I have the JT fan on my JTs, fits on top of the mask, under the peak (what can I say, it rains) and yes you can hear it, because it's so close to your ears, but you only have it on for 15-20 seconds just to get rid of any "fog" etc
So the last few times I have worn my JTs, the fan has been used for less than a minute, but I do just switch it on at the start of a game, then few seconds later its off.