Hmm....well..various factors to consider here.
Firstly...if a hole, large enough for you to fall down, and not bump teh sides...is a fairly wide hole...consider that this is borred directly through teh earth...now consider, that the earths core, is liquid iron...this would obviously...be affected, by teh hole...and I imagine...would leak...which would disrupt its flow...and hence, earths rotation...which would mean...taht while you fell through this hole....enjoying every minute of it, and never realiseing that you are going up...because you never would be...teh people on teh outside of teh world...including those special individuals, who have lived their lives upside down...would now realise that gravity is a good thing..as they gradually started to find themselves becoming lighter and ultimately floating away...that is of course, if the major environmental changes, happening all around them, didnt distract their attention...
Another factor, that you overlooked...is the fact that despite, your super heat proof suit...you would infact have limited oxygen, and would most likely be breathing dust and sulphur gasses for most of your journey...
These things considered...I would suggest, not only that you would never realise that you were traveling upwards, as it would never happen...even if you survived teh fall...but, if you did...consider that it is a very irresponsible thing to do.
