I honestly wouldn't worry yourself about it that much.
Having owned a few myself, and known many users, I have never known one to become weaker or seem to deteriorate in any which way due to using a squeegee or fluffy in them; though off the top of my head I don't remember which you aren't meant to use...