White After Labor Day

OK...I'm a guy, but my old boss (you know who you are) always used to give me grief about wearing white after Labor Day. Where did this come from? Is this REALLY a fashion rule? Does it apply equally to men? Does anybody give a rat's #*@ still? I just heard this again the other day and don't want people at the bowling alley tonight or the people from Walmart laughing and snickering behind my back. I don't think I could live with the shame. Guess I've gotta change.