I can't be bothered to scour the internet for a source, but Sondheim has spoken specifically about his use of hats. Completely paraphrasing this, but it went something along the lines of ... "Because I mention hats so often, no doubt some graduate student will write a dissertation on the subject. Well I can save them the trouble and tell you that it's just because of the jaunty nature of the word, and the amount of words it rhymes with." Or something. He was using hats specifically to make a bigger point about over-analyzing his work. I think I might have read it in one of lyric books, come to think of it.
Actually, the reference from GYPSY goes, "True, you could say, 'Hey, here's your hat'". (The "true-you" and "say-hey" word-play is important enough to notate.)