The memory of something happening to you concerning your fear on a particular day can stay with you, especially if your phobia is really gripping. Or anything else really. I get nervous around October because of something that happened to me, particularly Halloween, but you'll be fine with it either way.