AoD hit the nail on the head - only six percent of the US population gets norovirus in a given year. My family is also horrible about washing their hands, and they rarely get stomach viruses. My dad is not only careless with hand washing, but with eating foods that have been left out longer than most would like, and doesn't always care about expiration dates (depending on the product) and he has only gotten a stomach bug 5 times and food poisoning once in fifty years!

Keep up on your hygiene for your peace of mind. Your brother is nothing to worry about. If it worries you that much, don't eat "shared" foods, like bags of chips. Besides that you should be just fine, but I assume you would be anyway!