I think it would be a wonderful idea. I think communication is very important. If he understands that you have anxiety, he will possibly accommodate you. Anxiety isn't like you don't like something, it's a legitimate illness that needs to be understood.

Good luck!