You shouldn't feel guilty at all. You're young and healthy, work hard, are a great mom, and should take some time to have fun with your husband or friends!
My 25 year old stepdaughter is going to have a baby in December, and I've already told her that her dad and I will be available to watch the baby anytime she wants to go out with her husband, friends, or alone, just to get a break from 24/7 mommy duty.
I never had children, but I see how much work it is by watching my sister, who has two young ones. She used to feel guilty, too, about
going out without the kids, but I think she has realized that it's important to have a balance, so she doesn't feel like she never takes any time for herself. A happy mommy is a better mommy!
Plus, it's important for your son and his grandparents to spend time together without you around. I cherish the memories I have of the times I spent with my grandparents, while my parents were out of town, out to dinner, or whatever.