Everybody has baggage. He might not have Crohn's, but like all of us, he does have his own secrets.
At this stage, you don't have to bring it up if you don't want to...it sounds like you are just not ready, or don't know whether you can trust him. Because you are still getting to know one another.
In case you feel guilty, please consider that you are not withholding information he is entitled to. You just have a right to privacy, like he does.