Have you considered that what you are feeling might be a mild panic attack? A lot of IBSers have mild to severe panic attacks. Hard to say if the IBS causes the panic or the panic causes the IBS. I think, though, that you can have either one, depending on the situation.
If your doctor thinks you are having mild panic attacks, he might prescribe a low dose of some sort of anti-depressant to try and control them and lessen their impact on your bowels (some anti-depressants also make the bowels work better, oddly enough, so they might help lessen the bowel problems that trigger the attacks).
You might also try alternative therapies for relaxation, deep-breathing and even counseling to see it that helps any.