A childhood friend of mine traveled to Johns Hopkins from South Carolina by car and drove home (actually, his wife drove him) 3 days post-surgery without any problems.
I would guess that air travel might be more difficult, however, due to the fact that you will be around a large crowd of people, wearing your catheter and bag, subject to possible flight cancellations/delays and still weak from the surgery.
As a very frequent air traveler, even pre-surgery your flight schedule would have been a very tiring one for me, but then again I am 11 years older than you.
Sorry that I do not have more info to provide. Perhaps others who have actually done what you are scheduled to do can lend insight.
Best of luck to you.