Oh, Mamanan, that's horrible. You are like me. You don't like confrontations. I tell people off great...in front of a mirror! You wanted to tell him off but yet you didn't want the confrontation so you turned back and the anxiety kicked in. Do that now! Go to a mirror and tell that fool off. Say everything you wanted to say to him..and more... and you will feel better! You shouldn't feel like dirt! You didn't do anything wrong. YOU were the considerate one...not that jerk.
You just have to understand where the anxiety came from so, if it ever happens again, your anxiety won't be as bad. You handled it and made it back home safely so you deserve a big pat on the back! Congratulations!