I have just started taking meds for anxiety and panic in the last four weeks (although have suffered most of my life, I am 39 now) and this has been my experience just last week and over the weekend.
I took Xanax for 2 weeks which I had got from one doc as mine was away for 3 weeks and I was feeling really bad. When I went to see my original doc he did not like me taking it so switched me immediately to Valium. I think the Xanax was helping as I was waking up feeling dopey but more calm and managing to get through the days.
However, the morning after I stopped taking the Xanax I felt really awful again, v shaky, major anxiety and panic which I was finding v difficult to get under control. He didn't mention any withdrawal to me at the time but I called him again as I was concerned, at which point he said that it was probably the withdrawal of the Xanax and could also be partly the switch to the new med Valium which was not working yet. I wish he had told me this previously as I would have probably gone against the switch at this time! The feelings were equally as bad and sometimes worse than before I started taking it as you mentioned.
Could you maybe give your doc a call and tell them what you are feeling, at least then they can probably put your mind at ease (if that is possible, hard I know) as to what you could expect from the change etc.
Take care and hope you start to feel better.b