I'm not qualified to tell you how to come off a medication. What I can offer you though is some reassurance that certain meds have some funky side effects, including ironically anxiety.
My first psychiatrist put me on Wellbutrin because it worked for a family member who had depression. Personally I have panic attacks, generalized anxiety, some agoraphobia, and depression. The Wellbutrin caused me to be confused, lightheaded, and I lost my appetite completely and had to force myself to eat a fraction of what I would normally. I also had a hard time walking on my own accord (without leaning on furniture or another person for support).
More relevant though to your post, I also had a spike in my anxiety level. When I asked this doctor about it he got VERY defensive VERY quickly (though totally unprovoked). Long story short, I have a new psychiatrist and am off of the Wellbutrin. All of the side effects are gone. I did some research online and also found that Wellbutrin is only FDA approved for depression, not anxiety or panic disorder. Why would someone prescribe a medication that doesn't help with anxiety and actually causes anxiety as a side effect for an anxiety sufferer? The new doctor didn't have an answer, but is sooo much better than the first. I feel very comfortable with this new person. It sounds like you may need to look into making a change in doctors even if just until your regular one is back from leave.