This is a tough and personal decision!
I can get why your P-doc might want to get you a bit more stable...with more recovery time from panic...
...but I also understand you not wanting to wait...because as you said, you just don't know when that might be, and you are ready to grow your family at this time.
I had both my babies pre-anxiety...so I don't have personal advice to offer...but can say that I have read many accounts of moms whose A/P was very managable sans Rx during pregnancy.
I wish you much ease in making your decision...and of course support you in any direction you decide to ultimately go.