My allergist explained to me that molds are ubiquitous in the atmosphere, so the best solution for mold allergies is to take shots for the most serious one
to which you are allergic. Often, stopping the most serious allergy has a beneficial effect on the others. They seem to quiet down a bit.
I would consider staying with your allergy specialist who advised you of your allergens, asking about shots for the most serious ones, and follow Red's advice about using a supplemental antihistamine such as Zyrtec, Allegra, etc.
Xyzal (a prescirbed potent form of Zyrtec) was recommended for me, and
it really works wonders for me.
Hope you feel better soon.