It sounds like it might be worthwhile to try the goat cheese and milk. Maybe the proteins are enough different that you can tolerate them.
What is your allergic response? If it's just an upset stomach or reflux, you could do a challenge test. DO NOT DO IT IF YOU GET THROAT SWELLING!
Milk Challenge: Take 1 teaspoon first, then wait 20 minutes. If no reaction increase dose to 2, wait 20 min, then 4,8, and 16. That should be enough to bring on a response if you are subject to immediate reaction.
Do ask the allergist about
this test, if you can!
I am lactose and sulfite intolerant and do not tolerate the hard cheeses, but I can get away with the soft ones. I can drink lactaid milk, so I do not have a protein alergy.