Your concerns for your husband and his concerns about his diagnosis and treatment, given his symptoms, are all valid. This is where it can get so very confusing.
However, b/c of what we do know about the Lyme bacteria and it's co-infections, it is not wise to just stop antibiotics or even begin to remove the treatment protocol without your doctor overseeing. If it is a matter of confidence in your husband's current LLMD there is a question as to whether this came about b/c he/she is not listening to the symptoms or perhaps they have not been communicated to him/her yet. If the symptoms are not known yet by his doctor, you must call and inform them in detail of what he is experiencing.
If it is another physician that is needed to, as Ticker says, offer a second opinion, then please move forward, so that a bridge of treatment can be appropriately in place in the event your husband is fighting this bacteria in his body. To just stop could be detrimental.
For one, your husband is not detoxing properly. Is he drinking enough water each day to flush his system? How is his daily diet? There is a seemingly huge inflammatory response. There are things he can do in the meantime to help move the lymph system; key for detoxification.
There is a legitimate concern for reaction, co-infection, other complicating factors - none of which any of us can know. The fact that he tested positive is reason for caution in how you proceed so that the bacteria does not act as an opportunist in a vulnerable host.
I do not know where you are located geographically, but, if you are in need of another physician and depending on your location or ability to travel for care; please feel free to pm my way if I can be of help bridging that for you.
It may come to ending antibiotics; but your question was regarding wisdom. I am grateful you are watching out for him. Wisdom would have a bridge in place to protect his immune system and assure that it is just the medication and not complications stemming from Lyme and co-infections.
I second Ticker.