Is your current doctor a PM, or just your primary? Unfortunately, many, if not most, pain management clinics are referral only, so you would need to get a referral from your current doctor, if the one who is treating you currently is your primary (if not, then you could ask your pcp for a referral).
If it's your pcp who is prescribing to you now. I think that I'd have a polite conversation with him & explain that you would like to be placed under the care of a PM, as you feel that they may be able to better handle your case because they 1) have more options to offer (ie. injections, implants, etc) to better treat your pain and 2) are better apt to handle a long term pain problem because of their superior knowledge of pain & its treatment (worded in a nice way, as not to make your current doc feel like you think he is stupid). You could also mention that a PM would be able to set you up with other alternative treatments and services that may provide you pain relief (acupuncture, massage, PT, etc).
If it's your PM who is currently prescribing to you, then you could get a new referral for another PM from your pcp, and just send your PM a nice letter thanking him for his services & striving to help you live a pain free life, but that at present time, you have decided to switch your care to another provider for XXX reasons (such as you feel that you have exhausted all the available options that he has given you). If you want to keep the door open with this doc in the future, you could mention something about possibly needing his services again in the future & thank him again. I wrote something along those lines to my PM, when I switched (and the person who guided me through the process was a physician himself, so he knew how to write a polite "get me out of here" letter).
One other thing: Are you, or have you taken any anti-depressants? Depression is very common among CP patients. I am certainly one of those people who it affects, and as a result I have been on multiple medications. I really would urge you to talk to someone about this. If you don't feel that you can confide in your doctor, then why not look for a psychiatrist (who could prescribe), or at least see a psychologist/councillor in the mean time, while you are searching for a new doc, who could then take over that aspect of your case as well. There are also such things as pain psychologists (psychologists who specialize in CP). Some PM clinics even have these employed by or associated with them. You may want to keep this in mind when you search for a new doctor. And if you haven't explored this area already: some antidepressants (such as Cymbalta or amitriptylene) are even know to help alleviate nerve pain, and as such, are often prescribed by PM's.
I do wish you the best of luck in your search for a new doctor & during this time of transition. I know it is not an easy thing to do. It took a lot of gut and several months for me to "fire" my PM. But I am so much happier now that I did, because now I have a doctor who actually cares about me, listens to me, and works with me, for me. I hope you can find the same!