As you can see by my signature line, my stats are similar. My 3 month PSA was fine, but then a bit higher at 6 months. This is when I decided to undergo additional therapy, and thankfully, things are going well now. You need to relax now and just get stronger. The PSA tests later will help you make decisions later.
As far as your question, my surgeon did say that the radiation would probably zap the nerves he saved, but that was never a reason for me not to have the additional therapy. I'm also on hormones for a couple of years, so that lessens the "libido". I've always put LIFE in front of some of those other things, and I'm thankful that I had a second option following surgery.
You seem to be recovering well, so just keep that up and the rest will work its way out later.