Try not to be concerned about what other people think about you. There will always be people that say bad things about us. You know in your heart that you are a good person and that is what matters, though I know it doesn't make it seem any easier. I use to hang out with the guys too because I didn't trust women. But now I have women friends. Though I stick to myself most of the time.
It sounds like it would do you some good to see a counselor. They can help you to get self esteem, and they are super supportive.
The relationship that you are contemplating sounds very complicated. Does he have kids with this other woman, well, his wife? If so, you should know that they are always going to come first in your relationship. It does sound like there is a lot of excess baggage in this relationship. I don't know if you want to continue to deal with that. Plus if he is married to the other woman, is he going to get a divorce? I just don't like the bouncing back and forth he is doing. He doesn't sound like he would be stable to be with. You could date though and not get serious. If you want that. But I wouldn't get serious with him unless he divorces his wife. It sounds like he wants his cake and to eat it too. That is just my opinion.
Keep posting, I hope that this helped a little.
Moderator-Depression and fibromyalgia
fibromyalgia, Chronic fatigue, depression,allergies