I caught my first husband in 60 lies in the first month (or something like that--I've forgotten exactly what). He quit lying and a year later we got married. We each brought garbage to our marriage.
He brought guilty secrets. He showed that through the lying. Within a year, and maybe sooner, he began lying again. He went through times when he lied and times when he didn't for 20 years.
I brought secret fears of my weaknesses and failures. I showed that by being gullible and forgiving too often. Eventually it changed to anger and rage.
During our separation I told my counselor I trusted him. He asked me how I could trust him if he was a liar (also a cheat). That stumped me. I don't understand why I thought that--to this day. I also found out that his first wife had become an alcoholic, attempted suicide, landed in a state hospital for 6 months, and was still an alcoholic. That was not the picture he had created of her.
I realized that he was repeating what he did to her--only this time it was me. I swore I'd beat him at his game--and in a way I did--I didn't get locked up and I didn't bury myself with alcohol. Then we divorced. It wasn't a pretty divorce.
He continued on to victim wife number three, still hanging onto his intact guilty secrets. She also was left a shambles--only that was after he sucked her dry and died. I went on to become a fully functioning, happy citizen of the world. By the way, each of his three wives (including me) will tell you that if he ever loved anyone, it was them.
The bad gets worse with time, so if it's bugging you now either get it fixed or get out. Idle threats, reminders, none of that fixes it. Make demands for what you need to be happy. Either he will care or not, and then you will know what to do.