KB,
Your H said that or shall I say asked you that question because he see the marriage as having serious problems that will not be resolved over night if ever in his mind. He believes that coming home is just the tip of the iceberg and it doesn't address anything. He believes whatever issues you have is the reason he left and not the OW. He may have some validity to that and maybe he doesn't, problem is he believes he does.
What other issues besides his career does he have a problem with that you are aware of? What have you done to resolve those? My H sort of did give me ultimatums but I never cared or worried about them as I knew deep down he was not strong enough to stick to it. He would say things like "we either going to work on this marriage or I am done." and he would also say things like "If I don't want him let him go so he can move on." I never would respond because I didn't want to give him any hope but I also knew I wasn't going to let him go either. I kept stringing him along until he got off the rollercoaster ride I was giving him.
I see OM occasionally but I had to change my number recently, as he would still call or text from time to time trying to reminisce or perhaps trying to see if he had any chance with me. I like OM as a person, but as mate? No. At the time I was dealing with my depression I adored the OM, I thought he was the missing link! He could give me everything I was missing in my life. Yes I did go through withdrawals. I ended it and still went through withdrawals. I had some real feelings for him but they were not love so I was sad about breaking it off. I actually broke it off with him last July or August 2015. I was finally okay and not sad about it anymore probably in November.
I didn't call my H because I knew what he wanted and I was not ready to give him what he wanted if ever at that point. I felt guilt talking to him, I could hear he was sad, I knew he felt lonely, and hoping for something anything to hold on to. I didn't want to lead him on, but I wanted him to wait. So basically I didn't want to imply he should wait, or ask him to wait I wanted him to wait on his own. Just a very selfish time. I was pushing him away but secretly hoping he would stay put.
Denjef31