At least to me, the number matters. I'll explain. Tons of different women equal just that, tons of different women. No emotional attachment, no forming a lasting relationship.
2 or 3 or 4 OW/OM, or new woman/men (if the couple is long separated or divorced) with whom the MCLer forms a proper relationship is one more problem. More hurt, more issues, more people.
Your husband's colleague is different than Mr J, seems more like a single guy I know that is at the end of Replay.
Mr J spend 10 years with OW2. In 13 years there has been 3 women, the latest very recent. No one sees anything wrong with 3 women in 13 years, especially when one of them lasted 10 years.
He will never discover the problem. The number of women in 13 years is perfectly normal.
Sometimes good things fall apart so better things can fall together. (Marilyn Monroe)