Magnite, I am so sorry that you are dealing with this.
My kids were all early teens, and my H was also a very out of control monster, and they have all chosen NOT to ever stay over night with him, and they despise the OW, so try to have as little to do with her as possible.
I can't begin to imagine what this must be like with younger children.
I too would fight tooth and nail not to hand my baby over to those two. A recipe for disaster. I would research attachment disorders, and get a lawyer to be your advocate on this if you need one.
Fingers crossed the novelty of the overnighters wanes with the other two too, and the kids can all remain together. I know this is tough on you to always have them, but it's preferable to them being mucked around by a temporarily crazy Father, and a woman who you know very little about - and even less about her nurturing abilities.