Trying to see how it could possibly work...
I have a son, 16, and the guy I've been dating for a little over a year has 3 kids - G13, B16, B18. We both have shared parenting plans, which give us about 10 days/month that we can see each other w/out kids. We rarely see each other outside of this bc I try to respect the time he has with his kids and vice versa. His 18yo is in college, living at home, no job. Their mom doesn't work, w/ the exception of singing in bars on the weekends - even when she has the kids at home. Even when we are together, w/out kids, they are cosistently calling or texting their dad w/ questions or needing to come home because they forgot something or wanting him to take them somewhere. My guess is mom tells them to or she's not around to help. Sometimes he'll say no, but there are many times that our plans get interrupted because he's very much a "yes" parent. My son's dad is a parent when he has our son so I rarely have to intercede. I've been divorced for 11 years, him 2. Granted our kids are our priority and I wouldn't be concerned or frustrated except their mom isn't being a mom and he doesn't have boundaries that force her to be. He deals w/ all the school/teacher stuff, sports, etc. and pays her child support and alimony. He wants me to spend time w/ him and the kids but they are disrespectful to him and I find myself resenting them and him for it. He wants me to see myself in the future picture of their lives but I can't. I wouldn't subject my son to how they live. I've been at dinner w/ them and things errupted into awful - the oldest got mad at his dad and started cussing and calling him names. The youngest yells at her dad and the middle just kinda exists. When it's just the 2 of us together - we do great. And all this said - I do love him and if I walked away it would be because of his lack of boundaries with his ex and his kids. He's trying to change but it's slow and very little progress, and I fear that I will only be resented in the end - by him and his kids - because the change is only happening because of me. He says I'm good for him and his kids...I just don't know that it's good for me.
Congrats on thinking
Congrats on thinking logically. So many times we get wrapped up in emotionally thinking. The "i love him" reasoning is not always a healthy reason to stay or even get involved further. I'd say do what's best for you and your child. Move on from the boyfriend, it may be hard at first but if you are happier and less stressed it will be the right decision. You and your ex are proof there are healthy child- ex relationships that still exist.
He has only been divorced 2
He has only been divorced 2 years, so the changes are not necessarily because of you...they are because they are necessary for him as well. I am dealing with a similar thing. After being married a long time, it is difficult for a man to find the boundaries that are healthy to himself, the kids, a new interest. As far as your time together, I would flat out tell him that you come first and the kids can wait until the next day or whenever your time is over. They are old enough to get their acts together, even if the bm is not parenting. His youngest will be an adult in 5 years. That will fly by. Granted, you may be faced with future holidays with his family, but that doesn't mean you have to do anything you don't want to do. You have to decide if he is worth putting up with the rest of the crap, but remembering that the amount of crap will diminish as the kids get older and hopefully move out and on.