Try looking at it with another, hypothetical, example.
If you wanted children and your boyfriend didn't, you would have some serious issues. If you didn't have children, you would go through life miserable because it's one of your highest priorities. If you did have children, he'd be miserable because he's saddled with extra responsibility, taking care of children he never wanted in the first place. Sure, one of you could sacrifice to make the other happy, but it's a really, really big sacrifice to make. In that case, you'd probably just say "Okay, we have different priorities in life and won't be able to work in harness", and you'd go your separate ways.
In this case, he's okay with fooling around on the side and you aren't. If it's so important to him that he will force you to sacrifice your happiness in order to keep the relationship going and keep his extra nooky, you're probably not going to work well together in the future. This isn't either of your fault. You just have fundamentally different values, and that can be a dealbreaker for relationships.
All of that said, if I were you I would have a good talk with him. Tell him it's a dealbreaker for you to have him carrying on like this. Set clear boundaries as to what you're okay with and what you're not. Let him explain his side of things, but remain firm and let him know how awful it makes you feel when you're left behind while he fools around. If he refuses to let his something on the side go even though he knows it makes you unhappy, he might not be the best life partner. In that case, I would move on and find someone who was more compatible with me.
Attention: NOTHING on this site may be reproduced in any fashion whatsoever without explicit consent (in writing) of the owner of said material, unless otherwise stated on the page where the content originated. Search engines are free to index and cache our content. Users who post their account names or personal information in their questions have no expectation of privacy beyond that point for anything they disclose. Questions are otherwise considered anonymous to the general public.