My boyfriend's best friend hates me?

He is relentless with bringing me down and even though my boyfriend has told him off every time, it's usually because I threaten breaking up with him if he doesn't. I feel like he wouldn't say anything to defend me if I didn't make him do it. He promised me him and his friend are no longer friends time and time again, but they creep back into eachother's lives every single time like soul mates. I don't know what to do. I don't have any friends I can turn to, and his friend will always hate me and will cause problems and reek havic in my life. Would this be a good reason to break up with my boyfriend? I will never get rid of his friend and it causes nothing but stress in my life. What do I do?