Dating Forum - boyfriend doing drugs & cheating with ex girfriend

My boyfriend and I have been dating for almost 5 months now. i'm 22 and he is 24. We started off having a pretty great relationship but eventually trust became a major concern in it.
First off a little background info.

When we first started dating i knew he dabbled in a little drug use but didn't find out how much until about a month in. He was a regular user of cocaine and ketamine. I told him that it bothered me to be around him when he was under the influence of drugs and asked him not to do them around me and he agreed.
At first he seemed to be clean whenever I saw him, but then it turned into him hiding the fact he was doing drugs by going into another room, doing a line, then returning like nothing had happened. But of course my gut told me otherwise and he eventually fessed up.
I proceeded to tell him that since obviously I was getting more and more upset with the drug use that we couldn't continue the relationship unless he quit completely.
He relented and told me he would choose me over the drugs.
He slipped up once and did a bump of ketamine while we were out with some friends one evening and I lost it.
I broke down crying telling him how much he hurt me and how I couldn't continue because the trust was now gone. He told me it wouldn't happen again and we ended up staying together.
Since then he has been clean and much happier as have I.
But then the calls started.
His ex girlfriend whom he had dated 3 years prior to us dating would be calling him at 2,3,4 in the morning. We would both be sleeping but I would wake up whenever these calls would come in and noticed the display saying her name.
I was extremely upset by this and told him so. He stated I was over reacting and that he was not cheating on me.
I know he;s not cheating but his ex calling him at ludacris times isn't acceptable. He told me if I didn't like it to call her and tell her. Since then the calls have stopped and I didn't have to resort to calling her which I'm glad.

(a side note.. two of my previous long-term relationships ended with me being cheated on. Both of my partners cheated on me with their ex girlfriends)

Then today he finished work early and when he arrived at my place I asked him how his day went etc. He went to get a shower and while in the shower I got the sudden urge to look at his phone. I know that was wrong of me and I shouldn't have done it. But in his messages it showed he'd been texting someone who was just listed as "M"

I copied the number into my phone and called it hiding my number. A woman answered and immediately I hung up and started crying.

When he came out of the shower I confronted him asking him who "M" was and why he didn't mention anything about her.

He said I shouldn't have been snooping etc and told me it was an old friend and sister of one of his friends who called him up while he was at work so he met her for lunch.

I told him he should have told me about this and asked why he didnt tell me. He said "I wasn't sure if your reaction would be worse than it is now"

Of course it wouldn't be. The fact he was withholding information about seeing some girl I don't know while on his lunch, rather than seeing me isn't acceptable. When I asked him about his day he should have mentioned it then, should he not?
We're both at our breaking point now.

He know's I don't trust him.. but I cant immediately trust him again with how much he's lied to me in the past... trust has to be earned. He even said "I don't know if I want this anymore"

I love him to death and I want to make this work more than anything. I need help, opinions, advice.. anything. I know me snooping wasn't right.. but was my reaction uncalled for?
Should I have reacted differently?
How can I learn to trust him again?

Please please help... I don't know what to do and I have no one else to turn to for advice since I moved last year and have no close friends here yet.