pregnant with boyfriend ?

Kind of long so please bear with me.

My boyfriend and I did a full swap with a couple on April 27. (For those of you who don't know that means I had sex with the other man and he had sex with the woman.) We used a condom and it DID NOT break. But because I am a worry wart I took the morning after pill anyway a few hours after the encounter. A few days later I had some spotting. I thought it was my period starting, but then it stopped very quickly.

I'm a little vague as to when my last period was, but I believe it was around March 20 because I remember thinking right before our encounter with the couple that I was already late. So about a week and half to two weeks after we were with the couple I start feeling nausea and my breasts were very sore. We took a home pregnancy test on May 20th and it was positive. We went to the doctor and they did a pregnancy blood test and it was positive. They said judging from my hcg levels I am a little over two months along.

So you would think it's my boyfriend for sure right? We never use any protection in three years I have not gotten pregnant. He had a test done a year ago and was told the chances of him ever being a father were slim to none.

So my question is it possible it was from the couple encounter? I don't see how as a condom was used that didn't fail AND the morning after pill. All of the conception calculators say I got pregnant in early April. I also had an ultrasound done on May 29. The person performing the ultrasound said I was about 8-9 weeks. The actual doctor said 10 weeks.

What's your opinion? I need reassurance.