With my son (he's a holy terror but a healthy 9 year old now)the dr said i'd conceived 3 weeks I knew I had !!!! My husband & I were travelling in america with our 6 year old daughter & my mum & we were all sharing a room!! Now we're just not into sex with my mother & our daughter in the next bed!!!! So I knew i'd conceived either in san francisco(where we stayed 3 nights with a friend & had separate rooms or after we got back here - no way could it have been in between as nothing happened so obviously the dr kept saying one thing I said another, dr got due date right but everyone (even dr) admitted he was obviously overdue based on his birth weight & the condition of his skin at birth (what the midwife described as "overcooked"!!!!)
my daughter the dr had the due date ok, but I saw him the day before she was born (22hours before actually) & he said it would be at least another week or so as she hadn't turned & wasn't in position etc etc - well she arrived in a rush - so he got that wrong.
My point is drs are human & babies are all unique (& they have not read the books that tell them what rate to grow at etc!!!!) & will do their own unique things from word go - but the more stressed you are the more dangerous it can be for your babies health & well being - so relax. Yes there could be a problem but don't create one through stress, deal with it if & when you have to.
God bless you both.