Ovulation after IUD removal?

Hi. I have had Mirena for exactly a year. I have had a regular period the ENTIRE time. My last one was on June 22nd. According to my ovulation tracker I should ovulate sometime between July 10th and July 14th. I am scheduled to have my IUD removed on July 5th (I want to conceive asap ). My cervix has to be dilated to remove the IUD. My doctor said I should not have sex for two weeks after removal. This two week period is during my supposed ovulation time (5-9 days after removal). Is it absolutely necessary to wait the entire two weeks and miss out on this possible chance to conceive?