The headaches that are just "general" can be caused by hormone changes...the kind that a lot of women get when they have morning sickness. There is no way to directly target the cause of this kind of headache (except not be pregnant lol) but you can soothe it by cool compresses on your forehead and or aromatherapy. I used to have a special sachet of herbs and lavender that could be heated or cooled and it was really nice for that.
The ones in the front are usually sinus headaches (sometimes also from tense muscles in your face too). You may not feel very congested nasally, but hormones cause sinus issues for most pregnant women. If you take some olive oil (or some of your favorite moisturizer) you can massage the ache right out of your head by massaging your sinuses. Avoid getting anything on your eyes of course..... gently start rubbing from the bridge of your nose and move outwards underneath your eyes where your cheekbone is. Then do the same farther up: start from between your eyebrows and work your way outward above your eyebrows towards your ears. Then make long strokes from between your brows upwards to your hairline. This is very relaxing and will help you a ton and also provides anti-aging benefits for the face.
The other thing you can do is use "head-on", which is non-medicated and homeopathic. I have used it myself and it does seem to help. They have different kinds for different headache causes and it is pregnancy safe
Hope this helps.