Sorry you are having headaches. They can be very hard to live with.
Sounds like you might have rebound headaches. My husband also was taking Advil or Tylenol every day, every 4 to 6 hours and that is what happened to him. He has basilar migraines that start in the back of his head and the left side of his head. He now takes absolutely NO Advil, Aleve, Motrin or Tylenol! His migraines are much better after quitting them and starting on amitriptyline.
Seeing a doctor is the only way to find out.