The Exorcist, The Omen, and Rosemary's Baby are what I consider the three greatest horror movies of all time. I would consider The Exorcist and The Omen to be the scariest for some different reasons. It's hard to top The Exorcist for all around disturbing content. The Omen has just a chilling soundtrack and child Damien is terrifying.
A couple of darkhorses:
Jaws in its context scared a huge audience and it tends to linger in the back of the minds of several generations whenever they get in the water.
The Strangers and/or Funny Games are disturbing in their realism and completely pointless violence and terror. Those films just leave you unsettled.
The Descent comes off the rails at the end, but as someone who struggles with claustrophobia and has done some caving, the middle of that film makes me panic.