$400-600 a year is more than a nuisance for a lot of people. Gotta think a lot of out of state AP holders will scale back to one trip a year. The out of state pass is $1,299.
Agreed this is significant to many, but Disney has done the modeling and must have determined they'll get a more affluent annual passholder base with more propensity to spend in park on each visit and won't have a problem filling the parks by shifting their attendance more toward (more profitable) daily tickets. They basically got a free pass to test demand without annual passes in the last year and I was surprised they brought back annual passes at all.
If Disney continues to invest in the parks like they have been leading up to the 50th anniversary, I see the price of APs continuing to grow. If they don't get serious about replacing/modernizing rides though, I agree Universal and potentially others pose a big threat. The incoming generation of parents will be the best traveled, and once you've been to Shanghai or Tokyo Disney some of the US versions lose a lot of their appeal (Pirates in Shanghai or Winnie the Pooh in Tokyo are best examples, but even the queue for Tokyo's Tower of Terror is miles ahead). Rethemings like Splash Mountain to Princess & the Frog don't count - I can think of at least a handful of rides across WDW that should have been demolished years ago. Figment top of mind.