Shouldn't impact too much. If you end up in the Marina, as you've suggested, that's ground zero for Fleet Week activities. The plus side is you'll have the opportunity to watch some sick air shows for free, capped by the Blue Angels. If your VRBO has roof access, that's even more money. Pretty much all the rooftops in the Marina, Cow Hollow, Pac Heights, Russian Hill will be filled with people partying. It's fun. The cons are that it'll be a little crowded on Marina Green that weekend and in many of the restaurants along Chestnut and Union. All the touristy stuff along the Embarcadero (Ghirardelli Square, Pier 39, etc.) will also be pretty crowded. Might impact your ability to find a ride in those areas, too, although there will be plenty of extra drivers scoping fares.
Other than that, you won't notice too much of a change anywhere else in town.
Last year's Fleet Week was neutered during the federal governmnet shutdown fiasco, so people will probably be pumped this year.