As you point out, for early adopters like you (or me) there's no real issue of where can't you go in your EV. You can go everywhere. You just have to plan.
And that's kind of the rub for people like my wife. She didn't like the planning and she didn't like being locked in to stopping at certain locations. And no matter how many arguments I made about the amount of time we saved daily by refueling at our house as opposed to going to gas stations, she would focus on her annoyance on the few road trips.
As the "one level 3 charging spot every 50 miles of interstate" part of the infrastructure plan gets built out properly, this will become less and less of an issue. But for now, in some parts of the US, it is a real issue. If I were in charge of solving the problems, though, I would focus first on increasing reliability of existing non-Telsa chargers, then focus on adding additional stalls at existing locations, and finally add existing locations.