You realize his point isn't that you should pay more so you can keep pregnant people, it's to pay more so that you can retain good employees and not be forced to bear the costs of going through the transactional costs of hiring a new employee, right? Obviously if it was worth it for them to stay then they would. As is, it's a lose-lose: they lose because it's not worth it to stay at a job that doesn't pay enough to get them child care (or so we theorize) while you lose because you have to suffer the time and financial burden of replacing employees instead of just paying them more to stay.
And to piggyback off of 923, of course it's easier to just do what you do: discriminate. That's the whole reason we have laws preventing it, because it provides an efficient breach for the employer while wholly shifting the burden onto the shoulders of (in this case) women who may potentially get pregnant.
Bottomline is that regardless of the legality of the situation, which we all agree in your situation is not illegal SOLELY because you are a small employer, it's a despicable business practice and just shows that you value some extra bucks in your pocket over any sense of morality, empathy, or social compassion for fellow humankind.