Maybe the simplest and most compelling argument in favor of legalization is that it would enhance people's freedom to choose their own mates. No one on either side of the debate suggests that it should be legal to coerce anyone into a polygamous marriage. Jonathan Rauch's comment (above) that polygyny allows "high-status men to hoard wives" suggests wives being collected like possessions, but this overlooks the fact that in many cultures, women in polygynous marriages actively choose to enter them—and the same goes for men in most polyandrous marriages. If the government prohibits people from choosing to marry polygamously, this is the equivalent of telling them that instead of marrying their preferred partner, they must instead marry someone they would otherwise not choose, or else not marry at all.
So it does seem that by prohibiting polygamy between consenting adults, we restrict people's ability to choose their own mate(s). However this doesn't mean making it legal is a good idea. Personal freedom is not the only value we should strive to maximize, of course, and there may be a greater social good served by keeping polygamy illegal. Would the potential costs of legalizing polygamy—such as reduced gender equality, increased numbers of low-status unmarried men, decreased social stability, or some other unmentioned problem—exceed the potential benefits?