You wouldn't hear much, if any, difference if the tops aren't reproducing sufficient lows. However, they're still receiving the low frequency information, so the amps are producing low frequency power and the drivers are taking that power, which stresses both the amps and the drivers, while the THD of the tops will be higher than if high passed. You've got nothing to lose with high passing, and potentially much to gain with respect to amp and driver headroom and lower THD, so it should be done.