Can tighty-whities interfere with your sperm count? Surprisingly, though, this is a legitimate concern. In healthy fertile men, even when briefs reduced sperm counts, vital sperm health measurements were still in the normal range. For men who already have poor fertility, or for men on the borderline of normal sperm counts, changing from briefs to boxers might be helpful, but how helpful it may be is unknown. Sperm cells are sensitive to rises in temperature.