It's a tired tripe.
They use the pads argument, but most Euros and foreigners I have had this discussion with have no concept of how big and how explosive these guys are. They really don't, and there isn't convincing them of it. For as accusatory of American ignorance some pockets of the world can be, this is one issue where its just ignorant.
Rugby is a tough game, but the constant moving also lessons the blows, as no one is a sitting duck. And there seems to be more of an emphasis of wrapping up, and in football there is actual hitting.
Honestly, I find if funny that no one thinks hockey players are "girlie men" for wearing pads. But if you look at injury rates and career longevity, I don't see how anyone could argue against NFL being the "toughest."
I've also noticed that when backed into a corner, they point out stuff like stamina etc., which conflates the meaning of "tough." Acing Caluclus is "tough" but it doesn't make someone a bad-ass for doing so.
Of course having that kind of stamina is "tough" but no one winces in fear of the cardiovascular superstar either. Ohhhh marathon champion coming through, better give up your barstool.
It's just kind of a silly argument, as by any objective standard, NFL guys are tough as nails. But since you aren't talking to someone who is being objective, its not worth your breath.
Final note, I will say that I find it the height of bad manners to come here, then slam football. I'd never ever dream of going to europe and slam soccer (not that I would, but still). My gf has a friend who is from germany, and she does similar stuff all the time while here. And I'm like "why did you come then?"
The opening scene of the movie "Saving Private Ryan" is loosely based on games of dodgeball Brian Dawkins played in second grade.