There is a difference between "free speech" and "backlash"
You have the right to free speech. You don't have a right to getting no backlash.
Other example: Your right to free speech allows somebody to walk through New York and scream that they like Joseph Stalin. Their right to free speech doesn't mean that people can't tell them they're an idiot.