I am schizophrenic and stress management is my #1 priority. How I would handle it is never go there again. Being called a "simp," especially in-person is a clearly hostile environment, even if just socially.
I don't know what being "poly" has to do with this, but it is evidence that you probably overshare. Might need to work on that, I have a similar issue and I am working on it. People are ignorant. Be a designer with your words — less is more.