I've looked into the anti-soy claims and the evidence seems pretty weak. I recall that there's even some evidence that the phyto-estrogen in soy essentially crowds out estrogen in our bodies and leads to less effective estrogen.

Tofu isn't my favorite form to consume beans in, but we eat it somewhat regularly.