I think it depends a lot on the size of the venue. I have been to some events where the venue is quite small and with 20 people, it was pretty crowded.
If you have plenty of space, I think I meet up to about 40 people is good. Once you cross a certain threshold, I think you lose the intimate setting that allows for good relationships to be built.