They have to know you are watching.

Inspectors have no right to enter your property without permission. So on the extremely rare occassion one does show up, you can say just say no.

It's next to unenforcable. The only people who pay are the elderly who don't know any better (and BBC enthusiasts I guess?)

Maybe a few decades ago where TVs were less common the law could actually be applied effectively. To give you a sense of how archiac this system is, the government generously lowers the fee to ~£60 / year if you only have a black and white TV.