The heavy bots use different User Agent, so you can catch those. Also the main bots respect robots.txt.