如何处理Django中的web爬虫?

I've been getting a lot of errors sent to my e-mail due to web crawlers hitting parts of my site without any request data, and I was wondering what is the best way to handle web crawlers in Django? Should I issue a redirect when I come across an empty QueryDict?I've been getting a lot of errors sent to my e-