通过scrapy-redis框架,使用代理IP池,返回结果中出现
2017-05-27 11:45:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://xxx> (referer: None) ['partial'] 2017-05-27
2017-05-27 11:45:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://xxx> (referer: None) ['partial'] 2017-05-27