在Python中进行Web爬网的最佳预建库是什么[重复]

I need to crawl and store locally for future analysis the contents of a finite list of websites. I basically want to slurp in all pages and follow all internal links to get the entire publicly available site.I need to crawl and store locally for future an