是否有搜索引擎包括索引机器人，可以通过向机器人提供某些属性来构成特殊目录？

Our application (C#/.NET) needs a lot of queries to search. Google's 50,000 policy per day is not enough. We need something that would crawl Internet websites by specific rules we set (for ex. country domains) and gather URLs, Texts, keywords, name of websites and create our own internal catalogue so we wouldn't be limited to any massive external search engine like Google or Yahoo.Our application (C#/.NET) needs a lot of querie