The WebCrawler started out as a research project. As the Internet continues to grow, it becomes even more intresting as a playground for reasearch in information discovery and retrieval. We have built a strong team that's committed to continuing the WebCrawler tradition of basic research. Some of our past efforts are described here, with more to come!
If you'd like to learn more about the WebCrawler, here are some resources that can help.