Britannica AI Icon

Quick Summary

The World Wide Web Wanderer was a pioneering program created in June 1993 by Matthew Gray at the Massachusetts Institute of Technology (MIT). Its primary purpose was to measure the size of the World Wide Web, a task it performed until late 1995.1

The Wanderer was an early example of a web crawler, which is a computer program designed to systematically search web pages for specific information. These crawlers explore the web by following hyperlinks from one page to another, collecting data to help build indexes for search engines. While the Wanderer's goal was measurement, its function was a crucial step in the development of search engines that help users find information on the vast World Wide Web.21

Summary created from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles.
2 Britannica Sources
  1. web crawler
  2. search engine

Ask Britannica AI about this topic

Britannica doesn't have a full article on this topic, so we've curated a list of our relevant coverage.