Search engines such as Google use so-called spiders, otherwise simply known as bots, to crawl or search the internet for new web pages that it will include in its search database, for websites that it will list in its search results.