info

Spider: Search Engine Web Crawler

The term Spider is used to describe the special software programs that search engines use to index the huge number of web pages that are on the Internet. The Spider gets its name from the fact that it is crawling the world wide web. A Spider will visit a page anything from a daily basis to a more usual weekly or fortnightly basis. If a page does not change very much and has a low page rank and link popularity, the Spider will visit much less frequently.

Spiderability is another term for search engine friendliness - the ease with which a search engine Spider can navigate to all of the pages in a Website. A well designed site will have every single page visited by the Spider on a regular basis - even if that numbers tens of thousands of pages.

go back to edmonds commerce