Top Definition
File that stops search engines from crawling a specified page or directory.
Disallow all search engines to crawl the specified directories - /images and /cgi-bin:

User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
by KFM December 23, 2004
Free Daily Email

Type your email address below to get our free Urban Word of the Day every morning!

Emails are sent from daily@urbandictionary.com. We'll never spam you.

×