User-agent: * Disallow: /edit/ Disallow: /Edit/ Disallow: /userfiles/ Disallow: /imgs/ Disallow: /include/ Disallow: /Template/ Disallow: /cgi-bin/ Crawl-delay: 60 User-agent: sogou spider Disallow: / User-agent: libwww Disallow: / # # default robots.txt file for wikyblog software # # these settings allow robots to crawl normal pages # but prevents crawling higher bandwidth pages #