Właściwy robots.txt ?
: 30 maja 2009, 06:27
Mam taki robots.txt i nie jestem pewien czy jest dobrze sformułowany .Czy może ktoś przeanalizować ?
User-Agent: *
Allow: /
User-Agent: Googlebot
Disallow: /faq.php
Disallow: /misc
Disallow: /MAINTAINERS.txt
Disallow: /user/register
Disallow: /?q=logout
Disallow: /?q=user/login
Disallow: /INSTALL.txt
Disallow: /profiles
Disallow: /aggregator
Disallow: /UPGRADE.txt
Disallow: /logout
Disallow: /admin
Disallow: /xmlrpc.php
Disallow: /?q=user/password
Disallow: /?q=node/add
Disallow: /sites
Disallow: /CHANGELOG.txt
Disallow: /includes
Disallow: /?q=search
Disallow: /scripts
Disallow: /user/login
Disallow: /?q=admin
Disallow: /database
Disallow: /modules
Disallow: /search
Disallow: /themes
Disallow: /LICENSE.txt
Disallow: /updates
Disallow: /?q=comment/reply
Disallow: /?q=contact
Disallow: /comment/reply
Disallow: /?q=aggregator
Disallow: /?q=user/register
Disallow: /install.php
Disallow: /node/add
Disallow: /cron.php
Disallow: /contact
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /user/password
Disallow: /update.php
Allow: /