# Disallow all crawlers access to certain pages. User-agent: * Disallow: /admin/ Disallow: /paypal/ Disallow: /test/ Disallow: /junk-directory/ Disallow: /checkout.php