# All robots will spider the domain 

User-agent: *

User-agent: Slurp

User-agent: Googlebot

User-agent: MSNbot