################### #Format and semantics of the "/robots.txt" file are as follows: # #The file consists of one or more records separated by one or more blank lines # (terminated by CR,CR/NL, or NL). Each record contains lines of the form # :". The field name is case # insensitive. #Comments can be included in file using UNIX bourne shell conventions: the '#' # character is used to indicate that preceding space (if any) and the # remainder of the line up to the line termination is discarded. Lines # containing only a comment are discarded completely, and therefore do not # indicate a record boundary. #The record starts with one or more User-agent lines, followed by one or more # Disallow lines, as detailed below. Unrecognized headers are ignored. # #User-agent # The value of this field is the name of the robot the record is describing # access policy for. If more than one User-agent field is present the # record describes an identical access policy for more than one robot. # At least one field needs to be present per record. The robot should be # liberal in interpreting this field. A case insensitive substring match of # the name without version information is recommended. # If the value is '*', the record describes the default access policy for any # robot that has not matched any of the other records. It is not allowed to # have multiple such records in the "/robots.txt" file. #Disallow # The value of this field specifies a partial URL that is not to be visited. # This can be a full path, or a partial path; any URL that starts with this # value will not be retrieved. For example, Disallow: /help disallows both # /help.html and /help/index.html, whereas Disallow: /help/ would disallow # /help/index.html but allow /help.html. Any empty value, indicates that all # URLs can be retrieved. At least one Disallow field needs to be present in a # record. The presence of an empty "/robots.txt" file has no explicit # associated semantics, it will be treated as if it was not present, # i.e. all robots will consider themselves welcome. ################### User-agent: * Disallow: /admin/ Disallow: /demo/ Disallow: /test.php Disallow: /addoldjob.php Disallow: /saveoldjob.php Disallow: /siteadmin.php Disallow: /savenotice.php Disallow: /acctadmin.php Disallow: /histadmin.php Disallow: /saveacctadmin.php Disallow: /registadmin.php Disallow: /registsaveadmin.php Disallow: /transferadmin.php Disallow: /transfersaveadmin.php Disallow: /bmadmin* Disallow: /dmadmin*