for
Invision Power Board forums, to improve indexing and disable pages like registration / calendar / member profiles / PM to the web spiders, use the following robotx.txt file or add to the flat robotx.txt the lines :
CODE
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /style_images/
Disallow: /index.php?act=Search
Disallow: /index.php?act=Login
Disallow: /index.php?act=Reg
Disallow: /index.php?act=calendar
Disallow: /index.php?act=Msg
Disallow: /index.php?act=Mail
Disallow: /index.php?act=Forward
Disallow: /index.php?act=Track
Disallow: /index.php?act=Post
Disallow: /index.php?showuser
Disallow: /index.php?act=Print
Disallow: /index.php?act=ST