Hi EveryBody,i have been working working for a private company.
Could anyone please tell me How do I use a Robots file?
If anyone have example with this.. please reply me

Thanks in Advance for the reply

Here is an example.

User-agent: *
Disallow: /and the path or file you do not want indexed

Lets say you have a test php file you are working on and do not want it indexed. Then this is what the robos.txt file would look like.

User-agent: *
Disallow: /test.php

To add more then one file to the list,

User-agent: *
Disallow: /test1.php
Disallow: /test2.php
Disallow: /test3.php
Disallow: /test4.php
Disallow: /test5.php

To add a directory to the file
User-agent: *
Disallow: /test1.php
Disallow: /test-directory/

and that will stop all search engines from indexing the test-directory.

__________________
pc gaming | digital camera forum

Thanks for clarifying me easily

Thanks Kev and Hsvyas you guys explain very well about robot file.

You can also do it on per page basis using meta robot tag..

<meta name="robots" content="noindex nofollow" />

Yeah but you have to place this tag on each page. And with robots.txt you can place all instructions at single page.

Thanks

yeah its good that if you are using robots file to our site but could you tell me one thing how can identify that which folder we are trying to stop to cache or index its really working or not.

__________________
Textile Manufacturer - Home Furnishing Textile

please visit this site to understand more about robots.txt

http://www.javascriptkit.com/howto/robots.shtml

I can suggest you http://www.robotstxt.org/orig.html where you can find the details and formats for robots.txt file.