Technical SEO Issues

Robots.txt Sensitive Backdoor

June 3, 2010 by  
Filed under Technical SEO Issues

USC.edu Stats: http://www.usc.edu/stats/prev/hscweb-monthly-server-stats.html The URL above refers to the stats page of the University of Southern California. Robots.TXT files tell search engines what they shouldn’t index, but it’s no use if we forget to tell them not to index themselves (which could lead to clashing the index with the noindex operation). So, what do we [...]