BuildWith.com is a great tool for figuring out what a website was built with. Does the site use Analytics, AdSense, which web server, which web framework, etc.

http://builtwith.com/default.aspx?http://kbcafe.com/

This last weekend, I had major outages on some of my sites because Google was indexing the hell out of them. Google's indexer has no fear. If it can denial of service a website, then it will by sending boundless amounts of requests. I noticed today in the logs that Google was indexing some really low value pages and their were millions of them. The solution was simple, I added the noindex <meta> tag to these pages.

<meta name="robots" content="noindex, nofollow" />

http://www.robotstxt.org/wc/meta-user.html

Unfortunately, Google will continue to index these pages at least one more time each before it gets the points.

Lesson: Be careful what you let Google index.

Follow up: I looked at Google's webmasters site for a possible better solution. At this point, Google doesn't provide a mechanism for slowing the indexing of sites. Even the sitemap protocol does not provide a mechanism for specifying how to index millions of pages without actually listing the millions of pages in a sitemap. Google's suggested solution doesn't really address the problem.