Ernie has an interesting post about creating a decentralized reputation system that would rate both lawyers and judges. I like the idea -- one of the first things I want to know when I start work on a new case is the reputations of both opposing counsel and the assigned judge. Unfortunately, that type of information is tough to find -- we've got Martindale-Hubbell for lawyers, and Ernie mentions the Almanac of the Federal Judiciary, a pay service from Aspen Publishing.
There's another site worth mentioning, and that's TracFed, which describes itself as "a unique, Web-based service for understandable, authoritative, and complete information about the federal government." One of the many useful resources found here is a section on Federal Judges, which provides year-by-year records of criminal and civil matters disposed of by each judge, individual statistical profiles, comparative data, and more. Unfortunately, this site isn't free either -- you'll have to pay $50/month to access the TracFed service. But it's a step in the right direction.
Update: Sabrina would scold me if I forgot to mention the review of TracFed at LLRX.