Biased vs. Unbiased
The following is a nice example from Michael Hardy that shows how excessive focus on unbiasedness of an estimator leads to nonsense results :
A light source is at an unknown location µ somewhere in the unit disk D. A dart thrown at the disk strikes some random location U in the disk, casting a shadow at a point X on the boundary. The random variable U is uniformly distributed in the disk, i.