We examine how well principals can distinguish between more and less effective teachers. To put principal evaluations in context, we compare them with the traditional determinants of teacher compensation—education and experience—as well as value‐added measures of teacher effectiveness based on student achievement gains. We present “out‐of‐sample” predictions that mitigate concerns that the teacher quality and student achievement measures are determined simultaneously. We find that principals can generally identify teachers who produce the largest and smallest standardized achievement gains but have far less ability to distinguish between teachers in the middle of this distribution.