Our analysis of the teacher tests found that, although some of the testing procedures are adequate, other features of the process are problematic. In general, the CM tests used to measure teacher knowledge have adequate or near adequate levels of internal consistency and reliability, represent broad coverage of the curriculum, and are well aligned with test guides and specifications given to teachers. However, the majority of the items in the subject-matter section of the tests, which measure teacher knowledge of his subject, demands only low-level cognitive responses. In addition, other test sections evaluating knowledge of pedagogy and Mexico´s education system have low levels of internal consistency reliability.
The results from our impact analysis of CM incentives on student test scores suggest that these incentives do not have any discernible effects on student test scores for primary school teachers and very modest positive effects on student test scores for secondary school teachers who are vying for admission into CM. In secondary schools, these effects are more evident after Etapa 10, when states adhered more closely to the minimum cutoff required for admission. Magnitudes of the effects for secondary school teachers range from 3 to 15 percent of a standard deviation, and they are only for teachers we classified as being within a group most likely to benefit from the incentive program (which was comprised of fewer than 20 percent of the total sample). After teachers were admitted into the program or received a promotion, we observed modest negative effects on their student test scores.
These findings are consistent with the literature on teacher incentives, which offers mixed evidence on the effects of salary incentives offered to teachers to improve student achievement. Most of the programs we reviewed in this study showed only modest improvements in student test scores, and these improvements (when they were observed) were often short-lived. Moreover, there was evidence that teachers devoted extra time to test preparation and this extra attention might have been partly responsible for the positive effects observed in some programs.