Abstract
Objective To describe the Neonatal Research Network's efforts to improve the certification process for the Follow-Up Study neurologic exam and to evaluate inter-rater agreement before and after two annual training workshops.
Study design The Neonatal Research Network Follow-Up Study is a multi-center observational study that has examined more than 11 500 infants from 1998-2010 and born <= 26 weeks gestational age at 18-22 months corrected age for neurodevelopmental outcome. The percentages of examiners who agreed with the Gold Standard examiner on 4 neurodevelopmental outcomes on the initial training video and a test video were calculated. Consistency among examiners was assessed with the first-order agreement coefficient statistic.
Results Improvements in agreement among examiners occurred between 2009 and 2010 and between initial training and test. Examiner agreement with the Gold Standard during the initial training was 83%-91% in 2009 and 89%-99% in 2010. Examiner agreement on the workshop test video increased from 2009-2010 with agreement reaching 100% for all four neurodevelopmental outcomes examined in 2010. First-order agreement coefficient values for the four neurodevelopmental outcomes on the training videos ranged from 0.64-0.82 in 2009 and 0.77-0.97 in 2010.
Conclusions We demonstrate the importance of annual certification and the benefits of evaluation and revision of certification protocols to achieve high levels of confidence in neurodevelopmental study outcomes for multi-center networks. (J Pediatr 2012;161:1041-6).