{"id":331,"date":"2017-03-10T16:36:53","date_gmt":"2017-03-10T21:36:53","guid":{"rendered":"http:\/\/my.dev.vanderbilt.edu\/mathfollowup\/?page_id=331"},"modified":"2020-08-19T10:50:41","modified_gmt":"2020-08-19T15:50:41","slug":"student-direct-assessment-measures","status":"publish","type":"page","link":"https:\/\/my.dev.vanderbilt.edu\/mathfollowup\/resources\/student-direct-assessment-measures\/","title":{"rendered":"Student Direct Assessment Measures"},"content":{"rendered":"

Overall Data Collection Procedures<\/strong><\/h2>\n

Student direct assessment data for the Math Follow-Up Study were collected electronically, using either E-Prime or FileMaker software. For each of the computer-based assessments, E-Prime software was used to collect information about students\u2019 accuracy and response times. Assessors obtained student information for all other tasks using FileMaker software. The FileMaker system had built in checks which notified data collectors instantly if any questions were skipped, if the criterion for the ceiling and\/or basal was not met, etc.<\/p>\n

Assessors saved copies of their data and shared it remotely using Accellion, Vanderbilt\u2019s secure file-sharing platform. Individual assessor files were checked and then imported into a master database.<\/p>\n

By using a paperless system, we were able to eliminate the need for double-entry of the data, which allowed for quicker access to the final dataset at the conclusion of the data collection period.<\/p>\n

Student Direct Assessments<\/u><\/strong><\/p>\n

All student participants in the\u00a0Math Follow-Up Study also participated in a previous longitudinal study of early math skills. The following chart provides an overview of the student direct assessment data collection timepoints for the original study (\u201cScaling Up TRIAD\u201d), as well as the two follow-up grants.
\n<\/a><\/p>\n

\"AssessmentTimeline_Update\"<\/a>
\n<\/a><\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

As the chart illustrates, data collection for the follow-up studies began in Spring 2014 when most students were in 5th<\/sup> grade. Since that time, project staff have directly assessed the math knowledge and math-related skills of more than 500 students annually. From T5 through T8, direct assessments with student participants were administered each spring in two sessions lasting approximately 45 minutes each. Beginning with T9 data collection, participants completed one 45 minute assessment session. Assessments were individually administered inside a quiet room within the child\u2019s school.<\/p>\n

All data collectors completed a rigorous training, practice, and certification process before being sent out into the field to work with study participants. Assessors completed three days of formal training and then were given opportunities to practice the assessments with each other and with project staff. Following practice, data collectors administered their session to a member of the project team. If the pre-appraisal attempts were completed successfully, assessors next completed a video of themselves giving their session to a consented non-study child. Videos were scored using a scoring rubric, and assessors had to obtain a minimum score of 85% in order to become certified.<\/p>\n

Click here to view an example of the video certification form.<\/a><\/span>
\n<\/strong>
\nThe student direct assessment measures administered for this project fit into several broad categories: standardized assessments, nonstandard math assessments, math surveys, domain-general cognitive measures, and math-specific cognitive measures. Information about each of the measures is presented below.<\/p>\n

 <\/p>\n

Standardized Assessments<\/strong><\/h5>\n

KeyMath 3<\/strong> (Connolly, 2007) \u2013 The KeyMath 3 Diagnostic Assessment is a comprehensive, norm-referenced measure of essential mathematical concepts and skills. KeyMath content covers the full spectrum of math concepts and skills that are typically taught in kindergarten through tenth grade and can be used with individuals aged 4 \u00bd through 21 years who are functioning at these instructional levels. The instrument has three concept areas. From T5-T9, we administered three subscales out of the five in the Basic Concepts area.<\/p>\n