Assessment analysis (group)

This is a group assignment using spreadsheets to analyze student data.

Item difficulty and discrimination index

Download student data, perform analysis, then write a report of the analysis.

Student data is in Canvas. Each group should use different data.

The spreadsheet should:

• Compute the difficulty for each assessment item.
  • This is basically an overall percentage correct, but in decimal form.
  • A higher "difficulty" means a problem with more correct responses.
• Compute the item discrimination index for each item.
  1. If needed, use "Data > Text to columns" to separate data.
  2. Sort data by final score.
  3. Identify top 27% and bottom 27. The command percentile(range, percent in decimal form) might be helpful for finding top 27% and bottom 27%
     Caveat: If tests scores appear across percentile ranks, decide to include or not include all scores from the rank.
  4. Calculate difficulty(top 27%) - difficulty(bottom 27%).
• Compute the difficulty for each category of mathematical understanding.

The report

Please submit a report of your analyses with explanations. The report should …

1. Identify the category of mathematical understanding for each problem. A problem may fit in more than one category. Also include your definitions for each category.
   • conceptual understanding: Adding It Up (2001), NAEP, Dan Meyer
   • procedural knowledge: Adding It Up (2001), NCTM (2023), NAEP
   • mathematical reasoning: NCTM (2009), Adding It Up (2001)
   • problem solving: Wilson, et al., (1993), NAEP, Adding It Up (2001)
2. Use tables to organize computations from the spreadsheet.
3. Provide quantitative summaries of the data. This will include written interpretations of the numerical data.
4. If available, provide qualitative summaries of the data by using pictures of student work with your commentary.
5. Make inferences based on your analyses. This may include possible reasons why the data had the observed results and/or possible responsive actions by the teacher.

• Item discrimination indexes above 0.2 are typically desirable.
• Review Interpreting Test Scores as linked in the Class Schedule

Tips:

• Focus on "the data" or "the response", not "the student".
• "Data" is plural. So "The data are …" is correct whereas "The data is …" is incorrect.
• For each problem, consider the difficulty and item discrimination index together, not separately, when discussing results.

Analysis scoring guide 20 MTP

Submission

Submit the URL for your assignment to the correct Canvas discussion board. Recall that the discussion board is used as a way to share ideas between (preservice) teachers.

Please print relevant pages if you want more detailed feedback.

For group assignments, only one submission is needed per group, but please make sure all group members have included their names on the submission.