University of Connecticut University of UC Title Fallback Connecticut

Assignment: Reliability with Dichotomous Variables

Click here for sample data to use with this assignment. Your assignment is to determine two different types of internal consistency reliability for a  test or homework assignment. The two types of internal consistency reliability to be calculated will be Spearman-Brown and KR20 or KR21 (the latter depend on whether all of the test or homework items are considered to be of equal difficulty).

I have created an Excel spreadsheet for you to enter your data. The spreadsheet contains the formulas for Pearson’s r, Spearman-Brown Prophecy, KR20, and KR21. In the large blue area on the spreadsheet you will need to record the results for each question (whether students’ answered it correctly [1]  or incorrectly [0]). Please record data from at least 20 students for at least 10 questions. After you have finished entering your data, check the reliability estimates at the top of the spreadsheet. Those estimates are not correct until all of the data has been entered. Enter 1 if the student answered the question correctly, and 0 if the student missed the question.

In the sample below, Student 1 correctly answered the seven questions. Student 2 missed all of the questions. Student 3 answered all but one question correctly.

Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Question 7
Student 1 1 1 1 1 1 1 1
Student 2 0 0 0 0 0 0 0
Student 3 1 1 0 1 1 1 1
Student 4 0 0 0 0 0 0 1
Student 5 0 0 0 1 1 1 1
Student 6 1 0 1 0 1 1 1

Please report the reliability estimates you found for each method (1 point each — total of 2 points). You should also discuss differences in the reliability estimates from one method to another. Why do you think one method produces a higher reliability than the other (don’t say it is better or more accurate) (2 points). Include an attachment of your completed Excel spreadsheet with your assignment (1 point). Warning: Only print the first page of the spreadsheet…the spreadsheet is rather long and will print hundreds of blank pages if you let it.

Scoring for this assignment will be based on the following:

  • 1 point for the Spearman-Brown reliability estimate value
  • 1 point for the KR-20 or KR-21 reliabiliy estimate
  • 2 points for reason why the Spearman-Brown reliability estimate value was similar or different from the K-R value. Hint: You need to look at the data…examine how students did on the odd and even questions.
  • 1 point for you printout