Rubrics

Rubrics are tools, often structured as a matrix, that are designed to assess student performances. These performances may be observed while the student is doing something, or be a product of student work Rubric components include:

  • Criteria (rows): The aspects of the work that will be assessed. These can range from sections of a report to stated student learning outcomes.
  • Performance levels (columns): Identifies a student’s level of mastery within each criterion.
  • Scores: Assigned point value for each of the performance levels.
  • Descriptors (cells): Explicit descriptions of the performance level for each criterion. These tell students what they are expected to do in order to achieve a certain level of mastery.
Criteria
(learning outcomes in this example)		 Performance Levels
Exceeds Expectations Meets Expectations Approaches Expectations Does Not Meet Expectations
4 3 2 1
Students will use evidence and/or explain responsibilities of ethical, contributing members living in diverse societies. Explains, using sophisticated examples and evidence, what it means to be a responsible, ethical, contributing member of a diverse society. Clearly and substantially articulates ethical principles applicable in various contexts. Provides adequate explanations and examples, describing what it means to be a responsible, ethical, contributing member of a diverse society. Adequately articulates ethical principles applicable in various contexts. Provides limited, or few appropriate, explanations and examples, describing what it means to be a responsible, ethical, contributing member of a diverse society. Articulates few ethical principles applicable in various contexts. Provides no or inappropriate explanations and examples, describing what it means to be a responsible, ethical, contributing member of a diverse society. Does not articulate ethical principles applicable in various contexts.
Students will analyze and articulate the ways in which individuals, groups, and institutions influence society. Analyzes and explains, using substantial details and supporting evidence, the ways in which individuals, groups, and institutions influence society. Analyzes and explains, using adequate details and supporting evidence, the ways in which individuals, groups, and institutions influence society. Analyzes and explains, using limited details and supporting evidence, ways in which individuals, groups, and institutions influence society. Does not identify or explain, does not use supporting details or evidence, and/or does not explain clearly the ways in which individuals, groups, and institutions influence society.

Developing a Rubric

  1. Review learning outcomes: Examine the learning outcomes to be addressed by the assignment or article of work.
  2. Define criteria (rows): Define the specific observable attributes that you want (as well as those you don’t want) students to demonstrate in their work. Keep the list manageable. Edit the list so that each item is specific and concrete, using action verbs and descriptive, meaningful adjectives.
  3. Define the performance levels (columns): Aim for an even number (4 or 6) because when an odd number is used, scorers tend to gravitate to the middle.
  4. Write descriptions of criteria for each level (cells): Define the minimum that a student needs to meet expectations. Then define the highest and lowest levels, which are easier to describe. Finally, describe remaining intermediate levels using language that clearly distinguishes one level from the other.
  5. Test the rubric: Often your test is your first use on an actual assignment. That’s OK. Note where the rubric was difficult to use and make adjustments. Collect samples of work that exemplify each level to serve as benchmarks for future use.
  6. Review with colleagues: Rubrics promote shared expectations and grading consistency, which benefit both faculty members and student in the program. When used for program assessment, the rubric should undergo the norming process so that faculty interpret and apply the rubric in the same way.

Adapted from: Rochester Institute of Technology Educational Effectiveness Assessment, Rubrics, and University of Hawai’i Manoa, Assessment How-to, Creating and Using Rubrics.

Analytic and Holistic Rubrics

The example shown above is an analytic rubric and is the one most often used for formal assessment. Another type is a holistic rubric.

  Advantages Disadvantages Use when you want to
Analytic Rubric – specifies at least 2 criteria of student work to be assessed at each performance level individually
  • Provides more consistent assessment of students and between scorers
  • Provides detailed feedback about performance
  • Identifies strengths and weaknesses
  • Good for formative assessment
  • Time-consuming to develop and reach consensus among scorers
  • Identify strengths and weaknesses in SLOs or other criteria
  • Provide more detailed feedback about performance
Holistic Rubric – provides a single score based on an overall impression of student work
  • Quick scoring and efficient, especially for large groups
  • Provides an overview of achievement
  • Good for summative assessment
  • Does not provide detailed feedback about performance
  • Does not tell a student what was good and what to improve
  • May be difficult for scorers to decide on an overall score
  • Obtain a quick snapshot of achievement
  • Define performance quality as a single dimension

Adapted from: University of Hawai’I Manoa, Assessment How-to, Creating and Using Rubrics, and Brookhart (2013), How to Create and Use Rubrics for Formative Assessment and Scoring.

Task-Specific and General Rubrics

Two other types of rubrics are worth comparing. Task-specific rubrics describe themselves; they are designed to assess a specific performance task, such as an essay on a final exam. For this reason, task-specific rubrics are primarily used for scoring or grading. General rubrics, by contrast, are not limited to a specific task because the evaluative criteria and performance descriptions are generalized across different assignments that address the same learning outcome. They can be shared with students ahead of time so that they plan and self-assess their own work. This makes general rubrics powerful teaching and learning tools.

  Advantages Disadvantages Use when you want to
Task-specific rubric – designed for a single, specific task
  • Good for consistent scoring/grading
  • Easier to train scorers and use consistently
  • Cannot share with students ahead of time
  • Students can’t use them to plan and assess their own work
  • Not good for formative assessment
  • Students can’t use them to improve learning on the next task/assignment
  • Score/grade a specific task for summative assessment
General rubric – can be used with different tasks
  • Are generalized across different tasks assessing the same SLO
  • Clarify instructional content and learning outcomes
  • Help instructors to focus on student learning
  • Can be given to students in advance
  • Help students to understand what is expected of them
  • Help students to plan and self-assess their own work before submitting an assignment
  • Help students to recognize strengths and weaknesses
  • Students can use instructor feedback to improve performance
  • Time consuming to develop and establish scoring consistency
  

Is a Rubric Reliable and Valid?

Steps 5 and 6 address the reliability and validity of a rubric. Reliability describes whether the rubric measures student performance consistently. Validity describes whether the rubric measures what it is supposed to measure. More specifically, it describes whether the inferences made from applying the rubric are correct. If they are, i.e. the rubric is valid, then these interpretations are appropriate, meaningful, and useful.

Two forms of reliability are especially pertinent to developing rubrics, and both deal with those who will use the rubric to rate student performance. Interrater reliability refers to the extent that two or more scorers, or raters, yield consistent scores when applying the rubric. Intrarater reliability describes this attribute within a single rater using the rubric at different points in time, which is especially important for general rubrics.

Form of Reliability How You Can Address and Improve in Rubrics
Interrater Reliability – different individual raters apply the rubric consistently
  • Test and calculate a reliability coefficient. Interrater reliability = number of agreements/number of possible agreements. Values > 0.8 are acceptable.
  • Engage in rubric norming and recalculate the reliability coefficient until consistency is achieved (> 0.8)
Intrarater Reliability – the same rater applies the rubric consistently at different points in time
  • Develop well-defined scoring categories with clear differences in advance. Revisit these often while scoring to ensure consistency.
  • Create or gather and refer to examples that exemplify differences in scoring criteria.

Multiple forms of validity are important to rubric development and use:

Form of Validity How You Can Address and Improve in Rubrics
Content validity – the rubric reflects the knowledge and skills of the larger content domain Examine whether:
  • the rubric’s criteria reflect all of the intended content
  • extraneous content is present in the rubric criteria
  • important content is missing from rubric criteria
Construct validity – the rubric measures the constructs (interrelated skills, attributes, or behaviors based on theories) you intend it to measure Similar to content validity, examine whether:
  • the rubric’s criteria reflect all the important components of the construct
  • the rubric includes criteria that are irrelevant to the construct
  • important aspects of the construct are missing from rubric criteria
Criterion validity – the student’s performance measured by the rubric correlates to a related current or future event Examine whether:
  • rubric scoring criteria reflect skills or practices used in the discipline or future workplace
  • rubric scoring criteria reflect skills that would suggest future success in the discipline or future workplace
  • important skills or practices are missing from rubric criteria

Adapted from: Moskal and Leydens (2000), Scoring Rubric Development: Validity and Reliability.

Example Rubrics

Resources

  • Highly Recommended: Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. Alexandria: Association for Supervision & Curriculum Development.
  • Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation, 7(10), 1-6.
  • Instructure Community. Adding Rubrics in Canvas.