Adding Rubrics to Harmonize Discussions (Canvas)

  • Updated

Overview

Harmonize can take the instructions that you have built for your discussion topic, either on your own or through prompt generation feature, and build a set of focus areas or even an entire rubric that will be synced back to your course and available in SpeedGrader for grading.

Already have rubrics in your course that you'd like to use?  No problem! You can associate your existing rubrics with a Harmonize discussion assignment as well.

Please Note: Associating rubrics requires use of the Canvas API and requires an additional Developer Key to be installed with additional permissions by your Canvas Administrator. See Configuring Canvas API in Harmonize for more information.

 

Generating a Rubric Using the Wizard

After creating your instructions, you will have the choice to either Skip Rubric & AI Coach or Set up Rubric & AI Coach.

Button to set up rubric & AI Coach

 

A loading screen will appear while your focus areas are being generated.

Screenshot 2024-11-18 at 10.47.01 AM.png

 

You will be taken to a  focus areas draft that you can edit.

Focus area editing interface. See text equivalent below.
  • [1] Title and description fields where you can click and edit.
  • [2] Trash icons that you can click to delete focus areas.
  • [3] +Add Focus Area button, which you can click to add a new focus area.
  • [4] Not Coachable with AI warnings for any focus areas that will not be coachable.
  • [5] AI Student Coaching toggle for enabling or disabling this feature.
  • [6] Next button to click when you are happy with your rubric.

Note: If 1 or more of your criteria are marked "Not Coachable with AI," you can turn off AI Coaching or proceed with coaching enabled for the other areas. See What Does "Not Coachable" Mean?

After clicking Next, you will see the following pop-up if any of your focus areas are not coachable. You may choose Returning to Editing or Generate Full Rubric.

Pop-up to confirm non-coachable focus areas

After clicking Generate Full Rubric, you will see a loading screen. Then your rubric draft will appear.

Rubric editing interface. See text equivalent below.
  • [1] A pencil icon for the top row to edit the category names.
  • [2] Pencil icons for each focus area row to edit descriptions and points.
  • [3] Rotating arrows to regenerate an entire focus area row.
  • [4] Update Focus Areas button to return to the previous screen and edit focus areas.
  • [5] Save Rubric button to click when you're ready to move forward.

Your rubric is now complete.

 

What Does "Not Coachable" Mean?

When you generate a rubric, you will sometimes see a warning that a particular focus area is "Not Coachable with AI."

Not Coachable with AI button

This can happen for a few reasons.

Reasons for the "Not Coachable" Message

  • Subjectivity. The focus area involves something subjective that the AI can't evaluate. For example, "empathy toward classmates" or "analysis that is insightful and interesting."
  • Factuality. The focus area involves factuality or "correctness," which the AI can't evaluate. For example, "accurate statements about x topics."
  • Non-text components. The focus area requires assessment of non-text components, such as a student's recorded video or audio.

What to Do When Multiple Areas are Not Coachable

It's not an unusual to have at least one focus area that is not coachable. If you have multiple non-coachable focus areas, you have a few options.

  • Turn Off AI Coaching. If the focus areas accurately reflect your requirements but several of them are not coachable, consider turning off the coach.
  • Proceed with Coaching Partially Enabled. Maybe AI Coaching will only work for one or two of your focus areas, but you really want it to be available for those areas. In that case, proceed with AI Coaching enabled. Consider adding a warning/explanation to the assignment prompt.
  • Revise Focus Areas. Edit the focus areas to remove references to subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message). 
  • Revise Assignment. The focus areas are based on the assignment, so you may need to revise your initial prompt to de-emphasize subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message).

 

Generating or Editing a Rubric outside of the Wizard

To start, open your Harmonize discussion assignment and click the Edit button:

Edit discussion button

 

If you see an Authorize Canvas Extensions button at the top of the Topic Settings page, click it and follow the instructions for enabling syncing between Harmonize and your Canvas course (including rubric syncing):

Authorize canvas extensions button

 

Before you can begin generating a rubric, you need to instructions to your discussion.  You can do that by adding text in the Instructions tab or clicking on Generate Prompt to quickly build an engaging topic prompt (see our guide for more details: Harmonize Brainstorm):

Sample discussion topic showing generate prompt button

 

Now, scroll down to the bottom of the instructions and click on the Generate Focus Areas button:

Generate focus areas button

 

A dialog will open as your topic instructions are in the background to ChatGPT to produce a set of 4 criteria we call Focus Areas:

Rubrics: Generate Focus Areas Dialog Active Generation

 

Once this process completes, you'll be presented with 4 Focus Areas, each with a title and a description.  In this view, you can edit any of the titles or descriptions, removing any you do not want by clicking the trashcan button, and even add your own Focus Areas by clicking Add Focus Area button.  When you're happy with your Focus Areas, click the Next button:

Rubrics: Edit Focus Areas Next Button

 

At this point you, if you do not intend to grade with a rubric but want to use the Focus Areas as guide for student work, click the Use Focus Areas Only button.  If you intend to grade with a rubric, click the Generate Full Rubric button to continue with the full rubric generation:

Rubrics: Generate Rubric Decision Dialog

 

When you click Generate Full Rubric, Harmonize will take the Focus Areas and create 4 ratings for each focus area:

Rubrics: Generate Full Rubric Generate Ratings Active Generation

 

Once each Focus Area's ratings have been generated, you'll have several options for adjusting your rubric:

  • Click the Reverse Rating Order button to toggle order of the ratings
  • Click the Edit (pencil) button next to the rating labels to adjust their names
  • Click the Edit (pencil) button next to any Focus Area row to edit the text and/or the points of any of the ratings in that row
  • Click the Regenerate (cycle) button next to any Focus Area row to get updated rating text for each rating from ChatGPT (useful if you've recently changed the title or description of the Focus Area)
Rubrics: Edit Rubric Ratings Buttons

 

If you need to make any adjustments to the Focus Areas, click the Back button to return to the previous dialog.  Once you're happy with the rubric, click the Save Rubric button:

Rubrics: Edit Rubric Ratings Save Rubric Button

 

Your Focus Areas will now be viewable in your topic instructions.  Click the View Full Rubric button to view the entire rubric include descriptions for both Focus Areas and Ratings.  Make any other adjustments you need to your topic settings and then click the Publish button:

Discussion: Topic Settings Focus Areas Section Publish Button

 

Upon publishing, the Focus Areas and full rubric will be viewable to students as part of the instructions for the discussion and the rubric will be synced to your course in Canvas and associated to the assignment:

Discussion: Instructions Focus Areas Student View

 

Grading with Rubrics in Canvas

When you're ready to grade the discussion assignment in SpeedGrader, be sure to click the View Rubric button to expand out the rubric.  See the following Canvas guide for grading with rubrics in SpeedGrader: How do I use a rubric to grade submissions in SpeedGrader?

Canvas SpeedGrader: View Rubric Button

 

Associate an Existing Rubric with Harmonize

If you have an existing rubric in your Canvas course that you would like to use with your Harmonize discussion assignment, simply scroll to the bottom of the assignment page in Canvas and click the + Rubric button to select the rubric of your choosing:

Screenshot of Add Rubric button at the bottom of an assignment in Canvas.

 

Frequently Asked Questions

How do I grade with a rubric in a different Learning Management System?

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request