Overview
Harmonize can take the instructions that you have built for your discussion topic, either on your own or through prompt generation feature, and build a set of focus areas or even an entire rubric that will be synced back to your course and available in SpeedGrader for grading.
Generating a Rubric Using the Wizard
Watch the video or read the text instructions below it to generate a rubric using the wizard.
After creating your instructions, you will have the choice to either Skip Rubric & AI Coach or Set up Rubric & AI Coach.
A loading screen will appear while your focus areas are being generated.
You will be taken to a focus areas draft that you can edit.
- [1] Title and description fields where you can click and edit.
- [2] Trash icons that you can click to delete focus areas.
- [3] +Add Focus Area button, which you can click to add a new focus area.
- [4] Not Coachable with AI warnings for any focus areas that will not be coachable.
- [5] AI Student Coaching toggle for enabling or disabling this feature.
- [6] Next button to click when you are happy with your rubric.
Note: If 1 or more of your criteria are marked "Not Coachable with AI," you can turn off AI Coaching or proceed with coaching enabled for the other areas. See What Does "Not Coachable" Mean?
After clicking Next, you will see the following pop-up if any of your focus areas are not coachable. You may choose Returning to Editing or Generate Full Rubric.
After clicking Generate Full Rubric, you will see a loading screen. Then your rubric draft will appear.
- [1] A pencil icon for the top row to edit the category names.
- [2] Pencil icons for each focus area row to edit descriptions and points.
- [3] Rotating arrows to regenerate an entire focus area row.
- [4] Update Focus Areas button to return to the previous screen and edit focus areas.
- [5] Save Rubric button to click when you're ready to move forward.
Your rubric is now complete.
What Does "Not Coachable" Mean?
When you generate a rubric, you will sometimes see a warning that a particular focus area is "Not Coachable with AI."
This can happen for a few reasons.
Reasons for the "Not Coachable" Message
- Subjectivity. The focus area involves something subjective that the AI can't evaluate. For example, "empathy toward classmates" or "analysis that is insightful and interesting."
- Factuality. The focus area involves factuality or "correctness," which the AI can't evaluate. For example, "accurate statements about x topics."
- Non-text components. The focus area requires assessment of non-text components, such as a student's recorded video or audio.
What to Do When Multiple Areas are Not Coachable
It's not an unusual to have at least one focus area that is not coachable. If you have multiple non-coachable focus areas, you have a few options.
- Turn Off AI Coaching. If the focus areas accurately reflect your requirements but several of them are not coachable, consider turning off the coach.
- Proceed with Coaching Partially Enabled. Maybe AI Coaching will only work for one or two of your focus areas, but you really want it to be available for those areas. In that case, proceed with AI Coaching enabled. Consider adding a warning/explanation to the assignment prompt.
- Revise Focus Areas. Edit the focus areas to remove references to subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message).
- Revise Assignment. The focus areas are based on the assignment, so you may need to revise your initial prompt to de-emphasize subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message).
Generating or Editing a Rubric outside of the Wizard
Watch the video or read the text instructions below it to generate a rubric using the editor.
To start, open your Harmonize discussion assignment and click the Edit button:
If you see an Authorize Canvas Extensions button at the top of the Topic Settings page, click it and follow the instructions for enabling syncing between Harmonize and your Canvas course (including rubric syncing):
Before you can begin generating a rubric, you need to instructions to your discussion. You can do that by adding text in the Instructions tab or clicking on Generate Prompt to quickly build an engaging topic prompt (see our guide for more details: Harmonize Brainstorm):
Now, scroll down to the bottom of the instructions and click on the Generate Focus Areas button:
A dialog will open as your topic instructions are in the background to ChatGPT to produce a set of 4 criteria we call Focus Areas:
Once this process completes, you'll be presented with 4 Focus Areas, each with a title and a description. In this view, you can edit any of the titles or descriptions, removing any you do not want by clicking the trashcan button, and even add your own Focus Areas by clicking Add Focus Area button. When you're happy with your Focus Areas, click the Next button:
At this point you, if you do not intend to grade with a rubric but want to use the Focus Areas as guide for student work, click the Use Focus Areas Only button. If you intend to grade with a rubric, click the Generate Full Rubric button to continue with the full rubric generation:
When you click Generate Full Rubric, Harmonize will take the Focus Areas and create 4 ratings for each focus area:
Once each Focus Area's ratings have been generated, you'll have several options for adjusting your rubric:
- Click the Reverse Rating Order button to toggle order of the ratings
- Click the Edit (pencil) button next to the rating labels to adjust their names
- Click the Edit (pencil) button next to any Focus Area row to edit the text and/or the points of any of the ratings in that row
- Click the Regenerate (cycle) button next to any Focus Area row to get updated rating text for each rating from ChatGPT (useful if you've recently changed the title or description of the Focus Area)
If you need to make any adjustments to the Focus Areas, click the Back button to return to the previous dialog. Once you're happy with the rubric, click the Save Rubric button:
Your Focus Areas will now be viewable in your topic instructions. Click the View Full Rubric button to view the entire rubric include descriptions for both Focus Areas and Ratings. Make any other adjustments you need to your topic settings and then click the Publish button:
Upon publishing, the Focus Areas and full rubric will be viewable to students as part of the instructions for the discussion and the rubric will be synced to your course in Canvas and associated to the assignment:
Frequently Asked Questions
Q: How do I grade with a rubric?
- Canvas users should see Grading with Rubrics in Canvas. All others should see Grading with Auto-Generated Rubrics.
Q: I have an existing rubric in Canvas that I want to use. Can I?
Q: I accidentally closed the Wizard. Can I return to set up my rubric?
If you were in the middle of editing and navigating away, you should be able to return and resume. If you chose to configure the discussion manually, or if you already completed the setup process once, you won't be able to access the wizard again. In this case, you will need to see Generating a Rubric Using the Wizard.