Overview
Harmonize can take the instructions that you have built for your discussion topic, either on your own or through prompt generation feature, and build a set of focus areas or even an entire rubric that will be available in Grades & Activity for grading.
Generating a Rubric Using the Wizard
After creating your instructions, you will have the choice to either Skip Rubric & AI Coach or Set up Rubric & AI Coach.
A loading screen will appear while your focus areas are being generated.
You will be taken to a focus areas draft that you can edit.
- [1] Title and description fields where you can click and edit.
- [2] Trash icons that you can click to delete focus areas.
- [3] +Add Focus Area button, which you can click to add a new focus area.
- [4] Not Coachable with AI warnings for any focus areas that will not be coachable.
- [5] AI Student Coaching toggle for enabling or disabling this feature.
- [6] Next button to click when you are happy with your rubric.
Note: If 1 or more of your criteria are marked "Not Coachable with AI," you can turn off AI Coaching or proceed with coaching enabled for the other areas. See What Does "Not Coachable" Mean?
After clicking Next, you will see the following pop-up if any of your focus areas are not coachable. You may choose Returning to Editing or Generate Full Rubric.
After clicking Generate Full Rubric, you will see a loading screen. Then your rubric draft will appear.
- [1] A pencil icon for the top row to edit the category names.
- [2] Pencil icons for each focus area row to edit descriptions and points.
- [3] Rotating arrows to regenerate an entire focus area row.
- [4] Update Focus Areas button to return to the previous screen and edit focus areas.
- [5] Save Rubric button to click when you're ready to move forward.
Your rubric is now complete.
What Does "Not Coachable" Mean?
When you generate a rubric, you will sometimes see a warning that a particular focus area is "Not Coachable with AI."
This can happen for a few reasons.
Reasons for the "Not Coachable" Message
- Subjectivity. The focus area involves something subjective that the AI can't evaluate. For example, "empathy toward classmates" or "analysis that is insightful and interesting."
- Factuality. The focus area involves factuality or "correctness," which the AI can't evaluate. For example, "accurate statements about x topics."
- Non-text components. The focus area requires assessment of non-text components, such as a student's recorded video or audio.
What to Do When Multiple Areas are Not Coachable
It's not an unusual to have at least one focus area that is not coachable. If you have multiple non-coachable focus areas, you have a few options.
- Turn Off AI Coaching. If the focus areas accurately reflect your requirements but several of them are not coachable, consider turning off the coach.
- Proceed with Coaching Partially Enabled. Maybe AI Coaching will only work for one or two of your focus areas, but you really want it to be available for those areas. In that case, proceed with AI Coaching enabled. Consider adding a warning/explanation to the assignment prompt.
- Revise Focus Areas. Edit the focus areas to remove references to subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message).
- Revise Assignment. The focus areas are based on the assignment, so you may need to revise your initial prompt to de-emphasize subjectivity, factuality, and non-text components (see Reasons for the "Not Coachable" Message).
Generating or Editing a Rubric outside of the Wizard
To start, open your Harmonize discussion assignment and click the Edit button:
If you see an Authorize Canvas Extensions button at the top of the Topic Settings page, click it and follow the instructions for enabling syncing between Harmonize and your Canvas course (including rubric syncing):
Before you can begin generating a rubric, you need to instructions to your discussion. You can do that by adding text in the Instructions tab or clicking on Generate Prompt to quickly build an engaging topic prompt (see our guide for more details: Harmonize Brainstorm):
Now, scroll down to the bottom of the instructions and click on the Generate Focus Areas button:
A dialog will open as your topic instructions are in the background to ChatGPT to produce a set of 4 criteria we call Focus Areas:
Once this process completes, you'll be presented with 4 Focus Areas, each with a title and a description. In this view, you can edit any of the titles or descriptions, removing any you do not want by clicking the trashcan button, and even add your own Focus Areas by clicking Add Focus Area button. When you're happy with your Focus Areas, click the Next button:
At this point you, if you do not intend to grade with a rubric but want to use the Focus Areas as guide for student work, click the Use Focus Areas Only button. If you intend to grade with a rubric, click the Generate Full Rubric button to continue with the full rubric generation:
When you click Generate Full Rubric, Harmonize will take the Focus Areas and create 4 ratings for each focus area:
Once each Focus Area's ratings have been generated, you'll have several options for adjusting your rubric:
- Click the Reverse Rating Order button to toggle order of the ratings
- Click the Edit (pencil) button next to the rating labels to adjust their names
- Click the Edit (pencil) button next to any Focus Area row to edit the text and/or the points of any of the ratings in that row
- Click the Regenerate (cycle) button next to any Focus Area row to get updated rating text for each rating from ChatGPT (useful if you've recently changed the title or description of the Focus Area)
If you need to make any adjustments to the Focus Areas, click the Back button to return to the previous dialog. Once you're happy with the rubric, click the Save Rubric button:
Your Focus Areas will now be viewable in your topic instructions. Click the View Full Rubric button to view the entire rubric include descriptions for both Focus Areas and Ratings. Make any other adjustments you need to your topic settings and then click the Publish button:
Upon publishing, the Focus Areas and full rubric will be viewable to students as part of the instructions for the discussion and the rubric will be synced to your course in Canvas and associated to the assignment:
Grading with Auto-Generated Rubrics
- Begin by clicking Grades & Activity.
- This will open the "All Students" view of the Harmonize Discussion Activity Report.
- Click on any student name to open the Individual Activity view for that students. Alternatively, select a student from the "Showing Activity for" drop down menu.
- The rubric that you created will be available to the right of the Individual Student Activity window.
Rubric Feedback
- Click the arrows to expand each Focus Area in the Rubric Feedback section.
- Click the appropriate rating. The allotted points will automatically appear in the associated Focus Area points field. Feel free to to change the number of points as needed.
- Optionally, you may enter specific feedback into the Focus Area Comment field.
- To discard your changes, click the Discard button.
Overall Feedback
- The total sum of points from the individual Focus Area points fields above will automatically appear in the Overall Grade field.
- To manually adjust this total, first toggle "Override Grade" to ON, then adjust the grade as needed.
- Enter any grade comments in to the Gradebook Comment field. These comments will automatically appear in the Gradebook for students to view once overall grading is submitted.
- To include your Focus Area Comments in the grade submission toggle Include Rubric Feedback to ON.
- To preview your grade submission, click the Preview Submission button.
- When you are finished grading, click Submit & Next. Your grade submission will be sent to the corresponding column and row of your LMS gradebook.
Frequently Asked Questions
Q. Will any of my auto-generated discussion prompts or rubrics be used outside of Harmonize?
- All of your auto-generated discussion prompts and rubrics are private and will not be used outside of Harmonize.