What Is It?
AI Coaching lets your students get real-time feedback on their writing, empowering them to reflect and revise before their final submission. But what is it, and how does it work? The AI Coach evaluates your students’ submissions–both posts and comments–against a rubric you create in our system. The AI Coach doesn’t give specific feedback or suggestions (since these can be unreliable) but instead indicates how well students are doing in each focus area (a task where the AI is more reliable). Here’s what the AI Coach’s feedback looks like:
Pro Tip: Some items will be marked “Human Assessment Needed” if the AI determines it can’t accurately assess them.
Why Use the AI Coach?
- Self-assessment with rubrics improves student writing.
- Revising and reflecting promote writing achievement.
- First-generation college students are less likely to ask instructors for help. AI Coaching makes it easier to request and receive feedback.
Watch our half-hour webinar on AI Coaching
How to Set up AI Coach
- Create a Harmonize Discussion.
- Create a prompt from scratch or get help from Harmonize Brainstorm.
- Use the Rubric Generator to create a rubric for your assignment.
- Enable AI Coaching.
- Optional: View Student Feedback History.
What It Can and Can’t Do
The AI Coach will:
- Provide real-time feedback.
- Indicate strength or weakness in different focus areas.
- Provide students with data they can use to self-assess their work.
- Encourage reflection and revision.
The AI Coach will not:
- Revise work for the student.
- Offer specific feedback.
- Assign a grade to the student’s work.
- Coach on non-text content.
General Tips and Strategies
Explain the purpose.
Let students know they can always come to you for feedback, but the AI Coach provides real-time feedback that might help them improve their drafts on their own. The AI Coach is really a tool for self assessment and improvement.
Encourage using the AI Coach.
Encourage students to run the AI Coach on their discussion posts. Let them know you can see when they’ve run the AI Coach and revised their post (using View Student Feedback History), and be sure they know it’s a good thing! Students might be more likely to use the AI Coach if they know you can see (and fully approve of!) its use.
Give them a process.
Suggest that students begin the assignment early so they will have time to run the AI Coach and revise. Suggest revising once or twice or until they are satisfied with their score.
Pro Tip: Invite students to reach out to you if they’re revising their work but not seeing their AI Coaching evaluation improve.
Acknowledge that the AI isn’t perfect.
Make sure students know that the AI Coach can get it wrong. The AI Coach is a tool for them to use, but they have to decide what to use and what to ignore. This is similar to how peer review works: the feedback isn’t always perfect, so students need to learn to be discerning.
Double check the rubric
Be sure the rubric clearly conveys your expectations. You may need to revise the rubric to make it match. The AI Coach will evaluate students on whatever is in the rubric, so be sure it aligns with your grading criteria and your desired level of difficulty.
Pro Tip: The AI-generated rubrics are sometimes “too hard” compared to what teachers are really expecting.
Change visibility.
If you want students to reflect on and revise their writing privately, consider switching visibility to “Private between student and instructor.”
Don’t forget comments!
Consider setting a milestone for posting and another one for commenting. In the comment, they can explain what changes they made based on the AI Coach’s feedback. You can even make this reflection be one of the rubric criteria.
Specific Ideas
| Assignment | What | Why |
| Richer Discussions | Use Harmonize to generate a rubric for your discussion and then encourage students to run the AI Coach, which will evaluate both their posts and their comments. | Your students can make stronger contributions–and give their peers better feedback–with help from the AI Coach. |
| Essay Draft | Have students submit their drafts to a Harmonize discussion set to private visibility. Use Harmonize to generate a rubric. Set a milestone submitting the draft and another for running the AI Coach against the rubric and reflecting on the results. | Give students a chance to self-assess their work against the rubric with some support from the AI Coach. |
| Peer Review | Consider running peer review within Harmonize. In Canvas and Blackboard, you can even use “Groups.” Students can view each other’s feedback AND see the AI Coach’s feedback. | Students may not trust peer review or may want another data point. Consider doing peer review with the AI Coach enabled for comparison. |
| Self Evaluation | Have students submit their work and evaluate it with the AI Coach. Then ask students to provide their own evaluation, explaining if they agree or disagree with the AI Coach and why. | Helps develop metacognitive skills. Helps students develop a critical eye for their own work. |
Further Reading
- “Revising with Metacognition to Promote Writing Achievement: A Case Study” by Cassandra O'Sullivan Sachar
- “Self-Assessment Through Rubrics” by Heidi Andrade
- “Comparing the Quality of Human and ChatGPT Feedback on Students’ Writing” by Steiss et. al.