In the realm of education, rubrics are pivotal in guiding students and assessing their work. However, a concerning trend has emerged with the advent of AI teacher tools that promise to magically conjure lesson plans, quizzes, and rubrics. The issue at hand is not the AI itself but the misconception that its generated rubrics are the epitome of quality assessment. I argue that while AI can aid in phrasing and structure, it shouldn’t be hailed as the authority on crafting quality rubrics. In this post, I’ll dissect what constitutes a sound rubric, underscore the necessity for teacher training in this area, and provide guidance on creating mathematically sound and pedagogically effective rubrics.
Using AI Teaching Tools
Teaching is hard. We all need some help. AI absolutely can aide in teacher tasks. However, it is important we do not put blind trust in the outputs. Ultimately, you are highly trained in effective teaching methods and the AI tool is not. It simply looks for patterns in the sources it was trained on and those are not necessarily quality sources.
Teaching Tools Should Do Better
To elevate the utility and relevance of AI as a teaching tool, it’s important that these systems be trained with an in-depth understanding of the educational frameworks and pedagogical strategies that underpin effective teaching. The intricacies of lesson planning, with its multifaceted layers and nuanced considerations, pose a significant challenge for teachers strapped for time. AI, when adeptly trained in the principles of sound lesson structuring, rubric formulation, and assessment creation, could transcend its current capabilities to genuinely assist educators in crafting superior educational experiences.
Rubrics Creation Tools Should Be Trained
However, the rapid proliferation of AI tools in the educational sector, seemingly emerging overnight, raises substantial concerns regarding the quality and thoughtfulness of these solutions. Upon evaluating one such teacher marketed tool, I encountered a stark disconnect between its promised efficiency and the actual quality of the lessons and rubrics it generated. The experience was disappointing, highlighting the tool’s inadequacy in facilitating the creation of high-caliber lessons and its failure to understand the essence of a meaningful and effective rubric.
It is one thing to use ChatGPT or Bard to assist with lesson and rubric creation where you know you need to adjust for the lens of teaching. It is another to use a tool that claims to be designed for teachers but is little more than a fancy facade for ChatGPT without training the model for high quality teaching and rubric creation. Claiming it is made for teachers might give the false impression that it designed to create high quality teacher tasks.
The Allure and Pitfalls of AI-Generated Rubrics
The allure of AI in education is undeniable. It offers the promise of efficiency and innovation. However, this allure becomes problematic when AI-generated rubrics are seen as an authority on rubrics. It leads educators down a path where the art of teaching and the science of assessment are overshadowed by the convenience of automation.
My Attempt at an AI Rubric
I asked this teacher tool with a specific rubric feature to create one based on the standard. The result was horrifying.
What Is the Purpose of a Rubric?
We can probably all agree that a rubric looks something like a table. It has a list of the criteria the evaluator (teacher) is looking for and columns to evaluate how well the student has met the criteria.
It gets a little, or a lot, muddy after that. Is it for communicating standards mastery or for guidance in scoring? Is it about the standard or the expectations of the assignment itself?
Some experts I like to look to for guidance on better rubric creation are Dr. Susan Brookhart, Thomas Guskey, and Rick Wormeli.
Before using an AI tool to generate a rubric, research some of the different elements to consider in a high quality rubric to be able to evaluate and adapt the AI rubric’s rough draft.
| Researcher | Key Point 1 | Key Point 2 |
|---|---|---|
| Susan M. Brookhart | Rubrics should make the criteria for judging learning outcomes transparent. | Rubrics need to provide specific and relatable performance descriptions. |
| Thomas R. Guskey | Rubrics should align with learning standards and objectives. | The highest level of performance in rubrics should define what an ‘A’ grade represents. |
| James H. McMillan | Focuses on the practical application of rubrics in educational assessments. | Advocates for rubrics to provide clear and understandable performance expectations. |
| Heidi Andrade | Stresses the importance of rubrics in self-assessment and student learning. | Highlights how rubrics help students understand and improve their own learning. |
| Rick Stiggins | Emphasizes the role of rubrics in formative assessment. | Suggests rubrics are crucial for students to understand the criteria for success. |
| Dylan Wiliam | Discusses the role of rubrics in providing effective feedback. | Focuses on how rubrics foster student engagement and involvement in the learning process. |
| Rick Wormeli | Advocates for rubrics to clearly articulate proficiency and growth. | Emphasizes the need for rubrics to adapt to different levels of student understanding. |
Rubrics Use Bad Math
Prior to AI creating rubrics, there has been a challenge with understanding the math behind how some rubrics are constructed. The levels of a rubric are not a percentage.
It is tricky, to nearly impossible, to accurately use a grid of criteria to determine a students true achievement on an assessment. A made up weighting of categories has good intentions, but fails to take into account how the elements weave together. A single point rubric is one way to tackle that, but let’s leave that for another discussion.
The Horrors of Bad Math
If you have a 4 point rubric, then your grade options are A+, C, F, and really low F. This can not possibly be what the criteria is describing.
This is even worse with a 3 point rubric: A+, D, and Low F.
Categories offering 25% or 33% have no place on a scoring rubric.
Rubrics should be about helping students find success and clearly communicating how to be successful. Not about assigning failure.
Skip Exceeds Expectations
This is not universally agreed upon, but skip “Exceeds expectations” as a rubric category. To earn full credit a student should meet the expectations. You can communicate if they exceeded them, but that should not result in a reduced score.
What Do The Levels Mean?
If I said that a student approaches expectations what letter grade pops into your mind? Certainly not a D or an F!! Yet this is the mathematical trap some rubrics fall into when we allow for a 1 point gap between the columns.
Example Headings
| Rubric Heading | Potential Corresponding Letter Grade |
|---|---|
| Meets Expectations | A |
| Approaches Expectations | B |
| Partially Meets Expectations | C |
| Does Not Meet Expectations | D or F |
A Scoring Guide Vs. A Criteria Rubric
Scoring Guide
A scoring guide is utilized with the intention of “fairly” evaluating student work between students. Each cell on the rubric is assigned a point value and those points are summed up.
Criteria Rubric
A criteria rubric might be looked at more holistically. For the column, the rows detail the overall qualities of that level. Students may display higher or lower quality in some areas, but overall did the student meet a certain level. “Overall, this student mostly met the criteria for ‘approaches expectations’ and thus should fall into the B range.” Use some professional judgement as to if that is a high, middle or low B.
Statistically there is no difference in a 79 and an 80 on a 100 point scale so I would stay away from 80 myself as you’re essentially assigning a a C+.
A Poorly Designed AI Rubric
I could not have been more horrified when using the designated rubric creator of a popular teacher AI tool.
I would expect the rubric tool on an AI site created for teachers would front load the AI prompt with what makes a good rubric.
3 Points Exceeds
This particular site defaulted to a 3 point rubric, which I am in favor of having fewer columns. However, the headings indicated 3 points, 2 points, and 1 point. Which translates to 100%, 66%, and 33%. Is it perfection or failing as the only options? Worse, the key at the bottom indicated that the 3 was only for exceeds the expectations. So “meets expectations” is a D. That is a major problem.
“The letter lacks neatness” has no business in assessing the standard of using punctuation.
Standard: W.7.11 Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing. a. Use a comma to separate coordinate adjectives. b. Spell correctly.
Criteria | 3 Points | 2 Points | 1 Point |
Content | Student clearly identifies the problem and suggests a solution in the letter. The content is well-developed and relevant. | Student identifies the problem and suggests a solution in the letter. The content is somewhat developed and mostly relevant. | Student provides limited or unclear identification of the problem and/or solution. The content is underdeveloped and lacks relevance. |
Capitalization, Punctuation, and Spelling | Student consistently demonstrates command of standard English capitalization, punctuation, and spelling throughout the letter. | Student demonstrates some command of standard English capitalization, punctuation, and spelling with occasional errors that do not significantly impact understanding. | Student demonstrates limited command of standard English capitalization, punctuation, and spelling, with frequent errors that impact understanding. |
Comma Usage | Student consistently uses commas to separate coordinate adjectives in the letter. | Student uses commas to separate coordinate adjectives in the letter with occasional errors that do not significantly impact understanding. | Student inconsistently or incorrectly uses commas to separate coordinate adjectives in the letter, leading to confusion or lack of clarity. |
Overall Presentation | The letter is neatly written or typed, with appropriate formatting, and demonstrates attention to detail in organization and presentation. | The letter is generally neat and legible, with few formatting issues, but may lack some attention to detail in organization and presentation. | The letter lacks neatness or legibility, has formatting issues, and demonstrates minimal attention to detail in organization and presentation. |
Note: The point scale for each criterion is as follows:
- 3 Points: Exceeds expectations
- 2 Points: Meets expectations
- 1 Point: Below expectations
How About ChatGPT 4
I wanted to compare the output of the site designed for teachers with just plain old ChatGPT. I think ChatGPT did a better job on the standard. My only prompt was to make a rubric based on the standard. What I normally would do it feed the prompt with what makes a good rubric. I still strongly dislike the 1 point gap between the columns. The points should represent the percentage that heading embodies. Proficient certainly should not be a low C to a D.
| Criteria/Performance Level | Advanced (4 points) | Proficient (3 points) | Needs Improvement (2 points) |
|---|---|---|---|
| Use of Commas to Separate Coordinate Adjectives (W.7.11a) | Consistently uses commas correctly to separate coordinate adjectives in all sentences. | Generally uses commas correctly to separate coordinate adjectives with minor errors. | Demonstrates a basic understanding but makes frequent errors in using commas to separate coordinate adjectives. |
| Spelling (W.7.11b) | Spells all words correctly, including complex and subject-specific vocabulary. | Spells most words correctly with occasional spelling errors that do not impede understanding. | Makes frequent spelling errors, indicating a need for further learning and practice. |
Overall Performance Levels:
- Advanced (8-7 points): Demonstrates an excellent command of standard English conventions, with no or very few minor errors.
- Proficient (6-5 points): Shows a good understanding of standard English conventions with minor errors that do not hinder communication.
- Needs Improvement (4-0 points): Needs to improve understanding and application of standard English conventions; frequent errors are present, affecting clarity and coherence of writing.
Bard Falls Into the Same Trap
| Criteria | Exceeds Expectations (4 pts) | Meets Expectations (3 pts) | Approaching Expectations (2 pts) | Does Not Meet Expectations (1 pt) |
|---|---|---|---|---|
| Uses a comma to separate coordinate adjectives correctly. | Consistently and accurately uses commas to separate coordinate adjectives. | Mostly uses commas to separate coordinate adjectives correctly, with few errors. | Inconsistent use of commas to separate coordinate adjectives, with some errors. | Frequent errors in using commas to separate coordinate adjectives. |
| Spells all words correctly. | Spells all words correctly, even in challenging or unfamiliar contexts. | Spells most words correctly, with occasional minor errors. | Spells some words incorrectly, including some common errors. | Frequent spelling errors, including common errors and more challenging words. |
Additional notes:
- Coordinate adjectives: These are adjectives that equally describe a noun and can be reversed in order without changing the meaning (e.g., “The tall, green tree” or “The green, tall tree”).
- Spelling: Consider the difficulty of the words being used and the context of the writing.
- Adapt as needed: You may need to adjust the rubric based on your specific grade level or student needs.
- Provide examples: When giving feedback to students, provide specific examples of their strengths and areas for improvement.
- Encourage self-assessment: Allow students to use the rubric to assess their own work and set goals for improvement.
Google Bard is free and it did pretty much as good as the ChatGPT 4 that I pay $20 a month for. However, it falls into the same trap of the 1 point difference between columns. Which results in meets expectations as 75%, which is clearly ridiculous.
Making a Rubric with ChatGPT
Why use a so called teacher AI website when it is not trained on good pedagogy?
Use ChatGPT or Google Bard to request those same things. However, in your prompt instruct the generative AI to consider different pedagogical approaches.
AI is definitely helpful at filling in the boxes for an initial rough draft, but you have to first teach it what a good rubric looks like.
Prompt I Used
I am teaching this standard: W.7.11 Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing. a. Use a comma to separate coordinate adjectives. b. Spell correctly.
I need a lesson in the style of the 5 E’s. It needs to be student centered. Do not use worksheets. Students need to demonstrate creative thinking, critical thinking of at least DOK level 2, collaboration, and communicate their ideas. The lesson should be engaging. It should incoporate student interests. Many of my students are interested in Roblox or soccer. Create a meaningful assessment that involves students applying their knowledge of the standard in a real world context. Create a rubric that aligns meets expectations as 100%, Approaches expectations as 85%, Below expectations as 75%, and far below expectations as 65%.
From ChatGPT 4
Lesson Plan: Mastering English Conventions through Creative Storytelling
Standard: W.7.11 – Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing.
- a. Use a comma to separate coordinate adjectives.
- b. Spell correctly.
Lesson Overview:
This lesson uses the 5 E’s instructional model (Engage, Explore, Explain, Elaborate, Evaluate) to help students understand and apply the conventions of standard English. The lesson incorporates interests like Roblox and soccer to create an engaging and student-centered learning environment. Students will demonstrate creative and critical thinking, collaboration, and communication.
1. Engage (Setting the Context)
- Activity: Introduce a story relay game. Start a story with a sentence related to soccer or Roblox that includes coordinate adjectives. For example, “In the vast, open field, the skilled, agile player maneuvered the ball.”
- Objective: Capture students’ interest and curiosity, connect to their interests, and introduce the concept of using commas to separate coordinate adjectives in a fun, interactive way.
2. Explore (Investigation and Discovery)
- Activity: In small groups, students continue the story. Each student adds a sentence, ensuring the use of coordinate adjectives. Encourage the use of descriptive, creative language relevant to their interests (e.g., describing a soccer match or a Roblox adventure).
- Objective: Provide an opportunity for students to experiment with the standard in a collaborative setting, promoting peer learning and creative thinking.
3. Explain (Concept Introduction and Clarification)
- Activity: Discuss as a class. Highlight and clarify the correct use of commas with coordinate adjectives and spelling in the sentences students created. Address any misconceptions or errors in a constructive manner.
- Objective: Ensure students understand the standard through teacher clarification and peer examples. Encourage students to ask questions and express their understanding.
4. Elaborate (Application and Extension)
- Activity: Students create their own short story or dialogue based on a soccer game or a Roblox adventure. They must incorporate at least five examples of coordinate adjectives separated by commas and ensure correct spelling throughout the story.
- Objective: Allow students to apply their understanding in a creative context, integrating their interests to make the learning experience more relevant and engaging.
5. Evaluate (Assessment and Reflection)
- Activity: Peer review and group presentation. Students exchange stories with another group, provide constructive feedback on the use of commas and spelling, and then present the story to the class.
- Objective: Foster a supportive learning environment where students can assess each other’s work and reflect on their own learning.
Assessment:
- Real-World Application: Students will create a blog post or a video script for a new game strategy or a match report, applying their knowledge of standard English conventions. This task connects their learning to a real-world context, enhancing the relevance and utility of the lesson.
Rubric for Assessment:
| Criteria | Meets Expectations (100%) | Approaches Expectations (85%) | Below Expectations (75%) | Far Below Expectations (65%) |
|---|---|---|---|---|
| Use of Commas with Coordinate Adjectives | Correctly uses commas to separate coordinate adjectives in all instances. | Generally uses commas correctly with minor errors. | Demonstrates a basic understanding but makes frequent errors. | Lacks understanding of comma usage, with consistent errors. |
| Spelling | No spelling errors. | Occasional spelling errors that do not impede understanding. | Frequent spelling errors, indicating a need for further practice. | Persistent spelling errors, significantly hindering understanding. |
| Creativity and Relevance | Story is highly creative, engaging, and effectively integrates student interests. | Story is creative and integrates student interests but lacks some engagement. | Story shows an attempt at creativity and relevance but is somewhat disengaged. | Story lacks creativity, engagement, and relevance to student interests. |
| Peer Review and Presentation | Provides insightful feedback during peer review and presents the story confidently and clearly. | Provides useful feedback and presents the story with minor issues in clarity or confidence. | Feedback is basic and presentation lacks clarity or confidence. | Feedback is not constructive and presentation is unclear or disorganized. |
This lesson plan and rubric aim to provide a comprehensive, engaging, and student-centered approach to mastering standard W.7.11, aligning assessment with real-world application and student interests.
From Google Bard
5 E’s Lesson: Mastering Punctuation and Spelling for Real-World Impact
Grade Level: 7th
Standard: W.7.11 Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing.
Objectives:
- Students will explain the function of commas to separate coordinate adjectives.
- Students will identify and correct misspelled words, applying various strategies.
- Students will collaboratively create and revise engaging content (Roblox story or soccer match description) incorporating correct punctuation and spelling.
- Students will present their creations, providing constructive feedback to their peers.
Engage (15 minutes):
Interest Grabber: Present two versions of the same sentence, one with correct punctuation and spelling (“The exciting, action-packed Roblox adventure unfolded…”) and one with errors (“the exsiting actionpcked roblox adventuer unfoldeD…). Ask students which version is more professional and impactful. Discuss the importance of accurate writing in real-world contexts.
Student Choice: Offer two content themes, relatable to student interests:
- Roblox Challenge: Explore examples of well-written and engaging Roblox stories. Identify sentences with effective use of commas and correct spelling. Discuss the impact of these elements on reader engagement and immersion.
- Soccer Showdown: Analyze sports commentary excerpts, highlighting the importance of clarity and accuracy in describing action. Show examples of comma usage for listing adjectives and descriptive phrases related to players’ skills and movements.
Explore (20 minutes):
Interactive Activity: Divide students into small groups based on their chosen content theme (Roblox/Soccer).
- Roblox Group: Create a “Commas and Cools” scavenger hunt in a popular Roblox game. Plant sentences with and without correct comma usage within the game environment. Groups race to find and identify sentences, explaining their reasoning for comma placement.
- Soccer Group: Play a short video clip of a soccer match with no commentary. Challenge groups to collectively narrate the action, focusing on describing players and their movements using vivid adjectives. Provide prompts to encourage appropriate comma usage.
Spelling Strategies: Share mnemonic devices, online dictionaries, and word roots/prefixes/suffixes resources to equip students with effective spelling strategies.
Explain (15 minutes):
- Commas for Clarity: Use a whiteboard or visual aid to present a clear explanation of comma usage for separating coordinate adjectives. Use examples from the “Explore” activities.
- Spelling Sleuths: Review common spelling errors and discuss why correct spelling matters in various contexts (professional communication, online gaming, social media).
Elaborate (30 minutes):
- Creative Corner: Students return to their chosen content theme groups.
- Roblox Group: Each group collaboratively writes a short, engaging Roblox story incorporating correct comma usage and accurate spelling. Encourage descriptive language and clear narrative structure.
- Soccer Group: Groups draft a descriptive commentary for the same soccer clip used earlier, ensuring accurate comma placement and vivid adjective choices.
- Peer Revision: Groups exchange their creations and provide constructive feedback using a checklist focusing on comma usage and spelling accuracy. Students should suggest improvements while offering positive reinforcement.
Evaluate (20 minutes):
- Presentation Powerhouse: Each group presents their final draft (Roblox story or soccer commentary) to the class. Encourage expressive reading or dynamic narration depending on the theme.
- Real-World Relevance: Discuss the importance of clear and accurate communication in various applications: online gaming, social media interactions, personal writing, and future career pursuits.
- Metacognition Reflection: Ask students to reflect on their learning experience using prompts like “What was the most challenging aspect of using commas?”, “Which spelling strategies were most helpful?”, and “How can I improve my punctuation and spelling in my daily writing?”.
Assessment:
- Peer feedback checklist: Completed checklist by each group on the other group’s creation (50%)
- Presentation: Clarity, creativity, and accurate use of commas and spelling (30%)
- Metacognition reflection: Thoughtful responses demonstrating personal learning and understanding (20%)
Rubric:
| Criteria | Meets Expectations (100%) | Approaches Expectations (85%) | Below Expectations (75%) | Far Below Expectations (65%) |
|---|---|---|---|---|
| Comma Usage: | Follows comma rules for coordinate adjectives consistently with few to no errors. | Mostly correct comma usage with some minor errors or inconsistencies. | Frequent errors in comma placement. | Consistent disregard for comma rules. |
| Spelling: | Spells all words correctly, including challenging or unfamiliar vocabulary. | Occasional minor spelling errors in complex words. | Several spelling errors, including common mistakes and challenging words. | Frequent and/or major spelling errors that hinder meaning. |
| Content Engagement: | Creative and engaging story |
Keep Arguing with AI
The above samples are simply the first draft in my utilizing AI to assist with lesson planning and a rubric. What is great about generative AI is the ability to continue the conversation. Provide feedback about what specifically you liked about the lesson and the rubric and what improvements you need to make. The conversation is saved, which allows you to return to the conversation when you are needing to modify the lesson on the fly, apply more differentiation, or. ask for more resources. Since you have one conversation that contains all the elements of your lesson, when you use the conversation to assist with student feedback it is trained with all the things you are looking for.
No Magic Replacement for Good Teachers
Before considering a tool that is marketed at teachers, consider what research they brag that they used to develop the tools.
ChatGPT allows you to create your own GPT for a custom teacher tool. This would ensure that your AI teacher tool aligns to your schools goals and pedagogy.
It is difficult to know everything and new research about teaching is always available. Unfortunately, for many things sufficient professional development and time to really learn them has been lacking. AI is truly a godsend for tired teachers who are expected to do it all without the time to do it.
4 thoughts on “Your AI Rubric Stinks”
One aspects of rubrics that has alway perplexed me is the 4 point scale and how teachers will happily convert a 3 into a 75% – knowingly or unknowingly understanding the math and the imact on a student’s overall grade. I absolutely agree with your argument that a 1 point gap between columns is not a fair way to evaluate student work. As I recall my undergrad work, from a VERY long time ago, rubrics were never intended to be converted into pecentage grades. Yet, as soon as I started my teaching career, it seems that is how they were always used. I have started using the “spread” where each column is a range of points, and I think that is better. That gives me a little leverage to show the student work on a continuum instead of an absolute score. I am wondering if the example you created with ChatGPT was with 3.5 or 4 and if you have tried developing your own GPT with the premium version. I have been toying around with them and I think what you have written here has sparked an idea that I might test out! Thanks for the well thought out and research backed post.
Agreed, not sure how rubrics became percentage makers… that is not how I was taught to use them either.
I’m using a Marzano-is 4.0 scale (which is really five points) but I’m working it like this:
I have a LOT of students who are behind grade level on nearly everything, plus 30% who are either ELL or have an IEP, or both.
I’m focusing on mastery and bringing up the low kids, and (although it makes me cringe, as a very bright student myself) leaving the bright kids to mainly figure out their own enrichment for now.
I’m in Marzano-ish environment, where everything is the four-point scale (i.e., five levels), so this is how I handle it.
3.0 is mastery, and it’s an A. Critera for mastery are explicit and at the chunk-level.
4.0 is essentially enrichment, going beyond mastery to find out more or do more stuff. 4.0 is “more than mastery” and it’s an A+.
I don’t even include 1.0 and 2.0 in rubrics or assignment criteria. If it’s not 3.0, then obviously something is missing. If some is missing, it’s 2.0. If a lot is missing, it’s 1.0. If they do nothing, it’s 0.0.
Nothing is 50%. The other grade bands are 10 percentage points, so an “F” is 50%. Thus, the percentages I use are:
100% – 4.0 (This is exact)
95% – 3.0 (This is exact)
75% – 2.0 (This can be higher, depending on how close they come)
65% – 1.0 (This can also go higher, as they approach 2.0
50% – 0.0 (This keeps total non-participators from getting credit for the course, and lets kids who are just going through a rough patch feel like getting their grade up is possible.)
The students only know the letter grades for 3.0 and 4.0. Last semester when I published all the levels and percentages, I had lots of students who were content with a C or content with just passing the course, which blows the whole point of mastery learning.
I tell them they are not getting points for doing a pile of papers, but last semester it ended up mostly being that. So now, I’m just not telling them. We focus on getting up to 3.0 mastery. The percentage grades are just to placate the district and give the parents an indication of where they’re at. Eventually, of course, they’ll figure it out, but I’m hoping that by then I will have mastered (ha!) keeping activities and pacing more engaging.