Imagine spending hours writing the perfect essay for the state test. You triple check your grammar, you make it relatable to your audience, you hit submit, and then wait for your grade. But here’s the thing, it isn’t a teacher reading your words. It isn’t even a human at all. It’s a machine.
That’s exactly what is happening here in Ohio and a group of 8th graders at West Clermont Middle School are trying to crack the code behind the machine. Simply put, they want to know how Artificial Intelligence evaluates student essays and how it affects their grades. 8th grader, Grayson Queen, explains how the project came about.
“Our teacher asked us. She showed us some essay responses that the OST had graded and she asked us what we thought they would have been graded and the results were kind of shocking,” explained Queen. “Essays that we thought were good, they weren't really that good to the OST AI and then she told us how previously she had kids try to figure out things that the AI liked and that was our main goal was to try to figure out what the AI likes and how we can get better scores on the OST test.”
Their investigation found that while AI catches grammatical mistakes and likes big words - it misses what really matters - creativity, originality, personality - the things that make us human.
“We think writing is an art not a math formula so we believe that it needs to be addressed and we need to add personality,” said Ajay Elswick an 8th grader at West Clermont Middle School. “The core issue is basically that the AI is very deprived of personality. It doesn’t want the writer to write with personality in it. Like when you address somebody in an essay using the word I, me, you, etc the AI deducts points for using the pronouns. And that simply takes away personality. It wants to use a formulaic approach to things with claim evidence reasoning specifically.”
Ajay Elswick gives a real-life example of how this could harm students in the future.
“So I'll give you a simple example. You're writing your college application. And people reading that, they're human. And normally you want to write a college application with a lot of personality to touch the hearts of the people reading it. And if you are trained to write for an AI for your whole elementary through high school career, then you're going to be writing like a robot to them. And they're going to think you're boring and you're not going to fit into the school. Or on job applications, same thing.”
So how did they come to this conclusion? They actually asked AI for help. They fed it essays, changed a few words, then ran them through again. And again. They even learned code and developed their own AI that they say is better.
“Well, for one, recently I kind of learned how to use and manipulate code because me and AJ and like everyone kind of helped with it. But we tried to replicate the Ohio State test AI by making our own website, which is still a process right now. But I definitely knew nothing about code before that,” said Rory Hinkle.
Now they are getting national recognition. They first presented their findings at the Ohio Council of Teachers of English Language Arts annual conference and then to the National Council of Teachers of English Assembly for Research. And most recently they met with Ohio Representative Adam Bird.
“These guys, they were awesome. They knocked it out of the park. I was so happy to hear about how hard they've been working,” said Rep. Bird. “It's a little distressing at the same time, because what they had to share today is something that I don't think a lot of people know about. And you know, we've got to make some changes when it comes to the testing and accountability system in Ohio. I've tried to be a champion for that, and I'm trying to get more people to back me on making changes to the testing system here in Ohio.”
Students say they are glad those who have the power to make change are listening.
“I think it went great. He really considered the problems that we gave him and he wants to basically have us tell other lawmakers that he works with about this problem that we've been facing with the AI. So he invited you to come to Columbus in the next couple of months,” said Elswick.
“Well, you know, I want to see them come to Columbus. And I'm going to try to make that happen. We want to see them come and share the kinds of things that they've been investigating, researching. I want to see them share that with some of my colleagues in Columbus,” said Rep. Bird.
So what exactly would these students like to see changed?
“Ideally in a perfect world, we would basically like a new AI to be developed or at least started on because I know that they're in the future already making tests that we probably won't even take right now but they need to develop a new AI in order to save the art of
“And honestly, I think that it should be taught in schools how to write for AI because it's just so different,” said Hinkle.
Their teacher agrees - she’s already begun making changes in her classroom.
“I don't have them just write essays. They might write a review rather than an argument. They might write a little chapter from a memoir instead of doing informative writing about somebody. So I focus on genre. What would be an actual thing that we would see in the real world? And I also teach essays as a genre. This is yet one more genre, which is what I've been doing, but now it confirmed my thinking and the practices that I've been doing for the past couple of years, because I don't want to turn out students to high school who only know how to write five-paragraph essays that make an AI happy. I don't think that's the goal of education,” said teacher Mary Hufford.
To watch the full story click here.