It's a feature, not a bug.
When AI enters the classroom and when we let it stay for the course
When I stood at the front of a classroom in 2007 to teach my first English Composition class, I heard one of my favorite lines from “The King and I” echoing in my mind: “It’s a very ancient saying, but a true and honest thought, that if you become a teacher, by your pupils you’ll be taught.” I vowed in that moment to always strike a balance between authority and authenticity, to eagerly share my knowledge and subject matter expertise with my students while acknowledging that many of the teachable moments in that room would come from them as they invited me into the vulnerable spaces where learning happens and showed me how to fill those gaps.
Fast-forward to 2022, when my pupil turned out not to be a person, but a chatbot. It’s been nearly four years since ChatGPT burst into our classrooms like Frankenstein’s creature on steroids, and every time I think I’ve cracked the code of AI’s DNA, it has another growth spurt. Yet it would be grossly unfair to say I’ve gained nothing in the struggle. True, I’ve become increasingly frustrated by the constant loop of revising assessments to prevent academic dishonesty from wiggling under the guardrails, but at the heart of effective pedagogy is innovation; we can’t continue to teach the same way we always did as our student populations, needs, and instructional tools evolve. Frustration notwithstanding, the moving goalposts of AI have challenged me to be a better educator because I’m consistently racing to meet my students where they are in an era when educational technology is evolving faster than we can find the on-switch.
I teach first-year composition and introductory literature courses, two subject areas that have suffered massive casualties in the crossfire of the AI revolution. Why, students ask, do I need to learn to write if ChatGPT can write for me? Why, students ask, do I need to learn how to conduct research if AI Mode in Google can find the answers? As strange as it sounds, I invite these questions because they serve a clear pedagogical purpose that feeds directly into writing and research instruction. Educators claim that the best antidote to cognitive offloading with AI, if not a panacea, is to teach students the value of learning, but stating this outright sounds too much like rosy-eyed optimism in an increasingly dark educational landscape where learning for learning’s sake has been undermined by skills-based training and workforce education. College and university programs equip graduates with skills and degrees that make them marketable in the workforce, certainly, but they also equip graduates with the critical thinking skills to be productive, responsible citizens and savvy consumers of media and information.
When I introduce AI into classroom conversations, I emphasize that responsible AI usage hangs on two hinges: knowing when to use it and knowing when not to use it. Equipping students with the skill to make that discernment naturally involves teaching them the value of developing subject matter expertise rather than giving in to the lure of taking the AI shortcut route. Current college students are preparing to enter a workforce in which many of them will be permitted, even encouraged, to use AI, which makes subject matter expertise even more crucial because without such knowledge, they lack the ability to determine where AI automation can make their workflow more efficient and where human oversight is necessary to ensure accuracy and quality. The potentially catastrophic errors resulting from failure to evaluate AI output range from networks shutting down when AI incorrectly flags a data breach to a life-threatening medical diagnosis when a human medical professional incorrectly interprets AI-generated results.
When students use AI to cheat, the main reason they get caught has little to do with AI detection, which educators like José Antonio Bowen and C. Edward Watson, in their book Teaching with AI: A Practical Guide to a New Era of Human Learning, have likened to academic smoke detectors. Case in point: when was the last time burned toast or popcorn in your apartment triggered your smoke detector? I’m asking in the interest of science. Sort of thing that could happen to anyone. The point is: AI detectors might be useful in alerting us to potential abnormalities in student work, but the real detection tool is our subject matter expertise. If a student who has never read Hamlet generates an AI analysis of Hamlet’s famous “To be or not to be” soliloquy, that student has no way of knowing whether AI hallucinated or misattributed quotations from the play because without having read the play, they lack the subject matter expertise to evaluate the accuracy of the AI’s output.
Unfortunately, cognitive offloading that produces inaccurate output is the most common AI flag I identify in student writing, detection tools or not. Yet simply assigning a failing grade without analyzing the suspected AI-generated content and discussing it with the student is a missed opportunity to create a teachable moment. Many of my colleagues and I have adopted the approach of using AI-detection reports not as proof of academic dishonesty, but as a conversation starter. Students whose writing, however rarely, falls into the false positive trap will typically respond correctly to questions about their work; students whose essays are suspected of containing AI-generated content (even if detection tools return a 0% score) will wriggle uncomfortably like a worm on a hook, unable to answer the simplest questions about their thesis or even the topic of their essay. When we explain this and work through the assignment’s weaknesses, we emphasize the instructional value of learning how to write rather than outsourcing the task of writing to AI.
Nowhere does cognitive offloading harm students more than in the writing classroom, where the mantra of process over product is repeated with almost religious enthusiasm. As writing instructors, we don’t teach our students how to produce polished essays. We teach them how to write essays, which involves the metacognitive work of thinking through purpose and process: brainstorming, outlining, drafting, and revising based on feedback all foster thoughtful rhetorical choices informed by a metacognitive approach to audience-centered writing. Questions about word choice, tone, and evaluation of evidence prompt students to consider the building blocks of their writing, which is why, if a student submits a polished draft without an outline in one of my courses, they lose points. An outline is the writing equivalent of the “show your work” directive on a math test. Don’t just give me the answer. Show me the steps you took to get there. Outlines also demand self-reflection by requiring students to slow down, asking questions like: Where is my argument? Are my body paragraphs flowing sequentially? Am I developing an essay that stays within the parameters of the topic?
Redesigning my assignments in response to AI-generated writing now involves an even greater emphasis on process and reflection, challenging students to embrace the messiness of drafting and encouraging them to let me sit with them in the mess. I now ask students to include a prompt reflection with their outlines identifying which prompt from the assignment guidelines they chose and how they think their chosen topic will aid them in practicing whichever rhetorical device we’re developing. This serves the purpose of detecting and assessing simultaneously.
Chatbots can populate an outline worksheet with generated text, but chatbots don’t know my students and generally produce bland, nondescript reflections. I’ve tried. Running my outline worksheets through multiple chatbots consistently yields mediocre results, or the chatbot omits this section of the outline entirely. Including the prompt reflection challenges students to consider how researching and writing about a specific topic develops the skills the assignment is intended to capture. Principally, the rubric individually scores each section of the outline: prompt reflection, introduction, body paragraphs, conclusion, and applicable research. As a result, students who attempt to bypass the outline and submit a completed draft lose credit because, as I emphasize, the outline is a tangible artifact of their engagement in the process instead of their focus on turning in a product.
Requiring students to provide a skeletal outline that breaks the essay into its component parts prompts them to imagine what the linear structure of their essays might look like. Crucially, the section directly following the introduction includes a space for a working thesis that directs students to isolate the sentence from their introduction that they believe to be their thesis. This prompts them to identify their working argument or main idea and demonstrate that they understand the components of their thesis that set it apart from the rest of their essay. Chatbots routinely fail this test, instead generating entirely fresh statements for the thesis. Typically, students who present a thesis in their outlines, even an underdeveloped one, will often earn higher scores than students who attempt to bypass the draft to underscore my objective: process matters more than product. As counterintuitive as it sounds, I reward messiness at this stage of the writing process because messy thinking is honest thinking.
Not being a software developer, I can’t definitively explain why chatbots fail my thesis test, but I suspect it’s because the task is deeply metacognitive in nature. As an academic experiment, I asked Claude, my current preferred chatbot of choice, to provide some insight on this behavior. As a writing instructor, I find engaging with AI myself in this way fascinating to “get under the hood,” as it were, of AI and learn how students are interacting with it so I can tailor my teaching accordingly. Evaluating the output also reveals why it’s so tempting for students to reach for AI because the response sounds uncannily human in its self-reflection.
According to Claude:
The short version is that language models are optimized for production, not retrieval or self-analysis. When you ask a chatbot to complete an outline with a working thesis, its default behavior is to generate — to produce a well-formed thesis-shaped sentence that fits the prompt. When you then ask it to go back and isolate the thesis from the introduction it just wrote, you’re asking it to perform a different cognitive operation entirely: read its own prior output as a text, identify a functional structure within that text, and extract rather than create. It will tend to generate a new thesis-shaped sentence anyway, because that’s the path of least resistance, and it often won’t notice (or flag) that it has done so. There’s no internal ‘wait, I should look at what I already wrote’ mechanism that reliably kicks in.
What I found equally fascinating and unsettling was the way Claude acknowledged the metacognitive value of my outline assignment while simultaneously attempting to explain why AI generally struggles to mirror human metacognition. Claude essentially produced a response in which it conducted an analysis of an analytical process beyond what it’s optimized to do. If this feels like a technological house of mirrors to you, you’re not alone. If you’ve spent time watching Claude’s thought process in real time by reading its “extended thinking” notes, it appears to engage in textbook metacognition—thinking about the process of thinking, analyzing, evaluating its response, and sometimes even self-correcting. Yet its response still maintains that the metacognitive work I’m asking of students cannot be adequately produced by AI.
This response is equally encouraging and unnerving because it blurs the boundaries between AI-resistant and AI-inclusive pedagogy. While I do design specific AI-resistant assignments to ensure I’m teaching students foundational skills, I don’t teach writing with an AI-resistant approach. On the contrary, I help students gradually build the skills to use AI in ways that foster critical thinking by exploiting the very process, emphasizing that prompt engineering and asking follow-up questions constitute an iterative process that neatly dovetails with the process of drafting and revising based on feedback and evaluating output. Claude’s response, then, is encouraging because an AI model that recognizes its own limitations is more likely to refuse to complete a task when it identifies a gap in its skill set. In fact, in my experimentation with chatbots over the last three years to determine how they might behave as a writing tutor, Claude was the only one that consistently would not immediately revise a piece of writing without prompting me to work through it myself or asking if I’d like further help. ChatGPT, Copilot, and Gemini consistently produce polished writing without prompting, only refusing to do so when in guided learning or study mode. Yet as encouraging as Claude’s response was, it was equally unnerving because it demonstrates precisely the kind of self-reflection that both software developers and educators insist AI is incapable of producing. Not to mention, this is also precisely the kind of self-reflection that students resist through cognitive offloading.
Yet my approach is not, as it might appear, weaponizing the process as a punitive measure to “catch” students cheating with AI. The key ingredient here is feedback. I don’t simply deduct points; I show students the gaps in their critical thinking, AI-generated or not, to show them why, ultimately, learning is active, not passive. For example, teaching students the pitfalls of AI-generated research essays entails teaching them what research actually is—not simply Googling search terms and searching for sources, but seeking answers to questions through discovery. Locating sources is only the first piece of the puzzle. The remaining and more critical piece involves evaluating sources and making thoughtful rhetorical choices around weighing and selecting logical evidence. I lean heavily on students using the quote sandwich to analyze and contextualize research, something that AI, despite advancements in web searching and deep research capabilities, still executes inexpertly. My feedback will include clear comments such as “This quotation is dropped and doesn’t clearly support your research,” or “You’re writing about TikTok’s impact on teen mental health, but you’re citing a study from 1995 before TikTok even existed.” (This one beat all the competition when it comes to glaring AI-generated content.) The successes, when they occur, are equally satisfying and impressive, like the health science student who grappled with hours of research to locate data on algorithmic bias in AI-generated analyses of skin cancer screenings or the student who pored over mental health data across the state of Florida to argue in favor of a bill that codifies mental health days as excused absences.
Will some students still slip past the guardrails? Undoubtedly. No judicial system is flawless. That said, the payoff I’m finding is that all my students, not merely the ones seeking shortcuts, are challenged to throw more muscle behind their work. Grades this semester have been lower than previously—hardly surprising given more nuanced assessment strategies and heavier weighting to emphasize the importance of academic integrity. What I have found equally surprising and refreshing is that more students than previously are coming to me not to challenge their grades, but to question, to understand how I evaluate their work, and to learn how they can apply that feedback moving forward. This is precisely the growth mindset approach to learning that I’ve struggled for nearly twenty years to instill in students, and I suspect, like many of my colleagues, it’s taken a chatbot-turned-cheating-tool to bring us to this epiphany.
On my first day of teaching, I vowed to learn from my students as much as to teach, to receive as much as to give. AI has entirely reframed that philosophy because even as I teach students to inhabit the gaps in their knowledge and admit what they don’t know, I’m admitting that I’m not always the expert in the room. The nearly twenty-year gulf of experience between my desk and theirs suddenly shrinks when we’re all trying to wrap our arms around this creature called AI. AI has challenged me to step up my game as a teacher because responsible teaching is about more than imparting subject knowledge. It’s about teaching students to use the knowledge and tools at their disposal with personal and professional integrity. In the end, AI is no different than every other tool that invaded our classrooms. Every anti-AI argument we’ve made has been previously launched against computers and search engines that are now as ubiquitous in our workflow as were the pencil and notebook before them. No longer the elusive shape-shifter that simultaneously breaks the mold while constructing a new one, AI is becoming a recognized part of the framework. As we acknowledge its existence and learn its functions, AI feels less and less like that spare part we find when unboxing a toy only to stare at it and wonder where it fits. As the tech gurus are fond of saying, “It’s a feature, not a bug.”