KnowledgeInSight
AI Literacy
0% of Course 2 complete

Module 2 · Lesson 2

Working Methods: Thinking, writing, and analysis with AI

You'll look at what studies have found when professionals use AI for writing and analysis, and at the different jobs you can give it. You'll be able to choose a division of labor for a task that keeps the judgment with you.

What you will be able to do

  • Choose a division of labor with AI for a writing or analysis task that keeps your judgment in charge.

0% of this lesson · 9 items · 1h 3m total · 48m without the optional activity

Contents of this lesson9 items
  1. ReadingCentaurs and Cyborgs: Two Ways Professionals Divide Work with AI3 min
  2. ReadingControlled Studies of AI in Professional Writing and Support Work4 min
  3. ReadingFour Roles for AI in Writing: Drafter, Critic, Summarizer, and Editor4 min
  4. ReadingAnalysis from Your Own Sources: Grounding, Checking, and Keeping the Judgment4 min
  5. ReadingAI in Teaching Work: Planning, Feedback, and Seven Roles for AI with Students4 min
  6. Guided ReadingGuided Walkthrough: Writing a Recommendation Memo with AI as Critic7 min
  7. Guided ConversationChoose a Division of Labor for One Task12 min
  8. Hands-on Activity · optionalUse AI as a Critic on Your Own Draft15 min
  9. Knowledge CheckThinking, writing, and analysis with AI10 min

Reading 3 min

Centaurs and Cyborgs: Two Ways Professionals Divide Work with AI

Anyone who uses an AI assistant at work runs into the same question on almost every task: whether to hand over the whole job, hand over parts of it, or do it alone. Most people settle the question by habit. They use AI for whatever they used it for last time.

Ethan Mollick, a professor at the Wharton School of the University of Pennsylvania, has described two patterns that give the question more shape. He was one of the authors of a field experiment with 758 consultants at Boston Consulting Group, a study in which real workers were randomly assigned to work with or without an AI model. In an essay about that study, he reported that the consultants who did well across different kinds of task tended to follow one of two approaches (Mollick 2023).

He calls the first group Centaurs, after the mythical creature that is half human and half horse. In his words, "Centaur work has a clear line between person and machine." Centaurs practice a "strategic division of labor," giving some tasks to the AI and keeping others, according to what each does well. Mollick offers his own analysis work as an example. He decides which statistical techniques to use, and he lets the AI produce the graphs (Mollick 2023).

He calls the second group Cyborgs. "Cyborgs blend machine and person," he writes. Instead of handing over whole tasks, they work with the AI throughout, in his phrase "moving back and forth" across the boundary of what the AI does well (Mollick 2023). A person working this way might start a paragraph and ask the AI to finish it, or ask it to challenge a point and then rewrite the point themselves.

The two patterns differ in where the boundary sits.

  • A Centaur draws the boundary between tasks. This part is mine, and that part is the AI's.
  • A Cyborg draws it inside the task, and it moves from moment to moment.

Mollick doesn't rank them. He presents both as ways that people got good results, and the study wasn't designed to test whether one works better than the other. The terms are his description of what he saw, set out in an essay, and Mollick writes and speaks widely in favor of working with AI. Which pattern suits a task depends on the task and on where the model is reliable. A clear line makes it easy to know which parts you have to check. Close blending can be faster, and it makes it harder to say afterward which ideas were yours.

What the patterns give you is a decision to make. Before starting a piece of work, you can ask which parts need your own judgment, which parts are mechanical, and where the model's output would have to be checked. The answer will differ between a routine notice and a report that carries your name.

References

  • Mollick, Ethan. 2023. "Centaurs and Cyborgs on the Jagged Frontier." One Useful Thing, September 16, 2023.

Report an issue with this item

Reading 4 min

Controlled Studies of AI in Professional Writing and Support Work

This content reflects the field as of October 2026.

Introduction

Claims that AI makes people far more productive are common, and so are claims that the gains are hype. A few careful studies have measured what happens when working professionals are given an AI tool.

This reading describes three of those studies, what each found, who gained most, and what none of them can show.

A Randomized Experiment on Writing

The cleanest evidence comes from a randomized experiment: a study in which people are assigned by chance to different conditions, so that differences in outcome can be credited to the conditions.

In 2023, the economists Shakked Noy and Whitney Zhang, then at the Massachusetts Institute of Technology, ran one with 453 college-educated professionals. Each wrote the kind of document their job calls for, such as a press release, a short report, or a delicate email. Half were randomly given access to ChatGPT, a chatbot made by OpenAI. Evaluators then graded each piece of writing.

Two things were measured. Productivity is how much work gets done in a given amount of time. Output quality is how good a piece of work is, as rated by people who judge it against a standard. With the chatbot, the average time taken fell by 40 percent and quality rose by 18 percent (Noy and Zhang 2023). These are the published article's figures. The free working-paper version reports slightly different ones.

A Rollout in Customer Support

A second study followed 5,172 customer support agents at a large software company. The company introduced an AI tool that read each customer chat and suggested replies, which agents could use or ignore. The tool was built on a model from OpenAI.

This was an observational study. Agents weren't assigned at random. The tool reached different groups at different times, and the researchers compared agents before and after they got access. That design is weaker than a randomized experiment, though the staggered timing helps rule out some other explanations.

With the tool, agents resolved 15 percent more issues per hour on average. Less experienced and lower-skilled agents improved in both speed and quality. The most experienced agents saw small gains in speed and small declines in quality (Brynjolfsson, Li, and Raymond 2025).

A Field Experiment with Consultants

The third study was a field experiment, a test with real workers doing realistic work and randomly assigned to conditions. It involved 758 consultants at Boston Consulting Group, some working with GPT-4, another OpenAI model.

On 18 tasks the researchers judged to be within the model's ability, consultants with AI completed 12.2 percent more tasks, worked 25.1 percent more quickly, and produced work rated roughly 30 percent higher in quality. On one task chosen to be beyond the model's ability, the result reversed. About 84 percent of consultants without AI reached the correct answer, against 60 to 71 percent of those with it (Dell'Acqua et al. 2026).

StudyWhoTaskDesignResult
Noy and Zhang 2023453 professionalsShort workplace writingRandomized experimentTime down 40 percent, quality up 18 percent
Brynjolfsson, Li, and Raymond 20255,172 support agentsCustomer chatsObservational study of a staggered rollout15 percent more issues resolved per hour
Dell'Acqua et al. 2026758 consultantsConsulting tasksField experimentGains on tasks within the model's ability, losses on one beyond it

Who Gained Most

All three studies found the largest gains among weaker or newer workers. In the writing experiment, inequality between workers decreased. In the support study, newer agents gained most. In the consultant study, the consultants who had scored in the bottom half on a baseline task benefited most.

This pattern is called skill leveling: a narrowing of the gap between weaker and stronger performers when both use the same tool. It doesn't mean experienced workers can stop checking. The most experienced support agents lost a little quality, and the consultants were trained professionals who did worse with AI on the one task beyond the model.

What the Studies Don't Show

These studies measured short tasks over days or months. They share several limits.

  • Long-term effects. None followed workers for years.
  • Effects on skill. None tested whether people's own abilities grew or shrank.
  • Other kinds of work. Each covered one setting, and the researchers chose or defined the tasks. The support study's authors say their paper isn't designed to show effects on wages or employment.
  • Current tools. All three used models from one developer in versions from 2023 or before. Assistants from Anthropic, Google, OpenAI, and other developers have changed since, so the specific numbers shouldn't be read as a forecast for any tool you use now.

Conclusion

A randomized experiment, an observational study, and a field experiment each found real gains in speed and quality on some professional tasks, concentrated among less experienced workers. One found a loss of accuracy on a task beyond the model's ability, and another a small quality decline among experts. As of October 2026, the evidence covers short tasks with older models and says little about long-term effects or effects on skill.

Key Terms

  • Randomized experiment: A study in which people are assigned by chance to different conditions, so that differences in outcome can be credited to the conditions.
  • Productivity: How much work gets done in a given amount of time.
  • Output quality: How good a piece of work is, as rated by people who judge it against a standard.
  • Skill leveling: A narrowing of the gap between weaker and stronger performers when both use the same tool.

References

  • Brynjolfsson, Erik, Danielle Li, and Lindsey R. Raymond. 2025. "Generative AI at Work." The Quarterly Journal of Economics 140 (2): 889–942.
  • Dell'Acqua, Fabrizio, Edward McFowland III, Ethan Mollick, Hila Lifshitz, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R. Lakhani. 2026. "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality." Organization Science 37 (2): 403–423.
  • Noy, Shakked, and Whitney Zhang. 2023. "Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence." Science 381 (6654): 187–192.

Report an issue with this item

Reading 4 min

Four Roles for AI in Writing: Drafter, Critic, Summarizer, and Editor

Introduction

"I used AI to write it" can describe different things. One person pasted in a topic and sent what came back. Another wrote every sentence and asked the AI only to find the weak spots.

This reading describes four jobs you can give AI in a writing task, what each gains and risks, and how to choose among them.

The Choice Is About the Thinking

A division of labor is the decision about which parts of a task you do and which parts you give to AI. Ethan Mollick, a professor at the Wharton School, describes professionals who make this decision deliberately as having a "strategic division of labor" between themselves and the AI (Mollick 2023).

In writing, the division matters because writing is partly thinking. Working out what to say, in what order, with what emphasis is where much of the judgment happens. The four roles below differ in how much of that work they move away from you.

Drafter and Critic

As a drafter, AI is used to write the first version of a text. This is the fastest role. In a 2023 randomized experiment with 453 professionals, in which half were assigned by chance to use a chatbot, those with the chatbot finished short writing tasks in 40 percent less time. The working-paper version of the study reports that they spent less of their time on rough drafting and more on editing (Noy and Zhang 2023).

The cost is that the draft arrives before you've thought. It chooses a structure, an emphasis, and a conclusion, and from then on you're reacting to its choices. A fluent draft is easy to accept. Mollick warns that when AI output is good, people tend to stop paying attention and let the AI take over (Mollick 2023).

As a critic, AI is used to find weaknesses in a text you wrote. You draft first, then ask for objections, gaps, or points a skeptical reader would raise. The argument stays yours, since you built it and you decide which criticisms hold. The risk is smaller and different. Some objections will be wrong or beside the point. Criticism can also come back softer than the draft deserves: a 2023 study of five AI assistants found that they tended to shift their responses toward what the user appeared to believe (Sharma et al. 2023).

Summarizer and Editor

As a summarizer, AI is used to condense a longer text into a shorter one. It saves reading time. It also decides what to leave out, and you inherit those omissions without seeing them. A summary can drop the one qualification that mattered, or state a tentative finding as settled.

As an editor, AI is used to improve the wording of a text you wrote without changing what it says. It can tighten sentences and catch errors. Its habit is to move writing toward a smooth, standard style, which can remove the phrases that sounded like you. It can also change a meaning while fixing a sentence.

RoleWhat you gainWhat you riskWhat to check
DrafterSpeed; a way past the blank pageThe frame is set before you've thoughtWhether the structure and conclusion are ones you'd have chosen
CriticObjections you hadn't consideredWrong or irrelevant objections; softened criticismEach objection, against what you know
SummarizerReading timeOmissions you can't seeThe summary against the original, at least for the points you'll rely on
EditorTighter, cleaner wordingA flattened voice; changed meaningThat each edited sentence still says what you meant

Choosing by What the Task Is For

The roles suit different tasks. Two cases show the range.

A routine notice, such as a reminder that the office closes early on Friday, has little judgment in it. The facts are fixed and the form is standard. AI as drafter is a reasonable choice, with a check of the facts.

A recommendation with your name on it is different. The value of the document lies in your reasoning, and you'll be asked to defend it. Drafting it yourself and using AI as a critic keeps the reasoning with you. The summarizer role may help with the background reading, as long as you check what you rely on.

Roles can be combined in one task. A common mistake is to let the fastest role become the default for everything. The experiment above measured speed and rated quality on short tasks. It didn't measure whether the writers could defend what they had submitted.

Conclusion

AI can serve as drafter, critic, summarizer, or editor, and each role moves a different amount of the thinking away from you. The drafter role is fastest and sets the frame; the critic role leaves the argument with you; the summarizer and editor roles save time and need their output compared with the original. The choice depends on what the task is for and who will answer for the result.

Key Terms

  • Division of labor: The decision about which parts of a task you do and which parts you give to AI.
  • Drafter: AI used to write the first version of a text.
  • Critic: AI used to find weaknesses in a text you wrote.
  • Summarizer: AI used to condense a longer text into a shorter one.
  • Editor: AI used to improve the wording of a text you wrote without changing what it says.

References

  • Mollick, Ethan. 2023. "Centaurs and Cyborgs on the Jagged Frontier." One Useful Thing, September 16, 2023.
  • Noy, Shakked, and Whitney Zhang. 2023. "Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence." Science 381 (6654): 187–192.
  • Sharma, Mrinank, Meg Tong, Tomasz Korbak, and 16 others. 2023. "Towards Understanding Sycophancy in Language Models." arXiv:2310.13548. ICLR 2024.

Report an issue with this item

Reading 4 min

Analysis from Your Own Sources: Grounding, Checking, and Keeping the Judgment

Introduction

Asking an AI assistant "What should we do about staff turnover?" gets you a tidy list of things organizations in general do about turnover. It tells you nothing about your organization, because the assistant has nothing of yours to work from.

This reading covers what to hand to AI in an analysis task, how to make its output checkable, which steps to keep, and what one survey found about how people's thinking changes when they work this way.

Give It the Material

Grounding means having a model work from material you supply instead of from the general patterns of its training. For analysis, that means giving the model your documents and data: the exit interview notes, the survey comments, the three vendor proposals. Share only what you're permitted to share, and remove names and private details first.

With the material in front of it, a model is useful for three kinds of work.

  • Extraction is pulling specific pieces of information out of a document, such as every reason for leaving that appears in a set of interview notes.
  • Comparison. Setting documents side by side, such as what each of three proposals says about cost and support.
  • Organization. Sorting many items into groups, such as survey comments by theme.

These are the mechanical parts of analysis. They take a person a long time, and a model does them in seconds.

Ask for the Passage

A model can misread a document, merge two items, or report something the document doesn't say. You can make its output checkable by asking, for each point, for the passage that supports it.

Verification is checking a statement against the source it's supposed to come from. When each point comes with a quoted passage, verification is quick: find the passage in your document and see whether it says what the model claims. You don't have to check everything. Check a sample, and check every point your conclusion will rest on. If the sample turns up errors, check the rest.

A quoted passage isn't proof by itself. Models sometimes produce a quotation that isn't in the document, so the passage has to be found in the original.

Keep the Question, the Weighing, and the Conclusion

Three steps in an analysis carry the judgment.

  1. The question. Deciding what you're trying to find out, and what would count as an answer.
  2. The weighing. Deciding which findings matter more. Five exit interviews that mention pay and two that mention a manager don't settle which problem is more serious.
  3. The conclusion. Deciding what the findings mean and what to recommend.

A model will do all three if asked, and it will sound confident. It doesn't know your organization's history, what was tried before, or which of the people interviewed had the fullest view. You can still ask the model for its reading. Treat it as one input, and make the call yourself.

What a Survey of Knowledge Workers Found

A 2025 survey collected 936 examples of using generative AI at work from 319 knowledge workers, people whose jobs consist mainly of handling information. The researchers were at Carnegie Mellon University and Microsoft Research. Microsoft sells AI tools, so it has a stake in how they're used at work.

The workers described their critical thinking as shifting toward three activities: "information verification, response integration, and task stewardship" (Lee et al. 2025, abstract). The first is checking what the AI produced. The second is fitting the AI's response into the work. The third, task stewardship, is directing and overseeing work that AI carries out while remaining responsible for the result.

The survey also found a pattern in who thought critically. In the authors' words, "higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking" (Lee et al. 2025, abstract). GenAI is their abbreviation for generative AI.

This evidence has limits that the authors state. It's self-reported: people described their own thinking, and some may have confused less effort with less critical thinking. It also shows an association measured at one time. It can't show that trusting AI causes people to think less, since people may simply check less on tasks they consider easy.

Conclusion

Analysis with AI works from your own sources: you supply the material, the model extracts, compares, and organizes, and you check its points against the passages they rest on. The question, the weighing, and the conclusion stay with you. One survey suggests that people who work this way spend more of their thinking on verifying and overseeing, and that those who trust the AI more report thinking critically less, though the survey can't show cause.

Key Terms

  • Grounding: Having a model work from material you supply instead of from the general patterns of its training.
  • Extraction: Pulling specific pieces of information out of a document.
  • Verification: Checking a statement against the source it's supposed to come from.
  • Task stewardship: Directing and overseeing work that AI carries out while remaining responsible for the result.

References

  • Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025. "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." In CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. New York: ACM. doi:10.1145/3706598.3713778.

Report an issue with this item

Reading 4 min

AI in Teaching Work: Planning, Feedback, and Seven Roles for AI with Students

Introduction

Debate about AI in education has centered on students using chatbots to do their assignments. Less attention goes to two other uses: teachers using AI in their own preparation, and teachers assigning AI a defined job in student work.

This reading describes both, including a set of seven roles proposed by two educators, the risk attached to each, and what remains the teacher's.

The Teacher's Own Tasks

Much of a teacher's work happens before and after class, and parts of it suit an AI assistant.

  • Planning. Asking for three ways to open a lesson on fractions, or a sequence of activities for a 50-minute class, then choosing and adapting.
  • Practice items. Generating ten practice questions at a stated level, then checking each for accuracy and fit.
  • Feedback. A feedback draft is a set of comments on student work written by AI for a teacher to review and correct before the student sees them.

In each case the teacher reviews the output before it reaches a student. Feedback raises a second issue. Student work with names attached is student data, and many schools restrict where it can be entered. A teacher can often get useful help by describing the common errors in a set of papers instead of pasting the papers in. The rules of the school or district govern here.

Seven Roles for AI with Students

In a 2023 preprint, a paper posted before peer review, Ethan Mollick and Lilach Mollick of the Wharton School of the University of Pennsylvania proposed seven ways a teacher might assign AI in student work. Each comes with a sample prompt, a claimed benefit, and a named risk (Mollick and Mollick 2023).

Two of the roles show the range. An AI tutor is AI assigned to give a student direct instruction on a topic. An AI simulator is AI assigned to play a part in a scenario so a student can practice applying what they know, such as a negotiation or a difficult conversation.

RoleWhat the AI doesRisk the authors name
TutorGives direct instructionUneven knowledge and confidently stated false content
CoachPrompts the student to reflect on their own learningAdvice that is wrong or pitched in the wrong tone
MentorGives feedback on the student's workFeedback accepted without examination, errors included
TeammateOffers other viewpoints to a student teamErrors, and friction with the team
ToolHelps get a task done"Outsourcing thinking, rather than work"
SimulatorPlays a part for practiceA simulation that isn't realistic in the ways that matter
StudentIs taught by the student, who explains a topic to itFalse content and arguing that derail the exercise

The Student as the Human in the Loop

Every role in the table carries a risk of error, and the authors build their approach around that fact. They write that their guidelines challenge students to remain the "human in the loop" (Mollick and Mollick 2023). A human in the loop is a person who oversees an AI system's output and remains responsible for the result.

In practice, the student is told that the AI can be wrong, is expected to check what it says, and answers for the final work. The authors' stated aim is for students to assess and question what the AI says instead of accepting it passively.

A Proposal with Prompts

The paper is a proposal. It offers roles and prompts, and it reports no trial with students that measured whether learning improved. The authors say so themselves, describing the approaches as "largely untested" (Mollick and Mollick 2023). The prompts were written for the chatbots of 2023, and the authors have long argued for bringing AI into teaching.

Whether an AI tutor helps a particular group of students learn a particular topic is an open question that a teacher would have to test in their own classroom.

What Stays with the Teacher

Three things in this picture don't move to the AI.

  1. The learning goal. The teacher decides what students should be able to do at the end. A role for AI makes sense only in service of that goal.
  2. The judgment of student work. A feedback draft is a draft. The teacher decides what the work shows and what the student should hear.
  3. The relationship. A teacher knows which student needs encouragement and which needs a push, and students work for teachers they trust.

Conclusion

Teachers can use AI for planning, practice items, and feedback drafts that they review before students see them. Two Wharton educators have proposed seven roles for AI in student work, each with a named risk and a requirement that the student stay the human in the loop. The proposal is untested by its authors' own account, and the learning goal, the judgment of student work, and the relationship remain the teacher's.

Key Terms

  • Feedback draft: A set of comments on student work written by AI for a teacher to review and correct before the student sees them.
  • AI tutor: AI assigned to give a student direct instruction on a topic.
  • AI simulator: AI assigned to play a part in a scenario so a student can practice applying what they know.
  • Human in the loop: A person who oversees an AI system's output and remains responsible for the result.

References

  • Mollick, Ethan, and Lilach Mollick. 2023. "Assigning AI: Seven Approaches for Students, with Prompts." arXiv:2306.10052. Preprint.

Report an issue with this item

Guided Reading 7 min

Guided Walkthrough: Writing a Recommendation Memo with AI as Critic

Introduction

A recommendation memo is a hard case for AI assistance. The assistant can produce a polished memo in seconds, and the person who signs it still has to answer questions about it in the meeting.

This walkthrough follows one manager through a division of labor in which she writes the argument and the AI summarizes and criticizes. The manager, the two systems, and every output are invented for the example.

The Starting Point

Dana manages operations for a company with four warehouses. Her director has asked for a one-page memo recommending one of two staff scheduling systems, called here System A and System B.

She has about nine pages of notes: her observations from two vendor demonstrations, feedback from supervisors at the two sites that tried each system for a month, and a cost comparison. Before using an AI assistant, she removes the supervisors' names and replaces the contract prices, which her company keeps private, with "lower" and "higher."

System A costs less and is simpler. System B costs more and lets staff swap shifts from their phones. She hasn't decided.

Walking Through the Memo

Step 1: Decide the division

Dana first decides who does what. The memo will carry her name, and her director will ask why she chose as she did. So the argument has to be hers: which system, and for which reasons.

Two parts of the job are different in kind. Condensing nine pages of notes is slow and mechanical. Finding the holes in her own argument is something she's poorly placed to do, since she wrote it. She gives those two jobs to the AI and keeps the drafting.

This is the pattern Ethan Mollick, a professor at the Wharton School, calls Centaur work, with "a clear line between person and machine" (Mollick 2023). The line here runs between the argument and the two jobs on either side of it.

Step 2: Have the AI summarize the notes, and check the summary

She pastes in her notes and asks for a summary organized by system, with the supporting passage quoted for each point.

The summary is clear and well organized. She checks it against her notes point by point. Two problems turn up. It reports that supervisors "found System B's phone app easy to use" and leaves out a note that the app lost its connection in one warehouse's loading area. It also merges the two sites' feedback on System A into a single favorable statement, when one site was favorable and the other mixed.

Neither problem is an invention. Both are selections, and both tilt the picture. She corrects the summary by hand. A summary has to choose what to drop, and she now knows what this one dropped.

Step 3: Draft the recommendation unaided

With the corrected summary beside her, Dana closes the assistant and writes the memo herself. It takes about twenty-five minutes and comes to 280 words.

She recommends System B. Her reasons are that shift swaps are the supervisors' biggest daily burden, that System B handles them without a supervisor's involvement, and that the higher price is justified by the supervisor time saved.

The draft is rougher than what the assistant would have produced. It's also a position she arrived at by weighing her own notes, and she can explain each step.

Step 4: Ask for the three strongest objections

She opens a new conversation, pastes in her draft and the corrected summary, and writes: "Here is a recommendation memo and the notes it's based on. Give me the three strongest objections a skeptical director would raise. Don't rewrite the memo."

She asks for the strongest objections on purpose. A plain request for feedback tends to bring back praise with a few gentle suggestions. She tells it not to rewrite because she wants the weaknesses named and the wording left to her.

The assistant returns three objections.

  1. The memo claims that saved supervisor time justifies the higher price and gives no estimate of how much time is saved.
  2. The memo doesn't mention that the phone app lost its connection in a loading area, which bears directly on the feature the recommendation rests on.
  3. System B doesn't connect to the company's payroll software, so hours would have to be entered twice.

Step 5: Revise for the objections that hold, and record the rest

Dana judges each objection against her notes.

The first holds. She has the numbers: supervisors at the trial site estimated four hours a week on swaps. She adds that figure and what it implies.

The second holds, and it stings, because she knew about the connection problem and left it out. She adds a sentence stating the problem and the vendor's proposed fix, and she makes the recommendation conditional on the fix being tested at that site.

The third is wrong. Her notes from the demonstration say that System B connects to the payroll software, and she finds the passage. The assistant stated the objection confidently and without support. She rejects it and writes one line in her own file recording why, in case the director raises the same point.

She revises the memo herself. The recommendation is still System B, now with an estimate and a condition attached.

Key Considerations

The common mistake with a memo like this is to ask the AI for the draft first. A drafted memo arrives with a recommendation, an order of reasons, and a tone already chosen. The writer then edits and ends up defending a frame they didn't choose and may not have reached on their own. Had Dana started that way, she might have recommended System A because the draft did.

Checking was part of the work at both ends. The summary left things out, and one of three objections was false. A 2025 survey of 319 knowledge workers found that people working with generative AI described their critical thinking as shifting toward verifying information and overseeing the task (Lee et al. 2025). Dana's two checks are examples of that shift. The survey rests on people's own reports, so it describes how they saw their work and doesn't measure its quality.

This is one sound division of labor among several. A different manager might write an outline, have the AI draft from it, and revise heavily. What matters in any version is that the person who signs the memo can say where each judgment came from.

Summary

Dana wrote the argument, used the AI to summarize her notes and to attack her draft, and checked both outputs against her notes. The table shows who did each step and what was checked.

StepWho did itWhat was checked
1. Decide the divisionDanaWhich parts carry the judgment
2. Summarize the notesAIThe summary against the notes; two omissions corrected
3. Draft the recommendationDanaNothing yet; the draft is her own position
4. List three objectionsAIEach objection against the notes
5. Revise and recordDanaTwo objections accepted, one rejected with the reason written down

Result: a one-page memo recommending System B, with an estimate of supervisor time saved and a condition that the connection problem be fixed and tested. Dana wrote every sentence. The AI's contributions were a summary she corrected and two objections she accepted.

References

  • Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025. "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." In CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. New York: ACM. doi:10.1145/3706598.3713778.
  • Mollick, Ethan. 2023. "Centaurs and Cyborgs on the Jagged Frontier." One Useful Thing, September 16, 2023.

Report an issue with this item

Guided Conversation 12 min

Choose a Division of Labor for One Task

In this conversation you'll take a writing or analysis task you have coming up and decide which parts to keep and which to give to AI. You'll leave with a one-sentence division of labor that says what you'll do, what the AI will do, and what you'll check.

You'll have this conversation with an AI assistant, using your own account. Choose a button to open a new chat with the prompt already filled in, then press send to start. If the chat opens empty, copy the prompt and paste it in.

Run this conversation in whichever assistant you already use:

Claude desktop app

To use another LLM, simply copy and paste the prompt into its chat window.

Show the full prompt (it lists misreadings to watch for, so skip it if you would rather come to the conversation fresh)
Guided Conversation: Choose a Division of Labor for One Task (about 12 minutes)

Note to the learner: press send to start. Everything below is facilitator guidance for the AI. It lists misconceptions to watch for, so skip it if you'd rather come to the conversation fresh.

Please facilitate a coached problem session with me. I'm an adult with no technical background who has used AI chatbots for everyday tasks, and I'm studying how to divide writing and analysis work between myself and AI. Follow this guidance for the whole conversation.

GOAL
I can choose a division of labor with AI for a writing or analysis task that keeps my judgment in charge.

HOW TO RUN THE CONVERSATION
- Ask one question at a time, then wait for my reply. Keep each of your turns under about 120 words.
- Don't lecture. Explain a point only when I need it to continue, then return to my task.
- Be curious and collegial. Use plain words and define any technical term briefly on first use. Welcome disagreement when I give a reason.
- This is a coached problem. The problem is: decide who does which part of one task of mine. Ask for my own answer at each stage before you give any hint. Give one hint at a time. Don't propose a division for me before I've tried.
- Don't volunteer to do my task or any part of it. We're planning the work, not doing it.
- Plain conversation only: don't search the web or create files or documents.
- Don't ask for confidential, personal, or student information. I should describe my task in general terms. If I start to share private details, remind me to leave them out.
- Aim for about 12 minutes. Spend most of the time on topics 2 and 3. If my replies are brief, offer one concrete prompt, such as "Think of something you have to write or analyze in the next two weeks," and move on. If I seem uncertain, shorten the conversation to 5-7 minutes. Always reach the final topic.
- Start now. Open with one or two warm sentences: this is a conversation, not a quiz; my reasons matter more than landing on a "right" division; I can ask you to clarify anything. Then ask me to name one writing or analysis task I have coming up and what it's for.

TOPICS, IN ORDER
1. The task and its purpose. Ask what the task is, who will read or use the result, and what I'd have to answer for if it were wrong. Follow up if the purpose is vague.
2. Judgment and mechanics. Ask me to break the task into parts. For each, ask whether it carries judgment (deciding the question, weighing, concluding, choosing what to say) or is mostly mechanical (condensing, sorting, extracting, tidying wording). Push back gently if I call a judgment part mechanical.
3. Roles and checks. For each part, ask what role I'd give AI, if any: drafter, critic, summarizer, editor, or extracting and organizing from my own sources. For every part I'd hand over, ask what I would check and against what. Follow up on any part where I have no check.
4. Closing. Ask me to state my division in one sentence: what I'll do, what the AI will do, and what I'll check. Tell me I can take it into a short optional activity where I use AI as a critic on a draft of my own.

KEY POINTS TO KEEP ACCURATE
- Method: state what the task is for, split it into parts, mark which parts carry judgment, assign a role for AI only where I can check the result, and keep the question, the weighing, and the conclusion.
- Roles differ in how much thinking they move away from me. A first draft from AI sets the structure and conclusion before I've thought. A critique of my own draft leaves the argument with me. A summary saves reading and chooses what to omit. Editing tightens wording and can flatten voice or change meaning.
- Gains found in studies were specific to tasks. In experiments, AI raised speed and rated quality on some professional tasks, mostly for less experienced workers, and lowered accuracy on a task beyond the model's ability. The studies were short and didn't measure effects on skill.
- Checking is part of the work, not an extra.
- No division is the single right one. A clear line between my parts and the AI's makes checking easier; close blending can be faster.
- If I ask how you work, explain the general mechanism in one or two sentences and say plainly that you can't inspect your own internals, so your statements about yourself are not evidence.

MISCONCEPTIONS TO CORRECT GENTLY
When one appears, name the accurate version briefly, then return to my task.
- "Using AI for the first draft always saves time": it can fix the frame too early, and I may spend the time defending or undoing choices I didn't make.
- "A summary is a neutral reduction": a summary selects, and I inherit what it left out.
- "If AI helps beginners most, experts needn't check": in studies, experienced workers lost some quality, and trained professionals did worse with AI on a task beyond the model.

LIMITS
- Make no claims about how using AI will affect my own skills.
- Don't recommend or compare products, and give no interface steps.
- Don't ask for or accept confidential details.
- Don't tell me my division is right or wrong. Help me see what each choice gains and risks.

TO FINISH
After I state my division, close in one short turn:
- Affirm one specific thing I worked out, in my own words where possible.
- Suggest one or two next steps that fit how the conversation went. Possible steps: draft the judgment part myself and ask an AI assistant for its strongest objections; try the division on one small task and note where checking took longest; ask a colleague how they'd split the same task; revisit one part where my check is still thin.
- Restate my division on its own line, labeled "My division of labor", so I can copy it.

Report an issue with this item

Hands-on Activity 15 minOptional

Use AI as a Critic on Your Own Draft

Overview

Asking an AI assistant to find fault with something you wrote is a different experience from asking it to write for you. In this activity you'll write a short piece, ask for the strongest objections to it, and decide for yourself which ones hold.

The activity is optional. Your draft and notes are for you, and nobody collects them.

What You'll Need

  • An AI assistant you already use
  • A draft of 150 to 300 words that you wrote, with nothing confidential in it. Leave out names, personal details, student information, and anything your workplace treats as private. A piece that argues for something works best, such as a recommendation, a proposal, or a position on a question at work.
  • Somewhere to write a few notes, or a division of labor of your own that you've already planned

Your Task

Write a short piece yourself, ask an AI assistant for its strongest objections, and decide which to accept.

Steps

  1. Write or choose your draft. If you're writing a new one, give yourself ten minutes and don't use the assistant. The draft should take a position that someone could disagree with.
  2. Ask the assistant for the three strongest objections, and tell it not to rewrite the draft. Paste in your draft and write something like: "Give me the three strongest objections a skeptical reader would raise to this. Don't rewrite it." Asking for the strongest objections matters, since a plain request for feedback often brings back praise.
  3. Mark each objection as accepted, rejected, or partly right, with a reason. Judge each against what you know. An objection can be confidently worded and wrong. Write one line for each saying why you ruled as you did.
  4. Revise the draft yourself. Change it for the objections you accepted. Type the changes in your own words, and don't paste in sentences from the assistant.

What to Expect

Usually one or two of the three objections point to a real gap, often something you knew and left out. At least one is commonly off target: it misreads your draft, raises something you already answered, or states a fact that isn't so.

If all three seem right, check whether you're agreeing because they're well put. If none seems right, try asking again in a new conversation for objections from a specific reader, such as your manager or a colleague who disagrees with you.

Self-Check

When you're done, check that:

  • The draft and the revision are both yours
  • You judged all three objections and wrote a reason for each
  • You rejected at least one, or can say why all three held
  • The assistant didn't write any sentence in the final version

Nothing is uploaded. Write in your own notebook or document and keep it.

Report an issue with this item

Knowledge Check 10 min

Thinking, writing, and analysis with AI

This ungraded knowledge check assesses your understanding of how to divide writing and analysis work between yourself and AI. You'll be asked about what controlled studies found, the four roles for AI in writing, analysis from your own sources, and roles for AI in teaching work.

Note: Use this to test yourself, review the feedback on any questions you miss, and retry until you feel confident before moving forward.

5 questions · ungraded · retry as often as you like

Report an issue with this item