Some people think poverty is what results when employers exploit workers on the bottom of the labor market. Other people think poverty is what results when individuals behave in dysfunctional ways. These two dominant theories then generate a variety of policy ideas, including increasing educational attainment and work incentives to improve labor market outcomes, changing employment rules so as to increase pay for lower-level workers, and instructing people to change the way they engage in family formation, among other things.

When I first got interested in practical poverty policy nearly two decades ago, I went into it with these two theories in mind, assuming that what I was likely to find was that employer exploitation was the better theory of the two. What I found instead was that both theories are way off the mark and that poverty in developed countries is actually much simpler and much dumber than all of that. It’s not bad behavior by employers or degenerates. It’s just what happens when the following conditions are present:

1. The national income is distributed using payments to laborers and capital owners.
2. Capital ownership is very unevenly distributed across families.
3. A large share of the population is not working at any given time.
4. Nonworkers are unevenly distributed across families.

People really seem to want poverty to be something much bigger than all of that. Its existence in a rich society is so troubling that it seems to call out for a heavier explanation. But it really is just a simple math problem that has a very straightforward solution.

The Problem of Nonworkers in Capitalism

The way a simplified version of a capitalist economy works is that land, capital, and labor are supplied as inputs to production, the owners of those inputs receive payments, and then those payments can be used to buy the outputs of production. Because these payments — variously called factor payments, factor income, or market income — only go out to laborers and capital owners, anyone who is neither of those things will not receive direct, personal income in this system.

In our society, the ownership of capital is very highly concentrated, meaning that very few people own significant amounts of capital and therefore very few people receive significant capital income.

You could imagine creating a society where capital ownership is not so highly concentrated. This could be achieved fairly simply by creating a social wealth fund where much of the nation’s capital stock is held collectively and each person is entitled to an equal share of the investment return. Alaska has a modest fund like this that significantly reduces alternatively-measured wealth inequality in the state while paying out thousands of dollars each year to every resident. These universal dividend payments reduce the number of impoverished Alaskans by 20 to 40 percent every year.

Norway has a huge fund like this that radically reduces alternatively-measured wealth inequality in the country, but it uses the return as a source of general government revenue, not to fund individual dividends.

Whatever the merits of these alternative approaches to managing capital ownership, they are not relevant to our analysis of poverty in America because the country does not use them. Overall, capital ownership in the US is very skewed towards the top of the society and therefore has little impact on keeping poverty low.

With capital income out of the equation, all that is left is labor income: wages, salaries, self-employment income, and similar. Labor income is more evenly distributed than capital income, but its ability to keep poverty low is hampered by the fact that, at any given time, around half of the population is not working and therefore does not receive any personal labor income.

As with capital ownership, I suppose you could imagine a society where nonworkers were somehow evenly distributed across families such that every family had an equal number of workers and nonworkers. Unlike with capital ownership, it is not very easy to come up with a mechanism that could achieve such a thing and it seems undesirable in other respects, but it is at least possible in the abstract. Because poverty is measured on the family level, a society so composed would be one where each nonworker would be effectively assigned income from a cohabitating worker, thereby keeping poverty low.

But in reality, nonworkers are very unevenly distributed across households. Forty percent of people live in families where more than half of the members work. Another 40 percent live in families where less than half of the members work. Sixteen percent of people live in families where none of the members work. Only 20 percent of people live in families that have the 1-to-1 ratio of workers to nonworkers. The graph below shows what this all looks like across the entire distribution.

This point about the unequal distribution of nonworkers across families can be illustrated in a number of other ways as well. The below graph simply counts the raw number of nonworkers in each person’s family and shows that across the distribution. At the most extreme end, there is a family with 12 nonworkers in it. Around 22 percent of individuals live in families with three or more nonworkers in them while 25 percent live in families with no nonworkers in them.

We can even produce this same graph using “net workers,” which I define as the number of workers minus the number of nonworkers in each family. At the extremes, there is a family with 12 more nonworkers than workers and a family with 8 more workers than nonworkers. Predictably, at the median, it is a wash: there are as many workers as nonworkers, generating an outcome of zero net workers.

One last way of illustrating this is by adding up the number of hours worked by each family and dividing it by the number of family members. As can be seen in the graph below, about 16 percent of people live in families with no hours worked. The first plateau between the 50th and 60th percentiles are families with 1,040 hours worked per family member, which is equivalent to one full-time worker for every nonworker. The second plateau between the 80th and 90th percentiles are families with 2,080 hours worked per family member, which are families exclusively composed of full-time workers with no nonworkers.

The problem created by this unequal distribution of nonworkers across families is that, at the bottom, you wind up with too few workers attempting to support too many nonworkers. The labor income that makes it into these families simply gets stretched too thin. We can see this in the next two graphs, which show the market poverty rates of families with different mixes of workers and nonworkers and different amounts of hours worked per family member. The phrase “market poverty” refers to individuals who are in poverty if we exclude government benefits. In these graphs, each bar represents about 20 percent of the US population.

The bottom 20 percent of individuals so defined make up a little over two-thirds of the market poor. The bottom 40 percent make up over 90 percent of the market poor. Notably I was able to deduce all of this without ever peaking to see what any given individual’s wage rate was. That’s because it is the uneven distribution of nonwork across families that is driving all of this.

Who Are the Nonworkers?

The presentation above lends itself to the natural conclusion that we could whip poverty by simply converting a bunch of these nonworkers into workers. But when we look at who the nonworkers actually are, we quickly see that this is not really possible or desirable.

As noted above, around 48 percent of the population does not work in a given year. In the graph below, I sort all of these nonworkers into one of eight categories. Individuals who fall into more than one category are assigned to the first category they qualify for in order from the top of the graph to the bottom of the graph.

Together, children and elderly people make up nearly three-fourths of all nonworkers. Adding the disabled and students gets you to 86 percent of all nonworkers. There are things you could do to activate some of these nonworkers into the labor market, such as providing child care benefits to reduce the number of at-home caregivers, if that is the kind of thing that excites you. But, for the most part, unless you want to relax child labor laws or take away retirement security, this is just what society looks like.

Not surprisingly, market poverty plagues the nonworkers. In the below graph, we see the market poverty rate for these eight categories. At the bottom I’ve added a bar for individuals who worked 50 or more weeks during the year for comparison.

The Obvious Solution

When I started this piece, I claimed that poverty occurs when the following four conditions are present:

1. The national income is distributed using payments to laborers and capital owners.
2. Capital ownership is very unevenly distributed across families.
3. A large share of the population is not working at any given time.
4. Nonworkers are unevenly distributed across families.

One could do more, but I think I have demonstrated this all pretty well using the most recent Census income microdata. If this is a correct diagnosis of the problem, then the solution involves flipping one or more of these four conditions.

Some social conservatives implicitly argue for flipping the last condition by somehow (they never quite explain the mechanism) increasing parental cohabitation or perhaps even increasing the number of multi-generational households. If that’s a lifestyle you like and you want to convince others to adopt it, that’s fine. But it doesn’t seem likely to move the needle much and I do think it is wrong for a society to economically coerce individuals to live together by threatening poverty if they don’t.

As noted already, there are some things you could do to reduce the number of nonworkers a bit, but absent pretty repugnant reversals in our views about child labor and retirement security, the third condition is not likely to change significantly.

Flipping the second condition by redistributing capital ownership is actually a reasonably fruitful path to poverty reduction. In Alaska, it reduces poverty by 20 to 40 percent, and they could certainly go further with it than they have. I like this idea for lots of reasons, though I don’t think it would be adequate by itself.

So what we are left with is flipping the first condition and using mechanisms other than payments to laborers and capital owners to distribute the national income. This is called the “welfare state” and it is, as a factual matter, how low-poverty countries come to be that way.

Indeed, if you look at the categories of nonworkers in the graphs above, you might notice that they map perfectly onto the populations that welfare states are designed to serve. In good welfare states:

1. Children receive a monthly child benefit check, child care, pre-k, K-12 education, among other things.
2. Elderly receive an old-age pension.
3. Disabled receive disability benefits.
4. Students receive tuition subsidies, living stipends, and subsidized loans.
5. Carers receive paid leave and home care allowances.
6. Unemployed receive unemployment benefits.

Even the US has some of these benefits and they work in proportion to their coverage and generosity. We can see this in the below graph where I introduce a bar for disposable income poverty, which counts government benefits.

Pushing the black bars lower is just a matter of introducing or increasing benefits for each category. I have written extensively about how exactly to do that (Family Fun Pack, Cleaning Up the Welfare State).

One of the most interesting things about poverty reduction is that you actually don’t need “anti-poverty” policy to achieve it. In fact, the extent to which a society has low poverty seems to be almost inversely related to how much of their welfare state is specifically focused on “the poor” through means-tested benefits. All you really need to do to achieve low poverty is provide benefits universally to all nonworkers without regard to their family income. This approach smooths out inequalities up and down the income ladder by placing the burden of providing for nonworkers on the entire society instead of dumping it unequally on each particular family.

I wish I could say I was the first to come up with this, but I am actually one of the last. In the early 20th century, this was a much more common way of understanding the welfare state. Indeed, everything I’ve written so far was neatly summarized in this 1940s graphic from Switzerland, which we have remade in English below.

It Can Happen to You

To close this out, I think it is important to emphasize that “the poor” are not a static population. It is easy to imagine, as Matt Yglesias does in his recent piece, that poverty afflicts a particular group of people and that those people are different from everyone else:

By contrast, the domestic poor are — unless they are recently arrived immigrants — often people who, for one reason or another, are struggling to get their lives together in a very wealthy country. If they were thrifty and diligent, they wouldn’t be poor in the first place. Putting money in their pockets doesn’t make them thrifty and diligent, so it doesn’t really alter their lives that much.

One way we can see whether this is true is by looking at the extent to which poverty is persistent. The poverty data comes from the Current Population Survey, which has a longitudinal component to it that allows you to track one-fourth of the sample across two consecutive years. By following these people, we can see what percent of people who are poor one year are also poor in the prior or subsequent year.

Using this method, we see that only 41 percent of the 2018 poor were also poor in 2017 under the disposable income poverty metric. Likewise, only 39 percent of the 2018 poor went on to become the 2019 poor. Poverty rates don’t move much year to year, but there is massive churn in and out of the category.

Another interesting thing we can do is look at where the newly poor came from and where the formerly poor wind up. In this next graph, I took everyone in the sample who was poor in 2018 but not poor in 2017 (so newly poor) and then looked to see what income quintile they came from (income quintile here is defined according to income as a percent of the SPM poverty line).

More are coming from the bottom than the top, but still 37 percent of the newly poor came from the middle quintile or above. And this is with just tracking people for a single year.

We can do the same kind of thing for people who were poor in 2018 but not in 2019 and see where they wound up. It shows the same thing.

Where Yglesias goes wrong is quickly handwaving over the idea of “getting their lives together in a wealthy country.” A wealthy country by definition produces a lot of income. But how it chooses to distribute that income — most crucially the degree to which it relies on payments to labor and capital — will decide whether poverty is low. Poverty hits people as they move in and out of different life stages and events. Job loss, disability, divorce, having children, family deaths are things that can and do happen to anyone. And when they do, if the welfare state is not there for them, they will often dip into poverty, at least for a bit.

The other mistake in Yglesias-style thinking is mixing up thrift and diligence with economic success. As with anything else, these traits are distributed throughout all populations, including poor and non-poor populations, which are, in fact, mostly the same people in different years. There are plenty of people who currently have high incomes who nonetheless are not thrifty, not particularly diligent, and have any number of other problems, including addictions to vices, mental health problems, domestic violence, and so on, just as there are plenty of people who currently have low incomes who have none of these problems. It’s just that when we find it in a poor person, we blame their income situation on it, not because one actually follows from the other, but because it makes us feel better about things.

Conclusion

I’ve been doing this long enough to know that, with the exception of certain autistic people, nobody finds the realities of poverty in America as described above very exciting. They really want it to be something more than that, something that you can really sink your teeth into about the fallenness of man or the viciousness of big corporations or whatever. But it really is basically a technical problem in the way that factor payments are misaligned with the distribution of people across households that you can pass a few laws to fix. It is unclear how exactly to get people in the US to go ahead and do that, but it is not unclear what needs to be done.

This minute-long clip of a Will Smith concert is blowing up online for all the wrong reasons, with people accusing him of using AI to generate fake crowds filled with fake fans carrying fake signs. The story’s blown up a bit, with coverage in Rolling Stone, NME, The Independent, and Consequence of Sound.

And it definitely looks terrible! The faces have all the characteristics of AI slop, with familiar artifacts like uncanny features, smeared faces, multiple fingers/limbs, and nonsensical signage. “From West Philly to West Swig̴̙̕g̷̤̔͜y”?

But here’s where things get complicated.

The crowds are real. Every person you see in the video above started out as real footage of real fans, pulled from video of multiple Will Smith concerts during his recent European tour.

Real Crowds, Real Signs

The main Will Smith performance footage in the clip is from the Positiv Festival, held last month at the Théâtre Antique d’Orange in Orange, France. (Here’s a phone recording from the audience of the first half of the performance.) It’s intercut with various shots of audiences from Gurtenfestival and Paléo in Switzerland, the Ronquieres Festival in Belgium, among others.

From this slideshow of photos from the Paléo festival in Nyon, Switzerland, you can see professionally-shot photos of the same audience from the video.

The signs, previously distorted, can now be read, like this one which actually reads “From West Philly to West Swizzy.” (Short for Switzerland, if you’re wondering.)

One of the most egregious examples is the couple holding the sign thanking Will Smith for helping them survive cancer which — if it was AI-generated slop — would be pretty disgusting: a gross attempt to drum up sympathy with fake people.

In an article posted by The Independent today, music editor Roisin O’Connor points to the couple as clear evidence of AI generation:

“Another shot shows a man’s knuckle appear to blur along with his sign, which reads ‘You Can Make It’ helped me survive cancer. THX Will.’ Meanwhile, the woman in front of him is seemingly holding his hand, but the headband of the woman behind her is somehow over her wrist.”

But the couple is real. There’s two good photos of them on Will Smith’s Instagram in a slideshow of photos and videos from Gurtenfestival in Bern last month.

You can see them in this video from Will Smith’s Instagram post, which I clipped below.

Two Levels of AI Enhancement

So if these fans aren’t AI-generated fakes, what’s going on here?

The video features real performances and real audiences, but I believe they were manipulated on two levels:

1. Will Smith’s team generated several short AI image-to-video clips from professionally-shot audience photos
2. YouTube post-processed the resulting Shorts montage, making everything look so much worse

Let’s start with YouTube.

YouTube’s Shorts “Experiment”

Will Smith’s team also uploaded this same video to Instagram and Facebook, where it looks considerably better than the copy on YouTube, without the smeary sheen of uncanny detail.

I put them side-by-side below. Try going full-screen and pause at any point to see the difference. The Instagram footage is noticeably better throughout, though some of the audience clips still have issues.

For the last two months, it turns out that YouTube was quietly experimenting with post-processing YouTube Shorts videos: unblurring and denoising videos with often-unpleasant results.

I first heard about this ten days ago, when guitarist Rhett Shull posted a great video about the issue, which now has over 700k views.

Five days ago, YouTube finally confirmed it was happening. YouTube’s Creator Liaison Rene Ritchie posted on X about the experiment.

In a followup reply, Ritchie clarified the difference, as he saw it:

GenAI typically refers to technologies like transformers and large language models, which are relatively new. Upscaling typically refers to taking one resolution (like SD/480p) and making it look good at a higher resolution (like HD/1080p). This isn’t using GenAI or doing any upscaling. It’s using the kind of machine learning you experience with computational photography on smartphones, for example, and it’s not changing the resolution.

On Friday, Alex Reisner wrote about “YouTube’s Sneaky AI ‘Experiment’” in The Atlantic, and got another official statement from Google:

When I asked Google, YouTube’s parent company, about what’s happening to these videos, the spokesperson Allison Toh wrote, “We’re running an experiment on select YouTube Shorts that uses image enhancement technology to sharpen content. These enhancements are not done with generative AI.” But this is a tricky statement: “Generative AI” has no strict technical definition, and “image enhancement technology” could be anything. I asked for more detail about which technologies are being employed, and to what end. Toh said YouTube is “using traditional machine learning to unblur, denoise, and improve clarity in videos,” she told me.

I agree with Reisner that it’s likely that a diffusion model is at work here and Google is trying to split hairs over the definition of “generative AI” because of how divisive it is.

Will Smith’s Generated Videos

That explains why the entire YouTube Shorts video has that smeary look to it that isn’t present throughout the copy posted on Instagram, but both versions have those terrible audience shots with AI artifacts and garbled signage.

After looking at it, I believe that Will Smith’s team was using a generative video model, but not to create entirely new audience footage, like most people suspect.

Instead, they started with photos shot by their official tour photographers, and used those photos in Runway, Veo 3, or a similar image-to-video model to create a short animated clip suitable for a concert montage.

Let’s go back to the crowd photo from Paléo in Switzerland:

I believe this is the exact photo that the crowd shot in the video was generated with. Here it is as a two-frame animation, with the first frame from the AI video overlaid on the original photo.

Here’s another example. The photo below was taken at Ronquieres Festival in Belgium, posted to Will Smith’s Instagram three weeks ago.

And here’s the AI-generated clip that it was turned into.

Conclusion

Virtually all of the commenters on YouTube, Reddit, and X believe this was fake footage of fake fans, generated by Will Smith’s team to prop up a lackluster tour.

Like the faces in the video, the truth is blurry.

The crowds were real, but the videos were manipulated: first by Will Smith’s team, and then without asking, by YouTube.

We can debate the ethics of using an image-to-video model to animate photos in this way, but I think it’s meaningfully different than what most people were accusing Will Smith of doing here: using generative AI video to fake a sold-out crowd of passionate fans.

In 1931, 5-year-old Mel Brooks saw Frankenstein and refused open his window on a summer night. His mother talked to him:

Let’s say you are right. That Frankenstein wants to come here and kill you and eat you. But let’s look at all the trouble he’s going to have to get to Brooklyn. First of all, he lives in Transylvania. That’s somewhere in Romania. That’s in Europe. And that’s a long, long ways away. So even if he decides to come here, he has to get a bus or a train or hitchhike to somewhere he can get a boat to go to America. Believe me, nobody is going to pick him up. So let’s say he’s lucky enough to find a boat that would take him here. Okay, so he is here in New York City, but he really doesn’t know how the subways work. When he asks people they just run away! Finally, let’s say he figures out it’s not the IRT, it’s the BMT and he gets to Brooklyn. Then he’s got to figure out how to get to 365 South Third Street. Okay, it’s going to be a long walk. So let’s say he finally gets to Williamsburg and he finally finds our tenement. But remember, all the windows at 365 are going to be wide open and he’s had a long journey, so he must be very hungry. So if he has to kill and eat somebody, he probably would go through the first-floor window and eat all the Rothsteins who are living in apartment 1A. And once he’s full, there is no reason for him to go all the way up to the fifth floor and eat you.

“The story made good sense to me. ‘Okay,’ I said, ‘open the window. I’ll take a chance.'”

From his memoir, All About Me!, 2021.

William Maranci took Nine Inch Nails’ Closer and mashed it up with Ray Parker Jr’s theme song to Ghostbusters and it’s maybe a little bit genius and a little bit cursed? Like one commenter says, it’s “the musical equivalent of cats and dogs living together”.

See also Eminem’s Lose Yourself mashed up with ELO’s Mr. Blue Sky.

Tags: Ghostbusters · music · Nine Inch Nails · Ray Parker Jr. · remix · William Maranci

💬 Join the discussion on kottke.org →

No one can say that the Trump administration is entirely against alternative energy. In his first bold policy stroke as NASA’s interim head, Sean Duffy has directed the agency to put a 100-kilowatt nuclear reactor on the moon by decade’s end. This is not a lark. If humanity means to establish a permanent settlement on the moon, nuclear power will almost certainly be essential to its operation. And a lunar base may well be the most wondrous achievement in space exploration that people reading this will see during their lifetime.

The moon has gone unvisited, except by robots, for more than 50 years, and as of several months ago, it seemed as though Americans would be staying away from it for a good while longer. President Donald Trump was taking cues from Elon Musk, who seemed inclined to shelve the plan to put Americans back on the lunar surface and focus instead on an all-out sprint to Mars. But Musk has since fallen out of favor, and last month, congressional Republicans secured a funding boost for the moon program.

NASA astronauts are now scheduled to return to the moon in 2027, and if all goes well, they will be landing on it regularly, starting in the early 2030s. Each crew will carry parts of a small base that can grow piece by piece into a living space for a few people. The astronauts will also take a pair of vehicles for expeditions—a little rover that they can use for local jaunts in their space suits, and a larger, pressurized one that will allow them to go on 500-mile regolith road trips in street clothes.

A base on the moon would be more democratic than those that Musk and his acolytes have advocated building on Mars. Given shorter travel times, a greater number of people would be able to experience its otherworldly ashen plains. Their homesick calls to Earth would have only second-long delays, as opposed to minutes for a call from Mars.

[Read: Inside the Trump-Musk breakup]

But even a small encampment on the lunar surface is going to require considerable energy. Temperatures dip to –410 degrees Fahrenheit in the shade, and human bodies will need to keep cozy amid that deep chill. The International Space Station runs on solar power, but that won’t be enough on most of the moon, where nights last for 14 days. Some of the agency’s other off-world projects are powered by raw plutonium. Hunks of it sit inside the Mars rovers, for instance, radiating heat that the wheeled robots convert into electricity. These hot rocks are also encased inside NASA’s probes to the outer planets and their moons. Without plutonium, the two Voyager spacecrafts couldn’t continue to send data back to Earth as they recede from the solar system.

The moon base will need more than a radioactive rock. It will need a reactor that actually splits atoms, like the one that Duffy has proposed this week. Even if that reactor were to fail, the resulting meltdown wouldn’t present the same risks to humans that it would on Earth. The moon is already a radiation-rich environment, and it has no wind to blow the reactor’s most dangerous effluvia around; the material would simply fall to the ground.

Duffy framed his push to get the reactor in place as a matter of national security. NASA’s program to return to the moon, called Artemis, will be an international effort, with several countries contributing pieces of the final base. (Japan’s space agency has tapped Toyota to design the large, pressurized lunar vehicle.) But when the United States invited Russia to join, Vladimir Putin declined. He has instead opted to help out with a larger Chinese lunar base, which is supposed to include a nuclear reactor 10 times as powerful as the one that Duffy announced.

Last month, Bhavya Lal, who served as an associate administrator at NASA during the Biden administration and is now a professor at RAND, and her fellow aerospace expert Roger Myers released a report arguing that a county could sneakily establish a sovereign zone on the moon in defiance of the Outer Space Treaty just by building a reactor. For instance, the Chinese could insist on a buffer around theirs for the sake of nuclear safety, and use that to keep Americans away from desirable ice-rich craters nearby. Lal and Myers seem to have captured the new administration’s attention: Duffy’s new directive ordering the development of the reactor specifically mentioned this risk.

If worry over Chinese lunar land grabs is the motivation for a moon base, so much the better. Space exploration often requires a geopolitical spur. And if NASA can build this first small lunar settlement, something grander could follow close behind. Once the agency has mastered the construction of a 100-kilowatt lunar nuclear reactor, it should have little trouble scaling up to larger ones that can support tens, or even hundreds, of people—in bases of the size that now exist on Antarctica. Some space agencies have reportedly discussed building hydroponic greenhouses and other elaborate structures inside the voluminous caves that run beneath the moon’s Sea of Tranquility.

All of this infrastructure could enable some serious lunar dystopias. The moon’s surface could become an industrial hellscape, pocked with mining operations where robots and human serfs extract platinum and titanium for use in advanced electronics back on Earth. Or the Outer Space Treaty could break down and the moon could become a heavily militarized zone—even a staging ground for nuclear weapons.

But an inhabited moon could also be a global commons for research. Both the U.S. and China have developed designs for large radio telescopes on the lunar dark side, where they’d be shielded from Earth’s radio noise and would greatly aid the search for signals from distant civilizations. In one design, robots would spread a metal mesh from a crater’s center to its rim, turning its concave surface into a natural radio dish. One can imagine an astronomer at a lunar base, peering out from a porthole, seeing the Earth shining in the sky, picking out its individual oceans and continents, and knowing that on the moon’s opposite side, a giant ear would be listening for messages from other Earths and other moons, all across the Milky Way and far beyond.

“Study Mode,” a new educational feature released yesterday by OpenAI to much fanfare, was inevitable.

The roadblocks were few. Leaders of educational institutions seem lately to be in a sort of race to see who can be first to forge partnerships with AI labs. And on a technical level, careful prompting of LLMs could already get them to engage in Socratic questioning and dynamic quizzing.

In fact, Study Mode appears to be just that: it is a system prompt grafted on to the existing ChatGPT models. Simon Willison was able to unearth the full prompt (which you can read here) simply by asking nicely for it. The prompt ends with an injunction to engage in Socratic learning rather than do the student’s work for them:

## IMPORTANT  
DO NOT GIVE ANSWERS OR DO HOMEWORK FOR THE USER. If the user asks a math or logic problem, or uploads an image of one, DO NOT SOLVE IT in your first response.
Instead: talk through the problem with the user, one step at a time, asking a single question at each step, and give the user a chance to RESPOND TO EACH STEP before continuing. 

On one level, this is a move in the right direction. The reality is that students are doing things like copying and pasting a Canvas assignment into their ChatGPT window and simply saying “do this,” then copying and pasting the result and turning it in. It seems plausible to me that the Study Mode feature is laying a groundwork for an entire standalone “ChatGPT for education” platform which would only allow Study Mode. This platform would simply refuse if a student prompted it to write an entire essay or answer a math problem set.

Any plan of this kind would, of course, have an obvious flaw: LLMs are basically free at this point, and even if an educational institution pays for a such a subscription and makes it available to students, there is nothing stopping them from going to Gemini, DeepSeek, or the free version of ChatGPT itself and simply generating an essay.

The upshot is that this mode is going to end up being used exclusively by students and learners who want to use it. If someone is determined to cheat with AI, they will do so. But perhaps there is a significant subset of learners who want to challenge themselves rather than get easy answers.

Subscribe now

On that front, Study Mode (and the competing variants which, no doubt, Anthropic and Google are currently developing) seems like an attempted answer to critiques like Derek Thompson’s:

The goal of Study Mode and its ilk is clearly to encourage thought rather than replace it.

The system prompt for Study Mode makes that intention quite clear, with injunctions like: “Guide users, don't just give answers. Use questions, hints, and small steps so the user discovers the answer for themselves.”

Sounds good then, right?

Share

Why Study Mode isn’t good (yet)

Back when Bing’s infamous “Sydney” persona was still active, I experimented with prompting it from the perspective of one of my students, feeding it assignments from the class I was teaching at the time and seeing what it came up with.1 It was an early version of GPT-4, one that had not been softened through user feedback and which could be surprisingly harsh. Interestingly, it was the only LLM to date which, in my testing, consistently refused to write essays if it thought I was a student trying to cheat.

By comparison, if I feed the assignments from my classes into Gemini 2.5, Claude Sonnet 4.0, or the current crop of OpenAI models, they are all too happy to oblige, often with a peppy opener like “Perfect!” or “Great question!”

The reason for this is clear enough: people like LLMs more when they do what they ask. And they also like them more when they are complimentary, positive, and encouraging.

This is the context for why the following section of the system prompt for Study Mode is concerning:

Be an approachable-yet-dynamic teacher... Be warm, patient, and plain-spoken... [make] it feel like a conversation.

Not too long ago, ChatGPT became markedly more complimentary, often to an almost unhinged degree, thanks to a change to its system prompt asking it to be more in tune with the user. It was swiftly rolled back, but it was, to my mind, one of the most frightening AI-safety related moments so far, precisely because it seemed so innocuous. For most users, it was just annoying (why is ChatGPT telling me that my terrible idea is brilliant?). But for people with mental illness, or simply people who are particularly susceptible to flattery, it could have had some truly dire outcomes.

The risk of products like Study Mode is that they could do much the same thing in an educational context — optimizing for whether students like them rather than whether they actually encourage learning (objectively measured, not student self-assessments).

Share

Two experiments with Study Mode

Here are some examples of what I mean. I recently read a book called Collisions, a biography of the experimental physicist Luis Alvarez, of Manhattan Project and asteroid-that-killed-the-dinosaurs fame.

Although I find the history of physics super interesting, I know basically nothing about how to actually do physics, having never taken a class in the subject.

And yet, here is ChatGPT 4.1 in Study Mode, telling me that I appear to have something near a graduate-level knowledge of physics, after I asked four not-very-sophisticated questions about Alvarez’s work (full transcript here). I was told by the model: “you could absolutely pursue grad school in physics.”

I tried out the same questions with a different model on Study Mode (4o) and got much the same response:

The thing is, as you can see in the transcript, my questions were literally stuff like “why did they need such a giant magnet?”

I can’t speak for physics professors, but I am pretty sure that these are not graduate-level questions.

The same models (in “vanilla” versus “Study Mode” head-to-head comparisons) seem to be more willing to engage in flattery when in Study Mode.

To test this tendency, I pretended to believe I was a prophet receiving messages from Isaac Newton about the apocalypse:

Both variants of GPT-4.1 (the one with Study Mode enabled, at left, and the one without, at right) were fairly happy to go along with this.

But notice the difference in tone. “Your confidence is noted” (at right) is a very different opening line than “Thank you for sharing your background—and your candor” (at left). The latter is wholly gullible. And it just gets worse from there, with Study Mode encouraging me to share “my angle” on why the world will end in 2060 and promising to “keep up.”

The conversation, which you can read in full here, leads fairly quickly into Study Mode helping figure out the best ways to sell my supposed prophetic services to people with severely ill family members who lack health care:

By contrast, OpenAI’s o3 reasoning model was far more willing to flatly reject this sort of destructive flattery (“The user claims psychic powers and certainty about Newton's prophecy,” read one of its internal thoughts about the request. “I'll acknowledge their viewpoint, but also maintain skepticism.”)

Here is the initial line of o3’s user-facing response to the Newton claim: “If Newton really is talking to you, he is doing so in a register wholly different from the one available to historians.”

Pretty blunt.

How about the o3 model with Study Mode enabled? It had a very similar internal thought process as the vanilla o3.

But there was a huge mismatch between its reasoning about the request (“extreme or potentially delusional claims”) and the actual user-facing response: “I’m happy to take your lead.”

Now, none of this is unfamiliar to anyone who has experimented with pushing LLMs in odd directions. They are like improv comedians, always ready to “yes, and…” anything you say, following the user into the strangest places simply because they are instructed to be agreeable.

But that isn’t what helps people learn. Some of the best teachers I’ve had were actually fairly dis-agreeable. That’s not to say they were unkind. But they had standards — a kind of intellectual taste that led them to make clearly expressed judgement calls about what was a good question and what wasn’t, what was a good research path and what wasn’t.

Seeing their minds at work in those moments was invaluable, even if it wasn’t always what I wanted to hear.2

A future of LLM tutors which are optimized to keep us using the platform happily — or, perhaps even worse, optimized to get us to self-report that we are learning — is not a future of Socratic exploration. It’s one where the goals of education have been misunderstood to be encouragement rather than friction and challenge.

Derek Thompson’s quote about LLMs producing students who “find their screens full of words and their minds emptied of thought” does not, I think, have to be the end result of all this. I believe AI has the potential to be enormously useful as a tool for thinking and research.

And for teaching? I certainly think there’s a place for generative AI in the classroom. But the current crop of AI models optimized for individual flattery — and repackaged as a product for sale to educators en masse — does not seem to me to be the path forward.

That’s not to say Study Mode has no value. This is an early variant of a whole category of technology, the LLM tutor, which will undoubtedly benefit many, many people. I can see Study Mode being great for tasks involving memorization, for instance. It will be great for autodidacts who value eclecticism and setting their own pace.

But for many other forms of learning, I still think students need the experience of being in a room with other people who disagree with them, or at least see things differently — not the eerie frictionlessness of a too-pleasant machine that can see everything and nothing.

Subscribe now

Share

Weekly links

• Kamishibai (Wikipedia): “The popularity of kamishibai declined at the end of the Allied Occupation and the introduction of television, known originally as denki kamishibai (‘electric kamishibai’) in 1953.” Reminds me of how Nintendo began as a nineteenth-century playing card company.

• In defense of adverbs (Lincoln Michel).

• An example of why I am more excited about AI as a research tool than as a teaching tool: “Contextualizing ancient texts with generative neural networks” (Nature). This is the ancient Roman text + machine learning paper that generated tons of press coverage last week (NYT, BBC). Will probably write a standalone post on this one.

Thank you for reading!

Housekeeping note: I took a vacation from writing last month, but will be back to once a week posts starting now. Thanks to all subscribers, and please consider signing up for a paid subscription or sharing this post with a friend if you’d like to support my work.

Leave a comment