Posts tagged with "society"

• Quoting Andrej Karpathy regarding LLM intelligence Animals vs Ghosts

  10/21/2025

  Quoting Andrej Karpathy regarding LLM intelligence Animals vs Ghosts

  In my mind, animals are not an example of this at all - they are prepackaged with a ton of intelligence by evolution and the learning they do is quite minimal overall (example: Zebra at birth). Putting our engineering hats on, we’re not going to redo evolution. But with LLMs we have stumbled by an alternative approach to “prepackage” a ton of intelligence in a neural network - not by evolution, but by predicting the next token over the internet. This approach leads to a different kind of entity in the intelligence space. Distinct from animals, more like ghosts or spirits. But we can (and should) make them more animal like over time and in some ways that’s what a lot of frontier work is about.

  That’s beautifully said. It paints a vivid picture of how LLM intelligence differs from biological intelligence. By training on the collective content of the internet, these models become a form of us, our past selves, our ghosts. We recognize an intelligence in them, but it’s a mistake to equate it with human or animal intelligence.

  Is the current intelligence enough for AGI? Will the next AI winter come from trying to make models more like animals? Is that even a wise path to pursue?

  I don’t think today’s intelligence is sufficient for true AGI. As Karpathy pointed out, it’s a fundamentally different kind of intelligence. I don’t see how this architecture evolves into something truly general. It can get closer, sure, but there will always be holes needing to be plugged. This will bring forth the next AI winter, until the next breakthrough is discovered and our capabilities reach the next level.

  Still, I’m uneasy about that pursuit. There’s already so much potential in what we have now. Entire industries and creative fields haven’t even begun to fully explore it. And as a society, we’re not prepared for the intelligence we already face. However, it is in our nature to always be progressing. Perhaps by the time the next breakthrough occurs, society will have adjusted to the current level of intelligence, better preparing us us for the next level.

• Quoting Toby Stuart on Pedigree

  10/6/2025

  Quoting Toby Stuart on the EconTalk episode The Invisible Hierarchies that Rule Our World

  The episode ended with a discussion on the impact of AI as it relates to people’s status, that it will reinforce the prestige hierarchy.

  When you can’t judge quality that’s precisely the time in which you rely on pedigree.

  He goes on to say:

  So if you are a college admissions officer, take that problem at a place where a lot of people want to go, it’s really hard to read an essay and say “I’m admitting them because this is an outstanding essay,” if that ever happened. But what you can do is, I’ve heard of the high school, or there is some other status marker in the background, so I’m going to overweight that relative to information that formerly was a signal but it’s just noise.

  Writing has been used just about everywhere to evaluate people. Now the capability of crafting well written content is available to everyone. Consequently, we become more reliant on other indicators for evaluation, which, unfortunately, are often characteristics over which people have limited control

  This line of thought presents a sobering reality. What was once seen as an equalizer for those lacking inherited advantages potentially turns out to be detrimental to their advancement.

• The "banality of evil"

  9/16/2025

  I started reading Hannah Arendt. When she attended the trial of Adolf Eichmann, a Nazi and an organizer of the Holocaust, she observed what she called the “banality of evil”. Here’s a quote, emphasis mine.

  The deeds were monstrous, but the doer—at least the very effective one now on trial—was quite ordinary, commonplace, and neither demonic nor monstrous. There was no sign in him of firm ideological convictions or of specific evil motives, and the only notable characteristic one could detect in his past behavior as well as in his behavior during the trial and throughout the pre-trial police examination was something entirely negative: it was not stupidity but thoughtlessness.

  A few weeks ago I came across the opinion piece Will AI Destroy or Reinvent Education? I think this piece has lots of good thoughts.

  The piece begins with talking through two articles about an MIT study on the impact of AI on our brain when using it in writing, Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. The paper found LLM use led to weaker neural connectivity and less cognitive engagement.

  Initially I thought, this is concerning and a good reason to have an awareness of how we use AI. But if people use it this way, it is to their detriment.

  After reading Arendt, I’m even more concerned. Could AI use be fostering a new kind of “thoughtlessness”? If so, might horrific deeds be carried out by people who are “quite ordinary, commonplace, and neither demonic nor monstrous” simply because they stopped thinking critically?