Short Thoughts

Responding to I Do Not Want to Be a Programmer Anymore (After Losing an Argument to AI and My Wife)

The article begins by sharing a story of attempting to use AI to resolve a difference of opinion with his wife, which convinced him he was wrong. His wife reaction:

It wasn’t the victory that stuck with her. It was how easily I surrendered my judgment to a machine.

He gives another example from work, from which he writes:

That’s the unsettling part. We don’t just listen to the machine; we believe it. We defer to it. And sometimes, we even prefer its certainty over the reasoning of the actual humans in front of us.

His concerning conclusion:

Wisdom has always come as the byproduct of experience. But if experience itself is outsourced to machines, where will the young earn theirs?

I also have experienced myself being resistant to the arguments of another only to be won over by consulting a LLM and reasoning through the arguments. In part this seems reasonable, the ideas of others which are contrary to our own are costly for us. Ideas which we arrive at, or we think we arrive at, on our own we believe we have already been through the work to vet.

Therefore, the question is whether we ask AI’s answer on the first take, or do we go back and forth with the AI examining the rationale. The first is concerning, to blindly accept the response without any further examination. But I suspect that is not what occurs in most use cases. Instead we become convinced by it because it is a nonthreatening way to explore the topic. I wonder if there is intimations of that when he says:

Clients, colleagues, even strangers are emboldened not because the machine gives them ideas, but because it gives them confidence.

When he provides the example at work the person sent him a “detailed breakdown” of how to improve the system. It sounds to me the person invested a lot of effort and thought into this, not quickly typed a question and forwarded on the AI response.

Circling back to his concern about wisdom, or lack of, I believe this highlights the need for relationship. If relationships continue to erode, lack of mentorship, and trust in AI continues to rise then is wisdom lost?

It feels this may be the case. But humans still accumulate experiences, from both our failures and triumphs. And from those experiences wisdom will still either be derived or ignored. It’s hard to imagine a complete loss of wisdom. Even the author gain wisdom from the experiencing of bringing AI into the conversation with his wife. There is precious wisdom humankind has obtained across our existence, which would be a tragedy to lose. But I have a hope in humanity, that we will continue to push forward and adapt, accumulating wisdom. It is in our nature, I don’t think we can do anything otherwise.

Replying to “You have 18 months” The real deadline isn’t when AI outsmarts us — it’s when we stop using our own minds.

And I am much more concerned about the decline of thinking people than I am about the rise of thinking machines.

I’m not precisely concerned about this. I don’t believe my thinking has declined since using these tools. Maybe they have in some trivial ways. But I believe my thinking has become more active as a result of these tools, because I am able to explore and ask questions, to investigate in ways I either was not able to before, or at least not as easily.

My concern is that there will be a division between people on how they use these tools. One side’s thinking will decline, while the other side’s thinking will be enhanced, which will lead to a further imbalance in society. It appears the statistics he references support this, the declines he reports are not coming from those who already reported as high.

The author later he answers the question about what kids should study:

While I don’t know what field any particular student should major in, I do feel strongly about what skill they should value: It’s the very same skill that I see in decline. It’s the patience to read long and complex texts; to hold conflicting ideas in our heads and enjoy their dissonance

While I do not entirely agree with his phrasing, or at least I am uncertain with how he phrased it, I do believe being able to work with conflicting ideas is an important skill. Perhaps if someone “enjoys” the dissonance they become energized and thrive in these situations. And so maybe the language is not too strong. But at the minimum I have found being able to wrestle with conflicting ideas to be an important life skill.

Quoting Richard Matthew Stallman in Reasons not to use ChatGPT:

It does not know what its output means. It has no idea that words can mean anything.… people should not trust systems that mindlessly play with words to be correct in what those words mean.

My initial reaction is agreement. But then what are the implications? Does it follow that machines will never have intelligence? What makes our intelligence different? A calculator doesn’t know what its output means. Should we trust it? Is the way the calculator works with numbers significantly different than the way the LLM works with words. It’s empirically different, but when attempting to consider each domain they operate in is it significantly different?

Did he think about all of this and then come to his conclusion?

These are all interesting questions and conversations we should be having but I wonder if his real complaint is included in his trailing thoughts:

Another reason to reject ChatGPT in particular is that users cannot get a copy of it.

This weekend I was listening to the re-release of the How to Have Great Conversations episode of the People I Mostly Admire podcast. Levitt and Duhigg discuss the three types of conversations: problem solving, emotional and relational. At one point Levitt says it took him until his 40’s to realize people have various goals in conversations, not just to find solutions.

It only seems natural to me we overly focus seeing things from our perspective, even when taking into account theory of mind. No area is excluded from this, including politics; especially politics.

Enter Arnold Kling’s The Three Languages of Politics. His thesis is that Progressives, conservatives, and libertarians each speak a different language, they each have a different way of expressing their point of view.

• Progressives
  • Oppressor and Oppressed
  • Concerned with the power dynamics between groups of people or organizations. They see government as a means to address this by ensuring justice for the oppressed.
• Conservatives
  • Civilization and Barbarism
  • Concerned about the order, values, and structure of society. They see government as a way to protect these from forces that could disrupt them, which include a breakdown of the rule of law.
• Libertarians
  • Freedom and Coercion
  • Concerned about people being able to make their own choices. They see government as a threat to this, believing its coercion (through taxation and regulation) is the main impediment to individual liberty and prosperity.

All three deal with a form of injustice, but the injustice is seen through a different lens, and therefore differ on what should be done.

In an effort to understand how these play out here is my attempt to think through the issue of immigration, documented and undocumented, in the United States. Each position could be fleshed out further, but for this discussion the context is limited to the direct concerns of the three languages.

Progressives see immigrants as a vulnerable people. It is government’s responsibility to welcome them and provide aid.

Conservatives see immigrants as a disruption to society. It is the government’s responsibility to protect society and foster an environment for citizens to thrive.

Libertarians see immigrants as groups of people who have been denied the ability to make their own choices. The current situation has been brought about by government’s hindrances to immigration as well as the assistance it provides.

This highlights the challenges of coming to agreement about immigration, which is not unique to this topic. It it helpful to keep in mind that these do not represent the entire perspective of people who hold these positions, instead it is the language they come to the issue with. Therefore, agreements can still be found, and they are more apt to be found when keeping in mind that people are likely to describe their position in one of these terms. It is also important to remember that people’s perspective is rich and nuanced.

I often think about the word Sonder:

The profound feeling of realizing that everyone, including strangers passing in the street, has a life as complex as one’s own, which they are constantly living despite one’s personal lack of awareness of it.

The thought of this word produces a sense of wonder and amazement. I hope to carry this sense with me into more relationships and interactions, including conversations on immigration.

I started reading Hannah Arendt. When she attended the trial of Adolf Eichmann, a Nazi and an organizer of the Holocaust, she observed what she called the “banality of evil”. Here’s a quote, emphasis mine.

The deeds were monstrous, but the doer—at least the very effective one now on trial—was quite ordinary, commonplace, and neither demonic nor monstrous. There was no sign in him of firm ideological convictions or of specific evil motives, and the only notable characteristic one could detect in his past behavior as well as in his behavior during the trial and throughout the pre-trial police examination was something entirely negative: it was not stupidity but thoughtlessness.

A few weeks ago I came across the opinion piece Will AI Destroy or Reinvent Education? I think this piece has lots of good thoughts.

The piece begins with talking through two articles about an MIT study on the impact of AI on our brain when using it in writing, Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. The paper found LLM use led to weaker neural connectivity and less cognitive engagement.

Initially I thought, this is concerning and a good reason to have an awareness of how we use AI. But if people use it this way, it is to their detriment.

After reading Arendt, I’m even more concerned. Could AI use be fostering a new kind of “thoughtlessness”? If so, might horrific deeds be carried out by people who are “quite ordinary, commonplace, and neither demonic nor monstrous” simply because they stopped thinking critically?

Quoting Jessica Kerr, Austin Parker, Ken Rimple and Dr. Cat Hicks from AI/LLM in Software Teams: What’s Working and What’s Next

Empathy

People will do things for AI that they won’t do for each other. They’ll check the outcomes. They’ll make things explicit. They’ll document things. They’ll add tests.… And all of these things help people, but we weren’t willing to help the people. It’s almost like we have more empathy for that robot than for each other… we can imagine that the AI really doesn’t know this stuff and really needs this information. And at some level, we can’t actually imagine a human that doesn’t know what we do.

This comparison creates a visceral blow, because I feel it describes me. I consider myself a fairly empathic person, but I’m slow to create this information for other humans, yet find myself more willing to do so for AI.

Why do we behave this way? Here are some theories. Different expectation levels, ex: AI doesn’t have this background knowledge, but humans should, or at least can figure it out. Comparison and competition between ourselves and others. The impact is immediate when working with the AI, but unknown and in the future for humans. More self-serving when providing these to the AI, at least in the near term again.

Even with these plausible explanations, I can’t quite get myself off the hook. This nagging self-awareness, however, doesn’t diminish my fear that my behavior will remain unchanged.

Participation

Another topic of this interview deserves mention:

…have a training data problems, right? And we can question what we use it for, but it’s very difficult to do that if you sit outside of it. If you set yourself apart, you have to participate.

I do think that is incumbent upon us to grapple with, you know, the reality we’re faced with… We have the universal function approximator finally and there’s no putting that toothpaste back in the tube, so we can figure out how to build empathetic systems of people and technology that are humanistic in nature, or we can let the people whose moral compass orients slightly towards their bank account make those decisions, and I know which side of it I’m on.

AI is here and it will change a lot of things. It’s understandable to be worried about the negative impact of AI, but letting that prevent you from engaging is a way of sitting on the sidelines. Instead, we have a duty to participate and shape its future.

Quoting CasualPhysicsEnjoyer in You’re a Slow Thinker. Now what?

The broad thing that I’ve learnt so far is that it’s been better for me to lean into my thinking style, rather than fight against it. The funny part is that now, I come across as ‘quicker’. But that’s no accident - I’m just reciting things that I’ve worked on for a long time, slowly, behind the scenes!

This perfectly describes me, and probably a lot of other people too. When I appear “quicker” it’s because I’ve already had the conversation in my head many times, and covering many possible directions. I call it “pre-thoughts”.

My theory is because I am an introvert, both the characteristic of slow thinking as well as spending a lot of time in thought beforehand.

Quoting Robert P. George from the Pursuit: The Founders’ Guide to Happiness in the episode In Order to Be Happy.

Madison also understood that structural constraints by themselves would be insufficient. If we are to be self-governing people, that is, if we’re to be citizens of a republic, then we need to be citizens who are capable of self-control of self-mastery of holding in check, at least to some extent. Those powerful evil impulses in our nature, and that requires education.

So, Madison famously said only a well-instructed people can be permanently a free people. That is only a well-educated people can only be a people fit for self-government. They needed to be educated in their character. Capable of distinguishing good from bad, right from wrong, just for unjust.

Ethan Mollick writes that our relationship with AI is shifting, from being a partner we create with to the AI performing the work itself, On Working with Wizards. This aligns with the pursuit of AI agents, 2025 being dubbed the year of the agent. While Mollick doesn’t explicitly discuss agents, he uses the term “wizards” to describe a similar concept. He calls them wizards because:

Magic gets done, but we don’t always know what to do with the results

…wizards don’t want my help and work in secretive ways that even they can’t explain.

This presents two important challenges. First, we lose, or never develop, the skill to evaluate what was produced. We lose ground, and so we are forced to trust more. As Mollick states:

every time we hand work to a wizard, we lose a chance to develop our own expertise, to build the very judgment we need to evaluate the wizard’s work.

But what I found especially striking is that throughout time, when we did not understand something, it was considered magic. Science came along and brought a method of understanding of how things work. Technology replaced magic. Is this direction now reversing? Are we losing our understanding, is magic returning? Mollick writes:

The paradox of working with AI wizards is that competence and opacity rise together. We need these tools most for the tasks where we’re least able to verify them. It’s the old lesson from fairy tales: the better the magic, the deeper the mystery.

This is a shocking realization. While many would push back against this analogy, it seems to be at least partially accurate. It raises a pressing question: Why are we so willing to trade our understanding and accept the magic in this context?

A constant complaint I’ve heard from software developers is that there isn’t a product owner. No one is creating requirements, no one is curating the backlog. Instead the software delivery team attempts to suss out how applications and platforms are to be built. Fair enough, it’s not very efficient to be given a high level description of something and then have to determine what it means.

I’m not going to analyze or offer my thoughts on this predicament, but I was reflecting on it while considering my current software development workflow:

• Receive a high level feature request
• Use AI to create detailed requirements based on the request and the state of the current application
• Edit the generated requirements
• Provide the final version of the requirements to an AI agent to implement
• …

If the desires of software developers were fulfilled then pristine requirements would be created by a product owner. Software developers would then hand off the requirements to an AI agent for implementation, thus making software developers then new Tom, the product manager from Office Space.

What would you say… you do here?

I’ve read three different takes on hallucinations this week, and what struck me most was not how they agreed, but how differently they framed the problem. Each piece approaches hallucinations from a unique angle: technical, procedural, and philosophical. Taken together, they sketch a landscape of possibilities.

OpenAI’s Why language models hallucinate presents the view that not all questions have answers, and so models should be trained with an incentive to abstain rather than answer confidently.

One challenge remains stubbornly hard to fully solve: hallucinations. By this we mean instances where a model confidently generates an answer that isn’t true.

Most evaluations measure model performance in a way that encourages guessing rather than honesty about uncertainty.

Penalize confident errors more than you penalize uncertainty, and give partial credit for appropriate expressions of uncertainty.

Then there’s Is the LLM response wrong, or have you just failed to iterate it?, which suggests that inaccurate responses are often the result of receiving an answer too soon. If pushed further, by having the model iterate, it can examine the evidence and follow new lines of discovery, much like humans do.

But the initial response here isn’t a hallucination, it’s a mixture of conflation, incomplete discovery, and poor weighting of evidence. It looks a lot like what your average human would do when navigating a confusing information environment.

LLMs are no different. What often is deemed a “wrong” response is often merely a first pass at describing the beliefs out there. And the solution is the same: iterate the process.

Finally, there is Knowledge and memory, which suggests hallucinations will not go away because knowledge must be tied to memory. Humans feel the solidity of facts, while models lack the experiences required to ground their knowledge.

Language models don’t have memory at all, because they don’t have experiences that compound and inform each other.

Many engineers have pinned their hopes on the context window as a kind of memory, a place where “experiences” might accrue, leave useful traces. There’s certainly some utility there… but the analogy is waking up in a hotel room and finding a scratchpad full of notes that you don’t remember making… but the disorientation of that scenario should be clear.

The solid, structured memory that we use to understand what we know and don’t know — when and when not to guess — requires time, and probably also a sort of causal web, episodes and experiences all linked together.

Each of these pieces makes interesting points, and together they explain different facets of model hallucination. Models are too eager to provide an answer. There are many uncertainties and “it depends” in life. Incentivizing models to reflect this may irritate users, but it better mirrors reality.

However, these responses make clear that what we receive is only a first pass, one that should be refined by iterating, digging deeper, and pushing the model further. Perhaps this process of discovery is still not enough to create true memory, as the third author points out, but it does seem to edge closer to mimicking a brief experience.

Currently, model context is built by labeling messages as system, user, or agent. We’ve learned it would be better to create a hierarchy of significance for these categories, system messages should carry more weight and not be overridden by user messages. What if context were segmented by other dimensions, like time, so a model could build a clearer picture of what it has learned?

Humans also continue processing conversations or experiences outside the event itself. What if models pushed themselves to dig deeper without user prompting, allowing them to provide a more thoughtful answer after the interaction had ended?

There is still so much more to explore, we are far from exhausting what’s possible.

I’ve only watched one video but this seems like a promising course to get up and going in a productive way with Semantic Kernel agents.

11 Lessons to Get Started Building AI Agents

Quoting Lori Gottlieb in conversation with Andrew Huberman in How to Find & Be a Great Romantic Partner

Drama tends to be from all the assumptions. Like, I know that, my truth, is the right story… You were wrong. And I think it’s so interesting because people come to therapy with these faulty narratives…

We’re all storytellers. We all try to make sense of something, and we all believe that our story is the absolute accurate version of the story, and actually it’s so funny when you see couples and they experience the same experience, and they have wildly different versions.

And then there’s some part where the Venn diagram overlaps, and then finally, they can see, oh, that person’s not a bad person they were coming at it because they, in their story, they believe this. And that’s so important. So drama happens when assumptions are made. People characterize the other person’s story as inaccurate, their own story as accurate. And then there’s… no space for curiosity or connection. It’s all rupture, no repair. our-story-is-the-absolute-accurate-version.md

Quoting Russ Roberts in conversation with Lowry Pressly discussing his book The Right to Oblivion: Privacy and the Good Life

I think he’s saying one other thing, or at least I learned one other thing from it. Which is that—and this is something you learn from marriage or a close friendship—you feel an injustice, or you’re accused of an injustice, and you play back the tape in your mind and you realize, ‘Nope, I was right.’ I did the right thing, and my friend or my wife is misremembering this, and let’s bring it out in the light. Let’s play the tape. Let’s get the video. Let’s get the CCTV—the closed circuit television—of that conversation we had three weeks ago at breakfast. And I’m going to show her—she’s going to show me—how it really was.

And, that’s not available, that tape. And, your mind has—it’s an illusion to think you can bring that into the light and settle it. And, it’s one of the deepest things I think you learn from marriage, is that your self-righteousness is usually—it’s either wrong, it’s a fake memory. We don’t have the tape. And, it’s much better to just say, ‘Maybe I was wrong and I’m going to learn from it.’

But, the idea that you could just get that big flashlight and illuminate that dark corner from childhood, from two years ago, from last week, it’s just an illusion. And it’s a dangerous, unhelpful illusion for interacting with other human beings who have the same challenges you do.