joshua/joshua.seigler.net
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

further revision

Browse source
This commit is contained in:
Joshua Seigler 2025-04-26 01:00:48 -04:00
parent 4b7aba50a7
commit e506a7b44f
4 changed files with 20 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

34
site/posts/2025-04-24-thinking-machines.md
View file

@ -1,11 +1,9 @@
---
title: Thinking machines - some thoughts about large language models
title: Thinking machines
slug: thinking-machines
description: "The computers will start thinking, and people will stop."
---
## Work
There's an exchange early in the classic '80s movie TRON. Some scientists are talking shop:
> **ALAN:** Ever since he got that Master Control Program set up, system's got more bugs than a bait store.
@ -16,32 +14,34 @@ There's an exchange early in the classic '80s movie TRON. Some scientists are ta
>
> **GIBBS:** (wryly) Yes, won't that be grand -- *the computers will start thinking, and people will stop*.
Gibbs has a point. The modern vision of a utopian future is one where work is relieved, and people are free to pursue leisure, or exercise their creativity with art, writing, and poetry. Setting aside the irony that creative works are the first and most visible applications of this technology -- is that imagined future actually a good one?
Gibbs has a point. The modern vision of a utopian future is one where work is relieved, and people are free to pursue leisure, or exercise their creativity with art, writing, and poetry. Thinking computers are here now, in the form of "large language models" (LLMs) like ChatGPT. Setting aside the irony that creative works are the first and most visible applications of LLM technology -- is that imagined future actually a good one?
## Mom is always right
When I was a kid, I remember a day going to yard sales with my mom in the family minivan. It was early summer, a hot day. The windows were down, and I complained that if the vehicle has good air conditioning, we should use it. What was the point in getting all hot? "To get used to the warm weather," came the answer. What an injustice! We were sweating back there! Later in life, I took a short trip to Arizona in August. Everyone scurried from building to building. Where the sun was doubled, reflected off of glass skyscrapers, the temperature jump was alarming. It was actually unsafe to spend long stretches outside unprepared. But when I returned to Massachusetts, for the rest of the summer 85 or 90 degrees Fahrenheit felt like nothing.
All that to say, the work that this new technology offers to relieve isn't just about achieving a result. The effort maintains and builds our abilities. Work pushes us to connect to each other for help, or to persevere in doing something difficult. Outsourcing that work eventually means losing the ability to do it yourself.
All that to say, the work that LLM technology offers to relieve isn't just about achieving a result. The effort maintains and builds our abilities. Work pushes us to connect to each other for help, or to persevere in doing something difficult. Outsourcing that work eventually means losing the ability to do it yourself.
## Empathy
## Attention must be paid
LLMs (large language models like ChatGPT) clearly manifest a type of intelligence. Sure, it's "just" multidimensional vector embeddings. But it does exhibit a type of intelligence. One without empathy. Not being human, it *can't* have empathy -- and intelligence without empathy is dangerous [^1] [^2].
Simply put, an LLM is a document completion engine. You give it text, and it extends it. The result doesn't have to be true, it just has to be convincing. No amount of pre-training or guard rails will make it truthful. It does often say true things, but that's not the point, it's more of a happy accident.
Because it was built from essentially the whole public internet, LLMs also have a strong connection to The Algorithm. Algorithms that run social media feeds and online advertising are designed to attract human attention, a precious thing. These systems (LLMs as well as social media algorithms) are oriented towards capturing that attention. The seminal LLM paper is even called, "Attention is all you need". A prescient title. AI images have a magnetic quality. The early iterations would let you down if you look closely at fingers or text, but even those images are hard to ignore. So - the intelligence of AI is not like ours. It can't know what it's like to be a human, and if it could be said to want something, it would be attention.
Because they are built from essentially the whole public internet, LLMs also have a strong connection to The Algorithm. Algorithms that run social media feeds and online advertising are designed to attract human attention, a precious thing. Social media algorithms and LLMs are oriented towards capturing that attention. The foundational LLM paper is even called, "Attention is all you need". A prescient title. LLM intelligence is not like ours. It can't know what it's like to be a human.
## Truth
If this was a person, someone who wanted your attention and had this kind of indifference towards truth, they would be considered a con-man or bullshitter[^1]. Untrustworthy.
It also doesn't care about truth. Speaking reductively, an LLM is a document completion engine. You give it text, and it extends it. No amount of pre-training or guard rails will make it truthful, because it is built to be *convincing*, not accurate. It does often say true things. But that's not the point, it's more of a happy accident. A person who has this kind of indifference towards truth would be considered a con-man or bullshitter[^3]. Untrustworthy.
## Don't create the torment nexus
## Warnings
LLMs clearly manifest a type of intelligence. Sure, it's "just" some linear algebra and a ton of data. But it does exhibit a type of intelligence. One without empathy. Not being human, it *can't* have empathy -- and intelligence without empathy can be dangerous [^2] [^3].
Science fiction is littered with cautionary tales about inhuman intelligence. For that matter, so is myth: genies give people whatever they want, but because people have self-destructive desires (like the desire to avoid work), it goes wrong. In TRON, Infocom has the MCP (Master Control Program), an overgrown chess program that is given access to whatever information it can consume, until its intelligence and capabilities are seemingly endless. The company leadership comes to rely on the program so completely that it becomes their entire interface for understanding and operating the business. There is also the irony that Infocom's success was built on the misuse of intellectual property, much as LLM companies have done [^4] [^5].
I am not wise enough to safely use a genie in a bottle. And I won't outsource my creative efforts to an addictive, bullshitting alien intellect, even if it might save time and effort in the short term.
I don't think I am wise enough to safely use a genie in a bottle. And I don't want to outsource my creative efforts to an addictive, bullshitting alien intellect, even if it might save time and effort in the short term.
![End of line](/2025-04-24--end-of-line.jpg)
[^1]: [AI chatbot pushed teen to kill himself, lawsuit alleges | AP News](https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-artificial-intelligence-9d48adc572100822fdbc3c90d1456bd0)
[^2]: [Belgian man dies by suicide following exchanges with chatbot](https://www.brusselstimes.com/430098/belgian-man-commits-suicide-following-exchanges-with-chatgpt)
[^3]: [On Bullshit by Harry Frankfurt : Harry Frankfurt : Free Download, Borrow, and Streaming : Internet Archive](https://archive.org/details/on-bullshit-by-harry-frankfurt)
[^4]: [AI, Copyright, and the Law: The Ongoing Battle Over Intellectual Property Rights – IP & Technology Law Society](https://sites.usc.edu/iptls/2025/02/04/ai-copyright-and-the-law-the-ongoing-battle-over-intellectual-property-rights/)
[^5]: [Generative AI Has an Intellectual Property Problem](https://hbr.org/2023/04/generative-ai-has-an-intellectual-property-problem)
[^1]: [On Bullshit by Harry Frankfurt | Internet Archive](https://archive.org/details/on-bullshit-by-harry-frankfurt)
[^2]: [AI chatbot pushed teen to kill himself, lawsuit alleges | AP News](https://apnews.com/article/chatbot-ai-lawsuit-suicide-teen-artificial-intelligence-9d48adc572100822fdbc3c90d1456bd0)
[^3]: [Belgian man dies by suicide following exchanges with chatbot | Brussels Times](https://www.brusselstimes.com/430098/belgian-man-commits-suicide-following-exchanges-with-chatgpt)
[^4]: [AI, Copyright, and the Law: The Ongoing Battle Over Intellectual Property Rights | IP & Technology Law Society](https://sites.usc.edu/iptls/2025/02/04/ai-copyright-and-the-law-the-ongoing-battle-over-intellectual-property-rights/)
[^5]: [Generative AI Has an Intellectual Property Problem | Harvard Business Review](https://hbr.org/2023/04/generative-ai-has-an-intellectual-property-problem)

2
site/timeline/2017-2020.md
View file

@ -2,5 +2,5 @@
title: Software developer @ Dash Core Group
time: 2017-2020
description: |
After some successful freelance work, I became a full time contractor, working remotely. I was fortunate to sometimes travel to interesting places to represent Dash or to meet up worldwide with other developers. My focus here was on web development, React Native app development, and JavaScript API testing and documentation. The contract was ended due to an extended crypto price slump in 2019-2020.
After some successful freelance work, I became a full time contractor, working remotely. I was fortunate to sometimes travel to interesting places to represent Dash or to meet up worldwide with other developers. My focus here was on web development, React Native app development, and JavaScript API testing and documentation.
---

2
site/timeline/2021-2022.md
View file

@ -2,5 +2,5 @@
title: Senior software engineer @ Prenda
time: 2021-2022
description: |
This is at the intersection of many interests, including learning, communication, and decentralized solutions. Prenda has an excellent culture that fosters humility and recognizes others' humanity.
This role was at the intersection of many interests, including learning, communication, and decentralized solutions. Here I learned to own, plan, and solve larger and less-well-defined tasks. Prenda has an excellent and intentional culture that fosters humility and recognizes others' humanity.
---

2
site/timeline/2023-2025.md
View file

@ -2,5 +2,5 @@
title: Founding software engineer @ Capabuild
time: 2022-2025
description: |
Capabuild is a web and mobile platform for disaster recovery companies. This role was an unexpectedly great fit for some technologies I used in side projects: THREE.js and SVG. Here I worked with other experienced engineers building new things and continuously balancing speed and sustainability. Startup life is exciting! Priorities can change quickly, and in this role I could see every day how my work and execution could help our customers, and ultimately make the business a success. In this role I learned to accept and also pay down tech debt, saw the value of investing time into a good development environment, and became comfortable being responsible for larger and less-well-defined tasks.
Capabuild is a web and mobile platform for disaster recovery companies. This role was an unexpectedly great fit for some technologies I used in side projects: THREE.js and SVG. Here I worked with other experienced engineers building new things and continuously balancing speed and sustainability. Startup life is exciting! Priorities can change quickly, and in this role I could see every day how my work and execution could help our customers, and ultimately make the business a success. In this role I learned to accept and also pay down tech debt, and experienced the value of investing time into a good development environment.
---