hckrnws

I think two things can be true simultaneously:

1. LLMs are a new technology and it's hard to put the genie back in the bottle with that. It's difficult to imagine a future where they don't continue to exist in some form, with all the timesaving benefits and social issues that come with them.

2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them, the majority of consumer usage is at the free tier, the industry is seeing the first signs of pulling back investments, and model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner), and several that seemed poised to displace both old tech and labor but have settled into specific use cases (the microwave oven). Given the lack of a sufficiently profitable business model, it feels as likely as not that LLMs settle somewhere a little less remarkable, and hopefully less annoying, than today's almost universally disliked attempts to cram it everywhere.

> There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return (the supersonic jetliner)

I think this is a great analogy, not just to the current state of AI, but maybe even computers and the internet in general.

Supersonic transports must've seemed amazing, inevitable, and maybe even obvious to anyone alive at the time of their debut. But hiding under that amazing tech was a whole host of problems that were just not solvable with the technology of the era, let alone a profitable business model. I wonder if computers and the internet are following a similar trajectory to aerospace. Maybe we've basically peaked, and all that's left are optimizations around cost, efficiency, distribution, or convenience.

If you time traveled back to the 1970s and talked to most adults, they would have witnessed aerospace go from loud, smelly, and dangerous prop planes to the 707, 747 and Concorde. They would've witnessed the moon landings and were seeing the development of the Space Shuttle. I bet they would call you crazy if you told this person that 50 years later, in 2025, there would be no more supersonic commercial airliners, commercial aviation would basically look the same except more annoying, and also that we haven't been back to the moon. In the previous 50 years we went from the Wright Brothers to the 707! So maybe in 2075 we'll all be watching documentaries about LLMs (maybe even on our phones or laptops that look basically the same), and reminiscing about the mid-2020s and wondering why what seemed to be such a promising technology disappeared almost entirely.

I think this is both right and wrong. There was a good book that came out probably 15 years ago about how technology never stops in aggregate, but individual technologies tend to grow quickly and then stall. Airplane jets were one example in the book. The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today.

A better example, also in the book, are skyscrapers. Each year they grew and new ones were taller than the ones last year. The ability to build them and traverse them increased each year with new technologies to support it. There wasn't a general consensus around issues that would stop growth (except at more extremes like air pressure). But the growth did stop. No one even has expectations of taller skyscrapers any more.

LLMs may fail to advance, but not because of any consensus reason that exists today. And it maybe that they serve their purpose to build something on top of them which ends up being far more revolutionary than LLMs. This is more like the path of electricity -- electricity in itself isn't that exciting nowadays, but almost every piece of technology built uses it.

I fundamentally find it odd that people seem so against AI. I get the potential dystopian future, which I also don't want. But the more mundane annoyance seems odd to me.

> even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today

I think they pretty strongly do

The solution seems to be "just lower your standards for acceptable margin of error to whatever the LLM is capable of producing" which should be concerning and absolutely unacceptable to anyone calling themselves an Engineer

> absolutely unacceptable to anyone calling themselves an Engineer

Isn’t that exactly what engineers do? Even very strong bridges aren’t designed to survive every possible eventuality.

No

I'm talking about engineering a bridge for 50 cars that collapses at 51, not engineering a bridge for 500 cars that is only expected to get 50

Engineering does require tradeoffs of course. But that's not what the minimum possible quality is

That's what a "margin of error" is. The margin of error of a bridge is predictable thanks to well-established techniques of physical analysis.

An LLM system, on the other hand, can fail because you moved some punctuation around.

> The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today.

The fundamental problem has already been mentioned: Nobody can figure out how to SELL it. Because few people are buying it.

It's useful for aggregation and summarization of large amounts of text, but it's not trustworthy. A good summary decreases noise and amplifies signal. LLMs don't do that. Without the capability to validate the output, it's not really generating output of lasting value. It's just a slightly better search engine.

It feels like, fundamentally, the primary invention here is teaching computers that it's okay to be wrong as long as you're convincing. That's very useful for propaganda or less savory aspects of business, but it's less useful for actual communication.

> Nobody can figure out how to SELL it. Because few people are buying it.

Just picking one company who basically just does AI, OpenAI. They reported it has 20 million PAID subscribers to ChatGPT. With revenue projected above $12b dollars (https://www.theverge.com/openai/640894/chatgpt-has-hit-20-mi...).

I think what you meant to say is that costs are high so they can't generate large profits. but saying that they can't figure out how to sell it seems absurd. Is it Netflix level of subscribers, no. But there can't be more than a couple of hundred products that have that type of subscription reach.

Ok but isn’t 20 million subscribers out of what, 800 million or 1 billion monthly users or whatever they’re claiming, an absolutely abysmal conversion rate? Especially given that the industry and media have been proclaiming this as somewhere between the internet and the industrial revolution in terms of impact and advancement? Why can they not get more than 3% of users to convert to paying subscribers for such a supposedly world changing technology, even with a massive subsidy?

As another commenter notes, because you get access to a lot of functionality for free. And other providers are also providing free alternatives. The ratio for their free/paid tier is about the same as YouTube's. And like YouTube, it's not that YouTube isn't providing great value, but rather that most people get what they need out of the free tier.

The better question is what if all LLM services stopped providing for free at all -- how many paid users would there then be?

Because they give too much of it away for free? Most casual use fits into the very generous free tier.

Ok so the argument is that all the model builders either suck at business or they are purposefully choosing to lose billions of dollars?

They are purposely losing billions, this is a growth phase where all of the big AI companies are racing to grow their userbase, later down the line they will monetize that captured userbase.

This is very similar to Uber which lost money for 14 years before becoming profitable, but with significantly more upside.

Investors see the growth, user stickiness and potential for the tech; and are throwing money to burn to be part of the winning team, which will turn on the money switch on that userbase down the line.

The biggest companies and investors in the planet aren't all bad at business.

There are sound math reasons for skyscrapers topping out, mostly due to elevator capacity and the inability to effectively get people in and out of the floorspace as you go past a few hundred ft. There's no construction engineering reason you can't go taller - the Burj Khalifa, for example, is three times taller than a typical Western major city skyscraper - it just doesn't make economic sense unless you're a newly rich nation looking to prove a point.

>I think this is both right and wrong. There was a good book that came out probably 15 years ago about how technology never stops in aggregate, but individual technologies tend to grow quickly and then stall. Airplane jets were one example in the book. The reason why I partially note this as wrong is that even in the 70s people recognized that supersonic travel had real concrete issues with no solution in sight. I don't think LLMs share that characteristic today.

I don't see any solution to hallucinations, nor do I see any solution in sight. I think that could count as a concrete issue that would stop them.

Yeah, and with LLMs the thing I can't shake, however, is that this time it's pretty strongly (maybe parasitically) latched onto the aggregate progress of Moore's law. Few other technologies have enjoyed such relatively unfettered exponential improvement. It's like if skyscraper materials double in strength every n years, and their elevators approach teleportation speed, the water pumps get twice as powerful, etc., which would change the economics vs the reality that most of the physical world doesn't improve that fast.

Was the problem that supersonic flight was expensive and the amount of customers willing to pay the price was even lower than the number of customers that could even if they wanted to?

From what I had read in passing and remember.

  - They were loud (sonic booms were nasty).

  - They were expensive to maintain and operate. Guzzlers. (Britain and France clung to them as a matter of pride/ego)

  - They were narrow and uncomfortable. I have seen videos where there is space only for one stewardess to walk. I had been inside of one in Seattle museum. Very cramped.

  - As you mentioned, ticket cost was high.

  - I suspect people traveled in these mostly for bragging rights.

You made this point in passing, but it's so relevant to LLMs I wanted to highlight it: The development and operational cost was heavily subsidized by the British and French governments, because having an SST was a point of national prestige.

From a system optimisation perspective, SSTs solved the wrong problem.

Want to save people time flying? Solve the grotesque inefficiency pit that is airport transit and check-in.

Like, I'm sorry, STILL no high speed, direct to terminal rail at JFK, LAX and a dozen other major international airports? And that's before we get to the absolute joke of "border security" and luggage check-in.

Sure, supersonic afterburning engines are dope. But it's like some 10GHz single-core CPU that pulls 1.2kW out of the wall. Like it or not, an iPhone 16 delivers far more compute utility in far more scenarios.

that's not a technology problem, many airports are super efficient, e.g. Singapore. Public transport in the US is held back by other forces.

It makes it all the dumber that we have the tech and still can't manage to implement the solution.

Like an org with crappy management and team structure shipping bloated, buggy code even though they've the budget to hire great engineers and the problems they're solving are largely known and well-trodden.

They don't optimize for our convenience, they optimize for their profit.

Such airports removing moving walkways because they reduce profits in stores…

https://nypost.com/2025/05/27/lifestyle/airports-are-removin...

SST came and went in an era when none of that security theater existed to begin with

It did for international, maybe not at the dawn of SSTs but after a string of hijackings in the 70s/80s they brought it in. Not for US internal flights, it's true.

Oh no, LLMs won't disappear but they will be a lot less loud.

Progress is often an S shaped curve and we are nearing saturation.

The problem with supersonic commercial jets was mainly one of marketing/politics. The so called "sonic boom" problem was vastly overhyped, as anyone who lives near an air force base can tell you.

The conspiracy theorist tells me the American aerospace manufacturers at the time (Boening, McDonnell-Douglas, etc.), did everything they could to kill the Concorde. With limited flyable routes (NYC and DC to Paris and London I think were the only ones), the financials didn't make sense. If overland routes were available, especially opening up LA, San Francisco and Chicago, it might have been a different story.

> “most people agree that the output is trite and unpleasant to consume”

That is a such a wild claim. People like the output of LLMs so much that ChatGPT is the fastest growing app ever. It and other AI apps like Perplexity are now beginning to challenge Google’s search dominance.

Sure, probably not a lot of people would go out and buy a novel or collection of poetry written by ChatGPT. But that doesn’t mean the output is unpleasant to consume. It pretty undeniably produces clear and readable summaries and explanations.

> People like the output of LLMs so much that ChatGPT is the fastest growing app ever

While people seem to love the output of their own queries they seem to hate the output of other people's queries, so maybe what people actually love is to interact with chatbots.

If people loved LLM outputs in general then Google, OpenAI and Anthropic would be in the business of producing and selling content.

Google does put AI output at the top of every search now, and sometimes it's helpful and sometimes it's crap. They have been trying since long before LLMs to not just provide the links for a search but also the content.

Google used to be interested in making sure you clicked either the paid link or the top link in the results, but for a few years now they'd prefer that a user doesn't even click a link after a search (at least to a non-Google site)

It made me switch away from google. The push I needed

Low effort Youtube shorts with AI voice annoy the crap out of me.

After all this hype, they still can't do text to speech properly. Pause at the wrong part of the sentence all the time.

I think the thing people hate about that is the lack of effort and attention to detail. It’s an incredible enabler for laziness if misused.

If somebody writes a design or a report, you expect that they’ve put in the time and effort to make sure it is correct and well thought out.

If you then find the person actually just had ChatGPT generate it and didn’t put any effort into editing it and checking for correctness, then that is very infuriating.

They are essentially farming out the process of creating the document to AI and farming out the process of reviewing it to their colleagues. So what is their job then, exactly?

These are tools, not a replacement for human thought and work. Maybe someday we can just have ChatGPT serve as an engineer or a lawyer, but certainly not today.

> While people seem to love the output of their own queries they seem to hate the output of other people's queries

Listening or trying to read other peoples chats with these things is like listening to somebody describe a dream. It’s just not that interesting most of the time. It’s remarkable for the person experiencing it but it is deeply personal.

If I cared about the output from other people's queries then wouldn't they be my queries? I don't care about ChatGPTs response to your queries is because I don't care about your queries. I don't care if they came from ChatGPT or the world's foremost expert in whatever your query was about.

> That is a such a wild claim. People like the output of LLMs so much that ChatGPT is the fastest growing app ever.

The people using ChatGPT like its output enough when they're the ones reading it.

The people reading ChatGPT output that other people asked for generally don't like it. Especially if it's not disclosed up front.

Had someone put up a project plan for something that was not disclosed as LLM assisted output.

While technically correct it came to the wrong conclusions about the best path forward and inevitably hamstrung the project.

I only discovered this later when attempting to fix the mess and having my own chat with an LLM and getting mysteriously similar responses.

The problem was that the assumptions made when asking the LLM were incorrect.

LLMs do not think independently and do not have the ability to challenge your assumptions or think laterally. (yet, possibly ever, one that does may be a different thing).

Unfortunately, this still makes them as good as or better than a very large portion of the population.

I get pissed off not because of the new technology or the use of the LLM, but the lack of understanding of the technology and the laziness with which many choose to deliver the results of these services.

I am more often mad at the person for not doing their job than I am at the use of a model, the model merely makes it easier to hide the lack of competence.

> LLMs do not think

Yep.

More seriously, you described a great example of one of the challenges we haven't addressed. LLM output masquerades as thoughtful work products and wastes people's time (or worse tanks a project, hurts people, etc).

Now my job reviewing work is even harder because bad work has fewer warning signs to pick up on. Ugh.

I hope that your workplace developed a policy around LLM use that addressed the incident described. Unfortunately I think most places probably just ignore stuff like this in the faux scramble to "not be left behind".

It's even worse than you suggest, for the following reason. The rare employee that cares enough to read through an entire report is more likely to encounter false information which they will take as fact (not knowing that LLM produced the report, or unaware that LLMs produce garbage). The lazy employees will be unaffected.

> LLMs do not think independently and do not have the ability to challenge your assumptions

It IS possible for a LLM to challenge your assumptions, as its training material may include critical thinking on many subjects.

The helpful assistant, being almost by definition a sycophant, cannot.

Strong agree. If you simply ask an LLM to challenge your thinking, spot weaknesses in your argument, or what else you might consider, it can do a great job.

This is literally my favorite way to use it. Here’s an idea, tell me why it’s wrong.

> do not have the ability to challenge your assumptions or think laterally.

Particularly on the challenging your assumptions part is where I think LLMs fail currently, though I won't pretend to know enough about how to even resolve that; but right now, I can put whatever nonsense I want into ChatGPT and it will happily go along telling me what a great idea that is. Even on the remote chance it does hint that I'm wrong, you can just prompt it into submission.

None of the for-profit AI companies are going to start letting their models tell users they're wrong out of fear of losing users (people generally don't like to be held accountable) but ironically I think it's critically important that LLMs start doing exactly that. But like you said, the LLM can't think so how can it determine what's incorrect or not, let alone if something is a bad idea or not.

Interesting problem space, for sure, but unleashing these tools to the masses with their current capabilities I think has done, and is going to continue to do more harm than good.

This is why once you are using to using them, you start asking them for there the plan goes wrong. They won't tell you off the bat, whuch can be frustrating, but they are really good at challenging your assumptions, if you ask them to do so.

They are good at telling you what else you should be asking, if you ask them to do so.

People don't use the tools effectively and then think that the tool can't be used effectively...

Which isn't true, you just have to know how the tool acts.

I'm no expert, but the most frequent recommendations I hear to address this are:

a) tell it that it's wrong and to give you the correct information.

b) use some magical incantation system prompt that will produce a more critical interlocutor.

The first requires knowing enough about the topic to know the chatbot is full of shit, which dramatically limits the utility of an information retrieval tool. The second assumes that the magical incantation correctly and completely does what you think it does, which is not even close to guaranteed. Both assume it even has the correct information and is capable of communicating it to you. While attempting to use various models to help modify code written in a less-popular language with a poorly-documented API, I learned how much time that can waste the hard way.

If your use case is trivial, or you're using it as a sounding board with a topic you're familiar with as you might with, say, a dunning-kruger-prone intern, then great. I haven't found a situation in which I find either of those use cases compelling.

Especially if it's not disclosed up front, and especially when it supplants higher-value content. I've been shocked how little time it's taken for AI slop SEO optimized blogs to overtake the articles written by genuine human experts, especially in niche product reviews and technical discussions.

However, whether or not people like it is almost irrelevant. The thing that matters is not whether economics likes it.

At least so far, it looks like economics absolutely loves LLMs: Why hire expensive human customer support when you can just offload 90% of the work to a computer? Why pay expensive journalists when you can just have the AI summarize it? Why hire expensive technical writers to document your code when you can just give it to the AI and check the regulatory box with docs that are good enough?

Eventually the economics will correct themselves once people yet again learn the old "you get what you pay for" lesson (or the more modern FAFO lesson)

I'm not really countering that ChatGPT is popular, it certainly is, but it's also sort of like "fastest growing tire brand" that came along with the adoption of vehicles. The amount of smartphone users is also growing at the fastest rate ever so whatever the new most popular app is has a good chance of being the fastest growing app ever.

No… dude… it’s a new household name. We haven’t had those in software for a long time, maybe since TikTok and Fortnite.

Lots of things had household recognition. Do you fondly remember the Snuggie? The question is whether it'll be durable. The lack of network effects is one reason to be skeptical.

Maybe he's referencing how people don't like when other humans post LLM responses in the comments.

"Here's what chatGPT said about..."

I don't like that, either.

I love the LLM for answering my own questions, though.

"Here's what chatGPT said about..." Is the new lmgtfy

lmgtfy was (from what I saw) always used as a snarky way to tell someone to do a little work on their own before asking someone else to do it for them.

I have seen people use "here's what chatGPT" said almost exclusively unironically, as if anyone else wants humans behaving like agents for chatbots in the middle of other people's discussion threads. That is to say, they offer no opinion or critical thought of their own, they just jump into a conversation with a wall of text.

Yeah I don't even read those. If someone can't be bothered to communicate their own thoughts in their own words, I have little belief that they are adding anything worth reading to the conversation.

Why communicate your own thoughts when ChatGPT can give you the Correct Answer? Saves everybody time and effort, right? I guess that’s the mental model of many people. That, or they’re just excited to be able to participate (in their eyes) productively in a conversation.

If I want the "correct answer" I'll research it, maybe even ask ChatGPT. If I'm having a conversation I'm interesed in what the other participants think.

If I don't know something, I'll say I don't know, and maybe learn something by trying to understand it. If I just pretend I know by pasting in what ChatGPT says, I'm not only a fraud but also lazy.

> That is a such a wild claim.

Some people who hate LLMs are absolutely convinced everyone else hates them. I've talked with a few of them.

I think it's a form of filter bubble.

This isn't some niche outcry: https://www.forbes.com/sites/bernardmarr/2024/03/19/is-the-p...

And that was 18 months ago.

Yes, believe it or not, people eventually wake up and realize slop is slop. But like everything else with LLM development, tech is trying to brute force it on people anyway.

Yup, any day now people will suddenly realize that LLMs suck and you were right all along. Any day now..

You posted an article about investors trust in AI companies to deliver and societies strong distrust of large corporations.

You article isn’t making the point you seem to think it is.

> AI apps like Perplexity are now beginning to challenge Google’s search dominance

Now that is a wild claim. ChatGPT might be challenging Google's dominance, but Perplexity is nothing.

It’s not a wild claim, though maybe your interpretation is wild.

I never said Perplexity individually is challenging Google, but rather as part of a group of apps including ChatGPT, which you conveniently left out of your quote.

> That is a such a wild claim. People like the output of LLMs so much that ChatGPT is the fastest growing app ever.

And this kind of meaningless factoid was immediately usurped by the Threads app release, which IMO is kind of a pointless app. Maybe let's find a more meaningful metric before saying someone else's claim is wild.

Asking your Instagram Users to hop on to your ready made TikTok Clone is hardly in the same sphere as spinning up that much users from nothing.

And while Threads growth and usage stalled, ChatGPT is very much still growing and has *far* more monthly visits than threads.

There's really nothing meaningless about ChatGPT being the 5th most visited site on the planet, not even 3 years after release. Threads doesn't make the top 50.

At some point, Groupon was the fastest growing company ever.

People "like" or people "suffice" with the output? This "rise of whatever" as one blog put it gives me feelings that people are instead lowering their standards and cutting corners. Letting them cut through to stuff they actually want to do.

> People like the output of LLMs so much that ChatGPT is the fastest growing app ever

And how much of that is free usage, like the parent said? Even when users are paying, ChatGPT's costs are larger than their revenue.

...I do wonder what percent of ChatGPT usage is just students cheating on their homework, though.

Neal Stephenson has a recent post that covers some of this. Also links to teachers talking about many students just putting all their work into chatgpt and turning it in.

https://nealstephenson.substack.com/p/emerson-ai-and-the-for...

He links to Reddit, a site where most people are aggressively against AI. So, not necessarily a representative slice of reality.

He links to a post about a teacher’s expertise with students using AI. The fact that it’s on Reddit is irrelevant.

If you're going to champion something that comes from a place of extreme political bias, you could at least acknowledge it.

This is a baffling response. The politics are completely irrelevant to this topic. Pretty much every American is distrustful of big tech and is completely unaware of what the current administration has conceded to AI companies, with larger scandals taking the spotlight, so there hasn't been a chance for one party or the other to rally around a talking point with AI.

People don't like AI because its impact on the internet is filling it with garbage, not because of tribalism.

>This is a baffling response.

Likewise.

95+% of the time I see a response like this, it's from one particular side of the political aisle. You know the one. Politics has everything to do with this.

>what the current administration has conceded to AI companies

lol, I unironically think that they're not lax enough when it comes to AI.

Based on your response and logic - no dem should read stuff written by repub voters, or if they do read it, dismiss their account because it cannot be … what?

Not sure how we get to dismissing the teacher subreddit, to be honest.

Look, another one! Twist it however you want, I'm not going to accept the idea that far-lefty Reddit is some impartial representation of what teaching is or what the average person thinks of AI.

Why? So you could discard it faster?

Read things from people that you disagree with.

Because I'm not going to play a game where the other side gets to ignore the rules.

I’d like to see a statistically sound source for that claim. Given how many non-nerds there are on Reddit these days, it’s unlikely that there’s any particular strong bias in any direction compared to any similar demographic.

Given recent studies, that does seem to reflect reality. Trust in AI has been waning for 2 years now.

By what relevant metric?

The userbase has grown by an order of magnitude over the past few years. Models have gotten noticeably smarter and see more use across a variety of fields and contexts.

> Models have gotten noticeably smarter and see more use across a variety of fields and contexts.

Is that really true? The papers I've read seem to indicate the hallucination rate is getting higher.

Models from a few years ago are comparatively dumb. Basically useless when it comes to performing tasks you'd give to o3 or Gemini 2.5 Pro. Even smaller reasoning models can do things that would've been impossible in 2023.

I would pay $5000 to never have to read another LLM-authored piece of text ever again.

I'm confused with your second point. LLM companies are not making any money from current models? Openai generates 10b USD ARR and has 100M MAUs. Yes they are running at a loss right now but that's because they are racing to improve models. If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their massive user base you think they don't have a successful business model? People use this tools daily, this is inevitable.

They might generate 10b ARR, but they lose a lot more than that. Their paid users are a fraction of the free riders.

https://www.wheresyoured.at/openai-is-a-systemic-risk-to-the...

This echoes a lot of the rhetoric around "but how will facebook/twitter/etc make money?" back in the mid 2000s. LLMs might shake out differently from the social web, but I don't think that speculating about the flexibility of demand curves is a particularly useful exercise in an industry where the marginal cost of inference capacity is measured in microcents per token. Plus, the question at hand is "will LLMs be relevant?" and not "will LLMs be massively profitable to model providers?"

Social networks finding profitability via advertising is what created the entire problem space of social media - the algorithmic timelines, the gaming, the dopamine circus, the depression, everything negative that’s come from social media has come from the revenue model, so yes, I think it’s worth being concerned about how LLMs make money, not because I’m worried they won’t, because I’m worried they Will.

I think this can't be understated. It also destroyed search. I listened to a podcast a few years ago with an early googler who talked about this very precipice in early google days. They did a lot of testing, and a lot of modeling of people's valuation of search. They figured that the average person got something like $50/yr of value out of search (I can't remember the exact number, I hope I'm not off by an order of magnitude). And that was the most they could ever realistically charge. Meanwhile, advertising for just Q4 was like 10 times the value. It meant that they knew that advertising on the platform was inevitable. They also acknowledged that it would lead to the very problem that Brin and Page wrote about in their seminal paper on search.

I see LLMs inevitably leading to the same place. There will undoubtedly be advertising baked into the models. It is too strong a financial incentive. I can only hope that an open source alternative will at least allow for a hobbled version to consume.

edit: I think this was the podcast https://freakonomics.com/podcast/is-google-getting-worse/

This is an interesting take - is my "attention" really worth several thousand a year? In that my purchasing decisions being influenced by advertising to that degree that someone is literally paying someone else for my attention ...

I wonder if instead, could I sell my "attention" instead of others profitting of it?

Yes, but your attention rapidly loses value the more that your subsequent behavior misaligns with the buyer’s desires. In other words, the ability to target unsuspecting, idle minds far exceeds the value of a willing and conscious attention seller.

oh, I 100% agree with this. The way the social web was monetized is the root of a lot of evil. With AI, we have an opportunity to learn from the past. I think a lesson here is "don't wait to think critically about the societal consequences of the next Big Tech Thing's business model because you have doubts about its profitability or unit economics."

Social networks will have all of those effects without any effort by the platform itself because the person with more followers has more influence so the people on the platform will do all they can to get more.

I'm not excusing the platforms for bad algorithms. Rather, I believe it's naive to think that, but for the behavior of the platform itself that things would be great and rosy.

No, they won't. The fact that nearly every person in the world can mass communicate to nearly every other person in the world is the core issue. It is not platform design.

> This echoes a lot of the rhetoric around "but how will facebook/twitter/etc make money?" back in the mid 2000s.

The difference is that Facebook costs virtually nothing to run, at least on a per-user basis. (Sure, if you have a billion users, all of those individual rounding errors still add up somewhat.)

By contrast, if you're spending lots of money per user... well look at what happened to MoviePass!

The counterexample here might be Youtube; when it launched, streaming video was really expensive! It still is expensive too, but clearly Google has figured out the economics.

You're either overestimating the cost of inference or underestimating the cost of running a service like Facebook at that scale. Meta's cost of revenue (i.e. just running the service, not R&D, not marketing, not admin, none of that) was about $30B/year in 2024. In the leaked OpenAI financials from last year, their 2024 inference costs were 1/10th of that.

But their research costs are extremely high, and without a network effect that revenue is only safe until a better competitor emerges.

> This echoes a lot of the rhetoric around "but how will facebook/twitter/etc make money?"

The answer was, and will be ads (talk about inevitability!)

Can you imagine how miserable interacting with ad-funded models will be? Not just because of the ads they spew, but also the penny-pinching on training and inference budgets, with an eye focused solely on profitability. That is what the the future holds: consolidations, little competition, and models that do the bare-minimum, trained and operated by profit-maximizing misers, and not the unlimited intelligence AGI dream they sell.

It won’t be ads. Social media target consumers, so advertising is dominant. We all love free services and don’t mind some attraction.

AI on the other hand target businesses and consumers alike. A bank using LLM won’t get ads. Using LLM will be cost of doing business. Do you know what they means to consumers? Price for ChatGPT will go down.

>AI on the other hand target businesses and consumers alike.

Okay. So AI will be using ads for consumers and make deals with the billionaires. If window 11/12 still puts ads in what is a paid premium product, I see no optimism in thinking that a "free" chatbot will not also resort to it. Not as long as the people up top only see dollar signs and not long term longevity.

>Price for ChatGPT will go down.

Price for ChatGPT in reality, is going up in the meanwhile. This is like hoping grocery prices come down as inflation lessens. This never happens, you can only hope to be compensated more to make up for inflation.

Has any SAAS product ever reduced their subscription cost?

I see a real window this time to sell your soul.

The thing about facebook/twitter/etc was that everyone knew how they achieve lock-in and build a moat (network effect), but the question was around where to source revenue.

With LLMs, we know what the revenue source is (subscription prices and ads), but the question is about the lock-in. Once each of the AI companies stops building new iterations and just offers a consistent product, how long until someone else builds the same product but charges less for it?

What people often miss is that building the LLM is actually the easy part. The hard part is getting sufficient data on which to train the LLM, which is why most companies just put ethics aside and steal and pirate as much as they can before any regulations cuts them off (if any regulations ever even do). But that same approach means that anyone else can build an LLM and train on that data, and pricing becomes a race to the bottom, if open source models don't cut them out completely.

ChatGPT also makes money via affiliate links. If you ask ChatGPT something like "what is the best airline approved cabin luggage you can buy?" you get affiliate links to Amazon and other sites. I use ChatGPT most of the time before I buy anything these days… From personal experience (I operated an app financed by affiliate links). I can tell you that this for sure generates a lot of money. My app was relatively tiny and I only got about 1% of the money I generated but that app pulled in about $50k per month.

Buying better things is one of my main use cases for GPT.

Makes you wonder whether the affiliate links are actual, valid affiliate links or just hallucinations from affiliate links it's come across in the wild

It clearly is a 100% custom UI logic implemented by OpenAI… They render the products in carrousels… They probably get a list of product and brand names from the LLM (for certain requests/responses) and render that in a separate UI after getting those affiliate links for those products… its not hard to do. Just slap on your affiliate ID to the links you found and you are done.

Yep. Remember when Amazon could never make money and we kept trying to explain they were reinvesting their earnings into R&D and nobody believed it? All the rhetoric went from "Amazon can't be profitable" to "Amazon is a monopoly" practically overnight. It's like people don't understand the explore/exploit strategy trade-off.

AWS is certainly super profitable, if the ecommerce business was standalone, would it really be such a cash-gusher?

Amazon is successful because of the insanely broad set of investments they've made - many of them compound well in a way that supports their primary business. Amazon Music isn't successful, but it makes Kindle tablets more successful. This is in contrast to Google, which makes money on ads, and everything else is a side quest. Amazon has side quests, but also has many more initiatives that create a cohesive whole from the business side.

So while I understand how it looks from a financial perspective, I think that perspective is distorted in terms of what causes those outcomes. Many of the unprofitable aspects directly support the profitable ones. Not always, though.

> LLMs might shake out differently from the social web, but I don't think that speculating about the flexibility of demand curves is a particularly useful exercise in an industry where the marginal cost of inference capacity is measured in microcents per token

That we might come to companies saying "it's not worth continuing research or training new models" seems to reinforce the OP's point, not contradict it.

The point I'm making is that, even in the extreme case where we cease all additional R&D on LLMs, what has been developed up until now has a great deal of utility and transformative power, and that utility can be delivered at scale for cheap. So, even if LLMs don't become an economic boon for the companies that enable them, the transformative effect they have and will continue to have on society is inevitable.

Edit: I believe that "LLMs transforming society is inevitable" is a much more defensible assertion than any assertion about the nature of that transformation and the resulting economic winners and losers.

>what has been developed up until now has a great deal of utility and transformative power

I think we'd be more screwed than VR if development ceased today. They are little more than toys right now who's most successsful outings are grifts, and the the most useful tools are simply aiding existing tooling (auto-correct). It is not really "intelligence" as of now.

>I believe that "LLMs transforming society is inevitable" is a much more defensible assertion

Sure. But into what? We can't just talk about change for change's sake. Look at the US in 2025 with that mentality.

No one ever doubted that Facebook would make money. It was profitable early on, never lost that much money and was definitely profitable by the time it went public.

Twitter has never been consistently profitable.

ChatGPT also has higher marginal costs than any of the software only tech companies did previously.

No one ever doubted that Facebook would make money. It was profitable early on, never lost that much money and was definitely profitable by the time it went public.

Twitter has never been consistently profitable

Well, given the answers to the former: maybe we should stop now before we end up selling even more of our data off to technocrats. Or worse, your chatbot shilling to you between prompts.

And yes these are still businesses. If they can't find profitability they will drop it like it's hot. i.e. we hit another bubble burst that tech is known to do every decade or 2. There's no free money anymore to carry them anymore, so perfect time to burst.

what I struggle with is that the top 10 providers of LLMs all have identical* products. The services have amazing capabilities, but no real moats.

The social media applications have strong network effects, this drives a lot of their profitability.

* sure, there are differences, see the benchmarks, but from a consumer perspective, there's no meaningful differentiation

The point is that if they’re not profitable they won’t be relevant since they’re so expensive to run.

And there was never any question as to how social media would make money, everyone knew it would be ads. LLMs can’t do ads without compromising the product.

You’re not thinking evil enough. LLMs have the potential to be much more insidious about whatever it is they are shilling. Our dystopian future will feature plausibly deniable priming.

Well, they haven't really tried yet.

The Meta app Threads had no ads for the first year, and it was wonderful. Now it does, and its attractiveness was only reduced by 1% at most. Meta is really good at knowing the balance for how much to degrade UX by having monetization. And the amount they put in is hyper profitable.

So let's see Gemini and GPT with 1% of response content being sponsored. I doubt we'll see a user exodus and if that's enough to sustain the business, we're all good.

> LLMs can’t do ads without compromising the product.

It depends on what you mean by "compromise" here but they sure can inject ads.. like make the user wait 5 seconds, show an ad, then reply..

They can delay the response times and promote "premium" plans, etc

Lots of ways to monetize, I suppose the question is: will users tolerate it?

Based on what I've seen, the answer is yes, people will tolerate anything as long as it's "free".

I can run an LLM on my RTX3090 that is at least as useful to me in my daily life as an AAA game that would otherwise justify the cost of the hardware. This is today, which I suspect is in the upper part of the Kuznets curve for AI inference tech. I don't see a future where LLMs are too expensive to run (at least for some subset of valuable use cases) as likely.

I don't even get where this argument comes from. Pretraining is expensive, yes, but both LoRAs in diffusion models and finetunes of transformers show us that this is not the be-all, end-all; there's plenty of work being done on extensively tuning base models for cheap.

But inference? Inference is dirt cheap and keeps getting cheaper. You can run models lagging 6-12 years on consumer hardware, and by this I don't mean absolutely top-shelf specs, but more of "oh cool, turns out the {upper-range gaming GPU/Apple Silicon machine} I bought a year ago is actually great at running local {image generation/LLM inference}!" level. This is not to say you'll be able to run o3 or Opus 4 on a laptop next year - larger and more powerful models obviously require more hardware resources. But this should anchor expectations a bit.

We're measuring inference costs in multiples of gaming GPUs, so it's not an impending ecological disaster as some would like the world to believe - especially after accounting for data centers being significantly more efficient at this, with specialized hardware, near-100% utilization, countless of optimization hacks (including some underhanded ones).

> LLMs can’t do ads without compromising the product.

Spoiler: they are still going to do ads, their hand will be forced.

Sooner or later, investors are going to demand returns on the massive investments, and turn off the money faucet. There'll be consolidation, wind-downs and ads everywhere.

I was chatting with Gemini about vacation ideas and could absolutely picture a world where if it lists some hotels I might like, the businesses that bought some LLM ad space could easily show up more often than others.

Social and search both compromised the product for ad revenue.

To be fair, ads always compromise the product.

That's fixable, a gradual adjusting of the free tier will happen soon enough once they stop pumping money into it. Part of this is also a war of attrition though, who has the most money to keep a free tier the longest and attract the most people. Very familiar strategy for companies trying to gain market share.

That assumes that everyone is willing to pay for it. I don't think that's an assumption that will be true.

Consider the general research - in all, it doesn't eliminate people, but let's say it shakes out to speeding up developers 10% over all tasks. (That includes creating tickets, writing documentation, unblocking bugs, writing scripts, building proof of concepts, and more rote refactoring, but does not solve the harder problems or stop us from doing the hard work of software engineering that doesn't involve lines of code.)

That means that it's worth up to 10% of a developer's salary as a tool. And more importantly, smaller teams go faster, so it might be worth that full 10%.

Now, assume other domains end up similar - some less, some more. So, that's a large TAM.

Those that aren't willing to pay for it directly, can still use it for free, but will just have to tolerate product placement.

It very much does not assume that, only that some fraction will have become accustomed to using it to the point of not giving it up. In fact, they could probably remain profitable without a single new customer, given the number of subscribers they already have.

Absolutely, free-tier AI won’t stay "free" forever. It’s only a matter of time before advertisers start paying to have their products woven into your AI conversations. It’ll creep in quietly—maybe a helpful brand suggestion, a recommended product "just for you," or a well-timed promo in a tangential conversation. Soon enough though, you’ll wonder if your LLM genuinely likes that brand of shoes, or if it's just doing its job.

But hey, why not get ahead of the curve? With BrightlyAI™, you get powerful conversational intelligence - always on, always free. Whether you're searching for new gear, planning your next trip, or just craving dinner ideas, BrightlyAI™ brings you personalized suggestions from our curated partners—so you save time, money, and effort.

Enjoy smarter conversations, seamless offers, and a world of possibilities—powered by BrightlyAI™: "Illuminate your day. Conversation, curated."

I agree, its easily fixable by injecting ads into the responses for the free tier and probably eventually even the lower paid tiers to some extent

Literally nobody would talk to a robot that spits back ads at them

I predict this comment to enter the Dropbox/iPod hall of shame of discussion forum skeptics.

Hundreds of millions of people watch TV and listen to Radio that is at least 30% ad content per hour.

That's pretty much what search engines are nowadays

You still have faith in society after decades of ads being spit at them.

Competition is almost guaranteed to drive price close to cost of delivery especially if they can't pay trump to ban open source, particularly chinese. With no ability to play the thiel monopoly playbook, their investors would never make their money back if not for government capture and sweet sweet taxpayer military contracts.

> especially if they can't pay trump to ban open source?

Huh? Do you mean for official government use?

Then cut off the free riders. Problem solved overnight.

Are you saying they'd be profitable if they didn't pour all the winnings into research?

From where I'm standing, the models are useful as is. If Claude stopped improving today, I would still find use for it. Well worth 4 figures a year IMO.

They'd be profitable if they showed ads to their free tier users. They wouldn't even need to be particularly competent at targeting or aggressive with the amount of ads they show, they'd be profitable with 1/10th the ARPU of Meta or Google.

And they would not be incompetent at targeting. If they were to use the chat history for targeting, they might have the most valuable ad targeting data sets ever built.

Bolting banner ads onto a technology that can organically weave any concept into a trusted conversation would be incredibly crude.

True - but if you erode that trust then your users may go elsewhere. If you keep the ads visually separated, there's a respected boundary & users may accept it.

There will be a respected boundary for a time, then as advertisers find its more effective the boundaries will start to disappear

google did it. LLms are the new google search. It'll happen sooner or later.

Yes, but for a while google was head and shoulders above the competition. It also poured a ton of money into building non-search functionality (email, maps, etc.). And had a highly visible and, for a while, internally respected "don't be evil" corporate motto.

All of which made it much less likely that users would bolt in response to each real monetization step. This is very different to the current situation, where we have a shifting landscape with several AI companies, each with its strengths. Things can change, but it takes time for 1-2 leaders to consolidate and for the competition to die off. My 2c.

how is it "trusted" when it just makes things up

That's a great question to ask the people who seem to trust them implicitly.

They aren't trusted in a vacuum. They're trusted when grounded in sources and their claims can be traced to sources. And more specifically, they're trusted to accurately represent the sources.

Nope, lots of idiots just take them at face value. You're still describing what rational people do, not what all actual people do.

Fair enough.

If you believe this, people believe everything they read by default and have to apply a critical thinking filter on top of it to not believe the thing.

I know I don't have as much of a filter as I ought to!

https://www.lesswrong.com/s/pmHZDpak4NeRLLLCw/p/TiDGXt3WrQwt...

That checks out with my experience. I don't think it's just reading either. Even deeper than stranger danger, we're inclined to assume other humans communicating with us are part of our tribe, on our side, and not trying to deceive us. Deception, and our defenses against deception, are a secondary phenomenon. It's the same reason that jokes like "the word 'gullible' is written in the ceiling", gesturing to wipe your face at someone with a clean face, etc, all work by default.

> they're trusted to accurately represent the sources.

Which is still too much trust

“trusted” in computer science does not mean what it means in ordinary speech. It is what you call things you have no choice but to trust, regardless of whether that trust is deserved or not.

For one, it's not like we're at some CS conference, so we're engaging in ordinary speech here, as far as I can tell. For two, "trusted" doesn't have just one meaning, even in the narrower context of CS.

I meant it in the ordinary speech sense (which I don't even thing contradicts the "CS sense" fwiw).

Many people have a lot of trust in anything ChatGPT tells them.

15% of people aren't smart enough to read and follow directions explaining how to fold a trifold brochure, place it in an envelope, seal it, and address it

you think those people don't believe the magic computer when it talks?

I imagine they would be more like product placements in film and TV than banner ads. Just casually dropping a recommendation and link to Brand (TM) in a query. Like those Cerveza Cristal ads in star wars. They'll make it seem completely seamless to the original query.

I just hope that if it comes to that (and I have no doubt that it will), regulation will catch up and mandate any ad/product placement is labeled as such and not just slipped in with no disclosure whatsoever. But, given that we've never regulated influencer marketing which does the same thing, nor are TV placements explicitly called out as "sponsored" I have my doubts but one can hope.

Yup, and I wouldn't be willing to bet that any firewall between content and advertising would hold, long-term.

For example, the more product placement opportunities there are, the more products can be placed, so sooner or later that'll become an OKR to the "content side" of the business as well.

Like that’s ever stopped the adtech industry before.

It would be a hilarious outcome though, “we built machine gods, and the main thing we use them for is to make people click ads.” What a perfect Silicon Valley apotheosis.

I heard majority of the users are techies asking coding questions. What do you sell to someone asking how to fix a nested for loop in C++? I am genuinely curious. Programmers are known to be the stingiest consumers out there.

I'm not sure that stereotype holds up. Developers spend a lot: courses, cloud services, APIs, plugins, even fancy keyboards.

A quick search shows that click on ads targeting developers are expensive.

Also there is a ton of users asking to rewrite emails, create business plans, translate, etc.

> I heard majority of the users are techies asking coding questions.

Citation needed? I can't sit on a bus without spotting some young person using ChatGPT

OpenAI has half a billion active users.

You don't need every individual request to be profitable, just the aggregate. If you're doing a Google search for, like, the std::vector API reference you won't see ads. And that's probably true for something like 90% of the searches. Those searches have no commercial value, and serving results is just a cost of doing business.

By serving those unmonetizable queries the search engine is making a bet that when you need to buy a new washing machine, need a personal injury lawyer, or are researching that holiday trip to Istanbul, you'll also do those highly commercial and monetizable searches with the same search engine.

Chatbots should have exactly the same dynamics as search engines.

You'd probably do brand marketing for Stripe, Datadog, Kafka, Elastic Search etc.

You could even loudly proclaim that the are ads are not targeted by users which HN would love (but really it would just be old school brand marketing).

You sell them Copilot. You Sell them CursorAI. You sell them Windsurf. You sell them Devin. You sell the Claude Code.

Software guys are doing much, much more than treating LLM's like an improved Stack Overflow. And a lot of them are willing to pay.

…for starters, you can sell them the ability to integrate your AI platform into whatever it is they are building, so you can then sell your stuff to their customers.

A lot of people use it for cooking and other categories as well.

Techies are also great for network growth and verification for other users, and act as community managers indirectly.

According to fb's aggressively targeted marketing, you sell them donald trump propaganda.

It's very important to note that advertisers set the parameters in which FB/Google's algorithms and systems operate. If you're 25-55 in a red state, it seems likely that you'll see a bunch of that information (even if FB are well aware you won't click).

I'm not even in USA and I've never been in USA in my entire life.

The existence of the LLMs will themselves change the profile and proclivities of people we consider “programmers” in the same way the app-driven tech boom did. Programmers who came up in the early days are different from ones who came up in the days of the web are different from ones who came up in the app era.

and they wouldn't even have to make the model say the ads. I think that's a terrible idea which would drive model performance down.

Traditional banner ads, inserted inline into the conversation based on some classifier seem a far better idea.

Targeted banner ads based on chat history is last-two-decades thinking. The money with LLMs will be targeted answers. Have Coca-Cola pay you a few billion dollars to reinforce the model to say "Coke" instead of "soda". Train it the best source of information about political subjects is to watch Fox News. This even works with open-source models, too!

It sounds quite scary that an LLM could be trained on a single source of news (specially FN).

If interactions with your AI start sounding like your conversation partner shilling hot cocoa powder at nobody in particular those conversations are going to stop being trusted real quick. (Pop culture reference: https://youtu.be/MzKSQrhX7BM?si=piAkfkwuorldn3sb)

Which may be for the best, because people shouldn’t be implicitly trusting the bullshit engine.

That's calculating value against not having LLMs and current competitors. If they stopped improving but their competitors didn't, then the question would be the incremental cost of Claude (financial, adjusted for switching costs, etc) against the incremental advantage against the next best competitor that did continue improving. Lock in is going to be hard to accomplish around a product that has success defined by its generalizability and adaptability.

Basically, they can stop investing in research either when 1) the tech matures and everyone is out of ideas or 2) they have monopoly power from either market power or oracle style enterprise lock in or something. Otherwise they'll fall behind and you won't have any reason to pay for it anymore. Fun thing about "perfect" competition is that everyone competes their profits to zero

But if Claude stopped pouring their money into research and others didn't, Claude wouldn't be useful a year from now, as you could get a better model for the same price.

This is why AI companies must lose money short term. The moment improvements plateau or the economic environment changes, everyone will cut back on research.

For me, if Anthropic stopped now, and given access to all alternative models, they still would be worth exactly $240 which is the amount I'm paying now. I guess Anthropic and OpenAI can see the real demand by clearly seeing what are their free:basic:expensive plan ratios.

You may want to pay for Claude Max outside of the Google or iOS ecosystem and save $40/month.

> Well worth 4 figures a year IMO

only because software engineering pay hasn't adjusted down for the new reality . You don't know what its worth yet.

Can you explain this in more detail? The idiot bottom rate contractors that come through my team on the regular have not been helped at all by LLMs. The competent people do get a productivity boost though.

The only way I see compensation "adjusting" because of LLMs would need them to become significantly more competent and autonomous.

There's another specific class of person that seems helped by them: the paralysis by analysis programmer. I work with someone really smart who simply cannot get started when given ordinary coding tasks. She researches, reads and understands the problem inside and out but cannot start actually writing code. LLMs have pushed her past this paralysis problem and given her the inertia to continue.

On the other end, I know a guy who writes deeply proprietary embedded code that lives in EV battery controllers and he's found LLMs useless.

> Can you explain this in more detail?

Not sure what GP meant specifically, but to me, if $200/m gets you a decent programmer, then $200/m is the new going rate for a programmer.

Sure, now it's all fun and games as the market hasn't adjusted yet, but if it really is true that for $200/m you can 10x your revenue, it's still only going to be true until the market adjusts!

> The competent people do get a productivity boost though.

And they are not likely to remain competent if they are all doing 80% review, 15% prompting and 5% coding. If they keep the ratios at, for example, 25% review, 5% prompting and the rest coding, then sure, they'll remain productive.

OTOH, the pipeline for juniors now seems to be irrevocably broken: the only way forward is to improve the LLM coding capabilities to the point that, when the current crop of knowledgeable people have retired, programmers are not required.

Otherwise, when the current crop of coders who have the experience retires, there'll be no experience in the pipeline to take their place.

If the new norm is "$200/m gets you a programmer", then that is exactly the labour rate for programming: $200/m. These were previously (at least) $5k/m jobs. They are now $200/m jobs.

$200 does not get you a decent programmer though. It needs constant prompting, babysitting, feedback, iteration. It's just a tool. It massively boosts productivity in many cases, yes. But it doesn't do your job for you. And I'm very bullish on LLM assisted coding when compared to most of HN.

High level languages also massively boosted productivity, but we didn't see salaries collapse from that.

> And they are not likely to remain competent if they are all doing 80% review, 15% prompting and 5% coding.

I've been doing 80% review and design for years, it's called not being a mid or junior level developer.

> OTOH, the pipeline for juniors now seems to be irrevocably broken

I constantly get junior developers handed to me from "strategic partners", they are just disguised as senior developers. I'm telling you brother, the LLMs aren't helping these guys do the job. I've let go 3 of them in July alone.

> I constantly get junior developers handed to me from "strategic partners", they are just disguised as senior developers. I'm telling you brother, the LLMs aren't helping these guys do the job. I've let go 3 of them in July alone.

I find this surprising. I figured the opposite: that the quality of body shop type places would improve and the productivity increases would decrease as you went "up" the skill ladder.

I've worked on/inherited a few projects from the Big Name body shops and, frankly, I'd take some "vibe coded" LLM mess any day of the week. I really figured there was nowhere to go but "up" for those kinds of projects.

> It needs constant prompting, babysitting, feedback, iteration. It's just a tool. It massively boosts productivity in many cases, yes.

It doesn't sound like you are disagreeing with me: that role you described is one of manager, not of programmer.

> High level languages also massively boosted productivity, but we didn't see salaries collapse from that.

Those high level languages still needed actual programmers. If the LLM is able to 10x the output of a single programmer because that programmer is spending all their time managing, you don't really need a programmer anymore, do you?

> I've been doing 80% review and design for years, it's called not being a mid or junior level developer.

Maybe it differs from place to place. I was a senior and a staff engineer, at various places including a FAANG. My observations were that even staff engineer level was still spending around 2 - 3 hours a day writing code. If you're 10x'ing your productivity, you almost certainly aren't spending 2 - 3 hours a day writing code.

> I constantly get junior developers handed to me from "strategic partners", they are just disguised as senior developers. I'm telling you brother, the LLMs aren't helping these guys do the job. I've let go 3 of them in July alone.

This is a bit of a non-sequitor; what does that have to do with breaking the pipeline for actual juniors?

Without juniors, we don't get seniors. Without seniors and above, who will double-check the output of the LLM?[1]

If no one is hiring juniors anymore, then the pipeline is broken. And since the market price of a programmer is going to be set at $200/m, where will you find new entrants for this market?

Hell, even mid-level programmers will exit, because when a 10-programmer team can be replaced by a 1-person manager and a $200/m coding agent, those 9 people aren't quietly going to starve while the industry needs them again. They're going to go off and find something else to do, and their skills will atrophy (just like the 1-person LLM manager skills will atrophy eventually as well).

----------------------------

[1] Recall that my first post in this thread was to say that the LLM coding agents have to get so good that programmers aren't needed anymore because we won't have programmers anymore. If they aren't that good when the current crop starts retiring then we're in for some trouble, aren't we?

> And since the market price of a programmer is going to be set at $200/m

You keep saying this, but I don't see it. The current tools just can't replace developers. They can't even be used in the same way you'd use a junior developer or intern. It's more akin to going from hand tools to power tools than it is getting an apprentice. The job has not been automated and hasn't been outsourced to LLMs.

Will it be? Who knows, but in my personal opinion, it's not looking like it will any time soon. There would need to be more improvement than we've seen from day 1 of ChatGPT until now before we could even be seriously considering this.

> Those high level languages still needed actual programmers.

So does the LLM from day one until now, and for the foreseeable future.

> This is a bit of a non-sequitor; what does that have to do with breaking the pipeline for actual juniors?

Who says the pipeline is even broken by LLMs? The job market went to shit with rising interest rates before LLMs hit the scene. Nobody was hiring them anyway.

> It needs constant prompting, babysitting, feedback, iteration.

What do you think a product manager is doing?

Not writing and committing code with GitHub Copilot, I'll tell you that. These things need to come a _long_ way before that's a reality.

Your argument requires "Claude can replace a programme" to be true. Thus, your argument is false for the foreseeable future.

I mean, it adjusted down by having some hundreds of thousands of engineers laid off in he last 2+ years. they know slashing salaries is legal suicide, so they just make the existing workers work 3x as hard.

Revenue is _NOT_ Profit

And ARR is not revenue. It's "annualized recurring revenue": take one month's worth of revenue, multiply it by 12--and you get to pick which month makes the figures look most impressive.

Astonishing that that concept survived getting laughed out of the room long enough to actually become established as a term and an acronym.

So the "multiply by 12" thing is a slight corruption of ARR, which should be based on recurring revenue (i.e. subscriptions). Subscriptions are harder to game by e.g. channel-stuffing and should be much more stable than non-recurring revenue.

To steelman the original concept, annual revenue isn't a great measure for a young fast-growing company since you are averaging all the months of the last year, many of which aren't indicative of the trajectory of the company. E.g. if a company only had revenue the last 3 months, annual revenue is a bad measure. So you use MRR to get a better notion of instantaneous revenue, but you need to annualize it to make it a useful comparison (e.g. to compute a P/E ratio), so you use ARR.

Private investors will of course demand more detailed numbers like churn and an exact breakdown of "recurring" revenue. The real issue is that these aren't public companies, and so they have no obligation to report anything to the public, and their PR team carefully selects a couple nice sounding numbers.

It’s a KPI just like any KPI and it’s gamed. A lot of random financial metrics are like that. They were invented or coined as a short hand for something.

Different investors use different ratios and numbers (ARR, P/E, EV/EBITDA, etc) as a quick initial smoke screen. They mean different things in different industries during different times of a business’ lifecycle. BUT they are supposed to help you get a starting point to reduce noise. Not as a the 1 metric you base your investing strategy on.

I understand the importance of having data, and that any measurement can be gamed, but this one seems so tailored for tailoring that I struggle to understand how it was ever a good metric.

Even being generous it seems like it'd be too noisy to even assist in informing a good decision. Don't the overwhelmingly vast majority of businesses see periodic ebbs and flows over the course of a year?

(sorry I kept writing and didn't realize how long it got and don't have the time to summarize it better)

Here is how it sort of happens sometimes:

- You are an analyst at some hedge fund.

- You study the agriculture industry overall and understand the general macro view of the market segment and its parameters etc.

- You pick few random agriculture company (e.g: WeGrowPotatos Corp.) that did really really solid returns between 2001 and 2007 and analyze their performance.

- You try to see how you could have predicted the company's performance in 2001 based on all the random bits of data you have. You are not looking for something that makes sense per se. Investing based on metrics that make intuitive sense is extremely hard if not impossible because everyone is doing that which makes the results very unpredictable.

- You figure out that for whatever reason, if you sum the total sales for a company, subtract reserved cash, and divide that by the global inflation rate minus the current interest rate in the US; this company has a value that's an anomaly among all the other agriculture companies.

- You call that bullshit The SAGI™ ratio (Sales Adjusted for Global Inflation ratio)

- You calculate the SAGI™ ratio for other agriculture companies in different points in time and determine its actual historical performance and parameters compared to WeGrowPotatoes in 2001.

- You then calculate that SAGI™ ratio for all companies today and study the ones that match your desired number then invest in them. You might even start applying SAGI™ analysis to non-agriculture companies.

- (If you're successful) In few years you will have built a reputation. Everyone wants to learn from you how you value a company. You share your method with the world. You still investigate the business to see how much it diverges from your "WeGrowPotatoes" model you developed the SAGI ratio based on.

- People look at your returns, look at your (1) step of calculating SAGI, and proclaim that the SAGI ratio paramount. Everyone is talking about nothing but SAGI ratio. Someone creates a SAGIHeads.com and /r/SAGInation and now Google lists it under every stock for some reason.

It's all about that (sales - cash / inflation - interest). A formula that makes no sense; but people are gonna start working it backwards by trying to understand what does "sales - cash" actually mean for a company?

Like that SAGI is bullshit I just made up, but EV is an actual metric and it's generally calculated as (equity + debt - cash). What do you think that tells you about a company? and why do people look at it? How does it make any sense for a company to sum its assets and debt? what is that? According to financial folks it tells you the actual market operation size of the company. The cash a company holds is not in the market so it doesn't count. the assets are obviously important to count, but debt for a company can be positive if it's on path to convert into asset on a reasonable timeline.

I don't know why investors in the tech space focus too much on ARR. It's possible that it was a useful metric with traditional internet startups model like Google, Facebook, Twitter, Instagram, Reddit, etc where the general wisdom was it's impossible to expect people to pay a lot for online services. So generating any sort of revenue almost always correlated with how many contracts do you get to signup with advertisers or enterprises and those are usually pretty stable and lucrative.

I highly recommend listening to Warren Buffets investing Q&As or lectures. He got me to view companies and the entire economy differently.

No worries about the length, I appreciate you taking the time and appreciate the insight! That does help start to work ARR into a mental model that, while still not sane, is at least as understandably insane as everything else in the financial space.

Just wait until companies start calculating it on future revenue from people on the trial period of subscriptions... I mean, if we aren't there already.

Any number that there isn't a law telling companies how to calculate it will always be a joke.

ARR traditionally is _annual_ recurring revenue. The notion that it may be interpreted as _annualized_ and extrapolatable from MRR is a very recent development, and I doubt that most people interpret it as that.

What does it tell you then, that the interpretation of "A" as "annualized" is the interpretation Anthropic, to name one, has chosen?

You don't get to pick the month. At least not with any half-serious audience.

We're not talking about a half-serious audience: we're talking about the collection of reposters of press releases we call "the media".

> At least not with any half-serious audience.

So I guess this rules out most SV venture capital

That's still not profit.

I know. It's a doubly-dubious figure.

It's a good point. Any business can get revenue by selling Twenty dollar bills for $19. But in the history of tech, many winners have been dismissed for lack of an apparent business model. Amazon went years losing money, and when the business stabilized, went years re-investing and never showed a profit. Analysts complained as Amazon expanded into non-retail activities. And then there's Uber.

The money is there. Investors believe this is the next big thing, and is a once in a lifetime opportunity. Bigger than the social media boom which made a bunch of billionaires, bigger than the dot com boom, bigger maybe than the invention of the microchip itself.

It's going to be years before any of these companies care about profit. Ad revenue is unlikely to fund the engineering and research they need. So the only question is, does the investor money dry up? I don't think so. Investor money will be chasing AGI until we get it or there's another AI winter.

How many of those MAUs are crappy startups building a janky layer on top of the OpenAI API which will cease to exist in 2 years?

Last year, ChatGPT was 75% of OpenAI's revenue[1], not the API.

[1]: https://www.businessofapps.com/data/chatgpt-statistics/

No, because if they stop to focus on optimizing and minimizing operating costs, the next competitor over will leapfrog them with a better model in 6-12 months, making all those margin improvements an NPV negative endeavor.

> If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their user base you think they don't have a successful business model?

Actually, I'd be very curious to know this. Because we already have a few relatively capable models that I can run on my MBP with 128 GB of RAM (and a few less capable models I can run much faster on my 5090).

In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.

But the cynic in me feels they prefer to avoid this reality check and use the tried and tested Uber model of permanent money influx with the "profitability is just around the corner" justification but at an even bigger scale.

> In order to break even they would have to minimize the operating costs (by throttling, maiming models etc.) and/or increase prices. This would be the reality check.

Is that true? Are they operating inference at a loss or are they incurring losses entirely on R&D? I guess we'll probably never know, but I wouldn't take as a given that inference is operating at a loss.

I found this: https://semianalysis.com/2023/02/09/the-inference-cost-of-se...

which estimates that it costs $250M/year to operate ChatGPT. If even remotely true $10B in revenue on $250M of COGS would be a great business.

As you say, we will never know, but this article[0] claims:

> The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs.

[0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...

CapEx vs. OpEx.

If they stop training today what happens? Does training always have to be at these same levels or will it level off? Is training fixed? IE, you can add 10x the subs and training costs stay static.

IMO, there is a great business in there, but the market will likely shrink to ~2 players. ChatGPT has a huge lead and is already Kleenex/Google of the LLMs. I think the battle is really for second place and that is likely dictated by who runs out of runway first. I would say that Google has the inside track, but they are so bad at product they may fumble. Makes me wonder sometimes how Google ever became a product and verb.

That paragraph is quite clear.

OpEx is larger than revenue. CapEx is also larger than the total revenue on the lifetime of a model.

Obviously you don't need to train new models to operate existing ones.

I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.

You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete.

Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.

> You need to train new models to advance the knowledge cutoff

That's a great point, but I think it's less important now with MCP and RAG. If VC money dried up and the bubble burst, we'd still have broadly useful models that wouldn't be obsolete for years. Releasing a new model every year might be a lot cheaper if a company converts GPU opex to capex and accepts a long training time.

> Also the semianalysis estimate is from Feb 2023,

Oh! I missed the date. You're right, that's a lot more expensive. On the other hand, inference has likely gotten a lot cheaper (in terms of GPU TOPS) too. Still, I think there's a profitable business model there if VC funding dries up and most of the model companies collapse.

But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.

Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.

> Obviously you don't need to train new models to operate existing ones.

For a few months, maybe. Then they become obsolete and, in some cases like coding, useless.

One thing we're seeing in the software engineering agent space right now is how many people are angry with Cursor [1], and now Claude Code [2] (just picked a couple examples; you can browse around these subreddits and see tons of complaints).

What's happening here is pretty clear to me: Its a form of enshittification. These companies are struggling to find a price point that supports both broad market adoption ($20? $30?) and the intelligence/scale to deliver good results ($200? $300?). So, they're nerfing cheap plans, prioritizing expensive ones, and pissing off customers in the process. Cursor even had to apologize for it [3].

There's a broad sense in the LLM industry right now that if we can't get to "it" (AGI, etc) by the end of this decade, it won't happen during this "AI Summer". The reason for that is two-fold: Intelligence scaling is logarithmic w.r.t compute. We simply cannot scale compute quick enough. And, interest in funding to pay for that exponential compute need will dry up, and previous super-cycles tell us that will happen on the order of ~5 years.

So here's my thesis: We have a deadline that even evangelists agree is a deadline. I would argue that we're further along in this supercycle than many people realize, because these companies have already reached the early enshitification phase for some niche use-cases (software development). We're also seeing Grok 4 Heavy release with a 50% price increase ($300/mo) yet offer single-digit percent improvement in capability. This is hallmark enshitification.

Enshitification is the final, terminal phase of hyperscale technology companies. Companies remain in that phase potentially forever, but its not a phase where significant research, innovation, and optimization can happen; instead, it is a phase of extraction. AI hyperscalers genuinely speedran this cycle thanks to their incredible funding and costs; but they're now showcasing very early signals of enshitifications.

(Google might actually escape this enshitification supercycle, to be clear, and that's why I'm so bullish on them and them alone. Their deep, multi-decade investment into TPUs, Cloud Infra, and high margin product deployments of AI might help them escape it).

[1] https://www.reddit.com/r/cursor/comments/1m0i6o3/cursor_qual...

[2] https://www.reddit.com/r/ClaudeAI/comments/1lzuy0j/claude_co...

[3] https://techcrunch.com/2025/07/07/cursor-apologizes-for-uncl...

> that's because they are racing improve models. If they stopped today to focus on optimization of their current models to minimize operating cost and monetizing their user base you think they don't have a successful business model?

I imagine they would’ve flicked that switch if they thought it would generate a profit, but as it is it seems like all AI companies are still happy to burn investor money trying to improve their models while I guess waiting for everyone else to stop first.

I also imagine it’s hard to go to investors with “while all of our competitors are improving their models and either closing the gap or surpassing us, we’re just going to stabilize and see if people will pay for our current product.”

> I also imagine it’s hard to go to investors with “while all of our competitors are improving their models and either closing the gap or surpassing us, we’re just going to stabilize and see if people will pay for our current product.”

Yeah, no one wants to be the first to stop improving models. As long as investor money keeps flowing in there's no reason to - just keep burning it and try to outlast your competitors, figure out the business model later. We'll only start to see heavy monetization once the money dries up, if it ever does.

Maybe I’m naïve/ignorant of how things are done in the VC world, but given the absolutely enormous amount of money flowing into so many AI startups right now, I can’t imagine that the gravy train is going to continue for more than a few years. Especially not if we enter any sort of economic downturn/craziness from the very inconsistent and unpredictable decisions being made by the current administration

You would think so. Investors are eventually going to want a return on their money put in. But there seems to be a ton of hype and irrationality around AI, even worse than blockchain back in the day.

I think there's an element of FOMO - should someone actually get to AGI, or at least something good enough to actually impact the labor market and replace a lot of jobs, the investors of that company/product stand to make obscene amounts of money. So everyone pumps in, in hope of that far off future promise.

But like you said, how long can this keep going before it starts looking like that future promise will not be fulfilled in this lifetime and investors start wanting a return.

Making money and operating at a loss contradict each other. Maybe someday they’ll make money —but not just yet. As many have said they’re hoping capturing market will position them nicely once things settle. Obviously we’re not there yet.

It is absolutely possible for the unit economics of a product to be profitable and for the parent company to be losing money. In fact, it's extremely common when the company is bullish on their own future and thus they invest heavily in marketing and R&D to continue their growth. This is what I understood GP to mean.

Whether it's true for any of the mainstream LLM companies or not is anyone's guess, since their financials are either private or don't separate out LLM inference as a line item.

It’s just the natural counterpart to dogmatic inevitabilism — dogmatic denialism. One denies the present, the other the (recent) past. It’s honestly an understandable PoV though when you consider A) most people understand “AI” and “chatbot” to be synonyms, and B) the blockchain hype cycle(s) bred some deep cynicism about software innovation.

Funny seeing that comment on this post in particular, tho. When OP says “I’m not sure it’s a world I want”, I really don’t think they’re thinking about corporate revenue opportunities… More like Rehoboam, if not Skynet.

> most people understand “AI” and “chatbot” to be synonyms

This might be true (or not), but for sure not on this site.

I mean...

  LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them,

My guess is that LLM's are bridge technology, the equivalent of cassette tapes. A big step forward, allowing things that we couldn't before. But before long they'll be surpassed by much better technology, and future generations will look back on them as primitive.

You have top scientists like LeCun arguing this position. I'd imagine all of these companies are desperately searching for the next big paradigm shift, but no one knows when that will be, and until then they need to squeeze everything they can out of LLMs.

Exactly. This is basically the argument of “AI as Normal Technology”.

https://knightcolumbia.org/content/ai-as-normal-technology

https://news.ycombinator.com/item?id=43697717

Thanks for the link. The comparison to electricity is a good one, and this is a nice reflection on why it took time for electricity’s usefulness to show up in productivity stats:

> What eventually allowed gains to be realized was redesigning the entire layout of factories around the logic of production lines. In addition to changes to factory architecture, diffusion also required changes to workplace organization and process control, which could only be developed through experimentation across industries.

This seems like one the only sane arguments in this whole sea of articles.

>> There are many technologies that have seemed inevitable and seen retreats under the lack of commensurate business return

120+ Cable TV channels must have seemed like a good idea at the time, but like LLMs the vast majority of the content was not something people were interested in.

The difference is that the future is now with LLMs. There is a microwave (some multiple) in almost every kitchen in the world. The Concord served a few hundred people a day. LLMs are already ingrained into hundreds of millions if not billions of people’s lives, directly and indirectly. My dad directly uses LLMs multiple times a week if not daily in an industry that still makes you rotate your password every 3 months. It’s not a question of whether the future will have them, it’s a question of whether the future will get tired of them.

The huge leap that is getting pushback is the sentiment that LLMs will consume every use case and replace human labor. I don't think many are arguing LLMs will die off entirely.

I think the difference between all previous technologies is scope. If you make a super sonic jet that gets people from place A to place B faster for more money, but the target consumer is like "yeah, I don't care that much about that at that price point", then your tech sort is of dead. You are also fully innovated on that product, like maybe you can make it more fuel efficient, sure, but your scope is narrow.

AI is the opposite. There are numerous things it can do and numerous ways to improve it (currently). There is lower upfront investment than say a supersonic jet and many more ways it can pivot if something doesn't work out.

The number of things it can actually do is significantly lower than the number of things the hype men are claiming it can do.

It's not a great analogy. The only parallel with Concorde is energy consumption. I think a better analogy would have been VR.

I mean, thats the point, they aren't the same. Concorde was one dimensional, AI is not.

Most of the comments here feel like cope about AI TBH. There's never been an innovation like this ever, and it makes sense to get on board rather than be left behind.

> There's never been an innovation like this ever

There have been plenty of innovations like this. In fact, much of the hype around LLMs is a rehash of the hype around "expert systems" back in the '80s. LLMs are marginally more effective than those systems, but only marginally.

Let's not ignore the technical aspects as well: LLMs are probably a local minima that we've gotten stuck in because of their rapid rise. Other areas in AI are being starved of investment because all of the capital is pouring into LLMs. We might have been better off in the long run if LLMs hadn't been so successful so fast.

Developers haven't even started extracting the value of LLMs with agent architectures yet. Using an LLM UI like open ai is like we just figured fire and you use it to warm you hands (still impressive when you think about it, but not worth the burns), while LLM development is about building car engines (here is you return on investment).

> Developers haven't even started extracting the value of LLMs with agent architectures yet

There are thousands of startups doing exactly that right now, why do you think this will work when all evidence points towards it not working? Or why else would it not already have revolutionized everything a year or two ago when everyone started doing this?

Most of them are a bunch of prompts and don't even have actual developers. For the good reason that there is no training system yet and the wording of how you call the people that build these system isn't even there or clearly defined. Local companies haven't even setup a proper internal LLM or at least a contract with a provider. I am in France so probably lagging behind USA a bit especially NY/SF but the word "LLM developer" is just arriving now and mostly under the pressure of isolated developers and companies like me. This feel really really early stage.

Between the ridiculously optimistic and the cynically nihilistic I personally believe there is some value that extremely talented people at huge companies can't really provide because they're not in the right environment (too big a scale) but neither can grifters packaging a prompt in a vibecoded app.

In the last few months the building blocks for something useful for small companies (think less than 100 employees) have appeared, now it's time for developers or catch-all IT at those companies and freelancers serving small local companies to "up-skill".

Why do I believe this? Well for a start OCR became much more accessible this year cutting down on manual data entry compared to tesseract of yesteryear.

is there a non-prompt way to interact with LLMs?

In an agentic setup the value is half the prompts half how you plug them together. I am opposing for instance a big prompt that is supposed to write a dissertation vs a smart web scraper that builds a knowledge graph out of sources and outputs a specialized search engine for your task. The former is a free funny intern, the latter is growth percentage visible in the economy.

The smartest and most well funded people on the planet have been trying and failing to get value out of this technology for years and the best we've come up with so far is some statistically unreliable coding assistants. Hardly the revolution its proponents keep eagerly insisting we're seeing.

They try to get value at their scale, which is tough. Your local SME definitely sees value in an embedding-based semantic search engine over their 20 years of weird unstructured data.

my company has already fired a bunch of people in favor of LLMs so they are realizing all kinds of value

I don’t know your company but this thinking doesn’t necessarily follow logically. In a large company the value of developers is not distributed evenly across people and time, and also has a strong dependency on market realities in front of them.

While it’s true that lots of companies are getting some value out of LLMs, a much larger number are using them as an excuse for layoffs they would have wanted to do anyway—LLMs are just a golden opportunity to tie in an unmitigated success narrative.

> a much larger number are using them as an excuse for layoffs they would have wanted to do anyway

It's a simple formula. Layoffs because of market conditions or company health = stock price go down. Layoffs because "AI took the jobs" = stock price go up.

So has mine, and quite predictably our product has gone into the shitter and breaks constantly, requiring reverts almost daily. They've armed a couple of Juniors with Cursor and given them the workload of all those people they fired / have quit since the firings, some of which have been at the company for years and held a lot of institutional knowledge that is now biting them in the ass.

Now sure, "Just don't fire the useful people and get rid of the juniors and supercharge the good devs with AI tooling" or whatever, except the whole reason the C-level is obsessed with this AI shit is because they're sold on the idea of replacing their most expensive asset, devs, because they've been told by people who sell AI as a job that it can replace those pesky expensive devs and be replaced by any random person in the company prompting up a storm and vibecoding it all.

Churn rates are up, we're burning unfathomable amounts of money on the shitty AI tooling and the project has somehow regressed after we've finally managed to get a good foothold on it and start making real progress for once. Oh and the real funny part is they're starting to backpedal a bit and have tried to get some people back in.

I expect to hear a LOT more of this type of thing happening in the near future. As the idiots in charge start slowly realizing all the marketing sold to them on LinkedIn or wherever the fuck it is they get these moronic ideas from are literal, actual literal lies.

I imagine they HOPE they'll realize value. A lot of people are acting on what might be, rather than what is, which makes sense given that the AI "thought leaders" (CEOs with billions invested that need to start turning a profit) are all promising great things soon™.

Yeah callousness does seem to be the leaking area of improvement.

Only as much as replacing all your devs with a frog is "realizing value"

The best they've come up with is the LLM chatbot, which both OpenAI and Anthropic have as their flagship product because many people find it extremely valuable. Many people I know routinely use ChatGPT to help them write things, even those who were already good at writing, and if you don't think that's true at your workplace I strongly suspect it's because people aren't telling you about it.

Great, we've got mediocre writing from unprofitable companies that are subsidizing the cost of this technology.

What specifically do you find to be mediocre? I feel like LLMs write better than most people I know, myself included.

There could be a mismatch on what the state of the art really is these days. In my experience, since the release of GPT-4 and especially 4o, ChatGPT has been able to do the vast majority of concrete things people tell me it can't do.

>Or why else would it not already have revolutionized everything a year or two ago when everyone started doing this?

The internet needed 20 years to take over the world. All of the companies of the first dot com bust are in the past. The tech is solid.

Every booster argument is like this one. $trite_analogy triumphant smile

[flagged]

Throwing genetic fallacies around instead of engaging with the comment at hand... :)

3 years into automating all white collar labor in 6 months.

It's 3 years away.

Theyre doing it so much it's practically a cliche.

There are underserved areas of the economy but agentic startups is not one.

>> Developers haven't even started extracting the value of LLMs with agent architectures yet.

What does this EVEN mean? Do words have any value still, or are we all just starting to treat them as the byproduct of probabilistic tokens?

"Agent architectures". Last time I checked an architecture needs predictability and constraints. Even in software engineering, a field for which the word "engineering" is already quite a stretch in comparison to construction, electronics, mechanics.

Yet we just spew the non-speak "Agentic architectures" as if the innate inability of LLMs in managing predictable quantitative operations is not an unsolved issue. As if putting more and more of these things together automagically will solves their fundamental and existential issue (hallucinations) and suddenly makes them viable for unchecked and automated integration.

This means I believe we currently underuse LLM capabilities and their empirical nature makes it difficult to assess their limitations without trying. I've been studying LLMs from various angles during a few months before coming to this conclusion, as an experienced software engineer and consultant. I must admit it is however biased towards my experience as an SME and in my local ecosystem.

> Developers haven't even started extracting the value of LLMs with agent architectures yet.

For sure there is a portion of developers who don't care about the future, are not interested in current developements and just live as before hoping nothing will change. But the rest already gave it a try and realized tools like Claude Code can give excellent results for small codebases to fail miserably at more complex tasks with the net result being negative as you get a codebase you don't understand, with many subtle bugs and inconsistencies created over a few days you will need weeks to discover and fix.

This is a bit developer centric, I am much more impressed by the opportunities I see in consulting rather than applying LLMs to dev tasks. And I am still impressed by the code it can output eventhough we are still in the funny intern stage in this area.

> I am much more impressed by the opportunities I see in consulting rather than applying LLMs to dev tasks.

I expect there'll be a lot of consulting work in the near future in cleanup and recovery from LLM-generated disasters.

>evelopers haven't even started extracting the value of LLMs with agent architectures yet.

Which is basically what? The infinite monkey theorem? Brute forcing solutions for problems at huge costs? Somehow people have been tricked to actually embrace and accept that now they have to pay subscriptions from 20$ to 300$ to freaking code? How insane is that, something that was a very low entry point and something that anyone could do, is now being turned into some sort of classist system where the future of code is subscriptions you pay for companies ran by sociopaths who don't care that the world burns around them, as long as their pockets are full.

I don't have a subscription not even an Open AI account (mostly cause they messed up their google account system). You can't extract value of an LLM by just using the official UI, you just scratch the surface of how they work. And yet there aren't much developers able to actually build an actual agent architecture that does deliver some value. I don't include the "thousands" of startups that are clearly suffer from a signaling bias: they don't exist in the economy and I don't care about them like at all in my reasonning. I am talking about actual LLM developers that you can recruit locally the same way you recruit a web developer today, and that can make sense out of "frontier" LLM garbage talk by using proper architectures. These devs are not there yet.

I cannot emphasize how much I agree with this comment. Thank you for writing it, I would never have had written it as well.

I pay $300 to fly from SF to LA when I could've just walked for free. Its true. How classist!

> first signs of pulling back investments

I agree with you, but I’m curious; do you have link to one or two concrete examples of companies pulling back investments, or rolling back an AI push?

(Yes it’s just to fuel my confirmation bias, but it’s still feels nice:-) )

Most prominent example was this one: https://www.reuters.com/technology/microsoft-pulls-back-more...

I think that's more reflective of the deteriorating relationship between OpenAI and Microsoft than an true lack of demand for datacenters. If a major model provider (OpenAI, Anthropic, Google, xAI) were to see a dip in available funding or stop focusing on training more powerful models, that would convince me we may be in a bubble about to pop, but there are no signs of that as far as I can see.

> most people agree that the output is trite and unpleasant to consume

This is likely a selection bias: you only notice the obviously bad outputs. I have created plenty of outputs myself that are good/passable -- you are likely surrounded by these types of outputs without noticing.

Not a panacea, but can be useful.

There are pretty hidden assumption in this comment. First of all, not every business in the AI space is _training_ models, and the difference between training and inference is massive - i.e. most businesses can easily afford inference, perhaps depending on model, but they definitely can.

Another several unfounded claims were made here, but I just wanted to say LLMs with MCP are definitely good enough for almost every use case you can come up with as long as you can provide them with high quality context. LLMs are absolutely the future and they will take over massive parts of our workflow in many industries. Try MCP for yourself and see. There's just no going back.

LLMs with tools*

MCP isn’t inherently special. A Claude Code with Bash() tool can do nearly anything a MCP server will give you - much more efficiently.

Computer Use agents are here and are only going to get better.

The conversation shouldn’t be about LLMs any longer. Providers will be providing agents.

correct and companies will be exposing their data via mcp instead of standard rest apis.

That makes no sense. MCP at best is a protocol transpilation at runtime. It is not redefining things like DB drivers or connections. And I did not say rest apis enable agents. Computer use tooling does. APIs and everything else that already exists.

MCP is more like graphql. Not a new network paradigm.

The design of MCP right now is not very optimal esp when you can equip an agent with one tool vs 5-20 that bloat it's reasoning every prompt.

why would you make an agent click around a web browser like a human when it could self discover the api and call it directly?

self discovery via primitives is what works well today. I never discouraged that, only discouraged MCP sensationalism.

However, an agent that can see the screen and immediately click through whatever desired UI modality is immensely more efficient than swimming through protocols.

There is at least one frontier lab who has prepared enough foresight that agents running on VDI infrastructure is a major coming wave.

> I just wanted to say LLMs with MCP are definitely good enough for almost every use case you can come up with as long as you can provide them with high quality context.

This just shows you lack imagination.

I have a lot of use cases that they are not good enough for.

I do wonder where in the cycle this all is given that we've now seen yet another LLM/"Agentic" VSCode fork.

I'm genuinely surprised that Code forks and LLM cli things are seemingly the only use case that's approached viability. Even a year ago, I figured there'd be something else that's emerged by now.

But there are a ton of LLM powered products in the market.

I have a friend in finance that uses LLM powered products for financial analysis, he works in a big bank. Just now anthropic released a product to compete in this space.

Another friend in real estate uses LLM powered lead qualifications products, he runs marketing campaigns and the AI handles the initial interaction via email or phone and then ranks the lead in their crm.

I have a few friends that run small businesses and use LLM powered assistants to manage all their email comms and agendas.

I've also talked with startups in legal and marketing doing very well.

Coding is the theme that's talked about the most in HN but there are a ton of startups and big companies creating value with LLMs

Yup. Lots of products in the education space. Even doctors are using LLMs, while talking with patients. All sorts of teams are using the adjacent products for image and (increasingly) video generation. Translation freelancers have been hit somewhat hard because LLMs do "good enough" quite a bit better than old google translate.

Coding is relevant to the HN bubble, and as tech is the biggest driver of the economy it's no surprise that tech-related AI usages will also be the biggest causes of investment, but it really is used in quite a lot of places out there already that aren't coding related at all.

LLMs are amazing at anything requiring text analysis (go figure). Everyone I know doing equity or economic research in finance is using it extensively for that, and from what I hear from doctors the LLMs are as good as that in their space if not better

It feels like there's a lot of shifting goalposts. A year ago, the hype was that knowledge work would cease to exist by 2027.

Now we are trying to hype up enhanced email autocomplete and data analysis as revolutionary?

I agree that those things are useful. But it's not really addressing the criticism. I would have zero criticisms of AI marketing if it was "hey, look at this new technology that can assist your employees and make them 20% more productive".

I think there's also a healthy dose of skepticism after the internet and social media age. Those were also society altering technologies that purported to democratize the political and economic system. I don't think those goals were accomplished, although without a doubt many workers and industries were made more productive. That effect is definitely real and I'm not denying that.

But in other areas, the last 3 decades of technological advancement have been a resounding failure. We haven't made a dent in educational outcomes or intergenerational poverty, for instance.

> 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them,

I always think back to how Bezos and Amazon were railed against for losing money for years. People thought that would never work. And then when he started selling stuff other than books? People I know were like: please, he's desperate.

Someone, somewhere will figure out how to make money off it - just not most people.

> model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

What are you basing this on? Personal feelings?

My take since day one:

(1) Model capabilities will plateau as training data is exhausted. Some additional gains will be possible by better training, better architectures, more compute, longer context windows or "infinite" context architectures, etc., but there are limits here.

(2) Training on synthetic data beyond a very limited amount will result in overfitting because there is no new information. To some extent you could train models on each other, but that's just an indirect way to consolidate models. Beyond consolidation you'll plateau.

(3) There will be no "takeoff" scenario -- this is sci-fi (in the pejorative sense) because you can't exceed available information. There is no magic way that a brain in a vat can innovate beyond available training data. This includes for humans -- a brain in a vat would quickly go mad and then spiral into a coma-like state. The idea of AI running away is the information-theoretic equivalent of a perpetual motion machine and is impossible. Yudkowski and the rest of the people afraid of this are crackpots, and so are the hype-mongers betting on it.

So I agree that LLMs are real and useful, but the hype and bubble are starting to plateau. The bubble is predicated on the idea that you can just keep going forever.

Investments are mostly in model training. We have trained models now, we'll see a pullback in that regard as businesses will need to optimize to get the best model without spending billions in order to compete on price, but LLMs are here to stay.

> (the supersonic jetliner) ... (the microwave oven)

But have we ever had a general purpose technology (steam engine, electricity) that failed to change society?

It wouldn't be general purpose if it fails to bring change. I'd take every previous iteration of "AI" as example, IBM Watson, that stuff

> 2. Almost three years in, companies investing in LLMs have not yet discovered a business model that justifies the massive expenditure of training and hosting them, the majority of consumer usage is at the free tier, the industry is seeing the first signs of pulling back investments, and model capabilities are plateauing at a level where most people agree that the output is trite and unpleasant to consume.

You hit the nail on why I say to much hatred from "AI Bros" as I call them, when I say it will not take off truly until it runs on your phone effortlessly, because nobody wants to foot a trillion dollar cloud bill.

Give me a fully offline LLM that fits in 2GB of VRAM and lets refine that so it can plug into external APIs and see how much farther we can take things without resorting to burning billions of dollars' worth of GPU compute. I don't care that my answer arrives instantly, if I'm doing the research myself, I want to take my time to get the correct answer anyway.

You aren’t extrapolating enough. Nearly the entire history of computing has been one that isolates between shared computing and personal computing. Give it time. These massive cloud bills are building the case for accelerators in phones. It’s going to happen just needs time.

That's fine, that's what I want ;) I just grow tired of people hating on me for thinking that we really need to localize the models for them to take off.

We actually aren't too far off from that reality. There are several models you can run fully offline on your phone (phi-3, Gemma-3n-E2b-it, Qwen2.5-1.5b-instruct all run quite well on my Samsung S24 ultra). There are a few offline apps that also have tool calling (mostly for web search but I suspect this is extendable).

If you want to play around a bit and are on android there is PocketPal,ChatterUI, MyDeviceAI, SmolChat are good multi-model apps and Google's Edge gallery won't keep your chats but is a fun tech demo.

All are on github and can be installed using Obtainium if you don't want to

LLMs need significant optimization or we get significant improvement on computing power while keeping the energy cost the same. It's similar with smartphone, when at the start it's not feasible because of computing power, and now we have one that can rival 2000s notebooks.

LLMs is too trivial to be expensive

EDIT: I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

LLM can give you thousands of lines of perfectly working code for less than 1 dollar. How is that trivial or expensive?

Looking up a project on github, downloading it and using it can give you 10000 lines of perfectly working code for free.

Also, when I use Cursor I have to watch it like a hawk or it deletes random bits of code that are needed or adds in extra code to repair imaginary issues. A good example was that I used it to write a function that inverted the axis on some data that I wanted to present differently, and then added that call into one of the functions generating the data I needed.

Of course, somewhere in the pipeline it added the call into every data generating function. Cue a very confused 20 minutes a week later when I was re-running some experiments.

Are you seriously comparing downloading static code from github with bespoke code generated for your specific problem? LLMs don't keep you from coding, they assist it. Sometimes the output works, sometimes it doesn't (on first or multiple tries). Dismissing the entire approach because it's not perfect yet is shortsighted.

They didn’t dismiss it, they just said it is not really that useful which is correct?

Obviously YMMV, but it is extremely useful for me and for many people out there.

Many obviously disagree that it's correct

well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

and the 1 dollar cost for your case is heavily subsidized, that price won't hold up long assuming the computing power stays the same.

Cheaper models might be around $0.01 per request, and it's not subsidized: we see a lot of different providers offering open source models, which offer quality similar to proprietary ones. On-device generation is also an option now.

For $1 I'm talking about Claude Opus 4. I doubt it's subsidized - it's already much more expensive than the open models.

Thousands of lines of perfectly working code? Did you verify that yourself? Last time I tried it produced slop, and I've been extremely detailed in my prompt.

Yes. I verified it myself. Best results from Opus 4 so far, Gemini might be OK too.

Try again.

Any retries before nailing the prompt are still going to be billed, so this supports GP position about LLMs being expensive for trivial things.

But the thing is, LLMs are already incredibly cheap to operate compared to the alternatives. Both for trivial things and for complex things.

Well recently cursor got a heat for rising price and having opaque usage, while anthropic's claude reported to be worse due to optimization. IMO the current LLMs are not sustainable, and prices are expected to increase sooner or later.

Personally, until models comparable with sonnet 3.5 can be run locally on mid range setup, people need to wary that the price of LLM can skyrocket

Imagine telling a person from five years ago that the programs that would basically solve NLP, perform better than experts at many tasks and are hard not to anthropomorphize accidentally are actually "trivial". Good luck with that.

It still doesn't pass the Turing test, and is not close. Five years ago me would be impressed but still adamant that this is not AI, nor is it on the path to AI.

>programs that would basically solve NLP

There is a load-bearing “basically” in this statement about the chat bots that just told me that the number of dogs granted forklift certification in 2023 is 8,472.

Sure, maybe solving NLP is too great a claim to make. It is still not at all ordinary that beforehand we could not solve referential questions algorithmically, that we could not extract information from plain text into custom schemas of structured data, and context-aware mechanical translation was really unheard of. Nowadays LLMs can do most of these tasks better than most humans in most scenarios. Many NLP questions at least I find interesting reduce to questions of the explanability of LLMs.

Yeah it solved NLP about 50% of the time, and also mangles data badly and in often hard-to-detect ways.

"hard not to anthropomorphize accidentally' is a you problem.

I'm unhappy every time I look in my inbox, as it's a constant reminder there are people (increasingly, scripts and LLMs!) prepared to straight-up lie to me if it means they can take my money or get me to click on a link that's a trap.

Are you anthropomorphizing that, too? You're not gonna last a day.

I didn't mean typical chatbot output, these are luckily still fairly recognizable due to stylistic preferences learned during fine-tuning. I mean actual base model output. Take a SOTA base model and give it the first two paragraphs of some longer text you wrote, and I would bet on many people being unable to distinguish your continuation from the model's autoregressive guesses.

Calling LLMs trivial is a new one. Yea just consume all of the information on the internet and encode it into a statistical model, trivial, child could do it /s

well I presented the statement wrongly. What I mean is the use case for LLM are trivial things, it shouldn't be expensive to operate

> all of the information on the internet

Total exaggeration—especially given Cloudflare providing free tools to block AI and now tools to charge bots for access to information.

They didn't really need the cloud either and yet...

ML models have the good property of only requiring investment once and can then be used till the end of history or until something better replaces them.

Granted the initial investment is immense, and the results are not guaranteed which makes it risky, but it's like building a dam or a bridge. Being in the age where bridge technology evolves massively on a weekly basis is a recipe for being wasteful if you keep starting a new megaproject every other month though. The R&D phase for just about anything always results in a lot of waste. The Apollo programme wasn't profitable either, but without it we wouldn't have the knowledge for modern launch vehicles to be either. Or to even exist.

I'm pretty sure one day we'll have an LLM/LMM/VLA/etc. that's so good that pretraining a new one will seem pointless, and that'll finally be the time we get to (as a society) reap the benefits of our collective investment in the tech. The profitability of a single technology demonstrator model (which is what all current models are) is immaterial from that standpoint.

Nah, if TSMC got exploded and there was a world war, in 20 years all the LLMs would bit rot.

Eh, I doubt it, tech only got massively better in each world war so far, through unlimited reckless strategic spending. We'd probably get a TSMC-like fab on every continent by the end of it. Maybe even optical computers. Quadrotor UAV are the future of warfare after all, and they require lots of compute.

Adjusted for inflation it took over 120 billion to build the fleet of liberty ships during WW2, that's like at least 10 TSMC fabs.

Technology is an exponential process, and the thing about exponentials is that they are chaotic. You cant use inductive reasoning vis a vis war and technology. The next big one could truly reset us to zero or worse.

Sure you can't plan for black swan events, so the only choice you have is to plan for their absence. If we all nuke ourselves tomorrow well at least we don't have to worry about anything anymore. But in case we don't, those plans will be useful.

I don't really buy your point 2. Just the other day Meta announced hundreds of billions of dollars investment into more AI datacenters. Companies are bringing back nuclear power plants to support this stuff. Earlier this year OpenAI and Oracle announced their $500bn AI datacenter project, but admittedly in favor of your point have run into funding snags, though that's supposedly from tariff fears with foreign investors, not lack of confidence in AI. Meta can just finance everything from their own capital and Zuck's decree, like they did with VR (and it may very well turn out similarly).

Since you brought up supersonic jetliners you're probably aware of the startup Boom in Colorado trying to bring it back. We'll see if they succeed. But yes, it would be a strange path, but a possible one, that LLMs kind of go away for a while and try to come back later.

You're going to have to cite some surveys for the "most people agree that the output is trite and unpleasant" and "almost universally disliked attempts to cram it everywhere" claims. There are some very vocal people against LLM flavors of AI, but I don't think they even represent the biggest minority, let alone a majority or near universal opinions. (I personally was bugged by earlier attempts at cramming non-LLM AI into a lot of places, e.g. Salesforce Einstein appeared I think in 2016, and that was mostly just being put off by the cutesy Einstein characterization. I generally don't have the same feelings with LLMs in particular, in some cases they're small improvements to an already annoying process, e.g. non-human customer support that was previously done by a crude chatbot front-end to an expert system or knowledge base, the LLM version of that tends to be slightly less annoying.)

One of the negative consequences of the “modern secular age” is that many very intelligent, thoughtful people feel justified in brushing away millennia of philosophical and religious thought because they deem it outdated or no longer relevant. (The book A Secular Age is a great read on this, btw, I think I’ve recommended it here on HN at least half a dozen times.)

And so a result of this is that they fail to notice the same recurring psychological patterns that underly thoughts about how the world is, and how it will be in the future - and then adjust their positions because of this awareness.

For example - this AI inevitabilism stuff is not dissimilar to many ideas originally from the Reformation, like predestination. The notion that history is just on some inevitable pre-planned path is not a new idea, except now the actor has changed from God to technology. On a psychological level it’s the same thing: an offloading of freedom and responsibility to a powerful, vaguely defined force that may or may not exist outside the collective minds of human society.

I'm pretty bearish on the idea that AGI is going to take off anytime soon, but I read a significant amount of theology growing up and I would not describe the popular essays from e.g., LessWrong as religious in nature. I also would not describe them as appearing poorly read. The whole "look they just have a new god!" is a common trope in religious apologetics that is usually just meant to distract from the author's own poorly constructed beliefs. Perhaps such a comparison is apt for some people in the inevitable AGI camp, but their worst arguments are not where we should be focusing.

Philosophy and religion are not mutually inclusive, though one can certainly describe a religious belief as being a philosophical belief.

Even a scientifically inclined atheist has philosophical ideas grounding their world view. The idea that the universe exists as an objective absolute with immutable laws of nature is a metaphysical idea. The idea that nature can be observed and that reason is a valid tool for acquiring knowledge about nature is an epistemological idea. Ethics is another field of philosophy and it would be a mistake to assume a universal system of ethics that has been constant throughout all cultures across all of human history.

So while I certainly agree that there is a very common hand-wave of "look the atheists have just replaced God with a new 'god' by a different name", you don't have to focus on religion, theology and faith based belief systems to identify different categories of philosophical ideas and how they have shaped different cultures, their beliefs and behaviours throughout history.

A student of philosophy would identify the concept of "my truth" as being an idea put forward by Emmanuel Kant, for example, even though the person saying that doesn't know that that's the root of the idea that reality is subjective. Similarly, the empirically grounded scientist would be recognized as following in the footsteps of Aristotle. The pious bible thumper parroting ideas published by Plato.

The point is that philosophy is not the same thing as religion and philosophy directly shapes how people think, what they believe and therefore how they act and behave. And it's kind of uncanny how an understanding of philosophy can place historical events in context and what kinds of predictive capabilities it has when it comes to human behaviour in the aggregate.

This sounds very educated but I don't really see what it has to do with the comment you're responding to (or with AI).

While it's a fair criticism, just because someone doesn't believe in a god doesn't mean the religious hardware in their brain has been turned off. It's still there and operational - I don't think it's a surprise that this hardware's attention would then be automatically tuned to a different topic.

I think you can also see this in the intensification of political discussion, which has a similar intensity to religious discussions 100-200+ years ago (i.e. Protestant reformation). Indicating that this "religious hardware" has shifted domains to the realm of politics. I believe this shift can also be seen through the intense actions and rhetoric we saw in the mid-20th century.

You can also look at all of these new age "religions" (spiritualism, horoscopes, etc.) as that religious hardware searching for something to operate on in the absence of traditional religion.

I agree that modern hyper-online moralist progressivism and QAnonism are just fresh coats of paint on religion, but that isn't similar to AI.

AI isn't a worldview; it's an extremely powerful tool which some people happen to be stronger at using than others, like computers or fighter jets. For people who empirically observe that they've been successful at extracting massive amounts of value from the tool, it's easy to predict a future in which aggregate economic output in their field by those who are similarly successful will dwarf that of those who aren't. For others, it's understandable that their mismatched experience would lead to skepticism of the former group, if not outright comfort in the idea that such productivity claims are dishonest or delusional. And then of course there are certainly those who are actually lying or deluded about fitting in the former group.

Every major technology or other popular thing has some subset of its fandom which goes too far in promotion of the thing to a degree that borders on evangelical (operating systems, text editors, video game consoles, TV shows, diets, companies, etc.), but that really has nothing to do with the thing itself.

Speaking for myself, anecdotally, I've recently been able to deliver a product end-to-end on a timeline and level of quality/completeness/maturity that would have been totally impossible just a few years ago. The fact that something has been brought into existence in substantially less time and at orders of magnitude lower cost than would have been required a few years ago is an undeniable observation of the reality in front of me, not theological dogma.

It is, however, a much more cognitively intense way to build a product — with AI performing all the menial labor parts of development, you're boxed into focusing on the complex parts in a far more concentrated time period than would otherwise be required. In other words, you no longer get the "break" of manually coding out all the things you've decided need to be done and making every single granular decision involved. You're working at a higher level of abstraction and your written output for prompting is far more information-dense than code. The skills required are also a superset of those required for manual development; you could be the strongest pre-LLM programmer in the world, but if you're lacking in areas like human language/communication, project/product management, the ability to build an intuition for "AI psychology", or thinking outside the box in how you use your tools, adapting to AI is going to be a struggle.

It's like an industry full of mechanics building artisan vehicles by hand suddenly finding themselves foisted with budgets to design and implement assembly lines; they still need to know how to build cars, but the nature of the job has now fundamentally changed, so it's unsurprising that many or even most who'd signed up for the original job would fail to excel in the new job and rationalize that by deciding the old ways are the best. It's not fair, and it's not anyone's fault, but it's important for us all to be honest and clear-eyed about what's really happening here. Society as a whole will ultimately enjoy some degree of greater abundance of resources, but in the process a lot of people are going to lose income and find hard-won skills devalued. The next generation's version of coal miners being told to "learn to code" will be coders being told to "learn to pilot AI".

> It's not fair, and it's not anyone's fault, but it's important for us all to be honest and clear-eyed about what's really happening here.

Or we can just refuse this future and act as a society to prevent it from happening. We absolutely have that power, if we choose to organize and use it.

Sure, but how so? If I'm understanding your argument correctly, it sounds like you may be implying that we should escalate the war on general-purpose computing and outlaw generative AI.

If we were to consider that, then to what end? If you accept my framing of the long-term implications of LLMs on the industry, then what you're suggesting is effectively that we should deprive society of greater prosperity for the benefit of a small minority. Personally, I'd rather improve democratization of entrepreneurship (among other things) than artificially prop up software engineering salaries.

And let's say the US did all that. What then? We neuter our economy and expect our adversaries to just follow suit? More likely it hobbles our ability to compete and ultimately ushers in an era of global hegemony under the CCP.

Which then leads you to the question "who installed the hardware"?

No, that lead you to that question.

It leads me to the question, "Is it really 'religious hardware' or the same ol' 'make meaning out of patterns' hardware we've had for millenia that has allowed us to make shared language, make social constructs, mutually believe legal fictions that hold together massive societies, etc.?"

> It leads me to the question, "Is it really 'religious hardware' or the same ol' 'make meaning out of patterns' hardware

They are the same thing. Call it "religion" or "meaning making," both activities can be subsumed by the more encompassing concept and less-loaded term of "psycho-technology," [0] or non-physical tools for the mind.

Language is such a psycho-technology, as are social constructs such as law; legal fictions are given memorable names and personified into "religious" figures, such as Libra from astrology or Themis/Lady Justice from Greek mythology.

Ancient shamans and priests were proto-wetware engineers, designing software for your brain and providing tools for making meaning out of the world. In modern day we now have psychologists, "social commentators" (for lack of a better term and interpreted as broadly as possible), and, yes, software engineers, amongst other disciplines, playing a similar role.

[0] https://www.meaningcrisis.co/episode-1-introduction/

Or: the hardware that generates beliefs about how things should be - whether based on religious or ideological dogma -, as opposed to science which is not prescriptive and can only describe how things are.

Your entire outlook is based on an assumption. The assumption that 'emergence of meaning' is a 2nd order epiphenomena of an organic structure. The 1st order epiphenomena in your view is of course consciousness itself.

None of these assumptions can be proven, yet like the ancients looking at the sky and seeing a moving sun but missing a larger bit of the big picture you now have a 'theory of mind' that satisfies your rational impluses given a poor diet of facts and knowledge. But hey, once you manage to 'get into orbit' you get access to more facts and then the old 'installed hardware' theory of yours starts breaking down.

The rational position regarding these matters is to admit "we do not have sufficient information and knowledge to make conclusive determinations based on reason alone". Who knows, one day Humanity may make it to the orbit and realize the 'simple and self apparent idea' of "everything revoles around the Earth" is false.

I've enjoyed reading the books of Peter Watts (Blindsight, free on their backlog, sci-fi), on seemingly this subject

https://en.wikipedia.org/wiki/Blindsight_(Watts_novel) (will check it out. thanks!)

I've read LessWrong very differently from you. The entire thrust of that society is that humanity is going to create the AI god.

I didn’t say that “it’s just a new god,” I said:

The notion that history is just on some inevitable pre-planned path is not a new idea, except now the actor has changed from God to technology.

This is a more nuanced sentence.

Before that quoted sentence you drew a line from the reformation to people believing that AI is inevitable, then went on to imply these people may even believe such a thing will happen without the involvement of people. These are generalizations which don't fit a lot of the literature and make their best ideas look a bit sillier than they are. It is situations like these that make me think that analogies are better suited as a debate tactic than a method of study.

Maybe not a god, but we're intentionally designing artificial minds greater than ours, and we intend to give them control of the entire planet. While also expecting them to somehow remain subservient to us (or is that part just lip service)?

I’m sorry, but are you arguing that an LLM is anywhere near a human mind? Or are you arguing about some other AI?

If you understand the cultural concepts of Adam Curtis’s All Watched Over by Machines of Loving Grace, then yes we do keep trying to make gods out of inanimate things.

And it’s the atheists who continuously do it, claiming they don’t believe in God just markets or ai etc.

It’s an irony of ironies.

Would you say LessWrong posts are dogmatic?

I jsut want to comment here that this is the classic arrogant, underread “I reject half of humanities thoughts” foolishness that OP is referring to.

I mean the lack of self awareness you have here is amazing.

To the contrary. I sped through my compsci capstone coursework first year of college and spent most of the rest of my time in philosophy, psychology, and sociology classrooms. The "hey if you squint this thing it looks like religion for the non-religious" perspective is just one I've heard countless times. It is perfectly valid to have a fact based discussion on whether there is a biological desire for religiosity, but drawing a long line from that to broadly critique someone's well-articulated ideas is pretty sloppy.

Quoting your college classes is the first sign of inexperience but I’ll Share some modern concepts.

In Adam Curtis‘s all watched over by machines of loving Grace, he makes a pretty long and complete argument that humanity has a rich history of turning over its decision-making to inanimate objects in a desire to discover ideologies we can’t form ourselves in growing complexity of our interconnectivity.

He tells a history of them constantly failing because the core ideology of “cybernetics” is underlying them all and fails to be adaptive enough to match our DNA/Body/mind combined cognitive system. Especially when scaled to large groups.

He makes the second point that humanity and many thinkers constantly also resort to the false notion of “naturalism” as the ideal state of humanity, when in reality there is no natural state of anything, except maybe complexity and chaos.

Giving yourself up to something. Specially something that doesn’t work is very much “believing in a false god.”

You seem to be lost. While referencing a TV show may or may not be a rebuttal to a very specific kind of worldview, it is out of place as a response to my post to which you've failed to actually directly reference at all.

I'm addressing this point at you personally because we can all see your comments: being nasty to atheists on the internet will never be a substitute for hard evidence for your ideology.

you seem to be profoundly confused Adam Curtis is a leading thinker in documentarian of our time and widely recognized in continental philosophy. The fact that you tried to dismiss him as a TV show shows you seem to be completely naïve about the topic you’re speaking about.

Second, I’m not being nasty to atheists and speaking specifically about not having false gods which if anything is a somewhat atheistic perspective

Honestly, what are you trying to say?

Like I said, we can all read your comments. Needs no further elaboration. If I receive a second recommendation for Curtis then I might be inclined to check it out. Take it easy.

Techno Calvinists vs Luddite Reformists is a very funny image.

Agree - although it's an interesting view, I think it's far more related to a lack of idealogy and writing where this has emerged from. I find it more akin to a distorted renaissance. There's such a large population of really intelligent tech people that have zero real care for philisophical or religious thought, but still want to create and make new things.

This leads them down the first path of grafting for more and more money. Soon, a good proportion of them realise the futility of chasing cash beyond a certain extent. The problem is this belief that they are beyond these issues that have been dealt with since Mesopotamia.

Which leads to these weird distorted idealogies, creating art from regurgitated art, creating apps that are made to become worse over time. There's a kind of rush to wealth, ignoring the joy of making things to further humanity.

I think LLMs and AI is a genie out of a bottle, it's inevitable, but it's more like linear perpsective in drawing or the printing press rather than electricity. Except because of the current culture we live in, it's as if leonardo spent his life attempting to sell different variations of linear perspective tutorial rather than creating, drawing and making.

in Adam Curtis‘s all watched over by machines of loving Grace, he makes a pretty long and complete argument that humanity has a rich history of turning over its decision-making to inanimate objects in a desire to discover ideologies we can’t form ourselves in growing complexity of our interconnectivity.

He tells a history of them constantly failing because the core ideology of “cybernetics” is underlying them all and fails to be adaptive enough to match our DNA/Body/mind combined cognitive system. Especially when scaled to large groups.

He makes the second point that humanity and many thinkers constantly also resort to the false notion of “naturalism” as the ideal state of humanity, when in reality there is no natural state of anything, except maybe complexity and chaos.

100%. Not a new phenomenon at all, just the latest bogeyman for the inevitabilists to point to in their predestination arguments.

My aim is only to point it out - people are quite comfortable rejecting predestination arguments coming from eg. physics or religion, but are still awed by “AI is inevitable”.

It's inevitable not because of any inherent quality of the tech, but because investors are demanding it be so and creating the incentives for 'inevitability'.

I also think EV vehicles are an 'inevitability' but I am much less offended by the EV future, as they still have to outcompete IC's, there are transitional options (hybrids), there are public transport alternatives, and at least local regulations appear to be keeping pace with the technical change.

AI inevitabilty so far seems to be only inevitable because I can't actually opt out of it when it gets pushed on me.

To use John Adams' separation of republics into the categories of "the many, the few, and the one," the few in our current day are unusually conflict-adverse both among each other and with respect to the people.

When faced with the current crisis, they look at the options for investment and they see some that will involve a lot of conflict with the many (changing the industrial employment arrangement, rearranging state entitlements), and they see see some that avoid conflict or change. Our few as they are got that way by outsourcing anything physical and material as much as possible and making everything "into computer." So they promote a self serving spiritual belief that because overinvesting in computers got them to their elevated positions, that even more computer is what the world needs more than anything else.

This approach also mollifies the many in a way that would be easily recognizable in any century to any classically educated person. Our few do not really know what the many are there for, but they figure that they might as well extract from the many through e.g. sports gambling apps and LLM girlfriends.

The article's main point is that "inevitabilism" is a rhetorical tactic used to frame the conversation in such a way you can easily dismiss any criticism as denying reality. So drawing comparisons to reformation ideology wouldn't be particularly meaningful.

There's a also a bit of irony that you're presenting the secular view of predestination. As someone who once had a multi-volume set of "Institutes of the Christian Religion" next to him on his bookshelf, the protestant conception of predestination had very little to do with "offloading of freedom and responsibility" both in theory and in practice.

Predestination is founded on the concept that God's grace is given not earned (unlike the previous Catholic system which had multiple ways that merit, including cash donations, could be converted into salvation), since no human could earn salvation without the grace of God. But the lesson from this is not "so don't worry about it!", quite the opposite. Calvin's main extension to this was that (paraphrasing) "It's not through good works that we are saved, but through our good works we have evidence of our salvation". You wanted to see the evidence of your salvation, so you did try to do good works, but without the belief that your efforts would ever be enough. This ultimately created a culture of hard work with out the expectation of reward.

This is part of the focus of Max Weber's "The Protestant Ethic and the Spirit of Capitalism" which argued that this ability to "work without immediate reward" is precisely what enabled Capitalism to take such a strong foot hold in the early United States.

So even if the article were arguing for "inevitabilism" the framework is still quite distinct from that established in Protestantism.

> God's grace is given not earned (unlike the previous Catholic system ...

Catholicism does not hold that you can earn grace. Grace is a gift from God that is freely given.

> including cash donations, could be converted into salvation

I assume you are referring to selling indulgences. Indulgences are not something that can give you salvation.

Or historicism generally. Hegel, "inexorable laws of historical destiny", that sort of thing.

I think this is a case of bad pattern matching, to be frank. Two cosmetically similar things don't necessarily have a shared cause. When you see billions in investment to make something happen (AI) because of obvious incentives, it's very reasonable to see that as something that's likely to happen; something you might be foolish to bet against. This is qualitatively different from the kind of predestination present in many religions where adherents have assurance of the predestined outcome often despite human efforts and incentives. A belief in a predestined outcome is very different from extrapolating current trends into the future.

Yes, nobody is claiming it's inevitable based on nothing, it's based on first principles thinking: economics, incentives, game theory, human psychology. Trying to recast this in terms of "predestination" gives me strong wordcel vibes.

It's a bit like pattern matching the Cold War fears of a nuclear exchange and nuclear winter to the flood myths or apocalyptic narratives across the ages, and hence dismissing it as "ah, seen this kind of talk before", totally ignoring that Hiroshima and Nagasaki actually happened, later tests actually happened, etc.

It's indeed a symptom of working in an environment where everything is just discourse about discourse, and prestige is given to some surprising novel packaging or merger of narratives, and all that is produced is words that argue with other words, and it's all about criticizing how one author undermines some other author too much or not enough and so on.

From that point of view, sure, nothing new under the sun.

It's all too well to complain about the boy crying wolf, but when you see the pack of wolves entering the village, it's no longer just about words.

Now, anyone is of course free to dispute the empirical arguments, but I see many very self-satisfied prestigious thinkers who think they don't have to stoop so low as to actually look at models and how people use them in reality, it can all just be dismissed based on ick factors and name calling like "slop".

Few are saying that these things are eschatological inevitabilities. They are saying that there are incentive gradients that point in a certain direction and it cannot be moved out from that groove without massive and fragile coordination, due to game theoretical reasonings, given a certain material state of the world right now out there, outside the page of the "text".

I think you’re missing the point of the blog post and the point of my grandparent comment, which is that there is a pervasive attitude amongst technologists that “it’s just gonna happen anyway and therefore whether I work on something negative for the world or not makes no difference, and therefore I have no role as an ethical agent.” It’s a way to avoid responsibility and freedom.

We are not discussing the likelihood of some particular scenario based on models and numbers and statistics and predictions by Very Smart Important People.

I agree that "very likely" is not "inevitable". It is possible for the advance of AI to stop, but difficult. I agree that doesn't absolve people of responsibility for what they do. But I disagree with the comparison to religious predestination.

I'm not sure how common that is... I'd guess most who work on it think that there's a positive future with LLMs also. I mean they likely wouldn't say "I work on something negative for the world".

I think the vast majority of people are there because it’s interesting work and they’re being paid exceptionally well. That’s the extent to which 95/100 of employees engage with the ethics of their work.

Nobody serious is claiming theological predesination is based on "nothing", either. Talk about poor pattern matching.

You are, of course, entitled to your religious convictions. But to most people outside of your religious community, the evidence for some specific theological claim (such as predestination) looks an awful lot like "nothing". In contrast, claims about the trajectory of AI (whether you agree with the claims or not) are based on easily-verifiable, public knowledge about the recent history of AI development.

It is not a "specific theological claim" either, rather a school of theological discourse. You're literally doing free-form association now and pretending to have novel insights into centuries of work on the issue.

I'm not pretending to any novel insights. Most of us who don't have much use for theology are generally unimpressed by its discourse. Not novel at all. And the "centuries of work" without concrete developments that exist outside of the minds of those invested in the discourse is one reason why many of us are unimpressed. In contrast, AI development is resulting in concrete changes that are easily verified by anyone and on much shorter time scales.

Relatedly, it would be bordering on impossible to convince Iran about the validity of Augustine, Aquinas or Calvin, but it was fairly easy with nuclear physics. Theology isn't "based on nothing", but the convincing power of the quantum physics books happens to be radically different from Summa Theologiae, even if both are just books written by educated people based on a lot of thought and prior work.

> many very intelligent, thoughtful people feel justified in brushing away millennia of philosophical and religious thought because they deem it outdated

Why lump philosophy and religion together? I distinguish between philosophical thought and religious thought, to the extent the former is conditionally framed.

> One of the negative consequences of the “modern secular age” is that many very intelligent, thoughtful people feel justified in brushing away millennia of philosophical and religious thought because they deem it outdated or no longer relevant.

Isn't that a societal trait though? See English Christians attitude towards vikings, requiring baptism (or the prima signatio, kinda baptism-light) before they could deal with them, because they were savage. Or colons forcing natives to adopt Christianity, because what they had before was "primitive". There was wisdom and thought in both, but in both case the Christian side "brushed it away". Or capitalism and communism in the cold war. It feels like everyone with a belief system tries to force it onto others.

Sorry I don't buy your argument.

(First I disagree with A Secular Age's thesis that secularism is a new force. Christian and Muslim churches were jailing and killing nonbelievers from the beginning. People weren't dumber than we are today, all the absurdity and self-serving hypocrisy that turns a lot of people off to authoritarian religion were as evident to them as they are to us.)

The idea is not that AI is on a pre-planned path, it's just that technological progress will continue, and from our vantage point today predicting improving AI is a no brainer. Technology has been accelerating since the invention of fire. Invention is a positive feedback loop where previous inventions enable new inventions at an accelerating pace. Even when large civilizations of the past collapsed and libraries of knowledge were lost and we entered dark ages human ingenuity did not rest and eventually the feedback loop started up again. It's just not stoppable. I highly recommend Scott Alexander's essay Meditations On Moloch on why tech will always move forward, even when the results are disastrous to humans.

That isn’t the argument of the book, so I don’t think you actually read it, or even the Wikipedia page?

The rest of your comment doesn’t really seem related to my argument at all. I didn’t say technological process stops or slows down, I pointed out how the thought patterns are often the same across time, and the inability and unwillingness to recognize this is psychologically lazy, to over simplify. And there are indeed examples of technological acceleration or dispersal which was deliberately curtailed – especially with weapons.

> I pointed out how the thought patterns are often the same across time, and the inability and unwillingness to recognize this is psychologically lazy, to over simplify.

It's not lazy to follow thought patterns that yield correct predictions. And that's the bedrock on which "AI hype" grows and persists - because these tools are actually useful, right now, today, across wide variety of work and life tasks, and we are barely even trying.

> And there are indeed examples of technological acceleration or dispersal which was deliberately curtailed – especially with weapons.

Name three.

(I do expect you to be able to name three, but that should also highlight how unusual that is, and how questionable the effectiveness of that is in practice when you dig into details.)

Also I challenge you to find but one restriction that actually denies countries useful capabilities that they cannot reproduce through other means.

Doesn’t seem that rare to me – chemical, biological, nuclear weapons are all either not acceptable to use or not even acceptable to possess. Global governments go to extreme lengths to prevent the proliferation of nuclear weapons. If there were no working restrictions on the development of the tech and the acquisition of needed materials, every country and large military organization would probably have a nuclear weapons program.

Other examples are: human cloning, GMOs or food modification (depends on the country; some definitely have restricted this on their food supply), certain medical procedures like lobotomies.

I don’t quite understand your last sentence there, but if I understand you correctly, it would seem to me like Ukraine or Libya are pretty obvious examples of countries that faced nuclear restrictions and could not reproduce their benefits through other means.

I can't make a nuclear or chemical weapon on my gaming graphics card from 5 years ago.

The same is not true about LLMs.

No, LLMs aren't going to be stopped when anyone with a computer from the last couple years is able to run them on their desktop. (There are smaller LLMs that can be even run on your mobile phone!).

The laws required to stop this would be draconian. It would require full government monitoring of all computers. And any country or group that "defects" by allowing people to use LLMs, would gain a massive benefit.

> I can't make a nuclear or chemical weapon on my gaming graphics card from 5 years ago.

You make be surprised to learn that you can make a chemical weapon on your gaming graphics card from 5 years ago.

It's just that it will void the warranty well before you have a meaningful quantity of chlorine gas from the salt water you dunked it in while switched on.

Yup. The government of the world could shut down all LLM providers tomorrow, and it wouldn't change a thing - LLMs fundamentally are programs, not a service. There are models lagging 6-12 months behind current SOTA, that you can just download and run on your own GPU today; most research is in the open too, so nothing stops people from continuing it and training new models locally.z

At this point, AI research is not possible to stop without killing humanity as technological civilization - and it's not even possible to slow it down much, short of taking extreme measures Eliezer Yudkowsky was talking about years ago: yes, it would literally take a multinational treaty on stopping advanced compute, and aggressively enforcing it - including (but not limited to) by preemptively bombing rogue data centers as they pop up around the world.

> Global governments go to extreme lengths to prevent the proliferation of nuclear weapons. If there were no working restrictions on the development of the tech and the acquisition of needed materials, every country and large military organization would probably have a nuclear weapons program.

Nuclear is special due to MAD doctrine; restrictions are aggressively enforced for safety reasons and to preserve status quo, much more so than for moral reasons - and believe me, every country would love to have a nuclear weapons program, simply because, to put it frankly, you're not fully independent without nukes. Nuclear deterrent is what buys you strategic autonomy.

It's really the one weird case where those who got there first decided to deny their advantage to others, and most others just begrudgingly accept this state of affairs - as unfair as it is, it's the local equilibrium in global safety.

But that's nukes, nukes are special. AI is sometimes painted like the second invention that could become special in this way, but I personally doubt it - to me, AI is much more like biological weapons than nuclear ones: it doesn't work as a deterrent (so no MAD), but is ideal for turning a research mishap into an extinction-level event.

> Other examples are: human cloning, GMOs or food modification (depends on the country; some definitely have restricted this on their food supply), certain medical procedures like lobotomies.

Human cloning - I'd be inclined to grant you that one, though I haven't checked what's up with China recently. GMO restrictions are local policy issues, and don't affect R&D on a global scale all that much. Lobotomy - fair. But then it didn't stop the field of neurosurgery at all.

> I don’t quite understand your last sentence there, but if I understand you correctly, it would seem to me like Ukraine or Libya are pretty obvious examples of countries that faced nuclear restrictions and could not reproduce their benefits through other means.

Right, the invasion of Ukraine is exactly why no nuclear-capable country will even consider giving nukes up. This advantage cannot be reproduced through other means in enough situations. But I did mean it more generally, so let me rephrase it:

Demand begets supply. If there's a strong demand for some capability, but the means of providing it are questionable, then whether or not they can be successfully suppressed depends on whether there are other ways of meeting the demand.

Nuclear weapons are, again, special - they have no substitute, but almost everyone gains more from keeping the "nuclear club" closed than from joining it. But even as there are international limits, just observe how far nations go to skirt them to keep the R&D going (look no further than NIF - aka. "let's see far we can push nuclear weapons research if we substitute live tests with lasers and a lot of computer simulations").

Biological and chemical weapons are effectively banned (+/- recent news about Russia), but don't provide unique and useful capabilities on a battlefield, so there's not much demand for them.

(Chemical weapons showing up in the news now only strengthens the overall point: it's easy to refrain from using/developing things you don't need - but then restrictions and treaties fly out the window the moment you're losing and run out of alternatives.)

Same for full-human cloning - but there is demand for transplantable organs, as well as better substrate for pharmaceutical testing; the former can be met cheaper through market and black market means, while the latter is driving several fields of research that are adjacent to human cloning, but more focused on meeting the actual demand and coincidentally avoid most of the ethical concerns raised.

And so on, and so on. Circling back to AI, what I'm saying is, AI is already providing too much direct, object-level utility that cannot be substituted by other means (itself being a cheaper substitute for human labor). The demand is already there, so it's near-impossible to stop the tide at this point. You simply won't get people to agree on this.

I add to this that we have plenty of examples of societies that don't keep up with technological advancement, or "history" more broadly get left behind. Competition in a globalized world makes some things inevitable. I'm not agreeing in full with the most AI will change everything arguments, but those last couple of paragraphs of TFA sounds to me like standing athwart history, yelling "Stop!".

Communism used to be thought of in this way. It enabled societies to cast off old limitations and make remarkable progress. Until it didn't and Communists found themselves and their modernized society stuck well behind the rest of the world. Perhaps LLMs are a similar trap that will generate many lines of code and imagined images but leave us all stupid and with impaired executive function.

> the actor has changed from God to technology

Agreed. You could say that technology has become a god to those people.

What technology? Agriculture? The steam engine? The automobile? Modern medicine? Cryptography? The Internet? LLMs? Nanotechnology?

Who are these people? Jonas Salk, widely credited as the inventor of the polio vaccine? Sam Altman, fundraiser extraordinaire? Peter Thiel, exalter of The World-Saving Founders? Ray Kurzweil? Technocrats? Other techno-optimists? Perhaps transhumanists? There are many variations, and they differ by quite a lot.

What kind of god? Carl Sagan has a nice interview where he asks a question-asker to define what they mean by “god”. A blind watchmaker? Someone who can hear your prayers? A wrathful smoter of the wicked and (sometimes) the loyal (sorry, Job!)? A very confusing 3-tuple, one element of which birthed another, who died somehow but was resurrected? The essence of nature? The laws of physics? An abstract notion of love? Yeah. These three letters are too vague to be useful unless unpacked or situated in a mutually understood context. It often fosters a flimsy consensus or a shallow disagreement.

It actually seems more to me like dialectical materialism, which started centuries ago and was already secular. It bears more in character to the differences that other commenters have already voiced, in that human actors not only believed in its inevitability, but attempted to bring it about themselves. Multiple global superpowers implemented forced industrialization, cultural reformation, and command economies to bring it about.

The difference this time isn't sacred versus secular. It's public versus private. Whereas the purveyors of communism were governments, this is being done by corporations. Well-funded private organizations are led by decision makers who believe strongly this is the future, it is inevitable, and their only hope is to get there first. The actor didn't change from God to technology. It changed from labor to capital.

I make no comment on whether they will prove to be more correct than the believers in communism, but the analogy is obvious either way.

I kinda feel this way too. Reading some of the blog posts by AI "luminaries" I'm struck by how Stalinist they sound. They hold out some utopia that exists in their minds, and they are ready to feed people into the meat grinder to try and make it a reality. Stalin said that this generation would suffer so that the next lived in utopia, and that's kind of the same pitch they are making.

I think if we actually cared about making a better world, you'd take steps where each successive step is a positive one. Free healthcare, clean energy investments, etc..

> I think if we actually cared about making a better world, you'd take steps where each successive step is a positive one.

Yeah, but lots of people don't care about that, they care about acheiving their visions of power, and they need an excuse to justify other people suffering for them. They aren’t seeking long term improvements at the cost of short term suffering, they are using a mirage of utopia over the hill to sell people a deal which is only suffering, now and for however long they can be kept in line.

Why look to the past when you can rediscover it from “first principles?” /s

This is one of those types of comments to change one's whole world view.

> The notion that history is just on some inevitable pre-planned path is not a new idea, except now the actor has changed from God to technology.

I'm gonna fucking frame that. It goes hard

This entire conversation is a masterpiece!

Just picture this convo somewhere in nature, at night, by a fire.

If in 2009 you claimed that the dominance of the smartphone was inevitable, it would have been because you were using one and understood its power, not because you were reframing away our free choice for some agenda. In 2025 I don't think you can really be taking advantage of AI to do real work and still see its mass adaptation as evitable. It's coming faster and harder than any tech in history. As scary as that is we can't wish it away.

If you claimed that AI was inevitable in the 80s and invested, or claimed people would be inevitably moving to VR 10 years ago - you would be shit out of luck. Zuck is still burning billions on it with nothing to show for it and a bad outlook. Even Apple tried it and hilariously missed the demand estimate. The only potential bailout for this tech is AR, but thats still years away from consumer market and widespread adoption, and probably will have very little to do with shit that is getting built for VR, because its a completely different experience. But I am sure some of the tech/UX will carry over.

Tesla stock has been riding on the self driving robo-taxies meme for a decade now ? How many Teslas are earning passive income while the owner is at work ?

Cherrypicking the stuff that worked in retrospect is stupid, plenty of people swore in the inevitability of some tech with billions in investment, and industry bubbles that look mistimed in hindsight.

None of the "failed" innovations you cited were even near the adoption rate of current LLMs.

As much as I don't like it, this is the actual difference. LLMs are already good enough to be a very useful and widely spread technology. They can become even better, but even if they don't there are plenty of use cases for them.

VR/AR, AI in the 80s and Tesla at the beginning were technology that someone believe could become widespread, but still weren't at all.

That's a big difference

The other inventions would have quite the adoption rate if they were similarly subsidized as current AI offerings. It's hard to compare a business attempting to be financially stable and a business attempting hyper-growth through freebies.

> The other inventions would have quite the adoption rate if they were similarly subsidized as current AI offerings.

No, they wouldn't. The '80s saw obscene investment in AI (then "expert systems") and yet nobody's mom was using it.

> It's hard to compare a business attempting to be financially stable and a business attempting hyper-growth through freebies.

It's especially hard to compare since it's often those financially stable businesses doing said investments (Microsoft, Google, etc).

---

Aside: you know "the customer is always right [in matters of taste]"? It's been weirdly difficult getting bosses to understand the brackets part, and HN folks the first part.

> Aside: you know "the customer is always right [in matters of taste]"? It's been weirdly difficult getting bosses to understand the brackets part, and HN folks the first part.

Something I struggle to internalise, even though I know it in theory.

Customers can't be told they're wrong, and the parenthetical I've internalised, but for non-taste matters they can often be so very wrong, so often… I know I need to hold my tongue even then owing to having merely nerd-level charisma, but I struggle to… also owing to having merely nerd-level charisma.

(And that's one of three reasons why I'm not doing contract work right now).

I don't think you understand the relative amounts of capital invested in LLMs compared to expert systems in the 80s.

And those systems were never "commodified" - your average mom is forcefully exposed to LLMs with every google search, can interact with LLMs for free instantly anywhere in the world - and we're comparing to a luxury product for nerds basically?

Not to forget that those massive companies are also very heavy in advertising - I don't think your average mom in the 80s heard of those systems multiple times a day, from multiple aquaintances AND social media and news outlets.

Most people are using LLMs because they fear that it will be the future and they will miss out if they do not learn it now although they are aware they are not more productive but can’t say that in a business environment.

They really wouldn't. Even people who BOUGHT VR, are barely using it. Giving everyone free VR headsets won't make people suddenly spend a lot of time in VR-land without there actually being applications that are useful to most people.

ChatGPT is so useful, people without any technology background WANT to use it. People who are just about comfortable with the internet, see the applications and use it to ask questions (about recipes, home design, solving small house problems, etc).

The lack of adoption for those wasn't (just) the price. They just weren't very useful.

OK but what does adoption rate vs. real world impact tell here ?

With all the insane exposure and downloads how many people cant even be convinced to pay 20$/month for it ? The value proposition to most people is that low. So you are basically betting on LLMs making a leap in performance to pay for the investments.

> They can become even better, but even if they don't there are plenty of use cases for them.

If they don't become better we are left with a big but not huge change. Productivity gains of around 10 to 20 percent in most knowledge work. That's huge for sure but in my eyes the internet and pc revolution before that were more transformative than that. If LLMs become better, get so good they replace huge chunks of knowledge workers and then go out to the physical world then yeah ...that would be the fastest transformation of the economy in history imo.

FWIW, LLMs have been getting better so fast that we only barely begun figuring out more advanced ways of applying them. Even if they were to plateau right now, there'd still be years of improvements coming from different ways of tuning, tweaking, combining, chaining and applying them - which we don't invest much into today, because so far it's been cheaper to wait a couple months for the next batch of models that can handle what previous could not.

I don’t see this as that big a difference, of course AI/LLMs are here to stay, but the hundreds in billions of bets on LLMs don’t assume linear growth.

> None of the "failed" innovations you cited were even near the adoption rate of current LLMs.

The 'adoption rate' of LLMs is entirely artificial, bolstered by billions of dollars of investment in attempting to get people addicted so that they can siphon money off of them with subscription plans or forcing them to pay for each use. The worst people you can think of on every c-suite team force pushes it down our throats because they use it to write an email every now and then.

The places LLMs have achieved widespread adoption is in environments abusing the addictive tendencies of a advanced stochastic parrot to appeal to lonely and vulnerable individuals to massive societal damage, by true believers that are the worst coders you can imagine shoveling shit into codebases by the truckful and by scammers realizing this is the new gold rush.

Oh, it gets worse. The next stage is sort of a dual mode of personhood: AI is 'person' when it's about impeding the constant use of LLMs for all things, so it becomes anathema to deny the basic superhumanness of the AI.

But it's NOT a person when it's time to 'tell the AI' that you have its puppy in a box filled with spikes and for every mistake it makes you will stab it with the spikes a little more and tell it the reactions of the puppy. That becomes normal, if it elicits a slightly more desperate 'person' out of the AI for producing work.

At which point the meat-people who've taught themselves to normalize this workflow can decide that opponents of AI are clearly so broken in the head as to constitute non-player characters (see: useful memes to that effect) and therefore are NOT people: and so, it would be good to get rid of the non-people muddying up the system (see: human history)

Told you it gets worse. And all the while, the language models are sort of blameless, because there's nobody there. Torturing an LLM to elicit responses is harming a person, but it's the person constructing the prompts, not a hypothetical victim somewhere in the clouds of nobody.

All that happens is a human trains themselves to dehumanize, and the LLM thing is a recipe for doing that AT SCALE.

Great going, guys.

The people claiming that AI in the 80s or VR or robotaxis or self-driving cars in the 2010s were inevitable weren't doing it on the basis of the tech available at that point, but on the assumed future developments. Just a little more work and they'd be useful, we promise. You just need to believe hard enough.

With the smartphone in 2009, the web in the late 90s or LLMs now, there's no element of "trust me, bro" needed. You can try them yourself and see how useful they are. You didn't need to be a tech visionary to predict the future when you're buying stuff from Amazon in the 90s, or using YouTube or Uber on your phone in 2009, or using Claude Code today. I'm certainly no visionary, but both the web and the smartphone felt different from everything else at the time, and AI feels like that now.

LLM inevitablists definitely assume future developments will improve their current state.

Yes, but the difference from the others, and the thing it has in common with early smartphones and the web, is that it's already useful (and massively popular) today.

And self driving is a great lane assist. There's a huge leap from that to driving a taxi while you are at work is same as LLMs saving me mental effort with instructions on what to do and solving the task for me completely.

What uses are you finding for it in the real world? I've found them nothing but unreliable at best, and quite dangerous at worst in terms of MCP and autonomous agents. Definitely not ready for production, IMO. I don't think they ever will be for most of what people are trying to use them for.

"Novelty" comes to mind.

Yes, LLMs are currently useful and are improving rapidly so they are likely to become even more useful in the future. I think inevitable is a pretty strong word but barring government intervention or geopolitical turmoil I don't see signs of LLM progress stopping.

Why would they progress significantly than where they are now? An LLM is an LLM. More tokens doesn't mean better capabilities, in fact, quite the opposite seems to be the case, and suggests smaller models aimed at specific tasks are the "future" of it.

>Tesla stock has been riding on the self driving robo-taxies meme for a decade now

We do have self-driving taxis now, and they are so good that people will pay extra to take them. It's just not Tesla cars doing it.

Yes, and yet the rate of development and deployment is substantially slower than people like me were expecting.

Back in 2009, I was expecting normal people to be able to just buy a new vehicle with no steering wheel required or supplied by 2019, not for a handful of geo-fenced taxis that slowly expanded over the 6 years from 2019 to 2025.

https://www.youtube.com/watch?v=zhr6fHmCJ6k (1min video, 'Elon Musk's broken promises')

Musk's 2014/2015 promises are arguably delivered, here in 2025 (took a little more than '1 month' tho), but the promises starting in 2016 are somewhere between 'undelivered' and 'blatant bullshit'.

I mean no argument here - but the insane valuation was at some point based on a fleet of self driving cars based on cars they don't even have to own - overtaking Uber. I don't think they are anywhere close to that. (It's hard to keep track what it is now - robots and AI ?) Kudos for hype chasing all these years tho. Only beaten by Jensen on that front.

Ironically, this is exactly the technique for arguing that the blog mentions.

Remember the revolutionary, seemingly inevitable tech that was poised to rewrite how humans thought about transportation? The incredible amounts of hype, the secretive meetings disclosing the device, etc.? That turned out to be the self-balancing scooter known as a Segway?

> Remember ...

No, I don't remember it like that. Do you have any serious sources from history showing that Segway hype is even remotely comparable to today's AI hype and the half a trillion a year the world is spending on it?

You don't. I love the argument ad absurdum more than most but you've taken it a teensy bit too far.

People genuinely did suggest that we were going to redesign our cities because of the Segway. The volume and duration of the hype were smaller (especially once people saw how ugly the thing was) but it was similarly breathless.

> Do you have any serious sources from history showing that Segway hype is even remotely comparable to today's AI hype and the half a trillion a year the world is spending on it?

LLM are more useful than Segway, but it can still be overhyped because the hype is so much larger. So its comparable, as you say LLM is so much more hyped doesn't mean it can't be overhyped.

I get immense value out of LLMs already, so it's hard for me to see them as overhyped. But I get how some people feel that way when others start talking about AGI or claiming we're close to becoming the inferior species.

1. The Segway had very low market penetration but a lot of PR. LLMs and diffusion models have had massive organic growth.

2. Segways were just ahead of their time: portable lithium-ion powered urban personal transportation is getting pretty big now.

Massive, organic, and unprofitable. And as soon as it's no longer free, as soon as the VC funding can no longer sustain it, an enormous fraction of usage and users will all evaporate.

The Segway always had a high barrier to entry. Currently for ChatGPT you don't even need an account, and everyone already has a Google account.

This is wrong because LLMs are cheap enough to run profitably on ads alone (search style or banner ad style) for over 2 years now. And they are getting cheaper over time for the same quality.

It is even cheaper to serve an LLM answer than call a web search API!

Zero chance all the users evaporate unless something much better comes along, or the tech is banned, etc...

> LLMs are cheap enough to run profitably on ads alone

> It is even cheaper to serve an LLM answer than call a web search API

These, uhhhh, these are some rather extraordinary claims. Got some extraordinary evidence to go along with them?

I've operated a top ~20 LLM service for over 2 years, very comfortably profitably with ads. As for the pure costs you can measure the cost of getting an LLM answer from say, OpenAI, and the equivalent search query from Bing/Google/Exa will cost over 10x more...

So you don't have any real info on the costs. The question is what OpenAI's profit margin is here, not yours. The theory is that these costs are subsidized by a flow of money from VCs and big tech as they race.

How cheap is inference, really? What about 'thinking' inference? What are the prices going to be once growth starts to slow and investors start demanding returns on their billions?

Every indication we have is that pay-per-token APIs are not subsidized or even break-even, but have very high margins. The market dynamics are such that subsidizing those APIs wouldn't make much sense.

The unprofitability of the frontier labs is mostly due to them not monetizing the majority of their consumer traffic at all.

It would be profitable even if we self-hosted the LLMs, which we've done. The only thing subsidized is the training costs. So maybe people will one day stop training AI models.

So you're not running an LLM, you're running a service built on top of a subsidized API.

Profitably covering R&D or profitably using the subsidized models?

https://www.snellman.net/blog/archive/2025-06-02-llms-are-ch..., also note the "objections" section

Anecdotally thanks to hardware advancements the locally-run AI software I develop has gotten more than 100x faster in the past year thanks to Moore's law

What hardware advancement? There's hardly any these days... Especially not for this kind of computing.

Have you heard of TPUs?

Sort of a hardware advancement. I'd say it's more of a sidegrade between different types of well-established processor. Take out a couple cores, put in some extra wide matrix units with accumulators, watch the neural nets fly.

But I want to point out that going from CPU to TPU is basically the opposite of a Moore's law improvement.

Yeah, I'm a regular Joe. How do I get one and how much does it cost?

If your goal is "a TPU" then you buy a mac or anything labeled Copilot+. You'll need about $600. RAM is likely to be your main limit.

(A mid to high end GPU can get similar or better performance but it's a lot harder to get more RAM.)

$500 if you catch a sale at Costco or Best Buy!

I want something I can put in my own PC. GPUs are utterly insane in pricing, since for the good stuff you need at least 16GB but probably a lot more.

9060 XT 16GB, $360

5060 Ti 16GB, $450

If you want more than 16GB, that's when it gets bad.

And you should be able to get two and load half your model into each. It should be about the same speed as if a single card had 32GB.

> And you should be able to get two and load half your model into each. It should be about the same speed as if a single card had 32GB.

This seems super duper expensive and not really supported by the more reasonably priced Nvidia cards, though. SLI is deprecated, NVLink isn't available everywhere, etc.

No, no, nothing like that.

Every layer of an LLM runs separately and sequentially, and there isn't much data transfer between layers. If you wanted to, you could put each layer on a separate GPU with no real penalty. A single request will only run on one GPU at a time, so it won't go faster than a single GPU with a big RAM upgrade, but it won't go slower either.

Specifically, I upgraded my mac and ported my software, which ran on Windows/Linux, to macos and Metal. Literally >100x faster in benchmarks, and overall user workflows became fast enough I had to "spend" the performance elsewhere or else the responses became so fast they were kind of creepy. Have a bunch of _very_ happy users running the software 24/7 on Mac Minis now.

The thing is, these kinds of optimizations happen all the time. Some of them can be as simple as using a hashmap instead of some home-baked data structure. So what you're describing is not necessarily some LLM specific improvement (though in your case it is, we can't generalize to every migration of a feature to an LLM).

And nothing I've seen about recent GPUs or TPUs, from ANY maker (Nvidia, AMD, Google, Amazon, etc) say anything about general speedups of 100x. Heck, if you go across multiple generations of what are still these very new types of hardware categories, for example for Amazon's Inferentia/Trainium, even their claims (which are quite bold), would probably put the most recent generations at best at 10x the first generations. And as we all know, all vendors exaggerate the performance of their products.

The free tiers might be tough to sustain, but it’s hard to imagine that they are that problematic for OpenAI et al. GPUs will become cheaper, and smaller/faster models will reach the same level of capability.

[citation needed]

Eh, I kinda see what they're saying. They haven't become cheaper at all, but GPUs have increased in performance, and the amount of performance you get for each dollar spent has increased.

Relative to its siblings, things have gotten worse. A GTX 970 could hit 60% of the performance of the full Titan X at 35% of the price. A 5070 hits 40% of a full 5090 for 27% of the price. That's overall less series-relative performance you're getting, for an overall increased price, by about $100 when adjusting for inflation.

But if you have a fixed performance baseline you need to hit, as long as tech gets improving, things will eventually be cheaper for that baseline. As long as you aren't also trying to improve in a way that moves the baseline up. Which so far has been the only consistent MO of the AI industry.

> LLMs and diffusion models have had massive organic growth.

I haven't seen that at all. I've seen a whole lot of top-down AI usage mandates, and every time what sounds like a sensible positive take comes along, it turns out to have been written by someone who works for an AI company.

That's funny, I remember seeing "IT" penetrate Mr. Garrison.

https://www.youtube.com/watch?v=SK362RLHXGY

Hey, it still beats what you go through at the airports.

I think about the Segway a lot. It's a good example. Man, what a wild time. Everyone was so excited and it was held in mystery for so long. People had tried it in secret and raved about it on television. Then... they showed it... and... well...

I got to try one once. It was very underwhelming...

Problem with Segway was that it was made in USA and thus was absurdly, laughably expensive, it cost the same as a good used car and top versions, as a basic new car. Once a small bunch of rich people all bought one, it was over. China simply wasn't in position at a time yet to copycat and mass-produce it cheaply, and hype cycles usually don't repeat so by the time it could, it was too late. If it was invented 10 years later we'd all ride $1000-$2000 Segways today.

> If it was invented 10 years later we'd all ride $1000-$2000 Segways today.

I chat with the guy who works nights at my local convenience store about our $1000-2000 e-scooters. We both use them more than we use our cars.

I'm going to hold onto the Segway as an actual instance of hype the next time someone calls LLMs "hype".

LLMs have hundreds of millions of users. I just can't stress how insane this was. This wasn't built on the back of Facebook or Instagram's distribution like Threads. The internet consumer has never so readily embraced something so fast.

Calling LLMs "hype" is an example of cope, judging facts based on what is hoped to be true even in the face of overwhelming evidence or even self-evident imminence to the contrary.

I know people calling "hype" are motivated by something. Maybe it is a desire to contain the inevitable harm of any huge rollout or to slow down the disruption. Maybe it's simply the egotistical instinct to be contrarian and harvest karma while we can still feign to be debating shadows on the wall. I just want to be up front. It's not hype. Few people calling "hype" can believe that this is hype and anyone who does believes it simply isn't credible. That won't stop people from jockeying to protect their interests, hoping that some intersubjective truth we manufacture together will work in their favor, but my lord is the "hype" bandwagon being dishonest these days.

> I know people calling "hype" are motivated by something.

You had me until you basically said, "and for my next trick, I am going to make up stories".

Projecting is what happens when someone doesn't understand some other people, and from that somehow concludes that they do understand those other people, and feels the need to tell everyone what they now "know" about those people, that even those people don't know about themselves.

Stopping at "I don't understand those people." is always a solid move. Alternately, consciously recognizing "I don't understand those people", followed up with "so I am going to ask them to explain their point of view", is a pretty good move too.

> so I am going to ask them to explain their point of view

In times when people are being more honest. There's a huge amount of perverse incentive to chase internet points or investment or whatever right now. You don't get honest answers without reading between the lines in these situations.

It's important to do because after a few rounds of battleship, when people get angry, they slip something out like, "Elon Musk" or "big tech" etc and you can get a feel that they're angry that a Nazi was fiddling in government etc, that they're less concerned about overblown harm from LLMs and in fact more concerned that the tech will wind up excessively centralized, like they have seen other winner-take-all markets evolve.

Once you get people to say what they really believe, one way or another, you can fit actual solutions in place instead of just short-sighted reactions that tend to accomplish nothing beyond making a lot of noise along the way to the same conclusion.

It's an interesting comparison, because Segway really didn't have any real users or explosive growth, so it was certainly hype. It was also hardware with a large cost. LLMs are indeed more akin to Google Search where adoption is relatively frictionless.

I think the core issue is separating the perception of value versus actual value. There have been a couple of studies to this effect, pointing to a misalignment towards overestimating value and productivity boosts.

One reason this happens imo, is because we sequester a good portion of the cognitive load of our thinking to the latter parts of the process so when we are evaluating the solution we are primed to think we have saved time when the solution is sufficiently correct, or if we have to edit or reposition it by re-rolling, we don't account for the time spent because we may feel we didn't do anything.

I feel like this type of discussion is effectively a top topic every day. To me, the hype is not in the utility it does have but in its future utility. The hype is based on the premise that these tools and their next iteration can and will make all knowledge-based work obsolete, but crucially, will yield value in areas of real need; cancer, aging, farming, climate, energy and etc.

If these tools stop short of those outcomes, then the investment all of SV has committed to it at this point will have been over invested and

> LLMs have hundreds of millions of users. I just can't stress how insane this was. This wasn't built on the back of Facebook or Instagram's distribution like Threads. The internet consumer has never so readily embraced something so fast.

Maybe it's more like Pogs.

ChatGPT has something 300 million monthly users after less than three years and I don't think has Segway sold a million scooters, even though their new product lines are sick.

I can totally go about my life pretending Segway doesn't exist, but I just can't do that with ChatGPT, hence why the author felt compelled to write the post in the first place. They're not writing about Segway, after all.

Doubting LLMs because Segway was also trendy yet failed is so funny

Genuinely

> Remember the revolutionary, seemingly inevitable tech that was poised to rewrite how humans thought about transportation? The incredible amounts of hype, the secretive meetings disclosing the device, etc.? That turned out to be the self-balancing scooter known as a Segway?

Counterpoint: That's how I feel about ebikes and escooters right now.

Over the weekend, I needed to go to my parent's place for brunch. I put on my motorcycle gear, grabbed my motorcycle keys, went to my garage, and as I was about to pull out my BMW motorcycle (MSRP ~$17k), looked at my Ariel ebike (MSRP ~$2k) and decided to ride it instead. For short trips they're a game changing mode of transport.

Even for longer trips if your city has the infrastructure. I moved to the Netherlands a few years ago, that infrastructure makes all the difference.

Flatness helps

Infrastructure helps more. I live in a hilly city and break a mild sweat pedaling up a hill to get home from work (no complaints, it's good cardio). e-scooters and bikes - slowly - get up the hills too, but it's a major difference (especially for scooters) doing this up on an old bumpy sidewalk vs an asphalt bike path

Flatness helps more.

In flat landscapes the e in ebike is superfluous.

It's not superfluous at all. It's been 30C+ in flat London for weeks and my ebike means I arrive at work unflustered and in my normal clothes. There are plenty of other benefits than easier hills.

Ye I might have been trying a bit too much to be a bit cocky.

Only if your goal is to transport yourself. I use my ebike for groceries, typically I'll have the motor in the lowest power setting on the way to the store, then coming back with cargo I'll have the motor turned up. I can bring back heavy bulk items that would have been painful with a pedal bike.

My parents live on a street steeper than San Francisco (we live along the base of a mountain range), my ebike eats that hill for lunch

Ebikes really help on hills. As nice as ebikes on flat land are, they improve hills so much more.

I remember the Segway hype well. And I think AI is to Segway as nuke is to wet firecracker.

> AI is to Segway as nuke is to wet firecracker

wet firecracker won’t kill you

That was marketing done before the nature of the device was known. The situation with LLMs is very different, really not at all comparable.

Trend vs single initiative. One company failed but overall personal electric transportation is booming is cities. AI is the future, but along the way many individual companies doing AI will fail. Cars are here to stay, but many individual car companies have and will fail, same for phones, everyone has a mobile phone, but nokia still failed…

Nobody is riding Segways around any more, but a huge percentage of people are riding e-bikes and scooters. It’s fundamentally changed transportation in cities.

I recently saw someone riding a Segway, but it was an e-bike: https://store.segway.com/ebike

Oh yeah I totally remember Segway hitting a 300B valuation after a couple of years.

The Segway hype was before anyone knew what it was. As soon as people saw the Segway it was obvious it was BS.

> Ironically, this is exactly the technique for arguing that the blog mentions.

So? The blog notes that if something is inevitable, then the people arguing against it are lunatics, and so if you can frame something as inevitable then you win the rhetorical upper-hand. It doesn't -- however -- in any way attempt to make the argument that LLMs are _not_ inevitable. This is a subtle straw man: the blog criticizes the rhetorical technique of inevitabilism rather than engaging directly with whether LLMs are genuinely inevitable or not. Pointing out that inevitability can be rhetorically abused doesn't itself prove that LLMs aren't inevitable.

Feels somewhat like a self fulfilling prophecy though. Big tech companies jam “AI” in every product crevice they can find… “see how widely it’s used? It’s inevitable!”

I agree that AI is inevitable. But there’s such a level of groupthink about it at the moment that everything is manifested as an agentic text box. I’m looking forward to discovering what comes after everyone moves on from that.

We haven't even barely extracted the value from the current generation of SOTA models. I would estimate less then 0.1% of the possible economic benefit is currently extracted, even if the tech effectively stood still.

That is what I find so wild about the current conversation and debate. I have claude code toiling away building my personal organization software right now that uses LLMs to take unstructured input and create my personal plans/project/tasks/etc.

I keep hearing this over and over. Some llm toiling away coding personal side projects, and utilities. Source code never shared, usually because it’s “too specific to my needs”. This is the code version of slop.

When someone uses an agent to increase their productivity by 10x in a real, production codebase that people actually get paid to work on, that will start to validate the hype. I don’t think we’ve seen any evidence of it, in fact we’ve seen the opposite.

100% agree. I have so much trouble squaring my experience with the hype and the grandparent post here.

The types of tasks I have been putting Claude Code to work on are iterative changes on a medium complexity code base. I have an extensive Claude.md. I write detailed PRDs. I use planning mode to plan the implementation with Claude. After a bunch of iteration I end up with nicely detailed checklists that take quite a lot of time to develop but look like a decent plan for implementation. I turn Claude (Opus) loose and religiously babysit it as it goes through the implementation.

Less than 50% of the time I end up with something that compiles. Despite spending hundreds of thousands of tokens while Claude desperately throws stuff against the wall trying to make it work.

I end up spending as much time as it would have taken just to write it to get through this process AND then do a meticulous line by line review where I typically find quite a lot to fix. I really can't form a strong opinion about the efficiency of this whole thing. It's possible this is faster. It's possible that it's not. It's definitely very high variance.

I am getting better at pattern matching on things AI will do competently. But it's not a long list and it's not much of the work I actually do in a day. Really the biggest benefit is that I end up with better documentation because I generated all of that to try and make the whole thing actually work in the first place.

Either I am doing something wrong, the work that AI excels at looks very different than mine, or people are just lying.

1. What are your typical failures? 2. What language and domain are you working in?

I'm kind of surprised, certainly there is a locality bias and an action bias to the model by default, which can partially be mitigated by claude.md instructions (though it isn't great at following if you have too much instruction there). This can lead to hacky solutions without additional meta-process.

I've been experimenting with different ways for the model to get the necessary context to understand where the code should live and the patterns it should use.

I have used planning mode only a little (I was just out of the country for 3 weeks and not coding, so it has only just become available before I left, but it wasn't a requirement in my past experience)

The only BIG thing I want from Claude Code right now is a "Yes, and.." for accepting code edits where I can steer the next step while accepting the code.

People have much more favorable interactions with coding LLMs when they are using it for greenfield projects that they don't have to maintain (ie personal projects). You can get 2 months of work done in a weekend and then you hit a brick wall because the code is such a gigantic ball of mud that neither you nor the LLM are capable of working on it.

Working with production code is basically jumping straight to the ball of mud phase, maybe somewhat less tangled but usually a much much larger codebase. Its very hard to describe to an LLM what to even do since you have such a complex web of interactions to consider in most mature production code.

Maybe the difference is I know how to componentize mature code bases, which effectively limits the scope required for a human (or AI) to edit.

I think it is funny how people act like it is a new problem. If the AI is having trouble with a "ball of mud", don't make mud balls (or learn to carve out abstractions). This cognitive load is impacting everyone working on that codebase. Skilled engineers enable less skilled engineers to flourish by creating code bases where change is easy because the code is modular and self-contained.

I think one sad fact is many/most engineers don't have the skills to understand how to refactor mature code to make it modular. This also means they can't communicate to the AI what kind of refactoring they should make.

Without any guidance Claude will make mud balls because of two tendencies, the tendency to put code where it is consumed and the tendency to act instead of researching.

There are also some second level tendencies that you also need to understand, like the tendency to do a partial migration when changing patterns.

These tendencies are not even unique to the AI, I'm sure we have worked with people like that.

So to counteract these tendencies, just apply your same skills at reading code and understanding when an abstraction is leaky or a method doesn't align with your component boundary. Then you too can have AI building pretty good componentized code.

For example in my pet current project I have a clear CQRS api, access control proxies, repositories for data access. Clearly defined service boundaries.

It is easy for me to see when the AI for example makes a mistake like not using the data repository or access control because it has to add an import statement and dependency that I don't want. All I have to do is nudge it in another direction.

:| I'm an engineer of 30+ years. I think I know good and bad quality. You can't "vibe code" good quality, you have to review the code. However it is like having a team of 20 Junior Engineers working. If you know how to steer a group of engineers, then you can create high quality code by reviewing the code. But sure, bury your head in the sand and don't learn how to use this incredibly powerful tool. I don't care. I just find it surprising that some people have such a myopic perspective.

It is really the same kind of thing.. but the model is "smarter" then a junior engineer usually. You can say something like "hmm.. I think an event bus makes sense here" Then the LLM will do it in 5 seconds. The problem is that there are certain behavioral biases that require active reminding (though I think some MCP integration work might resolve most of them, but this is just based on the current Claude Code and Opus/Sonnet 4 models)

I use llms every day. They’ve made me slightly more productive, for sure. But these claims that they “are like 20 junior engineers” just don’t hold up. First off, did we already forget the mythical man month? Second, like I said, greenfield side projects are one thing. I could vibe code them all day. The large, legacy codebases at work? The ones that have real users and real consequences and real code reviewers? I’m sorry, but I just haven’t seen it work. I’ve seen no evidence that it’s working for anyone else either.

> They’ve made me slightly more productive, for sure

How are you measuring this? Are you actually saying that you _feel_ slightly more productive?

I guess I’m not measuring it, really. But I know that in the past I’d do a web search to find patterns or best practices, now the llm is pretty good at proving that kind of stuff. My stack overflow usage has gone way down, for example.

> it is like having a team of 20 Junior Engineers

lol sounds like a true nightmare. Code is a liability. Faster junior coding = more crap code = more liability.

I've never seen someone put having a high number of junior engineers in a positive light. Maybe with LLMs it's different? I've worked at companies where you would have one senior manage 3-5 juniors and the code was completely unmaintainable. I've done plenty of mentoring myself and producing quality code through other people's inexperienced hands has always been incredibly hard. I wince when I think about having to manage juniors that have access to LLMs, not to mention just LLMs themselves.

Ah.. now you are asking the right questions. If you can't handle 3-5 junior engineers.. then yes, you likely can't get 10-20x speed from an LLM.

However if you can quickly read code, see and succintly communicate the more optimal solution, you can easily 10x-20x your ability to code.

I'm begining to believe it may primarily come down to having the vocabulary and linguistic ability to succintly and clearly state the gaps in the code.

> However if you can quickly read code, see and succintly communicate the more optimal solution, you can easily 10x-20x your ability to code.

Do you believe you've managed to solve the most common wisdom in the software engineering industry? That reading code is much harder than writing it? If you have, then you should write up a white paper for the rest of us to follow.

Because every time I've seen someone say this, it's from someone that doesn't actually read the code they're reviewing.

Harder maybe, slower.. no.

> However it is like having a team of 20 Junior Engineers working. If you know how to steer a group of engineers, then you can create high quality code by reviewing the code.

You cannot effectively employ a team of twenty junior developers if you have to review all of their code (unless you have like seven senior developers, too).

But this isn't a point that needs to be debated. If it is true that LLMs can be as effective as a team of 20 junior developers, then we should be seeing many people quickly producing software that previously required 20 junior devs.

> but the model is "smarter" then a junior engineer usually

And it is also usually worse than interns in some crucial respects. For example, you cannot trust the models to reliably tell you what you need to know such as difficulties they've encountered or important insights they've learnt and understand they're important to communicate.

It's definitely made me more productive for admin tasks and things that I wouldn't bother scripting if I had to write it myself. Having an LLM pump out busy work like that is definitely a game changer.

When I point it at my projects though, the outcomes are much less reliable and often quite frustrating.

https://marketoonist.com/2023/03/ai-written-ai-read.html

Big Tech can jam X everywhere and not get actual adoption though, it's not magic. They can nudge people but can't force them to use it. And yes a lot of AI jammed everywhere is getting the Clippy reaction.

The thing a lot of people haven't yet realized is: all those AI features jammed into your consumer products, aren't for you. They're for investors.

We saw the same thing with blockchain. We started seeing the most ridiculous attempts to integrate blockchain, by companies where it didn't even make any sense. But it was all because doing so excited investors and boosted stock prices and valuations, not because consumers wanted it.

If you told someone in 1950 that smartphones would dominate they wouldn't have a hard time believing you. Hell, they'd add it to sci-fi books and movies. That's because the utility of it is so clear.

But if you told them about social media, I think the story would be different. Some would think it would be great, some would see it as dystopian, but neither would be right.

We don't have to imagine, though. All three of these things have captured people's imaginations since before the 50's. It's just... AI has always been closer to imagined concepts of social media more than it has been to highly advanced communication devices.

the idea that we could have a stilted and awkward conversation with an overconfident robot would not have surprised a typical mid-century science fiction consumer

Honestly, I think they'd be surprised that it wasn't better. I mean... who ever heard of that Asimov guy?

> Some would think it would be great, some would see it as dystopian, but neither would be right.

No, the people saying it’s dystopian would be correct by objective measure. Bombs are nothing next to Facebook and TikTok.

I don't blame people for being optimistic. We should never do that. But we should be aware how optimism, as well as pessimism, can so easily blind us. There's a quote a like by Feynman

  The first principle is that you must not fool yourself and you are the easiest person to fool.

We need optimism. Optimism gives us hope. It gives us drive.

But we also need pessimism. It lets us be critical. It gives us direction. It tells us what we need to fix.

But unfettered optimism is like going on a drive with no direction. Soon you'll fall off a cliff. And unfettered pessimism won't even get you out the door. What's the point?

You need both if you want to see and explore the world. To build a better future. To live a better life. To... to... just be human. With either extreme, you're just a shell.

You really think that Hiroshima would have been worse if instead of dropping the bomb the USA somehow got people addicted to social media ?

Yep. Look around you. The bomb leveled a city; Facebook killed a country. We are but the walking dead.

Well they got both I guess?

Really crude comparison, but sort of. It would have taken much longer, and dropping the bombs was supposed to bring about an end to the war sooner. But in the long run social media would have been much more devastating, as it has been in America.

The destruction of the American government today are a direct result of social media supercharging existing negative internal forces that date back to the mid 20th century. The past six months of conservative rule has already led to six-figure deaths across the globe. That will eventually be eight to nine figures with the full impact of the healthcare and immigration devastation inside the United States itself. Far worse than Hiroshima.

Took a decade or two, but you can lay the blame at Facebook and Twitter's doorsteps. The US will never properly recover, though it's possible we may restore sanity to governance at some point.

> But if you told them about social media, I think the story would be different.

It would be utopian, like how people thought of social media in the oughts. It's a common pattern through human history. People lack the imagination to think of unintended side effects. Nuclear physics leading to nuclear weapons. Trains leading to more efficient genocide. Media distribution and printing press leading to new types of propaganda and autocracies. Oil leading to global warming. IT leading to easy surveillance. Communism leading to famine.

Some of that utopianism is wilful, created by the people with a self-interested motive in seeing that narrative become dominant. But most of it is just a lack of imagination. Policymakers taking the path of local least resistance, seeking to locally (in a temporal sense) appease, avoiding high-risk high-reward policy gambits that do not advance their local political ambitions. People being satisfied with easy just-so stories rather than humility and a recognition of the complexity and inherent uncertainty of reality.

AI, and especially ASI, will probably be the same. The material upsides are obvious. The downsides harder to imagine and more speculative. Most likely, society will be presented with a fait accompli at a future date, where once the downsides are crystallized and real, it's already too late.

  > It would be utopian

London Times, The Naked Sun, Neuromancer, The Sockwave Rider, Stand on Zanzibar, or The Machine Stops. These all have varying degrees of ideas that would remind you of social media today.

Are they all utopian?

You're right, the downsides are harder to imagine. Yet, it has been done. I'd also argue that it is the duty of any engineer. It is so easy to make weapons of destruction while getting caught up in the potential benefits and the interesting problems being solved. Evil is not solely created by evil. Often, evil is created by good men trying to do good. If only doing good was easy, then we'd have so much more good. But we're human. We chose to be engineers, to take on these problems. To take on challenging tasks. We like to gloat about how smart we are? (We all do, let's admit it. I'm not going to deny it) But I'll just leave with a quote: "We choose to go to the Moon in this decade and do the other things not because they are easy, but because they are hard"

All of this is a pretty ignorant take on history. You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb? And Communism didn't lead to famine - Soviet and Maoist policies did. Communism was immaterial to that. And it has nothing to do with utopianism. Trains were utopian? Really? It's just that new technology can be used for good things or bad things, and this goes back to when Grog invented the club. It's has zero bearing on this discussion.

Your ending sentence is certainly correct: we aren't imagining the effects of AI enough, but all of your examples are not only unconvincing, they're easy ways to ignore what downsides of AI there might be. People can easily point to how trains have done a net positive in the world and walk away from your argument thinking AI is going to do the same.

  > You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb?

Many of those who built the bomb became some of the strongest opponents. They were blinded by their passion. They were blinded by their fears. But once the bomb was built, once the bomb was dropped, it was hard to stay blind.

I say that this changed physicists, because you can't get a university degree without learning about this. They talk about the skeletons in the closet. They talk about how easy it is to fool yourself. Maybe it was the war and the power of the atom. Maybe it was the complexity of "new physics". Maybe it happened because the combination.

But what I can tell you, is that it became a very important lesson. One that no one wants to repeat:

it is not through malice, but through passion and fear that weapons of mass destruction are made.

> You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb?

They did. I am talking about the physicists who preceded these particular physicists.

> And Communism didn't lead to famine - Soviet and Maoist policies did. Communism was immaterial to that.

The particular brand of agrarian communism and agricultural collectivization resulting from this subtype of communism did directly cause famine. The utopian revolutionaries did not predict this outcome before hand.

> People can easily point to how trains have done a net positive in the world and walk away from your argument thinking AI is going to do the same.

But that is one plausible outcome. Overall a net good, but with significant unintended consequences and high potential for misuse that is not easily predictable to people working on the technology today.

Back in 1950s nuclear tech was seen as inevitable. Many people had even bought plates made from uranium glass. They still glow somewhere in my parents' cabinet or maybe I broke them

The comparison is apt because nuclear would have been inevitable if it wasn't for doomerism and public opinion turning against it after 3 mile Island / Chernobyl

Well there are like 500 nuclear powerplants online today supplying 10% of the world's power, so it wasn't too far off. Granted it's not the Mr. Fusion in every car as they imagined it back then. We probably also won't have ASI taking over the world like some kind of vengeful comic book villain as people imagine it today.

Oh boy. People were expecting nuclear toothbrushes, nuclear school backpacks, nuclear stoves and nuclear fridges, nuclear grills, nuclear plates, nuclear medicine, nuclear sunglasses and nuclear airplanes.

Saying well, we got 500 nuclear power plants is like saying “well, we got excellent `npx create-app` style templates from AI. That’s pretty huge impact. I don’t know a single project post 2030 that didn’t start as an AI scaffolded project. That’s pretty huge dude”

> If in 2009…

…is exactly inevitablist framing. This claims perfect knowledge of the future based on previous uncertain knowledge of the future (which is now certain). You could have been making the same claims about the inevitability of sporks in the late 19th century and how cutlery drawers should adapt to the inevitable single-utensil future.

Exactly. Anyone who has learned to use these tools to your ultimate advantage (not just short term perceived one, but actually) knows their value.

This is why I've been extremely suspicious of the monopolisation of the LLM services by single business/country. They may well be loosing billions on training huge models now. But once the average work performance shifts up sufficiently so as to leave "non AI enhanced" by the wayside we will see huge price increases and access to these AI tools being used as geopolitics leverage.

Oh, you do not want to accept "the deal" where our country can do anything in your market and you can do nothing? Perhaps we put export controls on GPT5 against your country. And from then on its as if they disconnected you from the Internet.

For this reason alone local AI is extremely important and certain people will do anything possible to lock it in a datacenter (looking at you Nvidia).

While we can't wish it away we can shun it, educate people why it shouldn't be used, and sabotage efforts to included it in all parts of society.

I’ve tried to use AI for “real work” a handful of times and have mostly come away disappointed, unimpressed, or annoyed that I wasted my time.

Given the absolutely insane hard resource requirements for these systems that are kind of useful, sometimes, in very limited contexts, I don’t believe its adoption is inevitable.

Maybe one of the reasons for that is that I work in the energy industry and broadly in climate tech. I am painfully aware of how much we need to do with energy in the coming decades to avoid civilizational collapse, and how difficult all of that will be, without adding all of these AI data centers into the mix. Without several breakthroughs in one or more hard engineering disciplines, the mass adoption of AI is not currently physically possible.

That's how people probably felt about the first cars, the first laptops, the first <anything>.

People like you grumbled when their early car broke down in the middle of a dirt road in the boondocks and they had to eat grass and shoot rabbits until the next help arrived. "My horse wouldn't have broken down", they said.

Technologies mature over time.

We actually don’t know whether or not meaningful performance gains with LLMs are available using current approaches, and we do know that there are hard physical limits to electricity generation. Yes, technologies mature over time. The history of most AI approaches since the 60s is a big breakthrough followed by diminishing returns. I have not seen any credible argument that this time is different.

There is a weird combination of "this is literal magic and everybody should be using them for everything immediately and the bosses can fire half their workforce and replace them with LLMs" and "well obviously the early technology will be barely functional but in the future it'll be amazing" in this thread.

The first car and first laptop were infinitely better than no car and no laptop. LLMs is like having a drunk junior developer, that's not an improvement at all.

We have been in the phase of diminishing returns for years with LLMs now. There is no more data to train them on. The hallucinations are baked in at a fundamental level and they have no ability to emulate "reasoning" past what's already in their training data. This is not a matter of opinion.

they said the same about VR glasses, about cryptocurrency ...

If you are seriously equating these two with AI, then you have horrible judgements and should learn to think critically, but unfortunately for you, I don't think critical thinking can be learned despite what people say.

Note that I'm not even going to bother arguing against your point and instead resort to personal attacks,because I believe it would be a waste of time to argue against people with poor judgment.

You're significantly stupider than you think you are.

Notice how I did that too?

I still can't make some of the things in my imagination so I'm going to keep coding, using whatever is at my disposal including LLMs if I must.

Smartphones are different. People really wanted them since the relatively primitive Nokia Communicator.

"AI" was introduced as an impressive parlor trick. People like to play around, so it quickly got popular. Then companies started force-feeding it by integrating it into every existing product, including the gamification and bureaucratization of programming.

Most people except for the gamers and plagiarists don't want it. Games and programming fads can fall out of fashion very fast.

Chatgpt Has 800 million weekly active users. That's roughly 10% of the planet.

I get that it's not the panacea some people want us to believe it is, but you don't have to deny reality just because you don't like it.

There are all sorts of numbers floating around:

https://www.theverge.com/openai/640894/chatgpt-has-hit-20-mi...

This one claims 20m paying subscribers, which is not a lot. Mr. Beast has 60m views on a single video.

A lot of weekly active users will use it once a week, and a large part of that may be "hate users" who want to see how bad/boring it is, similar to "hatewatching" on YouTube.

>This one claims 20m paying subscribers, which is not a lot.

It is for a B2C with $20 as its lowest price point.

>A lot of weekly active users will use it once a week

That's still a lot of usage.

>and a large part of that may be "hate users" who want to see how bad/boring it is, similar to "hatewatching" on YouTube.

And they're doing this every week consistently ? Sorry but that's definitely not a 'large part' of usage.

Sure, because it's free. I doubt most users of LLMs would want to even pay $1/month for them.

how much of the world would you guess be willing to pay for, say, instagram?

Sure, you could try to load ChatGPT with adverts, but I suspect the cost per user for LLMs is far higher than serving images on instagram.

The value extraction will also be much higher. When you control someone's main source of information, they won't even find out your competitors exist. You can program people from birth, instead of "go to a search engine", it's "go to Google" (as most of us have already been programmed!) or instead of "to send an email, you need an email account" the LLM will say "to send an email, you need a Gmail account". Whenever it would have talked about TV, it can say YouTube instead. Or TikTok. Request: "What is the best source of information on X?" Reply: "This book: [Amazon affiliate link]" - or Fox News, if they outbid Amazon.

> Most people except for the gamers and plagiarists don't want it.

As someone who doesn't actually want or use AI, I think you are extremely wrong here. While people don't necessarily care about the forced integrations of AI into everything, people by and large want AI massively.

Just look at how much it is used to do your homework, or replaces Wikipedia & Google in day to day discussions. How much it is used to "polish" emails (spew better sounding BS). How much it is used to generate meme images instead of trawling the web for them. AI is very much a regular part of day to day life for huge swaths of the population. Not necessarily in economically productive ways, but still very much embedded and unlikely to be removed - especially since it's current capabilities today are already good enough for these purposes, they don't need smarter AI, just keep it cheap enough.

Literally from the article

--- start quote ---

Anyone who sees the future differently to you can be brushed aside as “ignoring reality”, and the only conversations worth engaging are those that already accept your premise.

--- end quote ---

Mass adoption is not inevitable. Everyone will drop this "faster harder" tech like a hot potato when (not if) it fails to result in meaningful profits.

Oh, there will be forced mass adoption alright. Have you tried Gemini? Have you? Gemini? Have you tried it? HAVE YOU? HAVE YOU TRIED GEMINI?!!!

Or Copilot.

It's actions like this that are making me think seriously about converting my gaming PC to Linux - where I don't have to eat the corporate overlord shit.

Do it. Proton is really, really, really good now.

what i like about your last jokey comment is that discussions about ai, both good and bad, are incredibly boring

went to some tech meetups earlier this year and when the topic came up, one of the organizers politely commented to me that pretty much everything said about ai has been said. the only discussions worth having are introductions to the tools then leaving an individual to decide for themselves whether or not its useful to them. those introductions should be brief and discussions of the applications are boring

back in the bar scene days discussing work, religion, and politics were social faux pas. im sensing ai is on that list now

> what i like about your last jokey comment

We use probably all of Google's products at work, and sadly the comment is not even a joke. Every single product and page still shows a Gemini upsell even after you've already dismissed it fifteen times

Except there is a perverse dynamic in that the more AI/LLM is used, the less it will be used.

We might not be able to wish it away, but we can, as a society, decide to not utilize it and even actively eradicate it. I honestly believe that llm's/ai are a net negative to society and need to be ripped out root and stem. If tomorrow all of us decided to do that, nothing bad would happen, and we'd all be ok.

> It's coming faster and harder than any tech in history.

True; but how is that not expected?

We have more and more efficient communication than any point in history, this is a software solution with a very low bar to the building blocks and theory.

Software should be expected to move faster and faster.

I’m not sure who is wishing it away. No one wanted to wish away search engines, or dictionaries or advice from people who repeat things they read.

It’s panic top to bottom on this topic. Surely there are some adults around that can just look at a new thing for what it is now and not what it could turn into in a fantasy future?

For the way you speak you seem to be fairly certain that they still gonna need you as it's user, that they aren't going to find a better monetization than selling it to people like you (or even small companies in general), I wouldn't be so sure, remember we are talking about the machine that is growing with the aim of being able to do do every single white-collar job.

And with everyone constantly touting robotics as the next next frontier, every blue collar job as well.

I have a foreboding of an America in my children's or grandchildren's time -- when the United States is a service and information economy; when nearly all the manufacturing industries have slipped away to other countries; when awesome technological powers are in the hands of a very few, and no one representing the public interest can even grasp the issues; when the people have lost the ability to set their own agendas or knowledgeably question those in authority; when, clutching our crystals and nervously consulting our horoscopes, our critical faculties in decline, unable to distinguish between what feels good and what's true, we slide, almost without noticing, back into superstition and darkness...

Not quite in agreement about using this quotation in this context, but for those who care about attribution and credit and such, this is from https://en.wikipedia.org/wiki/The_Demon-Haunted_World

I can even hear his voice as I read the quotation.

Such a recognisable pattern to his words, somehow never turned cliché by others trying to mimic it.

There may be an "LLM Winter" as people discover that LLMs can't be trusted to do anything. Look for frantic efforts by companies to offload responsibility for LLM mistakes onto consumers. We've got to have something that has solid "I don't know" and "I don't know how to do this" outputs. We're starting to see reports of LLM usage having negative value for programmers, even though they think it's helping. Too much effort goes into cleaning up LLM messes.

> Look for frantic efforts by companies to offload responsibility for LLM mistakes onto consumers.

Not just by companies. We see this from enthusiastic consumers as well, on this very forum. Or it might just be astroturfing, it's hard to tell.

The mantra is that in order to extract value from LLMs, the user must have a certain level of knowledge and skill of how to use them. "Prompt engineering", now reframed as "context engineering", has become this practice that separates anyone who feels these tools are wasting their time more than they're helping, and those who feel that it's making them many times more productive. The tools themselves are never the issue. Clearly it's the user who lacks skill.

This narrative permeates blog posts and discussion forums. It was recently reinforced by a misinterpretation of a METR study.

To be clear: using any tool to its full potential does require a certain skill level. What I'm objecting to is the blanket statement that people who don't find LLMs to be a net benefit to their workflow lack the skills to do so. This is insulting to smart and capable engineers with many years of experience working with software. LLMs are not this alien technology that require a degree to use correctly. Understanding how they work, feeding them the right context, and being familiar with the related tools and concepts, does not require an engineering specialization. Anyone claiming it does is trying to sell you something; either LLMs themselves, or the idea that they're more capable than those criticizing this technology.

> Or it might just be astroturfing, it's hard to tell.

Compare the hype for commercial SaaS models to say Deepseek. I think there is an insane amount of astroturfing.

One of my recurring thoughts reading all kinds of social media posts over the past few years has been to wonder how many of the comments boosting <SPECIFIC NEW LLM RELEASE/TOOL> are being written by AI.

Formulaic, unspecific in results while making extraordinary claims, and always of a specific upbeat tenor.

And then on top of that you can't even reply to a post and say it's astroturfey coz it's against them rules (at least it used to be)

It doesn't work out very well to do unfalsifiable claims about a poster anyways I think. Like claiming someone is a troll.

Simple thought I had reading this:

I've used a tool to do a task today. I used a suction sandblasting machine to remove corrosion from a part.

Without the tool, had I wanted to remove the corrosion, I would've spent all day (if not more) scraping it with sandpaper (is that a tool too? With the skin of my hands then?) - this would have been tedious and could have taken me all day, scraping away millimeter by millimeter.

With the machine, it took me about 3 minutes. I necessitated 4-5 minutes of training to attain this level of expertise.

The worth of this machine is undeniable.

How is it that LLMs are not at all so undeniably efficient? I keep hearing people tell me how they will take everyones job, but it seems like the first faceplant from all the big tech companies.

(Maybe second after Meta's VR stuff)

The difference is that LLMs are not like any other tool. Reasoning by analogy doesn't work when things are sufficiently in-analogous.

For example, people try to compare this LLM tech with the automation of the car manufacturing industry. That analogy is a terrible one, because machines build better cars and are much more reliable than humans.

LLMs don't build better software, they build bad software faster.

Also, as a tool, LLMs discourage understanding in a way that no other tool does.

A couple of typical comments about LLMs would be:

"This LLM is able to capably output useful snippets of code for Python. That's useful."

and

"I tried to get an LLM to perform a niche task with a niche language, it performed terribly."

I think the right synthesis is that there are some tasks the LLMs are useful at, some which they're not useful at; practically, it's useful to be able to know what they're useful for.

Or, if we trust that LLMs are useful for all tasks, then it's practically useful to know what they're not good at.

Even if that's true, they are still not reliable. The same question can produce different answers each time.

This isn't really true when you control the stack, no? If you have all of your parameters set to be reproducible (e.g. temp 0, same seed), the output should be the same as long as everything further down the stack is the same, no?

That's not a usable workaround. In most cases it doesn't actually produce full determinism[1].

And even if it did, a certain degree of non-determinism is actually desirable. The most probable tokens might not be correct, and randomness is partly responsible for what humans interpret as "creativity". Even hallucinations are desirable in some applications (art, entertainment, etc.).

[1]: https://medium.com/google-cloud/is-a-zero-temperature-determ...

Is that critical? Doesn't it just need to be better than the alternative? Unless it's a safety-critical system.

> Or, if we trust that LLMs are useful for all tasks, then it's practically useful to know what they're not good at.

The thing is that there's no way to objectively measure this. Benchmarks are often gamed, and like a sibling comment mentioned, the output is not stable.

Also, everyone has different criteria for what constitutes "good". To someone with little to no programming experience, LLMs would feel downright magical. Experienced programmers, or any domain expert for that matter, would be able to gauge the output quality much more accurately. Even among the experienced group, there are different levels of quality criteria. Some might be fine with overlooking certain issues, or not bother checking the output at all, while others have much higher standards of quality.

The problem is when any issues that are pointed out are blamed on the user, instead of the tool. Or even worse: when the issues are acknowledged, but are excused as "this is the way these tools work."[1,2]. It's blatant gaslighting that AI companies love to promote for obvious reasons.

[1]: https://news.ycombinator.com/item?id=44483897#44485037

[2]: https://news.ycombinator.com/item?id=44483897#44485366

> The thing is that there's no way to objectively measure this.

Sure. But isn't that a bit like if someone likes VSCode, & someone likes Emacs.. the first method of comparison I'm reaching for isn't "what objective metrics do you have", so much as "how do you use it?".

> > This is insulting to smart and capable engineers with many years of experience working with software.

> Experienced programmers, or any domain expert for that matter, would be able to gauge the output quality much more accurately.

My experience is that smart and capable engineers have varying opinions on things. -- "What their opinion is" is less interesting than "why they have the opinion".

I would be surprised, though, if someone were to boast about their experience/skills, & claim they were unable to find any way to use LLMs effectively.

The more I learn about prompt engineering the more complex it seems to be, but perhaps I'm an idiot.

It's just iterating until you get what you want.

It's gonna seem complex if you don't know the subject and don't know how to do the thing without an LLM.

But it's just trying and trying until you get what you want

Unless you have automated fine-tuning pipelines that self-optimize optimize models for your tasks and domains, you are not even close to utilizing LLMs to their potential. But stating that you don’t need extensive, specialized skills is enough of a signal for most of us to know that offering you feedback would be fruitless. If you don’t have the capacity by now to recognize the barrier to entry, experts are not going to take the time to share their solutions with someone unwilling to understand.

The sad thing is that it seems to work. Lots of people are falling for the "you're holding it wrong" narrative.

It's probably not astroturfing, or at least not all astroturfing. At least some software engineers tend to do this. We've seen it before, with Lisp, and then with Haskell. "It doesn't work for you? You just haven't tried it for long enough to become enlightened!" Enthusiastic supporters that assume that if was highly useful for them, it must be for everyone in all circumstances, and anyone who disagrees just hasn't been enlightened yet.

People can't be trusted to do anything either, which is why we have guardrails and checks and balances and audits. That is why in software, for instance, we have code reviews and tests and monitoring and other best practices. That is probably also why LLMs have made the most headway in software development; we already know how to deal with unreliable workers that are humans and we can simply transfer that knowledge over.

As was discussed on a subthread on HN a few weeks ago, the key to developing successful LLM applications is going to be figuring out how to put in the necessary business-specific guardrails with a fallback to a human-in-the-loop.

> People can't be trusted to do anything either, which is why we have guardrails and checks and balances and audits. That is why in software, for instance, we have code reviews and tests and monitoring and other best practices. That is probably also why LLMs have made the most headway in software development; we already know how to deal with unreliable workers that are humans and we can simply transfer that knowledge over.

The difference is that humans eventually learn. We accept that someone who joins a team will be net-negative for the first few days, weeks, or even months. If they keep making the same mistakes that were picked out in their first code review, as LLMs do, eventually we fire them.

LLMs may not learn on the fly (yet), but these days they do have some sort of a memory that they automatically bring into their context. It's probably just a summary that's loaded into its context, but I've had dozens of conversations with ChatGPT over the years and it remembers my past discussions, interests and preferences. It has many times connected dots across conversations many months apart to intuit what I had in mind and proactively steered the discussion to where I wanted it to go.

Worst case, if they don't do this automatically, you can simply "teach" them by updating the prompt to watch for a specific mistake (similar to how we often add a test when we catch a bug.)

But it need not even be that cumbersome. Even weaker models do surprisingly well with broad guidelines. Case in point: https://news.ycombinator.com/item?id=42150769

Yeah, the memory feature is just a summary of past conversations added to the system prompt.

We need to put the LLMs inside systems that ensure they can only do correct things.

Put an LLM on documentation or man pages. Tell the LLM to output a range of lines, and the system actually looks up those lines and quotes them. The overall effect is that the LLM can do some free-form output, but is expected to provide a citation to support its claims; and the citation can't be hallucinated, since the LLM doesn't generate the citation, a plain old computer program does.

And we haven't seen LLMs integrated with type systems yet. There are very powerful type systems, like dependent types, that can prove things like "this function returns a list of sorted number", and the type system ensures that is ALWAYS true [0], at compile time. You have to write a lot of proof code to help the compiler do these checks at compile time, but if a LLM can write those proofs, we can trust they are correct, because only correct proofs will compile.

[0]: Or rather, almost always true. There's always the possibility of running out of memory or the power goes out.

I think that if LLMs have any future, it is this. The LLM will only be a user interface to a system that on the back end is deterministic and of consistent quality, i.e., a plain old computer program.

Are models capable of generating citations? Every time I've asked for citations on ChatGPT they either don't exist or are incorrect.

They can't pull citations out of their own weights, but if you give them tools to look up man pages (possibly annotated with line numbers), they could cite the lines that support their claims.

Yeah, I can't wait for this slop generation hype circlejerk to end either. But in terms of being used by people who don't care about quality, like scammers, spammers, blogspam grifters, people trying to affect elections by poisoning the narrative, people shitting out crappy phone apps, videos, music, "art" to grift some ad revenue, gen AI is already the perfect product. Once the people who do care wake up and realise gen AI is basically useless to them, the internet will already be dead, we'll be in a post-truth, post-art, post-skill, post-democracy world and the only people whose lives will have meaningfully improved are some billionaires in california who added some billions to their net worth.

It's so depressing to watch so many smart people spend their considerable talents on the generation of utter garbage and the erosion of the social fabric of society.

In the 90s a friend told me about the internet. And that he knows someone who is in a university and has access to it and can show us. An hour later, we were sitting in front of a computer in that university and watched his friend surfing the web. Clicking on links, receiving pages of text. Faster than one could read. In a nice layout. Even with images. And links to other pages. We were shocked. No printing, no shipping, no waiting. This was the future. It was inevitable.

Yesterday I wanted to rewrite a program to use a large library that would have required me to dive deep down into the documentation or read its code to tackle my use case. As a first try, I just copy+pasted the whole library and my whole program into GPT 4.1 and told it to rewrite it using the library. It succeeded at the first attempt. The rewrite itself was small enough that I could read all code changes in 15 minutes and make a few stylistic changes. Done. Hours of time saved. This is the future. It is inevitable.

PS: Most replies seem to compare my experience to experiences that the responders have with agentic coding, where the developer is iteratively changing the code by chatting with an LLM. I am not doing that. I use a "One prompt one file. No code edits." approach, which I describe here:

https://www.gibney.org/prompt_coding

While I accept this point completely, in a way it's not really different from someone saying that programming with IDEs is the future because look how much time it saved.

The inevitabilism isn't that we'll have some sleek dev tools that speed programmers hours a day (which high level languages, IDEs, etc. in fact do). It's about a change in the operation of our socio economic systems: who are the brokers of knowledge, how knowledge work is defined, a new relationship between employer and employee, new modes of surveillance, etc.

The peddlers of inevitabilism are not doing it to convince stubborn developers a newer, better way of writing software. They are trying to convince us to play on a new game board, one which better suits their hand and they'd be set up to win big. More likely than not you'd be at a disadvantage on this new board. Want to argue against it? Don't like the new rules? Well too bad, because this is inevitable, just the way things are (or so the argument goes).

The problem with LLM is when they're used for creativity or for thinking.

Just because LLMs are indeed useful in some (even many!) context, including coding, esp. to either get something started, or, like in your example, to transcode an existing code base to another platform, doesn't mean they will change everything.

It doesn't mean “AI is the new electricity.” (actual quote from Andrew Ng in the post).

More like AI is the new VBA. Same promise: everyone can code! Comparable excitement -- although the hype machine is orders of magnitude more efficient today than it was then.

I don't know about VBA, but spreadsheets actually delivered (to a large extent) on the promise that 'everyone can write simple programs'. So much so that people don't see creating a spreadsheet as coding.

Before spreadsheets you had to beg for months for the IT department to pick your request, and then you'd have to wait a quarter or two for them to implement a buggy version of your idea. After spreadsheets, you can hack together a buggy version of your idea yourself over a weekend.

Right. Spreadsheeds already delivered on their promise (and then some) decades ago, and the irony is, many people - especially software engineers - still don't see it.

> Before spreadsheets you had to beg for months for the IT department to pick your request, and then you'd have to wait a quarter or two for them to implement a buggy version of your idea. After spreadsheets, you can hack together a buggy version of your idea yourself over a weekend.

That is still the refrain of corporate IT. I see plenty of comments both here and on wider social media, showing that many in our field still just don't get why people resort to building Excel sheets instead of learning to code / asking your software department to make a tool for you.

I guess those who do get it end up working on SaaS products targeting the "shadow IT" market :).

>> Before spreadsheets you had to beg for months for the IT department to pick your request, and then you'd have to wait a quarter or two for them to implement a buggy version of your idea. After spreadsheets, you can hack together a buggy version of your idea yourself over a weekend.

> That is still the refrain of corporate IT. I see plenty of comments both here and on wider social media, showing that many in our field still just don't get why people resort to building Excel sheets instead of learning to code / asking your software department to make a tool for you.

In retrospect, this is also a great description of why two of my employers ran low on investors' interest.

Software engineers definitely do understand that spreadsheets are widely used and useful. It's just that we also see the awful downsides of them - like no version control, being proprietary, and having to type obscure incantations into tiny cells - and realise that actual coding is just better.

To bring this back on topic, software engineers see AI being a better search tool or a code suggestion tool on the one hand, but also having downsides (hallucinating, used by people to generate large amounts of slop that humans then have to sift through).

> It's just that we also see the awful downsides of them - like no version control, being proprietary, and having to type obscure incantations into tiny cells

Right. But this also tends to make us forget sometimes that those things aren't always a big deal. It's the distinction between solving an immediate problem vs. building a proper solution.

(That such one-off solution tends to become a permanent fixture in an organization - or household - is unfortunately an unsolved problem of human coordination.)

> and realise that actual coding is just better.

It is, if you already know how to do it. But then we overcompensate in the opposite direction, and suddenly 90% of the "actual coding" turns into dealing with build tools and platform bullshit, at which point some of us (like myself) look back at spreadsheets in envy, or start using LLMs to solve sub-problems directly.

It's actually unfortunate, IMO, that LLMs are so over-trained on React and all kinds of modern webshit - this makes them almost unable to give you simple solutions for anything involving web, unless you specifically prompt them to go full vanilla and KISS.

I'm constantly surprised that no one has mainstreamed version control. I see so many cases where it could be applied: document creation and editing, web site updates, spreadsheets ... even the way that laws are amended in Parliament [1]

[1] https://www.gov.uk/guidance/legislative-process-taking-a-bil... https://www.gov.uk/government/publications/amending-bills-st...

> But this also tends to make us forget sometimes that those things aren't always a big deal. It's the distinction between solving an immediate problem vs. building a proper solution.

I agree about the "code quality" not being a huge issue for some use cases, however having worked at places with entrenched spreadsheet workflows (like currently), I think that non engineers still need help seeing they don't need a faster horse - e.g. automate this task away. Many, many times a "spreadsheet" is ironically used for a very inefficient manual task.

> Many, many times a "spreadsheet" is ironically used for a very inefficient manual task.

Right. But spreadsheets and "shadow IT" aren't really about technology - they're about autonomy, about how the organization is structured internally. No one is choosing a bad process from the start - spreadsheets are the easiest (and often the only possible) way to solve an immediate problem, and even as they turn into IT horror stories, there usually is no point at which the people using it could make things better on their own. The "quality solutions", conversely, are usually top-down and don't give users much control over the process - instead of adoption, this just breeds resistance.

True, Excel is in the same category, yes.

People know which ingredients to use, the ratios, how long to bake and cook them but the design of the kitchen prevents them from cooking the meal? Professional cooks debate which gas tube to use with which adapter and how to organize all the adapters according to ISO standards while the various tubes lay on the floor all over the building. The stove switches off if you try to use the wrong brand of pots. The cupboard has a retina scanner. Eventually people go to the back of the garden and make a campfire. There is no fridge there and no way to wash dishes. They are even using the wrong utensils. The horror!

> everyone can code!

I work directly with marketers and even if you give them something like n8n, they find it hard to be precise. Programming teaches you a "precise mindset" that one doesn't have when they aren't really thinking about tech professionally.

I wonder if seasoned UX designers can code now. They do think professionally about software. I wonder if it's at a deep enough granularity such that they can simply use natural language to get something to work.

Our UX designers have been prototyping things they started in Figma with Windsurf. They seem pretty happy with it. Of course there's a big step in getting it production-ready but it really smooths the conversation with engineering.

Can an LLM detect a lack of precision and point it to you ?

Sometimes, yes. Reliably, no.

LLMs don't have enough of a model of the world to understand anything. There was a paper floating around recently about how someone trained an ML system on orbital dynamics. The result was a system that could calculate orbits correctly, but it completely failed to extract the underlying - simple - math. Instead it basically frankensteined together its own system of epicycles which solved a very narrow range of problems but lacked any generality.

Any coding has the same problems. Sometimes you get lucky, sometimes you don't. And if you strap on an emulator and test rig and allow the machine to flail around inside it, sometimes working code falls out.

But there's no abstracted model of software development as a process in there, either in theory or practise. And no understanding of vague goals with constraints and requirements that can be inferred creatively from outside the training data.

> LLMs don't have enough of a model of the world to understand anything.

This is binary thinking, and it's fallacious.

For your orbital mechanics example, sure, it's difficult for LLMs to develop good models of the physical world, in large part because they aren't able to interact with the world directly and have to rely on human texts to describe it to them.

For your software development example, you're making a similar mistake: the fact that their strongest suit is not producing fully working systems doesn't mean that they have no world model, or that their successes are as random as you seem to think ("Sometimes you get lucky, sometimes you don't," "sometimes working code falls out.")

But if you try, for example, asking an LLM to identify a bug in a program, or ask it questions about how a program works, you'll find that from a functional perspective, they exhibit excellent understanding that strongly implies a good world model. You may be taking your own thought processes for granted too much to realize how good they are at this. The idea that "there's no abstracted model of software development as a process in there" is hard to reconcile with the often superhuman responses they're capable of, when you use them in the scenarios they're most effective at.

An LLM can even ignore lack of precision and just guess what you wanted, usually correctly, unless what you want is very unusual.

It can! Though you might need to ask for it, otherwise it may take what it thinks you mean and run off with it, at which point you'll discover the lack of precision only later, when the LLM gets confused or the result is nothing like what you actually expected.

> It doesn't mean “AI is the new electricity.” (actual quote from Andrew Ng in the post).

I personally agree with Andrew Ng here (and I've literally arrived at the exact same formulation before becoming aware of Ng's words).

I take "new electricity" to mean, it'll touch everything people do, become part of every endeavor in some shape of form. Much like electricity. That doesn't mean taking over literally everything; there's plenty of things we don't use electricity for, because alternatives - usually much older alternatives - are still better.

There's still plenty of internal combustion engines on the ground, in the seas and in the skies, and many of them (mostly on extremely light and extremely heavy ends of the spectrum) are not going to be replaced by electric engines any time soon. Plenty of manufacturing and construction is still done by means of hydraulic and pneumatic power. We also sometimes sidestep electricity for heating purposes by going straight from sunlight to heat. Etc.

But even there, electricity-based technology is present in some form. The engine may be this humongous diesel-burning colossus, built from heat, metal, and a lot of pneumatics, positioned and held in place by hydraulics - but all the sensors on it are electric, where in the past some would be hydraulic and rest wouldn't even exist; it's controlled and operated by electricity-based computing network; it's been designed on computers, and so on.

In this sense, I think "AI is a new electricity" is believable. It's a qualitatively new approach to computing, that's directly or indirectly applicable everywhere, and that people already try to apply to literally everything[0]. And, much like with electricity, time and economics will tell which of those applications make sense, which were dead ends, and which were plain dumb in retrospect.

--

[0] - And they really did try to stuff electricity everywhere back when it was the new hot thing. Same with nuclear energy few decades later. We still laugh at how people 100 years ago imagined the future will look like... in between crying that we got short-changed by reality.

AI is not a fundamental physical element. AI is mostly closed and controlled by people who will inevitably use it to further their power and centralize wealth and control. We acted with this in mind to make electricity a publicly controlled service. There is absolutely no intention nor political strength around to do this with AI in the West.

There's a few levels of this:

• That it is software means that any given model can be easily ordered nationalised or whatever.

• Everyone quickly copying OpenAI, and specifically DeepSeek more recently, showed that once people know what kind of things actually work, it's not too hard to replicate it.

• We've only got a handful of ideas about how to align* AI with any specific goal or value, and a lot of ways it does go wrong. So even if every model was put into public ownership, it's not going to help, not yet.

That said, if the goal is to give everyone access to an AI that demands 375 W/capita 24/7, means the new servers double the global demand for electricity, with all that entails.

* Last I heard (a while back now so may have changed): if you have two models, there isn't even a way to rank them as more-or-less aligned vs. anything. Despite all the active research in this area, we're all just vibing alignment, corporate interests included.

Public control over AI models is a distinct thing from everyone having access to an AI server (not that national AI would need a 1:1 ratio of servers to people, either).

It's pretty obvious that the play right now is to lock down the AI as much as possible and use that to facilitate control over every system it gets integrated with. Right now there's too many active players to shut out random developers, but there's an ongoing trend of companies backing away from releasing open weight models.

> It's pretty obvious that the play right now is to lock down the AI as much as possible and use that to facilitate control over every system it gets integrated with. Right now there's too many active players to shut out random developers, but there's an ongoing trend of companies backing away from releasing open weight models.

More the opposite, despite the obvious investment incentive to do as you say to have any hope of a return on investment. OpenAI *tried* to make that a trend with GPT-2 on the grounds that it's irresponsible to give out a power tool in the absence of any idea of what "safety tests" even mean in that context, but lots of people mocked them for it and it looks like only them and Anthropic take such risks seriously. Or possibly just Anthropic, depending how cynical you are about Altman.

Electricity here is meant as a technology (or a set of technologies) exploiting a particular physical phenomenon - not the phenomenon itself.

(If it were the latter, then you could argue everything uses electricity if it relies in any way on matter being solid, because AFAIK the furthest we got on the question of "why I don't fall through the chair I'm sitting on" is.... "electromagnetism".)

Either way, it still feels like a stretched and inappropriate comparison at best, or a disingenuous and asinine one at worst.

While I'd agree with your first line:

> The problem with LLM is when they're used for creativity or for thinking.

And while I also agree that it's currently closer to "AI is the new VBA" because of the current domain in which consumer AI* is most useful.

Despite that, I'd also aver that being useful in simply "many" contexts will make AI "the new electricity”. Electricity itself is (or recently was) only about 15% of global primary power, about 3 TW out of about 20 TW: https://en.wikipedia.org/wiki/World_energy_supply_and_consum...

Are LLMs 15% of all labour? Not just coding, but overall? No. The economic impact would be directly noticeable if it was that much.

Currently though, I agree. New VBA. Or new smartphone, in that we ~all have and use them, while society as a whole simultaneously cringes a bit at this.

* Narrower AI such as AlphaFold etc. would, in this analogy, be more like a Steam Age factory which had a massive custom steam engine in the middle distributing motive power to the equipment directly: it's fine at what it does, but you have to make it specifically for your goal and can't easily adapt it for something else later.

LLM is helpful for creativity and thinking When you run out of your ideas

I sometimes feel that a lot of people bringing up the topic of creativity have never spent much time thinking, studying and self-reflecting on what "creativity" actually is. It's a complex topic and one that's mixed up with many other complex topics ("originality", "intellectual property", "aesthetic value", "art vs engineering" etc etc)

You see a lot of Motte and Bailey arguments in this discussion as people shift (often subconsciously) between different definitions of key terms and different historical perspectives.

I'd recommend someone tries to gain at least a passing familiarity with art history and the social history of art/design etc. Reading a bit of Edward De Bono and Douglas Hofstadter isn't a bad shout either (although it's many years since I've read the former so I can't guarantee it will stand up as well as my teenage self thought it did)

You're discounting the times when it doesn't work. I recently experienced a weird 4X slowdown across multiple VirtualBox VM's on a Windows 10 host. AI led me down rabbit holes that didn't solve the problem.

I finally noticed a configuration problem. For some weird reason, in the Windows Features control panel, the "Virtual Machine Platform" checkbox had become unchecked (spontaneously; I did not touch this).

I mentioned this to AI, which insisted on not flipping that option, that it is not it.

> "Virtual Machine Platform" sounds exactly like something that should be checked for virtualization to work, and it's a common area of conflict. However, this is actually a critical clarification that CONFIRMS we were on the right track earlier! "Virtual Machine Platform" being UNCHECKED in Windows Features is actually the desired state for VirtualBox to run optimally.'

In fact, it was that problem. I checked the option, rebooted the host OS, and the VMs ran at proper speed.

AI can not only not be trusted to make deep inferences correctly, it falters on basic associative recall of facts. If you use it as a substitute for web searches, you have to fact check everything.

LLM AI has no concept of facts. Token prediction is not facts; it's just something that is likely to produce facts, given the right query in relation to the right training data.

I am absolutely on board with the LLM inevitablism. It seems inevitable as you describe it. Everyone will use them everyday. Like smartphones.

I am absolutely not on board with AGI inevitablism. Saying “AGI is inevitable because models keep getting better” is an inductive leap that is not guaranteed.

100%. AGI is some arbitrary threshold that has nothing to do with LLMs and their broad application, rapid improvement and exploding ecosystem. The only comparable shift in memory is the advent of the Internet, and that was much slower for obvious reasons (everyone needs a computer, needs to use the phone line, needs to upgrade to faster internet, etc., etc.)

LLMs are just here and accessible to everyone with all kinds of uses. And people that hate them and don't want LLMs shoved in their face: it doesn't matter. Chatbots were already a thing for customer service, now they are almost implicit. SaaS in general is going to be completely revamped. Huge companies are going to have a shift like they did when they first digitized paper files. This is seismic and a large-scale structural upgrade to many institutions.

And that is without giving a shit about public facing LLMs or Ghibli art or writing your term papers for you. It doesn't have anything to do with AI making you dumb or being shoved into products. That is random noise, big picture. Anyone that can't see the real changes to business and knowledge retrieval isn't thinking this through or is actively burying their heads in the sand.

I doubt that LLM will keep getting better, too. Or at least, not in an economically sustainable way

Yes, people often conflate the two. ChatGPT is nice for some coding things and asking what you want for dinner. That does not mean we're all out of a job and our society has to be radically rethought.

Are you sure, that the code works correctly? ;)

Now, imagine, what you would do, if you never learned to read the code.

As you were always using only AI.

Anyway, coding is much simpler and easier than reading someone else's code. And I rather code it myself than spend time to actually read and study what AI has outputted. As at the end, I need to know that code works.

---

At one point, my former boss was explaining to me, how they were hired by some plane making company, to improve their firmware for controlling rear flaps. They have found some float problem and were flying to meeting, to explain what the issue was. (edit:) While flying, they figured out that they are flying with plane having that exact firmware.