Andrej Karpathy: Software in the era of AI [video] (youtube.com)

I kind of say it in words (agreeing with you) but I agree the versioning is a bit confusing analogy because it usually additionally implies some kind of improvement. When I’m just trying to distinguish them as very different software categories.

It's not just the hardware constraints - it's also the training constraints, and the legibility constraints.

Training constraints: you need lots, and lots of data to build complex neural network systems. There are plenty of situations where the data just isn't available to you (whether for legal reasons, technical reasons, or just because it doesn't exist).

Legibility constraints: it is extremely hard to precisely debug and fix those systems. Let's say you build a software system to fill out tax forms - one the "traditional" way, and one that's a neural network. Now your system exhibits a bug where line 58(b) gets sometimes improperly filled out for software engineers who are married, have children, and also declared a source of overseas income. In a traditionally implemented system, you can step through the code and pinpoint why those specific conditions lead to a bug. In a neural network system, not so much.

So totally agreed with you that those are extra tools in the toolbelt - but their applicability is much, much more constrained than that of traditional code.

In short, they excel at situations where we are trying to model an extremely complex system - one that is impossible to nail down as a list of formal requirements - and where we have lots of data available. Signal processing (like self driving, OCR, etc) and human language-related problems are great examples of such problems where traditional programming approaches have failed to yield the kind of results we wanted (ie, beyond human performance) in 70+ years of research and where the modern, neural network approach finally got us the kind of results we wanted.

But if you can define the problem you're trying to solve as formal requirements, then those tools are probably ill-suited.

Weights are code being replaced by data; something I've been making heavy use of since the early 00s. After coding for 10 years you start to see the benefits of it and understand where you should use it.

LLMs give us another tool only this time it's far more accessible and powerful.

LLMs have already replaced some code directly for me eg NLP stuff. Previously I might write a bunch of code to do clustering now I just ask the LLM to group things. Obviously this is a very basic feature native to LLMs but there will be more first class LLM callable functions over time.

Why bother using human language to communicate with a computer? You interact with a computer using a programming language—code—which is more precise and effective. Specifically: → In 1.0, you communicate with computers using compiled code. → In 2.0, you communicate with compilers using high-level programming languages. → In 3.0, you interact with LLMs using prompts, which arguably should not be in natural human language. Nonetheless, you should communicate with AGIs using human language, just as you would with other human beings.

If this is what it comes to, it would explain the many, many software malfunctions in Star Trek. If everything is an LLM/LRM (or whatever super advanced version they have in the 23rd century) then everything can evolve into weird emergent behaviours.

stares at every weird holo-deck episode

LLMs are not inherently indeterministic. Batching, temperature, and other things make them appear so when run by big providers but a locally-run LLM model at zero temperature will always produce the same output given the same input.

Why does this remind me of COBOL.

    def __main__:
        You run main(). If there are issues, you edit __file__ to try to fix the errors and re-run it. You are determined, persistent, and never give up.

You know, the more I think about it, the more I like this model.

What we have today with ChatGPT and the like (and even IDE integrations and API use) is imperative right, it's like 'answer this question' or 'do this thing for me', it's a function invocation. Whereas the silly calculator program I presented above is (unintentionally) kind of a declarative probabilistic program - it's 'this is the behaviour I want, make it so' or 'I have these constraints and these unknowns, fill in the gaps'.

What if we had something like Prolog, but with the possibility of facts being kind of on-demand at runtime, powered by the LLM driving it?

This (sort of) is already a paradigm: https://en.m.wikipedia.org/wiki/Probabilistic_programming

I wonder when companies will remove the personality out of LLMs by default, especially for tools

> maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly

I can't believe someone would seriously write this and not realize how nonsensical it is.

"indeterministic programming", you seriously cannot come up with a bigger oxymoron.

[flagged]

> So the question is, can you hide the formal stuff under the hood, just like you can hide a calculator tool for arithmetic? Use informal English on the surface, while some of it is interpreted as a formal expression, put to work, and then reflected back in English?

The problem with trying to make "English -> formal language -> (anything else)" work is that informality is, by definition, not a formal specification and therefore subject to ambiguity. The inverse is not nearly as difficult to support.

Much like how a property in an API initially defined as being optional cannot be made mandatory without potentially breaking clients, whereas making a mandatory property optional can be backward compatible. IOW, the cardinality of "0 .. 1" is a strict superset of "1".

> Use informal English on the surface, while some of it is interpreted as a formal expression, put to work, and then reflected back in English? I think that is possible, if you have a formal language and logic that is flexible enough, and close enough to informal English.

That sounds like a paradox.

Formal verification can prove that constraints are held. English cannot. mapping between them necessarily requires disambiguation. How would you construct such a disambiguation algorithm which must, by its nature, be deterministic?

Not gonna lie, after skimming the website and a couple preprints for 10 minutes my crank detector is off the charts. Your very vague comments adds to it.

But maybe I just don't understand.

> "English is the new programming language."

For those who missed it, here's the viral tweet by Karpathy himself: https://x.com/karpathy/status/1617979122625712128

lean 4/5 will be a rising star!

This thread perfectly captures what Karpathy was getting at. We're witnessing a fundamental shift where the interface to computing is changing from formal syntax to natural language. But you can see people struggling to let go of the formal foundations they've built their careers on.

On one hand, I think Karpathy is a gifted educator in a way that's not repeatable as a science. On the other, if the conference leaders next year told every presenter to watch this talk and emulate how Karpathy focuses on concrete insights and suggests what to build now, then the overall quality of presentations would probably trend higher.

Fun demo of an early idea was posted by Oriol just yesterday :)

https://x.com/OriolVinyalsML/status/1935005985070084197

My friend Eric Pelz started a company called Malleable to do this very thing: https://www.linkedin.com/posts/epelz_every-piece-of-software...

An ever-shifting UI sounds unlearnable, and therefore unusable.

Like a HyperCard application?

This talk https://www.youtube.com/watch?v=MbWgRuM-7X8 explores the idea of generative / malleable personal user interfaces where LLMs can serve as the gateway to program how we want our UI to be rendered.

I love this concept and would love to know where to look for people working on this type of thing!

It's probably Jira. https://medium.com/question-park/all-aboard-the-ai-train-b03...

Humans are shit at interacting with systems in a non-linear way. Just look at Jupyter notebooks and the absolute mess that arises when you execute code blocks in arbitrary order.

Define non-essenti

The way I see dependency in office ("knowledge") work:

- pre-(computing) history. We are at the office, we work

- dawn of the pc: my computer is down, work halts

- dawn of the lan: the network is down, work halts

- dawn of the Internet: the Internet connection is down, work halts (<- we are basically all here)

- dawn of the LLM: ChatGPT is down, work halts (<- for many, we are here already)

> programming

The programming analogy is convenient but off. The joke has always been “the computer only does exactly what you tell it to do!” regarding logic bugs. Prompts and LLMs most certainly do not work like that.

I loved the parallels with modern LLMs and time sharing he presented though.

only in English, and also non-deterministic.

that's a really good analogy! It feels like wicked joke that llms behave in such a way that they're both intelligent and stupid at the same time

“You are an expert 10x software developer. Make me a billion dollar app.” Yeah this checks out

Here’s a few problems I foresee:

1. People get lazy when presented with four choices they had no hand in creating, and they don’t look over the four and just click one, ignoring the others. Why? Because they have ten more of these on the go at once, diminishing their overall focus.

2. Automated tests, end-to-end sim., linting, etc—tools already exist and work at scale. They should be robust and THOROUGHLY reviewed by both AI and humans ideally.

3. AI is good for code reviews and “another set of eyes” but man it makes serious mistakes sometimes.

An anecdote for (1), when ChatGPT tries to A/B test me with two answers, it’s incredibly burdensome for me to read twice virtually the same thing with minimal differences.

Code reviewing four things that do almost the same thing is more of a burden than writing the same thing once myself.

I don't think the human is the problem here, but the time it takes to run the full testing suite.

The problem is that every time you run your full automation with linting and tests, you’re filling up the context window more and more. I don’t know how people using Claude do it with its <300k context window. I get the “your message will exceed the length of this chat” message so many times.

In my experience with Jules and (worse) Codex, juggling multiple pull requests at once is not advised.

Even if you tell the git-aware Jules to handle a merge conflict within the context window the patch was generated, it is like sorry bro I have no idea what's wrong can you send me a diff with the conflict?

I find i have to be in the iteration loop at every stage or else the agent will forget what it's doing or why rapidly. for instance don't trust Jules to run your full test suite after every change without handholding and asking for specific run results every time.

It feels like to an LLM, gaslighting you with code that nominally addresses the core of what you just asked while completely breaking unrelated code or disregarding previously discussed parameters is an unmitigated success.

> Tight feedback loops are the key in working productively with software. […] even more tight loops than what a sane human would tolerate.

Why would a sane human be averse to things happening instantaneously?

> Or generating 4 versions of PR for the same task so that the human could just pick the best one.

That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality. Why are we doing this to ourselves and embracing it?

A few years ago, it would have been seen as a joke to say “the future of software development will be to have a million monkey interns banging on one million keyboards and submit a million PRs, then choose one”. Today, it’s lauded as a brilliant business and cost-saving idea.

We’re beyond doomed. The first major catastrophe caused by sloppy AI code can’t come soon enough. The sooner it happens, the better chance we have to self-correct.

I would venture that 'tightening the feedback loop' isn't necessarily 'increasing the number of back and forth prompts'- and what you're saying you want is ultimately his argument. i.e. if integral enough it can almost guess what you're going to say next...

Yeah I am currently enjoying giving the LLM relatively small chunks of code to write and then asking it to write accompanying tests. While I focus on testing the product myself. I then don't even bother to read the code it's written most of the time

I think that if you give the same task to three different developers you'll get three different implementations. It's not a random result if you do get the functionality that was expected, and at that, I do think the prompt plays an important role in offering a view of how the result was achieved.

Code is read much more often than it is written. Code generated by the machine today will be prompt read by the machine going forward. It's a closed loop.

Software is a world in motion. Software 1.0 was animated by developers pushing it around. Software 3.0 is additionally animated by AI agents.

How I understood it is that natural language will form relatively large portions of stacks (endpoint descriptions, instructions, prompts, documentations, etc…). In addition to code generated by agents (which would fall under 1.0)

It is not the code, which just like prompts is a written language. Software 3.0 will be branches of behaviors, by the software and by the users all documented in a feedback loop. The best behaviors will be merged by users and the best will become the new HEAD. Underneath it all will be machine code for the hardware, but it will be the results that dictate progress.

That blog post is a great illustration that most of the complexity/difficulty of a web app is in the hosting and not in the useful code.

clerk is an auth library - and finally one that doesnt require dozens of lines to do things like, i dont know, check if the user is logged in

and wild... you used gemini to process a screenshot to find the website for a 5 letter word library?

Audio maybe the better option.

[dead]

I am actually working on building a semantic TypeScript server right now. It's going really good, check it out. https://github.com/screencam/typescript-mcp-server

You don't "solve" autonomous driving as such. There's a long, slow grind of gradually improving things until failures become rare enough.

> Karpathy is consistently one of the clearest thinkers out there.

Eh, he ran Teslas self driving division and put them into a direction that is never going to fully work.

What they should have done is a) trained a neural net to represent sequence of frames into a physical environment, and b)leveraged Mu Zero, so that self driving system basically builds out parallel simulations into the future, and does a search on the best course of action to take.

Because thats pretty much what makes humans great drivers. We don't need to know what a cone is - we internally compute that something that is an object on the road that we are driving towards is going to result in a negative outcome when we collide with it.

I think this lines up with Apple’s thesis of on-device models being a useful feature for developers who don’t want to deal with calling out the OpenAI

https://developer.apple.com/documentation/foundationmodels

Yeah, hit this exact wall building a small AI tool. Ended up spinning up a whole backend just to keep the keys safe. Feels like there should be a simpler way, but haven’t seen anything that’s truly plug-and-play yet. Curious to see what you’re working on.

I think the way the friction could be reduced to almost zero was through OpenAI "custom GPTs" https://help.openai.com/en/articles/8554397-creating-a-gpt or "Alexa skills". how much easier can it get than the user using their own OpenAI account? Of course I'd rather have them on my own website but if were talking complete ease of use then I think that is a contender

Do you think Firebase and Superbase are working on this? Good luck but to me it sounds like a platform feature, not a standalone product.

[dead]

You can disagree with his conclusions but I don't think his understanding of small models is up for debate. This is the person who created micrograd/makemore/nanoGPT and who has produced a ton of educational materials showing how to build small and local models.

> suggests a lack of understanding of these smaller models capabilities

If anything, you're showing a lack of understanding of what he was talking about. The context is this specific time, where we're early in a ecosystem and things are expensive and likely centralized (ala mainframes) but if his analogy/prediction is correct, we'll have a "Linux" moment in the future where that equation changes (again) and local models are competitive.

And while I'm a huge fan of local models run them for maybe 60-70% of what I do with LLMs, they're nowhere near proprietary ones today, sadly. I want them to, really badly, but it's important to be realistic here and realize the differences of what a normal consumer can run, and what the current mainframes can run.

Of all the things you could suggest, a lack of understanding is not one that can be pinned on Karpathy. He does know his technical stuff.

He ain't dismissing them. Comparing local/"open" model to Linux (and closed services to Windows and MacOS) is high praise. It's also accurate.

I tried the local small models. They are slow, much less capable, and ironically much more expensive to run than the frontier cloud models.

We couldn't let that happen!

I like the idea of having a single source of truth RULES.md, however I'm wondering why you used symlinks as opposed to the ability to link/reference other files in cursor rules, CLAUDE.md, etc. I understand that functionality doesn't exist for all coding agents, but I think it gives you more flexibility when composing rules files (for example you can have the standard cursor rules headers and then point to @RULES.md lower in the file)

[flagged]

It will succeed due to the same reason other sloppy strategies succeed, it has large short term gains and moves risk into the nebulous future. Management LOVES these types of things.

I've found that as LLMs improve, some of their bugs become increasingly slippery - I think of it as the uncanny valley of code.

Put another way, when I cause bugs, they are often glaring (more typos, fewer logic mistakes). Plus, as the author it's often straightforward to debug since you already have a deep sense for how the code works - you lived through it.

So far, using LLMs has downgraded my productivity. The bugs LLMs introduce are often subtle logical errors, yet "working" code. These errors are especially hard to debug when you didn't write the code yourself — now you have to learn the code as if you wrote it anyway.

I also find it more stressful deploying LLM code. I know in my bones how carefully I write code, due to a decade of roughly "one non critical bug per 10k lines" that keeps me asleep at night. The quality of LLM code can be quite chaotic.

That said, I'm not holding my breath. I expect this to all flip someday, with an LLM becoming a better and more stable coder than I am, so I guess I will keep working with them to make sure I'm proficient when that day comes.

Just like when all regulated industries started only using decision trees and ordinary least-squares regression instead of any other models.

> how many IT leaders would accept the output code of something like a menugen app as production-viable.

probably all of the ones at microsoft

Well there are cases like OLMo where the process, dataset, and model are all open source. As expected though, it doesn't really compare well to the worst closed model since the dataset can't contain vast amounts of stolen copyrighted data that noticeably improves the model. Llama is not good because Meta knows what they're doing, it's good because it was pretrained on the entirety of Anna's Archive and every pirated ebook they could get their hands on. Same goes for Elevenlabs and pirated audiobooks.

Lack of compute on the Ai2's side also means the context OLMo is trained for is miniscule, the other thing that you need to throw brazillions of dollars at to make model that's maybe useful in the end if you're very lucky. Training needs high GPU interconnect bandwidth, it can't be done in distributed horde in any meaningful way even if people wanted to.

The only ones who have the power now are the Chinese, since they can easily ignore copyright for datasets, patents for compute, and have infinite state funding.

> Am I all of a sudden the one lacking imagination?

No, The reality of what these tools can do is sinking in.. The rubber is meeting the road and I can hear some screaching.

The boosters are in 5 stages of grief coming to terms with what was once AGI and is now a mere co-pilot, while the haters are coming to terms with the fact that LLMs can actually be useful in a variety of usecases.

> and must say that I'm a bit confused by this presentation, and it's a bit unclear to me what it adds.

I think the disconnect might come from the fact that Karpathy is speaking as someone who's day-to-day computing work has already been radically transformed by this technology (and he interacts with a ton of other people for whom this is the case), so he's not trying to sell the possibility of it: that would be like trying to sell the possibility of an airplane for someone who's already just cruising around in one every day. Instead the mode of the presentation is more: well, here we are at the dawn of a new era of computing, it really happened. Now how can we relate this to the history of computing to anticipate where we're headed next?

> ...but sometimes an LLM becomes the operating system, sometimes it's the CPU, sometimes it's the mainframe from the 60s with time-sharing, a big fab complex, or even outright electricity itself?

He uses these analogies in clear and distinct ways to characterize separate facets of the technology. If you were unclear on the meanings of the separate analogies it seems like the talk may offer some value for you after all but you may be missing some prerequisites.

> This demo app was in a presentable state for a demo after a day, and it took him a week to implement Googles OAuth2 stuff. Is that somehow exciting? What was that?

The point here was that he'd built the core of the app within a day without knowing the Swift language or ios app dev ecosystem by leveraging LLMs, but that part of the process remains old-fashioned and blocks people from leveraging LLMs as they can when writing code—and he goes on to show concretely how this could be improved.

The fundamental mistake I see is people applying LLMs to the current paradigm of software; enormous hulking codebases made to have as many features as possible to appeal to as many users as possible.

LLMs are excellent at helping non-programmers write narrow use case, bespoke programs. LLMs don't need to be able to one-shot excel.exe or Plantio.apk so that Christine can easily track when she watered and fed her plants nutrients.

The change that LLMs will bring to computing is much deeper than Garden Software trying to slot in some LLM workers to work on their sprawling feature-pack Plantio SaaS.

I can tell you first hand I have already done this numerous times as a non-programmer working a non-tech job.

What you're missing is the audience.

This talk is different from his others because it's directed at aspiring startup founders. It's about how we conceptualize the place of an LLM in a new business. It's designed to provide a series of analogies any one of which which may or may not help a given startup founder to break out of the tired, binary talking points they've absorbed from the internet ("AI all the things" vs "AI is terrible") in favor of a more nuanced perspective of the role of AI in their plans. It's soft and squishy rhetoric because it's not about engineering, it's about business and strategy.

I honestly left impressed that Karpathy has the dynamic range necessary to speak to both engineers and business people, but it also makes sense that a lot of engineers would come out of this very confused at what he's on about.

Couldn’t believe my eyes. The www is truly bankrupt. If anyone has a browser plugin which automatically redirects to llms.txt sign me up.

Website too confusing for humans? Add more design, modals, newsletter pop ups, cookie banners, ads, …

Website too confusing for LLMs? Add an accessible, clean, ad-free, concise, high entropy, plain text summary of your website. Make sure to hide it from the humans!

PS: it should be /.well-known/llms.txt but that feels futile at this point..

PPS: I enjoyed the talk, thanks.

If you have different representations of the same thing (llms.txt / HTML), how do you know it is actually equivalent to each other? I am wondering if there are scenarios where webpage publishers would be interested in gaming this.

[dead]

It does seem a bit silly long term to have something like Python which was developed as a human friendly language written by LLMs.

If AI is going to write all the code going forward, we can probably dispense with the user friendly part and just make everything efficient as possible for machines.

Is a world driven by the strengths and weaknesses of programming languages better than the one driven by the strengths and weaknesses of LLMs?

Given the plethora of programming languages that exist today, I'm not worried at all about AI taking over SWE jobs.

[dead]

[dead]

Its early days. Agree with your point that the "vision" of the future laid out by tech people doesn't have much of a chance of becoming (accepted) reality, because its necessarily a reflection of their own inner world, largely devoid of importance and interactions with other people. Prime example, see metaverse. Most of us don't want to replace the real world with a (crappy) digital one; the sooner we build things that respects that fundamental value, the sooner we can build things that actually improves our lives.

did you not have the computer room open to flash games and the like over lunch time? competitive 4 player bmtron was a blast way back whenhttps://www.games1729.com/archive/

I thought that video was generated. Everything about it seemed off

If the prompt is good, the LLM will tell you when it's wrong, but you can use production testing if necessary like Tesla.

I didn't get the impression that he's against text per se, just that LLMs should use a format that's most concise for humans in the given scenario. Example from the video: showing the (textual) diff between old and new versions of text/code, rather than just the new version. Or converting a text-only restaurant menu to photos+text.

You may be interested in: https://www.anthropic.com/research/sleeper-agents-training-d... https://arxiv.org/abs/2404.13660

This is what mechanistic interpretability studies are trying to achieve, and it's not yet realistically possible for a general case.

This is a good insight. There’s also a similar insight about compilers back in the days before AV.. we will have AV LLMs etc… basically reinvent everything for the new stack.

I was just talking to somebody at work about a "Trusting Trust" style attack from LLMs. I will remain deeply suspicious of them

Profit over security, outsource liability

I do feel like large scale LLM vulnerabilities will be the real Y2K

> Meanwhile

What do you mean "meanwhile", that's exactly (among other things) the kind of stuff he's talking about? The various frictions and how you need to approach it

> anyone actually working in the space

Is this trying to say that Karpathy doesn't "actually work" with LLMs or in the ML space?

I feel like your whole comment is just reacting to the title of the YouTube video, rather than actually thinking and reflecting on the content itself.

Seems to me that this is just another level of throwing compute at the problem.

Same way programs was way more efficient before and now they are "bloated" with packages, abstractions, slow implementations of algos and scaffolding.

The concept of what is good software development might be changing as well.

LLMs might not write the best code, but they sure can write a lot of it.

A manager in our company introduced Gemini as a chat bot coupled to our documentation.

> It failed to write out our company name.The rest was flawed with hallucinations also, hardly worth to mention.

I wish this is a rage bait towards others, but what should me feelings be? After all this is the tool thats sold to me, I am expected to work with.

The real question is how long it'll take until they're not brittle

On the other hand, posts like this are like watching someone writing ask jeeves search queries into google 20 years ago and then gesturing how google sucks while everyone else in the room has figured out how to be productive with it and cringes at his "boomer" queries.

If you're still struggling to make LLMs useful for you by now, you should probably ask someone. Don't let other noobs on HN +1'ing you hold you back.

https://theeducationist.info/everything-amazing-nobody-happy...

There's also those instances where Microsoft unleashed Copilot on the .NET repo, and it resulted in the most hilariously terrible PRs that required the maintainers to basically tell Copilot every single step it should take to fix the issue. They were basically writing the PRs themselves at that point, except doing it through an intermediary that was much dumber, slower and less practical than them.

And don't get me started on my own experiences with these things, and no, I'm not a luddite, I've tried my damndest and have followed all the cutting-edge advice you see posted on HN and elsewhere.

Time and time again, the reality of these tools falls flat on their face while people like Andrej hype things up as if we're 5 minutes away from having Claude become Skynet or whatever, or as he puts it, before we enter the world of "Software 3.0" (coincidentally totally unrelated to Web 3.0 and the grift we had to endure there, I'm sure).

To intercept the common arguments,

- no I'm not saying LLMs are useless or have no usecases

- yes there's a possibility if you extrapolate by current trends (https://xkcd.com/605/) that they indeed will be Skynet

- yes I've tried the latest and greatest model released 7 minutes ago to the best of my ability

- yes I've tried giving it prompts so detailed a literal infant could follow along and accomplish the task

- yes I've fiddled with providing it more/less context

- yes I've tried keeping it to a single chat rather than multiple chats, as well as vice versa

- yes I've tried Claude Code, Gemini Pro 2.5 With Deep Research, Roocode, Cursor, Junie, etc.

- yes I've tried having 50 different "agents" running and only choosing the best output form the lot.

I'm sure there's a new gotcha being written up as we speak, probably something along the lines of "Well for me it doubled my productivity!" and that's great, I'm genuinely happy for you if that's the case, but for me and my team who have been trying diligently to use these tools for anything that wasn't a microscopic toy project, it has fallen apart time and time again.

The idea of an application UI or god forbid an entire fucking Operating System being run via these bullshit generators is just laughable to me, it's like I'm living on a different planet.

Kids? Think about all the domain experts, entrepreneurs, researchers, designers, and creative people who have incredible ideas but have been locked out of software development because they couldn't invest 5-10 years learning to code.

A 50-year-old doctor who wants to build a specialized medical tool, a teacher who sees exactly what educational software should look like, a small business owner who knows their industry's pain points better than any developer. These people have been sitting on the sidelines because the barrier to entry was so high.

The "vibe coding" revolution isn't really about kids (though that's cute) - it's about unleashing all the pent-up innovation from people who understand problems deeply but couldn't translate that understanding into software.

It's like the web democratized publishing, or smartphones democratized photography. Suddenly expertise in the domain matters more than expertise in the tools.

> those kids produced with their vibe coding

No one, including Karpathy in this video, is advocating for "vibe coding". If nothing more, LLMs paired with configurable tool-usage, is basically a highly advanced and contextual search engine you can ask questions. Are you not using a search engine today?

Even without LLMs being able to produce code or act as agents they'd be useful, because of that.

But it sucks we cannot run competitive models locally, I agree, it is somewhat of a "rich people" tool today. Going by the talk and theme, I'd agree it's a phase, like computing itself had phases. But you're gonna have to actually watch and listen to the talk itself, right now you're basically agreeing with the video yet wrote your comment like you disagree.

This is most definitely not toys for rich people. Now perhaps depending on your country it may be considered rich but I would comfortably say that for most of the developed world, the costs for these tools are absolutely attainable, there is a reason ChatGPT has such a large subscriber base.

Also the disconnect for me here is I think back on the cost of electronics, prices for the level of compute have generally gone down significantly over time. The c64 launched around the $5-600 price level, not adjusted for inflation. You can go and buy a Mac mini for that price today.

>What would be interesting to see is what those kids produced with their vibe coding.

I think you are referring to what those kids in the vibe coding event produced. Wasn't their output available in the video itself?

> This is a toy for rich people

GitHub copilot has a free tier.

Google gives you thousands of free LLM API calls per day.

There are other free providers too.

Where was he was saying you could vibe code beyond a simple app?

He even said it could be a gateway to actual programming

There is another angle to this too.

Prior to LLMs, it was amusing to consider how ML folks and software folks would talk passed each other. It was amusing because both sides were great at what they do, neither side understood the other side, and they had to work together anyway.

After LLMs, we now have lots of ML folks talking about the future of software, so ething previously established to be so outside their expertise that communication with software engineers was an amusing challenge.

So I must ask, are ML folks actually qualified to know the future of software engineering? Shouldnt we be listening to software engineers instead?

I've seen evidence of "anyone can vibe code", but at this stage the result tends to be a 5,000-line application intricately entangled with 500,000 lines of irrelevant slop. Still, the wonder is that the bear can dance at all. That's a new thing under the sun.

He says that now we are in the mainframe phase. We will hit the personal computing phase hopefully soon. He says llama (and DeepSeek?) are like Linux in a way, OpenAI and Claude are like Windows and MacOS.

So, No, he’s actually saying it may be everywhere for cheap soon.

I find the talk to be refreshingly intellectually honest and unbiased. Like the opposite of a cringey LinkedIn post on AI.

I think it used to be like that before the GNU people made gcc, completely destroying the market of compilers.

> Also, as a lazy developer who is always trying to make AI do my job for me, it still kind of sucks, and its not clear that it will make my life easier any time soon.

Every time I have to write a simple self contained couple of functions I try… and it gets it completely wrong.

It's easier to just write it myself rather than to iterate 50 times and hope it will work, considering iterations are also very slow.

On a tangent, I find the analogies interesting as well. However, while Karpathy is an expert in Computer Science, NLP and machine vision, his understanding of how human psychology and brain work is as good as you an I (non-experts). So I take some of those comparisons as a lay person’s feelings about the subject. Still, they are fun to listen to.

> Traditional: Read 5000 lines → Find method → Replace → Write 5000 lines

What of today's agents work like this? None of the ones I've tried would do something like that, but instead would grep/search the file, then do a smaller edit (different tools do those in different ways).

Overall, it does feel like a strawman argument against "Traditional" when almost none of the tooling actually works like that.

how do i install this thing?

no, but a subset of English could be

Yeah, usually I'd steer my agents to never use the output directly from any command, and instead redirect it to a logfile, then force it to search/grep stuff directly from the log-file instead of just getting all the outputs at all times. Seems to work OK.

Governments obviously lead in military tech, but do you think they have access to better AI (in general) than consumers? Unless they do, I think it's fair to say that governments are behind the curve, since consumers tend to adopt things more quickly.

[dead]

No one wanted to! I think we might have bitten off more than we could chew in terms of video production. There is a lot of content to publish.

Once it was clear how high the demand was for this talk, the team adapted quickly.

That's how it goes sometimes! Future iterations will be different.

slides: https://docs.google.com/presentation/d/1sZqMAoIJDxz79cbC5ap5...

Product folks don’t know what they want a lot of the time and don’t know what’s possible

In his defense he clearly articulated that meaningful change has not yet been achieved and could be a decade away. Even pointing to specific examples of LLMs failing to count letters and do basic arithmetic.

What I find absent is where do we go from LLMs? More hardware, more training. "This isn't the scientific breakthrough you're looking for".

Reddit and YouTube are such huge social media platforms that it really depends on which bubble (read: subreddits/yt channels) you're looking at. There's the "AGI is here" people over at r/singularity and then the "AI is useless" people at r/programming. I'm simplifying arguments from both sides here but you get my point.

> coding is no longer the problem.

"Coding" - The art of literally using your fingers to type weird characters into a computer, was never a problem developers had.

The problem has always been understanding and communication, and neither of those have been solved at this moment. If anything, they have gotten even more important, as usually humans can infer things or pick up stuff by experience, but LLMs cannot, and you have to be very precise and exact about what you're telling them.

And so the problem remains the same. "How do I communicate what I want to this person, while keeping the context as small as possible as to not overflow, yet extensive enough to cover everything?" except you're sending it to endpoint A instead of endpoint B.

I will counter this with the fact that sometimes, and depending on the abstraction level that you are trying to solve/work at code or some other determinstic language is the right and easier way language to describe the context. This doesn't just apply to SWE, but all forms of engineering (electrical, civil, mechanical, etc).

We have math notation for maths, diagrams for circuits, plans for houses, etc etc. Would hate to have to give long paragraphs of "English" to my house builder and watch what the result could be. Feels like being a lawyer at this point. English can be appropriate and now we also have that in our toolbox.

Describing context at the abstraction level and accuracy you care about has always been the issue. The context of what matters though as you grow and the same system has to deal with more requirements at once together IMV is always the challenge in ANY engineering discipline.

[dead]

"We're not building Iron Man robots. We're building Iron Man suits"

2.0 programs (model weights) are created by running 1.0 programs (training runs).

I don't think it's currently possible to ask a model to generate the weights for a model.

The whole AI scene is starting to feel a lot like the cryptocurrency bubble before it burst. Don’t get me wrong, there’s real value in the field, but the hype, the influencers, and the flashy “salon tricks” are starting to drown out meaningful ML research (like Apple's critical research that actually improves AI robustness). It’s frustrating to see solid work being sidelined or even mocked in favor of vibe-coding.

Meanwhile, I asked this morning Claude 4 to write a simple EXIF normalizer. After two rounds of prompting it to double-check its code, I still had to point out that it makes no sense to load the entire image for re-orientating if the EXIF orientation is fine in the first place.

Vibe vs reality, and anyone actually working in the space daily can attest how brittle these systems are.

> "Because they all have slight pros and cons, and you may want to program some functionality in 1.0 or 2.0, or 3.0, or you're going to train in LLM, or you're going to just run from LLM"

He doesn't say they will fully replace each other (or had fully replaced each other, since his definition of 2.0 is quite old by now)

> The beginning was painful to watch as is the cheering in this comment section.

Yours is the second comment claiming there is "cheering" and "fanboying" in this comment section. What comments are you talking about? I've read through this submission multiple times since yesterday, yet I've seen none of that. What specific comments are the "cheering" ones?

The version numbers mean abrupt changes.

Analogy: how we "moved" from using Google to ChatGPT is an abrupt change, and we still use Google.

[dead]

Some people are annoyed at the hype, some are making good faith arguments about the pros/cons, and some people are just cranky. AI is a popular subject and we've all got our hot takes.

Foundational programmers form the base of where the seed can grow.

In a way programmers found where our roots grow, they can not find your limits.

Software 3.0 is a step into a different light, where software finds its own limits.

If we know where they are rooted, we will merge their best attempts. Only because we appreciate their resultant behavior.

If there are lesser-known voices who are as interesting as karpathy or simonw (to mention one other example), I'd love to know who they are so we can get them into circulation on HN.

No it’s incredibly annoying I agree.

The hype hysteria is ridiculous.

If anything I wished the conversation turned away from "vibe-coding" which was essentially coined as a "lol look at this go" thing, but media and corporations somehow picked up as "This is the new workflow all developers are adopting".

LLMs as another tool in your toolbox? Sure, use it where it makes sense, don't try to make them do 100% of everything.

LLMs as a "English to E2E product I'm charging for"? Lets maybe make sure the thing works well as a tool before letting it be responsible for stuff.

> English is a terrible language for deterministic outcomes in complex/complicated systems

I think that you seem to be under the impression that Karpathy somehow alluded to or hinted at that in his talk, which indicates you haven't actually watched the talk, which makes your first point kind of weird.

I feel like one of the stronger points he made, was that you cannot treat the LLMs as something they're explicitly not, so why would anyone expect deterministic outcomes from them?

He's making the case for coding with LLMs, not letting the LLMs go by themselves writing code ("vibe coding"), and understanding how they work before attempting to do so.

> English is a terrible language for deterministic outcomes in complex/complicated systems.

Someone here shared this ancient article by Dijkstra about this exact thing a few weeks ago: https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

AI is all about context window. If you figured out the context problem, you will see that all these "AI is bullshit, it doesn't work and can't produce working code" goes away. Same for everything else.

Who said I wanted my outcomes to be deterministic. Why is it that the only way we accept programming is for completely deterministic outcomes, when the reality is that is an implementation detail.

I am a real user and I am on a general purpose e-commerce site and my ask is "I want a TV that is not that expensive", then by definition the user request is barely deterministic. User requests are normally like this for any application. High level and vague at best. Then developers spend all their time on edge cases, user QA, in the weeds junk that the User does not care about at all. People dont want to click filters and fill out forms for your app. They want it to be easy.

While I agree with you broadly, remember that those that employ you don't have those skills either. They accept that they are ceding control of the details and trust us to make those decisions or ask clarifying questions (LLMs are getting better at those things too). Vibe coders are clients seeking an alternative, not developers.

I am not sure I got your point about English. I thought Karpathy was talking about English being the language of prompts, not output. Outputs can be English but if the goal is to compute using the output, then we need structured output (JSON, snippets of code, etc.), not English.

Time is a funny calculator, measuring how an individual is behind. And in the funny circumstance that an individual is human, they look back on this comment in 3 years and wonder why humans only see themselves.

Like biz logic requirements they need to be fine grained defined

I think you’re straw manning his argument.

He explicitly says that both LLMs and traditional software have very important roles to play.

LLMs though are incredibly useful when encoding the behavior of the system deterministically is impossible. Previously this fell under the umbrella of problems solved with ML. This would take a giant time investment and a highly competent team to pull off.

Now anyone can solve many of these same problems with a single API call. It’s easy to wave this off, but this a total paradigm shift.

You just described Software 4.0...

There is no significant software that has been written (plagiarized) by "AI".

How do you know?

As you haven't evidenced your claim, you could start by providing explicit examples of what is significant.

Even if you are correct, the amount of llm-assisted code is increasing all the time, and we are still only a couple of years in - give it time.

Why not ask the actual experts

Many would regard Karpathy in the expert category I think?

Microsoft is dogfooding Copilot in their dotnet/runtime [1] and dotnet/aspnetcore [2] repositories. This is the only time I have seen a company using its own AI Tools transparently. Yes, they label it an experiment, but I am pretty sure it’s mandated use within Microsoft.

I am an “AI skeptic”, so clearly I am biased here. What I am seeing in the repositories is, that Copilot hasn’t made any substantial contributions so far. The PRs, that went through? They often contain very, very detailed feedback, up to the point line by line replacements have been suggested.

The same engineers, that went up stage at “Microsoft Build 2025” to tell how amazing Copilot is and how it made them a 100x developer? They are not using Copilot in any of their PRs.

You said it’s a religion. I’d say it’s a cult. Whatever it is, outside the distortion bubble, this whole thing looks pretty bad to me.

[1] https://github.com/dotnet/runtime/pulls

[2] https://github.com/dotnet/aspnetcore/pulls

What counts as "significant software"? Only kernels I guess?

This is basically what Lovable is. It's a multi billion dollar company today

I'm not sure this makes sense as a question. Registers are 'controlled' by running code for a given state. An AI can write code that changes registers, as all code does in operation. An AI can't directly 'control registers' in any other way, just as you or I can't.

Because any precise description of what the computer is supposed to do is already code as we know it. AI can fill in the gaps between natural language and programming by guessing and because you don't always care about the "how" only about the "what". The more you care about the "how" you have to become more precise in your language to reduce the guess work of the AI to the point that your input to the AI is already code.

The question is: how much do we really care about the "how", even when we think we care about it? Modern programming language don't do guessing work, but they already abstract away quite a lot of the "how".

I believe that's the original argument in favor of coding in assembler and that it will stay relevant.

Following this argument, what AI is really missing is determinism to a far extend. I can't just save my input I have given to an AI and can be sure that it will produce the exact same output in a year from now on.

All you need is a framebuffer and AI.

Nice try, AI.

He's selling something.

Maybe so, but please don't post unsubstantive comments to Hacker News.

(Thoughtful criticism that we can learn from is welcome, of course. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.)

I'm about half way through the video and I'm really not seeing what all the praise is about, it just seems to be an AI optimism word salad.

edit: a lot of the comments giving praise on YouTube look like bots...

Yeah, not sure I ever saw anything similar on HN before, feels very odd.

I mean the talk is fine and all but that's about it?

It's pastor preaching for the already converted, not new in the area. The only thing new is that they are selling the kool-aid this time.

Replace with "Water Utility going down and the world becoming less sanitary", etc. Still a red flag?

He lives in a GenAI bubble where everyone is self-congratulating about the usage of LLMs.

The reality is that there's not a single critical component anywhere that is built on LLMs. There's absolutely no reliance on models, and ChatGPT being down has absolutely no impact on anything beside teenagers not being able to cheat on their homeworks and LLM wrappers not being able to wrap.

Plus coding is the fun bit, reviewing code is the hard and not fun bit, arguing with an overconfident machine sound like it'll be worse even than that. Thankfully I'm going to retire soon.

Because we are still using code as a proof that needs to be proven. Software 3.0 will not be about reviewing legible code, with its edge-cases and exploits and trying to impersonate hardware.

Hang in there! Your comment makes some really good points about the limits of analogies and the real control corporations have over LLMs.

Plus, your historical corrections were spot on. Sometimes, good criticisms just get lost in the noise online. Don't let it get to you!

> The presenter made certain to identify three interested actors (governments, corporations, "regular people") and how LLM offerings are not controlled by governments. This is a bit disingenuous.

I don't think that's what he said, he was identifying the first customers and uses.

The team adapted quickly, which is a good sign. I believe getting the videos out sooner (as in why-not-immediately) is going to be a priority in the future.

Classic under promise and over deliver.

I'm glad they got it out quickly.

You can't just put things there any time you want - the RFC requires that they go through a registration process.

Having said that, this won't work for llms.txt, since in the next version of the proposal they'll be allowed at any level of the path, not only the root.

https://github.com/AnswerDotAI/llms-txt/issues/2

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

"Don't be snarky."

https://news.ycombinator.com/newsguidelines.html