My experience working with programmers for decades (as an individual contributor, team lead, architect, release engineer, engineering manager) is that great programmers are rare and most programmers are good enough and a few flunk out. Since AI is trained on "the internet" it sees and learns a lot of bad habits, poor coding examples, shortcuts, oversimplified solutions, and therefore, by default, produces code that can be improved. To use AI coding agents effectively, you must be able to spot bad code, suggest better approaches, add guardrails, tests, checks, guidance, etc. to keep the agents on track and improve their output. If you do not deeply understand data structures, algorithms, idioms, best practices, footguns, design patterns, etc., you cannot help the agents provide better code.
> Since AI is trained on "the internet" it sees and learns a lot of bad habits, poor coding examples, shortcuts, oversimplified solutions, and therefore, by default, produces code that can be improved.
This is a huge simplification. AI is also trained on tweets, spam and explicit content, but ChatGPT isn't constantly swearing at you to send your bitcoin to a Nigerian prince.
The mistake people make here is that just because something is represented in the weights, doesn't mean it's representative of its output. And the reason for that is largely the same reason for why if I show you lots of bad code you don't become a worse coder. If anything you'll see bad code and learn to avoid writing similar code yourself.
So long as quality is being reinforced and that process is refined and improved then "model collapse theory" and arguments about how AI will write bad code because it reads bad code will continue to prove incorrect.
I have been using Claude CLI, Codex, Gemini, and OpenCode (Z.ai GLM-4x/5x) for quite a while. I've seen the code they all produce. I've had to add checks as forcing functions to get them to create more loosely-coupled, readable, maintainable, extensible, testable code. They often get stuck, "complaining" that the rules are too hard to satisfy, and then I show them how, make them take different approaches.
I do not mean to anthropomorphize their work, but I do need to improve what they produce. I think a lot of push-back to vibe-coded projects is that the person asking an AI Agent to make something do not or cannot improve what it produces, which goes back to the question asked. Is it worthwhile to learn programming now. I argue that it is.
So, it would be better if models were trained to produce provably correct code, and agent harnesses are improving to help do this, but the models themselves have been shown to be trained to provide the answer they think you want to hear, and declare a task done as fast as possible. They hallucinate (or lie) about running tests, even about writing code at all (sometimes).
How do cloud models "learn" to write better code? Where are the projects that fine-tune an open/local LLM to "learn" from its mistakes. I have yet to see any model improve on its own.
A lot of the best code I've seen is closed source, and unlikely to be represented in existing training data. A lot of the worse code I've see was online. Maybe newer coding models are getting trained better, but most I've tried are years out of date. Maybe my view is affected by trying so many LLMs. Very few have been great.
Your point makes sense. I think there is something about it. On the other side this reminds me that in life, not only code, I learn much more from good things than bad things. I learn more from people, books... that I admire, that do great things. I cannot learn much from the mistakes that others do. They can sometimes even be "noise" that distracts me from what I should focus.
In life I learn a lot from my mistakes, not much from others' mistakes. Could we apply that to AI and code?
Using agents to micromanage agents is something I ended up doing.
Part of the fun is solving puzzles where the coding agent gets stuck and me then giving it technical direction. Especially when an agent says that something cannot be done in software (I show them how it actually can be done).
But what if I use AI to write live demos where you can write machine code, debug it visually in your browser, punch data on virtual punch cards? What if I used AI to generate videos about this, generate the artwork, backgrounds, narration, intro music, etc. I guess I am just a meta-tinkerer :-)
I am also pursuing something similar (complementary?) to this (recently started writing a Rust-based "distributed OS" that manages ML resources in a network of heterogenous systems: varied cores, system RAM, GPU VRAM, I/O). So my focus is not so much distributed agents, but more distributed inference (and fine tuning) that would benefit agents, distributed or not. I may blog about my work soon.
Thanks, softwarewright; would love to hear more about your work. I've been implementing small-model inference in Zug inside WunderOS, but it's not done yet, so nothing public just yet.
I'm in a research/prototyping phase with no truly verified results yet, but I have all the hardware I need to do actual testing. I have an OS that boots in a VM, it just isn't verified to have practical value yet. It could be an AI coding agent's fever dream until I use it anger. So far Apple Silicon only, but most of my systems run Linux and most of my GPUs are NVIDIA, so moving development from Mac to Linux soon. It does have tests, demos, docs. Iterating on it daily.
I should clarify that by Apple Silicon I mean it boots Rust no_std on ARM. Does not use GPU yet. Plan is to use Rust without CUDA libraries. This project is more likely to use an NPU on a ARM dev board before it can use an NVIDIA GPU, and might never be able to use Apple GPUs. Goal: run on no-longer-supported by CUDA GPUs.
Good stuff here, much of which resonates with me. FWIW, my notion of an OS for agents isn't as close to the metal yet, ie, my view of agent lifecycle is that they look a lot like WhatsApp or Discord traffic. So, based on that DEEP analysis, I decided to build on BEAM/OTP for the control plane: basically, Elixir for the UX and Gleam for everything else. And the data plane is pure Zig: data sidecar into BEAM/OTP via NIF.
So that gives me a certain freedom for deploy: bare metal, containers, VMs, even K8S. And since there are some tools for pickling all that into a single binary, and running BEAM/OTP on u-kernel sorts of things, I can get all the way to the metal in the way that you are. Whether or when that happens remains to be seen, etc.
Interesting link/content and it seems complimentary.
One thing that is missing from both of our approaches is the ability re-train (fine-tune) coding models "overnight" so that they can "learn" from the prior day and changes since their training cutoff date.
I have found some things I can do to improve my work based on this, thanks.
_Pentad idea_ -/- _MLOS relevance_ -/- _Action_
Closed autonomic loops -/- Very high -/- Adopt architecture vocabulary
Deterministic replay -/- Very high -/- Strengthen event/replay contract
Model minimalism -/- Very high. -/- Extend later to compute-placement ladder
Durable vs active population -/- High -/- Define registered vs resident capacity metrics
Standing queries -/- High -/- Future policy/watch abstraction
Provenance by construction -/- High -/- Record policy decision causality
No model/NLP in hot path -/- High -/- State explicitly as invariant
One thing that falls out of Model Minimalism is adding native WunderOS model hosting, which I've been working on this week, natively in Zig, NIF'd into BEAM/OTP.
IMO vertical integration in AI infra is underrated; by adding model serving I can exploit a range of optimizations that 'best of breed'/glue code architecture makes harder. This week's example: native semantic entropy implementation -- following Spanda -- in Zig, such that hallucination detection at K=5 (batch size) is 650us per turn, i.e., in the noise.
I've spec'd how to do QLoRA, too, but it's unclear when or if I'll implement it, not least because it's not clear that I should bother given the training data integration issues.
I'm glad someone else is thinking about this stuff!
I was a hobby programmer in the 80s, but that was after college. I was a hobby programmer in the 70s via my high school's library having a terminal to access a mainframe for self-study programming courses, and I built mail-order computer kits which I programmed in machine language.
I think AI is a great tool, but requires safeguards. I use it every day, but I use a lot of tools to reign it in, keep it on task, check its results, and force it to follow an engineering process.
I don't know about the "as quickly as possible" part, since there needs to be guardrails, but improving AI should be done sooner rather than later. e.g, instead of training to give the answer it thinks you want to hear, it should be trained to give provably correct answers. instead of being trained and fine tuned and then let loose, it should be able to continuously learn and improve.
While people keep chasing tokens/second and vibe-coding, I'm more interested in correct results per kWh, sustainable and ethical AI. Also, I'm worried that there will be an increasing digital divide between the wealthy and the rest of us, so I do want AI to be available and not locked up just for corporations and governments.
To me, the fun is solving problems, and AI let's me solve more problems, harder problems and more quickly. I was happy to move from assembler programming to high level programming languages, to 4GL, to frameworks, tech stacks, etc. Moving to AI coding is just another shift in solving problems at a higher level with more productivity.
hacking microprocessors was fun. building complex systems using AI is fun, too.
I am working on developing a distributed operating system for managing machine learning resources (the entire memory hierarchy, networking and compute hierarchy). I worked in O/S development early in my career, now retired and ML is a hobby I've invested in over the past few years.
I'm curious about:
- how to "upcycle" end-of-life NVIDIA GPUs, like K80, M40, P100, etc.
- how to offload MoE expert calculations to MCUs with NPUs, possibly FPGAs
- how to manage a 10G LAN-attached network of older Xeon servers with the above GPUs and USB-attached MCUs as a machine learning "fabric"
- do ideas from IBM's ACP/TPF O/S (now Z/TPF) carry over?
- do ideas from IBM's Workload Manager (business priorities managed resources) carry over?
- can I bypass end-of-life CUDA driver support for older NVIDIA hardware by using Rust/Vulcan?
- can I leverage the architecture of Freetoken (MoE caching/routing, Engram, KV-cache compression) and other approaches like Baby Dragon Hatchlings, Hierarchical Reasoning Models, Tiny Recursive Models, Recurisve Language Models, Multi-token predicion, etc?
- is anyone else intersted in actually building something like this?
I've explored many of these parts individually, now combining them...
me too; and my coding agents are slowed down (from developing features) because I require them to refactor the code to be more readable; my code metrics tests force AIs to leverage functional programming and design patterns).
Yes agents can produce code that compiles and runs, but I had to add tools to keep them on track, document their work, follow a process, check their outputs. I also use other AIs to generate developer documentation and review code.
It is like managing a bunch of idiot savant eager-to-please interns, except unlike interns, coding agents do not (yet) learn and improve on their own.
I have the same "1/30th ... simultaneous" experience with coding agents, and I am having fun. Directing teams of agents to solve many problems (or create many useful things) for me has its own fun challenges/puzzles-to-solve.
Some of the fun is redirecting coding agents when they get stuck, by showing them code or design they could not work out on their own. Some of the fun is designing tools to rein-in AI coding agents, keep them on track, and verify their outputs. Some of the fun is exploring new areas for me: compilers, game dev, language design, emulators, embedded, etc.
So I am having fun at the CTO/PM/Engineering Manager level but also at the lowest coding level.
I've been coding since the early 70s. This brings back the joy I had discovering microprocessors and electronics.
Sure, if that is what you like. I for one did not get into this field because I wanted to hire people (artificial or natural) to write the programs for me. Programming as an activity / art is what I enjoyed the most. Managing people / agents is a total turn off.
serious question: do you program in machine code or assembler or use a higher level language? do you use design patterns and functional approaches? metaprogramming, code generation, templating, app frameworks?
Each tool, abstraction, language feature elevates coding to get more done with less (more concisely). Programming is a social activity, since source code is meant to be read by humans (at least historically) even if it is just yourself later.
I'm not trying to talk you into using AI coding agents, but explain why I find them fun, and I've written a lot of code by hand over decades, from spaghetti assembler code to structureed programming to object oriented to functional, etc. Each advance in the industry has been a new and diiferent kind of fun for me.
Are you saying that AI coding agents cannot produce working code? Or are you saying that it produces something that cannot be explained or understood?
If I used a macro to generate code that gets compiled and linked to code someone else provided, how is that different from asking a coding agent to generate similar code and linking to other coding agent code?
Is your concern that the output is probabilistic vs. a macro or code generator producing code deterministically? I apply engineering practices to ensure the results are suitable.
I do understand that AI is fundamentally different (and I've been studying ML in depth for more than a year). My comparison was in terms of tools increasing productivity, not how they do that.
I was a "10x programmer" on several projects and I am 10x more productive now than I was then (at least). I can create in a few days by myself (with agents) what used to take months (with a team of people).
Maybe I'm producing what some consider "AI slop" but the things I've built this year work to my satisfaction, are fun to make, and I am constantly learning new things. And I am doing this for fun but also to learn by teaching, so I build educational, visual things to share with others.
This is interesting. It would be even better if ones coding agent also got better at its coding abilities every night. Otherwise, agents might re-introduce the same class of bugs that your nightly improvements keep reacting to.
This would require logging agent actions during the day, analyzing it at night, identifying things the coding agent could do better, and update its AGENTS.md memory (relatively easy) or fine-tune the agent (if local LLM, and the fine-tuning cost was acceptable).
It would be nice that when a person corrects an agent's mistake this was captured and fed back into a nightly agent improving process.
reply