I spent a lot of time wandering around the border zones between countries on Street View. It's fascinating, the lengths we go to to keep the "other humans" out of "our country", and yet... Often even the most militarized borders are illusions, with huge swathes of wilderness making border walls or fences impossible.
My Mac Studio arrived yesterday and one of the first things I did was cancel my Claude subscription. Happy to be free of load bearing, price gouging, paternalistic "altruists" Anthropic.
Am keeping my Codex sub while I find the best local model, but my plan is to eventually stop with OpenAI too.
There's a continuum between "vibe coded by someone with no technical knowledge or inclination" and "hand written domain driven design development". You can absolutely use coding agents AND have maintainable code. But yes, the coding agents will not magically make everything maintainable if you don't tell them to.
"Code maintainability and good architecture don’t have good measurements that we can apply"
Who has no wisdom? There are dozens of ways to measure code maintainability. Cyclomatic complexity is just one.
Nothing stops you from wiring up something like SonarQube metrics to your agentic coding workflow.
Plenty of measurements exist, but the trick, as the article said, is finding good measurements.
Cyclomatic complexity has been pretty solidly discredited within the maintainability research community for decades.
Sonar's cognitive complexity metric is a bit better, but here's a study that found that it still only has about a 0.5 correlation with how much difficulty programmers actually had reading code as measured by multiple methods.
They found that the most accurate way to measure code complexity that didn't involve something like an eye tracker or EEG is still basically just vibes - asking programmers if they thought it was hard to understand.
Halstead Effort came in second, and scored pretty well, but here's another one where it doesn't do so well, either. And it scores the SonarQube metrics even worse, with only a 0.35 correlation: https://www.sciencedirect.com/science/article/abs/pii/S01641...
Yeah, we have this rule in our SQ and I absolutely hate it because of the number of times I'm forced to break something up. Complexity that matters is usually at a higher, more architectural level.
Yeah. Looking at the code samples in that first page was pretty interesting. They're cherry-picked, of course, but still it's a little galling to see which ones SQ gave a lower score.
I definitely wouldn't want AI to be autonomously using that as a guide without doing some fairly serious internal A/B testing first. Kind of like for cyclomatic complexity, it's just too easy to find ways to maliciously comply. And if that's what you ask AI to do then that's likely what you're going to get.
> yes, the coding agents will not magically make everything maintainable if you don't tell them to.
There's the rub. It requires knowing about and caring about maintainability. And a lot of the people who "haven't written a line of code since 2025" don't care
Humans aren't magical, so if people can figure out what maintainable code is and how to write it, so can a non-human. Furthermore, humans are rather unlikely to be the optimal form of intelligence for software architecture and engineering, so it's almost certainly the case that if humans can figure it out, some non-human could do it better. That it's not yet solved by models suggests that it's not easy and maybe even not practically achievable with current tools. But that's not the same thing as not solvable.
> Furthermore, humans are rather unlikely to be the optimal form of intelligence for software architecture and engineering
I like this framing. Humans invented software (and engineering in general) as a means to solve problems with methods that work best for us. There may be entirely different, and parallel, problem-solving methodologies outside of human best-practices.
I have not found it difficult to get these systems to write maintainable code. You're probably correct that most people don't care about this so they don't bother to do so. But IMO, this doesn't seem like a problem that is crying out for a solution.
I agree with the larger point, but here I'll disagree:
> There are dozens of ways to measure code maintainability.
There are no good ways. I'm averse to making absolute statements, but here I'll take that chance. I worked in dev producitivy for years with people who spent decades in that domain across multiple companies with very high volumes of code production. Everybody agreed: All metrics are flawed and even a combination of metrics is insufficient.
Just to give one fundamental reason (in addition to a lot of the sibling comments): for any given metric there are an infinite set of counter-examples that don't trigger any thresholds but are clearly bad code. So these metrics typically only help in trivial cases, don't catch a majority of the cases, and so often become more of an annoyance due to low SNR. A lot of dev productivity work ends up being wiring these metrics in and then providing escape hatches when they inevitably get too noisy!
And most relevant to this discussion: these tools do not say anything about higher-level concerns like architecture, over-engineering and design, which IME is where agents tend to mess up most. I've almost never had a complaint about the code itself; the logic, naming, functions, data structures, even a lot of the testing, are all on point. It's always been the higher-level structure and design: over-engineering, duplicate classes, suboptimal abstractions, redundant operations across layers that could be solved by adding a single variable in a class, etc. etc.
I think the problem, like with code written by humans, is lack of sufficient context while doing a task leading to tunnel-vision. This is why we need to oversee and ensure things are good holistically. I suspect models are now good enough to play the role of an architect as well, though, and I've read some indications of that online... I just haven't tried giving them that much control yet.
> But yes, the coding agents will not magically make everything maintainable if you don't tell them to.
You say that like adding "Make it maintainable." to your prompts solves the problem. But the reality is we only have weak metrics for measuring maintainability. For example, you can trivially optimize for Cyclomatic complexity by blowing away abstractions and duplicating code everywhere. That doesn't make the code better. Cyclomatic complexity is a tool that has to be applied judiciously.
That doesn't mean you can't or shouldn't use AI to generate code. But it does mean if you want your project to scale, you're still going to need a lot of developer involvement at the code level to ensure the code remains maintainable so that future developers can build on top of it. AI is not like compilers, which allow developers to build complex solutions without being proficient at the next level down (assembly).
> There's a continuum between "vibe coded by someone with no technical knowledge or inclination" and "hand written domain driven design development". You can absolutely use coding agents AND have maintainable code. But yes, the coding agents will not magically make everything maintainable if you don't tell them to.
Yes. But I think the idea is without "hand written domain driven design development" the result trends to "vibe coded by someone with no technical knowledge or inclination," as developers de-skill.
- lot of (most?) people can do basic multiplication (I’m too lazy to fetch any stats but I hope you’ll have some observations in your bubble dear reader)
So are you agreeing with me or not? Because it's not black and white. Having a few non-deskilled developers around who do the equivalent of "compete in mental calculations" is the same as having none at all.
It's sort of like the retort "AI won't take all the jobs from humans, some will be left [at the very top and very bottom]." Even if true, fat lot of good it does most people who would be unemployed in that scenario.
Also, if you get widespread deskilling, but massive increases in code production due to AI, you're probably still going to get "organizations falling into the trap" like the OP describes and the remaining skilled people getting burned out trying to hold it all together. Modern American business culture (in aggregate) is incapable of learning to not burn people out until everyone is already burned out, only then will it pay attention to the problem (and then probably forget what they learned and start repeat it in 10 years).
> So are you agreeing with me or not?
I’m disagreeing with black or white view.
You make it
A bit black or white.
Llm will not take jobs, at worst it will shift jobs.
If we will allow current llm to develop code without inspecting it, we will still want to provide direction and expect certain quality level. So in “worst” case you will need at least design/QA people.
If code will be free, and it will increase 1000x, you probably will need 100x design/QA people. (Unit tests written by same llm, if not reviewed are worthless as assurance). I’m absolutely certain that humans will have plenty of software related jobs.
As to programmer deskilling, another parallel: lot of programmers have very little knowledge of how to deal with databases correctly/efficiently, because they are somewhat lazy and good tooling/orms allow to get away with it, but there are also plenty of devs who do know how db works and how to design a db schema for their app access as it improves overall system performance and correctness. Same with LLM case, having knowledge of what happens in code, gives an edge, and competitive/curious/overly-responsible people will continue reading things in depth (llms also make learning somewhat easier). And its definitely not 1%.
Furthermore, llm prices are subsidised a lot now, so it’s yet to be seen if llms are economically viable as vibe coding agents on a large scale.
Our class sizes at the university that I teach at have plummeted with the crash. What I'm hoping is that this means I'm just going to be teaching people who are actually interested in producing quality code.
//There's a continuum between "vibe coded by someone with no technical knowledge or inclination" and "hand written domain driven design development".
Somewhere in the middle of that continuum sits - "domain driven specifications, described using high level english concepts(that are well defined) from the domain , combined with a selection of a few standard architectures"
I agree that "maintainability" is the wrong place to stake one's flag. It's a technical problem and will probably be overcome.
The "lack of wisdom" in AI coded projects which does not strike me as a soluble technical problem concerns questions like: which software is worth writing and which isn't? How are tradeoffs evaluated?
I don't think AIs are structurally capable of answering these questions in the same way humans are. But they make those decisions anyway!
Those metrics don't amount to wisdom. Wisdom is something like realizing that you don't have to special-case some code over <here>, you just need to create a new reusable primitive over <there> that fixes two bugs with one stone, and you'd probably wind up needing it anyway. Both bits of code could sail through all the measurements of code maintainability you could come up with, and an agent suggest to me the brittle solution, but I came up with the more maintainable one.
The author admits to a measurement: maintainability. There are quantitative and qualitative ways to measure this, the biggest signal being outright abandonment
That's terrible as a signal. It's too late for anything, and there's many reasons things get abandoned.
> code maintainability and good architecture don’t have good measurements that we can apply, because it takes months, years even, to notice the effects of bad architecture or of unmaintainable code.
One way of understanding "good" here is "actionable", imho.
> But yes, the coding agents will not magically make everything maintainable if you don't tell them to.
But that's not actually a change. Humans wouldn't magically make everything maintainable if you don't tell them to (and maybe not even if you do). You have to monitor them and train them, carefully, basically forever.
Maybe not forever. Humans you have to do it basically forever, because humans don't improve very fast. AI... you have to do it forever with the current level of tech. If the tech improves enough, who knows?
> You can absolutely use coding agents AND have maintainable code. But yes, the coding agents will not magically make everything maintainable if you don't tell them to.
The trillion dollar question is how you do this, if your employees do not care (they are optimising for salary & time spent not code quality) and you have no way of telling apart AI slop vs. good maintainable code. (If you could you would just train the AI.)
Before AI there was at least some way to tell apart good programmers from bad, because there was some human effort involved in coding. Now with AI and slop generation there is almost now way to do this.
> if your employees do not care (they are optimising for salary & time spent not code quality)...
From what I see is it's mainly managers and higher brass who doesn't care about code quality and sustainability, and aims to drive time to market metrics down aggressively with AI.
Any employee who cares about code quality will become a poor performer with a red luddite label because they dare to change what the AI has emitted for them.
I'd love to be wrong, very wrong about this, actually.
They don't care about maintainability because they're not going to be debugging it at 1am on Friday night but mainly because if something is wrong they have lots of people below them to blame for it.
Developers, however, are still responsible for the code! We must review the AI....all 80k lines of code it generated yesterday. If we don't then we are at fault. And we must go full throttle of course. So ....not be picky and retrograde about accepting what is generated.....
IOW we know who is going to get screwed and it isn't them.
Yeah but I mean, what is their compensation based on? I'm genuinely pretty curious about the point you're making here. I don't know anything about underwriting or how that profession works or how it is analogous to what you see as the future for software developers. But it seems interesting and possibly insightful.
> From what I see is it's mainly managers and higher brass who doesn't care about code quality and sustainability, and aims to drive time to market metrics down aggressively with AI.
Trends over time will drive more observable changes. If a whole generation of programmers picks up bad habits that their managers don't care about (think very junior), that will take some time to play out. It's like children's literacy. You don't notice overnight, but a decade of neglect and you have a reading problem in kids.
Yes, this is why it's good that we're having these debates. We should keep having them. Personally, I am not yet convinced that this is a real problem. But I also haven't yet worked on a team with people who have entered the field in the last few years.
> and aims to drive time to market metrics down aggressively with AI.
Go fast and break things has been a mantra for how long?
I think a lot of the laments about "good" code are really about "ownership" - and as someone who spent most of my working career in OTHER peoples code bases I have seen some things. There are a lot of you who think that your code bases are "great" when they are NOT. Personal understanding is not a good measure of quality.
The increased cadence from AI is just speed running to the legacy code base.
The answer: express the concern, and reiterate it after every issue that arises because of increased complexity. Start building a plan on how to "unravel" the mess, how to migrate things in place, how to start drawing boundaries in your systems. The system is designed to reward heroes who fix problems - you want to be super man who stops the bridge from falling apart, not the engineer who pushes the costly fixes it before it does.
> Go fast and break things has been a mantra for how long?
Practically since eternity, but just as we learnt to manage current rate of "fast" and "breakage", somebody attached a solid booster behind us. So we're trying to understand what happened and what's happening and what will happen.
> I think a lot of the laments about "good" code are really about "ownership"
People owning what they did, have responsibility and initiative about doing better is always a good thing, yes.
> and as someone who spent most of my working career in OTHER peoples code bases I have seen some things.
I can understand that, I'm sorry you had to go through this.
> There are a lot of you who think that your code bases are "great" when they are NOT. Personal understanding is not a good measure of quality.
My codebases are as great as my knowledge. I love when someone reads my code and points where I f'ed up. I also love to show what I did has achieved something I was aiming for and discuss how to achieve it betterer.
> The system is designed to reward heroes who fix problems...
And this is the problem. Because I work silently and diligently build something looking unimpressive while working like an atomic clock without any problems.
I often have to ask myself if I’m asking for something different because of preference or need. I don’t really know what others are doing but I see this comment a lot about needing to always correct agents. I can’t figure out if it’s an exaggeration or not because once I’ve planned how I want something done I pretty much have zero need to intervene.
Impossible to know without doing a detailed comparison. Are you using the same LLMs? Are your criteria for correcting the output the same? Are you working on similar code? Are your plans and prompts the same?
It's fine to have preferences. The biggest problem with bikeshedding is the time spent (wasted) debating. But you don't have to debate the AI, you just tell it your preferences, and ideally encode them so that they are repeatable.
Of course it's fine to have preferences. I think you missed the nuance. Bikeshedding implies the time is wasted because the topic wasn't important in the first place. The color of the bike shed, as it were, has nothing to do with the storing of the bikes.
I am equating the nerve that develops in people that get lost bikeshedding (wasting time on inconsequential parts of the problem) with fighting an llm on inconsequential implementation details.
We most certainly agree: what matters should always be the actual requirements (functional, security, performance, etc.) You can't bikeshed an important topic. Everything else is implementers decision. In my experience an experienced engineer understands the difference and trusts implementers to make the decisions that they do own.
No I understand the nuance. But it's only bad to have preferences about things that barely matter if you waste time on them. You don't have to fight the llm, you just tell it what you prefer and it does that, that's what's great about them. (If it is not listening to your directives, then you have other problems.)
You use the intentionally vague word "implementers" to abstract whether you're delegating to a human or to an AI. But the key point is that these are not the same thing. If I'm delegating to a person, that person is the "implementer". If I'm using an AI to generate an implementation, I am still the "implementer", it is merely a computer program working on my behalf.
Directing an AI's work is not "fighting it". You keep characterizing it that way, but it's the wrong characterization. "Implementer" is not blurry at all. Humans are responsible for the things they implement, whether or not they use AI tooling to do that implementation.
Is that my characterization? This subthread is about how it’s so time consuming fixing every little thing the AI does to be just like you would have done. My challenge to that sentiment is “give it some freedom, don’t micromanage it.” That’s all.
I understand the boundaries of ownership and responsibility. That’s why I can tell you if you are spending inordinate amounts of time correcting AI code then you’re doing something wrong. Either write the code yourself or reassess your ownership boundaries. You’re acting as a manager of a team of agents in an agentic coding paradigm. Managers don’t tell me how to write code.
Without going and re reading the whole thread, I'm pretty sure that both of your comments that I replied to included this "fighting the AI" characterization. It's certainly fair that you didn't start the thread about it but were just taking the premise of the sub thread. But I just disagree with that whole premise. I think what's nice about these tools is that I can just write a document that says things like "prefer immutability" and then I neither need to micromanage nor accept code I don't like, and there is no long slack thread about whether I'm right about any of the things I've written into those rules, the AIs are happy to do as I've asked.
I think this entire analogy about being a manager of a team of agents that is in vogue is completely misguided. Have I always been the manager of a team of bash scripts? No. These tools are way more capable, but they are still just tools that I'm using to do my own work, they are not people that I'm delegating responsibility to.
> I think what's nice about these tools is that I can just write a document that says things like "prefer immutability" and then I neither need to micromanage nor accept code I don't like
Why are we even arguing then? You and I agree. Did I ever say "don't share a single preference with the AI"? You set the guardrails and preferences and the AI follows them. This is how it's always worked so it's reasonable for me to assume that people "fighting the AI" have already done this and are being overly pedantic about the output. Otherwise it wouldn't be eating up inordinate amounts of time...
> Have I always been the manager of a team of bash scripts? No.
No. You're not even remotely close here. Let's revisit this once you've figure out how to have a team of bash scripts implement 100k lines of code and build entire systems in 2 weeks based on high level instructions and requirements shared in context and prompts. You have responsibility at a different level and scale in an AI native workflow.
I don't know why we're arguing about the first thing :)
But I think we have a real disagreement about the second thing. You don't "have responsibility at a different level and scale in an AI native workflow", you're still using tools. I understand that what you're saying is that it's such a difference in scale that it is a difference in kind. But I don't agree. I'm fundamentally at odds with this entire framing of ai agents as a team that is being managed. I don't like any of the anthropomorphizing of ai tools. It's fine if it's just an analogy, but people take it way too seriously as a real thing IMO. I fully recognize that I'm out of step with the prevailing discourse on this, but it's a genuine disagreement, I'm not confused about what other people think.
"Hey look I got this brand new tool it can do everything my old tools did and more but I'm scared to use it to do more--might shoot myself in the foot."
I think most people have felt that way before. Only way forward is to practice with the new tool (=
> Any employee who cares about code quality will become a poor performer with a red luddite label because they dare to change what the AI has emitted for them.
I think that's true but it's a special case. AI is here to stay and with AI coding IS faster and quality is better than ever before. Ideally you want your luddite fired along with the slop generators and keep the ones who are using AI and taking their time to deliver a maintainable code.
"Luddites" can just be the ones pointing out the Emperor's new clothes have a big hole in them. Removing them is just ensuring that whatever mistakes you're making get reinforced.
Chances are high the luddite is the one with the deepest understanding about the product and the code base - the one who is actually passionate about the project and actually cares.
Maybe? This does not seem to be in evidence to me. In my experience thus far, this seems to be way more of a personality and ideology split than a depth of understanding split. On my teams thus far, I've seen the deepest experts land on opposite sides of this question.
That could be the case. But if other companies figure out a way to deliver good code with AI, then the company with the luddites is going bankrupt. Luddites are better than slop generators, but both are worse against a developer with AI who actually cares and has a deep understanding of the code base and programming.
Well, and that's how you get enshittification everywhere. Everyone just cares about time to market, consumers are left to drown among a sea of slop, and honest businesses go down because they cannot stand a chance against slop peddlers flush with VC money.
Enshittification happens after the company was quick to deliver a working solution. They successfully captured the market, killed off the competition and now they are ready to start milking the customers who have no other choice.
AI slop can actually help with this, because it reduces the cost of replacing their enshittificated software.
The reason its hard to dethrone Facebook, Instagram, YouTube, Twitch, etc. Is not because it's hard to recreate the software (it may or may not be) it's because of network effects, budgets, etc.
And unless you have ownership level stake in the company, you *shouldnt* care about what happens downstream of selling labor to the company. Yes, I highly recommend 8 and skate.
Ive seen soooo many people burnt out, or "ive given years to the company and i got hit with layoffs", or "$200 software would have saved $1000000 when I brought it up to them". And companies will throw you away the MOMENT your usefulness is gone, even if just perceived. So, use them just as much as they use you.
And that idea of slacker is ALSO a way to generate more money for you, by slyly withholding or slowing work. I didnt get my paltry 3% last year. Inflation up 15% or whatever stupid number. But I can control how much work I do, so my effective wage/hour stays with inflation.
Save your caring for your personal projects, nonprofits you help at, your and family/friends labor you help with.
With some. You insisted, so let's get into the details.
1. Caring about the company when you are a worker and not owner?
Yes and no. I don't care about the company. I do care about what I do. It's a self-respect matter. I do good work not because I'm a slave to company, but because of self respect. My deal is simple: "I'll do my best to produce the best artifact and push the company further as long as it doesn't conflict with my personal principles, you'll buy that time for that amount of money".
I have a simple, foundational rule: I'll sleep sound at night, and this rule is rooted in my ethics. So, I don't shortchange anyone, incl. my employer. If terms change between us, we will discuss, but this probability is not a reason to do shitty work (or optimize for money, or which sugarcoated absurdity others name this).
2. Companies will throw away/layoff people with no notice?
Yes, this is bad. This is life. It's not nice, fair or acceptable, but without unionization, you can't act against this. So, you either try to change this or you just accept it. Realities of work life is not a predicament to shortchange your employer again.
This is as absurd as saying "I'll die anyway, why do all these things? I can just die on-demand".
Meaningless...
3. Work slowage (work-to-rule) as a counter to low/no pay raises in accordance to general inflation
We can accept that, but you all shall really unionize. It's not scary. Try organizing. It's a force multiplier.
4. Invest emotional and physical labor in ventures you gain completely out of
Everybody should have hobbies either productive or unproductive. I can't find the question.
There are companies where you can spend years doing as you describe. More and more, though, you’re competing with people who care even though they don’t own, put in the same effort YoY, and invest in their job. Companies love these employees.
So I guess it really depends on your values and what you want out of life. If you enjoy hobbies and time outside of work, sure find a job where you can coast. Don’t get frustrated when you get laid off just find another place to work. Etc.
Plenty of people want to grow within the industry and build a career, though. And many have what we call basic self-respect and care about how they are perceived.
> ... with AI coding IS faster and quality is better than ever before.
Is this claim based on something?
I'm not against or "for" AI (whatever that means), I try to use it as effectively I can, but for me it's not at all obvious that quality is better than ever before.
Speed I can buy, especially in new projects and utilities, but quality? At least I haven't seen this in practice, if anything I'm just seeing more code, issues, PR's and pressure ==> more slop, more bugs, less quality.
You can always say "skill issue" and "process issue", but that's partly my point here, AI doesn't magically solve this.
Presumably it is based on that person's personal experiences, like your own comment and all the other comments here?
Speaking for myself, based on my own personal experiences, quality is by far the bigger advantage of these tools. It has never ever been easier to write automated tests and to automate tedious manual validation. I'm running my code through like 10x more paces than I ever did before, because I can just say "hey try running this in these twenty different ways" (including with browser automation, if that's relevant), without needing to either do the tedious steps to run all that or to take the time to write a script to do it, and to compile and attach the findings to the PR. This saves me hours to days of work on validation, but the reality is that I just wouldn't have spent that time in the past, I just stopped at a lower bar for quality, because I couldn't justify the ROI for spending all that time on it. But now the ROI is huge, so it's a no brainer.
If people are not taking advantage of this, then yes, that is literally a skill issue.
For some reason this reminds me A LOT of past discussions about microservices, most wonderful on paper and forever debated, but I've never seen it work out perfectly in practice, for me it's mostly been a cluster F in most companies that adopted them.
Currently I see AI similarly, in theory perfect, in practice I don't see the claimed effects. So yes, skill issue, but skills are relevant and your company probably can't hire a rockstar team (if that matters in the future).
But your comment on personal experiences was very good! Spot on, we are all biased, easy to forget. Thank you for that.
> If people are not taking advantage of this, then yes, that is literally a skill issue.
...or domain. You said "including with browser automation", so you do web or web-adjacent development.
Not all of us are doing that. What I work on doesn't have any UI or output besides a log file most of the time, but it connects to many places and does many things like an octopus, but nobody sees that, but feels that it's there because their environment keeps on working.
I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.
The octopus you just described sounds to me like an excellent example of what having the ability to more easily do tedious validation is most useful for. If you know that the "environment keeps on working", there must be some way for you to observe that fact. And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes. I find it so much easier to do this exact kind of thing now. Or, "easier" really isn't the right word. It's that the activation energy is low enough now that I'm able to do a lot of things up front that I used to rely on runtime monitoring to validate.
I guess YMMV, and it's not magic, but for me it totally changes the calculation on when it makes sense to automate something (like that chart from the old xkcd about how many times you'll do the thing and how long it takes to automate) in a way that means I'm doing a bunch of things that are useful for quality that just would never have passed the bar in the past.
> I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.
I didn't. I made a guess. I might be wrong, that's OK. I love to be wrong, because I learn things by being wrong. Also no offense was intended, and I don't consider webdev inferior anything. What I tried to mean is, if AI has more training data for a domain, it does better. If you fire the same model on a niche domain, it falls flat.
> If you know that the "environment keeps on working", there must be some way for you to observe that fact.
Yes.
> And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes.
Nope. On the contrary, because there's so much innate knowledge that is required to know what to do, simulating in mind, deploying and testing on real world is much easier and faster than letting loose an ML model on it. You need real data, real data comes in slow, but you can catch problems early and easily.
Considering it's a niche area, AI also doesn't have much training on that domain, so it's doubly inapplicable for what we do.
> but for me it totally changes the calculation on when it makes sense to automate something ... (snipped for brevity)
It's great that if it works for you, but YMMV part is way more correct than people want to accept and want to learn. AI is a pneumatic hammer, but not everything is a nail which can be driven in with that.
When it works, it works. When it doesn't, well people still pretend it does or insists it shall. We must accept the limitations.
> deploying and testing on real world is much easier and faster than letting loose an ML model on it.
No this is what you're not getting. It is "doing the things I would do to deploy and test in the real world, but faster and in the background while I do other things", it is not "letting loose an ML model on it". This is the new capability. If you have any process like "do {action}, wait until {something}, check {something}, determine if it matches expectation", it is now possible to run that loop way more times in way more variants without either spending the time on it synchronously oneself or writing a script to do it. (If you do that specific action loop often enough, it's probably worth writing the script anyway, but that's also much quicker to do now.)
The AI doesn't need training on the domain, it just needs to be told "these are the things I would do, please do them for me and report back".
I'm sympathetic to not everything being nail-like, but I really think you're leaving a lot of chips on the table if you can't imagine any of this kind of action-check-evaluate loop you have that you could offload.
I'm saying you could do more things. Instead of waiting until a new thing is merged so that it can be deployed and waiting for monitoring to catch issues, you could be deploying pre-merge to testbeds, using different variants of the code or different configurations of the whole system or different inputs to exercise edge and corner cases. Maybe it's too complex of a system to set up that kind of testbed or simulation? Well, it's easier to do that now too! I can spin up and down environments, either with containers on my workstation, or in cloud deployments, that I would have not attempted in the past, because it would have taken me too long to set them up. But now it doesn't take that long, and I find it super valuable to be able to try more things out. (Cost is still a real constraint, I'm not saying that constraints magically no longer exist.)
Obviously I have no idea what your work looks like! But what I'm saying is that time savings are not just time savings. There can be a point at which the time savings bring you under an "activation energy" such that it unlocks a new capability, not just a speedup. And some of those unlocked capabilities can be directed toward improving the quality of software. And I think that's awesome and useful, is my prevailing point here. I won't claim that it will usher in an industry wide improvement in quality or anything, but for me personally, I'm making better software more quickly now, and I'm very pleased that I can do that.
I fully agree with this point about quality. I am doing so much more testing than I used to, because I have so much more time to do it, and it's so much easier to automate the more tedious kinds of validation.
Maybe it's true that lots of people aren't taking advantage of this and are shipping trash, but that's their own problem, and there have always been people who do the job poorly.
That's true, but it doesn't have to stay in this form.
> with AI coding IS faster and quality is better than ever before.
Citation needed, because the last study I read about was painting a completely different picture about code quality. Also, just because the AI pulling and remixing code from a known repository with high quality doesn't mean your code will be at the same quality automatically. Passing tests is not enough.
> Ideally you want your luddite fired along with the slop generators.
The thing is it's not possible to see who generates slop and who generates code, and if you fire the only people who knows about the codebase intimately, you'll be on a very exciting, possibly fatal ride. I don't recommend this. AI doesn't know your history and trade-offs. These guys do, and can guide you to clear.
AI can't.
Believing that AI will create bug-free code from start is believing that Rust is the silver bullet.
The reason that we can (not that everyone, or even most people will) create higher quality software now is that it is now much easier to try out a bunch of edge and corner cases, which would have previously often required a prohibitive amount of time to set up and run. It's a dream! I can certainly believe that people aren't taking advantage of this, but they should be!
But I totally agree with you about the measurement problem. I think it's a very difficult time to be a hiring and firing manager.
> The thing is it's not possible to see who generates slop and who generates code, and if you fire the only people who knows about the codebase intimately, you'll be on a very exciting, possibly fatal ride. I don't recommend this.
This is exactly what I said in my top comment. This is a huge problem.
"Just do everything to hold things together, and let me abuse you without complaint" is exactly the kind of patronizing sentiment that infests the tech adjacent scene these days.
The answer to that is "Fuck you, no. I have worth, and you will respect it". Gilded Age paternalisms are not something that needs to be brought back into vogue unchallenged, especially when the intent is to keep the rabble quiet, and the checks rolling in and up.
No, the luddite is the person in one bad location on the continuum, opposite the AI psychotic on the other bad end of the spectrum. The person holding everything together is the one applying wisdom to the use of tools for the things they are good at while avoiding the things they are bad at, rather than blindly following one ideology or its opposite.
Sorry, but you're distorting my framing of the term in my original comment.
I labeled the person who uses AI to generate code and uses their brain and wisdom about the system to refine that code as the luddite since they will work slower when compared to other "higher performers" who don't care about the code quality.
In my framing I'm aware that the person is not a luddite per-se, but will look like it since they will be slower while trying to create better code, albeit using AI in the process as well.
Citing myself:
> Any employee who cares about code quality will become a poor performer with a red luddite label because they dare to change what the AI has emitted for them.
One way to get workers to care about the quality of the product they are being paid to build (for someone else's profit) is for the management to show some loyalty and build trust over the course of a decades-long career by treating them fairly and avoiding layoffs.
Sounds like a culture problem. I’ve never had a team I lead not give a shit. Even if the project is dumb (some projects are dumb), there’s enough pride in competence and desire not to dump garbage on the people around you to ensure they at least want to try.
Disagree, throwing more money at devs hoping it will improve quality while continuing to take away their authority will only perpetuate the slop machine.
It's not that hard: treat people with dignity and take their contributions seriously, not as a disposable meat mass. In fact, not only will this improve code quality, it's likely to improve employee retention too.
You pay someone and give them the explicit responsibility for code quality in your system, empowering them to gate check-in with any static analysis and adversarial agent review they feel like.
It’s not a silver bullet but you absolutely can do better than just giving up.
Anything involving an AI won't work. If it did AI companies would already train their AI with it. Review solutions during training, generate synthetic data etc., or as a budget solution just route the requests through more models before giving you an answer.
This isn't true, there's a limit to how much feed-forward processes can correct themselves and well-poisoning of context is far too common.
Try using an LLM to rewrite an LLM output without the slop (vs asking for no slop to begin with) or sandboxed subagents that critique a parent's draft.
There is absolutely a step-function improvement in quality but:
1) not everyone wants to explode their cost by adding extra calls
2) this can't just be "trained in" to a system as obviously they have attempted this but the technique still provides an uplift.
If you can do adversarial agent review, or use an LLM to fix the slop and it works, then technically you already have an LLM trained which knows what to do. You can just use them to improve the other model. The issue is that we don't have these at the moment.
> Before AI there was at least some way to tell apart good programmers from bad
There were only bad ways, and the best way was to just find people who were both good programmers and cared about quality to keep an eye on the rest. Nothing much has changed in that respect.
I don't think it's that hard of a question to answer. I've noticed on my team, our thinking has shifted from how do you directly solve a problem, to how you get an agent to effectively solve the problem and not produce slop in the process.
One thing that we have done that's probably made the biggest impact is alot more upfront architecture with the knowledge that pretty soon agents will be running wild all over the code. Having worked with these agents for a while now, you get a very good sense of how they will go about solving a problem and the various footguns they will encounter along the way. Editing an AGENTS.md file or building a skill is not nearly as fun as coding by hand but it will pay dividends over and over if you do it right.
Another big thing is doing refactoring passes. Early on in our projects our agents generated ALOT of slop and we had to go back and fix alot of it. But every time we did one of these passes, a major aspect was improving agent instructions / skills / etc so it doesn't happen again. It can be a painful process at first but I found that over time, the amount of slop the agent produces goes down by orders of magnitude.
I feel like we're still very much programming, but we're now doing it at a "higher level" where we are not writing the code ourselves but instructing the agent to. And IMHO, properly instructing an agent on a production codebase is not a trivial task.
> and you have no way of telling apart AI slop vs. good maintainable code. (If you could you would just train the AI.)
I know how to distinguish good maintainable code from garbage. I have known for quite a few years. But knowing how to train someone, or an AI? I'm a good coder, not necessarily a good teacher. And there are things about code that I _feel_, not that I can rationally explain.
I completely agree with you on this. I was talking about the employer perspective. They have no way to tell apart the two and thus have no way to set the incentives.
You might write good, maintainable code, but they will prefer the slop generator who delivers quicker.
> Before AI there was at least some way to tell apart good programmers from bad
That's why companies were interviewing people on tasks that had nothing to do with writing maintainable software. /s
I understand the concern and it should be addressed and researched. But, simply saying "humans were writing code themselves" doesn't provide any evidence for better quality.
The comment you replied to did not say anything about unit tests. And "software quality" is much broader than "data structure and code complexity".
I do think this gets to the heart of the matter. I think many programmers have missed the forest for the trees on why things like data structure and code complexity matter. They do matter, but they don't matter in and of themselves. They matter because they are the best techniques we have for making software that is of high quality (the software, that is, not the code) and which remains so over time, while continuing to be developed and adapted.
I strongly believe that it is now much easier to create software that is of high quality and adaptability, orthogonally to the data structure and code complexity concerns. Those concerns remain relevant, but it's a mistake to think of them as the primary thing rather than things that support the primary thing
I'm spreading my bets a lot lately, trying a bunch of stuff:
https://modelrigs.com - a website for running open weights models on various hardware setups, including benchmarks, how to run them, and a TCO economics analysis
https://riverfish.uk - this one's a bit different! It's basically a pSEO experiment in combining fish population surveys and river data across England
https://findclients.org - a B2B client prospecting database of UK companies across different cities and industries, with a market analysis of the different sizes and ages of companies
https://octoloops.com - AI growth app. I've downgraded my time on this as although I could get users to sign up, getting them to actually use the app hasn't worked out at all; new users won't reply to my nice founder emails either. Bit of a bummer because I invested quite a lot of effort but it just doesn't seem to be working. Ironically I still use it to dogfood on my own projects...
https://signalbump.com - another experiment: enrich people's projects in monday.com with data on new (capital) projects and expansions of their clients. It's a monday.com app, the marketplace submission process was very challenging as a solopreneur. We'll see, very early days.
There's a lot going on in energy, manufacturing, mining, and of course defence at the moment. For example, we're still making progress towards economically positive fusion power; Vehicle-to-Grid for EVs is starting to roll out as trials; hydrogen is still being seriously looked at in different parts of the energy supply chain; the first flight of a mid-sized electric aircraft happened the other day; a "quantum" based navigation system that doesn't need GPS did a couple of trials in the last weeks; laser based mining, new geothermal tech, and lots and lots of autonomous/drone defence tech is being developed.
Then there's also "newspace", an interesting area to watch!
Nope. The projects I'm on where we use it, we're carefully migrating to the newer models. Where we can we test with evals to try and get an understanding of how the models have changed.
It's not all roses -- I've seen some regressions -- but generally the 3.x Flash models are pretty great for our use cases.
The great thing about LLMs though is it's incredibly easy to diversify and have fallbacks. But of course that means additional costs, mostly centered around engineering efforts to test and integrate them.
https://octoloops.com - AI outbound and community outreach for indie/vibe coders. Unfortunately it's not going well: people sign up, but then they don't use the app. I'm running out of ideas to try quite frankly.
reply