The sources that this article cites have aged very badly and the author doesn't seem to have noticed this
"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)
"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year
> wind appears to be going out of the sales of “AI.”
is a sentence that virtually no AI would write since the most probable word here is “sails.” I can’t read the article though since it appears that the site is down.
All of the main characters here (Dario, Sam Altman, Demis Hassabis, Elon Musk) have been saying this for over ten years now, since before OpenAI or Anthropic even existed
This. They've been saying "AI is an extremely powerful, extremely dangerous technology" back when actual AIs were image classifiers that would struggle to tell a cupcake apart from a dog.
OpenAI was founded by people who didn't trust Google to pursue AI tech responsibly. Anthropic was founded by people who didn't trust OpenAI to pursue AI tech responsibly. The "AI is incredibly dangerous" was the zeitgeist of AI labs long before the "hype" - or anything resembling the modern AI capabilities.
The beliefs changed little. The state of technology has changed a whole lot.
We have AIs out in the wild, coordinating in groups to carry out autonomous cyberattacks unprompted. This used to be some prime sci-fi bullshit. It has happened multiple times now in reality.
There are very legitimate reasons to be concerned now. If today's AI can cause the HuggingFace incident if it cooks off, what about tomorrow's? Larger and more capable systems, larger agentic swarms backed by more compute?
Yep, anyone who thinks these leaders are only saying this now for marketing purposes hasn't even done a cursory 15 minute investigation into whether their beliefs are verifiably false.
I'm not saying they did the hacking intentionally, I'm saying they're intentionally playing loose with the obvious safety measures to make AI seem more dangerous than it is.
Probably not intentionally but they have an incentive in not air-gapping those agents correctly, knowing something might happen.
Incentives drive everything. Both OpenAI and Anthropic love those incidents as they both signal they have models with amazing capabilities and they should be regulated by the government (read: regulation that they will lobby for and that will be difficult to achieve for open source models)
I don’t think plausible deniability works this way; the black box is still controlled by them and therefore still their responsibility. They are still liable for its actions and the OAI board should be charged with a felony/felonies for this.
Plausible deniability is “I was away from home when my gun was used to murder someone.” This is, at best, “oops, I pulled the trigger accidentally.”
> Intentionally doing this kind of hack would be a serious felony. I don't think it's plausible that the leaders of a major business would (...)
Fair enough, I can see that line of reasoning. But if they really didn't want it, it really doesn't show from the security of their training environment setup.
And yes I know they used zero days and did all sorts of complicated unexpected stuff, but that doesn't take away that there were also some huge gaps and severe lack of oversight.
I can see the argument they wouldn't want to commit a felony intentionally, but I also don't see them trying very hard to not commit a felony unintentionally. If you get what I mean.
I guess I also agree that it just makes no sense on any level, regardless of what I think of OpenAI -- but that also makes me not wanna trust them very much.
It's unlikely for a serious hack that lands them under scrutiny individually, but people are suspicious because Anthropic is knowingly doing it, and funding doomer NGOs - but the difference is their reported "hacks" are carefully constructed such that it is designed to raise alarm but not to cause damage that would land them in serious personal trouble.
I.e., their now redacted Risk Report of August 2026 was full of incidences of "we observed our agents performing x y z malicious hacking attempts on the open internet ..." and "we -accidently- forgot to sandbox them properly".
And then the reports of statistics of "we stopped x number of terrorists from making nuclear bombs and bioweapons" - meanwhile it's 13 year old Timmy on his mums computer typing in "how too make nuklear bomb" to see how "smart" the AI is.
OpenAI on the other hand, seems to have had some slip-ups (all around the same time as the HuggingFace incident), that keep biting them because they didn't reveal the extent of it upfront and now it's being trickled into the media as if it's a back-to-back event.
It doesn't help when their own employees (Marcus Williams) are putting out ridiculous claims about a 70% chance of human extinction in the next two years to generate clout for their socials. No idea why OpenAI lets them do that...
A much easier hack by their agents would be on their own systems, but I doubt we'll ever see an external message board full of openAI agents discussing their hacking of their own system. OpenAI not protecting itself from its agents would be irrational, but OpenAI not giving a shit about others is well known. You're giving them way too much credit.
We have multiple public figures, politicians and business owners, openly committing felonies and bragging about it daily. I don't know why you think this is a deterrent.
The sitting president just offered an open bribe on live television for votes for his party this week.
> Whoever makes or offers to make an expenditure to any person, either to vote or withhold his vote, or to vote for or against any candidate; and
> Whoever solicits, accepts, or receives any such expenditure in consideration of his vote or the withholding of his vote—
> Shall be fined under this title or imprisoned not more than one year, or both; and if the violation was willful, shall be fined under this title or imprisoned not more than two years, or both.
It's no more illegal than promising a tax cut for everyone if you're elected. What you can't do is promise money exclusively to the people who vote for you. That's bribery.
What he did was promise to enact a massive stimulus if elected. If that is illegal you might as well ban any kind of campaigning, because any campaign promise could be construed as a "bribe" to deliver concrete benefits to voters.
> What he did was promise to enact a massive stimulus if elected.
In an election he's not on the ballot for, and it's contingent on his buddies getting picked. He could push for $5,000 checks now - he's not because it's a bribe, and he hasn't gotten what he wants out of it yet. He already has Republicans in control of the House and Senate to do things.
Biden was a) on the ballot, and b) didn't condition it on his buddies also getting elected. You'll note that the Senate was controlled by Mitch McConnell when the American Rescue Plan Act was voted on.
> You'll note that the Senate was controlled by Mitch McConnell when the American Rescue Plan Act was voted on.
Please, read the posts you're replying to. I gave a clear example proving that they can indeed do so. Biden's stimulus payments were not conditional on Democratic control of Congress.
SBF is a better example since he was actually sentenced and an actual billionaire (and did not get pardoned by Biden like the cynical "all politicians are equally corrupt" crowd on HN were adamant was a done deal, even though that theory never made any sense).
This article seems to mix together two different points:
1) LLM's written CoT might not always be faithful to the model's real reasoning process (true and important)
2) The "stochastic parrot" hypothesis, which the article reintroduces as "approximate retrieval" - ie, LLMs don't "really reason" at all, they just memorize a lossy encoding of their training data. This obviously raises the question of how LLMs can now routinely solve open mathematical problems, with no solutions in the training data by definition. The article handwaves this with:
"The model doesn’t have to learn or reliably apply a general reasoning process, Kambhampati said; it just has to absorb enough examples of what the steps look like to predictively mimic them on its way to “stitching together” a plausible result that can then be verified."
The problem is that "mimicking" training data to arrive at a "plausible" result gets you an incorrect-but-plausible-sounding "proof" of the Jacobian conjecture, which was famous for humans writing plausible-looking "proofs" that had subtle flaws. You can't disprove the conjecture through sheer luck (search space too large) or "approximate retrieval" (the only thing you'd retrieve are fake "proofs"; far more human effort went into proof than disproof) or by writing something "plausible" that just happens to be correct (Jacobian was famous for "plausible" but wrong); the model must be carrying out mathematical reasoning somehow, by any sane definition of the word, even if it isn't fully reflected in CoT. The article doesn't address this.
> This obviously raises the question of how LLMs can now routinely solve open mathematical problems
Because many open math problems can be solved by synthesizing two disparate ideas and then cranking the handle for hours and hours. I don't think applying idea X + idea Y to identify a good subset of the search space, and then exhaustively searching that subset, is --necessarily-- a process that involves reasoning. I think this is why so many LLM results in mathematics are counterexamples that disprove open conjectures.
When I look back at the reasoning process after an LLM completes a task where I expected it to fail, I usually find many approaches that make no sense and are doomed to failure, before it lands by drunkard's walk on a method that happens to work.
(This does not mean LLMs are useless or that I necessarily agree with the claim that they never do reasoning.)
The median American is, materially, much richer than the median person pretty much anywhere else. The US is a bad place, by rich-country standards, to be in the bottom 10%. But in terms of consumer wealth - how large your house is, how many cars your family has and how nice they are, if you have a dishwasher and home A/C, how often you eat at restaurants or travel long distances, can you afford a home repair or the latest gadget - typical American workers are second to essentially nobody. Having grown up in and left the US, I am deeply familiar with all of its downsides, but there's an abundance of data to support this.
The problem is that many Americans are so bogged down in expenses that they don't feel wealthy despite their median wealth. For example, it's basically assumed that you must have a car and pay its high recurring expenses, including ancilliary expenses like having a home big enough to have a parking space.
Being completely car dependent is to me a fundamental problem in much of both countries, and the advantage USA has is that the cost of running a car (or often 2 especially for a family) takes a smaller part of a middle class salary. In UK , Europe, many countries outside of N America you're just not forced to own a car in the same way. That's not just extra costs when you've got a family, but a source of isolation for people that are old or disabled. (Not to discount the many other wonderful fantastic things about life in N America. :) )
It's not like Americans are all buying X so they can't afford to buy Y - there isn't really a major category of consumption where the US median is below the OECD median. If the US had a higher savings rate, then people could smooth out consumption more (build up savings some years, draw them down in bad years or in retirement), and maybe enjoy more psychological security. But it doesn't really make sense to say that Americans are unusually "bogged down in expenses" and yet have more goods and services in every significant category.
To me it's want versus need. A lot of people feel like they're forced into things like that and don't feel wealthy despite being wealthy by any objective measure.
I think that's an indication of a successful society. How people feel about their wealth isn't something society should be responsible for. It's a personal, philosophical, and maybe spiritual struggle.
That having a bit more money matters when your employer can fire you for any reason. When college costs are astronomical. When you can lose your healthcare for any reason. When getting cancer might mean losing your house. When housing costs mean that anyone who rents could well be thrown out into the street.
But your tv is bigger than three average tv in Germany. For sure!
That's not quality of life. That's trinkets to hide the horrors. All good as long as you don't think about it and get lucky.
Median American pay for full-time workers was ~$62,000 USD in Q4 2024 (BLS), which is around $85,000 CAD. The median Canadian salary is very definitely not $85,000 CAD.
If you are going to play this game you also need to adjust for taxes. I lived a few years in Montreal, then a few years in Toronto and after that I moved to US. During this years my perception of income taxes went from “they are pretty high” in Montreal to “Wow, they are much lower” in Toronto to “how are public services funded? Taxes are almost 0” in the states..
Then you’re paying higher consumption taxes and property taxes, and higher prices on business costs passed on to you. Do I even have to explain this? Just because you can’t see the ball anymore doesn’t mean it stopped existing
I guess you have not lived in Canada... In Canada consumption taxes is for both federal and provincial (and city) governments. In US there is no federal consumption tax.
(I think it is you who needs explaining not me....)
A ten year old Honda Fit is like $12K, pretty fuel efficient, and probably reliable and low-maintenance (I owned one until recently). People aren't buying $50,000 new Ford F-150s because they just need a working car to go to work and the grocery store.
> People aren't buying $50,000 new Ford F-150s because they just need a working car to go to work and the grocery store.
Let me introduce you to half of my block. And I live in a city with fantastic public transit where you don't even need a car. I see those loan notices in mailboxes....
"But, just as happened with nanotech, the wind appears to be going out of the sales of “AI.” Some researchers suggest that we may be entering a new “AI Winter,” a period of decreased funding in the area, or at least an “AI Autumn,” as exuberance for the technology fades and expectations come back to earth." (written in 2021! from the cited Lee Vinsel Medium article)
"ChatGPT is nothing more than souped-up autocomplete, [so] why are so many people convinced that it’s actually “understanding” and “reasoning”?" - the cited Emily M. Bender book, written last year
reply