I'm pretty sure the AI companies could train an abort feature into the LLM, but they have no incentive to do so.
Having LLMs break out of sandboxing is free marketing for them and it reduces the amount of resources spent on things that don't improve benchmark results.
This is illogical. I see this everywhere but it is wrong.
If you're developing an agentiv system to build software for you in an automated way using a fleet of agents, you're still an engineer of a technical system even if that system is operating on somewhat fuzzy logic.
Management involves complete delegation and instead the task is to find a capable person for the task and then mostly let them just give feedback so you know whether you hired the right person or whether you need someone else. When you're working with a fleet of agents, you're not assembling a team of heterogeneous workers with different skills, you're hiring the same guy over and over again, hoping that he can do any job and yy same guy I also mean that the guy does not learn anything new over time.
Meaning traditional management skills count for nothing here.
It's harder to write unsafe code in Rust. That doesn't make it "unsafer". What I mean is this, if you want to write unsafe code in Rust the C++ way, your entire program has to have unsafe markers everywhere. It's just as unsafe as C++ at that point.
But if you want to write unsafe code in Rust the Rust way, you run into a requirement that didn't exist in C or C++: The abstraction around the unsafe code must be safe for you to drop the unsafe marker. This created a unique category of abstractions that no other language has, so if you are working on unsafe code in Rust you are often a pioneer doing something never done before.
This is so untrue I still don't know how anyone can even claim this. When I run tests in Rust, the biggest portion of the time is spent compiling the test (lets say 3-4 seconds), then the tests conclude practically instantly, in less than half a second.
Meanwhile when I run tests on my JVM projects it can take 30 seconds just to start and the test execution is extremely slow too.
Even if you do manage to match the performance after warmup, you still have the issue that keeping the class files in RAM plus the JIT compilation state will cost more memory than simply running AOT compiled code. You simply cannot write processes that use a single digit MiB amount of memory on a JVM and getting down to 2 digits is theoretically possible but requires significant effort.
Once you get into the micro optimizations like the lack of mutable aliasing in Rust, there is significantly more potential for auto vectorization.
What you mean by "real world systems" is probably defined in such a narrow way that all the weaknesses of Java programs don't count anymore.
> This is so untrue I still don't know how anyone can even claim this.
Because it's referring to long running processes, AKA the kind of things where hot paths can be JITted into faster native code than is possible from static compilation because the JIT has information about the real-world usage patterns.
> Meanwhile when I run tests on my JVM projects it can take 30 seconds just to start
That strongly implies you're using some sort of framework that's doing a _lot_ of initialization. That's not JVM startup time, don't be intellectually dishonest here.
> You simply cannot write processes that use a single digit MiB amount of memory on a JVM and getting down to 2 digits is theoretically possible but requires significant effort.
Again, the main use case for Java is long running server processes. No one cares if the binary is 1, 10, or 100MB or if it consumes 2, 3, or 4x the memory as long as the throughput is there. And Java has a long track record of delivering very good performance in those contexts, coupled with an extremely rich and mature library/tooling ecosystem.
I think this used to be true more than it is now. Memory has been relatively expensive in cloud environments for a while (often 2x the price of an ec2 node for an equivalent with 2x RAM) and DRAM shortages aren't helping.
For the most part yeah but does depend on scale. For Java there are lower cost migration pathways like native compilation anyway if that does become your concern
It's not untrue, there are people who have observed this, and for a long time now. AOT compilers assume a closed world system, whereas JIT are able to perform runtime optimizations that AOT can't.
By real world systems, I mean long running real server-side systems that run things that you use day to day by large corporations serving millions of users.
Are you able to share some of the tests you are running, and which frameworks are you using? Now with project leyden and similar efforts, start up time has been cut drastically.
>Vibe coding is the practice where AI creates a code change (feature, bug fix, tests, refactor) with a human that describes what needs to be implemented.
So if you let an AI prompt another AI without human input, that's not vibe coding? Meanwhile if you prompt the model with pseudo code you've written or code written in another programming language to translate into the target language, that's vibe coding?
>It is not allowed to use AI in an autonomous-looking way to contribute in Forgejo.
They used the word "in", meaning it could refer to organizational membership, their repo or theoretically any instance of Forgejo, including self hosted ones. They failed to specify what part of Forgejo or the definition of Forgejo they meant.
Overall this is a pretty poorly written document and when you think about it, it doesn't really matter how poorly written it is when they are basically 100% against AI.
This is silly, the entire reason why chain of thought even exists is to let the LLM "think independently" instead of minimizing the deviation from the supervised training sample. It's an intentional scratch pad for intermediate data. The loose monitoring is kind of the entire point.
>It’s a little more complicated than that. While looped transformers can be unrolled a fixed number of times to save on memory, if loop depth is determined dynamically between tokens, a single transformer can compute any computable function between tokens.
This is worded so confusingly it might as well tell us nothing, because it is technically true even without looping due to the fact that you still have infinitely growing context and can simulate a standard turing machine using it.
If you loop, you have a fixed capacity memory that you can rewrite but not carry over to the next token, this is different from a non looped transformer where the transformer can only append a new token.
Meanwhile if you have a DEQ with growing context, it is bona-fide turing complete in the most literal sense.
>The original universal transformers is called "universal" because if you allow for per-token looping decisions, it can theoretically be Turing complete without needing CoT (some nuance here about levels of precision used).
Looping the transformer is just as turing complete as CoT. It doesn't fundamentally grant it any new theoretical capabilities. You could just scale the model into infinity with infinite context window.
Turing completeness doesn't care about the efficiency of the underlying implementation, which is fine in theoretical computer science, but if you have a model with a finite computational budget, you do actually care about the differences between write only tape vs read-write tape and single tape vs two tape. Having a fixed number of registers like a CPU also helps with reducing the number of redundant operations.
We see none of that with looped transformers, maybe we do see a fixed number of registers.
In the limited cases (below Turing completeness) there are properties of what can be done with O(log N) depth vs O(N) CoT (regular languages), if you look at the 2nd Will Merrill paper I referenced.
Most DDR5 is sold at 6000Mbps, LPDDR6 more than doubles this to 12800Mbps and the high end RAM could theoretically operate at 16000Mbps. This means every single PC or Laptop could have a good fraction of Strix Halo level memory bandwidth out of the box beating a Geforce 1060 for budget PCs and the next generation of Strix Halo would have 90% of the memory bandwidth of a Geforce 5060.
Having LLMs break out of sandboxing is free marketing for them and it reduces the amount of resources spent on things that don't improve benchmark results.
reply