But code is the mean to get an idea across to someone. It’s the difference between “I think this can work” and “I’ve worked on this and it does work”. I’m part of the OpenBSD mailing list and it quite nice someone sharing a diff for an idea or experiment. It may not get merged, but it’s better than realm of deliberations.
> as someone coming from software I find learning electronics pretty hard as it requires a completely different mental model and way to approach systems
I went from electronics to software and one thing that has puzzled me is how much people dislike reading docs as in reference manuals. People can get by with sloppy code full of hidden bugs and when those bugs arise they’re like deers frozen by headlights.
Imagine building a circuit without any ideas how it operates. I’ve encountered web devs that don’t understand how http works.
> Open source programs could be more like motivated explanations of computation.
It is already that. Every time a method/function is created, a structure is defined, a variable is added, a file is created or renamed,… It’s all for the purpose of human communication. The computer only need binary in a single file.
But people feels like they should be able to jumpninto curl code without any understanding of networking, or linux code with no knowlede of computer architecture. Few code are meant for total beginners.
We have good engineers that gave us curl, ffmpeg, the 4.4BSD, tmux, vim and emacs, x11,… To this day, no one can show any particular important software that comes from LLM assistance at scale. It’s all slop.
Well it's been barely a year, but basically all commercial software, and most open-source software, is engineered with AI assistance. It's not all slop, you just aren't looking around. Ask any engineer at any company - are they using LLMs to assist in writing code? All of them will say yes. Source: I'm a software engineer working on commercial software at a company, I also talk to many engineers at other companies.
The thing is a text generator. It generates text. You can couple that with any code that gives rhe ikkusion of a normal decision workflow, but it does not make any decision more than a software like latex. According to your definition, the latter would “decide” the amount of words to put on a sheet of paper.
> I’m not saying gradient descent was empirically discovered, I’m saying that its use in machine learning is itself a form of empiricism. A repeated observe/adjust-based-on-data cycle
The data is the input, the output is to generally find the lowest amount of a loss function. It’s a greedy approach because brute forcing is inefficient.
It’s no more empirical than a greedy algorithm for scheduling.
> It’s no more empirical than a greedy algorithm for scheduling.
Right, GP is drawing a distinction between search, ie mechanical exploration of a space, with understanding, ie having a map of the territory such that you don’t need trial and error.
> The best moments of these is even when they are confusing on purpose, because imprecise language is a gateway to deeper levels of meaning through fourth-wall breaks, metaphor, analogy, and humor.
I don't agree. Because those metaphors and other word plays reach deep into the human mind (at least for the purported audience), while most technical writing try to be more explicit.
Here is the introduction for Laravel Socialite
In addition to typical, form based authentication, Laravel also provides a simple, convenient way to authenticate with OAuth providers using Laravel Socialite. Socialite currently supports authentication via Facebook, X, LinkedIn, Google, GitHub, GitLab, Bitbucket, and Slack.
And this is the marketing blug of Shadows of the Gods by John Gwyne
A century has passed since the gods fought and drove themselves to extinction. Now only their bones remain, promising great power to those brave enough to seek them out.
As whispers of war echo across the land of Vigrid, fate follows in the footsteps of three warriors: a huntress on a dangerous quest, a noblewoman pursuing battle fame, and a thrall seeking vengeance among the mercenaries known as the Bloodsworn.
All three will shape the fate of the world as it once more falls under the shadow of the gods.
One is direct, with not a lot of imagery, but rather use specific concepts which has precise meaning. The other is just as clear, but use concepts that ties to bigger ones. They are not imprecise, they just let you be aware there's a bigger canvas than the literal interpretation.
If you use a loaded dice, you can be pretty confident about where it will lands. It may not be 100% accurate, but can be quite close to certain. Without training the weight are pure noises. After training, it leans towards coherent sentences and particular statements.
Yes, and I believe my point still stands. We thoroughly understand the principle by which a loaded die can be intentionally biased despite not being able to predict the outcome of any given throw due to the system in question being a chaotic one.
In contrast, we do not understand LLMs in the same way (nor biological brains). Claiming that anything of that nature is simply biased towards coherent output seems entirely reductive to me - the question is how such coherence arises in the first place. There is no meaning encoded or computation performed by the particular pathway a die travels through the chaotic landscape.
Sure an argument can be made that it's "just" a next token predictor thus how is it really any different from a markov model? Yet the output is not even remotely the same.
> In contrast, we do not understand LLMs in the same way
From my point of view, (not a ML researcher), it’s due to the magic of numbers. The same thing happens with computer vision and neural networks. There’s a bunch of magic weights that get created which has no meaning by themselves, but computing them does help with detecting objects.
So if you take words, derives them into tokens, use the attention techniques to extract the “coherency” aspect, it’s no wonder you can replicate “coherency”. Add reinforcement learning to that to increase towards certain aspects like correct code syntax and you have heavily loaded the dice again.
We have used maths to model chemistry, biology, and physics, as well as economics and sociologic phenomena. Then we use maths (more specifically logic and set theory) to usher in the age of information and computing. Now you want us to act surprised that maths, through ML, can model language.
Maybe further down the line, we can have a simpler set of formulas for language coherency, but for now we have to make to with using the whole internet and a bazillion watts of power to guess the weights for the generic ML model.
I'm comparing it with chess. Chess is pretty complex, complex enough that only a small subset of humans can play it at a very high level. Introducing computers to chess first led to a statistical and brute force approach. But once that had paid off and the results were in people spent a lot of time analyzing those results and this led to an entirely new class of engine that was far more efficient than what had gone before and which performed even better than the 'big iron'.
I would not be surprised at all if we will find that AI will go the same route. The fact that we don't know how it works is where the opportunity for improvement lies.
reply