Yes, but the the implementation with the additional iteration also has to have extra checks for that iteration, because it can't blindly iterate to the ith file, as there may not be i files anymore. So the implementation gets more complex overall.
Can you tell me what the zstd invocation is that corresponds to the default invocation of bzip3, which uses block size 16 MiB (according to the man page)?
I got some really good results with bzip3 compression Wikipedia XML dumps, and I would like to check if it's actually better or if I was just calling zstd wrong.
If you want a 16MB window, use `--long=24` (2^24 is 16M). (I believe this is larger than the default window for zstd level 3, but smaller than the default window at higher compression levels.)
>iv. Never use the passive where you can use the active.
Orwell himself routinely ignores it, even in the first sentence of the essay:
>Most people who bother with the matter at all would admit that the English language is in a bad way, but it is generally assumed that we cannot by conscious action do anything about it.
The second clause could be rewritten in active voice by changing it to "but people generally assume". But this would make the writing worse, and Orwell, as a good writer, probably didn't even consider the option of making it worse, and therefore didn't notice the passive voice.
Passive voice is an essential tool for all good writers of English. I always give the example of the opening of Pride and Prejudice [0]:
>It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.
The joke doesn't work in active voice. If you attribute this acknowledgement to some specific group of people then it's simply false, not a comedic exaggeration.
Technically, I think, that first sentence, by Austen, uses a passive participle but does not use a passive voice for any finite verb. I don't think that advice to avoid "the passive" is intended to apply to that situation.
For example, nobody would seriously suggest avoiding the passive participle in a sentence like "Put the broken plate in the bin". ("Put the plate that someone broke into the bin"?)
There's a rather simple reason this is so: A thought is typically most powerfully expressed in a sentence that puts the most surprising or notable word last, immediately before the period. Ideally, without any qualifying adjective/adverb.
And "... a wife." really turns on the ears, no?
So advice to prefer active voice is not wrong -- but the skilled writer adds other values into the balance.
> It's a bit of a pet peeve when people include quotes on a blog post without linking or otherwise references their source.
You’re peeved with good reason. It’s the blog equivalent of posting a screenshot of an article to social media. People, please post your sources! In the age of misinformation, that’s more important than ever.
You shouldn't have to click away from a blog post to understand what the blog post is about. It should start with a clear self-contained introduction.
atproto.com contains a succinct discription:
> Atproto is a big-world open social protocol. Users publish JSON records into repositories. The changestreams of those records then sync across the network to drive applications.
Still a bit abstract, but OK. Personally I would include that AT protocol stands for Authenticated Transfer Protocol and that it is the protocol powering Bluesky.
> When the data fed into an AI model is based on real data you can get predictability. When the AI inputs start coming from the AI outputs the wobble is introduced which results inevitably into delirium.
This is asserted without evidence. There is absolutely no proof that AI requires humans in the loop to function or improve itself.
Compare it with Deepmind's go-playing program, AlphaGo, which mainly involved training a neural network on a large database of high-level human games. It defeated one of the top-ranked players in the world, Lee Sedol, but arguably it was drawing from human experience just like you described.
But it didn't stop there. After that, Deepmind developed AlphaGo Zero, a version that was trained exclusively through self-play, with no human feedback in the loop. That's what you would call "AI inputs coming from AI outputs" but it didn't have the result of “resulting in delirium”: instead, it became orders of magnitudes stronger than the original version (which it defeated in a 100 to 0 competition after 3 days of training).
This shows that AI can improve itself without having access to any human knowledge, and indeed transcend human performance by orders of magnitudes. There is absolutely nothing to suggest that general AI cannot improve itself the same way.
People who claim otherwise are engaging in wishful thinking; they just assert their conclusion, but have no rational arguments to back it up.
You have no evidence that self improvement can work at generalized tasks. Go is a simple game with a clear win condition, but deep strategy and near infinite permutations of how a game plays out. Winning a game of Go is a task well suited to machine learning.
One might say you are engaging in wishful thinking by believing it’ll just continue to work across all domains. The world is much bigger than a Go board.
> You have no evidence that self improvement can work at generalized tasks.
I never made that claim. I just said it's way too early to rule it out: there is no logical reason why AIs will (always) need to have a human in the loop, and we don't have enough experience with LLMs to know what their true limits are.
I referenced AlphaGo not because the game of Go is exactly like every other task AI might perform in the future, but because the evolution from AlphaGo (which was trained on human games) to AlphaGo Zero (which was not) shows that at least in certain domains, it's not only possible to take the human out of the loop, it can actually make AI perform better.
I'm not claiming this will definitely be possible in every other domain, but people who state it definitely won't be, are jumping the gun.
Both programs had programmed in to their routines the rules of go (and for Alpha Zero the rules of chess and shogi). The programs then followed an optimization program. Absent the extremely specific "rules of this game" set by a human none of the programs work.
Amusingly, all three games are in the game theory sense trivial because they provably have a correct solution (probably a draw for chess), but the search space is so large that it is unlikely we will ever collectively invest the resources to discover the answer.
AlphaZero works in simple and fully observable environments like board games.
RLVR for LLMs works for coding and math domains which can be verified by code.
AlphaZero, RLVR and other RL methods don't work for learning dextrous manipulation for robotics, for example. Learning in robotics depends on human data.
The key word in that paper is “indiscriminate”, as in:
> We find that _indiscriminate_ use of model-generated content in training causes irreversible defects in the resulting models
If you view AI training as lossy compression of their training data, then lossily compressing the same data repeatedly will result in data degredation; this is well known from other domains (try repeatedly compressing a JPEG image, for example).
That means it's extremely important that there is some content curation in the loop. But there is no reason to believe this content curation must be done by humans, or that it must exclude all AI-generated content by default.
For example, the recent LLM-generated disproof of the Jacobian conjecture would probably be beneficial to include in the training data, despite being the result of an LLM.
That paper has had a pretty turbulent reception and looks pretty conclusively wrong at this point.
It used an incorrect theoretical framing that assumed that data was being replaced rather than accumulated as a result of more training (see https://arxiv.org/abs/2404.01413 which explores this). This is incorrect because this simply isn't how real-world datasets are created via synthetic data generation (which generally accumulate more data over time rather than replace their data). As a result most of the theoretical results were invalid.
Empirical evidence has also cast a considerable amount of doubt on the paper. For example, Microsoft Phi-4 was an empirical test in specifically what happens if the majority of your training data is synthetic rather than human and it turns out that Phi-4 did significantly better than previous models which relied primarily on human data.
There's some nuance to all of this in how exactly you do this, but the original claims of the paper are looking really shaky at this point.
You are referring to gans which have complete understanding of the rules and an api interface to leverage interacting with the world.
While I think there could be an argument made here you are mixing apples and rocks if you believe ChatGPT is using gans.
I hate to be that person but you are showing that you are unable to differentiate between linear regressions, clustering, gradient descent, and generative adversarial networks. You are over your ski-tips and don't see itc
reply