More and more such experiments. I felt sad for a couple months when I realized that writing code will not be the same since. Now I am on the other side.
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
I think I am on your arc as well. My learning on different topics is growing every day, but there’s a limit to how much I can absorb. With the LLMs the experiments stay just beyond that horizon and I keep chasing.
Stated too strongly, but I think this could be the model for education (some subjects anyway). Everything personalized to your learning goals, grounded in experiments that give a tight feedback loop and with a model that never gets tired of re-explaining something for the 10th time.
Yes they are great for learning at own pace, trying out new things.
I have accepted two things that make me a happy engineer now: AGI is not here no matter what they say and LLMs are still very useful if one knows how to use them.
They are another layer of abstraction and like you said they do not tire. There is a lot of optimization needed so we can reduce wastage (running 1T+ LLMs for most work is wastage).
Haha, I thought about this today. I'm traveling and walked through a university campus. The new students are coming in and I thought wow, wouldn't it be wild to be a fresh in student in the age of AI (it's been a long time since I was college). But, then I thought, actually this could be an incredible time to be a student. If your curious and motivated the agents can be amazing partners. I think the students who crack how to use AI now are going to be the ones who really run the table as this economic wave crests. But, yeah, you're right on with the Wall-E comment. I would guess many (most?) will turn their brain off and then what's the point of even being there?
When we hyper focus on finding something, we find it all the time.
The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?
This is a tiny LLM doing all the heavy-lifting. Any mention of the training process? I am obsessed with tiny LLMs and the do-one-thing-really-well approach that they seem to fit very well.
I experiment a lot with local LLMs, particularly small ones like Qwen3.5 4B and 9B. I have build multiple experiments to make harnesses that use these models for code generation, planning, local search, etc.
These are really good models but the harness has to be built around them. I have a ton of generated system prompts for specific purposes. Even parts of a SolidJS stack, for example Route management, has its own prompt. These are experiments but the results are real. If we build harnesses around small models, we can build a locally running WYSIWYG editor which works on plain text prompts.
The performance, in simple tokens/second, is not the most important factor. For many private data points, like emails, I would rather have a local graph based search and LLM on top where the harness is specific to problems like calendar, contacts, finance, etc.
I run all experiments on an 16GB M4 Mac Mini but coding agents building the harness are a mix of Codex, Claude Code and opencode.
Somewhere around early 2000, I used to work part-time with a local video editing agency. Matrox video capture cards were the really high bar, other than Avid. At least that is what I remember. We had a couple of Matrox capture cards, expensive for our tiny budget. Other than that and a few 3D cards also around that time, I forgot about Matrox.
I do not think Cloudflare was a less-than-peers optimized product when they launched. This is one of their blog posts which describes taking one aspect even further.
I think Cloudflare became big only because they were so much more optimized than others that they offered some services for free that others were not offering. If running costs are high, you only burn (VC) cash and then you exit.
There is the entirely plausible option of the NSA indirectly bankrolling them to counteract the growing number of TLS connections. We terminate your TLS for free in our server and you don't have to change anything was a sweet deal for website operators _and_ those that want to look at unencrypted traffic. Given enough sustained funding they could undercut competitors and grow to what they are today.
I use Claude Code, Codex and opencode pretty much interchangeably. I am currently using Claude more this month because (stupidly) I paid for Max ($100) since I have a large client project.
I barely spend time or thought in making prompts. I have a simple text file with a few combinations. They refer all the common files (README, AGENTS, DEVELOP, etc.)
All reference software is cloned locally and the docs mention that. The prompt templates then boil down to research mode (write Epic) or worker mode (write software) or review mode (leave review notes in Epic). That's it.
Many of my harness experiments are about text manipulation, text search, graph on text. Because that is what LLMs are - text processing systems. Cut parts of prompts, cut parts of response, cut parts of user's intent. Join, break into epics/tasks, run with LLMs, repeat.
I can understand that this is not the exact location where the floods actually happened. But I am sad and I am frustrated and I am angry.
We had floods in Sikkim, which is practically a "stone throw" away from a world-level map point of view. I was there then. There was past research but everyone was so surprised. Activists talking about that event 10 years prior. Published reports.
Yet, we do nothing to move people away. To pause and think what the hell is going wrong. Just keep hustling and pretend shit is not going to go wrong. I am tired of our* culture.
our: all of us, I am not only talking about India, Nepal or China here.
I'm from Iran and sadly it is the same here as well. Our country is in a bad shape given the political/economic situation and there are so many warnings that are common knowledge, but given the economy of fixing them, nothing really happens to prevent them. We had a mall that was decades too old and very unstable. The fire department had issued many safety warnings and had tried to close the place for safety; it did not happen and the building went on fire and crashed to the ground, killing many civilians and firefighters. Its a shame things like this happen; made me incredibly sad to see this report. Hope nothing like this ever happens to anyone.
Someone said that it's because we humans are very good at reacting at immediate danger and very bad at reacting at risks building up slowly.
I don't know if it is true, but by today still voting for parties denying climate change all the way and that have done nothing in past years to prevent this type of risks is crazy.
> Someone said that it's because we humans are very good at reacting at immediate danger and very bad at reacting at risks building up slowly.
I have a tiny lithium battery on a desk in my lab that started swelling about a week ago. I've been meaning to go to the recycle place every day since then to drop it off, but nothing exploded yet, so here I sit with a slightly swollen battery some meters away from me.
If you have an oven light or a resister, you can use it to almost fully discharge the Lipo battery. Doing that will mostly render it inert, but if it's already swollen, you'll want to do it in a metal bucket/container outside. It can take awhile depending on the capacity of the battery. (This is how I dispose my FPV drone batteries).
I have plenty of fire alarms, both local and remote, and extinguishers of different types around, I'm not too worried :) But thanks for the concern kind stranger!
The other thing to do is discharge it, so it has little energy stored to maintain/enlarge a fire
A drone engineer I worked with would immediately put any suspect battery into a barrel of water for several days, then get them disposed of properly.
The risk really seems trivial, probably because of our extensive experience with ordinary alkaline and NiCad batteries with a very benign degradation pattern, but the risk with Li-Ion is no joke.
All your fire extinguishers will do you no good if it goes off while you are out of the house. Also, the fire extinguishers will only suppress the surrounding fire, and do nothing about the core Li-Ion fire itself.
It's trivial until they go, and then it is instant misery. I've done it on purpose just to see how bad it would be and even knowing it will happen it will still surprise you with how sudden, total and hot those things burn. You have < 2 seconds from first signs of ignition to all-out fire with the battery moving all over the place due to outgassing. It's a thing to behold.
How am I supposed to be able to observe it slowly swell more over time then? I think I might have overstated how big this battery is, or understated how small it is. It's fiiine :)
PSA: check drawer with old devices regularly. Found a swollen battery in my PSP, after not touching it 7-8 years. Same with an old motorola A920 (quite old). Who knows how many years both had swollen batteries but, then again, newer batteries have much more power to contain.
If you think about it though, there are billions of shoddily made lithium batteries sitting in drawers everywhere for years now, and yet there are only about 10 fire incidents a day.
So like many other boogeymen, driving to work is still far more dangerous. Back of the napkin shows about 100x more likely to die in a car crash than experience a battery incident in any given year.
People are good at reacting to a common danger. Look at the Boxing Day Tsunami - people standing and watching the sea as they just didn't know what it was as they had never seen it before.
This month a 'hunger stone' appeared in the Danube, indicating the levels for pre-20th century famine.
And various other markers in many countries for 'the water reached this level', ironically, on the side of existing buildings.
Yet we build closer and closer to rivers because (flash) flood events happen rarely, and typically not in our lifetime.
You're a little bit older than me - and I recall reading that 'our era' is actually pretty unusual for how quiet it has been in terms of natural disasters and their magnitudes.
Last major volcanic eruption around lots of humans: Krakatoa - 1883
Last major M9 earthquake before Sumatra: Chile - 1960
and so on. Things seem a bit more 'major' now because there's 7bn+ people.
It must be hard to discuss that in Italy, where millions of tourist (like me) queue to visit Pompei (or, better in my opinion, Herculanum), and then go back to their AirBnB in... Napoli, aka the backyard of an active volcano.
As a friend says, "Our collective cognitive dissonance is something I would not want to climb with a bike."
It’s very different to visit a risk area as a tourist. Your chances of being buried are slim for a particle week. But for the people living there every day, the statistics compound. For a long enough period it’s almost guaranteed you will suffer. In the case of the downhill villages in Nepal, everyone knew those lakes would eventually burst , and in a matter of a few years. To continue building and living in those areas was completely reckless and I want to believe that anywhere in Europe they would have evacuated such villages decades before disaster.
By the amount of (some of it useless) tech we've come to build in the last 3 decades, we should have arrived at the stage where we can collectively control the CO2 levels in our atmosphere, by now. For ever.
We can increase them, it's just really hard to decrease them which is why scientists have been telling us to stop increasing the CO2 levels for about a century.
Cutting down on annual emissions will not be accompanied by reduction in atmospheric CO2, unless the current rate of emission is lower than that of natural processes that take CO2 out of the atmosphere.
That's why atmospheric CO2 continues to rise, despite cuts, if the cuts at global scale are at all real.
It does not just take a lot of energy. If we got unlimited free energy today it would take decades if not centuries to build enough carbon capture facilities to just remove as much CO2 as we're producing.
Of course the free energy would instantly allow us to eliminate a huge amount of emissions but that's not really relevant to the point. All the concrete for the tens of thousands of facilities would add up a good amount of additional CO2 to scrub as well.
>Someone said that it's because we humans are very good at reacting at immediate danger and very bad at reacting at risks building up slowly.
Humans are really good at stealing, lying, beating their wives and all sorts of other bad shit too but you don't see people going around saying "aw shucks, it's just nature, whatcha gonna do". One of the benefits of society is to tame human nature in such a way that we all benefit more than we lose and those benefits let us outcompete the adjacent society whether that's the cavemen or a nation state.
Whichever society figures out how to marginalize the voice of the unprincipled, the emotional, those who lie with plausible deniability for their own (often not even financial, just in their own damn head) self interest and overall detriment will mop the floor with the rest of them.
There is also an effect being caused by social media and video sharing - loss of context.
The entire Himalayan foothills area (which includes what, at least 6 different major countries off the top of my head) are potentially impacted by something like this.
Most of these areas are barely surviving, and extremely poor, and often very disconnected from the rest of the country due to the terrain.
Realistically, what do you think they could have done to prevent this, given the likely 1/1000 or lower chance it would happen to them specifically?
People live there already. How do you propose to move them and support them in the place you moved them to? Please don't make me break out the Aquaman joke.
I am not talking to this specific case but about a general phenomenon that I see in my country, where people are being incentivized to build on hazardous areas by way of corruption.
Depending on the scale one could incentivized people to move to safer areas.
For some who are very poor yes I agree. But many are baited in by corrupt policies by a nexus of politicians and contractors/builders -- cheap land, cheap permits.
There is a lot of land that does not have such extreme dangers. Of course, some countries have it worse than others. Japan would be one such. Good or bad, the population has been on the decline there for unrelated reasons.
For earthquakes this is what my country looks like
This is at a very broad resolution. One can refine to make this more local. Earthquakes are just one type of hazard. One has to take other hazards into account.
That's not quite it. Its about policies that take into account the cost of future events and balance it against the ease of building. The future needs to be priced in, in today's prices.
Regulations (for example, you cannot build unless and insurer is willing to insure your property against known local hazards) and enforcing the regulations can make that happen. Often we move in the other direction by adopting policies that encourages mispricing.
You’re the one making it all or nothing. give some concrete examples to prove me wrong, or stop bs’ing.
don’t forget major earthquakes (most of California and the entire Pacific Rim, to name some well known areas), floods, serious droughts, tornados, hurricanes/cyclones, fires, subsidence (into the ocean or sinkholes), volcanic eruptions, tsunamis and lahars.
It’s about frequency and preparation, but it’s never perfectly safe. No building code handles everything, it’s impossible. Hazards are somewhat mitigated or ignored, not entirely dealt with.
Not sure what you mean by give concrete examples, when my position is normative.
I am not sure where you are getting "all or nothing" in my comments. If you are interested, you can read my other comments on this story to get perhaps a fuller picture of my position.
I understand my comments have worked you up, but I am not sure why. If you have lost people in this incident, I am genuinely sorry for your loss and can understand emotional outbursts. If you have dear ones engaged in practices that I am disapproving of, well, then in that case too I can understand your displeasure.
If you thought that I am claiming that adhering to my position will ensure that future death tolls would magically become zero then that's uncharitable reading.
My position is quite simple, price in the hazards based on scientific hazard assessment. Enforce policies that encourages correct pricing, rather than part take in corrupt practices that encourages mispricing of the kind that benefits the political class and builder class nexus that operate from the safety of being out of range.
This can be done through regulations. For example, we have mandatory third party accident insurance for vehicles. That's an example of a regulation that encourages pricing in the hazards.
Hazard zoning is not all or nothing of the type (a) permitted and (b) not permitted. It is about assigning areas to different grades of quantized hazards. See my other comment for example of such zoning for Earthquake hazard.
With hazard priced in, the markets would mostly take care of where to build new homes and what kinds to build.
In Himalayan states of India we have the opposite going on. Politicians, in concert with the enforcement bodies that are peopled by their flunkies, encourage illegal constructions in known hazardous areas, deregulates zoning so that dangerous areas are cheaper to own property on in exchange of kickbacks from the builders.
> don’t forget major earthquakes ...
I am not forgetting that for sure but not sure what your point is. If it's -- look here are other dangerously built up areas in the US -- then yes I agree.
There was corruption involved in managing the levies that were supposed to protect New Orleans too.
> or stop bs’ing
And yes there is no need to be rude. It does not make your point stronger.
On the otherhand I live here, with large part of my childhood in an earthquake hazard zone, the highest in fact. If you have lived here and know your high school geography you would be well equipped to guess where.
I am not sure I hear any constructive suggestions from you other than lulz. Unfortunately that's not going to make a difference.
In any case, since this has devolved into personal insults. I recuse myself from a wrestling bout in the mud, sorry about denying the pleasure.
For an idea about the rampant corruption in the Himalayan states of India that promotes illegal and dangerous real estate projects. There is a cartel of politicians and builders who are making a fast buck selling to those who are not fully aware of the magnitude of the dangers.
Even in the US you'll find states which ban the phrase "climate change" from state law, or scientific study of sea level rise. In regions profoundly at risk from both.
The way I justify it in my mind is that, by all accounts, there's never been a global civilization of any kind on this planet before. Especially not one with sentient primates connected by instant telecommunications and powered by capitalism. There is no existence proof that this can work. So when it seems like it is failing, I take solace in that maybe it's not even possible.
I'm really not sure if any combination of political and economic systems could have avoided the mess we find ourselves in. Capitalism and free enterprise powered by fossil fuels does seem like it strapped a booster rocket to our inherent greed and inability to live in balance with our surrounding ecosystems, and now we've got an unhealthy combination of massive momentum and no political will to redirect human development towards sustainability. It's going to be chaos because global civilization seems to be uncontrollable and voracious. So, buckle up?
It argues that following the global economic crises of the 1970s, governments, financiers and technological utopians gave up on trying to shape the complex "real world" and instead established a simpler "fake world" for the benefit of multi-national corporations that is kept stable by neoliberal governments
> This will continue till we wisen up and elect the right people, select the right people.
This is part of the problem:
The problem is not necessarily democracy, but rather the insight the lack of bandwidth in voting ballots. A few bits every 4 years or whatever? Contrast this with the rate of modification in the law of a jurisdiction. Define the "democratic grade" as the ratio of binding democratic input in bits/per year to the changes in the binding law in bits/per year.
The problems aren't the concepts of democracy, but that humanity was sold a muzzle with a nameplate stating "democracy".
One may argue that it was necessary in the era where most democracies were initiated, until large enough fractions of the population become educated, but justified or not, its hard to defend today.
Every time you spread the message that we just need to "vote right" you too are in a novel way a part of the problem.
I think you will find little support for your suggestion that when you huurtehoog give up on something, the whole world should also give up on it, without rigorous proof of the proclaimed hopelessness.
Are there "right people"? Or does power corrupt, and your cozy wish is just not realistic? Looking at local politics, I see misuse of power and on all sides. Left, right, middle. Nobody can be trusted.
I don't think it is necessarily corruption. That is an easy accusation to make but what actually happens when someone joins a government? You learn a whole tonne of stuff you didn't know about what is broken and need fixing and you learn about how much money you don't have to spend but you don't want the country to fall apart in depression but telling everyone they are screwed so you try and deflect as much as possible.
The other issue is that the machinery is not 1 person but 10s or 100s of 1000s of people who are actually doing the work you want to do. There is a difference between "£10M will sort out that hospital easily" and going through the machine ends up making that cost more like £100M not just "corruption" although there will be some but also massive inefficiency, lack of agency for people to make things happen without having 100 meetings to agree it and then having the government change their mind and throw the whole thing in the trash.
Devolving decisions to local government helps to a point but some things have to be managed centrally, which is where the bureaucracy is at its worst.
I was just a while ago writing elsewhere (sorry, not totally on topic):
"But now almost everyone has figured out what the signs of the corruption are so we can spot the ones getting ill from the power and remove them.
I was joking that we (Finland) could sacrifice a island f Åland to be a Matrix for the power corrupted elite. We can simulate plenty of virtual sycophants fro them to be perfectly happy narcissists while isolated from doing us harm.
And I'm happy to volunteer to train as a therapist, should someone want to get better!
The golden rule and its formulations is pretty damn good litmus test for checking the level of corruption. If you're deep enough this will create a rage reaction. Think of Trump when someone talks about empathy or standard deviation. He will get hostile as he doesn't understand what the aliens around him are talking about.
I referred to the moral development with the lollipop and the levels are:
Pre-conventional (narcissist level in childhood or later in life regression by trauma, addiction or ...)
Conventional (the sheep level that understands moral based on what others do around - google banality of evil if you don't know this)
Post-conventional is the level where we do the golden rule thinking.
Most of us are in the conventional in action, BUT can understand the golden rule and that's enough. Have the decision makers reflect on some, even mundane, issues on the basis of "what if all 8 billion of us did this" and you'll get a short sircuit or division by zero, if the capability has regressed or never developed. These are no evil people by no means, but in most cases ones that have not had a healthy people around them. Either as a child or later on. We just know where we're going to end up should we not remove them from the power. Whether the society is powered by communism, or capitalism, or whateverism the outcome is the same if we don't figure out a way for this.
We cannot allow replacement via "good great leaders" as the power itself also creates the regression, just like addiction or trauma. So if I'm seemingly moral now and you'll select me as the great spagetti leader that dictates over the world and other puny religions, eventually I will corrupt by the admiration. And so will you.
The predicament of the human race is IMO decided by the fact that do we dare to start talking about this human trait honestly :D Sorry a long rant, now I'll shut up!"
So thanks for a question I had an answer almost available!
> I was joking that we (Finland) could sacrifice a island f Åland to be a Matrix for the power corrupted elite.
"Take all your overgrown infants away somewhere
And build them a home, a little place of their own
The Fletcher Memorial Home
For Incurable Tyrants and Kings"
...
"They can polish their medals and sharpen their smiles,
And abuse themselves playing games for a while
Boom boom, bang bang, lie down you're dead"
We do the opposite. The political class and the contractor class regularly flout safety norms and build where they shouldn't. They make the money from the safety of being out of range.
When the political class are questioned they utter inanities like people should be grateful that God Ganga maia waters have come to bless their lands and homes.
It is a racket. This will continue till we start electing people who respect the scientific methods and their conclusions.
Okay, but isn't it likely that the outcome of those scientific investigations will be that the Himalayan foothills should not be inhabited at all, except maybe a couple of places? What I mean is: Sure, politicians and contractors could do better, but I believe the primary reason for villages and bridges in those valleys is that people want them there. Standard of living is exceptionally low and this is just one of many hazards.
> Okay, but isn't it likely that the outcome of those scientific investigations will be that the Himalayan foothills should not be inhabited at all, except maybe a couple of places
Yes and that is broadly the safe option. One can build on geological stable high grounds. They are rather and more difficult to build on. We know what happens when we don't.
> Sure, politicians and contractors could do better
It's not mistake of the "oops a typo" variety.
They are knowingly making a freakton of money by endangering the people who ought to know better. I agree sometimes the poor might not have viable alternatives that they can fund with their own bank balance but it seems this is where the governments should step in to fund alternatives. But why do that when there is easy money to be made.
While I usually prefer to attribute incompetence rather than malice, it's hard to deny corruption is a likely factor here too. But the "freakton of money" aspect IMO needs nuance. I wager that the wealth of those cronies doesn't come close to my western living standards, so who am I to judge?
The question is why people build new on known dangerous places. The new is an important word. They do because the future is not priced in (make insurance mandatory?) and because of acts of corruption that actively promote/incentivize development in hazardous areas by rewriting zoning, repricing permits, rewriting saner policies.
Some people's job is building new houses to sell them. Your question is why they are allowed to build in some areas. Usually there is no point working unless it's for money so take your guess.
BTW, I can't build anything in half of my garden unless I also build huge concrete structures to anchor the terrain, which maybe would make matters worse. My house has been there for about 100 years, the other houses too, maybe it's stable enough but often they allow what's already there and deny anything new. Anyway, it's fine, I don't want to build something there.
> Your question is why they are allowed to build in some areas
No that is not my question at all. In fact I don't have a question, I have a position.
My position is that population density in many of these super dangerous areas is because of corruption laced government policies and practices. The government follows these policies in concert with builders because they can knowingly profit from endangering people (i) in dire straits, (ii) people who are not informed about magnitude of the risks.
In addition to that their corrupt practices distort the market prices so that people end up suckered. If these corrupt practices were to stop or reduce and better more people friendly policies were followed then these place will not become uninhabited but plausible less populated. If you look at my other comments on this story you will get more details of my position.
Some people are in such dire straits that they may still agree to stay in a hazard equivalent to living on Chernobyl lands. Some who are more fortunate would take other options should the government be doing their job.
And most of this "well obviously nobody should build anywhere where there will be serious destruction every couple generations" type sentiment ultimately just to funnels wealth and power to whoever took that good land previously.
If there is serious distruction every couple of generations it happens either to you or to your children. So, do you really want to build our buy there? If it happened to your parents there is no you.
But these people who want to make a buck building garbage somewhere it won't last are largely counterbalanced by the equal and opposite people who seek to financially benefit at society's expense by harnessing the screeching of the useful idiots to get sprinkler systems mandated in garden sheds and mail order carports require engineered plans.
They're basically two sides of the same "slimy people who'll lie for a buck" coin.
Maybe true in the first world. In 3rd world countries some times the information about the hazards is not properly appreciated.
For example, there were recent floods, is a location were previous governments had not developed. Now the current government rather than taking accountability are claiming that affected houses should be grateful that they have been touched by the river god.
Difficult to believe, but some people fall for the propaganda.
“the people”, those instinct-driven will-less victims of the “Epstein class”, right?
Why do they have to be “moved”? They are perfectly capable of moving themselves as I can attest to seeing them flooding into my country by the thousands. “The people” make their own choices.
HF is the default platform to find models. Not just ones published by big labs but also a lot of the distilled or fine-tuned versions, etc.
If Nvidia buying HF makes it tough for all the diverse models on HF, then what are some alternatives?
It seems models are the best things to be available on a Torrent platform? Of course HF is much more than just the files but perhaps the metadata can be separate and hosted on multiple community platforms.
I'd honestly prefer that. I'm usually on 5g if not 4g and pulling models is an absolute nightmare when I basically have to use HF and even with a token getting throttled. Although I have a pretty unique use case. Pulling wads of 5gb tensor files over cellular is painful.
I am on the same boat, I live in a small Himalayan village. I have 2x5G based devices and one Wireless bridge (Ubiquiti LiteBeam M5) for a local broadband. Generally I get 50 Mbps, sometimes up to 100 Mbps
i'm traveling in an RV and can always (80% of the time) get up to 3MBps, max to 300. But with cellular it's mega spotty so pulling 3-10 5GB files then losing your connection for a few seconds/minutes completely kills that files download. I've tried about 10 different keep-alive-ing type hf downloader clients and with my 64gb ram I'm usually downloading 25-32GB models and I don't even try unless I'm in sitting in a city next to a library anymore.
It's less of a pain now that I'm not in my LLM newbie phase of trying to test out 30 different models and stick with Qwen, but I couldn't even try to download qwen 3.8 for a week or two until I got back to the city.
Starlink is the obvious option and I have it on my roof but I do everything I can to not pay that $170 a month vs my $170/quarter 5g service.
I don't know anyone who "finds" models on HF. The announcements always come through Reddit, HN, Xitter, and various Discords. HF is just a place where some people bought a domain name, spun up a file server, and struck it rich. If they went away tomorrow, they'd be replaced the next day.
I am quite skeptical that an acquisition by Nvidia will make people move somewhere else ... The network effect remains central, if you are researcher/lab/company and you want to promote your model/dataset you go to the popular platform where most of the community is. The same way most of the public repo are still being published on GitHub despite people being pissed of by Microsoft acquisition. (I you add on top of that the cost changing your workflow, not using HF library, ... it make the transition even more difficult)
I already do this. I live in a small village where there isn't even a wired Internet connection (wireless only). I work full-time with LLMs, on own product ideas and client projects (all LLM led).
I started investing in farms, have 50 pigs and 100+ chickens now. We are planning to grow to 100 pigs and 2000 chickens in a year. We will start growing Shiitake mushrooms in a few months too.
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
reply