I wonder if this will incentivize pushing compute back toward the user instead of in big centralized facilities, sonething like a repeat of the PC revolution.
Or will it just incentivize harsher treatment of noncompliant parts of the population?
Or just pushing these things far from urban areas where they probably should have been already?
Probably a bit of each, and some more I'm not thinking of.
> the user instead of in big centralized facilities
Although that would be exponentially worse in terms of electricity and resource use, etc. - instead 1 TPU handling x,000 customers per month, now you need to build and power n,000 separate devices, where the vast majority of them are idle the vast majority of the time, and even then without all of the more efficient power, batching, etc that DCs offer.
> urban areas
The photo at the very top of the OP article shows a very low-density area. If someone was expecting to see urban-type density, it isn’t in the location shown in the photo.
That's working under the (imo wrong) assumption that LLMs are what we need/want. First, because pushing toward the user can happen as shared hosting in your municipality and doesn't have to be in every home. Second, because if you take slop, adtech, and surveillance out of the equation, the material impact is several orders of magnitude less than what's being built today. Or for what it's worth, several orders of magnitude less than what's already built in terms of DC capacity.
1. Every single municipality would have AI data center "hosting in your municipality".
2. You'd have to majorly overbuild every municipality's capacity, as usage would likely have peak hours and 'dead' hours each day based on work, sleep, etc. if usage is local to that municipality.
> several orders of magnitude less than what's already built in terms of DC capacity.
Are you saying you believe already-built data centers are 100 times (2 orders of magnitude) as much as needed? I presume this is not what you think will actually happen in the real world though?
1. In the scenario i'm mentioning, there's no LLMs being deployed at all for end users. Social networks, netflix-style video and email fits in a single rack for thousands of people (tens of thousands?).
2. Yes. That's already the case. Today a single smartphone has enough CPU/RAM to host email services for many thousands of users. A single Netflix OCA 2U serves 400Gbit/s of video: https://news.ycombinator.com/item?id=28584738
The only reason we had huge DCs before LLMs was so Google/Facebook could spread their surveillance/adtech empire. Nothing of that scale is needed to provide useful services to end users.
Or will it just incentivize harsher treatment of noncompliant parts of the population?
Or just pushing these things far from urban areas where they probably should have been already?
Probably a bit of each, and some more I'm not thinking of.