The problem is a lack of funding, which leads to excessive competition and ties continued employment to sustained contributions.
Many results are obvious in retrospect, and such results are often the best ones. The difficult part with such results is framing the problem in the right way and asking the right questions. If you manage to do that, the result simply follows. You may still need funding and hard work to confirm your finding, in which case someone with more resources can claim your result, if they are aware of the idea.
I think the GP was suggesting that their reluctance was more about someone taking their idea. I still think it might suggest a problem with academia, but the summary would be closer to
"I don't want to live in a world where someone else follows through with my ideas without giving me credit"
It's still a problem because a lot of academics aren't especially well equipped to follow through with their ideas, which can create information silos that lead to ideas never being implemented. Still, I don't know if this is the biggest fish to fry: you have other silos like IP law and NDAs etc.
This feels like an unwarranted strawman. There are plenty of reasons for researchers to share openly at times and plenty of times it makes sense to wait until the meal is ready to serve before publishing.
I think the sentiment is misplaced here (there is a legitimate concern for IP protection), but this is my absolute favorite line from Silicon Valley - small correction though: “… makes the world a better place better than we do”
That's mostly correct, but I believe the INT8 support on M5 changes the equation pretty drastically. I also believe that diffusion models generally output _better_ results if they have shaping for code (they often will miss delimiter matching inside of a canvas).
My immediate reaction to a "what is with this loud music, I can barely hear the actual audio" comment on YouTube was to assume the channel was afraid of being copystruck by the iceberg's lawyers.
$50B is essentially 100% of the annual NIH budget, which funds the vast majority of JUST life sciences basic research. So you may want to update your beliefs
e: oh and while we're at it, California spent over $24 billion over a five-year period (2019–2024) specifically targeting homelessness
This over-reaction is why people can't see over a long time horizon.
This is great news for Google as they realize that Sundar is the problem and he will soon leave Google for Demis to be the new CEO of Alphabet (Google) which I am predicting. [0]
AI is critically important to Google, but there's a lot more to Google than just having a frontier AI model. Do Demis skills line up with what the whole company needs? It's going to be tough to beat Sundar's 1200% increase in stock price.
I sold out of my position. I can imagine a story where it works out in the long term, but I don't see how this doesn't cause terrible retention problems in the short to medium term. I felt a pull to launch a startup when I heard Jeff Dean was leaving, and I'm a long time big corp employee who hasn't been at Google in over a decade.
TBH, I think there's some truth to that. I spent _ages_ tuning the kernels to match the tested FLOP count of my M3's processor. I only have an M3 though and wasn't able to push int8 very far on it, but I think there's a chance that M5-class machines and higher might have more capability in this regard.
What I also learned is that MLX/vLLM is probably within ~20% or so of the absolute max perf on Mac. I found some improvements over what they were doing, but we're at the point where it's challenging to optimize without per-stepping kernels.
I found a few improvements over stock DiffusionGemma along the way, like using top-k attention, which drastically improves perf on my mac without sacrificing any of the benchmarks I was able to throw at it.
FWIW some of the issues with Gemma being slow on Mac are specific choices they've made in the architecture that make it challenging to make use various optimizations that have popped up recently. I think a Kimi K3-style network hybrid with the diffusion bits of DiffusionGemma could have some serious sway.
I think that diffusion still has an edge locally, but with some architecture tweaks and CPU improvements it would actually be a winner (ie: training the network for smaller token batch sizes or flexibility in attention heads, a less expensive attention mechanism, and others).
"I don't want to live in a world where someone else makes the world a better place than we do."
reply