Nothing with “AI” is awesome.
you people would have opposed the train rail back in the day because it was against god’s will.
1800s meme
You oppose the train rail because it’s against God’s will. I oppose the train rail because they were owned by tycoons and built with slave labor. We are not the same.
I was not expecting AI to be a catalyst to reinvigorate the game modding landscape, but AI tech seems to be filled with unintended consequences. Hopefully these types of processes can be achieved with local hardware running open models in the near future. Micro-scale community compute could completely facilitate a departure from the tech behemoths’ walled gardens.
Qwen 3.8 is actually fairly capable already. You do need a beefy machine to run it, but it’s very much in consumer grade at this point. My expectation is that within a few years we’ll be able to buy something like this, hopefully at a cost of a raspberryPi to run it https://wccftech.com/alibabas-tsmc-built-5nm-risc-v-chip-xuantie-c950-now-runs-qwen-3-8-27b-model-natively-unlocking-massive-vertical-integration-tailwinds/
This is really damn cool chip design. The liberation of technology for the proletariat by China really just keeps rolling along. When the AI crash hits the financial markets, China will be ready with commodity solutions.
Exactly, local models are obviously the future, and China is building it.
I’d say self hosted models are the future, not local ones. As far as I’m aware, hardware requirements haven’t been going down, and hardware keeps getting more expensive, meaning you’re probably better off renting compute from a data center than you are shelling out tens of thousands for AI-capable hardware.
Or maybe I’m just not up to date on stuff. I only have a distilled deepseek R1 model that I barely ever touch, running on a 4060ti 16gb. I find single-purpose ML models a lot more useful and cost-efficient for homelab stuff than a general purpose LLM.
You can run Qwen 3.8 27b with 16-64gb vram depending on the quant. The 8bit version that would need 64gb is comparable to Opus 4.6 from less than a year ago. While 64gb vram is still expensive, but definitely in consumer grade already. Meanwhile, Alibaba is already working on a dedicated chip for running Qwen that doesn’t even need a ton of vram.
There are also interesting efforts like DwarfStar where they quantized parts of the model, so the reasoning layers stay at high qunats but the routing ones drop to low. And this lets you run a really large model locally.
And I do expect there will likely be more push towards making models single-purpose too. For example, you probably can train one to just work with a specific programming language really well. The holy grain would be to extract the reasoning parts of the model, and then build the context around that at runtime. Then you could have a fairly small model that could be adopted to a wide range of problems as needed.
