minus-squareProperlyProperTea@lemmy.mltoSelfhosted@lemmy.world•Is anyone self-hosting their own AI?linkfedilinkEnglisharrow-up4arrow-down1·7 days agoLast night I managed to run Qwen Flash Next iq2 XS using Strata in docker on a VM I passed through my 3090, 64GB of DDR4, 12 vcores of my 5960x, Managed 90t/s I’ll probably try a larger version soon but QFN already seems a lot more capable than 3.6 35b A3B, and runs better than Qwen 3.8 dense in llama.cpp linkfedilink
Last night I managed to run Qwen Flash Next iq2 XS using Strata in docker on a VM
I passed through my 3090, 64GB of DDR4, 12 vcores of my 5960x,
Managed 90t/s
I’ll probably try a larger version soon but QFN already seems a lot more capable than 3.6 35b A3B, and runs better than Qwen 3.8 dense in llama.cpp