Actually still no

github.com/JustVugg/colibri

Everyone was desperate to be first because capitalism. But we are getting good models without the insane build out requirement. Which will be hilarious to leave the cunts holding the bag. Not that the planet is better for it in the end.

~1 token per second (storage bound gen4 nvme)… Some of us have places to be.

Don’t get me wrong. Its impressive that it can run at all, but honestly the usecase is exceedingly narrow. You’d have better results with a structured quantized gpu-only gemma or qwen workflow. Quality over quantity, rely on validation and a structured process: lots of cross-model review and iteration loops with spec and test driven dev. You could probably get a working alpha by the time colibri set up the environment.

Yeah I’m just beginning my local AI journey on a 5080, tried Qwen3.6 27b Q4 and was getting like 1tps because of the vram overflow. Ran it over night at it was still chewing on generating a prompt for a sub agent when I got up in the middle of the night until it simply ended in some kind of “fetch failure” lol. I think I gave it something too large to tackle, but either way 1tps is kinda garbage.

Is that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.

It’s the q4 quantization, but it requires 20+GB vram and my 5080 only has 16