Edit history

Deleted

It’s one of those “if you have to ask, you can’t afford it” scenarios.

I’ve heard FP16 is likely to be 6TB of VRAM, when Kvcache is factored in.

Even a more sensible quant will likely be 1TB.

… most of us are not running this at home.