← Back to post

Edit history

Most recent

+1 for Hermes.

If you have a newer Nvidia GPU, you can run Qwen 27B via exllamav3 and get good quality/speed in 16GB. And it’s worth the trouble, as 27B is an amazing model.

If it’s AMD, yeah, a3B is a good bet, depending on how much spare CPU RAM you have.

Original

+1 for Hermes.

If you have a newer Nvidia GPU, you can run Qwen 27B via exllamav3 and get good quality/speed in 16GB.

If it’s AMD, yeah, a3B is a good bet, depending on how much spare CPU RAM you have.