Aren’t diffusion models and LLMs (ollama) too big for an external NPU? As far as I know something like a Coral runs specific models only. And it’s limited to the 1 or maybe 2GB of memory on it. It’d do tasks like voice recognition, or image classification. But not generate images or text.
If you want to run arbitrary AI models and generative AI, I think you should be looking for a graphics card?!
Aren’t diffusion models and LLMs (ollama) too big for an external NPU? As far as I know something like a Coral runs specific models only. And it’s limited to the 1 or maybe 2GB of memory on it. It’d do tasks like voice recognition, or image classification. But not generate images or text.
If you want to run arbitrary AI models and generative AI, I think you should be looking for a graphics card?!
(Edit: From what I know NPUs are very good at some very specific tasks. Like you have a 24/7 video stream to process and you want to get rid of that constant 30% CPU load and do it a bit more efficiently and save on power. Or you do speech recognition / noise detection and that needs to be at a low latency and not waste too much electricity. That’s a good use-case for buying one of those NPU cards/sticks. For general experimentation, you’re probably better off doing it on CPU (can’t be large models anyway) or GPU.)
Aren’t diffusion models and LLMs (ollama) too big for an external NPU? As far as I know something like a Coral runs specific models only. And it’s limited to the 1 or maybe 2GB of memory on it. It’d do tasks like voice recognition, or image classification. But not generate images or text.
If you want to run arbitrary AI models and generative AI, I think you should be looking for a graphics card?!
Aren’t diffusion models and LLMs (ollama) too big for an external NPU? As far as I know something like a Coral runs specific models only. And it’s limited to the 1 or maybe 2GB of memory on it. It’d do tasks like voice recognition, or image classification. But not generate images or text.
Aren’t diffusion models and LLMs (ollama) too big for a NPU? As far as I know something like a Coral runs specific models only. And it’s limited to the 1 or 2GB of memory on it. It’d do tasks like voice recognition, or image classification. But not generate images or text.