Replying to @⁨eicker@lemmy.world⁩

A bit of both though. In house models don’t need the power and process of a small city, they need the power to power what their business needs. Also it would seem to me that has a huge advantage on the whole as a model would be more efficient, if it’s focused on the companies needs rather than being a jack of all trades, from poetry to law to code. Cut out the being everything to everyone and you can do far more with far less hardware.

Replying to @⁨TheFogan@programming.dev⁩

Exactly. At work, we’re already looking at getting our own compute for an open model for some automatons we built. Google’s models are alright, but they are retiring them too fast. So we want to get off that treadmill. Plus a local model gives us opportunities to experiment with LoRAs and techniques like what cactus hybrid did with a small head predicting certainty (news.ycombinator.com/item?id=49010782). Plus, the big players are removing sampling options thinking they know what’s best when it is obvious they don’t.