Local LLM by Ente

Summary

LLMs are too important to be left to big tech. There is a gap between frontier models and models that can run on your device, but local models improve each day, and once they cross a certain capability threshold, they will be good enough for most purposes; and will come with full privacy and control.Based on these assumptions, we've been working on Ensu, Ente's offline LLM app. Today is our first release.Download it here.In the rest of this post, we'll explain why we think the assumptions hold, what we're doing, and how you can get involved.WhyLLMs are too important to be left to big tech. We've written in depth about this earlier, here and here.Briefly, those posts come at it from two angles:If you're someone who hates LLMs, you would still be able to recognize in clearer moments of thought that LLMs are a technology that can't just be wished away.If you're someone who finds joy in interacting with LLMs, you would recognize the lack of privacy and the high dependency (arbitrary bans, content shaping, non-portable memory) you have on centralized providers.And in both cases it is also clear that LLMs can be used to manipulate people en masse. Ergo, we can't be at the mercy of big tech controlling them.The issue is that there is a capability gap between large centralized models and smaller models that can be run offline on your device.But we're problem solvers, and this is not our first rodeo. When we first started Ente Photos, it seemed unthinkable that we'd be able to deliver face recognition, person clustering and natural language image search all running locally on your device. People called us crazy.It took many years, but we did it. Our users enjoy these features every day. Everything is done locally on device, and also synced, end-to-end encrypted, across all your devices. Full privacy, full control, without loss of convenience; technology in service of people, not as a tool of surveillance.In the same vein, while we have been itching for a long time to do some...