I'm using this as a frontend locally: https://docs.openwebui.com


itsrealfake

did you get any further?

I gotta say that I really like the frontend i mentioned.. I'm hosting it on digital ocean now, and pointing it at an LLM where I have tokens.

it's quite effective, though non-local... which gives me a certain kind of feel that I don't know how to feel about: shouldn't we all aspire to do this stuff locally? are we too early to realistically use it locally?



Are you using a local LLM?

Have a colleague that was showing me a proof of concept using some models to create a symatic search over some data... Kinda wild how simple it is to do. That's what surprised me most. The complexity is really in the models. Been dipping my toe into running stuff locally as well. It's interesting. 