I can definitely help assist you in this search, but importantly, in order to give you the right kind of answers, I'll need to know the following info:
  • You say a "home desktop computer", that's fine -- but I need more information, specifics on the hardware specs. What kind of a CPU do you have and what kind of a GPU does it have?
  • When you say you want it to be self-hosted on your home server, I'm assuming that you mean you want some kind of a web GUI to be able to access it remotely?
I'll also point out in advance that while you can do this, depending upon what kind of information you might be specifically desiring (i.e. do you want an LLM model that is more focused on helping with coding tasks and answers, or something that's more "generalized" in it's responses for typical chatbot conversations?), and more importantly, based on your expectations as far as performance is concerned, you may not want to do it.
While there are certainly LLM models out there which are both reasonable in model size for your downloading to set it up and with enough parameters to train the model (some of the LLAMA models are ideal in this respect), unfortunately the performance you're going to get running these on the average computer, like a gaming PC, is going to be pretty piss-poor. The response speed/output generation will be excruciatingly slow in most cases.
Thanks a lot for this answer. I have to go to sleep for now. I'll write specs tomorrow!
reply