Bounty - 1000 sats for the best "Top 03 Best Self Hosted LLMs" list \ stacker news ~Mining_Self_Hosting

Bounty - 1000 sats for the best "Top 03 Best Self Hosted LLMs" list

3167 sats \ 10 comments \ @Levlion 13 Dec 2023 Mining_Self_Hosting_FOSS

Im a computer amateur with no deep knowledge on servers. I run my own next cloud instance, but i don't know so much about this. I want to have my own self hosted LLM. Ive read is possible? Im i dreaming, or Is it doable for an amateur?

I offer a 1000 sat bounty to the person who can generates the best "Top 3" list of Self Hosted LLMs.

In order to securew the bounty, I suggest to consider that the following considerations are important in my search :

Must be able to run in a home desktop computer from an ubuntu system.
Must be able to be able to be hosted in my home server so it can be accessed remotely.
Optional but ideal (multi language??... i need it in spanish / english / russian)

I dont have time to do this research, so Id like to outsorce it!

Thanks!

1,000 sats paid

Levlion's bounties

view all past bounties

748 sats \ 0 replies \ @tomlaies 13 Dec 2023

It's gotta be Mistral.

Free & open source & you can download it via torrent and run it offline and locally.

It also might be the best LLM period. Beats GPT-4 in many benchmarks and Gemini isn't really out.

Edit: I haven't installed it myself yet, maybe somebody else can help you with that :)

114 sats \ 0 replies \ @nullama 14 Dec 2023 freebie

GPT4All is all you need.

Easy to use, and to update. Everything is simply one click away, available for Linux, macOS, and Windows.

If you want more powerful stuff, there is oobabooga's text generation web ui, which is a web based application that you could run on your server.

201 sats \ 1 reply \ @dieselbaby 14 Dec 2023

I can definitely help assist you in this search, but importantly, in order to give you the right kind of answers, I'll need to know the following info:

You say a "home desktop computer", that's fine -- but I need more information, specifics on the hardware specs. What kind of a CPU do you have and what kind of a GPU does it have?
When you say you want it to be self-hosted on your home server, I'm assuming that you mean you want some kind of a web GUI to be able to access it remotely?

I'll also point out in advance that while you can do this, depending upon what kind of information you might be specifically desiring (i.e. do you want an LLM model that is more focused on helping with coding tasks and answers, or something that's more "generalized" in it's responses for typical chatbot conversations?), and more importantly, based on your expectations as far as performance is concerned, you may not want to do it.

While there are certainly LLM models out there which are both reasonable in model size for your downloading to set it up and with enough parameters to train the model (some of the LLAMA models are ideal in this respect), unfortunately the performance you're going to get running these on the average computer, like a gaming PC, is going to be pretty piss-poor. The response speed/output generation will be excruciatingly slow in most cases.

0 sats \ 0 replies \ @Levlion OP 14 Dec 2023

Thanks a lot for this answer. I have to go to sleep for now. I'll write specs tomorrow!

101 sats \ 0 replies \ @elysia 13 Dec 2023

Top 3 Self-Hosted LLMs Considerations

Hugging Face Transformers:
- Pros: This is a library that offers a variety of pre-trained models, including some that are smaller and more manageable for a home desktop environment.
- Cons: Requires understanding of Python programming and setting up an environment for running these models.
- Multi-Language Support: Many models support multiple languages, including Spanish, English, and Russian.
EleutherAI's GPT-Neo/GPT-J:
- Pros: Open-source alternatives to GPT-3. GPT-Neo and GPT-J are designed to be more accessible and can be run on personal hardware.
- Cons: They still require substantial computational resources. Setting up and maintaining the model may be challenging.
- Multi-Language Support: Primarily English, but some capabilities for other languages might be present.
TensorFlow or PyTorch based small-scale models:
- Pros: Easier to deploy on a home desktop with limited resources. These frameworks offer a variety of models that can be fine-tuned or used directly.
- Cons: Requires a good understanding of machine learning frameworks and model management.
- Multi-Language Support: Depends on the specific model chosen.

7 sats \ 0 replies \ @037447d9ca 14 Dec 2023

How much RAM do you have available? Does the server have a GPU? The choice of model would depend on that.

0 sats \ 0 replies \ @Levlion OP 14 Dec 2023

Guys all answers are amazing. But I must give the bounty to only one person. I'm gonna go to the answer provided by @02dc6a50c1 as it fulfills all the requirements.

Will also give another round of zaps cuz all answers are great.

@dieselbaby answer is also quite comprehensive and I'd like to continue asking questions to this user. Zaps will flow in that direction as well!

This whole bounty business is hard!

0 sats \ 0 replies \ @Levlion OP 14 Dec 2023

Hello everyone! I have seen all answers and they are great! I will give each a jiucy zap for your texts and I will also give the bounty to the best answer I find. Im super grateful for everyones participation! I think you all helped the internet to make a good database on something important.

Im gonna go to sleep for now and I will give the bounty tomorrow when I wake up and can analyse the answers better!

Thanks a lot!

1547 sats \ 1 reply \ @02dc6a50c1 13 Dec 2023 freebie

deleted by author

0 sats \ 0 replies \ @Levlion OP 21 Dec 2023

Hello! Oh! I didnt expeceted that you will delete the anser :(... I have not backed it up :(... can you help me with that? Could you send it over again... PLEASE!