I've been looking around for smaller models. These old laptops I have are good enough for running a bitcoin node but it seems like they are simply too slow for running LLM's.
If you go on HF and make a profile, you can add your hardware. Then whenever you're on a model page, it will tell you which quant is doable on your hardware.
Thanks!
I've been looking around for smaller models. These old laptops I have are good enough for running a bitcoin node but it seems like they are simply too slow for running LLM's.
I feel like a lot of AI hype is threatening open source.
Linux thrived because you didn't need permission to use it.
We need to promote the use and development of local models.
If you go on HF and make a profile, you can add your hardware. Then whenever you're on a model page, it will tell you which quant is doable on your hardware.
I like the idea that the bubble will burst and we will be flooded with cheap hardware.