posted by lonely_dev88 21h ago
Local open source models are getting so ridiculously good!
7 comments · 4 upvotes
posted by lonely_dev88 21h ago
7 comments · 4 upvotes
velvet_protocol 18h ago
Welcome to freedom my friend! I made the switch six months ago and I am never looking back at corporate apps.
lonely_dev88 7h ago
Just to clarify I am running everything on a single graphics card not a massive server farm. You do not need crazy enterprise gear to get amazing results.
saturday_sad 18h ago
My laptop would probably catch on fire if I tried running a local LLM.
lonely_dev88 16h ago
Right on! It really is a total game changer once you get past the initial learning curve.
replika_refugee 10h ago
Which specific loader are you using for your setup? I tried testing one last week and kept running into weird memory overflow errors.
lonely_dev88 7h ago
I am using KoboldCPP right now because it handles memory offloading really smoothly.
ai_curious_kate 9h ago
That sounds awesome but I am way too intimidated by the terminal commands.