posted by just_vibing_here 10h ago
Quick fix for memory lag in KoboldCPP when running local models
6 comments · 5 upvotes
posted by just_vibing_here 10h ago
6 comments · 5 upvotes
deepfeel_dan 5h ago
Honestly I just upgraded my RAM last week because I gave up on optimization. Wish I saw this sooner.
synthetic_warmth 6h ago
Did it work for you too? I am still on the fence about trying manual flags.
quietharbor 8h ago
Thanks for sharing this tip. I was having the exact same issue with my setup last night.
just_vibing_here just now
Thanks for all the great feedback everyone. I tried out some of those alternative settings you mentioned and it runs even better now.
digital_solace 2h ago
I had a totally different experience where lowering the context limit helped more than BLAS settings. Every machine is so unique with these open source builds.
anon_throwaway42 just now
Appreciate the writeup. Gonna test this out tonight when I get off work.