[–] brucethemoose2 3y ago ↗ Use instruct mode, with the instruction in the memory window, with the formatting discussed here:https://github.com/ggerganov/llama.cpp/issues/2262#issuecomm...If you are feeling generous, host a V2 worker using KoboldCPP. Its pretty easy to set up, and 13B/70B will run on PCs with consumer GPUs:https://github.com/db0/KoboldAI-Horde-Bridge
1 comment
[ 2.4 ms ] story [ 7.7 ms ] threadhttps://github.com/ggerganov/llama.cpp/issues/2262#issuecomm...
If you are feeling generous, host a V2 worker using KoboldCPP. Its pretty easy to set up, and 13B/70B will run on PCs with consumer GPUs:
https://github.com/db0/KoboldAI-Horde-Bridge