Local Code Chatbot Running on 2GB RAM
by brucethemoose2 on 6/28/2023, 12:01:34 AM
There is even some untapped headroom, as they quantized to Q4 instead of using K-quant.
Not to speak of the potential hooked up to a vector db, swapping out LORAs for different languages and such.
There is even some untapped headroom, as they quantized to Q4 instead of using K-quant.
Not to speak of the potential hooked up to a vector db, swapping out LORAs for different languages and such.