You can also use the "Oogabooga" client, which supports a bunch of different models including Llama and Alpaca. If you use 8, 4 or 2 bit quantized models you can fit even the largest 65b model in 64GB RAM.
You can also use the "Oogabooga" client, which supports a bunch of different models including Llama and Alpaca. If you use 8, 4 or 2 bit quantized models you can fit even the largest 65b model in 64GB RAM.
(post is archived)