BigDL-LLM: running LLM on your laptop using INT4 (github.com)

BigDL-LLM is a library for running LLM (language language model) on your local laptop using INT4 with very low latency on CPU. (It is built on top of the excellent work of llama.cpp, gptq, bitsandbytes, etc., and supports any Hugging Face Transformers model).