Plain C/C++ implementation (of inference of LLaMA model) without
dependencies. AVX, AVX2 and AVX512 support for x86 architectures.
Mixed F16/F32 precision. 4-bit, 5-bit and 8-bit integer quantization
support. Runs on the CPU. Supported models:

   LLaMA
   LLaMA 2
   Alpaca
   GPT4All
   Chinese LLaMA / Alpaca
   Vigogne (French)
   Vicuna
   Koala
   OpenBuddy (Multilingual)
   Pygmalion 7B / Metharme 7B
   WizardLM
   Baichuan-7B and its derivations (such as baichuan-7b-sft)

NOTE 1: You will need to:

 pip3 install -r /usr/share/llama.cpp/requirements.txt

for data format conversion scripts to work.

NOTE 2:
 MODELS ARE NOT PROVIDED. You need to download them from original
 sites and place them into "./models" directory.

 For example, LLaMA downloaded via public torrent link is 220 GB.

Overall this is all raw and experimental, no warranty, no support.

• Vitaly Chikunov