Running Llama2 Models Locally
I get approval from meta, then I downloaded all Llama2 models (7B,7B-chat,13B,13B-chat,70B,70B-chat) locally, when I tried to run the script to test the 7B model using the command that is mentioned on Llama github "torchrun --nproc_per_node 1 example_text_completion.py \
--ckpt_dir llama-2-7b/ \
--tokenizer_path tokenizer.model \
--max_seq_len 128 --max_batch_size 4" I got this error "Distributed package doesn't have NCCL built in". My environment : Windows 10, Nvidia GeForce RTX 3090, CUDA 11.8, torch 2.0.1+cu118. I have free enough space and I have everything but I don't know why it's not working. Any help please. I'm blocked in this task for 3 days.
2 comments
[ 4.1 ms ] story [ 17.7 ms ] threadI found this: https://discuss.pytorch.org/t/runtimeerror-distributed-packa...