Let me clarify. Mixtral-8x22B-v0.1 was released a couple days ago. The "mixtral:8x22b" tag on ollama currently refers to it, so it's what you got when you did "ollama run mixtral:8x22b". It's a base model only capable…
Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.
That's not the model this post is about. You used the base model, not trained for tasks. (The instruct model is probably not on ollama yet.)
Mistral 7B Instruct v0.2 and Mistral 7B v0.2 are different models. Judging by the title, I suspect OP meant to post about the latter, which was released a few days ago, but accidentally linked to the former instead.
A rocket on a typical orbital launch profile spends less than 60 seconds in air dense enough for jet engines to have good performance, so there is little to gain. Pegasus is an orbital rocket launched from an aircraft,…
An air-breathing jet engine doesn't need to carry oxidizer, which in a rocket is most of the propellant weight. It also has access to unlimited reaction mass, so it can be much more energy-efficient in producing thrust…
You can partially offload with some backends (e.g. llama.cpp and derivatives) but speed gains from that don't come in until it's mostly offloaded. I have 8GB VRAM and it's not enough to get any boost on mixtral in Q8.…
A reactor that has never been turned on isn't a significant radiation hazard. It's the fission products that are hazardous, not the fuel, if it's never gone critical there are no fission products yet.
The bazarek is fun but in reality even less relevant that this post makes it out to be. Since people with real information cannot prove it and it takes zero effort to post fakes the bazarekposts are not any more…
Preparations for pad repairs and upgrades were well underway before the first flight - the question was not whether they'd be necessary, but how much and how soon. In particular if I remember correctly manufacturing of…
Anything with 64GB of memory will run a quantized 70B model. What else you need depends on what is acceptable speed for you. With a decent CPU but without any GPU assistance, expect output on the order of 1 token per…
The simulation is just so fake, almost everything that goes on is just decorative. There is a budget, but after the first 30 minutes you'll always be running an enormous surplus without trying. Citizen commute to work,…
Yes, many, huggingface is full of chat-tuned LLaMA derivatives that are supposed to replicate its performance, and tools like text-generation-webui or kobold.cpp can be used to run them with chat-style UX. But for most…
Experimental Falcon inference via ggml (so on CPU): https://github.com/cmp-nct/ggllm.cpp It has problems but it does work
ggml is a library that provides operations for running machine learning models llama.cpp is a project that uses ggml to run LLaMA, a large language model (like GPT) by Meta whisper.cpp is a project that uses ggml to run…
It doesn't matter very much that the "official" instruct tune is censored as anyone can create their own and there will probably be many freely available ones as happened with LLaMA. There is one already:…
You have to turn down temperature and/or p when you want accuracy. Otherwise you don't know if the model's read is bad, or if you just happened to get a low-probability outlier. With the playground's defaults of t=0.8,…
You can just barely fit a 33B GPTQ model in 24GB VRAM. It will be in 4-bit mode, and without maximum context size, but it will be quite fast. Or you can run from RAM+VRAM in GGML format with llama.cpp (or a derivative),…
Quantization isn't (and wasn't) expensive, it's mostly just data shuffling. A good PC will do a 7B model in half a minute, up to a few minutes for a larger model. Quantized models being made available for download is…
Almost every UI for LLMs I've seen has a way to specify an initial prompt that never goes out of context, it's strange that it's not a feature in ChatGPT.
I definitely noticed a drop in quality when the gimped (but presumably dramatically cheaper to run) GPT-3.5-turbo model was introduced on the free version. As a paying subscriber I think you should still have access to…
The charges worked fine. They blew holes in the tanks as planned. The issue is that this didn't cause immediate structural failure as intended. But structural failure of the rocket while flying through upper atmosphere…
There is nothing out there that quite matches ChatGPT quality but you can get a similar kind of experience by running an instruction-tuned derivative of LLaMA with llama.cpp. Try something like vicuna-13b-free or…
I suspect you could train a model to just shut up and follow instructions. I.e. instead of "Do X -> Sure! As a large language model, I'd love to help you with X!", just "Do X -> X". This would avoid giving the model a…
I really want a high powered instruction-following LLM that's capable of assisting with writing fiction. GPT-3.5-turbo is fantastically capable at this if you can get it to drop its stilted AI-assistant voice, but doing…
Let me clarify. Mixtral-8x22B-v0.1 was released a couple days ago. The "mixtral:8x22b" tag on ollama currently refers to it, so it's what you got when you did "ollama run mixtral:8x22b". It's a base model only capable…
Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.
That's not the model this post is about. You used the base model, not trained for tasks. (The instruct model is probably not on ollama yet.)
Mistral 7B Instruct v0.2 and Mistral 7B v0.2 are different models. Judging by the title, I suspect OP meant to post about the latter, which was released a few days ago, but accidentally linked to the former instead.
A rocket on a typical orbital launch profile spends less than 60 seconds in air dense enough for jet engines to have good performance, so there is little to gain. Pegasus is an orbital rocket launched from an aircraft,…
An air-breathing jet engine doesn't need to carry oxidizer, which in a rocket is most of the propellant weight. It also has access to unlimited reaction mass, so it can be much more energy-efficient in producing thrust…
You can partially offload with some backends (e.g. llama.cpp and derivatives) but speed gains from that don't come in until it's mostly offloaded. I have 8GB VRAM and it's not enough to get any boost on mixtral in Q8.…
A reactor that has never been turned on isn't a significant radiation hazard. It's the fission products that are hazardous, not the fuel, if it's never gone critical there are no fission products yet.
The bazarek is fun but in reality even less relevant that this post makes it out to be. Since people with real information cannot prove it and it takes zero effort to post fakes the bazarekposts are not any more…
Preparations for pad repairs and upgrades were well underway before the first flight - the question was not whether they'd be necessary, but how much and how soon. In particular if I remember correctly manufacturing of…
Anything with 64GB of memory will run a quantized 70B model. What else you need depends on what is acceptable speed for you. With a decent CPU but without any GPU assistance, expect output on the order of 1 token per…
The simulation is just so fake, almost everything that goes on is just decorative. There is a budget, but after the first 30 minutes you'll always be running an enormous surplus without trying. Citizen commute to work,…
Yes, many, huggingface is full of chat-tuned LLaMA derivatives that are supposed to replicate its performance, and tools like text-generation-webui or kobold.cpp can be used to run them with chat-style UX. But for most…
Experimental Falcon inference via ggml (so on CPU): https://github.com/cmp-nct/ggllm.cpp It has problems but it does work
ggml is a library that provides operations for running machine learning models llama.cpp is a project that uses ggml to run LLaMA, a large language model (like GPT) by Meta whisper.cpp is a project that uses ggml to run…
It doesn't matter very much that the "official" instruct tune is censored as anyone can create their own and there will probably be many freely available ones as happened with LLaMA. There is one already:…
You have to turn down temperature and/or p when you want accuracy. Otherwise you don't know if the model's read is bad, or if you just happened to get a low-probability outlier. With the playground's defaults of t=0.8,…
You can just barely fit a 33B GPTQ model in 24GB VRAM. It will be in 4-bit mode, and without maximum context size, but it will be quite fast. Or you can run from RAM+VRAM in GGML format with llama.cpp (or a derivative),…
Quantization isn't (and wasn't) expensive, it's mostly just data shuffling. A good PC will do a 7B model in half a minute, up to a few minutes for a larger model. Quantized models being made available for download is…
Almost every UI for LLMs I've seen has a way to specify an initial prompt that never goes out of context, it's strange that it's not a feature in ChatGPT.
I definitely noticed a drop in quality when the gimped (but presumably dramatically cheaper to run) GPT-3.5-turbo model was introduced on the free version. As a paying subscriber I think you should still have access to…
The charges worked fine. They blew holes in the tanks as planned. The issue is that this didn't cause immediate structural failure as intended. But structural failure of the rocket while flying through upper atmosphere…
There is nothing out there that quite matches ChatGPT quality but you can get a similar kind of experience by running an instruction-tuned derivative of LLaMA with llama.cpp. Try something like vicuna-13b-free or…
I suspect you could train a model to just shut up and follow instructions. I.e. instead of "Do X -> Sure! As a large language model, I'd love to help you with X!", just "Do X -> X". This would avoid giving the model a…
I really want a high powered instruction-following LLM that's capable of assisting with writing fiction. GPT-3.5-turbo is fantastically capable at this if you can get it to drop its stilted AI-assistant voice, but doing…