I was trying to distance myself from this situation, but this is just too painful to read. I am sincerely sorry that people have harassed you on my behalf, but I have no control over what some people say or do on an…
So because ChatGPT exists now, less experienced programmers will be hired to developed critical software under the assumption that they can use ChatGPT to fill the gaps in their knowledge? Even in that case, I would…
What is the issue with the GDPR? If OpenAI is violating the GDPR, why should European governments ignore it?
I would trust ChatGPT code about as much as I trust the code produced by any human. All the Therac-25 code was written by a human, so what is the argument here exactly? At least when you tell ChatGPT that its code is…
Price controls and rationing go hand in hand. Ultimately the goal is to ensure that everybody is able to acquire the goods that they need to survive. We are not talking about luxury yachts here, we are talking about…
I know this is probably going to give a heart attack to some of the libertarian tech-bros here, but there is an easy and tested solution for that: price controls. Inflation has a very clear source, it happens when…
I used it for a while but I found that too many suggestions are worthless and having to consider them makes me waste more time than just writing the code myself. For the things that it is useful like snippets, I found…
Most likely yes. I bought a new 980 Pro 2TB a few weeks ago and it came with the latest firmware. Manufacture date is Jan 2023. The firmware is already more than a year old so unless you get very old stock it should…
Could you share what you did to convert the models? The mirrors aren't very reliable and it would be good to have it documented somewhere. There is some discussion about this at…
The llama models were trained with a context size of 2048. By default llama.cpp limits it to 512, but you can use -c 2048 -n 2048 to get the full context window.
llama.cpp is using RTN at the moment.
The python implementation[1] ran some tests using the same quantization algorithm as llama.cpp (4 bit RTN). 1: https://github.com/qwopqwop200/GPTQ-for-LLaMa
What are you comparing it to? Without instruction tuning and a two character prompt "He" I am not sure why you would expect it to perform any better.
The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.
They have recently merged support for x86. I get 230ms/token on the 13B model on a 8 core 9900k under WSL2.
Of course, in practice that is not a possibility for most people. You can't just choose to work hourly if nobody is willing to hire you on these terms. Without regulation the balance of power is strongly in the hands of…
Is there anything about sitting in a line manufacturing electronics that requires 12 hours shifts?
I see many comments here every day that would get anyone instantly banned from most of the biggest subreddits. However, I think that most people would agree that this place is a lot less toxic than reddit. Smaller…
That's not my experience, but maybe it is my adblocker hiding the Chrome ads. You can't even use the chat feature in bing with firefox without spoofing the user agent.
Wasn't expecting to see Adam Ragusea here. He has a popular youtube cooking channel nowadays.
I have noticed this too while giving the new bing a try the last few days. It doesn't matter that you have an AI bot summarizing and answering questions about the search results if your search results are worthless…
I wonder how this is implemented in the GPU. From my time working on a 3D renderer a long time ago, triangles with offscreen vertices would be clipped into smaller triangles, so in the end you would still be rendering…
Isn't this problem solved with a chiplet design like AMD does? Just manufacture the SRAM using an older and cheaper node and keep the logic in the most advanced node.
Seeing the performance of implementations like FlexGen [1], I don't think it would be entirely unreasonable to run a 13B model on a single GPU for personal usage purposes. You are not going to a run a public service out…
I was mostly talking about access to the trained model weights. The OpenAI API is certainly better than nothing, but it is very restrictive and cost prohibitive for many purposes. For instance, you have to adhere to the…
I was trying to distance myself from this situation, but this is just too painful to read. I am sincerely sorry that people have harassed you on my behalf, but I have no control over what some people say or do on an…
So because ChatGPT exists now, less experienced programmers will be hired to developed critical software under the assumption that they can use ChatGPT to fill the gaps in their knowledge? Even in that case, I would…
What is the issue with the GDPR? If OpenAI is violating the GDPR, why should European governments ignore it?
I would trust ChatGPT code about as much as I trust the code produced by any human. All the Therac-25 code was written by a human, so what is the argument here exactly? At least when you tell ChatGPT that its code is…
Price controls and rationing go hand in hand. Ultimately the goal is to ensure that everybody is able to acquire the goods that they need to survive. We are not talking about luxury yachts here, we are talking about…
I know this is probably going to give a heart attack to some of the libertarian tech-bros here, but there is an easy and tested solution for that: price controls. Inflation has a very clear source, it happens when…
I used it for a while but I found that too many suggestions are worthless and having to consider them makes me waste more time than just writing the code myself. For the things that it is useful like snippets, I found…
Most likely yes. I bought a new 980 Pro 2TB a few weeks ago and it came with the latest firmware. Manufacture date is Jan 2023. The firmware is already more than a year old so unless you get very old stock it should…
Could you share what you did to convert the models? The mirrors aren't very reliable and it would be good to have it documented somewhere. There is some discussion about this at…
The llama models were trained with a context size of 2048. By default llama.cpp limits it to 512, but you can use -c 2048 -n 2048 to get the full context window.
llama.cpp is using RTN at the moment.
The python implementation[1] ran some tests using the same quantization algorithm as llama.cpp (4 bit RTN). 1: https://github.com/qwopqwop200/GPTQ-for-LLaMa
What are you comparing it to? Without instruction tuning and a two character prompt "He" I am not sure why you would expect it to perform any better.
The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.
They have recently merged support for x86. I get 230ms/token on the 13B model on a 8 core 9900k under WSL2.
Of course, in practice that is not a possibility for most people. You can't just choose to work hourly if nobody is willing to hire you on these terms. Without regulation the balance of power is strongly in the hands of…
Is there anything about sitting in a line manufacturing electronics that requires 12 hours shifts?
I see many comments here every day that would get anyone instantly banned from most of the biggest subreddits. However, I think that most people would agree that this place is a lot less toxic than reddit. Smaller…
That's not my experience, but maybe it is my adblocker hiding the Chrome ads. You can't even use the chat feature in bing with firefox without spoofing the user agent.
Wasn't expecting to see Adam Ragusea here. He has a popular youtube cooking channel nowadays.
I have noticed this too while giving the new bing a try the last few days. It doesn't matter that you have an AI bot summarizing and answering questions about the search results if your search results are worthless…
I wonder how this is implemented in the GPU. From my time working on a 3D renderer a long time ago, triangles with offscreen vertices would be clipped into smaller triangles, so in the end you would still be rendering…
Isn't this problem solved with a chiplet design like AMD does? Just manufacture the SRAM using an older and cheaper node and keep the logic in the most advanced node.
Seeing the performance of implementations like FlexGen [1], I don't think it would be entirely unreasonable to run a 13B model on a single GPU for personal usage purposes. You are not going to a run a public service out…
I was mostly talking about access to the trained model weights. The OpenAI API is certainly better than nothing, but it is very restrictive and cost prohibitive for many purposes. For instance, you have to adhere to the…