Key takeway from the article: OpenAI's ChatGPT APIs might actually be, surprise surprise, profitable.
"As of June 2024, OpenAI's API was very likely profitable, with surprisingly high margins. Our median estimate for gross margin (not including model training costs or employee salaries) was 75%." "Once all traffic switches over to the new August GPT-4o model and pricing, OpenAI plausibly still will have a healthy profit margin. Our median estimate for the profit margin is 55%."
Ye the article is pretty interesting - I was trying to calculate inference provider's profit margins for say Llama-3.1 8b including electricity costs and the power consumption of GPUs, and I seem to always get to around $0.20/1M tokens as the minimum cost if you amortize a GPU's cost over 1 GPU. If you amortize it over 2 years, then maybe $0.15/1M tokens.
I'm very intrigued about OpenAI's profit margins, if they are indeed true.
Fair point! Ye if it's 3-4 year amortization that's reasonable - quantization could be another reason why some benchmarks for 4o aren't that good vs pure float8 / float16, and also how they made it much faster
4 comments
[ 4.3 ms ] story [ 26.5 ms ] thread"As of June 2024, OpenAI's API was very likely profitable, with surprisingly high margins. Our median estimate for gross margin (not including model training costs or employee salaries) was 75%." "Once all traffic switches over to the new August GPT-4o model and pricing, OpenAI plausibly still will have a healthy profit margin. Our median estimate for the profit margin is 55%."
I'm very intrigued about OpenAI's profit margins, if they are indeed true.