HN Explorer

Ask HN: Is synthetic data generation practical outside academia?

5 points by cpard ↗ HN
I keep seeing synthetic data pipelines powering the latest LLM “breakthroughs”: • TinyZero’s $30 fine-tuning workflow • Sky-T1’s $450 reasoning-model build • Meta AI’s Llama 3 herd (2024 paper detailing their synthetic-data training) • Berkeley OpenThoughts (“Data Recipes for Reasoning Models”), published yesterday

There are also open-source toolkits you can experiment with:

https://github.com/meta-llama/synthetic-data-kit https://github.com/bespokelabsai/curator

But it still feels very research-oriented. I haven’t found many examples of these pipelines running in real-world products.

I’m curious:

1. Who is using synthetic-data pipelines in production today?

2. What tasks does it actually improve. E.g. fine-tuning smaller models for specific tasks?

Any real-world stories, pointers, or further reading would be hugely appreciated. Thanks!

13 comments

[ 5.6 ms ] story [ 35.6 ms ] thread
sargstuff
Non-AI specific 'synthetic data generation':

historically used for processes which make use of time-series / simulations & modeling / forcasting. aka weather forcasting, related points in [0]

2) a) Testing with actual 'sensitive' data may not be possible for security reasons (aka payroll information, stock market price influences)[1]. b) insufficent/incomplete information. aka figure out how well what's known matches 'reality' and/or may suggest areas to look for 'missing' pieces in model.

-----

[0] : https://www.oreilly.com/library/view/practical-time-series/9...

[1] : https://www.k2view.com/what-is-synthetic-data-generation/

cpard
This is great. Synthetic data has been around for a long time, I think the difference with LLM related cases is that in the past it was primarily structured data that was a bit easier to approximate with some distribution or some grammar.

With synthetic data for large languages models it’s more about QA pairs and reasoning trails for solving complicated problems

sargstuff
Non-physics Much Ado about Shrodinger's Cat. Just tool(s) for quickly building higher order associations/abstractions from 'base term information'.[1][2][3]. aka dynamically generate a unique catlan number(s) for given Tromp lambda calcui as way of reducing tree height/lisp parentheses down to a single pair while dynamically computing/recomputing the determinant (appropriate base / number symbols ratio) to minimize length between parentheses.

----------------

[1] : I told AI to make me a protein. Here's what it came up with : https://www.nature.com/articles/d41586-025-01586-y

[2] : AI Models for Protein Structure Prediction : https://frontlinegenomics.com/ai-models-for-protein-structur...

[3] : AI model deciphers the code in proteins that tells them where to go : https://news.mit.edu/2025/ai-model-deciphers-code-proteins-t...

cpard
Love the references but I’m having a hard time deciphering your comment. Quantum physics where always fascinating to me but not always easy to comprehend I guess
sargstuff
yes/no. Don't know what Shrodinger's cat's state is until put into context. aka fundamental theorm of calculus provides symbolic evaluation context; but without expressions / 'numbers', theorm is just a 'shrodinger's cat' in different clothing.

One can escape the original NIL, NULL, None issue by using boolean logic, but that implies rules.

The 'strange thing' about shrodinger's cat, one can never be certain that one didn't pick the context before the cat existed and/or the references after the deceased remains were no longer visible. So, exercise is arguably statistically skewed toward 3/4 deceased, 1/4 alive. Add statistical sampling, and one can get an approximation of where things might be relative to cat life span. Only works one finds at least one instance of an 'alive cat' first. Much easier to just start with Boole's cat to avoid shrodinger's cat issues (aka lambda term). LLM's will happily supply relevant Boole's cat expression with/or without shrodinger's cat input.

Pauli might consider that a half baked cracker.

publicdaniel
It’s really useful for generating synthetic data for search and recommendations that you can use to train a smaller / faster model. This is especially useful if you don’t have lots of click-through data or with cold start scenarios. There are some good articles that cover this, if you’re interested I’ll try to find them and share
publicdaniel
I’m currently working on a document parsing engine for a specific type of document. The inputs are usually PDFs. I’m able to get great structured output from both the latest Gemini Flash models and the latest Llama Scout models. The best latency I get with Gemini is about 5 seconds end to end. With llama hosted on groq it’s about 3 seconds.

My use case is latency constrained, so I’m exploring fine tuning / distilling to see if I can get latency sub second. I imagine these are the kinds of scenarios where it’s still worth it to fine-tune and distill.

My plan is to generate a lot of synthetic training data using more capable slower foundation models and use that to train the smaller model.

cpard
Do you use any framework to generate the data and how do you evaluate the quality of the generated data?
Jugurtha
When I was in EE at university, I worked on heart anomaly detection and multi-phase flow classification for oil & gas. The papers I was reading used synthetic data with a nice noise dust sprinkled on it. Meanwhile, I worked on data from hospitals acquired on restless, sweaty, hairy, dudes with rusty, banged up electrodes and abused probes.

Needless to say, the data I saw on these papers looked nothing like the data I worked with, whether from hospitals or what I saw at Schlumberger in the Sahara.

The real world tends to be ... interesting.

cpard
That makes sense, do you think LLMs have or can potentially change that and end up having more realistic synthetic data than what you've seen in the past? I guess the data you were working were more like time series data but still if an large language model can be perceived as a universal approximator of some sort, might be able to generate more realistic synthetic data than the approach you described with the noise dust sprinkled on data.