> "I think that medical data is a really good example of a use case that we don't want to work on," says Elbaz. "In order to model medical diseases, you need real doctors to help you. "There's a lot of specialized knowledge that you would need in order to create this medical synthetic data. Even though medical data is extremely valuable. It's something that I think requires a separate company. It's just too hard. Anything that requires very, very, specialized knowledge is hard," he concluded.
One of the clear uses of synthetic data is areas where data is inaccessible and/or expensive to gather like medical imaging. Turns out, there may be "no free lunch".
I worked at a company (Rad AI) and we evaluated another company that made synthetic data for medical companies. Our physician data scientists reviewed the data- it was bad. Really, really bad. We ultimately decided that it would be irresponsible to use it (and would make for a pretty crappy model anyways).
I'm really glad to hear that this company acknowledges how hard that problem is and avoids it. The risk is so high that there is just no value in it.
At my last job we were doing ML with full motion video. Our data scientists kept looking at synthetic data (being able to use it would have solved a lot of practical problems for us) and rejecting it :)
In my PhD research (on computer vision at University of Tokyo), one of the topics I was exploring was reliability of models to synthetic data. Using generative models, we could show that models which were built on binary data sources were somewhat more reliable than others. Consider different geometries of rashes on skin & different skin location - the model could still be closer to ground truth (i.e model performance on real skin lesions)
However when other variables came into the mix (different tones, ethnicity, participant age etc), the model predictions were up for tosses.
Now, medical data is much more harder in ML. The variability is quite extreme since all our bodies are little different from one another. In my thesis, I very firmly proposed that generative models should never ever be used in synthetic medical information as part of the thesis proposal - and there were those field studies to back it up. (The thesis is under non-disclosure at University of Tokyo until September 2023 since an embargo is involved from participating hospitals & data sources. I would have been eager to share otherwise). I am glad people aren't wildly trusting if task A, B & C are doable, task AB, CA, ABC, BC etc should be similarly tractable.
> I very firmly proposed that generative models should never ever be used
Isn't this a bit strong? E.g. just because it is not a good idea for your use case right now, it may not mean that in the future something else happens and it could invalidate your firm proposal.
Apologies. Yes it seems a bit strong, but going by the understanding of field-data variability - I and few others in the area, feel that it would take nothing short of an absolute miracle to get to human-level error rate in diagnostic medicine. Hence, the confidence for the near future (5-10 years) and a disclaimer against the false notion of improving ML based outcomes. Sometimes negative results paint much truer pictures than amazing ones. Maybe if the paradigm changes for how we design generator networks, perhaps one day this presumption will be invalidated as you correctly pointed.
The whole craze about my application i.e. dermatology successes in ML spurred from the 2017 Nature paper, where skin cancer was detected as good or better than dermatologists. But technically, such experiments have design problems: We had apriori knowledge of the dataset (White N American Melanoma data) & hence we could ascertain the model performance. Real world data is much more variable. Further, later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions. 3 years later my experiments could model reliably only on 10 very common diseases & of a very uniform skin type and ethnicity. Those results were nowhere close to perfect.
The proposal to keep Human-in-the-loop is a much fairer alternative in ML aided medicine, than end to end machine learning. Most direction of research is headed that way. Physician assistance is much more reliable than potential replacement.
The trouble with synthetic data is that it doesn't address the extended variability in real population & generalization will always suffer. Also, doctors take multi-path decision, choice by elimination, past cases - based on several diagnostic inputs & even gut intuition. At that scale of input multimodality, Type I & II errors are at a scale higher than correct identifications. And we don't know how to teach intuition or imagination to machine models well enough. Those fall back to rule based methods & edge cases.
The dermatologists make small dots/arrow markings on the positively identified lesions. What the Nature paper apparently didn't do was remove the small markings in their training. It is probably an innocuous flaw since gradient-based investigation/ interpretation only took off later than their publication (2017).
As a result, instead of positively identifying the lesion based on disease pathology, it identified overwhelmingly based on presence or absence of medical marks. There was following up discussion in a certain paper of this experimental design (I can't remember the exact name), but they did gradient based activation mapping and those pointed to the marks as the identifying feature. It felt quite a revelation of why this worked so well.
In statistical modeling the equivalent of synthetic data is heavily used, but not for training the actual model on.
Building a generative model allows you to sample observations from a known distribution and confirm that your approach to learn is even able to recover the correct parameters. It's used as a basic sanity checking tool.
Synthetic data is great for this purpose. Before setting out on a complex and expensive modeling task, at least verify that in the best case (i.e. your model is literally the data generating process) you could even learn the specific problem. It's surprisingly easy to define models that are fundamentally not learnable.
However the idea that you could learn from synthetic data doesn't make sense, as you could just literally use the model that generates the data in the first place. If you're just learning known parameters, why waste the time training when you already have the answer?
Using real data that has been heavily anonymized and stripped of PII (perturbing dates, hashing strings etc) makes some sense, assuming that PII contains no information (which is unlikely). But the more clever you're trying to be with synthetically generating features the more your are essentially just manually building a model of how the world works with obviates the need to learn a model anyway.
>> However the idea that you could learn from synthetic data doesn't make sense, as you could just literally use the model that generates the data in the first place. If you're just learning known parameters, why waste the time training when you already have the answer?
Hey, that's a really good point. But the synthetic data generators in the article are not statistical models, but 3d models. Even if it were possible to create a 3d model that is a 100% faithful likeness of some arbitrary human being, we'd still need some way to compare the 3d model to a human.
Still, yeah, if we could simulate the real world with perfect fidelity we wouldn't need no machine learning.
When your AI models operate on a vectorized space, then yes. Pipelines from real-to-vectorspace and synthetic-to-vectorspace can be extremely indistinguishable from each other. Basically a video game. This makes training on synthetic data extremely effective, if not for real inference, at the very least for transfer and curriculum learning.
Generative models cannot do better than a well balanced and representative training dataset. I've worked on generative models in the medical space in the past. If you are looking to use a generative model to fix class imbalance, there is always the alternative of weighting your training dataset to account for the class imbalance. Both methods are effectively doing the same thing: oversampling the underrepresented class.
The issues that come with generative models are not unique to machine learning. Here's a human example: imagine you are a radiologist with 20 years of experience. Throughout your career you have specialized in MRI scans. You are able to identify a wide range of pathologies. Now one day a friend walks into your office with a brain MRI of a giraffe. They ask you to determine if the giraffe is healthy. You have _some_ prior from the tens of thousands of human scans you have analyzed. Despite that, you can't make a determination with any high degree of confidence. You simply don't have a good baseline for giraffes. Any analysis you do make is assuming that learnings from human scans extend to giraffes.
Generative models are useful if we are _certain_ that training data covers essentially all possible variations. Training a model on MNIST to produce numbers/digits is trivial, capturing 99.99% of the perceptible variation in handwriting is (relatively) easy. The issue with doing this for most medical applications is twofold: 1) medical training data is difficult to acquire and much less abundant than most other data types, 2) we don't have a good measure for how much of the possible variation we have captured and how much we have missed
Good comment, but- what is it with giraffes as an example of classification in machine learning? I thought it was just me but it seems everyone does it.
So, if you already understand the patterns in your domain, then you can leverage your understanding to generate data, to train your models to recognize the patterns in your domain's data?
It sounds silly, but maybe it's easier than hand-encoding the rules yourself?
> "And bias-wise we can generate whatever distribution of ethnicities, ages, genders you want in your data, so we are not biased in any way," he says as he shows us a three-dimensional fake face.
Am I missing something, or isn't this a tool that can be used to generate a highly biased dataset to train an ML system on? This is how you'd insert biases into your 'algorithm' to get the desired result, isn't it? And then you could use that system to say, select from a pool of loan applicants, and any biases would just be 'due to the algorithm'?
I'm not in the AI field, but what does "synthetic data" really mean? I'm seeing different definitions in the comments.
If I were to add synthetic lens flare to an existing image, would that be synthetic? Change the white balance? Add dirt, scratches, replace paint colors, etc? Change someones skin tone? Add blemishes?
Knowing nothing, all of these seem like they would be very beneficial, but I always see people saying any synthetic data is bad data. With the probability of intercept being so incredibly low for some of this stuff, I can't see how that could be true.
19 comments
[ 4.5 ms ] story [ 48.0 ms ] thread> "I think that medical data is a really good example of a use case that we don't want to work on," says Elbaz. "In order to model medical diseases, you need real doctors to help you. "There's a lot of specialized knowledge that you would need in order to create this medical synthetic data. Even though medical data is extremely valuable. It's something that I think requires a separate company. It's just too hard. Anything that requires very, very, specialized knowledge is hard," he concluded.
One of the clear uses of synthetic data is areas where data is inaccessible and/or expensive to gather like medical imaging. Turns out, there may be "no free lunch".
I'm really glad to hear that this company acknowledges how hard that problem is and avoids it. The risk is so high that there is just no value in it.
However when other variables came into the mix (different tones, ethnicity, participant age etc), the model predictions were up for tosses.
Now, medical data is much more harder in ML. The variability is quite extreme since all our bodies are little different from one another. In my thesis, I very firmly proposed that generative models should never ever be used in synthetic medical information as part of the thesis proposal - and there were those field studies to back it up. (The thesis is under non-disclosure at University of Tokyo until September 2023 since an embargo is involved from participating hospitals & data sources. I would have been eager to share otherwise). I am glad people aren't wildly trusting if task A, B & C are doable, task AB, CA, ABC, BC etc should be similarly tractable.
Isn't this a bit strong? E.g. just because it is not a good idea for your use case right now, it may not mean that in the future something else happens and it could invalidate your firm proposal.
The whole craze about my application i.e. dermatology successes in ML spurred from the 2017 Nature paper, where skin cancer was detected as good or better than dermatologists. But technically, such experiments have design problems: We had apriori knowledge of the dataset (White N American Melanoma data) & hence we could ascertain the model performance. Real world data is much more variable. Further, later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions. 3 years later my experiments could model reliably only on 10 very common diseases & of a very uniform skin type and ethnicity. Those results were nowhere close to perfect.
The proposal to keep Human-in-the-loop is a much fairer alternative in ML aided medicine, than end to end machine learning. Most direction of research is headed that way. Physician assistance is much more reliable than potential replacement.
The trouble with synthetic data is that it doesn't address the extended variability in real population & generalization will always suffer. Also, doctors take multi-path decision, choice by elimination, past cases - based on several diagnostic inputs & even gut intuition. At that scale of input multimodality, Type I & II errors are at a scale higher than correct identifications. And we don't know how to teach intuition or imagination to machine models well enough. Those fall back to rule based methods & edge cases.
... excuse me?
As a result, instead of positively identifying the lesion based on disease pathology, it identified overwhelmingly based on presence or absence of medical marks. There was following up discussion in a certain paper of this experimental design (I can't remember the exact name), but they did gradient based activation mapping and those pointed to the marks as the identifying feature. It felt quite a revelation of why this worked so well.
More information:
1.https://jamanetwork.com/journals/jamadermatology/fullarticle...
2.Swetter (2020) .Novel Technologies to Improve Melanoma Detection and Care Focusing on Artificial Intelligence
3.ISIC Workshop 2019 at CVPR (Slides online)
4.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8074854/
Building a generative model allows you to sample observations from a known distribution and confirm that your approach to learn is even able to recover the correct parameters. It's used as a basic sanity checking tool.
Synthetic data is great for this purpose. Before setting out on a complex and expensive modeling task, at least verify that in the best case (i.e. your model is literally the data generating process) you could even learn the specific problem. It's surprisingly easy to define models that are fundamentally not learnable.
However the idea that you could learn from synthetic data doesn't make sense, as you could just literally use the model that generates the data in the first place. If you're just learning known parameters, why waste the time training when you already have the answer?
Using real data that has been heavily anonymized and stripped of PII (perturbing dates, hashing strings etc) makes some sense, assuming that PII contains no information (which is unlikely). But the more clever you're trying to be with synthetically generating features the more your are essentially just manually building a model of how the world works with obviates the need to learn a model anyway.
Hey, that's a really good point. But the synthetic data generators in the article are not statistical models, but 3d models. Even if it were possible to create a 3d model that is a 100% faithful likeness of some arbitrary human being, we'd still need some way to compare the 3d model to a human.
Still, yeah, if we could simulate the real world with perfect fidelity we wouldn't need no machine learning.
The issues that come with generative models are not unique to machine learning. Here's a human example: imagine you are a radiologist with 20 years of experience. Throughout your career you have specialized in MRI scans. You are able to identify a wide range of pathologies. Now one day a friend walks into your office with a brain MRI of a giraffe. They ask you to determine if the giraffe is healthy. You have _some_ prior from the tens of thousands of human scans you have analyzed. Despite that, you can't make a determination with any high degree of confidence. You simply don't have a good baseline for giraffes. Any analysis you do make is assuming that learnings from human scans extend to giraffes.
Generative models are useful if we are _certain_ that training data covers essentially all possible variations. Training a model on MNIST to produce numbers/digits is trivial, capturing 99.99% of the perceptible variation in handwriting is (relatively) easy. The issue with doing this for most medical applications is twofold: 1) medical training data is difficult to acquire and much less abundant than most other data types, 2) we don't have a good measure for how much of the possible variation we have captured and how much we have missed
It sounds silly, but maybe it's easier than hand-encoding the rules yourself?
[0] https://www.microsoft.com/en-us/research/wp-content/uploads/...
Am I missing something, or isn't this a tool that can be used to generate a highly biased dataset to train an ML system on? This is how you'd insert biases into your 'algorithm' to get the desired result, isn't it? And then you could use that system to say, select from a pool of loan applicants, and any biases would just be 'due to the algorithm'?
If I were to add synthetic lens flare to an existing image, would that be synthetic? Change the white balance? Add dirt, scratches, replace paint colors, etc? Change someones skin tone? Add blemishes?
Knowing nothing, all of these seem like they would be very beneficial, but I always see people saying any synthetic data is bad data. With the probability of intercept being so incredibly low for some of this stuff, I can't see how that could be true.