Deepfakes, manipulated media, etc have been a concern of mine for a while. I believe it’s only a matter of time before anyone with a smartphone can generate perfect video, audio, images, etc of anyone saying anything or doing anything.
The current approaches to this issue focus on a “fight fire with fire approach” - using newer and newer ML models to return probability on the likelihood of given media being “real” or “fake”. This approach is appropriate for digital media that’s already out there but it has a lot of issues. Which model to trust? What’s the probability like? How does the legal system treat probability? What happens when/if the generation approaches outright beat the detection models?
I got to thinking - detection probability will never be perfect. What if this issue was approached from the other side of the coin - 100% certainty with verification of digital media assets? We already have tried and proven cryptographical approaches that could provide this but they’re onerous and difficult to use (PKI, PGP, etc).
I want to bring digital media asset verification to everyone at scale by making it easy enough for anyone to use. This is how Tovera was born.
Tovera takes your digital media assets (currently JPG only - PNG, MPEG4 in PoC) and embeds an anonymous, randomly generated unique identifier (hex UUID) in the file metadata. We then create a SHA256 checksum of the file and associate it with the user account in our backend. Once uploaded to the platform users can share media on existing social media networks (currently Twitter and Facebook) or any website with copy and paste iframe HTML. Because the unique identifier is embedded in the asset itself we can also (eventually) use various other client approaches - browser plugins, mobile SDKs, third party clients, etc to verify media anywhere it ends up being consumed.
When viewers consume media, our client implementation (currently JavaScript only) hits our API with the metadata-embedded unique identifier and returns the stored checksum. The checksum of the media being viewed on the client is compared to the checksum returned from the API. If they match the viewer can explore the verification information and provenance. If they don’t match the viewer is warned. Then I realized “Hey, while we’re hitting an API every time an asset is consumed, why don’t we let users attach other attributes to it? Why not let them update these attributes after the media has been released so it updates dynamically everywhere?”
Using our dashboard users can set things like media description, their public name, and asset use license (currently well-known Creative Commons options). Additionally, users can opt for our API to hide this information. When extended dynamic metadata is marked as hidden our API will only return the checksum of the asset for verification. All of this can be updated at any time and reflected everywhere near instantly (with CDN cache purging, etc currently a few minutes). Of course we’re going to keep working on this so users can further protect, market, brand, and otherwise enrich their digital media published with Tovera.
Then I realized - why trust us not to modify this verification data? On upload or anytime later, users can opt-in for the anonymous verification record (random unique identifier and checksum) to be added as JSON to IPFS (don’t worry - we pin it to one or more pinning services so it’s not going anywhere). We then use blockchain smart contracts to create an immutable blockchain record (currently Polygon mainnet) pointing to that IPFS JSON URI. With this anyone (users, viewers, security researchers, third parties, etc) can compare our API results to these completely independent and immutable verification records to assure there’s no funny business going on at Tovera. Currently when selected it takes a few minutes for IPFS, blockchain minting, etc to happen but once completed these records are refreshed in the user dashboard and the viewer.
For all of the usual reasons we strip metadata too. Then we add our unique identifier in a custom XMP namespace. But we have a workaround for the issue you describe.
Users upload their assets to our storage with the unique ID embedded. They then get a shareable link or HTML that loads the asset and our verification Javascript directly from our embed platform.
The image itself shows up in a social media post, website, etc just as it normally would but if a user clicks it they see the above, with all of the extended metadata, etc.
5 comments
[ 3.3 ms ] story [ 28.4 ms ] threadThe current approaches to this issue focus on a “fight fire with fire approach” - using newer and newer ML models to return probability on the likelihood of given media being “real” or “fake”. This approach is appropriate for digital media that’s already out there but it has a lot of issues. Which model to trust? What’s the probability like? How does the legal system treat probability? What happens when/if the generation approaches outright beat the detection models?
I got to thinking - detection probability will never be perfect. What if this issue was approached from the other side of the coin - 100% certainty with verification of digital media assets? We already have tried and proven cryptographical approaches that could provide this but they’re onerous and difficult to use (PKI, PGP, etc).
I want to bring digital media asset verification to everyone at scale by making it easy enough for anyone to use. This is how Tovera was born.
Tovera takes your digital media assets (currently JPG only - PNG, MPEG4 in PoC) and embeds an anonymous, randomly generated unique identifier (hex UUID) in the file metadata. We then create a SHA256 checksum of the file and associate it with the user account in our backend. Once uploaded to the platform users can share media on existing social media networks (currently Twitter and Facebook) or any website with copy and paste iframe HTML. Because the unique identifier is embedded in the asset itself we can also (eventually) use various other client approaches - browser plugins, mobile SDKs, third party clients, etc to verify media anywhere it ends up being consumed.
When viewers consume media, our client implementation (currently JavaScript only) hits our API with the metadata-embedded unique identifier and returns the stored checksum. The checksum of the media being viewed on the client is compared to the checksum returned from the API. If they match the viewer can explore the verification information and provenance. If they don’t match the viewer is warned. Then I realized “Hey, while we’re hitting an API every time an asset is consumed, why don’t we let users attach other attributes to it? Why not let them update these attributes after the media has been released so it updates dynamically everywhere?”
Using our dashboard users can set things like media description, their public name, and asset use license (currently well-known Creative Commons options). Additionally, users can opt for our API to hide this information. When extended dynamic metadata is marked as hidden our API will only return the checksum of the asset for verification. All of this can be updated at any time and reflected everywhere near instantly (with CDN cache purging, etc currently a few minutes). Of course we’re going to keep working on this so users can further protect, market, brand, and otherwise enrich their digital media published with Tovera.
Then I realized - why trust us not to modify this verification data? On upload or anytime later, users can opt-in for the anonymous verification record (random unique identifier and checksum) to be added as JSON to IPFS (don’t worry - we pin it to one or more pinning services so it’s not going anywhere). We then use blockchain smart contracts to create an immutable blockchain record (currently Polygon mainnet) pointing to that IPFS JSON URI. With this anyone (users, viewers, security researchers, third parties, etc) can compare our API results to these completely independent and immutable verification records to assure there’s no funny business going on at Tovera. Currently when selected it takes a few minutes for IPFS, blockchain minting, etc to happen but once completed these records are refreshed in the user dashboard and the viewer.
Today users can sign up for free and add up to...
File metadata is routinely stripped out of uploads as a matter of security/privacy.
Users upload their assets to our storage with the unique ID embedded. They then get a shareable link or HTML that loads the asset and our verification Javascript directly from our embed platform.
You can see it in action here:
https://embed.tovera.com/preview/c65b0658ab6e4d89963b1e0a319...
The image itself shows up in a social media post, website, etc just as it normally would but if a user clicks it they see the above, with all of the extended metadata, etc.