I've never seen a compelling explanation of how this is supposed to work. Do people self-host their pods? Is the pod one more thing to sign up with an external provider?
If I'm self-hosting my pod anyway, then wouldn't I want my self-hosting to include the (for example) photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? If I did give them access, then how is it more private than if they hosted it themselves? Maybe the pitch is more about mobility of services and avoiding lock-in than it is about privacy and security, but that doesn't seem to be the message I get. Does anyone have any pro-solid blog posts or articles you think might be helpful in convincing me? It smells like it is well aligned with my values but I just can't see how it will actually work.
Solid seems like an idea that only makes sense if your key goal is allowing cloud providers to offer proprietary services that use your data, while still storing it elsewhere.
But I'd say most people fall either into "I don't mind proprietary services holding my data" or "I want to self-host the preferably open source application and my data". I don't see the benefit of separating the two at all, and I think the only reason this project keeps coming up and around is the name of the guy pushing it.
"Solid is a mid-course correction for the Web by its inventor, Sir Tim Berners-Lee" is very literally the only selling point I think Solid has.
Sandstorm or Cloudron have a better model, self-host (or pay someone to host for you) all of your apps and data together, in an easy interface for adding new apps like an app store.
Spritely project is developing Goblins, implementing the capability-based CapTP transport protocol for distributed programming and the fediverse, which looks very interesting.
You can choose whether to self-host or whether to have someone else handle that for you.
As for also hosting your apps: I think the answer to that is the same as for why you might still choose to use an email client even if you're hosting your email server.
However, I would expect the majority of people not to self-host, but to have that handled by a party they trust. They would then not be limited by that host also having to support the particular photo-viewing layer they are interested in, but instead are able to choose one independently.
(Disclosure: I work for Inrupt, but opinions are my own.)
I had this idea a while ago when trying to figure out how we're going to do web-tech in space. Obviously you don't want to send an entire web page, just the data.
If data is clearly templated, it's up to the user to choose a UI for displaying it. Many UIs can compete. They don't have to proprietary.
The hosting story is still a little messy, but in my version of this, I think it needs a trusted third-party to offer hosting, along with an open committee to manage the data specifications.
The advantages of separating data storage from usage are similar to those in application development. If you have a robust model for defining and retrieving your data, the tools for working with that data can be iterated on independently from the data itself.
So let's say your photos are in Google Photos, because your phone backs them up there automatically, but all your friends who you want to see the photos are on Facebook. Theoretically, "Solid" could be pulling those photos into a standard data specification, and provide easy tools for automating how they get shared into Facebook.
Then let's say you wanted to edit some of those photos, so you open up PhotoShop Solid, and get editing.
But then you join Instagram, and you want to quickly show off your Photoshop skills. All you need to do is connect Solid.
The key aspect here is that you can use as many different tools as you like to work with your data, but it's the same data, and if Facebook goes belly-up tomorrow, or you just don't want your photos there anymore, Solid has your back.
One thing that I don't think is well defined yet, but really should be, is how services will request access to your data. This needs some kind of standard interface, very similar to how phones let you customize app access. It should make it very clear what data you are "selling" to the Facebook, and what they are providing in return for that data.
That makes sense, but it also means the selling point is data portability more than data privacy/security. Since I would already need to give an application access to my data, I might as well have them host it for me. It still feels like Solid is a more complex and harder-to-reason-about solution to the same problem that GDPR data takeout tries to solve. Consistent open standards are nice, but it doesn't take that much work for Facebook (or others) to accept a Google Photos takeout dump as an input format.
If the only real problem that Solid solves is "I loose my data if provider X locks me out or goes bankrupt", then it isn't even good enough since the third-party pod-hosting company can have the same failure mode. Maybe we should expand on GDPR data takeout legislation to require something like API-driven access that would allow people/companies to build automated backup/export solutions.
That becomes a much simpler thing to build and get buy-in for instead of a whole new paradigm which companies aren't incentivized to follow and is hard for users to understand.
I think data portability is the most obvious value proposition for most people, especially between potentially hostile services (Facebook and Google Photos could easily inter-operate, but that's not something either company wants to invest in). For someone like me, who has moved a streaming music "collection" between 4 services over the past decade or so, it would have been real handy.
It also, conceptually at least, dramatically lowers the barrier to entry for new services in the same space, since they can use the existing data models and persistence layers.
The privacy aspects are secondary, at least to start, though I think the last few years have helped establish more context for them. Providing granular controls for data access in a common format across services seems like a big win, at least compared to my experience hunting through preferences/settings/account details/etc menus in Facebook and other places.
Finally, I think there's significant potential value in this being used as way to authenticate the source of media. As deepfakes are getting better and easier to create, having all the video and audio of you connected to a known identity seems like a necessity. Letting Facebook or Google be that identity provider would be very bad.
The privacy aspects need more attention. There does need to be a neutral and trusted identity provider. It should be possible for the pod owner to control access via some kind of scope/role/policy setup. That's going to be complicated for most people. Using an example from elsewhere in this thread, I probably would want PhotoShop to be able to have full access to many (but not all) images my pod, but Instagram should really only be able to read the images I want to post there.
I think federated identity is the way to go. Technically that's what we have now, ie every identity provider I'm aware of lets you reset your password via email, which is federated.
But we tried federated and users didn't care. They want convenience. Maybe with privacy apparently picking up some public interest, we can try again.
> every identity provider I'm aware of lets you reset your password via email, which is federated
I'm not sure I understand what you're getting out. Federated identity is far more than self-service password reset. And as far as convenience goes, how many places do you use your google/github/facebook/linkedIn/twitter credentials to log into a third party? How convenient is that? That's identity federation. The problem is that those companies own your identity profile and use it for their purposes, not yours.
>photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod?
IMHO, privacy is the wrong tack for these arguments. The issue is that Flickr or Google Photos can shut down and/or raise prices while all your data is stuck there.
You may also like Etebase (I created it), which is an open-source and end-to-end encrypted backend for apps. So it removes the extra layer (an additional provider) but still maintains the privacy and security benefits of self-hosting.
I read and it says you have to use a third party host or host on your own [1]. I’m not sure how this differs from the WWW - could someone explain? Thank you!
It's an inversion of control over your data. Instead of all the data being stored in multiple places outside of your control, your data is stored on a pod and you have better control over what happens to it.
I attended the presentation of Tim Berners Lee at MozFest 2018 where he outlined the Solid Project, and I've been waiting to see this project evolve. I'd like to see some more documentation and a 'get started' kind of tutorial to make this pretty abstract concept understandable. Looking around the project pages doesn't give me much incentive to start using it yet.
I'd love to see a 'Solid for dummies' kind of approach in the documentation, things are quite technical still. Some easy to understand examples would be nice!
Solid seems like a great technological idea. It empowers people to own their data and give others and apps access to it. This is empowering.
But!!!! people want useful things. Things that help them solve problems so they want to pick up those things. Solid is lacking these solutions right now.
Where is my todo list, contact app, calendar app, and so forth... all built on solid? This is an opportunity. But, until these kinds of the solutions exist for solid it's going to stay as a neat technology that's not bridged the gap to be useful.
Indeed, I feel projects like this can invest for years on platform, but fail to foster or develop the core useful apps people want to use. Ultimately, if your platform lacks that, few people will invest long enough to bring those apps from third parties.
Lovely to see this come up again. It's really a great time to promote Solid, with the evidence of shady practices at the big corporations mounting, I know lots of folk would like a way to reclaim some of their data.
That said, I feel like the big starting point for Solid is exfiltration/syncing of data between services, and that appears to be missing, for the most part.
Give folk an app that will pull their music preferences from Spotify, and let them try out Amazon or Apple music with the same collection. It's a very simple data set, and a fairly common and comprehensible use case.
Doing photos from Facebook/Instagram/Twitter/etc would be a bigger lift, but not too much different in concept, and I think very compelling in terms of selling the value of the service.
I think something like this makes sense at some # of users closer to 10-20, or maybe even higher numbers, kind of monkeysphere / Dunbar's number level.
That is, a smaller group of people (an extended family, a neighborhood street, a collective ... fine, a cult) band their resources together to pay for pod hardware to realize some savings / distributed back-up, and have "edge" apps for trusted sharing within the pod, and then P2P (pod-to-pod?) apps for sharing outside the pod.
Right now, the economics just don't quite make sense.
I absolutely think the self-hosting story definitely only works for the widespread world if those of us capable of self-hosting provide services to less-technical users we know.
If this were to take off, I would expect to see some governments offering it to all citizens, both to support whatever digital services they provide, and as a tool for helping users manage their privacy.
How would governments finance this? Developing these offerings with the quality of service of the GAFAM (which users take now for granted) costs a fortune (financed by advertising in practice as of today).
My favorite example of this is image sharing for large extended families.
If most people thought about it, they don't actually want pictures of their kids all over Instagram, but it's a way to distribute them and, after a fashion, back them up. We don't have to worry about the matriarch's house burning down and taking out our photo archive with it. Or her clicking on a malware link and wiping it all out.
If they had a piece of software that wasn't designed for building a startup (deeply technical, highly fiddly), Uncle Bob and you and your nephew Timmy could host geographically redundant copies of the data that the group has access to.
I have a glacier store of my phone and SD card data that I upload every weekend. I'm good with 5 or so days of phone data loss in that event. (I use a desktop VM that does nothing other than touch Glacier, and my phone, for this.)
Something like Solid is more useful if you want to share data with other people, or use apps that only support cloud storage. There are many such apps (Google Docs for example), but most of them are currently embedded in their respective cloud storage provider. Solid decouples the apps from the storage.
Philosophically, and in the long-term, Solid is compelling to me for all of the reasons that the project purports to exist. But we have a classic chicken-and-egg problem with the absence of both reputable pod providers and the development of an application ecosystem.
Practically, and immediately, it is compelling to me as a developer of small-ish applications (plugins, etc.) in which I want to give users the ability to store some data (preferences, etc.) in the cloud without my having to manage that data, or the associated services/infrastructure - including having to deal with absorbing or recouping the cost.
Dropbox once offered the Datastore API[0], which was a handy little bring-your-own-database service allowing apps read/write access to a key/value store in a user's account (Dropbox accounts being quite common then), but it was deprecated[1] due to lack of traction at the time.
Do pods support queries or just HTTP GET/POST/PUT/DELETE?
Queries are important for performance. They avoid transferring all data over the network from the pod to the application.
Although it's possible to define an open index format (metadata built from the actual data), it's probably still less efficient to access and also tricky to update without race conditions, data corruption, etc.
Every now and then this gets shared again and every time I am struck by what a terrible job the front page does of explaining what Solid is and why anyone would want to use it. Like, okay, yeah, "control my own data" is a nice pitch in this day and age I guess, but... what can I do with it? "Any kind of data can be stored in a Solid pod, including normal files like you might store in Google Drive or Dropbox" - that sounds kinda useful, where's the link to SolidFile? Is there even a SolidFile app or is that just a thing someone could make with this protocol?
Hell, where's the front page link to check out the list of any apps using Solid? It's like two or three random clicks to find this page (https://solidproject.org/apps) if you are curious, and it's super uninspiring - why does one of the apps in the "Showcase" section at the top not even have a description?
Solid always just feels like a solution looking for a problem.
I'm in the early stages of bootstrapping a SaaS app that prioritizes security and privacy. For my business, this would be great: users are far more likely to trust an app if the app won't be storing any of the application data itself -- if they can choose their own data hosting provider.
Google Cloud Drive, Box, and others already allow this kind of model to an extent. Third-party apps can access and modify user data hosted there, although not through a standardized API.
E2E encrypted firebase equivalent, with CalDAV frontend for E2E encrypted calendar from mobile & Thunderbird clients, with OSS clients, server and multiple language bindings? Seems too good to be true! Thanks for the pointer.
I'm the founder of Etebase, cool to see it mentioned it here!
If anyone has any questions, please let me know (email in profile). We will also be giving a talk about Etebase at FOSDEM this year if anyone is interested: https://fosdem.org/2021/schedule/event/etebase/
You're right. It is still not clear to me what/how this will be used (I think I have an idea but not sure; the front page should have made it quite clear).
It would have been great if they had described a target use-case right there on the front page.
The writeup seems to be targeting developers but it looks to me like the product is for the public (end-users). The site doesn't seem to have done a good enough job linking both of them.
How would querying the data work? Will a pod provide querying capabilities as well? Example use case would be finding all pictures around a certain location. It "seems" unavoidable that certain data still has to be at a service
provider since they enrich the original data. How would that work?
No, you're looking at a Getting Started Page targeted at web developers, hence they use the tools they are familiar with. These tools take care of auth/data format and other glue code so you can get started easily... But because Solid is based on open standards, you could write a desktop app that makes HTTP requests using your language's clients without issues, but you'll have to write some boilerplate code yourself until there's frameworks in your language that handle that for you.
I believe working with Solid is mostly about working with HTTP and RDF... so, if you're into the JVM, try Apache Jena[0].
Working with RDF is computationally intensive. And writing UIs that fit to various ontologies is not solved. So everyone inevitably ends up writing applications to fit a particular ontology.
Excluding theoretical quibbles, what changes? In practice, the particular ontology that your application supports is the standard.
The evaluation of SPARQL patterns is PSPACE-complete.
Which means that for trying to approach the problem of writing UIs (working on a realistic hardware) that adapt to different ontologies you need to restrict the set of possible solutions...
That is, to limit the number and place constraints on the ontologies you accept.
The ActivityPub ecosystem is a glaring example of this.
ActivityPub defines a set of baseline requirements (and a basic ontology) that all implementations need to support.
Once you meet the baseline, you can use linked data tools to extend the standard. Applications that understand your extension will also interact with it in the UI, those that don't understand your extension usually just ignore it.
But you need to specify base requirements (set the baseline for an extensible standard)! You can't just throw totally random ontologies at devs, it won't work. I think this is the error Solid makes.
This project is doomed in isolation. It needs to be coupled with at least one major popular/killer service, and preferably more than that. Plus make that as easy as uploading data to “the cloud,” and as transparent, plus “free” to meaningfully compete.
Right now it just looks like technology in search of a problem.
> Right now it just looks like technology in search of a problem.
This is a common turn of phrase (which has turned up more than once here, even), but it's not appropriate to describe Solid. Solid is a half-implemented solution to a real problem.
1) I am a big fan of the general philosophy of Solid. But as others have mentioned, I've found it extremely difficult to understand specifics of the proposal or to see concrete progress, based on publicly available resources.
The best place I've found is Ruben Verborgh's blog (he's a researcher who's affiliated with Solid). For example here's a nice post which goes into more detail on the ideas behind Solid:
2) In terms of applications, I'd personally like to see more of a focus on productivity applications, rather than so much focus on social media. Better interop between cloud SaaS apps would be valuable to businesses and professionals, and would sidestep many thorny challenges like decentralized moderation.
As a concrete starting point: what if I could store my Google Docs text files on my own storage layer, and edit them in realtime using a variety of editor apps? This would resemble my ability to edit a .txt file on my computer in vim or TextEdit, but would port that metaphor to the world of modern online collaboration.
Here's a short Twitter thread I wrote on this topic:
3) I'm very curious to see more incremental paths from the current web to a decentralized approach. For example, what if we could start annotating existing websites with private data and sharing those annotations P2P, rather than starting over from scratch?
I've explored this a bit with my Wildcard project, where users can store annotations in a "spreadsheet" that is linked to a web page:
I think that with a problem statement of "I want to control my data and how third parties use my data" Solid makes a good deal of sense. Personal information should be owned by you and you should be able to take it from service to service as you see fit. The two problems that I find with Solid, based on my own limited understanding are as follows:
(1) Services don't need to give back to Solid - Services, Facebook, your medical provider, whatever else you are using, does not have a clear incentive to provide their own data on you back to your Solid pod. It is far easier for them to keep it: it lets them do offline processing, and it keeps you more locked into their service. I'm not sure how one would solve this issue.
(2) Much like mobile apps with excessive permissions and the abuse of tracking elements - I don't see how Solid prevents the abuse of its service. If Solid catches on and Facebook has a permissions check saying "Let Facebook do 'SELECT * FROM .;' on your Solid data, how many people will click yes? Even if you request it each time, once the data is copied out, it is out there and can be packaged and resold, used to build advertising profiles, etc. You're back to the original problem of not being able to limit access to your data, but with extra steps. Where I think this could be solved is by Solid not providing the data directly, but by being a service which can answer queries. Queries could be items such as "Does user like cats? y/n/m". Or it could be something like "Here is an anonymized dataset being built out. Please add your input to it." Replies to queries could also have an amount of deliberately wrong or misleading answers given, depending on the service and endpoint to obfuscate your personal data on places that don't need it. While this can still be abused, it raises the bar for abuse.
The problem with Facebook is not that they are using user uploaded content, is that the user has no clue whatsoever the data is being collected by means of tracking actions (likes, time spend on a video...) and cross site tracking and what is going to be used for (Ads, make the site more addictive ?, recommendations?).
It's an issue of collecting user generated data without awareness, and with lack of transparency, that's very different from "I upload a picture and I share it with my closest friends only" for that issue, one could argue Facebook has a fair enough UI/UX.
I'm assuming that what would go into a pod is decided by the user and therefore is a separate issue.
I imagine the permission you are describing more like, "Let Facebook Access your Wedding photo's Pod" if well implemented, and "Let Garage Band access your Music pod".
another problem is that once data has been accessed, it can be stored and analyzed. maybe we need to play big tech's own game and apply ToS to any data we allow them to access?
This exactly. This problem can never get better until normal individuals have the capacity and leverage to demand reasonable contractual limits on the use of their data, and the confidence that these limits will be enforced.
That's a tall order when we don't even have the language to describe what data-usage limits even mean.
You raise a good point - my example of asking if the user likes cats is something that could be an inferred metric based on their clicking on cat pictures and having lots of pictures of cats. It isn't the user data itself which Solid aims to control.
Trying to subdivide pods so that services map access from album A -> pod B and album B -> pod C sounds like a painful UX problem as well.. Do I place my ambient music in my music pod or my "office music" pod? Do I have a hierarchy of musical pods? I am joking a little, but it is a hard thing to have fit into an easy flow.
> The problem with Facebook is not that they are using user uploaded content, is that the user has no clue whatsoever the data is being collected by means of tracking actions (likes, time spend on a video...) and cross site tracking and what is going to be used for (Ads, make the site more addictive ?, recommendations?).
I'd say the two biggest questions that outweigh any others are:
Who will access the data?
What will they do with it?
The problem is that few of us know how our data is being used against us. "We will use the latest artificial intelligence methods to convince you that [political issue] is a good thing, since you're just on the other side of the fence."
The key thing you didn't mention is who the data will be sold to. I doubt the extremist nonsense, not related at all to any videos I've watched, creeps into my YouTube recommendations by accident. Who's paying them to generate clicks and viewing hours? What conversion are they attempting with my information? Only one side understands this game. Is there any justification for not having to reveal who paid them for my information or my clicks?
> If Solid catches on and Facebook has a permissions check saying "Let Facebook do 'SELECT * FROM .;' on your Solid data, how many people will click yes?
Excellent point. For the answer, just look at how many people give FB app permission to all their contact and photos.
Hint, if you add an observer on android, you will see that whatsapp and fb app, both scan your contact list every few minutes!
If you allow anything (access my contact list so i can find person X), they will take it literally everything they can (you entire contact list, every few minutes).
One possible answer to both of your questions is "with legislation", and I feel like to some extent a tightening of the rules here is what Solid anticipates. Perhaps not to the point that we could expect the big players of today to adopt a platform like Solid, but perhaps where the companies of tomorrow looking to avoid the headaches of compliance see offloading data storage to a dedicated entity managed by the user as an appealing option.
I think that may be the answer! If data is toxic waste then offloading the storage of toxic waste may be the new way to focus on your "core competency" of running your service.
With respect to (1), data is a liability (running afoul of GDPR, etc), so if someone else can manage the storage then that reduced my liability considerably. Facebook might decide that their incentive to track you around the web exceeds their incentive to avoid this liability (of the two are even mutually exclusive), but for many avoiding liability is a powerful incentive.
March 2020 I was investigating Solid for legaltech projects I'm designing. I thought it sounded like a great way for a user to create a master profile that can be used with multiple web apps and service providers, while maintaining more control over their data. I engaged with the Solid community and with Ruben, who was very nice and helpful. However, I found that the tech was still at the hobby stage, and I didn't really think the toys being built on it were very compelling. It was very disappointing considering it was already a couple years old and had a lot of hype around it. I hope that this moves forward, but it's almost a year later and seems to be the same story.
We are launching both a Solid server TrinPod and a suite of productivity apps with TrinApp launchpad - free versions will be available end of the month https://graphmetrix.com
To me the whole pitch is a bit too nebulous and seemingly
inconsistent. What does it mean with "portable" and
"interoperable" data standards? Does that mean specific file
formats? Protocols? Apps?
What do they mean with "Linked Data"? What's novel about this,
what makes it different from hyperlinks or shared Dropbox
folders?
Why does it say pods are decentralized data stores, but then on
the "about" page it tells me I have to host the pod on a server
- how is that decentralized?
Linked Data does have a concrete definition: RDF data that is accessible through SPARQL, with well defined semantics. That is, following a published ontology and re-using existing vocabularies as much as possible.
Reminds me of https://remotestorage.io/ which I have used in a (mostly personal) website. I like that I can provide a service as a static website, and users pay for, manage and have control over their own storage. One of the things that could help it bootstrap is that they also support client-side Google Drive and Dropbox shims, so users who want to use something else can, but most users will just use one or the other.
There are downsides though.
- The API may be less than ideal for some use cases. This can make a well-performing app hard. For example there is no relational-sql API.
- You can't migrate user's data. So if you make changes to the format you need to retain backwards compatibility "forever".
- If you want to provide "discovery" or cross-user features you will need to have your own storage anyways.
I actually started working on just such a shim yesterday. The idea is to represent all major cloud storage through a single simple frontend API. You have to implement each provider's OAuth flow, but once you have the files it's pretty much the same.
Dropbox, Google, and my own protocol[0] will come first, but Solid is planned eventually.
What I've learned so far:
* Google is extremely draconian unless you use their JavaScript picker. To really integrate nicely you would have to spend at least $15000-$75000 getting your app audited by a 3rd party.
* Dropbox picker integration is very easy and slick.
Is this related to Jaron Lanier's vision of union (in the labor sense) groups which sell to companies access to the group's anonymized data? In his model, one individual can belong to many groups and their information is like a share/stock they have in a pool of data with other people. They sell in a cooperative fashion to select companies, gaining power through anonymity and group bargaining power.
Data in those contexts nearly always have "reciprocial actions" fundamentally and those who gather the data know what they actually want out of it. Company-user interactions would likely produce better data. It feels like half of an understanding of both data usage and union style collective barganing.
Unless I am missing something it seems a bit like trying to combine a semitruck and a bicycle to get green bulk cargo transportation. Sure both things individually have these advantages and it would be great to have all in one but implementing it is fundamentally nonviable.
"Mediators of Individual Data (MIDs) are a union-like organization championed by technologist Jaron Lanier as a framework for addressing the key issues around user-generated data like ownership, monetization, and reparations. An effective MID or data union could perform a number of critical functions on behalf of its members that are currently lacking in the digital economy."
There exists also a great Harvard Business Review article, but it is paywalled.
Love the idea but how does Solid prevent any malicious actor from over time replicating all the data that user authorize into a centralized repository - in this case, it only makes it even easier for data consolidation.
In my (personal) view, Solid is technology that enables service providers to put you in control of the data, not that forces them to do so. In other words, it's just part of the puzzle, where the other part is e.g. customer demand or regulation.
This post[0] by Ruben Verborgh helped me understand the problem Solid is trying to solve:
> In December 2019, Google and Facebook proudly announced a major milestone, which was echoed in news media all around the world: it is now possible to copy a picture from Facebook to Google Photos. This news came in mere months after we celebrated the 50th anniversary of another technological feat: the moon landing of 20 July 1969, when millions of households witnessed Neil Armstrong take a giant leap for mankind.
> So let me get this straight: two of the largest tech companies in history make headlines because in 2019, they move a single photo over the whopping distance of 11 km it takes from the Facebook headquarters in Menlo Park to the Googleplex in Mountain View, whereas in 1969, we sent live video signals from 380,000 km away on the actual moon?
> If those two companies, both widely hailed as pinnacles of technology, genuinely consider this to be innovation they are proud of, the only logical conclusion is that data-driven innovation today is fundamentally broken.
> The problem is widespread and not limited to technology or social media. Any sector that requires personal data to deliver services, from retail over insurance to health, suffers from the damaging effects of siloization. Companies increasingly need more access to data, but they won’t get there if they keep on collecting that data themselves.[...]
I like the concept of Solid, and proposed it to the BeWelcome community (who are running a hospitality exchange network like CouchSurfing, but free).
Although Solid is a better technical idea, I'm not sure how the migration from PHP/MySQL would work. Nobody on the Solid forum replied to my post asking about it. Another guy, Chagai, is pushing for Matrix and the fediverse. Ultimately, there's many great technical ideas, but the real solution doesn't lie in writing code - it's in building the community.
Therefore I'm trying harder to create more activity in BeWelcome's existing community (which is hard when nobody's travelling), like an online meetup every Thursday. If it gets critical mass, I believe it could be as widespread as Wikipedia or OpenStreetMap (which it models in org structure). I wish that Solid could be a part of it, but that requires active developers, and currently it's hard enough to find someone willing to make an API or native iOS app, never mind a breaking back-end change that people aren't asking for.
How does this work in practice, I didn't see a worked example on the site.
For instance, say I give Solid Social access to my contacts and my photos. At that point, Solid Social can access all my contacts and photos, copy them, and then do whatever they please with them (even if I later revoke permissions). In the current web, I upload my photos to a website, which I then trust not to use them inappropriately. In the Solid world, it seems like I give photo-access to a website (which could then copy them) and trust that website to not use them inappropriately. This seems like the current system, but with extra steps.
In terms of data compatibility, I think that there's a potentially compelling case (user-driven standards vs. bespoke handling per website), but again I fail to see the practical implications. For example, if I upload a video to a website and it becomes incompatible with other websites, my assumption is that there's some reason for that step. It could be out of walled-garden malice, but it also could be that the website is encoding it and co-locating it in such a way that it'll be really fast for other users to watch. In Solid-world, it's unclear what the flow here should be: does the website only get to consume the videos as I have them on my pod (potentially terrible performance)? Does the website get to still do optimizations (and potentially keep artifacts of my video after I revoke permissions?)
I like the drive towards consistent standards for interfacing with data, but other than that I could just be missing the important bits here (and please let me know if so), and I fail to see the positive change in most web-based user-experiences for the most part.
Solid is just the technical part. You're also going to need the regulatory part that says it's illegal for a company stores data after you revoke access.
125 comments
[ 2.4 ms ] story [ 188 ms ] threadIf I'm self-hosting my pod anyway, then wouldn't I want my self-hosting to include the (for example) photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? If I did give them access, then how is it more private than if they hosted it themselves? Maybe the pitch is more about mobility of services and avoiding lock-in than it is about privacy and security, but that doesn't seem to be the message I get. Does anyone have any pro-solid blog posts or articles you think might be helpful in convincing me? It smells like it is well aligned with my values but I just can't see how it will actually work.
But I'd say most people fall either into "I don't mind proprietary services holding my data" or "I want to self-host the preferably open source application and my data". I don't see the benefit of separating the two at all, and I think the only reason this project keeps coming up and around is the name of the guy pushing it.
"Solid is a mid-course correction for the Web by its inventor, Sir Tim Berners-Lee" is very literally the only selling point I think Solid has.
Sandstorm or Cloudron have a better model, self-host (or pay someone to host for you) all of your apps and data together, in an easy interface for adding new apps like an app store.
https://spritelyproject.org#goblins
As for also hosting your apps: I think the answer to that is the same as for why you might still choose to use an email client even if you're hosting your email server.
However, I would expect the majority of people not to self-host, but to have that handled by a party they trust. They would then not be limited by that host also having to support the particular photo-viewing layer they are interested in, but instead are able to choose one independently.
(Disclosure: I work for Inrupt, but opinions are my own.)
If data is clearly templated, it's up to the user to choose a UI for displaying it. Many UIs can compete. They don't have to proprietary.
In time, UI would become an art form; like music.
The advantages of separating data storage from usage are similar to those in application development. If you have a robust model for defining and retrieving your data, the tools for working with that data can be iterated on independently from the data itself.
So let's say your photos are in Google Photos, because your phone backs them up there automatically, but all your friends who you want to see the photos are on Facebook. Theoretically, "Solid" could be pulling those photos into a standard data specification, and provide easy tools for automating how they get shared into Facebook.
Then let's say you wanted to edit some of those photos, so you open up PhotoShop Solid, and get editing.
But then you join Instagram, and you want to quickly show off your Photoshop skills. All you need to do is connect Solid.
The key aspect here is that you can use as many different tools as you like to work with your data, but it's the same data, and if Facebook goes belly-up tomorrow, or you just don't want your photos there anymore, Solid has your back.
One thing that I don't think is well defined yet, but really should be, is how services will request access to your data. This needs some kind of standard interface, very similar to how phones let you customize app access. It should make it very clear what data you are "selling" to the Facebook, and what they are providing in return for that data.
If the only real problem that Solid solves is "I loose my data if provider X locks me out or goes bankrupt", then it isn't even good enough since the third-party pod-hosting company can have the same failure mode. Maybe we should expand on GDPR data takeout legislation to require something like API-driven access that would allow people/companies to build automated backup/export solutions.
That becomes a much simpler thing to build and get buy-in for instead of a whole new paradigm which companies aren't incentivized to follow and is hard for users to understand.
It also, conceptually at least, dramatically lowers the barrier to entry for new services in the same space, since they can use the existing data models and persistence layers.
The privacy aspects are secondary, at least to start, though I think the last few years have helped establish more context for them. Providing granular controls for data access in a common format across services seems like a big win, at least compared to my experience hunting through preferences/settings/account details/etc menus in Facebook and other places.
Finally, I think there's significant potential value in this being used as way to authenticate the source of media. As deepfakes are getting better and easier to create, having all the video and audio of you connected to a known identity seems like a necessity. Letting Facebook or Google be that identity provider would be very bad.
But we tried federated and users didn't care. They want convenience. Maybe with privacy apparently picking up some public interest, we can try again.
I'm not sure I understand what you're getting out. Federated identity is far more than self-service password reset. And as far as convenience goes, how many places do you use your google/github/facebook/linkedIn/twitter credentials to log into a third party? How convenient is that? That's identity federation. The problem is that those companies own your identity profile and use it for their purposes, not yours.
IMHO, privacy is the wrong tack for these arguments. The issue is that Flickr or Google Photos can shut down and/or raise prices while all your data is stuck there.
[1] https://solidproject.org/users/get-a-pod
> Solid lets people store their data securely in decentralized data stores called Pods.
Then in the "Get-a-pod" it suggest you should get Amazon for hosting?
I'd love to see a 'Solid for dummies' kind of approach in the documentation, things are quite technical still. Some easy to understand examples would be nice!
But!!!! people want useful things. Things that help them solve problems so they want to pick up those things. Solid is lacking these solutions right now.
Where is my todo list, contact app, calendar app, and so forth... all built on solid? This is an opportunity. But, until these kinds of the solutions exist for solid it's going to stay as a neat technology that's not bridged the gap to be useful.
That said, I feel like the big starting point for Solid is exfiltration/syncing of data between services, and that appears to be missing, for the most part.
Give folk an app that will pull their music preferences from Spotify, and let them try out Amazon or Apple music with the same collection. It's a very simple data set, and a fairly common and comprehensible use case.
Doing photos from Facebook/Instagram/Twitter/etc would be a bigger lift, but not too much different in concept, and I think very compelling in terms of selling the value of the service.
User profiles as well, à la gravatar.
How about work history, LinkedIn exports, standard resume formats?
A lot could be done with medical data, using FHIR's ontologies.
Fitness data also deserves a place there.
It's difficult to understand how Solid isn't a major success after all these years.
Given that none my prospective users have a pod, or would want to go through additional sign-up steps.
Also all pod providers seems to be experimental, and have no pricing.
That is, a smaller group of people (an extended family, a neighborhood street, a collective ... fine, a cult) band their resources together to pay for pod hardware to realize some savings / distributed back-up, and have "edge" apps for trusted sharing within the pod, and then P2P (pod-to-pod?) apps for sharing outside the pod.
Right now, the economics just don't quite make sense.
"The people who collaborated with the Germans had their reasons. Me, I was not alive during that time."
I don’t trust GAFAM to protect my privacy, but I wouldn’t trust this government either.
If most people thought about it, they don't actually want pictures of their kids all over Instagram, but it's a way to distribute them and, after a fashion, back them up. We don't have to worry about the matriarch's house burning down and taking out our photo archive with it. Or her clicking on a malware link and wiping it all out.
If they had a piece of software that wasn't designed for building a startup (deeply technical, highly fiddly), Uncle Bob and you and your nephew Timmy could host geographically redundant copies of the data that the group has access to.
Practically, and immediately, it is compelling to me as a developer of small-ish applications (plugins, etc.) in which I want to give users the ability to store some data (preferences, etc.) in the cloud without my having to manage that data, or the associated services/infrastructure - including having to deal with absorbing or recouping the cost.
Dropbox once offered the Datastore API[0], which was a handy little bring-your-own-database service allowing apps read/write access to a key/value store in a user's account (Dropbox accounts being quite common then), but it was deprecated[1] due to lack of traction at the time.
[0] https://dropbox.tech/developers/the-datastore-api-a-new-way-...
[1] https://dropbox.tech/developers/deprecating-the-sync-and-dat...
Queries are important for performance. They avoid transferring all data over the network from the pod to the application.
Although it's possible to define an open index format (metadata built from the actual data), it's probably still less efficient to access and also tricky to update without race conditions, data corruption, etc.
Hell, where's the front page link to check out the list of any apps using Solid? It's like two or three random clicks to find this page (https://solidproject.org/apps) if you are curious, and it's super uninspiring - why does one of the apps in the "Showcase" section at the top not even have a description?
Solid always just feels like a solution looking for a problem.
Google Cloud Drive, Box, and others already allow this kind of model to an extent. Third-party apps can access and modify user data hosted there, although not through a standardized API.
Let me know if you have any questions, my email is tom @ the domain.
Has this made front page in HN yet at any point?
If anyone has any questions, please let me know (email in profile). We will also be giving a talk about Etebase at FOSDEM this year if anyone is interested: https://fosdem.org/2021/schedule/event/etebase/
It would have been great if they had described a target use-case right there on the front page.
The writeup seems to be targeting developers but it looks to me like the product is for the public (end-users). The site doesn't seem to have done a good enough job linking both of them.
I believe working with Solid is mostly about working with HTTP and RDF... so, if you're into the JVM, try Apache Jena[0].
PHP? Try EasyRDF[1].
Python? rdflib[2]
[0] https://jena.apache.org/
[1] https://www.easyrdf.org/
[2] https://rdflib.readthedocs.io/en/stable/
Surely this incarnation of the RDF-based Semantic Web nirvana will succeed where all other attempts have failed! :)
I don't expect SPARQL and ontologies to go mainstream. So why "force your hand" and make solid depend on the standards of the semantic web?
There are much simpler and productive approaches to solve the problem of data interoperability.
Really? Which ones?
Working with RDF is computationally intensive. And writing UIs that fit to various ontologies is not solved. So everyone inevitably ends up writing applications to fit a particular ontology.
Excluding theoretical quibbles, what changes? In practice, the particular ontology that your application supports is the standard.
Are you aware of any projects that try to solve this? Very interested.
Which means that for trying to approach the problem of writing UIs (working on a realistic hardware) that adapt to different ontologies you need to restrict the set of possible solutions...
That is, to limit the number and place constraints on the ontologies you accept.
The ActivityPub ecosystem is a glaring example of this. ActivityPub defines a set of baseline requirements (and a basic ontology) that all implementations need to support.
Once you meet the baseline, you can use linked data tools to extend the standard. Applications that understand your extension will also interact with it in the UI, those that don't understand your extension usually just ignore it.
But you need to specify base requirements (set the baseline for an extensible standard)! You can't just throw totally random ontologies at devs, it won't work. I think this is the error Solid makes.
Right now it just looks like technology in search of a problem.
This is a common turn of phrase (which has turned up more than once here, even), but it's not appropriate to describe Solid. Solid is a half-implemented solution to a real problem.
The best place I've found is Ruben Verborgh's blog (he's a researcher who's affiliated with Solid). For example here's a nice post which goes into more detail on the ideas behind Solid:
https://ruben.verborgh.org/blog/2017/12/20/paradigm-shifts-f...
2) In terms of applications, I'd personally like to see more of a focus on productivity applications, rather than so much focus on social media. Better interop between cloud SaaS apps would be valuable to businesses and professionals, and would sidestep many thorny challenges like decentralized moderation.
As a concrete starting point: what if I could store my Google Docs text files on my own storage layer, and edit them in realtime using a variety of editor apps? This would resemble my ability to edit a .txt file on my computer in vim or TextEdit, but would port that metaphor to the world of modern online collaboration.
Here's a short Twitter thread I wrote on this topic:
https://twitter.com/geoffreylitt/status/1355255162626068482
3) I'm very curious to see more incremental paths from the current web to a decentralized approach. For example, what if we could start annotating existing websites with private data and sharing those annotations P2P, rather than starting over from scratch?
I've explored this a bit with my Wildcard project, where users can store annotations in a "spreadsheet" that is linked to a web page:
https://www.geoffreylitt.com/wildcard/
(1) Services don't need to give back to Solid - Services, Facebook, your medical provider, whatever else you are using, does not have a clear incentive to provide their own data on you back to your Solid pod. It is far easier for them to keep it: it lets them do offline processing, and it keeps you more locked into their service. I'm not sure how one would solve this issue.
(2) Much like mobile apps with excessive permissions and the abuse of tracking elements - I don't see how Solid prevents the abuse of its service. If Solid catches on and Facebook has a permissions check saying "Let Facebook do 'SELECT * FROM .;' on your Solid data, how many people will click yes? Even if you request it each time, once the data is copied out, it is out there and can be packaged and resold, used to build advertising profiles, etc. You're back to the original problem of not being able to limit access to your data, but with extra steps. Where I think this could be solved is by Solid not providing the data directly, but by being a service which can answer queries. Queries could be items such as "Does user like cats? y/n/m". Or it could be something like "Here is an anonymized dataset being built out. Please add your input to it." Replies to queries could also have an amount of deliberately wrong or misleading answers given, depending on the service and endpoint to obfuscate your personal data on places that don't need it. While this can still be abused, it raises the bar for abuse.
It's an issue of collecting user generated data without awareness, and with lack of transparency, that's very different from "I upload a picture and I share it with my closest friends only" for that issue, one could argue Facebook has a fair enough UI/UX.
I'm assuming that what would go into a pod is decided by the user and therefore is a separate issue.
I imagine the permission you are describing more like, "Let Facebook Access your Wedding photo's Pod" if well implemented, and "Let Garage Band access your Music pod".
That's a tall order when we don't even have the language to describe what data-usage limits even mean.
Trying to subdivide pods so that services map access from album A -> pod B and album B -> pod C sounds like a painful UX problem as well.. Do I place my ambient music in my music pod or my "office music" pod? Do I have a hierarchy of musical pods? I am joking a little, but it is a hard thing to have fit into an easy flow.
I'd say the two biggest questions that outweigh any others are:
Who will access the data?
What will they do with it?
The problem is that few of us know how our data is being used against us. "We will use the latest artificial intelligence methods to convince you that [political issue] is a good thing, since you're just on the other side of the fence."
The key thing you didn't mention is who the data will be sold to. I doubt the extremist nonsense, not related at all to any videos I've watched, creeps into my YouTube recommendations by accident. Who's paying them to generate clicks and viewing hours? What conversion are they attempting with my information? Only one side understands this game. Is there any justification for not having to reveal who paid them for my information or my clicks?
Excellent point. For the answer, just look at how many people give FB app permission to all their contact and photos.
Hint, if you add an observer on android, you will see that whatsapp and fb app, both scan your contact list every few minutes!
If you allow anything (access my contact list so i can find person X), they will take it literally everything they can (you entire contact list, every few minutes).
https://github.com/hyperhyperspace
Its goals are similar, the approach is more pragmatic (p2p data layer using standard web browsers and webrtc).
What do they mean with "Linked Data"? What's novel about this, what makes it different from hyperlinks or shared Dropbox folders?
Why does it say pods are decentralized data stores, but then on the "about" page it tells me I have to host the pod on a server - how is that decentralized?
Linked Data itself is not novel: https://www.w3.org/standards/semanticweb/data
There are downsides though.
- The API may be less than ideal for some use cases. This can make a well-performing app hard. For example there is no relational-sql API.
- You can't migrate user's data. So if you make changes to the format you need to retain backwards compatibility "forever".
- If you want to provide "discovery" or cross-user features you will need to have your own storage anyways.
Dropbox, Google, and my own protocol[0] will come first, but Solid is planned eventually.
What I've learned so far:
* Google is extremely draconian unless you use their JavaScript picker. To really integrate nicely you would have to spend at least $15000-$75000 getting your app audited by a 3rd party.
* Dropbox picker integration is very easy and slick.
[0]: https://github.com/gemdrive
Unless I am missing something it seems a bit like trying to combine a semitruck and a bicycle to get green bulk cargo transportation. Sure both things individually have these advantages and it would be great to have all in one but implementing it is fundamentally nonviable.
"Mediators of Individual Data (MIDs) are a union-like organization championed by technologist Jaron Lanier as a framework for addressing the key issues around user-generated data like ownership, monetization, and reparations. An effective MID or data union could perform a number of critical functions on behalf of its members that are currently lacking in the digital economy."
There exists also a great Harvard Business Review article, but it is paywalled.
[1] https://blog.datadividendproject.com/this-week-in-data-101-w...
http://archive.today/swdlk
Edit: DataDividendProject.com (https://archive.is/iFhT0). Given that it is all blocked, it is not that interesting to me.
> In December 2019, Google and Facebook proudly announced a major milestone, which was echoed in news media all around the world: it is now possible to copy a picture from Facebook to Google Photos. This news came in mere months after we celebrated the 50th anniversary of another technological feat: the moon landing of 20 July 1969, when millions of households witnessed Neil Armstrong take a giant leap for mankind.
> So let me get this straight: two of the largest tech companies in history make headlines because in 2019, they move a single photo over the whopping distance of 11 km it takes from the Facebook headquarters in Menlo Park to the Googleplex in Mountain View, whereas in 1969, we sent live video signals from 380,000 km away on the actual moon?
> If those two companies, both widely hailed as pinnacles of technology, genuinely consider this to be innovation they are proud of, the only logical conclusion is that data-driven innovation today is fundamentally broken.
> The problem is widespread and not limited to technology or social media. Any sector that requires personal data to deliver services, from retail over insurance to health, suffers from the damaging effects of siloization. Companies increasingly need more access to data, but they won’t get there if they keep on collecting that data themselves.[...]
0. https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-...
Although Solid is a better technical idea, I'm not sure how the migration from PHP/MySQL would work. Nobody on the Solid forum replied to my post asking about it. Another guy, Chagai, is pushing for Matrix and the fediverse. Ultimately, there's many great technical ideas, but the real solution doesn't lie in writing code - it's in building the community.
Therefore I'm trying harder to create more activity in BeWelcome's existing community (which is hard when nobody's travelling), like an online meetup every Thursday. If it gets critical mass, I believe it could be as widespread as Wikipedia or OpenStreetMap (which it models in org structure). I wish that Solid could be a part of it, but that requires active developers, and currently it's hard enough to find someone willing to make an API or native iOS app, never mind a breaking back-end change that people aren't asking for.
For instance, say I give Solid Social access to my contacts and my photos. At that point, Solid Social can access all my contacts and photos, copy them, and then do whatever they please with them (even if I later revoke permissions). In the current web, I upload my photos to a website, which I then trust not to use them inappropriately. In the Solid world, it seems like I give photo-access to a website (which could then copy them) and trust that website to not use them inappropriately. This seems like the current system, but with extra steps.
In terms of data compatibility, I think that there's a potentially compelling case (user-driven standards vs. bespoke handling per website), but again I fail to see the practical implications. For example, if I upload a video to a website and it becomes incompatible with other websites, my assumption is that there's some reason for that step. It could be out of walled-garden malice, but it also could be that the website is encoding it and co-locating it in such a way that it'll be really fast for other users to watch. In Solid-world, it's unclear what the flow here should be: does the website only get to consume the videos as I have them on my pod (potentially terrible performance)? Does the website get to still do optimizations (and potentially keep artifacts of my video after I revoke permissions?)
I like the drive towards consistent standards for interfacing with data, but other than that I could just be missing the important bits here (and please let me know if so), and I fail to see the positive change in most web-based user-experiences for the most part.
But the technical part is still necessary.