The problem isn't the technology, it's the people. People don't want to share their data. You have to get people to actually accept that their competitors might be able to easily scrape data out of their website. They don't want that, and they'll pay money to keep it from happening.
Where did you get that notion? Just take a look at the internet this last decade.
What people crave now is a semantic web that is connected to a community of computers able to very quickly verify and distinguish what’s real and what’s disconnected from the truth.
I think they meant that companies don’t want to share their data. If you look at things like the Twitter API and how it’s been slowly locked down over the years, it’s clear that this is the case.
That really depends on the type of data, I think. For example, I'm quite sure restaurant owners love it when their restaurant address is shared far and wide, so people can find the restaurant.
When the data is your USP, sure. But there's a ton of data that's not.
Former semantic web researcher here. The problem with semantic web is that it's way too complicated for the average user (unlike html), with no clear payoff/ I see the need for Machine learning to automate several of these tasks. It is a laudable effort, but I'm not sure if it'd be anything more than some marginal improvements in data interop on a web-scale.
You're right. It's hard to publish data with semantic markup and understand the schemas (or even the benefit of using them). Nobody is writing FOAF or any other RDF/OWL by hand.
In order to bootstrap the process, authorship tools need to produce semantic markup automatically on the behalf of users. Unfortunately the content silos do not want to do that. They're happy to keep the data to themselves and run Snowflake / big data queries on them all day long. They just don't want you doing the same. It's no benefit to them if your friend or interest graph escapes their hold.
This is a difficult chicken and egg problem.
A good angle of attack might be for the distributed web folks to take an interest. ActivityPub, Mastodon, and Matrix/Riot/(whatever they rebranded to) would be beneficiaries of schema ingestion. The more they can mark up the data they produce, the more useful it becomes. A tooling ecosystem can grow around semantic data.
Technically open standard APIs in a distributed web are already semantic, it may just be that their schemas don't map well to queryable triple stores. But that's just a transformation away. RDF is just a format for expressing your ontology in an open and compatible way others can reference and build upon.
The ActivityPub standard is built atop JSON-LD, which is a Semantic Web standard. I don't think anyone has really tried to parse the data exposed by the Fediverse for much though.
I was working in RDF/ontology space. IMO the additional domain knowledge required to model data correctly and publish them in a way an average user can comprehend and navigate properly is disproportional to the benefit of such data model. By the time everything is properly modelled the knowledge itself might be a bit obsolete (speaking from bio-tech related fields).
The semantic web was at one time "the next big thing". It was one of those technologies that topped the Gartner Hype Cycle sometime during the first Web 2.0 years.
Interesting to see it back again. It had many problems but was always an interesting concept.
I've had to dive into this recently as part of my work (I'm in academia).
While the original vision seems to have failed from a lack of incentives, the idea behind RDF as way to model connected data and its many serialisation formats is actually quite helpful. It seems to have found a niche in academia when it comes to sharing and integrating large graph datasets. I just consider RDF and most of the other semantic web standards as a framework for modelling and serialising graph data. It's pretty good at that.
No, it’s not even good for that. The fact you need yo reify properties (relationships) to annotate them is a big failure, which is not present in more general graph models.
I think this is a matter of opinion. I'm personally a fan of reified properties, but this is the source of differences between property graph databases and RDF triple stores.
Works fine for my use cases (e.g. representing multiple WordNets as labeled property graphs in Neo4j) and I quite like the simplicity of data modelling based on triplets of data literals.
Not true. See "reification is a red herring" [1]. Also, there's now a more natural way to add properties to relationships if really needed, it is call RDF* [2], and it is already implemented of a lot of triplestores, including RDF4J, Jena, StarDog and more.
I'm a committer on Eclipse RDF4J and currently working on our implementation of SHACL. I've worked with semantic web now for 7-8 years I think. Seen quite a few projects struggle or fail mainly due to performance issues. That is why I'm doing my best with optimising RDF4J.
Hey there! I actually saw this on my own experiments!
I was experimenting with validating some data through the SHACL version of schema.org [1]. To be fair my findings are not much more than "anecdata" for now, since a proper benchmark should use a bigger dataset, allow the JVM to warm up, etc... But from a quick comparison, the RDF4j implementation was on my tests from 2 to 3 times faster than the "reference" one [2] when validating some small in-memory data.
You will notice a much bigger difference once you have a larger database and want to make changes to your data. RDF4J is able to analyse your changes and validate only the smallest amount of data to make sure that everything is still valid.
I appreciate the spirit of the semantic web, but I've never really been clear on how it would accomplish its stated goals.
Let's imagine the web has lots of annotated, machine-readable data. You still need a client to explore that data, right? The browser is not going to know how to make sense of the relationships between your data points. How is the end user going to consume the information, a web app? As the dominance of companies like Facebook, Google and Twitter have shown, the average user doesn't care about de-centralization. They care about a good user experience.
Let's take the recipe scenario presented in the article. What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
> What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results?
Yes, I think the semantic web is suited to big companies with large teams and huge processing capabilities – they hoover the data up and add it to their own graph. As you've pointed out, the trend is for Google to pull it out of the website and display it on the results page, cutting the content creator out entirely.
Building a tool or service on top of the semantic web is out of reach for most developers because of inconsistencies and complexities, unless they use the tools provided by the giants – and that goes directly against the principles of the semantic web.
The exception here is niche use cases and academia working with huge datasets, where the semantic web seems to be incredibly useful.
> What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
> Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
Sure but without some way for smaller players to get access to the same data that the giants have, there's no way to compete. There's a reason Google, FB, LI, et al have spent as much energy as they have siloing their data, cutting off interop, and consolidating their services. Technical or product or user experience superiority means nothing when you don't have the data; a polished, quick search engine means nothing if it has no results to search through.
I think that the even more powerful pitfall of that web scenario would be plain and simple lack of truthfulness. Machine readable lies.
If you are old enough then you will remember a time when SEO wasn't given a name yet, and mostly consisted of spamming the head tag with untrue META tags. The entire idea of the semantic web is basically a hypothetical extrapolation of the assumption "what if META didn't fell victim to lies?"
But that idea quickly evolved away from that web use case full of deliberate deception and evolved into technology for interoperable knowledge representation. The "web" use case quickly turned into the science funding equivalent of spamming META tags: systems (decision-makers) that were still falling for it where considered so far behind that influencing them with that method did not even feel like lying.
Interoperable knowledge representation isn't valuable because you might want to plug together random knowledge bases (you wouldn't), but because it would be an enabling technology to build one knowledge base on top of another (or on top of multiple others). This would be particularly valuable when modeling is done in short-lived iterative steps with changing teams, organisations and goals, like the quick iterations of student generations that each push ahead their niche of science with some miniscule contribution to progress. With sem-web tech, one generation of knowledge modelers would be able to do their work on top of the work of those who came before them, without being restricted to the technology of those earlier generations. Think of the "web" in "semantic web" as a big spelling error of "stack".
(source: did my diploma during peak semantic web, shoveling OWL from a Java DOM into a Lisp server to brute force maximal consistent subsets of an inconsistent superset in fantastically un-scalable ways)
The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects that require a big leap of faith to get invested on. I see solidproject.org as an example of this.
I've been exploring semantic technologies from the cold and incredible practical angle of "my own needs" :-). The RDF model offers a lot even when keeping data for myself: I can use RDF to infer new things from existing data and visualize relationships. An RDF triplestore can be used to store pretty much anything, and works well for CRUD. RDF is about graphs, so a triple store can be used to model documents too, since trees are a subset of all graphs.
RDF4J has really nice documentation and supports a lot of modern features. The getting started guide is great [1], and there's also a super friendly gitter room to ask questions [2].
>The problem with the "semantic web" projects is that they are always proposing "pie in the sky" projects
I have read this sentiment countless of times, but I don't quite understand it. Maybe because I am not familiar with the semantic web. Can you please suggest some reading list for a person who is unfamiliar with semantic web to better understand these issues?
The first thing you should read is the RDF primer [1]. It explains the basic data model that the semantic web uses as building block for anything else.
The gist of it is to provide a way to model data as a graph, and make it almost trivial to exchange data between parties. A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that.
The "pie in the sky" part is that, from this basic building block, a whole host of other technologies can be built, some of them dealing with artificial intelligence, machines being able to _infer_ more information from what you have already said [2]. Inferences can work best when there's more data, that's were the ability to mix your data with third party databases comes handy.
Solid adds even more stuff to the table. In their words: derived from "social linked data", is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. I think "decentralized" is the keyword here. I haven't even started looking at it :-)
> A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that.
What solution does RDF provide for that? I know of `owl:sameAs`, but even if it were feasible to look up those statements and reason about them (assuming someone reliable went to the effort to actually define them, and did so correctly), that already breaks down if I have to match a single field "name" (or, say, vcard:fn) with two fields "first name" and "last name" (or, say, foaf:givenName and foaf:surName). So as far as I know, there's still no practical solution - or is there?
Not an expert on OWL but I heard owl:sameAs is supposed to be a strong equivalence assertion (commutative) and perhaps not necessarily the tool you want to use for integration
To be able to say that two subgraphs describe the same thing, at some point, a mapping is needed between the different sub graphs (for users could be, say, a common schema:email statement). A good practice is to use some existing ontology when modeling (say, FOAF or schema.org), this can fix the problem of having to map fields since both sources would already be using the same conventions.
I think what's normally done is some sort of normalization pass or passes after ingesting the triples, so at the end of the process your app can always use your preferred property (say vcard:fn) or just ignore the subgraph in the worst case. There are a number of tools to perform this kind of normalization.. I know of SPARQL construct queries but I've seen others out there (rml.io).
If you are asking if RDF provides a silver bullet to integrate data sources, the answer is no, everything requires effort.
But I think the integration story is way better than when integrating other data models, say, relational databases.
The other aspect is that URLs being the ids for statements, I can peek at the implementation of a 3rd party data model and use that _in advance_, instead of integrating later.
For instance, when I write my blog I can use the URLs of existing wikidata entities to model my own entities. If wikipedia entities used a relational model, the IDs of these entities would be opaque and could change at any time, so I wouldn't be able to do that.
The semantic web makes sense for large datasets provided by tech-enabled companies, but the overwhelming majority of entities represented on the web don't have large datasets or tech teams. They have a small amount of standard data they want to get out to customers – contact data, company data (tax registration or whatever), GPS locations for stores, etc. They don't need 99.9% of the complexities of Schema.org.
They only consider a website from their viewpoint – a human viewing the website through browser. The HTML (including microdata, JSON-LD, etc) is terrifying for most of them. So at best, their web designer builds in some semantic web standards but the data can be inaccurate or become out of data quickly because the website owner doesn't even know it's there.
Even big companies have adopted semantic web standards in such a confusing and inconsistent way that it's difficult to rely on the data being available in an expected format, so that a developer can build something on top of it.
Unless semantic web standards can be adopted by small businesses in a simple way (point, click and type) I don't think they will ever be widely adopted.
Finally, I don't think it's efficient to hide the 1kb of machine usable data inside 99kb of HTML designed for browser rendering.
Shameless plug:
A small team and I have built an alternative to the semantic web, to store machine readable data in DNS. It can be populated by simple user friendly online forms.
Interesting that the article doesn't touch on Tim Berners-Lee's current focus: https://solidproject.org (Disclosure: I work for his startup, Inrupt, which he founded to give that project a boost. Opinions here are my own though.)
Although it's primarily known for allowing apps to store their users' data in a place under their users' control, for better or worse, it's built on many SemWeb technologies.
And even though I'm still somewhat disillusioned about the Semantic Web (like Aaron Schwarz, I used to be enthusiastic about it way back when), but there certainly is a chance it will be interesting again when your personal (potentially private) data can be part of the Semantic Web as well. For example, if they'd have written their reviews in a Solid-based website, linking them to MusicBrainz would be a relatively small step.
The universal login aspect of Solid sounds rather like Mozilla’s Persona. What about Solid makes it more likely to succeed against the incumbents than Persona?
There are several things that make it different, making the reason Persona failed not apply (though obviously it might still fail for different reasons):
- There are several organisations behind Solid that whose raison d'etre is Solid. I'd argue that Mozilla possibly pulled the plug for Persona too early, specifically, without first adding support for it in Firefox, thereby creating massive incentive for website owners to support it. But then, it wasn't their core focus, and there was no return on investment for them, so I can see why they cancelled it.
- Ironically, a wider focus. The incentive for website owners to support Persona would have been that it would be easier for potential users to join - although they already got most of that benefit by allowing sign-in with Facebook, and most people did not actually use Persona. Solid, on the other hand, has several incentives to support it. It can give organisations access to data that the user is actively curating, rather than having to provide it to different organisations individually, with it getting outdated the moment they shared it. Additionally, given how, for many organisations, data is increasingly (post-GDPR and all that) a liability and not their primary competency, outsourcing management of user data to the user themselves makes sense. And finally, there will hopefully be more people with a Solid Pod, especially in specific markets - for example, IIRC the government of Flanders was looking at giving every citizen a Pod.
- It can lean on the marketing benefits of it being TimBL's main focus, which may or may not be stronger than being one of Mozilla's projects.
- Depending on your definition of success, it can be as successful as Mastodon/the Fediverse is. As opposed to Persona, people can and do write apps specifically to support Solid, because it's more than just a login.
Just off the top of my head, and again, personal views. But hopefully that gives some insight.
The problem is the economic incentives. The people who have to do extra work are the people providing the data. The people who get the benefit are the people consuming the data.
The semantic web currently scales poorly in two different dimensions. As long as this remains the case, it will see little adoption.
First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers.
Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI.
If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.
I think you said it pretty well, but I'll re-iterate some of your second point.
The human element defies any solution other than social coordination such as a specs registry or org. At some point the software needs to understand each other and that requires people to work together. A technical solution around schemas can assist the coordination work, but it can't replace it.
EDIT to add: Since we can't remove human coordination, I question the value of arbitrary schema composition when it incurs the huge DX cost of using a URL to represent every attribute.
The "semantic web" is not a special technology, it's just a bunch of standards for data interchange. You can definitely expose highly proven and scalable technology stacks that have nothing to do with silly "graph database" models/platforms, while still using semantic web and Linked Data standards to great effect.
Yes but couldn't you imagine a web and a society that doesn't run on big data?
What if the unscalability of the semantic web were a feature rather than a bug?
The current political and economic fallout of the big data web that we all know and love arises precisely due to the ungodly effects of network size on power laws. E.g., the rich get richer and the poor get poorer on a global scale.
Why not a more semantic and a more decentralized web.
I think that AI is getting close to the point that it is going to be a significant factor in the story of the semantic web from here on.
What comes to mind is people training AI to parse web pages and mark them up to indicate different types of data inside of them. Such as the recipe example.
So someone who authors a WordPress plugin for recipes could incorporate it. Or, and this is more scalable but obviously very difficult, it may be possible to create plugins for web frameworks that automatically inject the semantic markup using AI. Maybe.
Then there could be another AI program that converts natural language queries into semantic knowledge queries.
But I think that AI could play a big part in rolling it out. I mean, if you had the AI systems without the semantic web, for open systems you would still want a way for them to share data unambiguously.
The semantic web can only work via automation -- pretty much complete automation. Semantic information depends on a "stance" (those humanities classes do pay off) since the same information can mean different things depending on context and perspective.
In addition spam will continue to be a significant problem.
We've been working on an alternative to SemWeb based on a new language for marking up facts inside of text. It doesn't rely on schema but allows the user to define fact relationships as they happen in writing. More info here: https://news.ycombinator.com/item?id=24662705
52 comments
[ 5.1 ms ] story [ 124 ms ] threadWhere did you get that notion? Just take a look at the internet this last decade.
What people crave now is a semantic web that is connected to a community of computers able to very quickly verify and distinguish what’s real and what’s disconnected from the truth.
When the data is your USP, sure. But there's a ton of data that's not.
In order to bootstrap the process, authorship tools need to produce semantic markup automatically on the behalf of users. Unfortunately the content silos do not want to do that. They're happy to keep the data to themselves and run Snowflake / big data queries on them all day long. They just don't want you doing the same. It's no benefit to them if your friend or interest graph escapes their hold.
This is a difficult chicken and egg problem.
A good angle of attack might be for the distributed web folks to take an interest. ActivityPub, Mastodon, and Matrix/Riot/(whatever they rebranded to) would be beneficiaries of schema ingestion. The more they can mark up the data they produce, the more useful it becomes. A tooling ecosystem can grow around semantic data.
Technically open standard APIs in a distributed web are already semantic, it may just be that their schemas don't map well to queryable triple stores. But that's just a transformation away. RDF is just a format for expressing your ontology in an open and compatible way others can reference and build upon.
Interesting to see it back again. It had many problems but was always an interesting concept.
While the original vision seems to have failed from a lack of incentives, the idea behind RDF as way to model connected data and its many serialisation formats is actually quite helpful. It seems to have found a niche in academia when it comes to sharing and integrating large graph datasets. I just consider RDF and most of the other semantic web standards as a framework for modelling and serialising graph data. It's pretty good at that.
What do you suggest as an alternative to RDF?
1: http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-...
2: https://blog.liu.se/olafhartig/2019/01/10/position-statement...
I was experimenting with validating some data through the SHACL version of schema.org [1]. To be fair my findings are not much more than "anecdata" for now, since a proper benchmark should use a bigger dataset, allow the JVM to warm up, etc... But from a quick comparison, the RDF4j implementation was on my tests from 2 to 3 times faster than the "reference" one [2] when validating some small in-memory data.
1: http://datashapes.org/schema
2: https://github.com/topquadrant/shacl
Let's imagine the web has lots of annotated, machine-readable data. You still need a client to explore that data, right? The browser is not going to know how to make sense of the relationships between your data points. How is the end user going to consume the information, a web app? As the dominance of companies like Facebook, Google and Twitter have shown, the average user doesn't care about de-centralization. They care about a good user experience.
Let's take the recipe scenario presented in the article. What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
that's pretty much exactly what they were doing with freebase (https://developers.google.com/freebase/guide/basic_concepts)
Building a tool or service on top of the semantic web is out of reach for most developers because of inconsistencies and complexities, unless they use the tools provided by the giants – and that goes directly against the principles of the semantic web.
The exception here is niche use cases and academia working with huge datasets, where the semantic web seems to be incredibly useful.
http://client.linkeddatafragments.org/
> How is the end user going to consume the information, a web app?
https://github.com/linkeddata/rdflib.js
There's many more RDF libraries out there
> What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
That's already what Google does with Schema.org
https://www.searchenginejournal.com/technical-seo/schema/
is a result I pulled up on DDG
> Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
Sure but without some way for smaller players to get access to the same data that the giants have, there's no way to compete. There's a reason Google, FB, LI, et al have spent as much energy as they have siloing their data, cutting off interop, and consolidating their services. Technical or product or user experience superiority means nothing when you don't have the data; a polished, quick search engine means nothing if it has no results to search through.
If you are old enough then you will remember a time when SEO wasn't given a name yet, and mostly consisted of spamming the head tag with untrue META tags. The entire idea of the semantic web is basically a hypothetical extrapolation of the assumption "what if META didn't fell victim to lies?"
But that idea quickly evolved away from that web use case full of deliberate deception and evolved into technology for interoperable knowledge representation. The "web" use case quickly turned into the science funding equivalent of spamming META tags: systems (decision-makers) that were still falling for it where considered so far behind that influencing them with that method did not even feel like lying.
Interoperable knowledge representation isn't valuable because you might want to plug together random knowledge bases (you wouldn't), but because it would be an enabling technology to build one knowledge base on top of another (or on top of multiple others). This would be particularly valuable when modeling is done in short-lived iterative steps with changing teams, organisations and goals, like the quick iterations of student generations that each push ahead their niche of science with some miniscule contribution to progress. With sem-web tech, one generation of knowledge modelers would be able to do their work on top of the work of those who came before them, without being restricted to the technology of those earlier generations. Think of the "web" in "semantic web" as a big spelling error of "stack".
(source: did my diploma during peak semantic web, shoveling OWL from a Java DOM into a Lisp server to brute force maximal consistent subsets of an inconsistent superset in fantastically un-scalable ways)
I've been exploring semantic technologies from the cold and incredible practical angle of "my own needs" :-). The RDF model offers a lot even when keeping data for myself: I can use RDF to infer new things from existing data and visualize relationships. An RDF triplestore can be used to store pretty much anything, and works well for CRUD. RDF is about graphs, so a triple store can be used to model documents too, since trees are a subset of all graphs.
RDF4J has really nice documentation and supports a lot of modern features. The getting started guide is great [1], and there's also a super friendly gitter room to ask questions [2].
1: https://rdf4j.org/documentation/tutorials/getting-started/
2: https://gitter.im/eclipse/rdf4j
I have read this sentiment countless of times, but I don't quite understand it. Maybe because I am not familiar with the semantic web. Can you please suggest some reading list for a person who is unfamiliar with semantic web to better understand these issues?
The gist of it is to provide a way to model data as a graph, and make it almost trivial to exchange data between parties. A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that.
The "pie in the sky" part is that, from this basic building block, a whole host of other technologies can be built, some of them dealing with artificial intelligence, machines being able to _infer_ more information from what you have already said [2]. Inferences can work best when there's more data, that's were the ability to mix your data with third party databases comes handy.
Solid adds even more stuff to the table. In their words: derived from "social linked data", is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. I think "decentralized" is the keyword here. I haven't even started looking at it :-)
1: https://www.w3.org/TR/rdf11-primer/
2: https://www.w3.org/standards/semanticweb/inference
What solution does RDF provide for that? I know of `owl:sameAs`, but even if it were feasible to look up those statements and reason about them (assuming someone reliable went to the effort to actually define them, and did so correctly), that already breaks down if I have to match a single field "name" (or, say, vcard:fn) with two fields "first name" and "last name" (or, say, foaf:givenName and foaf:surName). So as far as I know, there's still no practical solution - or is there?
To be able to say that two subgraphs describe the same thing, at some point, a mapping is needed between the different sub graphs (for users could be, say, a common schema:email statement). A good practice is to use some existing ontology when modeling (say, FOAF or schema.org), this can fix the problem of having to map fields since both sources would already be using the same conventions.
I think what's normally done is some sort of normalization pass or passes after ingesting the triples, so at the end of the process your app can always use your preferred property (say vcard:fn) or just ignore the subgraph in the worst case. There are a number of tools to perform this kind of normalization.. I know of SPARQL construct queries but I've seen others out there (rml.io).
- Having to create your own mapping for every piece of data of every divergent data source, with a not-insignificant amount of work involved?
- Those mappings inevitably containing mistakes/different interpretations that result in invalid data?
- Some of the data not being amenable to lossless mapping in the first place?
In other words: is "RDF provides a solution for that" not overselling it a bit?
But I think the integration story is way better than when integrating other data models, say, relational databases.
The other aspect is that URLs being the ids for statements, I can peek at the implementation of a 3rd party data model and use that _in advance_, instead of integrating later.
For instance, when I write my blog I can use the URLs of existing wikidata entities to model my own entities. If wikipedia entities used a relational model, the IDs of these entities would be opaque and could change at any time, so I wouldn't be able to do that.
They only consider a website from their viewpoint – a human viewing the website through browser. The HTML (including microdata, JSON-LD, etc) is terrifying for most of them. So at best, their web designer builds in some semantic web standards but the data can be inaccurate or become out of data quickly because the website owner doesn't even know it's there.
Even big companies have adopted semantic web standards in such a confusing and inconsistent way that it's difficult to rely on the data being available in an expected format, so that a developer can build something on top of it.
Unless semantic web standards can be adopted by small businesses in a simple way (point, click and type) I don't think they will ever be widely adopted.
Finally, I don't think it's efficient to hide the 1kb of machine usable data inside 99kb of HTML designed for browser rendering.
Shameless plug: A small team and I have built an alternative to the semantic web, to store machine readable data in DNS. It can be populated by simple user friendly online forms.
Explainer and summary: https://www.num.uk
Best way to play around with it: https://app.numserver.com/tools/editor/add
Although it's primarily known for allowing apps to store their users' data in a place under their users' control, for better or worse, it's built on many SemWeb technologies.
And even though I'm still somewhat disillusioned about the Semantic Web (like Aaron Schwarz, I used to be enthusiastic about it way back when), but there certainly is a chance it will be interesting again when your personal (potentially private) data can be part of the Semantic Web as well. For example, if they'd have written their reviews in a Solid-based website, linking them to MusicBrainz would be a relatively small step.
- There are several organisations behind Solid that whose raison d'etre is Solid. I'd argue that Mozilla possibly pulled the plug for Persona too early, specifically, without first adding support for it in Firefox, thereby creating massive incentive for website owners to support it. But then, it wasn't their core focus, and there was no return on investment for them, so I can see why they cancelled it.
- Ironically, a wider focus. The incentive for website owners to support Persona would have been that it would be easier for potential users to join - although they already got most of that benefit by allowing sign-in with Facebook, and most people did not actually use Persona. Solid, on the other hand, has several incentives to support it. It can give organisations access to data that the user is actively curating, rather than having to provide it to different organisations individually, with it getting outdated the moment they shared it. Additionally, given how, for many organisations, data is increasingly (post-GDPR and all that) a liability and not their primary competency, outsourcing management of user data to the user themselves makes sense. And finally, there will hopefully be more people with a Solid Pod, especially in specific markets - for example, IIRC the government of Flanders was looking at giving every citizen a Pod.
- It can lean on the marketing benefits of it being TimBL's main focus, which may or may not be stronger than being one of Mozilla's projects.
- Depending on your definition of success, it can be as successful as Mastodon/the Fediverse is. As opposed to Persona, people can and do write apps specifically to support Solid, because it's more than just a login.
Just off the top of my head, and again, personal views. But hopefully that gives some insight.
First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers.
Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI.
If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.
The human element defies any solution other than social coordination such as a specs registry or org. At some point the software needs to understand each other and that requires people to work together. A technical solution around schemas can assist the coordination work, but it can't replace it.
EDIT to add: Since we can't remove human coordination, I question the value of arbitrary schema composition when it incurs the huge DX cost of using a URL to represent every attribute.
https://pierrelevyblog.com/my-research-in-a-nutshell/
https://github.com/IEMLdev/ieml
Interesting ideas, would highly recommend checking it out!
https://pierrelevyblog.com/my-research-in-a-nutshell/
What if the unscalability of the semantic web were a feature rather than a bug?
The current political and economic fallout of the big data web that we all know and love arises precisely due to the ungodly effects of network size on power laws. E.g., the rich get richer and the poor get poorer on a global scale.
Why not a more semantic and a more decentralized web.
What comes to mind is people training AI to parse web pages and mark them up to indicate different types of data inside of them. Such as the recipe example.
So someone who authors a WordPress plugin for recipes could incorporate it. Or, and this is more scalable but obviously very difficult, it may be possible to create plugins for web frameworks that automatically inject the semantic markup using AI. Maybe.
Then there could be another AI program that converts natural language queries into semantic knowledge queries.
But I think that AI could play a big part in rolling it out. I mean, if you had the AI systems without the semantic web, for open systems you would still want a way for them to share data unambiguously.
In addition spam will continue to be a significant problem.
We can have semantic social networking now, https://github.com/patchworkuniversity/radiojade