I've said it before, I'll say it again. The semantic web is a pipedream. It's never, ever going to happen. The whole idiotic scheme hinges on the notion that people are going to spend a whole bunch of extra time adding intelligent metadata to their data. Never. Going. To. Happen.
I find it far more likely that we'll have built a practical AI before the semantic web ever happens, at which point all semantic efforts will be obsolete.
Of course, if there's anything more masturbatory than the semantic web, it's microformats. Hey! You there! Don't link to a vcard object, that's yesterday's deal. Instead, painstakingly reproduce your vcard in XML! Yeah, that's a good idea, isn't it. Lets take a widely supported data format that many people already have, and turn it in to a completely unsupported, poorly specified fantasy format. Then you can maintain two copies of the same data! Endless fun.
It's just a symptom of the general web malaise. Am I crazy or was it once possible for real human beings to write web pages? You know, before you had to remember 90 pages of how CSS worked? The W3C these days seems to be where small minds go to die, and even TBL isn't helping.
The average Joe seems to be moving towards a web presence based on a blogging platform or a social network or a CMS. If app providers build semantically and enterprises or large data set publishers build semantically, you could conceivably end up with a reasonably semantic web for content of value.
I think the idealists are on to something, but as usual they are whacking off over the thought that one day everything will be semantic. That's the pipedream to which you rightly refer. What I said above will not take place for many home-spun sites or apps or businesses that can't justify the cost with added value or increased revenue.
It's perfectly reasonable for, say, a CMS to semantically mark up when an article was posted. That's free. The server knows the date, so why not? It's not hurting anyone. Similarly, a camera with a GPS chip could easily add LAT/LONG EXIF metadata to every photo, except for the power, that's free too. You could use that to answer queries like "give me all photos taken in Laos in the past year", for example. Nothing wrong with that.
Where it breaks down is that there isn't enough free metadata to make the semantic web worthwhile. It relies on people actually saying "This is an address I'm typing right now... this is the street component of the address... This is the City..." It's ridiculous. Certainly, there'll be special purpose applications making use of free metadata in the future, but the whole web being semantic will never happen.
Google Maps seems to do a pretty good job of parsing addresses. Given a database of say, US city names, state names and zip codes, could an API not be built that would recognize something like that and wrap it in a microformat either by a stupid simple API call or just as an auto-expanding template type feature?
I agree with you that the average user (which includes business, etc.) will not do it significantly unless it's effortless so I wonder if app platforms or frameworks can make it more effortless in this way. That combined with some level of justified business value in some sectors could be our best stab at a practically, usefully, partially semantic web.
Being pragmatic, I'm not one to say that unless the whole web is semantic then we give up. I'm also not one to believe the whole web will be semantic or that all of it really needs to be.
If you had a machine that could recognise addresses with reasonable accuracy and zero false positives, you wouldn't NEED microformats or the semantic web. That's the whole point, smart humans spending their time compensating for the stupidity of machines. It's a boil the ocean strategy that only makes any kind of economic sense of consider your users peons whose time has no value. Spending time making machines smarter is an investment in the future, the semantic web is an admission of defeat.
Also, Google doesn't parse addresses at all, it uses its index data the same way it does spell checking.
Not all websites are basically just documents with a management interface, however.
Remember, there are an awful lot of sites with database back ends that store structured data. The technical difficulty of emitting that structured data out into a web page using a microformat (or RDFa, or whatever) is going to be pretty low, so if it's useful (for SEO or anything else), I expect it will start happening.
What use is semantic metadata that is generated automatically? The whole point of "semantic" is that it's created by humans to describe the data; you can't get around the "extra work" requirement without defeating the uniqueness of the semantic tags. And if the extra work still stands, the average Joe will not bother with it. Hell, I wouldn't bother with it, in any form.
I hear what you're saying but I don't think it's as easy to dismiss as that. There are ways that microformats or semantic data could easily start showing up in websites. If a web framework, like say Rails, supported microformat creation based on your models then it wouldn't cost the developer or user any more time to markup that data. All of a sudden your movie app starts automatically inserting semantic movie info about the actors, director, etc which can link back to authoritative sources.
I'm not saying that it WILL happen, but I think it is definitely doable. With a framework like Rails your schema is already available to the program. If we just had one more generator (or more likely a gem for each supported microformat) then you could get semantic data in your app for almost free.
Well, imagine you're right. What then? Once you have all the magical, microformatted metadata what is it exactly you do with it?
The canonical semantic web idea is that you build ontologies and search them. Now you have a new problem. How do you know which metadata is trustworthy? You'd be a fool to think people aren't going to attempt to game the system. If I search for "all articles about Lucy Liu", how is it that your semantic web search engine filters articles that are really about Lucy Liu from those that are merely marked up to make them appear so? You've just substituted a problem that requires as much intelligence as adding the metadata in the first place. Hell, what am I saying? Trust is a far harder problem. Even Google haven't cracked that one yet.
So now that I've answered the 'How do you do it question' I have to also explain what would be done with it? Alright that's fine.
Yahoo is already parsing microformats out of it's search results, so if two sites are otherwise equal in content and my site has microformat information about the data I'm going to get a better relevancy ranking from Yahoo than you are, which translates to more hits and more money. More data beats better algorithms, I've seen that on this site about a dozen times in the last few weeks but when the 'more data' is overhyped like the Semantic Web everyone decides that it's totally pointless.
Your other point, that 'people will game it' is an amazingly bad reason to not do something. Think if Google had looked at PageRank and said:
"yeah that's cool but all people have to do is make a link farm and they can game the system, making a new search engine is pointless."
With the level of intelligence and Hackertude that is available on this site I'm pretty astounded that there are so much vocal opposition to the Semantic Web. You're not going to beat Google in standard search but something like this could give a startup a leg-up in the market place. Maybe we should start looking at ways to answer the questions instead of just throwing our hands up in the air and saying that it won't work.
I think all of you have missed an important point. Sure, purists like me use microformats all the time, and you're right when you say it's going to take more than that.
You're missing the advantage for the business people. It isn't just going to be the standardists that will care. As web developers, it's our job to explain the technology, why it matters, and more importantly, how they can benefit from it. "Get ahead of your competition by using semantic information, start future-proofing your web site."
I recently built a website for a client, filled with microformats. Advantage? I cut down on development time because I could re-use CSS from previous jobs, pretty much just copy-and-paste. I used hCards to markup people's contact info, and as a result I could create a simple link to Technorati's hCard conversion service. It means I'm maintaining one copy of the data (instead of creating an XML format too), and I can add extra functionality with no extra effort. Google now use microformats to add location addresses to it's search results. For a small-to-medium business, this is great!
Also, Standardists will typically keep things semantic anyway, so you're not really asking them to change their methods, just giving them a nice easy format to markup their format.
Businesses will adopt the technology, especially if us web developers explain to them the advantages. Plus, they get to use a whole bunch of nifty buzzwords at the next big meeting.
To me, semantic markup is about making reusable patterns of information that relate to the content. Can I harness that to my advantage in CSS? Of course. It's really no different than saying "I'm going to make every sidebar I ever make have a class called 'aside'." Is technique grounds for issues over ownership?
Technique is probably not an issue but "copying and pasting" work you've done for another client is likely problematic. Well, hopefully your clients aren't the kind that get litigious over these things but I still wouldn't go around telling everyone that I'm reusing work from previous jobs.
1) Well, that depends what you want to call "next version". If you called "Web 2.0" next version, then you can call the Semantic Web next version.
2) He was obviously talking about the direction of the technology, not implying that "everything" as it is now is going to become semantic. I think that was pretty clear.
3) Same line here. "Standards" are never going to fully be implemented, it doesn't mean that they mean nothing. Semantic web is just one step up. Most standards-based websites are designed semantically anyway, plus from a back-end perspective, REST is on the rise.
Yahoo, Flickr, Google, Apple, Microsoft and Technorati are using microformats already. They're all pushing the technology, they have plans as to how they want to use them. It's up to the developers to come up with solutions as to how to make them easier for the general public to use.
1) Personally, I don't, nor do I count either it or "Semantic web" as tangible, exact things. I just meant to point out that you can use "next version" to anecdotally refer to a shift in trend, which was how it was used in this case.
2) That which the bigger companies are pushing, which ultimately relates to a not insubstantial section of the web to follow suit.
3) You're absolutely right that microformats haven't been marketed yet. If it's news to you, have an exploration around the microformats wiki. You'd be surprised who's doing what.
If you're interested in some examples of the companies I mentioned:
- Yahoo uses hCalendar, hCard, hReview on almost all business listings, search results, events, and reviews.
- Flickr supports XFN and hCard on all profile pages, and use geo on their photos with geo tagged photos.
- Google use microformats to enhance their search results. I believe that XFN in particular plays a part in determining link relevance and they certainly use rel-nofollow. Across various Google products you will find hCards, XFN, rel-license and so on.
- Apple's .Mac webmail supports hCard.
- Microsoft are supposedly doing some big things with IE8 and microformats. They already use hCalendar in their WindowsLiveWriter.
- Technorati started a lot of the microformats. Basically no part of their site isn't covered with examples of them. They have a variety of services that you can use to do clever things with the microformats you use (such as their hCard to vCard converter - add a link to your page which when clicked parses the hCard and turns it into a vCard for easy addition to your address book).
Aside from these, you can add Twitter, Digg, Dreamweaver, Drupal, Facebook, Last.fm, Moveable Type, Pownce and Wordpress to the list of microformat users.
If everything were available in microformats, wouldn't that dramatically lower the barrier to entry for search startups who want to be the next Google?
That said, wouldn't it be in Google's best interest to keep microformats at bay? Natural language processing + strong AI is how Google stays on top, no?
"The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw
I see half-and-half solutions as more possible: slightly smarter markup and much smarter web spiders would reap about as much benefit as a fully semantic web. I mean, just marking headers properly in <h\d> tags rather than <font size="\d"> and quotes in <quote> is a pretty decent hint for sniffing out semantically important text, and having grammar parsers for named entities like proper names and addresses and dates and such that are as forgiving with that text as BeautifulSoup is with the terrible HTML out there is a good amount of work in the right direction. I worked on a research project doing just that for two years and you can get a LOT of useful information out of a lot of modern HTML pages just by applying some simple statistical/AI magic to it. Anything more sophisticated would be near impossible without a breakthrough bordering on miracle.
Hey, it goes both ways. Mark section headers with <h%d> tags instead of <font size="foo"> or <div style="foo">, but don't use <h%d> tags where all you want is formatting. In the same way, use <em> instead of <i> only when the italics are actually for emphasis, as in this sentence. In other cases, such as denoting internal thoughts in a story or something like that, don't use <em>.
I publish data in microformats whenever I get the chance. It's not hard, it doesn't take long and the added classes aren't doing any harm.
If microformats drift away and the possibilities never eventuate, what harm has been done? It's just semantic markup.
The reason is that microformats are a perversion of CSS/HTML. CSS is not markup; it is to define visual style. I think it rather humorous that web "developers" rejoiced at the leaps and bounds HTML made when it went from HTML 4.01 to XHTML 1.0. One of the reasons is that browsers made up what ever HTML tags they saw fit. XHTML provided a way to define a standard set of tags and attributes and give application developers the means to extend them without breaking other peoples code. This includes semantic descriptions.
Most of the arguments for using microformats are forgetting that many smart people have already, more or less, solved the content description issue. Thats what the X in XHTML is. By specifying that XHTML is valid XML, you can abide by all the rules for XML... mainly the extensible portion. You can add custom attributes to any tag you feel. There are DTDs, XSD, RelaxNG, Schemetron, etc; DTDs and XSD seem to have won the battle over document definitions. The only arguments against custom XHTML attributes are becoming more and more irrelevant. Arguments based on browser support for custom XSD or DTD documents are already irrelevant because most browsers have, or will, add support for them as a requirement to support novel things like XSLT. Arguments that DTDs and Schemas are two difficult are not really arguments. They are difficult for a reason, semantic definition is difficult. Many of the problems with microformats, i.e. namespace collisions, have been "solved" in the various XML standards. There are already many, many tools for parsing and using XML including namespaces, XIncludes, XSD, DTD, XSLT, etc... However, the tools to make microformats actually usable and useful, do not exist. Other arguments about custom attributes not being XHTML Strict valid are also irrelevant. Adding custom attributes by definition shouldn't be strict valid. Strict validity is checked against the xhtml1-strict.dtd DTD. Not being XHTML strict valid isn't a bad thing as long as the document is still valid XML and a schema/DTD document exists to prove it. This is, in my opinion, a major flaw in microformats. There is no source of validity. Anyone can use any thing for microformat descriptions. There is no perception of validity what so ever.
Most of the arguments for microformats, especially coming from the community tend towards ignorance of details. When you don't fully understand the details of the XML roots of XHTML, you tend to try and hack it or reinvent the wheel. Really diving into XML technology beyond XHTML sheds insight and vision on where the XML trend is moving, why, and where XHTML can follow along.
The only questionably valid arguments are for the current landscape not fully supporting XHTML+XML. However, as I have stated, that is quickly becoming a non-issue. You can also argue that XML wasn't meant to be the solution to the semantic web... but I ask, why isn't it? Tell me why a language designed to mark-up any data cannot describe the semantics of that data?
Coming from a non-techie perspective, the reason I'm not using it is because its all way too confusing. I mean, have you read an article about this stuff? Everyone defines it in different ways, hardly anyone can explain with specificity what exactly it will do for you, and its very hard for anyone to found out information about what to do when or where. By that I mean, no one has told me what I can do as a non-programmer. Business people won't start using it until there is a specific advantage to doing it, but no one can point that out yet. Furthermore, I haven't been able to find a way to figure this out without hiring someone.
For example, I have a TypePad blog. So someone explain what I'm supposed to do, how it will help me, and how I will do it. In a way that doesn't involve coding. I dare you.
34 comments
[ 3.1 ms ] story [ 81.3 ms ] threadI find it far more likely that we'll have built a practical AI before the semantic web ever happens, at which point all semantic efforts will be obsolete.
Of course, if there's anything more masturbatory than the semantic web, it's microformats. Hey! You there! Don't link to a vcard object, that's yesterday's deal. Instead, painstakingly reproduce your vcard in XML! Yeah, that's a good idea, isn't it. Lets take a widely supported data format that many people already have, and turn it in to a completely unsupported, poorly specified fantasy format. Then you can maintain two copies of the same data! Endless fun.
It's just a symptom of the general web malaise. Am I crazy or was it once possible for real human beings to write web pages? You know, before you had to remember 90 pages of how CSS worked? The W3C these days seems to be where small minds go to die, and even TBL isn't helping.
I think the idealists are on to something, but as usual they are whacking off over the thought that one day everything will be semantic. That's the pipedream to which you rightly refer. What I said above will not take place for many home-spun sites or apps or businesses that can't justify the cost with added value or increased revenue.
It's perfectly reasonable for, say, a CMS to semantically mark up when an article was posted. That's free. The server knows the date, so why not? It's not hurting anyone. Similarly, a camera with a GPS chip could easily add LAT/LONG EXIF metadata to every photo, except for the power, that's free too. You could use that to answer queries like "give me all photos taken in Laos in the past year", for example. Nothing wrong with that.
Where it breaks down is that there isn't enough free metadata to make the semantic web worthwhile. It relies on people actually saying "This is an address I'm typing right now... this is the street component of the address... This is the City..." It's ridiculous. Certainly, there'll be special purpose applications making use of free metadata in the future, but the whole web being semantic will never happen.
I agree with you that the average user (which includes business, etc.) will not do it significantly unless it's effortless so I wonder if app platforms or frameworks can make it more effortless in this way. That combined with some level of justified business value in some sectors could be our best stab at a practically, usefully, partially semantic web.
Being pragmatic, I'm not one to say that unless the whole web is semantic then we give up. I'm also not one to believe the whole web will be semantic or that all of it really needs to be.
Also, Google doesn't parse addresses at all, it uses its index data the same way it does spell checking.
Remember, there are an awful lot of sites with database back ends that store structured data. The technical difficulty of emitting that structured data out into a web page using a microformat (or RDFa, or whatever) is going to be pretty low, so if it's useful (for SEO or anything else), I expect it will start happening.
I'm not saying that it WILL happen, but I think it is definitely doable. With a framework like Rails your schema is already available to the program. If we just had one more generator (or more likely a gem for each supported microformat) then you could get semantic data in your app for almost free.
The canonical semantic web idea is that you build ontologies and search them. Now you have a new problem. How do you know which metadata is trustworthy? You'd be a fool to think people aren't going to attempt to game the system. If I search for "all articles about Lucy Liu", how is it that your semantic web search engine filters articles that are really about Lucy Liu from those that are merely marked up to make them appear so? You've just substituted a problem that requires as much intelligence as adding the metadata in the first place. Hell, what am I saying? Trust is a far harder problem. Even Google haven't cracked that one yet.
Yahoo is already parsing microformats out of it's search results, so if two sites are otherwise equal in content and my site has microformat information about the data I'm going to get a better relevancy ranking from Yahoo than you are, which translates to more hits and more money. More data beats better algorithms, I've seen that on this site about a dozen times in the last few weeks but when the 'more data' is overhyped like the Semantic Web everyone decides that it's totally pointless.
Your other point, that 'people will game it' is an amazingly bad reason to not do something. Think if Google had looked at PageRank and said: "yeah that's cool but all people have to do is make a link farm and they can game the system, making a new search engine is pointless."
With the level of intelligence and Hackertude that is available on this site I'm pretty astounded that there are so much vocal opposition to the Semantic Web. You're not going to beat Google in standard search but something like this could give a startup a leg-up in the market place. Maybe we should start looking at ways to answer the questions instead of just throwing our hands up in the air and saying that it won't work.
You're missing the advantage for the business people. It isn't just going to be the standardists that will care. As web developers, it's our job to explain the technology, why it matters, and more importantly, how they can benefit from it. "Get ahead of your competition by using semantic information, start future-proofing your web site."
I recently built a website for a client, filled with microformats. Advantage? I cut down on development time because I could re-use CSS from previous jobs, pretty much just copy-and-paste. I used hCards to markup people's contact info, and as a result I could create a simple link to Technorati's hCard conversion service. It means I'm maintaining one copy of the data (instead of creating an XML format too), and I can add extra functionality with no extra effort. Google now use microformats to add location addresses to it's search results. For a small-to-medium business, this is great!
Also, Standardists will typically keep things semantic anyway, so you're not really asking them to change their methods, just giving them a nice easy format to markup their format.
Businesses will adopt the technology, especially if us web developers explain to them the advantages. Plus, they get to use a whole bunch of nifty buzzwords at the next big meeting.
Take your time, what's the rush?
1) Who says there will be a "next version of the web"
2) Who says it will be "about" anything? How can a decentralized communications network be "about" something? What planet are you from?
3) See jamess's comment
As for microformats, it's mainly been embraced by navel-gazing bloggers who don't write much code if at all.
2) He was obviously talking about the direction of the technology, not implying that "everything" as it is now is going to become semantic. I think that was pretty clear.
3) Same line here. "Standards" are never going to fully be implemented, it doesn't mean that they mean nothing. Semantic web is just one step up. Most standards-based websites are designed semantically anyway, plus from a back-end perspective, REST is on the rise.
Yahoo, Flickr, Google, Apple, Microsoft and Technorati are using microformats already. They're all pushing the technology, they have plans as to how they want to use them. It's up to the developers to come up with solutions as to how to make them easier for the general public to use.
2) What direction?
3) I code for a living and it's news to me that Yahoo et al. use microformats in some capacity. The point is that the marketing isn't there.
2) That which the bigger companies are pushing, which ultimately relates to a not insubstantial section of the web to follow suit.
3) You're absolutely right that microformats haven't been marketed yet. If it's news to you, have an exploration around the microformats wiki. You'd be surprised who's doing what.
If you're interested in some examples of the companies I mentioned:
- Yahoo uses hCalendar, hCard, hReview on almost all business listings, search results, events, and reviews.
- Flickr supports XFN and hCard on all profile pages, and use geo on their photos with geo tagged photos.
- Google use microformats to enhance their search results. I believe that XFN in particular plays a part in determining link relevance and they certainly use rel-nofollow. Across various Google products you will find hCards, XFN, rel-license and so on.
- Apple's .Mac webmail supports hCard.
- Microsoft are supposedly doing some big things with IE8 and microformats. They already use hCalendar in their WindowsLiveWriter.
- Technorati started a lot of the microformats. Basically no part of their site isn't covered with examples of them. They have a variety of services that you can use to do clever things with the microformats you use (such as their hCard to vCard converter - add a link to your page which when clicked parses the hCard and turns it into a vCard for easy addition to your address book).
Aside from these, you can add Twitter, Digg, Dreamweaver, Drupal, Facebook, Last.fm, Moveable Type, Pownce and Wordpress to the list of microformat users.
Until that day, or the day Google dies, there's not much point.
That said, wouldn't it be in Google's best interest to keep microformats at bay? Natural language processing + strong AI is how Google stays on top, no?
"The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw
Most of the arguments for using microformats are forgetting that many smart people have already, more or less, solved the content description issue. Thats what the X in XHTML is. By specifying that XHTML is valid XML, you can abide by all the rules for XML... mainly the extensible portion. You can add custom attributes to any tag you feel. There are DTDs, XSD, RelaxNG, Schemetron, etc; DTDs and XSD seem to have won the battle over document definitions. The only arguments against custom XHTML attributes are becoming more and more irrelevant. Arguments based on browser support for custom XSD or DTD documents are already irrelevant because most browsers have, or will, add support for them as a requirement to support novel things like XSLT. Arguments that DTDs and Schemas are two difficult are not really arguments. They are difficult for a reason, semantic definition is difficult. Many of the problems with microformats, i.e. namespace collisions, have been "solved" in the various XML standards. There are already many, many tools for parsing and using XML including namespaces, XIncludes, XSD, DTD, XSLT, etc... However, the tools to make microformats actually usable and useful, do not exist. Other arguments about custom attributes not being XHTML Strict valid are also irrelevant. Adding custom attributes by definition shouldn't be strict valid. Strict validity is checked against the xhtml1-strict.dtd DTD. Not being XHTML strict valid isn't a bad thing as long as the document is still valid XML and a schema/DTD document exists to prove it. This is, in my opinion, a major flaw in microformats. There is no source of validity. Anyone can use any thing for microformat descriptions. There is no perception of validity what so ever.
Most of the arguments for microformats, especially coming from the community tend towards ignorance of details. When you don't fully understand the details of the XML roots of XHTML, you tend to try and hack it or reinvent the wheel. Really diving into XML technology beyond XHTML sheds insight and vision on where the XML trend is moving, why, and where XHTML can follow along.
The only questionably valid arguments are for the current landscape not fully supporting XHTML+XML. However, as I have stated, that is quickly becoming a non-issue. You can also argue that XML wasn't meant to be the solution to the semantic web... but I ask, why isn't it? Tell me why a language designed to mark-up any data cannot describe the semantics of that data?