This is a fascinating article, but I can't help thinking that I wish they were more technically incompetent so it harmed their business. The Daily Mail has to be one of the most repugnant media organizations and if there is ever a justification for using the word 'evil' in relation to an institution then they are it.
Apparently they realised that there's zero chance of the two readerships ever intersecting, so they let their guard down. It's pretty shocking, even to someone who already reviles them.
My own thoughts were along the the lines of "what a waste of talent on such a scumbag publication".
"Tables are co-located, meters away from the news desk ‘the customers’, meaning it only takes minutes if something breaks for the communication between the developer and the user."
Meh, can you imagine sitting across from journalistic horrors like Melanie Phillips or Peter Hitchens, and being torn between fixing your shit quickly or doing the nation a favour and delaying publication of their right wing tripe from poisoning the weak minded for just a few minutes/hours longer.
Their journalists hold their opinions because they can earn a good living by doing so, not because they particularly believe them.
> The Mail attracts writers, who ought to oppose it, because it pays them top rates on one condition only: they say exactly what the editor wants them to say. You can get at least £1,000 for a morning’s work, and Dacre will fill your pockets even if he decides not to use your piece. Writers will bark like a performing seal for money as easy as that.
That sounds familiar. I know somebody who worked on our student paper that swallowed their beliefs to go and work for the Mail because it was the only paper still hiring. Not believing in it doesn't absolve you from being complicit in it, though.
I actually have more respect for someone who is a genuine right-wing nutjob and tells it like it is than someone who abandons their principles and sells out. As Marina (of 'and The Diamonds') so eloquently put it "better to be hated / than loved, loved, loved for what you're not".
My point was that if you are making a decision explicitly on salary and interesting technology it was a good choice. I'm sorry if this was not clear.
Were you asking me to explain my point because you find the existence of a rational explanation for intelligent people working for the company extremely offensive?
It's a common fallacy to assume that political beliefs are connected to intelligence.
You should always consider the possibility that there are smart people who hold different political/ethical positions to you.
The engineers that joined might not consider the publication evil, and they might not like the grossly-offensive insinuation by you that they are like western prostitutes in Saudi Arabia (?!).
> The Daily Mail has to be one of the most repugnant media organizations
Hm, I don't know. I sort of respect them, in a way. In some ways I think of them as the authentic voice of the impotent lower middle classes, lashing out at everything around them, greedy, cynical, childish and shallow, blaming others, ever seeking some outraged scandal to distract them from their otherwise mundane lives. Hate the Mail if you will, but you're basically hating their audience too.
I don't like the Mail, but at least they're independent, seemingly serving no-one's agenda but their own. Murdoch is much worse, in my opinion.
I hate the Mail because it encourages the worst in its audience. I have to believe that if the people that read it and I could reach a compromise if we weren't being so polarized by their reading material.
Being "greedy, cynical, childish and shallow, blaming others" are all vices at the end of the day (FYI despite the tone of my comment here, I'm actually not religious).
I agree totally that Murdoch is worse, but I think the Mail does hold more sway in a vocal area of the population with political power. The fact that they never acknowledge their hypocrisy or the damage they do I think is also offensive on an intellectual level that the Sun et al never quite reach as they're so cartoon-like.
I agree with everything you wrote. But the problem is, what can anyone do to fix it?
I mean it's easy to come up with a plan to "fix" Murdoch. Simply tighten media ownership rules, so that one family or organisation can't own more than a certain small percentage of media. And I take your point that the Sun is pretty clownish in the UK, but in Australia they're far more insidious. (¶)
But how do you fix the Mail? They're like this perfect endgame of free press. There's no big conspiracy; they're simply the answer to the question: "what does the angry disenfranchised demographic want to read about?". And while I lament the means by which said class seeks to assuage its anger, I do sympathise that they have a lot to be angry about.
All this is why I find it hard to wrestle with the question of the Mail. I want to hate them, but some part of me holds back, because I feel that's too close to simple class discrimination, and because I can't shake the feeling that they're the symptom, not the cause, of society's ills.
As a fully paid-up member of the impotent lower middle class, and a fervent opponent of pretty much everything the daily mail stands for, I resent that implication!
Maybe the core audience of the daily mail IS lower middle class, but only the segment of the lower middle class that is intolerant and reactionary.
Yeah, I shouldn't have really worded it like that. It's difficult making characterisations and I went a bit too far. I do believe, however, that there's a segment of the middle classes who have a lot of dissatisfaction with their general lot, and the Mail is a way for them to vicariously lash out at "the system", johnny foreigner, the rich, the paedophiles, and all of those other easy targets.
I'm going to leave what I wrote as is and reap what I sow, but regret putting it in such a nasty tone now. Sorry!
That hardly qualifies as a 'diatribe.' Also, no need to get so defensive, their comments against the Mail weren't directed at you personally. While I'm offering advice, thicker skin is helpful when on the interwebs.
>I think of them as the authentic voice of the impotent lower middle classes, lashing out at everything around them, greedy, cynical, childish and shallow, blaming others...
Odd, "greedy, cynical, childish and shallow, blaming others" sounds like an accurate representation of a large chunk of the elites and 1 percerters, it must be a species thing.
It's interesting, I know several engineers who've worked at (or currently work at) the Mail Online and they all talk about the 'Daily Mail Tax' salary bump they required in order to slice off a little bit of their soul to work there.
Technical recruiters I know talk about it too, how they describe this very cool roll their trying to fill, deliberately obfuscating who it's for, before eventually having to reveal it and usually loosing most of their potentials.
I'm curious, can you be more specific about why they are "repugnant" and "evil?" Preferably something longer than 318 characters and shorter than the rambling 8,500 word New Yorker essay.
"From a content management perspective, writing code in a very functional way is very low friction vs traditional OO models of building content management systems". It sounds like "using letters is very low friction vs using numbers". A new silver bullet/hammer?
>It sounds like "using letters is very low friction vs using numbers".
No, it sounds like "using flexible, isolated, easily changable functional components and avoiding side-efects" is very low friction vs "designing and creating an OO class hierarchy, with all the tedium it entails in most common languages".
I think I misunderstood him but it did sound like OOP vs functional :). What he meant was probably something like "we stopped writing bad code with lots of mutable state and caught up with modern best-practices in OOP".
My theory goes, it's part and parcel of phase 3 in the language adoption cycle: it's now what a lot of people learn straight out of college.
It's really popular and quickly maturing - so lots of cool stuff that pays money is being built with it - but it hasn't crystallized into a morass of legacy work. Oh shucks, version 3 came out but it's largely incompatible with 50% of existing libraries, blah blah.
If your main experience developing software came from tweaking wordpress or uni C++/Java 101 then that's totally a reasonable feeling to have - "my goodness I am so much more productive".
Excellent submission. I wish it went into some more detail on how they integrate those mini apps into the site or handle their failure but it was probably already long enough.
This
a Table also rebuilt the entire legacy Java frontend replacing the entire 155,000 lines of have with about 4000 lines of Clojure
was particularly impressive since a Table is just a small team of up to six people.
I also skimmed the front page of http://www.dailymail.co.uk and don't understand the allergic reactions in the comments here. A guy built a hobbit hut, some celebrity gossip, sports, more gossip... rather unremarkable.
For a website this big (32k pixels according to the article) it does load pretty quickly. Faster than a product page on Amazon (at least for me from Central Europe).
I believe the use of the phrase "all grown up" juxtaposed with loud headlines about the evils of paedophilia is one such reason the Daily Mail is reviled.
There is a difference between the Daily Mail newspaper, which is a scare mongering right wing hate rag which has as much relation to truth and news as say Fox News, and its website which is pretty much a celeb stuff and sort of a soft porn peddler.
Its a weird pairing under the same brand. I don't know, but I don't think there is much crossover between the users of the paper and website. Personally, I find both hideous, but Im at a loss to accept that some one who gets all up set about gay marriage, immigrants, and the alleged loss of Englishness, what ever that is or was, is in any way interested in a Kardashian thing.
It's very difficult to hold any extreme view - right or left - without begin hypocritical[1]. The DM just excels at it.
[1] And, at an individual level, without encouraging the trashier elements of the media to expend enormous amounts of energy to expose those hypocrisies, however significant.
Well, sort of, the web site is the newspaper's hypocrisy. No?
You are correct, the DM does excel at what it does. On the face of it, the impression I get is that the paper does have a consistent vibe. I mean, I despise its views, but I do have to admit that it is very good at what it does. It is fairly rock solid in its opinions and politics. I would use lots of words to slag the paper off, but I don't think hypocrisy is one of them. But then maybe, people who read it, for either pleasure or pain, may well have many examples of hypocrisy.
I don't read it, but I am aware of a) its rabid obsession with pedophiles b) it's 'sidebar of shame' which frequently features very young women in various states of undress. That, I call hypocrisy.
A recent scandal in the UK happened when the Mail publicly called a leading politician's father 'evil' as he was a Marxist. He was also a respected academic at a leading institution and fought for Britain against the Nazis, whereas the Mail was pro-appeasement and its owner friends with Hitler. Without wishing to appeal to Godwin's law, it's that kind of bald-faced moral pronouncement (with no regard to institutional hypocrisy) on subjects from immigration to gay rights that makes the Mail so divisive.
I think programmers these days focus too much on having whizz bang technologies.
90% of successful project implementation is culture, good practices and so forth.
I've seen well run cobol projects, and badly run projects using all the latest trends.
It doesn't really matter if your software is using Java, ruby or node.js. I can guarantee you from a business perspective, success does not rely on that. I much rather have well tested, well maintained Java Spring Code base, then a badly tested, unreadable clojure codebase.
The success of the story is that they probably went from a bad codebase to a good one. And you can in any language, node.js or not.
Except maybe for the 4th paragraph. Using node.js instead of PHP cut my server costs to 20% of what it was before (simply because I can run the same thing on less number of cheaper servers). Lower cost means more profit and I count profit under "success from business perpective". ;)
Jerry Weinberg has an essay on fads in computer programming[1]. Basically, it's written such that you can swap in whatever today's hottest fad is, and the article still makes sense (it was originally written for Structured Programming, but OO Programing, Agile Development, Node.js, et al would fit well within it).
One of his conclusions is that the organizations that are likely to succeed with the latest whiz-bang technologies are also the ones who were most likely to succeed without them. Sure, what's new can be better, but it's hardly sufficient.
Based on my experience, I would agree, and so I think you're right. With the exception of this being specific to "programmers these days". I think our industry has always been plagued by this trend. It's just that now, with the internet, the stories have a longer reach.
The entire backlog of 1.2 million articles are stored in elastic-search, which is used to drive the site
Elasticsearch is awesome and I'd heard rumors of people using it as a data store rather than just a search index but it's awesome to see it being used as a data store at such scale.
At the last place I worked, we used ES for both search and storage of 2.5MM+, and quickly growing, items (the data was also stored in DynamoDB for safety). It worked quite well, saved us some development time, and reduced the latency of search requests a little since there was no 2nd network call to fetch the documents found by ES from separate storage.
What are the basic tradeoffs at that scale? I'm only using it for search at a very low scale but it almost sounds like a dream database if it works well at scale and for canonical storage.
>Enter Fred George (of Programmer Anarchy fame). Fred came on board as Chief Architect, and focused on supporting the organisational move from traditional enterprise technology team to a nimbler, highly skilled organisation built around a strong culture of personal responsibility and trust.
I think all the other details in this write-up are not as important as this. The specific tech choice isn't the driver, it is this cultural shift. This doesn't mean the tech choice doesn't matter though. In fact, I would argue that the most rapid way to build such a culture in your organization is to self-select for people who would do well in that environment. I hate painting people with too broad a brush, but my experience has been that choosing safe, established technologies as your stack will cause the sort of people who like safe, predictable work to show up for jobs. It can be hard (though certainly not impossible) to recruit the alpha geeks and the people with a more entrepreneurial perspective. There are plenty of super-talented people who have very pedestrian technologies as their preferred platforms. But if you're looking to mix things up quickly and go for a more nontraditional environment, the shortcut to getting that is to choose tech that is beloved by people who primarily want to work in a nontraditional manner. The sort of person who dabbles with go or node in their spare time does it not because they expect that it's going to be their meal ticket to a cushy job doing software development at a bank somewhere. They do it because they have got intrinsic motivation to learn new things and be on the cusp of the latest tech out of sheer enjoyment.
it’s not possible to serve a 32,000 pixel webpage to a mobile device
I reject this prejudice.
The biggest annoyance I have with browsing websites on mobile devices is that they nearly always disable zoom.
Zoom is the probably the single easiest way to navigate large documents, because it lets the user scale effective panning speed. Without zoom, you're forced to pan at a fixed speed, and if the document is very long, it takes forever. But if you can zoom out, you can pan much much faster, and focus in on interesting bits of content much more easily. Many many times I've given up on websites because I wasn't able to get to the desktop view on my phone.
As to the infeasibility of a 32k page? The Nexus 7 tablet has the same resolution as my 24" main monitor. My phone isn't far behind. If anything, the question should be if 32k pages are infeasible for desktop machines, not mobiles.
It's a interesting presentation but can anyone tell me why you would build an analytics tool and an advertising tool when there are already tools out there to do this? (Chartbeat, Doubleclick, etc...)
Maybe there have very specific use cases or have grown to the point were it makes economic sense to rebuild these but seems a bit redundant.
59 comments
[ 2.9 ms ] story [ 108 ms ] threadApparently they realised that there's zero chance of the two readerships ever intersecting, so they let their guard down. It's pretty shocking, even to someone who already reviles them.
"Tables are co-located, meters away from the news desk ‘the customers’, meaning it only takes minutes if something breaks for the communication between the developer and the user."
Meh, can you imagine sitting across from journalistic horrors like Melanie Phillips or Peter Hitchens, and being torn between fixing your shit quickly or doing the nation a favour and delaying publication of their right wing tripe from poisoning the weak minded for just a few minutes/hours longer.
> The Mail attracts writers, who ought to oppose it, because it pays them top rates on one condition only: they say exactly what the editor wants them to say. You can get at least £1,000 for a morning’s work, and Dacre will fill your pockets even if he decides not to use your piece. Writers will bark like a performing seal for money as easy as that.
http://blogs.spectator.co.uk/nick-cohen/2013/10/who-is-the-g...
That + being able to offer people a greenfield project using Clojure and Node.JS probably attracted great engineers.
Were you asking me to explain my point because you find the existence of a rational explanation for intelligent people working for the company extremely offensive?
It's a common fallacy to assume that political beliefs are connected to intelligence.
You should always consider the possibility that there are smart people who hold different political/ethical positions to you.
The engineers that joined might not consider the publication evil, and they might not like the grossly-offensive insinuation by you that they are like western prostitutes in Saudi Arabia (?!).
Hm, I don't know. I sort of respect them, in a way. In some ways I think of them as the authentic voice of the impotent lower middle classes, lashing out at everything around them, greedy, cynical, childish and shallow, blaming others, ever seeking some outraged scandal to distract them from their otherwise mundane lives. Hate the Mail if you will, but you're basically hating their audience too.
I don't like the Mail, but at least they're independent, seemingly serving no-one's agenda but their own. Murdoch is much worse, in my opinion.
Being "greedy, cynical, childish and shallow, blaming others" are all vices at the end of the day (FYI despite the tone of my comment here, I'm actually not religious).
I agree totally that Murdoch is worse, but I think the Mail does hold more sway in a vocal area of the population with political power. The fact that they never acknowledge their hypocrisy or the damage they do I think is also offensive on an intellectual level that the Sun et al never quite reach as they're so cartoon-like.
I mean it's easy to come up with a plan to "fix" Murdoch. Simply tighten media ownership rules, so that one family or organisation can't own more than a certain small percentage of media. And I take your point that the Sun is pretty clownish in the UK, but in Australia they're far more insidious. (¶)
But how do you fix the Mail? They're like this perfect endgame of free press. There's no big conspiracy; they're simply the answer to the question: "what does the angry disenfranchised demographic want to read about?". And while I lament the means by which said class seeks to assuage its anger, I do sympathise that they have a lot to be angry about.
All this is why I find it hard to wrestle with the question of the Mail. I want to hate them, but some part of me holds back, because I feel that's too close to simple class discrimination, and because I can't shake the feeling that they're the symptom, not the cause, of society's ills.
(¶) Check out the Murdoch press's outrageous bias in the recent Australian election: http://www.theguardian.com/commentisfree/2013/aug/05/daily-t...
Maybe the core audience of the daily mail IS lower middle class, but only the segment of the lower middle class that is intolerant and reactionary.
I'm going to leave what I wrote as is and reap what I sow, but regret putting it in such a nasty tone now. Sorry!
Which is another reason I have a kind of grudging respect for them. You'll never read a word against the 1-percenters in the Murdoch press.
Technical recruiters I know talk about it too, how they describe this very cool roll their trying to fill, deliberately obfuscating who it's for, before eventually having to reveal it and usually loosing most of their potentials.
Dunno where we get the numbers, but perhaps you do and can knock the claim down. A nation might thank you for doing so.
http://en.wikipedia.org/wiki/List_of_newspapers_in_the_world...
Edit: Saw now that the list above doesn't show numbers for the last couple of years, but in the UK The Sun is still ahead as of January this year:
http://en.wikipedia.org/wiki/List_of_newspapers_in_the_Unite...
"Associated Newspapers network remains world's most popular newspaper website with more than 99m monthly browsers"
This is published by the Guardian, the Mail's left wing equivalent.
http://www.theguardian.com/media/2012/feb/23/mail-online-clo...
Ouch!
The original article says: "Every month they service over 150 million unique visitors"
This article from Forbes(2011) claims that The New York Times is the leader, followed by Times of India.
http://www.forbes.com/sites/kenrapoza/2011/09/05/tech-savvy-...
The article was based on Alexa ranking, which have changed since:
http://www.alexa.com/siteinfo/dailymail.co.uk
http://www.alexa.com/siteinfo/nytimes.com
http://www.alexa.com/siteinfo/china.com.cn
I guess they are using Alexa rankings to support those claims.
EDIT: If we use Alexa, the Huffington Post ranks above:
http://www.alexa.com/siteinfo/huffingtonpost.com
127.0.0.1 dailymail.co.uk www.dailymail.co.uk
This ensures that there is no possibility of my giving any traffic to this utterly repugnant shitfest of a newspaper.
I would encourage everyone to do the same.
PS: I like functional programming a lot.
No, it sounds like "using flexible, isolated, easily changable functional components and avoiding side-efects" is very low friction vs "designing and creating an OO class hierarchy, with all the tedium it entails in most common languages".
It's really popular and quickly maturing - so lots of cool stuff that pays money is being built with it - but it hasn't crystallized into a morass of legacy work. Oh shucks, version 3 came out but it's largely incompatible with 50% of existing libraries, blah blah.
If your main experience developing software came from tweaking wordpress or uni C++/Java 101 then that's totally a reasonable feeling to have - "my goodness I am so much more productive".
This
a Table also rebuilt the entire legacy Java frontend replacing the entire 155,000 lines of have with about 4000 lines of Clojure
was particularly impressive since a Table is just a small team of up to six people.
I also skimmed the front page of http://www.dailymail.co.uk and don't understand the allergic reactions in the comments here. A guy built a hobbit hut, some celebrity gossip, sports, more gossip... rather unremarkable.
For a website this big (32k pixels according to the article) it does load pretty quickly. Faster than a product page on Amazon (at least for me from Central Europe).
Its a weird pairing under the same brand. I don't know, but I don't think there is much crossover between the users of the paper and website. Personally, I find both hideous, but Im at a loss to accept that some one who gets all up set about gay marriage, immigrants, and the alleged loss of Englishness, what ever that is or was, is in any way interested in a Kardashian thing.
[1] And, at an individual level, without encouraging the trashier elements of the media to expend enormous amounts of energy to expose those hypocrisies, however significant.
You are correct, the DM does excel at what it does. On the face of it, the impression I get is that the paper does have a consistent vibe. I mean, I despise its views, but I do have to admit that it is very good at what it does. It is fairly rock solid in its opinions and politics. I would use lots of words to slag the paper off, but I don't think hypocrisy is one of them. But then maybe, people who read it, for either pleasure or pain, may well have many examples of hypocrisy.
http://cdn.recombu.com/digital/images/news/M10326/1335269711...
90% of successful project implementation is culture, good practices and so forth.
I've seen well run cobol projects, and badly run projects using all the latest trends.
It doesn't really matter if your software is using Java, ruby or node.js. I can guarantee you from a business perspective, success does not rely on that. I much rather have well tested, well maintained Java Spring Code base, then a badly tested, unreadable clojure codebase.
The success of the story is that they probably went from a bad codebase to a good one. And you can in any language, node.js or not.
Except maybe for the 4th paragraph. Using node.js instead of PHP cut my server costs to 20% of what it was before (simply because I can run the same thing on less number of cheaper servers). Lower cost means more profit and I count profit under "success from business perpective". ;)
One of his conclusions is that the organizations that are likely to succeed with the latest whiz-bang technologies are also the ones who were most likely to succeed without them. Sure, what's new can be better, but it's hardly sufficient.
Based on my experience, I would agree, and so I think you're right. With the exception of this being specific to "programmers these days". I think our industry has always been plagued by this trend. It's just that now, with the internet, the stories have a longer reach.
[1] http://secretsofconsulting.blogspot.com/2011/06/beyond-agile...
Elasticsearch is awesome and I'd heard rumors of people using it as a data store rather than just a search index but it's awesome to see it being used as a data store at such scale.
I think all the other details in this write-up are not as important as this. The specific tech choice isn't the driver, it is this cultural shift. This doesn't mean the tech choice doesn't matter though. In fact, I would argue that the most rapid way to build such a culture in your organization is to self-select for people who would do well in that environment. I hate painting people with too broad a brush, but my experience has been that choosing safe, established technologies as your stack will cause the sort of people who like safe, predictable work to show up for jobs. It can be hard (though certainly not impossible) to recruit the alpha geeks and the people with a more entrepreneurial perspective. There are plenty of super-talented people who have very pedestrian technologies as their preferred platforms. But if you're looking to mix things up quickly and go for a more nontraditional environment, the shortcut to getting that is to choose tech that is beloved by people who primarily want to work in a nontraditional manner. The sort of person who dabbles with go or node in their spare time does it not because they expect that it's going to be their meal ticket to a cushy job doing software development at a bank somewhere. They do it because they have got intrinsic motivation to learn new things and be on the cusp of the latest tech out of sheer enjoyment.
I reject this prejudice.
The biggest annoyance I have with browsing websites on mobile devices is that they nearly always disable zoom.
Zoom is the probably the single easiest way to navigate large documents, because it lets the user scale effective panning speed. Without zoom, you're forced to pan at a fixed speed, and if the document is very long, it takes forever. But if you can zoom out, you can pan much much faster, and focus in on interesting bits of content much more easily. Many many times I've given up on websites because I wasn't able to get to the desktop view on my phone.
As to the infeasibility of a 32k page? The Nexus 7 tablet has the same resolution as my 24" main monitor. My phone isn't far behind. If anything, the question should be if 32k pages are infeasible for desktop machines, not mobiles.
Maybe there have very specific use cases or have grown to the point were it makes economic sense to rebuild these but seems a bit redundant.