I don't know if Go or Toggl fans rallied and upvoted this article, but I don't have anything with that.
That being said, I'd like to see if HN could provide some kind of fraud protection for upvotes/downvotes. I know that, for example, ad providers have some kind of algorithm detecting if the click is valid. Does anyone know of something like that in use in some of the community recommendation sites like HN?
As it is now, it would be fairly easy to game HN for karma or marketing reasons. All it takes for an article to take off is less than 10 votes at the right moment.
Site's been down a lot, and I'm not fond of the campaign tracking stuff in the original URL, so here's the text:
Moving to Go
Posted on September 19, 2012 [by Alari?]
During the last few months we have systematically improved both front-end user interface, and backend server code for the Toggl main time tracking page. It was motivated by the fact that we found it increasingly hard to cope with the steady growth of users and traffic, resulting in serious slowdowns of the system and in some cases even downtime.
The majority of Toggl program code originated from the time when our user base was 10x smaller, so the system needed a major overhaul. There were also several functional shortcomings, for example entries were not refreshed automatically when changes were made elsewhere (e.g. mobile phone). Also we had no offline support of any kind on the web.
Our situation in May this year
It took 5-7 seconds to load a time tracking page in Toggl, quite often even up to 20 seconds and more. Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data.
The whole backend was based on Ruby on Rails, we used Ruby 1.8 at that time.
User interface could not be used offline.
Javascript code was bloated and contained a lot of unnecessary code (for example we had multiple date parsing/formatting libraries, etc)
The loading and parsing of Javascript started to block our UI because of its size.
Backend API calls were not optimized for the purpose, we requested too much data which strained both the database and bandwidth.
New implementation
After careful consideration we decided to re-use the offline-enabled time tracking code that is also used in our mobile apps and in Toggl Desktop, only retaining the visual of the existing time tracking page. So basically we decided to replace the whole backend code of that page.
We had run some experiments with the Go programming language (http://golang.org/) before, and decided that we should re-implement some parts of our backend with this new platform. So far, it has paid off, as the development process was fast, and deployment surprisingly simple. The resulting code is a big improvement in terms of speed. We’ll continue to replace our backend code with Go.
Secondly we implemented a Redis-backed (http://redis.io/) WebSocket server in Go to enable realtime synchronization between different Toggl clients – so your time entries, projects etc. would be updated automatically if you use Toggl on multiple devices.
Thirdly we added HTML5 manifest and local storage to support offline usage of Toggl. This served also as the speed enabler. Offline was already implemented with our mobile interface m.toggl.com, so we reused a lot of that.
In frontend Javascript code, we’re moving to Backbone.js (http://backbonejs.org/). As the amount of Javascript code is increasing fast, this library enables to structure it better. For parsing, formatting and manipulating dates in Javascript, we’ve moved to Moment.js (http://momentjs.com/) It has simplified our code a lot.
Another update we made was to upgrade to Ruby 1.9. The upgrade gave the system another speed boost.
Finally we spent time measuring HTML/CSS/JS load times and optimizing the milliseconds there. Our goal was to get the load time to under 1 second, or even faster. While the tracking page now loads faster, it’s still a work in progress as we’re doing too many API requests when loading the timer. Also, we...
I'm curious about the LABjs usage. In my own testing it ends up not having any significant impact in load times, sometimes it's even slower (same for other loaders). Just concatenating+minifying files has a huge effect already, the added complexity doesn't justify itself.
Agreed. I recently moved from LABjs to RequireJS. I regret ever going down the LJS route because it really doesn't encourage / enable you to modularize your JS. It was over complicated to get my head around RJS, but once I did, it really cleaned up a lot of my code and I'm super glad that I made the switch.
If you look for the stream of articles that invaded HN in the last weeks, it is usually because someone implemented a web site in Ruby or Python and when it gets scalability problems all is heaven on earth with Go.
Of course if you're doing server side development already with more performant languages and runtimes, there is little incentive.
I switched from ruby to clojure because jvm is awesome
and it's a lisp and I also use coffescript, backbone,
requirejs and raphael. I've been using this stack for
about a month and a half, it's awesome.
Does this help? How does this help you solve your problem? I'm going to go with "It doesn't". All of these posts boil down to that form.
The good ones start with: "Over the last year we deployed [insert stack] into production..". And then tell me what sucked because not everything is awesome.
This is very cyclical. We're in the honeymoon period with some of these technologies. Two years ago you saw a lot of posts proclaiming that NodeJS and MongoDB were fantastic, world changing technologies.
Now you see a lot of posts saying "We're switching from MongoDB to Postgres/Riak because of X, Y, and Z". I suspect we'll start seeing similar posts about Node.js very soon.
No telling what will happen with Go, but just enjoy the honeymoon period for what it is and stay tuned to find out what happens later.
I've got nothing against Go, but this smells fishy. 5-20 second load times on a page in Rails? That doesn't sound like an issue with the programming language, it sounds like an underpowered server, lack of proper caching, or an under-tuned database (or data model).
The only interesting thing would be to see how much of a performance improvement each of the separate changes brought.
Phrases like "Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data." seems to hint at:
a) missing profiling (was it database queries, actual computation, network I/O, disk I/O, ... ?)
b) architectural problems (a database schema that didn't perform, missing indexes, ...)
I'm sure Go is a nice language, but I don't see how Go could improve any of those besides maybe a more managable approach to parallel computations.
Given the amount of seemingly parallel changes that happened, it's pretty much impossible to determine what actually caused the improvement (at least with the information in this blogpost).
A layman's question, I'm not a Ruby expert, but I was wondering - did you try jRuby and were not happy with the performance? or you already run on the JVM? My narrow notion on the Ruby world was that if it doesn't scale, put it on the JVM and you are done, am I living in fantasy land?
36 comments
[ 4.5 ms ] story [ 88.1 ms ] threadhttp://webcache.googleusercontent.com/search?q=cache:3vDKN5A...
That being said, I'd like to see if HN could provide some kind of fraud protection for upvotes/downvotes. I know that, for example, ad providers have some kind of algorithm detecting if the click is valid. Does anyone know of something like that in use in some of the community recommendation sites like HN?
As it is now, it would be fairly easy to game HN for karma or marketing reasons. All it takes for an article to take off is less than 10 votes at the right moment.
Seems like their switch to Go didn't go so well if HN entry with 7 upvotes can break it ;)
Site's been down a lot, and I'm not fond of the campaign tracking stuff in the original URL, so here's the text:
Moving to Go
Posted on September 19, 2012 [by Alari?]
During the last few months we have systematically improved both front-end user interface, and backend server code for the Toggl main time tracking page. It was motivated by the fact that we found it increasingly hard to cope with the steady growth of users and traffic, resulting in serious slowdowns of the system and in some cases even downtime.
The majority of Toggl program code originated from the time when our user base was 10x smaller, so the system needed a major overhaul. There were also several functional shortcomings, for example entries were not refreshed automatically when changes were made elsewhere (e.g. mobile phone). Also we had no offline support of any kind on the web.
Our situation in May this year
It took 5-7 seconds to load a time tracking page in Toggl, quite often even up to 20 seconds and more. Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data. The whole backend was based on Ruby on Rails, we used Ruby 1.8 at that time. User interface could not be used offline. Javascript code was bloated and contained a lot of unnecessary code (for example we had multiple date parsing/formatting libraries, etc) The loading and parsing of Javascript started to block our UI because of its size. Backend API calls were not optimized for the purpose, we requested too much data which strained both the database and bandwidth. New implementation
After careful consideration we decided to re-use the offline-enabled time tracking code that is also used in our mobile apps and in Toggl Desktop, only retaining the visual of the existing time tracking page. So basically we decided to replace the whole backend code of that page.
We had run some experiments with the Go programming language (http://golang.org/) before, and decided that we should re-implement some parts of our backend with this new platform. So far, it has paid off, as the development process was fast, and deployment surprisingly simple. The resulting code is a big improvement in terms of speed. We’ll continue to replace our backend code with Go.
Secondly we implemented a Redis-backed (http://redis.io/) WebSocket server in Go to enable realtime synchronization between different Toggl clients – so your time entries, projects etc. would be updated automatically if you use Toggl on multiple devices.
Thirdly we added HTML5 manifest and local storage to support offline usage of Toggl. This served also as the speed enabler. Offline was already implemented with our mobile interface m.toggl.com, so we reused a lot of that.
In frontend Javascript code, we’re moving to Backbone.js (http://backbonejs.org/). As the amount of Javascript code is increasing fast, this library enables to structure it better. For parsing, formatting and manipulating dates in Javascript, we’ve moved to Moment.js (http://momentjs.com/) It has simplified our code a lot.
Following Google Page Speed (https://developers.google.com/speed/pagespeed/) tips, we’ve started to use a Javascript loader to reduce resource blocking. At the moment, we’re using LABjs (http://labjs.com/).
Another update we made was to upgrade to Ruby 1.9. The upgrade gave the system another speed boost.
Finally we spent time measuring HTML/CSS/JS load times and optimizing the milliseconds there. Our goal was to get the load time to under 1 second, or even faster. While the tracking page now loads faster, it’s still a work in progress as we’re doing too many API requests when loading the timer. Also, we...
http://freemoviesite24.blogspot.com/2012/09/barfi-2012-strea...
Of course if you're doing server side development already with more performant languages and runtimes, there is little incentive.
Why would anyone senselessly troll HN?
The good ones start with: "Over the last year we deployed [insert stack] into production..". And then tell me what sucked because not everything is awesome.
Now you see a lot of posts saying "We're switching from MongoDB to Postgres/Riak because of X, Y, and Z". I suspect we'll start seeing similar posts about Node.js very soon.
No telling what will happen with Go, but just enjoy the honeymoon period for what it is and stay tuned to find out what happens later.
Phrases like "Most of the time was spent by the server to compile the necessary dataset of time entries, but also related data." seems to hint at:
a) missing profiling (was it database queries, actual computation, network I/O, disk I/O, ... ?)
b) architectural problems (a database schema that didn't perform, missing indexes, ...)
I'm sure Go is a nice language, but I don't see how Go could improve any of those besides maybe a more managable approach to parallel computations.
Given the amount of seemingly parallel changes that happened, it's pretty much impossible to determine what actually caused the improvement (at least with the information in this blogpost).