If the runtime was the main issue, couldn't you just mod jekyll to only regen based on new data (modding site.process basically)...obviously if you changed templating information you'd want to regen everything, but....
Regarding the time it takes to build the site - is everything rebuilt every time in Jekyll?
You'd think there would be some sort of dependency build tree ala make/rake so only items whose source components would need to be rebuilt (which, in some cases could be the whole site).
It has to regenerate the entire site each time. I guess the reason for this might be because it can't be sure every page is updated otherwise. Take for example adding a new post and you have a "recent posts" widget in your sidebar. This affects every page on the site and therefore it must be rebuilt.
Is parsing a page template on the server and including a 'recent posts' snippet really so challenging that it justifies every client doing another HTTP request for that very same snippet?
Dependencies are hard to figure out automatically and a bit of a nuisance if you'd have to declare whether every plugin/script use has dynamic side effects. Considering that jekyll is a pretty small (and somewhat opinionated) system by default, it probably wasn't considered worth the effort.
And once you approach the level of a more typical WordPress site in both volume and information per page, I'm not sure how much effort you'd save. It's probably better to investigate how to make the general run time a bit faster. (Although it would probably be possible to have mostly static pages with few dependencies, then make the dynamic/dependent changes via a minimal web service and AJAX. At least when the load is too much for that, you'd still see the static content. Wouldn't be as generic/small as jekyll then, though.)
One more "geekCred--;" for letting WordPress (or whatever) convert two '-' to a single '–'.
More on topic, though: I don't think anyone is seriously claiming that both solutions are equivalent or easily interchangeable. For me the simple and 'process the files once' approach makes a lot of sense and even seems more transparent than a big beast like WordPress. Whether Jekyll or Blosxom - something that feels less like a CMS and more like a way to write simple texts.
I can't tell from the site itself, since it's non-responsive (back to WordPress?), but from the comments here it appears the author objects to Jekyll regenerating the entire site every time.
If that's the case then I suggest he tries Hakyll, which has a dependency system built in so that it only rebuilds what it needs when files change.
haha ironic that it's crashing now eh? That's the one upside of Jekyll for sure. I was hosting the site on 512MB of ram. Just did a resize to handle the HN traffic, hopefully it comes back online soon.
Hakyll looks pretty slick. I'll check it out for sure. Thanks
Have you implemented any of the caching tools for WordPress? I also run WordPress on my blog, which is hosted at Dreamhost (shared hosting), which is about the worst you can have. I've never been hit terribly hard, but I did manage to pick up around 2,000 visitors in the span of about 3 hours one time, which is the most my blog has ever seen. WP with WP SuperCache kept it up, even on a shared host.
I'd be curious to read a post-mortem once you get the site back up.
I had my wordpress blog on shared Dreamhost for about a year.. I was slowly increasing traffic then I noticed the dreamhost daemons killing off my processes.. after much, much hassle I finally got the offical line of "1000 unique visitors per day is using too much memory for Dreamhost shared - you need to upgrade".
I moved to rackspace.
Caching with WordPress.org is as easy to set up as Jekyll, if not easier. With W3 Total Cache or WP Super Cache, performance is roughly equivalent to a baked site being hosted by the same web server.
Edit: Oops, I lost, as he already said in another thread that he had WP Super Cache turned on. But it seems to me that 512MB would be enough if the web server was set up properly, and in fact caching was set up so the web server would serve pages directly.
Maybe I didn't get something about this static blog generating trend thing, but what would be wrong about having a 'normal' database-backed blog software that simply renders the pages staticly to a file (as it is an option with Rails caching) ? That would work with some simple inexpensive DB like SQLite, but still leave the system flexible and easy to edit with a backend.
I use wp-super-cache for this. However, it's still not the same as static site generation. The entire Wordpress system still gets loaded at least to the plugin loading phase.
There are numerous db hits that take place before this stage (mainly to figure out which plugins need to be loaded).
Also, the PHP must be interpreted in order to render html. The static site generation has _none_ of this as it's already pre-rendered in html.
Regarding your use of Markdown, I had a similar experience.
I've had Markdown on my WordPress install for many years, but only recently decided to commit to learning the main commands off by heart, and use it whenever possible. Markdown's not perfect, but committing to it has sped up writing a lot.
There are some solid Markdown graphical text editors too, if you're into that sort of thing. Most of these allow you to preview your documents while writing them, which is a big plus.
I developed MarkdownPad (http://markdownpad.com) for Windows, and there's also Mou for Mac and Retext for Linux.
I am amused by the site being down. Going back to WordPress, he presumably forgot that a cache plugin is pretty much required. Not that Jekyll is perfect, but that's one area where it's pretty much impossible to beat.
I actually have wp-supercache enabled. However, the site was only running on 512MB ram :( Getting featured on HN is a lot different than daily traffic. A static site would not have saved me here.
Indeed. I've been on the HN front page a couple of times with Jekyll generated sites (http://os-blog.com is one). Hosted on an Amazon micro instance (613 mb) and served by nginx, none of my sites have broken a sweat under the increased traffic.
HN traffic isn't really that much. I was watching the stats under the new Google Analytics realtime mode and the site was averaging about 3 - 6 requests per second.
I remember when everybody moved on to WordPress while I stayed on MovableType 2.6. I finally moved one year and a half later, when my 400+ posts made the building time absolutely unbearable.
Are there any Jekyll blogs with hundreds or thousands of posts?
I am about to move my personal blog over to blogofile (like Jekyll, but written in Python) which has about 500 or so entries. Will let you know how long they take to build when done...
I like that I can run my Jekyll blog for free on Heroku, and it is served entirely out of Varnish. Seems like you are paying more money for crap performance.
In addition, my blog regenerates in < 5 seconds on Ruby 1.9.3.
OK, so obviously I suck at server configuration/optimization. Any tips for optimizations that could have enabled me to support the influx of Hacker News traffic?
Note:
* Ubuntu 512MB on Rackspace
* Apache
* Wordpress with wp-super-cache and minimal plugins enabled
I had a similar issue a while back after having a blog post of mine featured on HN front page for some time. The problem is really apache. Either configure apache with workers (which is ain't the default config) or add nginx in front. You might be interested in this particular post:
A week ago, Jack Baty, also having been allured by static site generation hotness, switched back to Wordpress after it took 8 minutes to regenerate his Jekyll blog with over 1,600 posts. "Life's too short."
At the moment, Jekyll (via Octopress) is more interesting/educational to me than plugging away at familiar Wordpress, so I'll cross that bridge when I get there.
If it's purely about generating the whole site every time you make a change, then try using the `--limit-posts` option on jekyll.
If you set it to 1, it just re-generates the most recent post, which is usually what you're working on anyways.
If that doesn't work, the Octopress has a `rake isolate` command that stashes everything but the post you ask for in a separate directory. `rake integrate` puts you back to normal.
48 comments
[ 8.2 ms ] story [ 124 ms ] threadYou'd think there would be some sort of dependency build tree ala make/rake so only items whose source components would need to be rebuilt (which, in some cases could be the whole site).
And once you approach the level of a more typical WordPress site in both volume and information per page, I'm not sure how much effort you'd save. It's probably better to investigate how to make the general run time a bit faster. (Although it would probably be possible to have mostly static pages with few dependencies, then make the dynamic/dependent changes via a minimal web service and AJAX. At least when the load is too much for that, you'd still see the static content. Wouldn't be as generic/small as jekyll then, though.)
More on topic, though: I don't think anyone is seriously claiming that both solutions are equivalent or easily interchangeable. For me the simple and 'process the files once' approach makes a lot of sense and even seems more transparent than a big beast like WordPress. Whether Jekyll or Blosxom - something that feels less like a CMS and more like a way to write simple texts.
1: http://www.blosxom.com/
[1]: http://daringfireball.net/projects/markdown/
If that's the case then I suggest he tries Hakyll, which has a dependency system built in so that it only rebuilds what it needs when files change.
http://jaspervdj.be/hakyll/
Hakyll looks pretty slick. I'll check it out for sure. Thanks
I'd be curious to read a post-mortem once you get the site back up.
Caching with WordPress.org is as easy to set up as Jekyll, if not easier. With W3 Total Cache or WP Super Cache, performance is roughly equivalent to a baked site being hosted by the same web server.
Edit: Oops, I lost, as he already said in another thread that he had WP Super Cache turned on. But it seems to me that 512MB would be enough if the web server was set up properly, and in fact caching was set up so the web server would serve pages directly.
http://wordpress.org/extend/plugins/wp-super-cache/
There are numerous db hits that take place before this stage (mainly to figure out which plugins need to be loaded).
Also, the PHP must be interpreted in order to render html. The static site generation has _none_ of this as it's already pre-rendered in html.
I've had Markdown on my WordPress install for many years, but only recently decided to commit to learning the main commands off by heart, and use it whenever possible. Markdown's not perfect, but committing to it has sped up writing a lot.
I developed MarkdownPad (http://markdownpad.com) for Windows, and there's also Mou for Mac and Retext for Linux.
You can see more from his own comments when he posted it on Hacker news[3].
[1] http://tom.preston-werner.com/2008/11/17/blogging-like-a-hac...
[2] http://www.allthingsdistributed.com/2011/08/Jekyll-amazon-s3...
[3] http://news.ycombinator.com/item?id=2896860
(you can run WordPress locally to generate the site, or use a 3rd party service just to run the process but not serve from it).
The hacker solution that the Chief Technology Officer of Amazon uses is much more a "power to the people" approach.
I gave the WPEngine link just to prove people are doing it with WordPress and it's not a pipedream.
The author obviously don't want to compile the site as it takes too long for him, so compiling a wordpress blog would not yield any time benefits.
And while Jekyll still has some downsides it is superior when it comes to handling a load spike from a HN post.
Wordpress cacheing HN traffic should do fine too. There's something else going on here.
HN traffic isn't really that much. I was watching the stats under the new Google Analytics realtime mode and the site was averaging about 3 - 6 requests per second.
Are there any Jekyll blogs with hundreds or thousands of posts?
http://paulstamatiou.com
In addition, my blog regenerates in < 5 seconds on Ruby 1.9.3.
Note:
* Ubuntu 512MB on Rackspace
* Apache
* Wordpress with wp-super-cache and minimal plugins enabled
Thanks for the feedback!
* Run PHP with php-fpm (bundled in latest PHP 5.3)
* Use a PHP accelerator like APC.
http://blog.rassemblr.com/2011/01/wordpress-need-for-speed-o...
This has been throughly tested with yet another post on HN front page a while later! My blog handled the load without a glitch.
Looks like I'm replying with the same comment over and over again. But I have to share nginx love :-)
http://jackbaty.com/2011/10/new-blogging-hotness/
At the moment, Jekyll (via Octopress) is more interesting/educational to me than plugging away at familiar Wordpress, so I'll cross that bridge when I get there.
If you set it to 1, it just re-generates the most recent post, which is usually what you're working on anyways.
If that doesn't work, the Octopress has a `rake isolate` command that stashes everything but the post you ask for in a separate directory. `rake integrate` puts you back to normal.