I think the big challenge will be with the languages themselves. LESS, Stylus, CoffeeScript, and Jade are all implemented in JavaScript. Finding C implementations would be tough. Though SASS has a C implementation.
"The obvous question with building front-end applications is where does the state live? Fortunately there are many services ..." Does any body know what is a great solution for simple Contact Forms?
This doesn't seem to be on the right level of abstraction - why replacing the entire web server, adding dynamic translating (compilation) as a main feature?
I look at it this way, we basically have two types of web servers, static and full-stack web frameworks. What if all you want is a static web server but you need a way for your files to share a common layout? If you don't have any server side compilation a full-stack framework is overkill. We could use a static site generator such as Jekyll and then put behind a static web server but now we have ugly urls, no 404 pages, and way more dependencies than needed.
The simple fact is a static web site on its own is not good enough but its 95% there. Using a full-stack framework is overkill, and needing an Static Site Generator just seems counter productive when considering simplicity is one of the biggest upsides to shipping static assets.
There is no good reason for static web servers to do a little extra for you like smarter redirects, clean urls, layouts/partials. This makes sense to me considering the direction we are headed.
Partly because few if any static generators talk to clients over a TCP socket; mostly, I suspect, because people tend to hear "static site generator" and immediately think "oh, boy, here's another goober who's going to try to push his bizarre little pile of shell scripts on me."
Harp does work great as a Static Site Generator but the reason why we avoid that term is that there have been several cases of people using Harp as a SSG by running the compile step each time they save a file. This is how they expected it to work (likely because of their experience with Jekyll). It has even gone so far as someone requesting a way to "preview" the compiled output not realizing that not only can harp serve a preview, it removes the need to generate the compiled output all together.
It really comes down to the mental modal of how people look at the tool and how they expect it to behave. Avoiding the term SSG has helped with that.
I have been using simple Node HTTP server to fulfill my small need[http://stackoverflow.com/a/13635318]. And of course, Node's event-driven architecture will help Harp. But, I think Nginx or Apache is quite featured web server you are trying to replace.
I like the idea, but I don't understand the problem they're trying to solve. For example I have an app using node as the backend behind nginx and ember as the front end.
Right now I'm using grunt to precompile my app to plan old HTML / JS /CSS but there are modules for node that can also compile and cache for me, Either way, I don't have to mess with something that works (NGINX)
Configuring Grunt to do this every time you want to ship a web app seems like a path to insanity to me. Also, NGINX doesn't by default give you clean urls and 404 fallbacks. Web frameworks do this for you and IMHO It's time static servers do as well.
Regenerating all the assets every time there is a change just feels sloppy to me. Also, I think it is asking a lot from novice developers to setup these configuration files and the knowledge one needs to have to do it effectively is a needless burden on someone who specializes on front-end development.
No disrespect to NGINX. Its too busy powering the internet to be concerned with these problems :)
This looks like it would be useful for a simple AngularJS app of mine where we use LESS, CoffeeScript, and Haml. We had written a simple Rack server to compile assets on demand. It'd be nice to get rid of the config.ru file and just use Harp as a drop-in replacement.
Nginx is 11 years old and faced similar skepticism from many people in 2002 who made statements about how feature-packed, stable and popular Apache was.
I've been using Harp for about 3 months and really like the sweet spot they have found between a bucket of static files and a full-blown web app stack. It's a very new paradigm and you probably need to try it to fully appreciate it.
So far we don't deal with the authoring abstraction layer. Our focus is on solving deployment by automating performance optimization like asset bundling, caching & CDN integration.
The big vision is to provide static hosting for the programmable web. We make HTML forms work and provide an API for sending the input wherever you want. This OAuth2 based API also handles deploying sites, injecting code snippets, and updating individual files.
The real moonshot here is developing a standard similar to JSON Patch for updating HTML via PATCH requests. If Harp, or anyone reading this is interested in this idea, we'd love to collaborate on developing a spec.
That Rails vs. Angular graph would sure be useful in arguing for a move away from Rails, except that it's unsourced and unitless on the Y axis. I'd love to know how it was generated and from what data, so I can make one that represents the same information in a way that's suitable for my purposes.
Not really. Those data points are independent. A lot of people use Angular and Rails together. Rails still provides a nice controller and model layer for an API.
I find it undesirably weighty for such purposes, and potentially limited in future prospects by the politics of its developer community. Decoupling the view layer from the backend makes replacement of the latter, with something offering an identical API but better prospects, easier to contemplate.
"Build on request" is a good idea. We do something similar in our stack: tup[0] as the (insanely fast) build system, and developer VMs run mitmproxy[1] on port 8080 which runs tup before every request.
I prefer this style of a setup to coupling them together a-la Harp. If the webserver doesn't work out, I can swap it out. Same with the build system. Tying them together makes me antsy. And for good reason I think: since 2004, we have switched from Apache -> Lighttpd -> Nginx, and have switched from custom shell scripts -> Make -> Tup (with some fabricate.py, and with calls out to other build systems like Leiningen)
I don't see this as tying it to the web server. You're not using some custom language that only Harp supports. If you need to switch to a completely static site, you can still use Harp to generate the static site. If you can't use Harp at all, there are several other static generators out there that support those formats.
Yes, the lazy compile (JIT) or "build on request" as you put it plays a huge factor when it comes to performance. Because it doesn't rebuild the world each time there is a change it is able to expend the cpu cycles when it matters most.
Under the hood we have the HTTP and the precompiling system decoupled. The precompiling is done with a library called Terraform which acts as a uniform source agnostic compiler. Harp wraps this with a HTTP layer and and SSG layer. See https://github.com/sintaxi/terraform
As a sidenote, I appreciate the sense of humor that the tup author has:
"In a typical build system, the dependency arrows go down. Although this is the way they would naturally go due to gravity, it is unfortunately also where the enemy's gate is. ... In tup, the arrows go up. This is obviously true because it rhymes."
"<diagram of tup vs make> See the difference? The arrows go up."
So this is like Apache's mod_* but for modern things. Sounds neat. One of the things I like most about the Play Framework is that it autocompiles LESS, CoffeeScript and Scala on refresh. Extending that concept to more languages is a good thing.
Reminds me of Middleman (http://middlemanapp.com/) and of my own https://github.com/stdbrouw/draughtsman, which has sadly languished. I've found it indispensable for prototyping, not just because of the precompilation but also because it'll search for metadata on the filesystem that you can use to flesh out those prototypes with real data.
middleman is fantastic. It does all this from what I can tell, and has an amazing plugin system, with plugins written for things like livereload so when I save a file, it re-pregenerates it and reloads my browser. It's SUCH a nice workflow especially for smaller sites.
Does Harp have livereload support? Kill all the refresh buttons (for dev)! That would be my only must have for using a different dev server than grunt based for JS app dev
NodeJS is plenty fast enough. When the dust settles we will provide benchmarks alongside NGINX. We're not pulling the wool over anyones eyes, I don't expect Harp to reach NGINX speeds anytime soon (or perhaps ever). That is not the focus of this server. Faster web server than NGINX is not what developers need. They need something that is easier to use, install, doesn't require any configuration, and more importantly is a suitable development tool on its own. Our focus are on the needs of front-end developers not Server operators.
We are publishing a spec for how Harp compatible servers should behave and we expect to see other implementations in the future.
Naming conflicts aside (what if your definition of "empty" doesn't match some other part of the code?), consider the `Array.prototype.empty` example. You extend `Array`, then some other bit of code anywhere else in the environment decides to do the following:
var pets = ["dog", "cat", "rabbit", "turtle", "owl", "alligator"];
for(var i in pets) {
console.log(pets[i]);
}
And what do you get?
dog
cat
rabbit
turtle
owl
alligator
function () {
return !this.length;
}
Oops! And here you could argue that the other coder should've known to use hasOwnProperty or forEach or whatever, but, all that aside, extending native object prototypes is an easy way to break code all over the environment.
> And here you could argue that the other coder should've known to use hasOwnProperty or forEach or whatever, but, all that aside...
I don't think you can dismiss that argument so easily. Other people modifying natives is the reason that if you're unsure of your environment, you shouldn't rely on `for(el in array/object)` without appropriate `hasOwnProperty` checks.
If you have control over your environment and the developers who will work in/with it (ex. the code for your website vs an npm module you plan to publish), then I see nothing wrong with naked `for (el in array/object)` or with extending natives if it's worth the added cost of developer overhead (they have to know that they're working with extended natives).
If you have to support IE 8, you probably shouldn't extend prototypes without serious thought, especially in a legacy codebase, because the interpreter support isn't there to do it right.
On the other hand, Windows XP is barely six months shy of EOL, and IE 8 currently has something like 4% share.
Now would seem to be an excellent time to start dropping support for IE 8. Such support is never going to gain you more users, and many other browsers run on Windows XP.
"Don't use Object.defineProperty because a dying browser on a dying platform doesn't support it" strikes me as a rather weak argument. I mean, there are still people out there using IE 6, God help them. Why not shoot as low as we can?
Naming conflicts aren't all that hard to resolve with a little forethought; if you're concerned about them, as you should be if you're thinking about extending prototypes in a codebase with which you're not intimately familiar, fire up your REPL and see what it returns for the name of the prototype extension you're thinking about adding. (If it's already there, that's one function you didn't have to write!)
As for the enumeration example, you're not making a point about extending the environment in general, but only about extending it wrong -- that is, by simply assigning functions to prototype slots, rather than using Object.defineProperty to do it right.
There are arguments to be made for writing style guides and programmer's cheatsheets with a half-competent audience in mind. All of those arguments are bad ones. The linked guide is generally reasonable, but I'd be a lot happier with it if it said "extending prototypes is usually a bad idea, but if you have to do it, here's how to stay off the landmines", rather than a superficially snicker-worthy insult to the reader such as "don't do this or you'll go to hell".
We use Harp to power our development platform that has Dropbox integration. It is currently in public Beta and I'll definitely be talking about it in the future.
There is no mention of it in my article because we didn't want our business to get in the way of the open source initiative. We believe in Tim O'Reilly's guidance "Create More Value Than You Capture".
One interesting LESS feature I found when we were evaluating it for letting some users use it for customization: it can evaluate arbitrary Javascript and open and embed files. Just in case you thought it was some pure transformative shorthand CSS syntax.
Of course, letting users upload arbitrary CSS served from your domain is an XSS-level risk, but let's say you let each user have his own domain and let them upload .less files.
Or you have a say, node.js APP that lets a user enter LESS and spit out the -- escaped -- CSS.
When compiled with lessc 1.4.2, this embeds the contents of that file as base64.
Also, with 1.3.x."@import" lets you open any filename and insert it, but it's required to be valid .less. I suppose you could try to @import /dev/random or urandom although I don't know if there's any practical attack surface for decreasing entropy. I don't think I could offhand find a valid file.
However in the theorethical web page evaluating .less, the error output will contain the contents of the file @imported.
Oh, and the evaluation of Javascript... I couldn't make it do much useful, as it seems there's some kind of limited JS environment that code executes in. The worst I could do was:
div {
background: `process.kill(-1, "SIGKIL")`;
}
The "fs" object does not seem to exist there, but I don't know enough about node.js/V8 environment to see whether there's equivalent of jailbreaks.
So, uh, don't compile arbitrary LESS files and send users their output, and LESS files someone sends you or checks into a project can kill your processes (at least).
In 2013, does it make sense to put each user in a docker/lxc container? I don't know the resource overhead, but I'd hope/expect that it would be within tolerances.
Then you can have rich features like includes without worrying about file-based info leak. (cpu-timing attacks are perhaps another kettle of fish).
For all skeptics, this can actually make lot sense with the current service-oriented web.
Actually, this is where I tried to go with Punch(http://laktek.github.com/punch) too. Vast majority of the content in web can essentially be static, it's just the minor inconveniences of plugging data sources, updating and managing them is what should be addressed.
79 comments
[ 3.4 ms ] story [ 195 ms ] thread[1]: https://github.com/chaoslawful/lua-nginx-module
http://spec.harp.io/
I think the big challenge will be with the languages themselves. LESS, Stylus, CoffeeScript, and Jade are all implemented in JavaScript. Finding C implementations would be tough. Though SASS has a C implementation.
The simple fact is a static web site on its own is not good enough but its 95% there. Using a full-stack framework is overkill, and needing an Static Site Generator just seems counter productive when considering simplicity is one of the biggest upsides to shipping static assets.
There is no good reason for static web servers to do a little extra for you like smarter redirects, clean urls, layouts/partials. This makes sense to me considering the direction we are headed.
It really comes down to the mental modal of how people look at the tool and how they expect it to behave. Avoiding the term SSG has helped with that.
Right now I'm using grunt to precompile my app to plan old HTML / JS /CSS but there are modules for node that can also compile and cache for me, Either way, I don't have to mess with something that works (NGINX)
Regenerating all the assets every time there is a change just feels sloppy to me. Also, I think it is asking a lot from novice developers to setup these configuration files and the knowledge one needs to have to do it effectively is a needless burden on someone who specializes on front-end development.
No disrespect to NGINX. Its too busy powering the internet to be concerned with these problems :)
Does Harp handle Haml?
I've been using Harp for about 3 months and really like the sweet spot they have found between a bucket of static files and a full-blown web app stack. It's a very new paradigm and you probably need to try it to fully appreciate it.
https://www.bitballoon.com
So far we don't deal with the authoring abstraction layer. Our focus is on solving deployment by automating performance optimization like asset bundling, caching & CDN integration.
The big vision is to provide static hosting for the programmable web. We make HTML forms work and provide an API for sending the input wherever you want. This OAuth2 based API also handles deploying sites, injecting code snippets, and updating individual files.
The real moonshot here is developing a standard similar to JSON Patch for updating HTML via PATCH requests. If Harp, or anyone reading this is interested in this idea, we'd love to collaborate on developing a spec.
I prefer this style of a setup to coupling them together a-la Harp. If the webserver doesn't work out, I can swap it out. Same with the build system. Tying them together makes me antsy. And for good reason I think: since 2004, we have switched from Apache -> Lighttpd -> Nginx, and have switched from custom shell scripts -> Make -> Tup (with some fabricate.py, and with calls out to other build systems like Leiningen)
[0] http://gittup.org/tup/
[1] http://mitmproxy.org/
Yes, the lazy compile (JIT) or "build on request" as you put it plays a huge factor when it comes to performance. Because it doesn't rebuild the world each time there is a change it is able to expend the cpu cycles when it matters most.
Under the hood we have the HTTP and the precompiling system decoupled. The precompiling is done with a library called Terraform which acts as a uniform source agnostic compiler. Harp wraps this with a HTTP layer and and SSG layer. See https://github.com/sintaxi/terraform
"In a typical build system, the dependency arrows go down. Although this is the way they would naturally go due to gravity, it is unfortunately also where the enemy's gate is. ... In tup, the arrows go up. This is obviously true because it rhymes."
"<diagram of tup vs make> See the difference? The arrows go up."
http://gittup.org/tup/tup_vs_mordor.html
Looks like an interesting project, though!
Just ran a spellcheck on it, man!
The most horrible thing about trying to build out Node.js applications these days is wishing for the Rails asset pipeline.
http://harpjs.com/docs/environment/lib looks like it should make it very possible to tie this up next to rendr and get the best of all worlds.
"We already have extreamly reliable"
Also, why compare RoR and AngularJS - completely irrelevant technologies.
We are publishing a spec for how Harp compatible servers should behave and we expect to see other implementations in the future.
They're 7 times slower than a static site server that I wrote in Go.
Forget comparing with Nginx.
Let the code breath a little -- no whitespace around control structures is a bit jarring:
Instead: Maybe adopt this Node style guide: http://nodeguide.com/style.htmlA strong statement whose motivation escapes me; it's not as though you can't trivially get the source of the extension in your REPL of choice.
I don't think you can dismiss that argument so easily. Other people modifying natives is the reason that if you're unsure of your environment, you shouldn't rely on `for(el in array/object)` without appropriate `hasOwnProperty` checks.
If you have control over your environment and the developers who will work in/with it (ex. the code for your website vs an npm module you plan to publish), then I see nothing wrong with naked `for (el in array/object)` or with extending natives if it's worth the added cost of developer overhead (they have to know that they're working with extended natives).
IE8 is the last version of IE for WinXP.
On the other hand, Windows XP is barely six months shy of EOL, and IE 8 currently has something like 4% share. Now would seem to be an excellent time to start dropping support for IE 8. Such support is never going to gain you more users, and many other browsers run on Windows XP.
"Don't use Object.defineProperty because a dying browser on a dying platform doesn't support it" strikes me as a rather weak argument. I mean, there are still people out there using IE 6, God help them. Why not shoot as low as we can?
Neither WinXP nor IE8 are dead there, not by a long shot.
As for the enumeration example, you're not making a point about extending the environment in general, but only about extending it wrong -- that is, by simply assigning functions to prototype slots, rather than using Object.defineProperty to do it right.
There are arguments to be made for writing style guides and programmer's cheatsheets with a half-competent audience in mind. All of those arguments are bad ones. The linked guide is generally reasonable, but I'd be a lot happier with it if it said "extending prototypes is usually a bad idea, but if you have to do it, here's how to stay off the landmines", rather than a superficially snicker-worthy insult to the reader such as "don't do this or you'll go to hell".
We use Harp to power our development platform that has Dropbox integration. It is currently in public Beta and I'll definitely be talking about it in the future.
There is no mention of it in my article because we didn't want our business to get in the way of the open source initiative. We believe in Tim O'Reilly's guidance "Create More Value Than You Capture".
For now, I use Wintersmith. It supports Jade and Markdown out of the box, and you can extend it by writing plugins.
I wrote one to help me "load" dependencies, without having to use <script> tags. I use Devon Govett's importer[1] for that.
[1] https://github.com/devongovett/importer
Of course, letting users upload arbitrary CSS served from your domain is an XSS-level risk, but let's say you let each user have his own domain and let them upload .less files.
Or you have a say, node.js APP that lets a user enter LESS and spit out the -- escaped -- CSS.
Now they can do:
When compiled with lessc 1.4.2, this embeds the contents of that file as base64.Also, with 1.3.x."@import" lets you open any filename and insert it, but it's required to be valid .less. I suppose you could try to @import /dev/random or urandom although I don't know if there's any practical attack surface for decreasing entropy. I don't think I could offhand find a valid file.
However in the theorethical web page evaluating .less, the error output will contain the contents of the file @imported.
Oh, and the evaluation of Javascript... I couldn't make it do much useful, as it seems there's some kind of limited JS environment that code executes in. The worst I could do was:
The "fs" object does not seem to exist there, but I don't know enough about node.js/V8 environment to see whether there's equivalent of jailbreaks.So, uh, don't compile arbitrary LESS files and send users their output, and LESS files someone sends you or checks into a project can kill your processes (at least).
Then you can have rich features like includes without worrying about file-based info leak. (cpu-timing attacks are perhaps another kettle of fish).
Actually, this is where I tried to go with Punch(http://laktek.github.com/punch) too. Vast majority of the content in web can essentially be static, it's just the minor inconveniences of plugging data sources, updating and managing them is what should be addressed.