Using a fork per request is the easy way out. The problem as you've all seen, is that memory will be used up very very quickly.
The solution is to rewrite the app to use an event loop like select or epoll, or to queue the requests in a message queue: doable, but worth the effort? I doubt it, since the website looks like a hobby rather than a commercial product.
That is, if someone forks hurl (http://github.com/defunkt/hurl) and rewrites it to use one of the fixes you referred to above, I'm sure they'd merge that in and deploy that.
Doing it with Resque would make a lot of sense since hurl uses Redis already.
Yeah, I know. I just wanted to point out that we're not totally helpless here like we would be with closed-source software, and maybe encourage someone to fix it :)
I made myself a Flash app a while ago that creates POST requests and has a little profiles system. I would not be able to fix half the bugs I find without it.
Since both my app and this website have unavoidable network delays, I think I'll be retiring my program. That is, when the "Cannot allocate memory" bug gets fixed.
It's open sourced, and contains some great code: http://github.com/defunkt/hurl. I used it as an inspiration when working on my first Redis project using Sinatra.
Like this really: http://web-sniffer.net/ (but you can store urls looked-up in your user history, alter headers sent, add parameters, ajax, follow redirects, use Basic Auth, and have permalinks)
This little app won the 2009 Rails Rumble, but they put it on Sinatra after that I suppose. It's a really nice example of a proxy server coded in ruby if you're looking for that sort of thing. And reading the source is a good introduction to Redis integration as well. I actually have a local copy running on my computer most of the time for debugging web apps.
They won the completness category and really deserved it - the little bits of polish made it shine. It's also worth noting they actually wrote it in Sinatra during the competition, not after, a railsrumble 2009 was more a rack rumble ( but that doesn't quite have the same ring to it)
And you get no syntax highlighting, no prettyfying of the response (adding linebreaks and stuff), etc.
Plus, my experience (as a novice Pythonista) trying to get http basic auth working with urllib was hell. It takes like 5 obscure lines, you have to know the "realm", which I didn't even know about before. That's way more complicated than just typing stuff into text boxes
You just have to base64 encode the username/password and add it in as the value of an HTTP header. Tedious compared to most other HTTP libraries though, I agree.
32 comments
[ 3.0 ms ] story [ 75.4 ms ] threadYou can check out this video to learn more about hurl http://vimeo.com/6238577
Using a fork per request is the easy way out. The problem as you've all seen, is that memory will be used up very very quickly.
The solution is to rewrite the app to use an event loop like select or epoll, or to queue the requests in a message queue: doable, but worth the effort? I doubt it, since the website looks like a hobby rather than a commercial product.
That is, if someone forks hurl (http://github.com/defunkt/hurl) and rewrites it to use one of the fixes you referred to above, I'm sure they'd merge that in and deploy that.
Doing it with Resque would make a lot of sense since hurl uses Redis already.
I'm saying that's probably why Chris and Leah haven't fixed it already and probably for many days to come.
Since both my app and this website have unavoidable network delays, I think I'll be retiring my program. That is, when the "Cannot allocate memory" bug gets fixed.
It's open sourced, and contains some great code: http://github.com/defunkt/hurl. I used it as an inspiration when working on my first Redis project using Sinatra.
Rex Swain's HTTP Viewer - See exactly what an HTTP request returns to your browser:
http://www.rexswain.com/httpview.html
Plus, my experience (as a novice Pythonista) trying to get http basic auth working with urllib was hell. It takes like 5 obscure lines, you have to know the "realm", which I didn't even know about before. That's way more complicated than just typing stuff into text boxes
Has the added benefit of working on stuff you are developing on your own machine.