Is it worth starting "from scratch" in web dev? Straight Python, or Django?

Were I you, I would start with Django so that I would have an application up and running faster. And then, since Django is open source, start poking around at the internals to see how it works.

I would wait to roll your own web framework until you've had a little more programming time under your belt. Regardless of whether you may be a quick study or not, rolling your own framework is going to have all sorts of strange issues come up, and sometimes it's nice to be able to ask around on a forum, mailing list or IRC for help. Django has one of the largest (if not _the_ largest) Python community; at least as far as currently common frameworks go.

Django is quite nice as a framework as well, it may turn out that it's all you ever need. There are quite a few people out there who use it for every project they work on, with great results.

[Edit: Changed "webapp" to read "web framework"]

I write my CGI scripts in C -- but only after writing all the libraries I'm going to want.

There are merits to "understanding what you're doing at a more fundamental level" as you put it -- but that doesn't mean that you should waste time writing the same code many times. Understand it, write a library, and be done with it.

I am not a professional developer, and am more akin to a hobbiest programmer/developer, so take this for what it is worth.

Like icey said, pick up Django and learn it - this will get you results far faster and of a higher quality then rolling you own straight out of the gate.

From your post, it sounds like you will be learning to program, learning python and possibly learning Django all at once - an ambitious goal to say the least. Combining the three - centered around Django I think will be the most practical approach for diving in - far easier than learning to program in python and tackling the particulars of designing a web app framework from the ground up. djangobook.com , despite its (over-come-able) errors, will give you a nice base to start from. "Practical Django Projects" is also getting some good reviews, but I have not had the chance to check this one out yet.

I think you will find most of Django relatively easy to learn - and what is not so easy, will highlight some of the tougher aspects of developing a web app framework so you know what to look closer at and beef up on in the case that you do roll your own in the future.

Also, like icey said, once you become versed in Django you can decide to dig around in its internals. This will give you the chance to see what is already optimally designed (no one should reinvent the wheel) and what you can stand reinvention or redesign.

I believe this was said about django (and a few other frameworks) when you write a site with django you are writing it in django and not python. well, that was the gist of it anyway. I didn't like django myself, I found it fragile, and I don't want my app to break unless I start dropping tables, or deleting code. a insert into should not break my app.

I'm a big fan of the "build a framework that you'll never use" philosophy. That isn't a joke. You aren't going to be able to match the quality of the thousands upon thousands of hours that have gone into something like Django, but making a framework will make you a better framework user. It makes you understand the flow, what building hooks is like, etc. It also means that you'll be more ready to dive into some of the framework code than someone who just uses a framework.

But that's just my opinion from seeing so many people not understand how frameworks work and try to use them.

While I'm not a big fan of PHP in general, the fact that you feel/felt comfortable with ExpressionEngine could mean that a related PHP framework would be a good choice. CodeIgniter (http://codeigniter.com) is similar to what EE is built on, and EllisLab is going to have the next version of EE built directly on CI. It's a really slim framework that doesn't force much on you, so you would get acquainted with PHP in general as well.

Along with that, I think it is a good idea to play with a number of languages and frameworks. To me, the best test is building a simple application in them, a todo list or the like. Try things like Django, Rails, Merb, CodeIgniter, Cake, and others to see what feels most natural and fun to you.

Be sure to also think of what else matters to you about a language/framework. Check out communities, plugins and modules, and the documentation.

If you're doing web development for money, then you should go with whatever is the most expedient solution that produces reasonably attractive results. The danger here of course is that in the long run what's expedient now may lead to problems down the road - such as when you hit a problem the pre-built solution can't handle, or if it causes you to do more work than you would if you only had a better understanding of the underlying technology.

With that said, if you're doing this to try and learn something, then you will be doing yourself a disservice if you don't at least try to roll your own version of everything you will want to do with the web. Even if you start out with your own stuff and decide to swap in freely available components (open source, etc.), at least then you'll understand the infrastructure you're working with at a level you would otherwise not.

Any free on inexpensive Python and Django server hosting around for playing around?

I suggest looking at the "Worse Is Better"/"Right Thing" discussion, as it maps perfectly to your situation: http://www.jwz.org/doc/worse-is-better.html

Worse Is Better gives you fast results with tradeoffs that you already knew of, while Right Thing approaches will allow you to discover hidden problems as you attempt to write the ideal program. So you learn more when you start from scratch, and that is a worthy goal.

I usually whip up a dirt-simple "Django-inspired" WSGI microframework (regexp-based url dispatcher, RESTful views, jinja or genshi templates, decoupled optional request and response objects (no shared state between requests), signed-cookie-based sessions, etc.) based on what I need as I go.

Instead of using an ORM, I've discovered an amazingly flexible, efficient DSL for dealing with Postgres. It's called SQL. ;-)

I wrap my model objects around sql queries, so, externally, the objects have a nice ORMey OO interface, but internally I can do all sorts of wonderful Postgresql magic).

After the first time you do it, making a new microframework usually takes less than a day, depending on what else I have to get done. (The reason I reimplement is to prevent myself from adding useless features that I "might" need someday on some future project).

One of these days, I might release one, but strictly as a code-dump example, not as an ongoing project.

(Back on topic, my recommendation to the OP is to start with Django. It's beautifully designed, and has served me well even after I stopped needing its code).

Use django, not because it's easy to get into, but because later you can use the large number of django apps to get kickstarts into things like adding a forum, blog, etc.

One thing you should be aware of is that Django's philosophy is to cleanly separate between what designers do and what developers do. That's why you don't (by default) get to use Python in your view templates as you would in php/jsp/rails. Instead Django provides yet another mini language just for views, including control sturctures, expressions, etc.

If you are a designer and developer in one person, Django might not be the best match for you, unless you love something else about it (there are many things to love about Django)

I found that I don't use most of what web frameworks provide. But you could argue that just using Django's nice regex based request dispatching is worth it, even if you decide not to use their template engine at all (it's replaceable after all)

To sum it up, what is it that web frameworks do for you? It's basically request dispatching, forms processing (validation and stuff), view templates and ORM. If your applications handle a lot of form to database CRUD, using a framework is helpful. Otherwise you could just as well use naked mod_python because request dispatching is actually rather trivial and python already has a simple template engine built into the language.

After reading these comments I've realized that trying to "go it alone" in Python is biting off too much at once. It will be too long before I can finish anything usable, and I'll lose motivation.

So, I've decided to aim for "FAST" and use Django to get a working website out the door as soon as possible, and study programming at greater depth on the side (without expecting that effort to pay for itself immediately).

I started going through the book "Practical Django Projects" this evening and it feels right at my level.

Thanks for all your perspectives! I appreciate it a lot.

I had a similar situation a few months ago. My goals were to get something up and running relatively quickly but it was my first python web app and I wanted to make sure that I actually understood what was going on underneath the covers. I used web.py (http://webpy.org/). Overall I've been pretty happy with it -- it is small enough that looking at the code is actually meaningful to me (although there is definitely some strangeness in there from my perspective) but I also can mostly just write the app and not think about the framework.

Maybe you should start thinking in going to another field, where your knowledge is more required?

naa.. you are asking the right question.

but let me put it this way. isn't it time that we forget about frameworks, starting from scratch, a lot of code that does the same thing 90% and only 10% innovation? Maybe it is better stitching actual services and "plugging" in the actual difference.

The wheel itself never was invention. It was its use. I bet the time when technology will be obsolete is approaching. It is what we can achieve what matters, not whats in the box.

disclaimers: I didn't read the rest of the thread, and i don't use Django (I've been coding C, perl, python and ruby for years, and now rails).

I think the path of least resistance is to learn python and Django well, put some apps up, find out wehre the dev environment on Linux/BSD or mac is strong (editors, test tools, git/SVN), (and maybe make some decent money!).

Then branch out, start digging in to problems that web app frameworks solve, and solutions. These solutions would be the ORM, templating system, admin interface (Django's is admired and frequently copied), debuggers, performance profilers, testing stack, Ajax libs.

Look at the SQL generated, the HTML and CSS, how easy it is to rotate/analyze logs, DNS (subdomains, permanent redirects, etc), monitor/restart the proxy and servers, caching at all levels (pages and fragments, db queries, fixed assets in the browser). Also look at the mechanisms to plug in functionality, e.g. authentication, file uploads, multiple db connections, etc. That should teach you a lot.

I think that you might really struggle trying to pick up Django without really understanding programming or Python. That doesn't mean that you shouldn't try, I'm just suggesting that you might struggle. A lot of people approach web frameworks from scratch and think that they can learn it in a couple of weeks and be proficient at them. It takes a lot of time. I'm thinking a good 6 months before becoming proficient.

Sure you can cut and paste the code from the tutorials and have a simple blog application and running in a week, but you won't really have a clue as to what's going on inside the framework.

There are a lot of parts of Python that Django uses that can be a little overwhelming to a noob programmer. For instance, URL's are crafted by using Regular expressions. Regexes might take you a couple of weeks to get your head around if you've never used them before. Django also tends to pass dictionaries around as arguments, which can be confusing and overwhelming if you've never seen them before. Same thing with it's use of list comprehensions under the hood. The use of args, kwargs or variable length arguments can be a bit confusing as well.

So, be patient with yourself. Django is certainly a great framework and well worth the time investment to learn. Python is also a great language, but again, these things take time to learn well.