38 comments

[ 2.8 ms ] story [ 82.8 ms ] thread
How Python's super() actually gets used:

http://www.google.com/codesearch?as_q=super\(&as_lang=py...

Indeed. super() is a hook / callback which permits a library to be used in ways not envisaged by the original author.
This definitely is not correct. Though there are many who argue that classes should be designed for inheritance and that classes which are not should not be inheritable (I'll leave that to your own judgement), when classes are designed for inheritance you do need a way to call your superclass.

And in Python's case, due to MI, you need a way to call your superclasses in a consistent and sensible order even though these superclasses are unaware of one another (one might be "normally" inheritable while the other is a mixin-type superclass for instance, or used to inject metaclass-based behaviors). This is, as far as I know, a perfectly valid use case and one which requires a `super`-type utility (though Python's own `super` isn't quite a flawless implementation of this).

Good post with solid examples.

At the risk of being down voted into oblivion (I know I would be on proggit by the fundamentalist Pythonistas there): it seems to me that this just adds to the list of Python issues that make the language far more complicated than it really should be.

Examples that spring to mind:

- The ongoing saga with the GIL (although I realize this has advantages in simplifying C extension development) [1]

- String formatting changing seemingly from 2.4 and every version thereafter [2]

- The 2.x/3.x split (yes I know there are reasons)

- Version issues for scripting [3]

- Syntax inconsistency. I know this one will be controversial but I'm referring to things like the fact that del() is a operation

- Other issues [4]

Isn't something like Ruby or Lua a lot cleaner in this regard? I ask this question seriously.

[1]: http://www.dabeaz.com/python/GIL.pdf

[2]: http://kuoi.com/~kamikaze/read.php?id=287

[3]: http://news.ycombinator.net/item?id=1712035

[4]: http://www.quora.com/What-are-the-main-weaknesses-of-Python-...

> At the risk of being down voted into oblivion (I know I would be on proggit by the fundamentalist Pythonistas there)

You seem to be very well acquainted with proggit indeed as this kind of pre-whining is quite common there. Though the little twist about it at the end is a new (and quite despicable) addition to it.

> it seems to me that this just adds to the list of Python issues

Considering this page is at least two years old, it's older than e.g. most of the GIL hubbub.

> The ongoing saga with the GIL (although I realize this has advantages in simplifying C extension development)

The GIL saga makes the whole language simpler (running directly opposite your thesis that all these "points" make the language more complex), though it does make taking advantage of multiple cores harder (if you somehow have a beef against multiple processes).

> String formatting changing seemingly from 2.4 and every version thereafter

You might want to read and test things before that kind of claims. When the article you link says:

> In Python 2.5 and later, there's a new formatter

it means the old one is still there. It was not removed in Python 2.5 (or later, `"It is %s!" % foo` still works perfectly in Python 3.1). The second "point" of your article is hellishly disingenuous as it has nothing to do with formatting. It's also a pretty terrible article as it uses "long form" formatting in the `format` calls (which are used to make bindings clearer, whereas you can use C#-style positional formatting if your strings are short and readable in and of themselves)… and then whines that the formatting calls are too verbose.

And of course, coming from Perl the article you link to suggests… implicitly pulling variables out of the context instead of explicitly sending them to formatting.

> The 2.x/3.x split (yes I know there are reasons)

Indeed, and a number of these reasons center around making the language simpler and more consistent (`print` as a function, reorganization of the stdlib, ...).

> Version issues for scripting [2]

This has nothing to do with Python as a language.

> Syntax inconsistency. I know this one will be controversial but I'm referring to things like the fact that del() is a operation

What are you talking about? What is `del()`? Are you talking about `del` as in `del foo` and expressing your dislike of it being a statement? Considering the original job of `del` (completely remove a binding from a scope), I'm not sure what else it could have been (though I can't say I like it much, be it to remove a binding from a scope or remove elements from structures). I'd also enjoy your explanation of how it demonstrates inconsistency: which other Python non-statement operation is able to destructively alter scopes? And is a majority of the scope-altering operation non-statements? If it is not, would not the non-statement scope-altering operations be inconsistent?

> Other issues [3]

Few of these are universally valid. In fact, I can find none that is apart from whitespace making it harder (if not impossible) to generate code[0]. Some of these complaints are plain nonsensical (Python being an "untyped language"), most others are personal preferences which you may or may not agree with (Python's scope confusion and crappy lambdas — or more precisely overuse of statements as Haskell's lambdas have exactly the same limitation — bug me a log, while TFAA's dislike of `eval` basically amounts to "I hate languages where not everything is decided at compile time)

> Isn't something like Ruby or Lua a lot cleaner in this regard?

Ruby definitely is not, though the choices it made are most definitely different. I don't know enough Lua to comment on it.

But if you want orthogonal and consistent languages, you probably should not look at even remotely popular languages anyway: they're mostly hobbled together. Even Haskell, which your [3] seems to hold as some kind of holy grail, is a complex and not necessarily consistent language. Consistent ...

(comment deleted)
> Just to clarify, the GIL is worse than making writing multithreaded programs harder, it makes CPU bound singlethreaded tasks perform worse on multicore machines than on single core machines.

This is incorrect. I'm guessing you're making that claim based on David's discoveries on the old (2.x) GIL last year and didn't actually bother seeing the presentations (or reading the slides, or the blog posts). All the issues he outlined are in multithreaded workloads (why would the GIL do anything for single-threaded loads?). His original outline (the one which basically went viral) was about "GIL battles" (GIL contention) in the old GIL [0], and GIL contention happens when the OS schedules two Python threads for running simultaneously and they both try to run at the same time. How do you do that without running two Python threads simultaneously?

Later, David also exposed a convoy effect with IO-bound threads in the new (Python 3.2) GIL [1] [2]

See also his Interactive Python Thread Visualizations [3] and note that all of them involve at least 2 threads.

[0] http://dabeaz.blogspot.com/2010/01/python-gil-visualized.htm...

[1] http://bugs.python.org/issue7946

[2] http://dabeaz.blogspot.com/2010/02/revisiting-thread-priorit...

[3] http://www.dabeaz.com/GIL/gilvis/index.html

The GIL is actually a way of keeping Python internally uncomplicated at the price of lost efficiency.

String formatting: that's a case of the "explicit is better than implicit" dogma. As for "complicated": I'd rather explain to a newbie that % takes a string template and data and substitutes, than that string literals are actually complex expressions with possibly side-effect-ful subexpressions.

Versioning problems in language distributions vs Linux distributions: is there a language community without?

> The GIL is actually a way of keeping Python internally uncomplicated at the price of lost efficiency.

Lost efficiency? Barring significant engineering effort, the GIL is definitely more efficient than more fine-grained locking in a single-threaded situation. Which the Python core team considers of prime importance and the main use-case for Python (whether you agree or not is a different issue): previous tentatives to break-up the GIL lead to performance hits considered unacceptable (~20% across the board I believe).

It also tends to keep Python code less complicated as — much as with Javascript — you can not have two threads of Python code running at the exact same time within a process (this, on the other hand, is a very mixed blessing: it prevents people from learning about the brokenness of thread-based concurrency, and it generally breaks code migrated on a GIL-less implementation such as Jython or IronPython)

Splitting the same hair from the other side:

If an experimental fine-grained locking variant of CPython scales better for, say, CPU-intensive multi-threaded programs on eight-cores, then the decision to use a GIL instead of fine-grained locking loses efficiency in the case of CPU-intensive multi-threaded programs on eight-cores.

(Edit:) I agree with you on GIL being better for the typical case.

> If an experimental fine-grained locking variant of CPython scales better for, say, CPU-intensive multi-threaded programs on eight-cores, then the decision to use a GIL instead of fine-grained locking loses efficiency in the case of CPU-intensive multi-threaded programs on eight-cores.

True, but as far as I know, so far Python core developers haven't been very interested in switching locking strategy depending on the hardware and workload (to be fair, the GIL is so embedded throughout the interpreter and code utilities the mere act of making this more pluggable would likely generate a higher workload than just ripping out the GIL and building a brand new and more modern interpreter/VM from scratch)

Well I'm mainly a python programmer and a fan of the language (but refuse to use the word "pythonista" and believe that whoever coined that word deserves a punch in the face) so take what I say with an appropriate grain of salt.

Yes python has a few warts you need to aware, and when you list then they sound quite overwhelming, but having programmed in python for, on and off, 10 years, very few of them have actually tripped me up in any serious way.

The GIL for example gets talked about more than it actually causes problems (and there are python implementations that don't have the GIL if it is truly a problem). If I really need multiprocessing within python, the multiprocessing library normally solves my problem.

The string formatting "problem" you link to is hardly a problem at all in python 2. The "%s" % (foo) approach didn't stopped working after 2.4. They simply added a way to deal with non-positional arguments. That being said I'm not really a fan of the new python3 print syntax, but maybe I'll eventually get used to it.

as for the 2.x/3.x thing, yea I'll agree it's sub-optimal. Let's just say I still write all my python code using 2.x since most of the libraries I use still don't have stable (or any) 3.x versions.

The version issue you link to is problem of forwards compatibility rather than backwards compatibility and many languages fail at forwards compatibility. As far as I know there has only been one backwards compatibility break in python2, and that was that 'with' became a keyword.

Some syntax inconsistency is annoying I agree, but they're mostly there for backwards compatibility, and many have already been fixed. Hopefully they've gotten it mostly cleaned up in python 3.

The other issues you link to seem to be mix of personal opinion and minor annoyances.

Sure the scope rules are a bit odd a times, but I've never really had problems with them in actual real code.

I really don't get the complaint about lack of information hiding. Yes python doesn't have "real" information hiding, but you can get most of the useful parts of information hiding by adding underscore to start of your variables and method names.

The only point on that list that I actually agree with to any extent, is the lambdas/anonymous functions, complaint. But again, it is a minor annoyance rather than a major deal breaker.

At the end of the day, I agree that there are a lot of odd corners in Python that have arisen from how the language has grown organically over the years, and yes there are 'cleaner' languages out there (although I'm not sure Ruby is one of them). But for when it comes to getting things done in a quick, clean and efficient manner, python just works for me.

Lua is cleaner because it's a tiny language. Whether that's better or not depends on what you want from it. But let's just say people don't solve their problems with OOP when OOP isn't a built-in.
And even in its small size, Lua has its ways of being complicated. The interaction of multiple return values with varargs, for example. Or having to use pcall when you want exception handling. Or the distinction between the length and the size of a table.

Not to say that these things are bad! Just that you find ugliness in any language if you look for it hard enough.

I'm always irritated when I see such post.

My kitchen is full of knives, but I accidentally stab my wife only once or twice per year. That's because I'm extra careful.

Same thing go with language constructs. I just don't throw features around and see what happens.

That's extra careful? I have been married 18 years and I have stabbed my wife a total of ZERO times.
this. I am yet to stab my girlfriend and we're living together for two years now.

Perhaps shin_lao's subconsciousness is trying to tell him something? Or he's trying to see if his explanation of "accidentally stabbing his wife to death" is believable?

I think classical-inheritance subclassing is itself harmful. Subclassing creates less flexible software than other approaches, and creates a need for lots of boilerplate.

This issue - super in python - drives many new python programmers crazy. However, I suspect most seasoned devs never think about it.

These days I never subclass or feel like I want to. Occasionally I'll use 'duck' typing where I would have subclassed in the past but I find that I rarely even need that. The desire to use classes like this sets off an alarm in my head that I'm making my code too complicated.

You do a lot of subclassing in Java. Java steers you there because it's so onerous to use dictionaries and arrays/lists/tuples. The syntax for these structures is simpler in python, and so they're more commonly-used.

Subclassing is particularly important for creating frameworks in java. You build your world with packages, class hierarchies and 'bean' style objects. Whereas in python you tend to abstract by module - sets of statics, functions and lightweight structures.

I believe python would be stronger still had it had removed subclassing altogether, and instead taken the prototype approach. In prototype languages there's no such thing as a class. Everything is cloned off an original master object. Functions and properties are tapped into the clone. Iolanguage has many of these advantages but is far slower, lacks the reach of python's standard library and (I think) also lacks python's awesome list-comprehension syntax.

Someone with better articulation than me could generate blog traffic for themselves by creating a pithy post about the difference in style between java and python, and why subclassing is unpythonic. Then we could link to it when these issues come up.

I don't see much difference between a duck-typed language in which callables with capitalized name are just expected to produce objects which obey a certain duck type, and a prototype language.

The problem to which super is a (general and tricky) solution is the consequences of multiple class inheritance rather than class inheritance. A single-inheritance super could be realized in a less tricky way.

    > I don't see much difference [..]
Agree, there's not much. But I have some scenarios in mind where I prefer the prototype approach.

1) Inline classes

A pattern I sometimes use involves creating an object to maintain state before a for loop. Example scenario: parsing characters as they come in over a network line. Say the data has qutation blocks that create nesting situtaion. You want to support both ' and " blocks.

It's a simple situation. You want an object that will store some kind of mode for what scope you're in at the moment, and you'll be accumulating the structure you're parsing. [1] You might want the object to have a render(self) method as well, to dump the data once you've parsed it.

In io you can just say "Make me an object that does this". In python you declare a class, and then instantiate an instance of it. It's needless to create the class - there is absolutely only one time that I'm going to build this thing. [2]

You could create a syntax that worked like python, but had a syntax that addressed this without technically being a prototype language.

2) Adding functions to existing objects

In python there is ugliness involved in tapping a new function onto an existing object. Python's a bit confusing here partly because of the way self works. When I encounter this in python I usually do a lambda trick to make the function a member of the object, and then I'll have some method on the object that will refer to it. The equivalent in io is more straightforward.

    > A single-inheritance super could be realized in a less
    > tricky way.
True. I have and will never use multiple inheritance in python and would prefer the simple syntax that would be made possible with single inheritance.

I remain interested in this broader issue of - is class-based inheritance a good thing at all?

==

[1] In python you have to use a structure for this, because once you're in the 'for' loop, it won't let you rebind variables outside the scope of the for loop, only modify them.

I haven't played with 'inline' yet.

[2] You could also use a dictionary, but I think that's a bit more verbose. An object is just what I want in these situations.

1) Good example for a one-off object implementation; however, creating a one-off class and then an instance is just one more line.

2) Actually, I don't mind if adding methods to other developers' objects is hard; it always seemed to me like a dirty hack.

In ruby, #2 is as simple as

    num = 42

    def num.to_whatever
      "whatever"
    end

    num.to_whatever # => 42
    41.to_whatever # => NoMethodError
I agree with my sibling, though. It might be better that this is harder in Python. It's a bad idea, generally.
> In io you can just say "Make me an object that does this". In python you declare a class, and then instantiate an instance of it. It's needless to create the class - there is absolutely only one time that I'm going to build this thing. [2]

You can create classes inline via `type`, and Python 2.6 has named tuples which are basically record types. You could also create a one-off class onto which you'd graft attributes as needed:

    >>> class Object(object): pass
    … 
    >>> a1 = Object()
    >>> a1.a = 3
    >>> a1.b = 'foo'
    >>> a2 = Object()
    >>> a2.stuff = 42
you can even use `copy.copy` (or `copy.deepcopy`) on these to have something prototypey:

    >>> a3 = copy.copy(a2)
    >>> a2.stuff
    42
    >>> a3.stuff
    42
    >>> a3.stuff = 43
    >>> a2.stuff
    42
    >>> a3.stuff
    43

> You could also use a dictionary, but I think that's a bit more verbose.

By about 4 characters per key access.

> In python there is ugliness involved in tapping a new function onto an existing object. […] When I encounter this in python I usually do a lambda trick to make the function a member of the object

What lambda trick? Why not just create a (bound) instance method from your function and graft that onto your object? Granted it's not going to be as sexy as doing so in IO, but it takes all of one line:

    >>> def foo(self): print 'Instance method of', self
    … 
    >>> # naïve version, not going to work as it doesn't bind the function to the instance, it just sets a function as a member of the object
    >>> a3.foo = foo
    >>> a3.foo()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: foo() takes exactly 1 argument (0 given)
    >>> a3.foo = types.MethodType(foo, a3, Object)
    >>> a3.foo()
    Instance method of <__main__.Object object at 0x1004cdf50>
> I have and will never use multiple inheritance in python and would prefer the simple syntax that would be made possible with single inheritance.

If you use single inheritance always, you can just get rid of `super` altogether: just call the relevant method on your superclass.

> [1] In python you have to use a structure for this, because once you're in the 'for' loop, it won't let you rebind variables outside the scope of the for loop, only modify them.

I hope I misunderstand what you mean, because what I'm reading here is downright wrong: Python only has function-level scoping (and a special scope for listcomps, but that's a different issue) so any binding you create or modify within a for loop (or a `with` statement, or an `if` or whatever else you may think of) will be visible from outside the block:

    >>> a
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'a' is not defined
    >>> for i in range(1):
    ...     a = i
    ... 
    >>> a
    0
And of course, you can rebind existing variables just fine:

    >>> for i in range(2):
    ...     a = i
    ... 
    >>> a
    1
did I miss something?

    > did I miss something?
In your examples you're binding or rebinding to vars in the current scope. The issue is rebinding vars outside of the current scope.

Perhaps this code demonstates what I'm getting at.

    def my_function():
        a = 1
        def modify_a():
            a = 2
        modify_a()
        print a # prints 1
        #
        b = [1]
        def modify_b():
            b = 2
        modify_b()
        print b[0] # prints 1
        #
        c = [1]
        def modify_c():
            c[0] = 2
        modify_c()
        print c[0] # prints 2

    my_function()
This is clearer, but it has absolutely nothing to do with `for` loops (which is why your statement utterly confused me) and everything to do with Python's broken scoping in general: Python 2 gives no writing access to lexical scopes between global and local (Python 3 added a new keyword, `nonlocal`, which is used in the same manner as `global` to set a binding as writable in a parent scope).
(comment deleted)
For #1 you can just use a closure.

For #2, the method documented in http://stackoverflow.com/questions/972/adding-a-method-to-an... doesn't seem so bad to me.

  object.methodname = new.instancemethod(function, object, class)
It may be deprecated. But it still works.
(comment deleted)

    > For #1 you can just use a closure.
I don't think that will work. See the discussion about rebinding in the post or elsewhere in the thread.
It is unclear to me what you think doesn't work. A closure can easily store state. As you yourself demonstrate in your rebinding example, you can store and mutate scope if you use a mutable datastructure (like a list) to avoid rebinding problems.

Perhaps if you gave an example of what you think doesn't work, I might understand your point.

Iolanguage has many of these advantages but is far slower, lacks the reach of python's standard library and (I think) also lacks python's awesome list-comprehension syntax.

Lua is prototype based. It's also quite fast. What if we ported the Python standard library and added list-comprehension?

(comment deleted)
Well, there's already Lunatic Python, so any Python libraries you lack can just be called from a hosted Python VM instance. Also, I think It's possible to add comprehension-like syntax to Lua already.
Click on our website: ==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

"Priority, my friend!! Christmas is coming, quick to our website shopping, our web site shopping there will be something different, unexpected things to you, let you have different sense, our website wholesale various fashion shoes, such as Nike, Jordan, prada, also includes the jeans, shirt, bags, hats and decoration. All these products are our free transport, prices are competitive, we can also accept paypal j, after the payment within short time, can ship. = New era cap $12;

Air jordan(1-24)shoes $30;

jordan air max oakland raiders $34a€“39;

Ed Hardy AF JUICY POLO Bikini $25;

Christan Audigier BIKINI JACKET $25;

Tshirts (Polo ,ed hardy,lacoste) $15

coogi DG edhardy gucci t-shirts $18;

gstar coogi evisu true jeans $35;

coach chanel gucci LV handbags $36;

Sunglasses(Oakey,coach,gucci,A r m a i n i) $15;

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====

==== ( http://www.etradinglife.com ) ====