It's precisely all the help you need to go and look up your notes on coroutines from college. You did go to college didn't you? Or are you an anti-college hipster, suddenly finding the limitations of that approach.
I studied computer science at a university, not just a college, and coroutines weren't covered during the course. So even if you've got a degree, there's no guarantee you'll have met a coroutine!
(I do agree that a formal education is valuable, even if you already think (or know) you're good at programming. I learned a lot from my course. There's a lot to be said for being forced to learn a bunch of stuff, and when you're under 25 it's not like it's going to hurt.)
(replying to myself because I can't reply to stch directly):
What does that mean/how does it affect me?
It means that only people who have specifically turned "showdead" on can see your posts. No one can reply to you posts directly. Your upvotes do nothing and don't count. But crucially, to you, everything seems completely normal.
And howd that happen?
An administrator decided that your first ever comment didn't fit the tone of this site (which it didn't, but well, you were new), so they decided to instantly and silently ban your account forever. So you have been commenting, for a year, and basically no one has seen your posts. I feel really bad about that and I hate this objectionable practise.
Wow, that's pretty ridiculous. You'd figure after a few months they'd tell me, right? Is that also why I get "Unknown or expired link" all the time and every page always takes 15-20 seconds to load? I noticed when I logged out that issue went away. I always just assumed the forum was really poorly designed. :o)
This is why you study CS at university before you start work as a programmer. Otherwise you end up being amazed by the basic CS concepts, or worse, you end up reinventing them badly.
Come on, I've got a reasonable point here. People post on forums asking these questions which should have been answered during their educations.
If civil engineers had a forum site and someone asked "how do I calculate the load a simple beam", people would say, what the fuck are you doing in your job if you don't know that? Who is employing you and why? Where did you go to college? Who was your tutor because next time I'm at a dinner at my college I'm going to ask him how the fuck did they pass you.
Our current trend of saying college doesn't matter really starts to be a problem when professional programmers (yeah I'm assuming that the guy in the question is) are asking basic questions like this. This shows that we do need college education for programmers.
It also irritates me - this guy could have taken the time to learn all these basics, but for whatever reason he thought it didn't matter, and now he's paying for it.
Neither a college education nor continued employment in the field is a requirement to ask a stackoverflow question. In fact, I'd venture to say that a version of stackoverflow where the default answer to everything was "you should have learned this in college" would not be very useful (or used) at all. If you find the discussion beneath you, I suggest that you not participate and find a better use of your time. Others may be getting value from it and I can't for the life of me understand why you think this is a bad thing.
Loads of people didn't go to college, and program. Loads of people went to college doing something other than CS, and program. Loads of people haven't yet been to college, and intend to go and do CS, and program.
Python has a lot of users who are not programmers by profession, but rather mathematicians or scientists or other professions where Python is an excellent tool for getting their main job done.
So assuming that only well-educated programmers can have questions about a programming language is asinine.
My university education didn't cover coroutines either.
SO is not a site for software engineers, it's a site for programmers, regardless of their level of knowledge, education or age.
And by the way, I'd rather work with fellow programmers who don't know what a coroutine is than with an insufferable elitist like your posts make you out to be.
One of the false assumptions you're making here is that CS == programming. There's a difference between being a civil engineer, where you have to be able to calculate loads on beams, and being a physicist, where you try to understand why beams behave the way they do.
It is one thing suggesting it is unwise to develop a new language without a CS background or broad and deep experience with other languages of some form as you might end up with something like PHP but there are plenty of ideas to develop that a CS background helps fairly little.
Generally curiosity and willingness to learn is worth more.
This does reflect very sadly on schools, and shows up when trying to explain things to programmers used to conventional languages, especially if you're trying to explain why C is fundamentally broken.
Roughly there are three fundamental control transfer operations: conditional jumps, subroutine calls, and coroutine calls. A subroutine is a slave, the caller is a master. With coroutines, the caller and callee are peers.
Coroutine calling is more fundamental and easier to program, but it isn't available in C.
The theory is about continuations. A continuation is just "the rest of the program". You can think of it as the program counter (PC). Subroutine calling works by passing a continuation to a routine, when the routine is finished it invokes the continuation. This is done by pushing the program counter on the stack, and the subroutine popping it with a return statement.
Coroutines work by exchange of control. When you call a coroutine you give it your current continuation to call when its ready. But also you do not call the coroutine at the beginning. You call it where it last left off: at its last continuation point.
A set of coroutines are usually called fibres. They represent interleaving of control. You can emulate them with pre-emptive threads and locks, but coroutines are synchronous and non-premptive.
Felix and Go both make heavy use of coroutines (called fthreads and goroutines). Iterators as in Python are a special case.
In general on today's badly designed CPU's you have to think of coroutines as requiring stack swapping. Threads do this which is why you can emulate coroutines with threads.
By far the most well known coroutine scheduler is .. the Operating System. Its basically a coroutine of applications. This is why you can read and write to files: the real world is event driven but callbacks are impossible to program with. So the operating control inverts the events by stack swapping so your application can be written as a master.
You think you're calling the OS, and the OS thinks its calling you. You're both masters. That's coroutines.
Python (and Felix) both have yield, but that's a special case. In general the model is reading and writing channels: yielding is just writing the "sole" channel and getting a function result is just reading it. A channel is basically a "place to swap stacks".
> Coroutine calling is more fundamental and easier to program
Citation needed.
For coroutines, each coroutine needs its own stack. That means you have to have a dynamic memory system baked into the language. And maybe garbage collection too.
> on today's badly designed CPU's you have to think of coroutines as requiring stack swapping
I can't think of an implementation of yield (let alone general coroutines) that doesn't require a separate stack for each coroutine. I admit that I haven't learned very many of the stranger forgotten architectures that are out there, so I might be blinded by the limitations of a somewhat conventional experience.
But I'm also thinking it might even be provable that each coroutine needs its own stack: Think of a program that has m generator functions, where f_1 calls f_2, f_2 calls f_3, ..., f_{m-1} calls f_m. Each of these subroutines creates n copies of its next-level generator, and steps those generators and yields to the parent unpredictably (for example, depending on input from a user-supplied file). It seems like if m and n are large enough, you'll have no choice but to resort to swapping stacks.
> For coroutines, each coroutine needs its own stack. That means you have to have a dynamic memory system baked into the language. And maybe garbage collection too.
Why? Just statically give every co-routine it's own stack.
The number of coroutine invocations, and the order in which they're cleaned up, could depend on user input. The former could be unbounded.
More formally, I'm pretty sure you can write a program that always halts, but for every pair of large numbers N, K > 0, there are at least K different input values that result in (1) at least N coroutine invocations being simultaneously active, and (2) for each of those K different input values, those invocations finish in a different order.
What I find weird about the yield keyword is that it seems to effect the execution of code that comes before the statement, causing it not to execute until later.
The presence of the yield keyword in a function definition means the function will return a generator when it's called. Therefore "print "aaa"" is not executed on "mygen = createGenerator()" as expected. "print "aaa"" is infact executed when mygen.next() is called.
To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute any of the code. This is why the following results in an error:
I think I mostly understand the idea, the presence of the yield statement transforms the nature of the parent function itself and makes evaluation lazy.
This seems a little weird syntactically though because it feels like the interpreter is somehow reading ahead in the program. It also means you don't necessarily know that this is a lazy function unless you read to the end.
This syntax would against the Python principle DRY (Don't Repeat Yourself): The "lazy" keyword conveys no information that doesn't exist in the function body.
Also, Python has precedent for this scanning behavior. One of the fundamental features of Python is that the scope of a variable is determined by the presence of an assignment statement in a function. For example:
a = 1
def f():
print a # UnboundLocalError occurs here
a = 2
f()
If you know Javascript, think about how often you need to use the "var" keyword in that language, and how easy it is to accidentally pollute the global namespace by omitting it.
This is an insightful point. I tend to regard this scanning behaviour as a "quirk" rather than something fitting in more with some deeper Python philosophy.
A specific syntax for generators would in my view be generally superior, containing the same number of new keywords (generator now replacing yield) and providing enhanced clarity.
generator hello(n):
for i in range(n):
return i
I am generally neutral about variable scope being determined by the assignment of a statement in a function - in 99.99% of cases the intuitive thing happens.
Except you can use `return` without argument to return early from a generator, and `yield` without argument yields None, so they need to be separate keywords.
Early return is definitely needed. I guess you could make bare `return` early return, and `return None` the way to yield None, but that's kind of confusing.
I agree that would be confusing, but I question that early return is needed - I don't think I've ever seen a generator example where early return is used. It could be that it's because I haven't seen enough examples of course. Please share some examples of a generator with early return which can't be accomplished easily and clearly in some other way.
> To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is bit tricky :-)
Because it allows another thread to run. Yes, they actually are coroutines, but that is an implementation detail. Just look at it as an optimization for the case where at most one of the two threads (the one producing the values and the one consuming it) runs at a time.
52 comments
[ 3.3 ms ] story [ 76.1 ms ] thread(I do agree that a formal education is valuable, even if you already think (or know) you're good at programming. I learned a lot from my course. There's a lot to be said for being forced to learn a bunch of stuff, and when you're under 25 it's not like it's going to hurt.)
College is (most of the time) American for university.
What does that mean/how does it affect me?
It means that only people who have specifically turned "showdead" on can see your posts. No one can reply to you posts directly. Your upvotes do nothing and don't count. But crucially, to you, everything seems completely normal.
And howd that happen?
An administrator decided that your first ever comment didn't fit the tone of this site (which it didn't, but well, you were new), so they decided to instantly and silently ban your account forever. So you have been commenting, for a year, and basically no one has seen your posts. I feel really bad about that and I hate this objectionable practise.
No, that is just something HN does.
If civil engineers had a forum site and someone asked "how do I calculate the load a simple beam", people would say, what the fuck are you doing in your job if you don't know that? Who is employing you and why? Where did you go to college? Who was your tutor because next time I'm at a dinner at my college I'm going to ask him how the fuck did they pass you.
Our current trend of saying college doesn't matter really starts to be a problem when professional programmers (yeah I'm assuming that the guy in the question is) are asking basic questions like this. This shows that we do need college education for programmers.
It also irritates me - this guy could have taken the time to learn all these basics, but for whatever reason he thought it didn't matter, and now he's paying for it.
Why do you look down on all these people?
So assuming that only well-educated programmers can have questions about a programming language is asinine.
My university education didn't cover coroutines either.
And by the way, I'd rather work with fellow programmers who don't know what a coroutine is than with an insufferable elitist like your posts make you out to be.
Generally curiosity and willingness to learn is worth more.
Roughly there are three fundamental control transfer operations: conditional jumps, subroutine calls, and coroutine calls. A subroutine is a slave, the caller is a master. With coroutines, the caller and callee are peers.
Coroutine calling is more fundamental and easier to program, but it isn't available in C.
The theory is about continuations. A continuation is just "the rest of the program". You can think of it as the program counter (PC). Subroutine calling works by passing a continuation to a routine, when the routine is finished it invokes the continuation. This is done by pushing the program counter on the stack, and the subroutine popping it with a return statement.
Coroutines work by exchange of control. When you call a coroutine you give it your current continuation to call when its ready. But also you do not call the coroutine at the beginning. You call it where it last left off: at its last continuation point.
A set of coroutines are usually called fibres. They represent interleaving of control. You can emulate them with pre-emptive threads and locks, but coroutines are synchronous and non-premptive.
Felix and Go both make heavy use of coroutines (called fthreads and goroutines). Iterators as in Python are a special case.
In general on today's badly designed CPU's you have to think of coroutines as requiring stack swapping. Threads do this which is why you can emulate coroutines with threads.
By far the most well known coroutine scheduler is .. the Operating System. Its basically a coroutine of applications. This is why you can read and write to files: the real world is event driven but callbacks are impossible to program with. So the operating control inverts the events by stack swapping so your application can be written as a master.
You think you're calling the OS, and the OS thinks its calling you. You're both masters. That's coroutines.
Python (and Felix) both have yield, but that's a special case. In general the model is reading and writing channels: yielding is just writing the "sole" channel and getting a function result is just reading it. A channel is basically a "place to swap stacks".
Citation needed.
For coroutines, each coroutine needs its own stack. That means you have to have a dynamic memory system baked into the language. And maybe garbage collection too.
> on today's badly designed CPU's you have to think of coroutines as requiring stack swapping
I can't think of an implementation of yield (let alone general coroutines) that doesn't require a separate stack for each coroutine. I admit that I haven't learned very many of the stranger forgotten architectures that are out there, so I might be blinded by the limitations of a somewhat conventional experience.
But I'm also thinking it might even be provable that each coroutine needs its own stack: Think of a program that has m generator functions, where f_1 calls f_2, f_2 calls f_3, ..., f_{m-1} calls f_m. Each of these subroutines creates n copies of its next-level generator, and steps those generators and yields to the parent unpredictably (for example, depending on input from a user-supplied file). It seems like if m and n are large enough, you'll have no choice but to resort to swapping stacks.
Why? Just statically give every co-routine it's own stack.
More formally, I'm pretty sure you can write a program that always halts, but for every pair of large numbers N, K > 0, there are at least K different input values that result in (1) at least N coroutine invocations being simultaneously active, and (2) for each of those K different input values, those invocations finish in a different order.
What is a metaclass in Python - http://stackoverflow.com/questions/100003/what-is-a-metaclas...
Understanding Python decorators - http://stackoverflow.com/questions/739654/understanding-pyth...
For example:
outputs:aaa
Whereas
Outputs nothing.To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute any of the code. This is why the following results in an error:
A return keyword cannot exist within the body of a generator - it makes no sense.A more detailed explanation can be found here http://docs.python.org/2.5/ref/yieldexpr.html
This seems a little weird syntactically though because it feels like the interpreter is somehow reading ahead in the program. It also means you don't necessarily know that this is a lazy function unless you read to the end.
A more intuitive syntax might be something like:
Also, Python has precedent for this scanning behavior. One of the fundamental features of Python is that the scope of a variable is determined by the presence of an assignment statement in a function. For example:
If you know Javascript, think about how often you need to use the "var" keyword in that language, and how easy it is to accidentally pollute the global namespace by omitting it.A specific syntax for generators would in my view be generally superior, containing the same number of new keywords (generator now replacing yield) and providing enhanced clarity.
I am generally neutral about variable scope being determined by the assignment of a statement in a function - in 99.99% of cases the intuitive thing happens.> To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is bit tricky :-)
Not Yield as in the common multithreading command to allow another thread to run.
PS. Anyone know why they chose such a confusing name?
Thanks for the feedback. Because of comments like these, I decided to spend more time at training people for a living, and I'm loving it.
So don't underestimate the power of web comments, it can literally change the way people live.