Does the research take into account that charter schools are opt-in, and thus likely to be populated by children with parents more heavily involved in their upbringing? Maybe I missed it being mentioned in the article, but it seems like that'd be a much bigger factor.
If that wasn't controlled for, then it's less that the schools themselves are necessarily any better (though that could also be the case) and more that students with already better chances of doing better are being "self"-selected into a different population.
Simple logic would dictate that parents who opt in for better schools for their kids are more involved than those who don't opt in for better schools..
It's getting a bit out of hand where every single statement online is now getting a [Citation Needed] afterwards. There isn't a randomized controlled study for every single aspect of the universe, we'll have to use intuition occasionally.
Statements of preference are different than assertions about the world. The comic is funny, but it does have an antiphilosophical spirit, especially if you think it relates to the present conversation.
I'm sorry. I care. (sorry (sorry)) ((((((sorry))))))
>It's getting a bit out of hand where every single statement online is now getting a [Citation Needed] afterwards
I wasn't asking if there was evidence just to be annoying, I was asking because the statement was exactly opposite of my expectation and I wanted to know if there was evidence to back it up.
My personal experience with charter schools is that it's where many parents "dump" kids that were either kicked or failed out of public school. This comes from many conversations I've had with a high-school teacher I've become good friends at that switched from teaching public school to charter school ~5 years ago. I admit my perspective is very narrow but was hoping there was actual data to support the chart-school-parents-more-involved statement.
First, that parents more heavily involved are more likely to opt-in to charter schools (or similarly though not equivalently, that parents who choose to opt-in are likely to have been more heavily involved). I consider it to be largely self-evident that parents who are not involved are very unlikely to opt-in, and while the converse isn't necessarily true, I think that fact alone would significantly sway the overall correlation of the "non opt-in" population.
And second, that being more heavily involved leads to better outcomes for children, which I feel is self-evident, and is also highly supported by a quick google search for 'involved parents better students'.
Does it also take into account that charter schools have a tendency to boot out any low-performing or "behavioral problem" students after a certain date (usually right after state funding levels for schools are set for the next year)?
Your logic doesn't work. Of those people who simply participated in the lottery, the ones who won it and went to charter school did better than the ones who weren't selected and didn't. This means the groups being compared - the winners and losers - both "had active parents"; this factor does not constitute a difference between the two groups.
"One could argue that simply participating in the lotter shows that those kids had active parents, and therefore are self-selecting as the OP stated."
Exactly the point: comparing students with equal motivation and parental involvement, who are separated only by luck, is a convenient way to control for the self-selection bias.
Your objection shouldn't be so lightly dismissed. It's possible that merely being placed in an environment populated with other students whose parents are supportive is enough to account for the gains. The students who failed to win the lottery were placed back into a general population whose influence (direct and indirect) could be the reason for their poor performance relative to the charter school kids.
I didn't reply to the others because I didn't want to get into a whole fight about it. But the OP's original message about selection-bias is still correct.
The article's original claim is that charter schools does work for disadvantage and black students.
But that is NOT what the data supports.
The data supports that it works for disadvantaged and black students who have active parents. That conclusion raises more questions than it answers.
I think your theory makes a lot of sense. Thank you for sharing that.
You're making an assumption that itself needs to be evaluated. My city (Albany, NY) was selected as an early experiment in Charter Schools in the state, as it's the state capital, and the public schools have a bad reputation.
In the media, the message is always about quality of education and high standards -- because the audience is the political class who funds the schools. But the outreach to parents (found in bus shelters, pamphlets, hip hop radio, etc) is a message that usually is around childcare and uniforms. Longer school hours + longer school year == less daycare expenses and arrangements.
In terms of outcomes, it's all over the map. Several charters have closed for poor performance, and some do well.
I'm sorry, but your comment wasn't clear. What assumption am I making? I mean, I'm sure I made several, but which do you think needs to be evaluated/supported?
> While overall charters and public schools compare relatively closely, both the 2009 and 2013 study found that charters did better for students in poverty. In addition, performance gap is growing over time:
It's not that hard to explain reversion to the mean. The schools in the lowest socioeconomic areas are generally, a priori, the worst of the bunch.
Consequently, just about anything will perform better. This is especially true if there is any ability to self-select (lottery is insufficient evidence as expelling "bad" students back to the public school is the same as self-selection).
When charter schools are compared in the suburbs, they generally perform worse than a public school.
I don't agree that this is reversion to the mean. If we saw it be the case that school districts with particularly good outcomes one quarter were more likely to have a decline in performance the next quarter independent of all observables, THAT would be reversion to the mean.
If there is a statistically significant difference in outcomes between charter schools and non-charter then I don't think we can dismiss it on the grounds you state.
But you did bring up an excellent point in the self-selection bias. We are potentially comparing apples and oranges if the charter schools get a different sort of student to teach.
Almost all the evidence is based on lotteries and addresses attrition. Expelling bad students is a potential problem, but students are tracked across schools. Some students are lost, of course. However, the evidence in the KIPP study shows that attrition actually raises the scores of the lottery losers more than lottery winners. This is because winning the lottery encourages some bad students to stick with it.
I don't see the lottery as a bad thing. It lets kids ezcape from a bad environment. In a inner city public school, there is often too much peer pressure which forces the kids into poor performance or gang life.
The lottery is not a bad thing. And, I'm not even against non-lottery based selection for schools. They are allowed to do so.
The only issue is when trying to compare private school type X vs public school Y. Public school Y is required to take all comers ... no exceptions. If you want to claim that Private School X is better, worse, etc. you need to control for that because the worst students consume the most resource--even the expulsion of just a very small number of students can make a disproportionate difference.
I work at one of those high-performing charter schools that serves primarily low-income students. It's interesting hearing the criticisms of the charter movement as a whole considering its actually made up of many different types of schools serving many different demographics. That said, there are quite a few studies, the Stanford CREDO study cited in the article chief among them, that demonstrate the effectiveness high-performing charter networks aimed at lower income populations (Uncommon Schools, Achievement Prep, and RePublic Schools in the south -- also where I work).
Also, we have, to my knowledge, one of the only compulsory middle school computer programming courses in the state (that's what I teach), maybe the country--the goal being that all of our kids have solid computer programming and web development fundamentals under their belt before they graduate high school.
I wish San Francisco could bring itself to embrace charter schools. SF is a perennial laggard in educational performance for its public schools. There are a few good schools, mostly heavily leaning Asian --due to parental acculturation. But, for the most part the schools are very under-performing. I wish there was something to get them to address the issue --and parents need to stop thinking 'education' and aptitude begin and stop in the classroom.
Also, it would help to have special schools for children of parents for whom education was not part of their upbringing --those kids need some full blown indoctrination to get them to see schooling as an essential part of their lives and not something which interrupts their lives.
I don't know about SF proper, but there's a pretty sizable charter movement in the Bay Area.
See:
http://www.kippbayarea.org/http://www.rsed.org/bayarea/
Those are two off the top of my head, but I'm pretty sure there are others...
I don't know if comparing charter schools to regular schools is fair, when I think their may be a significant difference in the population of the school. Or at least I don't know that there isn't.
Even with a lottery, you have a bias towards students with parents that enter the lottery. People with caretakers that are less concerned about their well-being probably are less likely to be entered in the lottery.
You should really compare students who attempted to get into a charter school vs those that didn't.
If you want to show evidence of charter school success that's definitive, you would need a randomized controlled trial. split a school district into half charter schools half public schools. And you wouldn't let the charter schools kick out students who underperform or have behavior problems, or otherwise make sure that such students don't skew the charter school's metrics. See what happens.
I guess I'm not sure how much the success of a student population can be attributed to teaching methods, and how much to aspects of the child's life. We all know that wealth has dramatic effect at college attendance and graduate rates for those that attend college.
Most of the lottery evidence suggests that the gains from charter schools are largest for students that that are more disadvantaged (limited english, learning disabilities, and low initial test scores). So while these studies only give causal estimates for students that apply to the lottery, it's quite plausible that average effects would be even larger if all students were included.
Charter school grad here. From a school which often came out as the top school in the state on standardized tests, in a high income suburb in a state with an already strong public education system (MN).
Anecdotal observations:
From what I could tell, the success of the school was due to three things:
1. Parents who had available time, money, and energy to devote to the school. This would likely be less true in a lower income area, but as has been mentioned elsewhere, could account for higher average performance of schools that parents actively chose to put their kids in, regardless of location.
2. Excellent staffing, mostly due to the attractiveness of working in an environment with more freedom to innovate.
3. That the environment was different enough from the huge suburban public schools nearby that some students would perform better in that kind of environment.
One other thing I noticed that I haven't heard mentioned before, which is a situation that is unique to looking at the upper end of public schools and income brackets rather than the lower: the school I went to was not good at playing the college prep game. All the courses were difficult, so nobody had any "honors" courses on their transcript. GPAs were generally lower. Fewer extracurricular activities were available because of the size of the school. These facts resulted in a lower number of students going on to prestigious post-secondary institutions than you would expect from an otherwise high performing school.
ALL of these observations point to the conclusion that Charter schools are really just an "alternative", and naturally some alternatives will be an improvement and some will be a failure.
I like this article because it indicates that charter schools specifically help marginalized groups (black, poor), which fits nicely with the idea that having alternative environments in general is good for some students, and also matches my personal experience. (I promise this is the only time I will ever claim "white male nerd" as a marginalized group.)
Charter schools may not be a panacea for low income neighborhoods. But there are a lot of low income parents who do really care and can devote time and energy to education. For those families, charter schools can change lives, so lets give them that option.
I agree, and I disagree. In most cases (each state is different) the charter school is paid a stipend per pupil from the local school district. Here in Philadelphia, and in other places, the sheer amount of charter schools pulls significant money out of public schools, which are already almost destitute.
I love the idea of a child getting a chance that they wouldn't normally have. But I wonder if it's at the expense of children of uneducated or uninvolved parents?
The one thing you can never do is blame the child for the parents' level of involvement. But it seems this may be punishing them doubly.
Here in the East Bay (San Fran Area), the charter school my kids attend gets ~$1300 less per student. So every student that moves from a non-charter to a charter frees up ~$1300 to be used another way by the district (or at least reduces the financial burden of the district if it is already almost destitute). So I wonder... if all those ~1200 charter students were attending the non-charter schools in the district, what happens to that ~$1.5 Million gap? I don't imagine that the school district just has an extra $1.5M sitting around to cover the difference. So does it spread that cut around so now all public schools get less per student? It seems to me that the charter school is saving money not sucking it out. Even if they are only as good as the non-charter schools, they would be doing the same job for less money.
A friend of mine is a public school teacher, and her specialty is special needs children. She also helped organize a revolt in Austin against charter schools a couple of years ago, which led to the replacement of the entire school board in the next election, and cancelling the contracts with at least some of the charter schools in the city. She's a friend whose opinions I respect, certainly on an issue where she is vastly better educated than I, so I was more inclined to listen to her, despite having a relatively strongly held belief that charters and market-oriented solutions were the right way forward for education.
She thoroughly convinced me, that at least in the case of IDEA (the company that was contracted in Austin) charter schools were not superior, especially for the students most in need of better schools.
Her arguments, backed by data (though I'd be hard pressed to find it again, since it's been a couple of years), included pointing out that IDEA schools had performed better specifically because they'd expelled and otherwise removed from their student pool the students least likely to perform well.
This served two goals for the charter schools: It insured their standard test performance would be better, and insured that the surrounding public schools (which are required by law to accept those students IDEA had refused to teach) performed more poorly on average, regardless of how well they teach.
Similarly, the special needs programs in public schools must accommodate children that charter schools aren't required to accommodate, often through exceptions and guidelines built into the charter program by the legislators who have ideological reasons for wanting more privatization of schools, received contributions from the very companies that will operate the charter schools, and have every reason to see charter schools work, at least on paper.
In the case in Austin, there was a pretty wide variety of benefits the charters received that the public schools they replaced did not. For example, a large budget was provided to renovate schools for the charter; when those same school buildings had been in service as public schools for decades, and the budgets were woefully inadequate to keep it up to date. Students obviously do better with better facilities.
I'm still open-minded about the possibility of alternative schools. I hate public school, as I know it; always have, as I was very poorly served by public schools (as a bright kid with a learning disability). But, when the political/economic deck is stacked soundly in favor of the charter school, as it was in the one case I really studied, if they perform about the same or slightly better than public schools, I have to assume that the only benefit is to the corporations making a profit on the schools.
My confidence in market solutions to problems like education has somewhat waned in the face of evidence that the market tends to favor the best students and forgets those that are challenging. I don't know what the right solution is, but I'm mistrustful of the motivations of the charters I've seen.
This is a nice piece by a professional consultant. My bet is that his company ESI consultants gets a fair amount of business from those invested in charter schools.
39 comments
[ 3.9 ms ] story [ 59.0 ms ] threadIf that wasn't controlled for, then it's less that the schools themselves are necessarily any better (though that could also be the case) and more that students with already better chances of doing better are being "self"-selected into a different population.
Is there evidence to back up this statement?
It's getting a bit out of hand where every single statement online is now getting a [Citation Needed] afterwards. There isn't a randomized controlled study for every single aspect of the universe, we'll have to use intuition occasionally.
I'm sorry. I care. (sorry (sorry)) ((((((sorry))))))
I wasn't asking if there was evidence just to be annoying, I was asking because the statement was exactly opposite of my expectation and I wanted to know if there was evidence to back it up.
My personal experience with charter schools is that it's where many parents "dump" kids that were either kicked or failed out of public school. This comes from many conversations I've had with a high-school teacher I've become good friends at that switched from teaching public school to charter school ~5 years ago. I admit my perspective is very narrow but was hoping there was actual data to support the chart-school-parents-more-involved statement.
First, that parents more heavily involved are more likely to opt-in to charter schools (or similarly though not equivalently, that parents who choose to opt-in are likely to have been more heavily involved). I consider it to be largely self-evident that parents who are not involved are very unlikely to opt-in, and while the converse isn't necessarily true, I think that fact alone would significantly sway the overall correlation of the "non opt-in" population.
And second, that being more heavily involved leads to better outcomes for children, which I feel is self-evident, and is also highly supported by a quick google search for 'involved parents better students'.
If you click links in the article, you will see they compared to kids who failed in the lottery. And how they address other methodological concerns.
One could argue that simply participating in the lotter shows that those kids had active parents, and therefore are self-selecting as the OP stated.
Exactly the point: comparing students with equal motivation and parental involvement, who are separated only by luck, is a convenient way to control for the self-selection bias.
The article's original claim is that charter schools does work for disadvantage and black students.
But that is NOT what the data supports.
The data supports that it works for disadvantaged and black students who have active parents. That conclusion raises more questions than it answers.
I think your theory makes a lot of sense. Thank you for sharing that.
In the media, the message is always about quality of education and high standards -- because the audience is the political class who funds the schools. But the outreach to parents (found in bus shelters, pamphlets, hip hop radio, etc) is a message that usually is around childcare and uniforms. Longer school hours + longer school year == less daycare expenses and arrangements.
In terms of outcomes, it's all over the map. Several charters have closed for poor performance, and some do well.
It's not that hard to explain reversion to the mean. The schools in the lowest socioeconomic areas are generally, a priori, the worst of the bunch.
Consequently, just about anything will perform better. This is especially true if there is any ability to self-select (lottery is insufficient evidence as expelling "bad" students back to the public school is the same as self-selection).
When charter schools are compared in the suburbs, they generally perform worse than a public school.
If there is a statistically significant difference in outcomes between charter schools and non-charter then I don't think we can dismiss it on the grounds you state.
But you did bring up an excellent point in the self-selection bias. We are potentially comparing apples and oranges if the charter schools get a different sort of student to teach.
The only issue is when trying to compare private school type X vs public school Y. Public school Y is required to take all comers ... no exceptions. If you want to claim that Private School X is better, worse, etc. you need to control for that because the worst students consume the most resource--even the expulsion of just a very small number of students can make a disproportionate difference.
Also, we have, to my knowledge, one of the only compulsory middle school computer programming courses in the state (that's what I teach), maybe the country--the goal being that all of our kids have solid computer programming and web development fundamentals under their belt before they graduate high school.
Also, it would help to have special schools for children of parents for whom education was not part of their upbringing --those kids need some full blown indoctrination to get them to see schooling as an essential part of their lives and not something which interrupts their lives.
Even with a lottery, you have a bias towards students with parents that enter the lottery. People with caretakers that are less concerned about their well-being probably are less likely to be entered in the lottery.
You should really compare students who attempted to get into a charter school vs those that didn't.
If you want to show evidence of charter school success that's definitive, you would need a randomized controlled trial. split a school district into half charter schools half public schools. And you wouldn't let the charter schools kick out students who underperform or have behavior problems, or otherwise make sure that such students don't skew the charter school's metrics. See what happens.
I guess I'm not sure how much the success of a student population can be attributed to teaching methods, and how much to aspects of the child's life. We all know that wealth has dramatic effect at college attendance and graduate rates for those that attend college.
Source: http://users.nber.org/~dynarski/kipp_may05_2010_web.pdf
Anecdotal observations:
From what I could tell, the success of the school was due to three things:
1. Parents who had available time, money, and energy to devote to the school. This would likely be less true in a lower income area, but as has been mentioned elsewhere, could account for higher average performance of schools that parents actively chose to put their kids in, regardless of location.
2. Excellent staffing, mostly due to the attractiveness of working in an environment with more freedom to innovate.
3. That the environment was different enough from the huge suburban public schools nearby that some students would perform better in that kind of environment.
One other thing I noticed that I haven't heard mentioned before, which is a situation that is unique to looking at the upper end of public schools and income brackets rather than the lower: the school I went to was not good at playing the college prep game. All the courses were difficult, so nobody had any "honors" courses on their transcript. GPAs were generally lower. Fewer extracurricular activities were available because of the size of the school. These facts resulted in a lower number of students going on to prestigious post-secondary institutions than you would expect from an otherwise high performing school.
ALL of these observations point to the conclusion that Charter schools are really just an "alternative", and naturally some alternatives will be an improvement and some will be a failure.
I like this article because it indicates that charter schools specifically help marginalized groups (black, poor), which fits nicely with the idea that having alternative environments in general is good for some students, and also matches my personal experience. (I promise this is the only time I will ever claim "white male nerd" as a marginalized group.)
I love the idea of a child getting a chance that they wouldn't normally have. But I wonder if it's at the expense of children of uneducated or uninvolved parents?
The one thing you can never do is blame the child for the parents' level of involvement. But it seems this may be punishing them doubly.
It's a tough nut to crack...
She thoroughly convinced me, that at least in the case of IDEA (the company that was contracted in Austin) charter schools were not superior, especially for the students most in need of better schools.
Her arguments, backed by data (though I'd be hard pressed to find it again, since it's been a couple of years), included pointing out that IDEA schools had performed better specifically because they'd expelled and otherwise removed from their student pool the students least likely to perform well.
This served two goals for the charter schools: It insured their standard test performance would be better, and insured that the surrounding public schools (which are required by law to accept those students IDEA had refused to teach) performed more poorly on average, regardless of how well they teach.
Similarly, the special needs programs in public schools must accommodate children that charter schools aren't required to accommodate, often through exceptions and guidelines built into the charter program by the legislators who have ideological reasons for wanting more privatization of schools, received contributions from the very companies that will operate the charter schools, and have every reason to see charter schools work, at least on paper.
In the case in Austin, there was a pretty wide variety of benefits the charters received that the public schools they replaced did not. For example, a large budget was provided to renovate schools for the charter; when those same school buildings had been in service as public schools for decades, and the budgets were woefully inadequate to keep it up to date. Students obviously do better with better facilities.
I'm still open-minded about the possibility of alternative schools. I hate public school, as I know it; always have, as I was very poorly served by public schools (as a bright kid with a learning disability). But, when the political/economic deck is stacked soundly in favor of the charter school, as it was in the one case I really studied, if they perform about the same or slightly better than public schools, I have to assume that the only benefit is to the corporations making a profit on the schools.
My confidence in market solutions to problems like education has somewhat waned in the face of evidence that the market tends to favor the best students and forgets those that are challenging. I don't know what the right solution is, but I'm mistrustful of the motivations of the charters I've seen.