For the jQuery project we've run all of our community discussions through Google Group mailing lists for the past three years. At this moment the main jQuery group is the second most popular programming group (next to Android developers) clocking in at over 21,000 members. We also have the jQuery Dev and jQuery UI groups. The main jQuery group averages around 83-143 messages per day. I also use Google Groups for discussion on a number of my other projects (Processing.js, Env.js, Sizzle.js, and TestSwarm).
This post isn't so much about the usefulness of mailing lists as a discussion medium, it's the complete failure of Google Groups as an adequate purveyor of public discussion software. For the jQuery project we're already in the process of moving the full discussion area to a forum that we control. We should have it set up, and everything moved over, within the next month or two.
There is one area in which Google Groups continues to shine: Private, or restricted, mailing list discussions. However any attempts at using it for a public discussion medium are completely futile.
The primary problem with Google Groups boils down to a systemic failure to contain and manage spam. Only a bottom-up overhaul of the Google Groups system would be able to fix the problems that every Google Group faces.
To better illustrate the problem, let's step through the common experience of running a Google Group.
The Beginning
When you create a public group everything will go well for a couple days, at most. Without fail an onslaught of spam will start to come through your group - I've even seen it happen within the first day. It happens to every group and doesn't matter how well you advertise it (or try to hide it). After having watched Google Groups for as long as I have I can only assume that there exists no spam filtering whatsoever. Or, if there is any, it's the most grossly incompetent spam filter I've ever seen.
When these spam messages start to come to your group a couple things will happen. First, you may not even notice the spams coming through. Since you're likely reading the list in a competent email client (such as Gmail) it'll detect the messages and dump them into your personal spam folder. Don't be surprised if you visit your group and see a pile of spammy messages sitting there greeting your new visitors.
Most email client spam detection software is smart. It looks for common points of failure and tries to take care of the root problem. One such tactic is to realize that a lot of spam is coming from a single address (like a Google Group) and start to flag most of it as it comes through (regardless of the actual content). The result is that much of your list is being flagged as a false positive. In the case of Gmail people will then start to un-flag the falsely-binned group messages. This works well until the system starts to think that all group messages are ok - and here comes the spam again.
To fight the spam you'll likely start flagging emails as "spam" in the groups interface. This works well (the user is permanently banned and the message deleted) - until a couple hours pass, that is. You'll see the spammer return, with a slightly different username, posting the same exact spam messages. Flagging a user/message as spam does absolutely nothing to train the groups spam detection system (for reasons that aren't entirely clear and only be explained by incompetence).
It's a horrible game of cat and mouse with the spam destroying the quality of your group. It's at this point that you say "enough is enough" and you turn on moderation for your group.
Moderation
Google Group moderation seems like a palatable idea but in practice is aggravating and crippling. To start, it creates a horrible first-participation experience for your users. For example, let's say you go to bed at the same time as someone in Tokyo attempts to post a message to the group; you won't be able to moderate the message through for many hours (and...
Isn't the problem rather that e-mail (or rather smtp) is dead (or should be). It seems to me the only solution to this problem is a web-based BBS or replace e-mail with something else. Or is there a mailling list server that requires users to authenticate before accepting a post?
Not really a problem with email. It's more a series of critical errors on the part of the Google Groups system. If they were to implement Gmail-quality spam filtering then most of my concerns would be moot (since I never would have had to turn on moderation and these sketchy spoofers would've been caught right at the gate).
http://jquery.markmail.org/ seems to have a good chunk of the lists archived. They can probably get you a copy (marked up in XML) if you ask - that might be better than scraping.
This is odd - Sender Policy Framework was instituted to stop spoofing, & Gmail actually uses it. So allowing Gmail spoofers is bypassing the spam filters & SPF.
I used Mailman for the discussion group for beta testers of my app, and people were always confused about how it works or why they were suddenly getting emails from other people. But once I switched to Google Groups, people get it and participate a lot more.
Yeah, there is something of a distinction between a communication channel for something that's already "made it", and a project that still needs a low barrier to entry. You can impose higher hurdles for people to jump over to participate without really losing much if your project is already 'famous'.
However, if you're trying to gain traction, having something that's easy to get started with is important, and Google Groups fits that bill pretty well: 1) people with Google accounts can sign up easily. 2) You can treat it as either a web-based forum, or a mailing list, which allows people to interact in the way that's best for them.
Indeed, I just switched Hecl over to using Google Groups, from a SourceForge mailing list, and so far I'm reasonably happy, despite a few spam problems.
> there is something of a distinction between a communication channel for something that's already "made it"
> MADE IT
This was true for us even on a small student-organized webapp project. Even some of the guys on the team protested using MailMan and went on safari looking for alternatives while we were prototyping, then when the project was officially adopted by the college and all students were told "yeah, the old channels exist, but this is a new, officially sanctioned channel also", suddenly there was a lot more activity on MailMan.
What's a good alternative? The spam problems in Google Groups totally suck, especially for very active groups, but it's a familiar system, so even non-techies can sign up and post fairly easily.
I was thinking of the same thing. What is really needed is a clean group management solution with a nice inviting UI, a good spam filter (something like Akismet) and good access control for various levels of admins. The last would really hit it off with open source projects having a BDFL sorta guy with multiple pillars.
It could even be freemium with free access to public groups involved in open source projects. Maybe even some nice integration with whatever seems logical.
You should chat with the moderators of those groups - I'm positive that they're having a similar experience.
For people that have already been whitelisted in (and assuming that you don't get spoofed) the experience is great. For those running the group? Not so much.
Our list is obviously smaller, but with somewhere around 1000 members, I don't find the moderation duties difficult to keep up with. And we've only let spam through a couple of times.
Rather than central moderation, which is appropriate in some cases but would lead to silly turf fights on others, GG ought to implement a simple voting system, not unlike YouTube comments (where particularly stupid things disappear after 6 downvotes).
Google really ought to consider the idea of putting NNTP back; it's obvious they don't really want to develop Groups as a tool (we can still hope for Wave to save the day, but...) and so GG looks more and more like the zombified corpse of Usenet. Perhaps if there was a push for Groups to have it's own Google Code API we might see some innovation.
Groups does have a 1-5 star rating system; I've noticed spam often gets one-star votes in groups I frequent. But that doesn't really address the problem; they ought to have a "report spam" featuer as well.
This actually happens in webmail quite a bit as well. Spammers open up a bunch of accounts, spam themselves, then mark their messages as "not spam". But the major vendors have figured this out.
This is a problem I see plaguing several code-related groups that I'm following, and it is very annoying (and up till now, mystifying) to see not only showing up in regular message flow, but also in abridged once-a-day summaries.
It's nice to see Google maintaining stuff like Groups and Code, but they don't seem to get very good maintenance. I've been finding myself saying the phrase "I wish they'd move this project away from google code and onto github" quite frequently lately. I suspect I'll be saying something similar for groups very soon..
I've been finding myself saying the phrase "I wish they'd move this project away from google code and onto github" quite frequently lately.
Funny, I usually find myself saying "I wish they'd move this project away from github, provide a proper API-stable versioned release, properly document the project and APIs, and otherwise behave like a well-maintained open source project" a lot lately. =)
If they did all that, then I wouldn't need to fork the project, maintain any local patches/modifications, etc.
You're right that many github-hosted projects suffer from overly-brief documentation. Perhaps Github attracts a certain type of project maintainer at the moment..
Or, "Why I Use Mailman and Spam Filtering". I just talked to a list moderator about this issue yesterday, actually. It's highly annoying and some project contributors have actually posted on lists to say "Sorry, there's too much spam, I'm unsubscribing".
not so easy. email is sent to thousands (if not millions) of address. when someone presses spam in gmail, most likely if the mail is truly spam hundreds of others marked the same mail as spam. the email could be quickly added to their training data, the sending agent black listed, etc.
Now groups is a different thing. How many of you mark messages that are spam? A few I am sure, but the point is the offending person is long gone and the message is already been posted. Harder problem unless you can wall off the source and control the users access.
Google should split USENET from Google Groups. The first is useful, the second is questionable. The first is suffering terribly from it's conjunction (by Google) with the second.
I wish Google Groups worked out better for moderators, because as a user it's the only message system I enjoy using (versus my 300th phpBB or vBulletin registration/confirmation/login cycle).
I know some groups have promoted frequent posters to moderator status and this helps with the workload (though it sounds like the jQuery group already does this and it's still too much).
Even vBulletin requires manual spam control these days.
Spammers are more than happy to pay sweatshop wages to people who manually register and log in to site after site and post their spam until the account is hand-banned by a moderator.
I haven't had much trouble with it. I just moderate the first post by a user and let the rest through. This has prevented any spam from getting through to my 1200+ member group.
Same here. Perhaps I'm not moderating large enough groups for this to be an issue, but it's never been a hassle.
What I really like is that I get mail telling me there's a moderated post, and I can approve simply by replaying to the meta-message.
Now, it is annoying that every so often users with approved posts are not correctly then set as auto-approve, so there's some occasional intervention needed, but Google groups has worked quite well for the 4 or so groups I look after.
Stack Exchange is designed for one type of interaction: answering focused questions.
It's not appropriate for just putting an idea out there and discussing it, or for that matter even having a discussion about answers to an individual question.
I just went to see if my ISP still had a usenet server I could use so I could use a spam-filtering tool at home but it seems that Bell has discontinued it. Looking at a list of usenet servers, it seems that more than half of US ISPs and the major Canadian ISPs no longer have usenet servers :/
69 comments
[ 2.9 ms ] story [ 177 ms ] threadAs far as I'm concerned, Google Groups is dead.
For the jQuery project we've run all of our community discussions through Google Group mailing lists for the past three years. At this moment the main jQuery group is the second most popular programming group (next to Android developers) clocking in at over 21,000 members. We also have the jQuery Dev and jQuery UI groups. The main jQuery group averages around 83-143 messages per day. I also use Google Groups for discussion on a number of my other projects (Processing.js, Env.js, Sizzle.js, and TestSwarm).
This post isn't so much about the usefulness of mailing lists as a discussion medium, it's the complete failure of Google Groups as an adequate purveyor of public discussion software. For the jQuery project we're already in the process of moving the full discussion area to a forum that we control. We should have it set up, and everything moved over, within the next month or two.
There is one area in which Google Groups continues to shine: Private, or restricted, mailing list discussions. However any attempts at using it for a public discussion medium are completely futile.
The primary problem with Google Groups boils down to a systemic failure to contain and manage spam. Only a bottom-up overhaul of the Google Groups system would be able to fix the problems that every Google Group faces.
To better illustrate the problem, let's step through the common experience of running a Google Group.
The Beginning
When you create a public group everything will go well for a couple days, at most. Without fail an onslaught of spam will start to come through your group - I've even seen it happen within the first day. It happens to every group and doesn't matter how well you advertise it (or try to hide it). After having watched Google Groups for as long as I have I can only assume that there exists no spam filtering whatsoever. Or, if there is any, it's the most grossly incompetent spam filter I've ever seen.
When these spam messages start to come to your group a couple things will happen. First, you may not even notice the spams coming through. Since you're likely reading the list in a competent email client (such as Gmail) it'll detect the messages and dump them into your personal spam folder. Don't be surprised if you visit your group and see a pile of spammy messages sitting there greeting your new visitors.
Most email client spam detection software is smart. It looks for common points of failure and tries to take care of the root problem. One such tactic is to realize that a lot of spam is coming from a single address (like a Google Group) and start to flag most of it as it comes through (regardless of the actual content). The result is that much of your list is being flagged as a false positive. In the case of Gmail people will then start to un-flag the falsely-binned group messages. This works well until the system starts to think that all group messages are ok - and here comes the spam again.
To fight the spam you'll likely start flagging emails as "spam" in the groups interface. This works well (the user is permanently banned and the message deleted) - until a couple hours pass, that is. You'll see the spammer return, with a slightly different username, posting the same exact spam messages. Flagging a user/message as spam does absolutely nothing to train the groups spam detection system (for reasons that aren't entirely clear and only be explained by incompetence).
It's a horrible game of cat and mouse with the spam destroying the quality of your group. It's at this point that you say "enough is enough" and you turn on moderation for your group.
Moderation
Google Group moderation seems like a palatable idea but in practice is aggravating and crippling. To start, it creates a horrible first-participation experience for your users. For example, let's say you go to bed at the same time as someone in Tokyo attempts to post a message to the group; you won't be able to moderate the message through for many hours (and...
http://74.125.77.132/search?hl=en&safe=off&rls=en-us...
However, if you're trying to gain traction, having something that's easy to get started with is important, and Google Groups fits that bill pretty well: 1) people with Google accounts can sign up easily. 2) You can treat it as either a web-based forum, or a mailing list, which allows people to interact in the way that's best for them.
Indeed, I just switched Hecl over to using Google Groups, from a SourceForge mailing list, and so far I'm reasonably happy, despite a few spam problems.
> MADE IT
This was true for us even on a small student-organized webapp project. Even some of the guys on the team protested using MailMan and went on safari looking for alternatives while we were prototyping, then when the project was officially adopted by the college and all students were told "yeah, the old channels exist, but this is a new, officially sanctioned channel also", suddenly there was a lot more activity on MailMan.
"This is a mailing list, not a discussion forum. Just post it to the Google Group."
I was really, really confused.
It's really puzzling, since gmail already has a decent spam filter - can't they apply that to groups?
Is Yahoo groups even worse?
It could even be freemium with free access to public groups involved in open source projects. Maybe even some nice integration with whatever seems logical.
They have switched their own support forums from Google Groups to a much more usable classical forum : http://www.google.com/support/forum
Google Groups was such a mess, I never understood what was really going on.
I'd kinda like to see either the Groups team fix the problems or some startup come up and really own that space.
For people that have already been whitelisted in (and assuming that you don't get spoofed) the experience is great. For those running the group? Not so much.
Link: http://freelists.org
Google really ought to consider the idea of putting NNTP back; it's obvious they don't really want to develop Groups as a tool (we can still hope for Wave to save the day, but...) and so GG looks more and more like the zombified corpse of Usenet. Perhaps if there was a push for Groups to have it's own Google Code API we might see some innovation.
Google has failed to contain spam, even on discussion lists for their own offerings.
It is actually good advise to curate your own data.
It's nice to see Google maintaining stuff like Groups and Code, but they don't seem to get very good maintenance. I've been finding myself saying the phrase "I wish they'd move this project away from google code and onto github" quite frequently lately. I suspect I'll be saying something similar for groups very soon..
It seems like a perfect project to use Lamson (http://www.zedshaw.com/projects/lamson/) as a starting point for.
Funny, I usually find myself saying "I wish they'd move this project away from github, provide a proper API-stable versioned release, properly document the project and APIs, and otherwise behave like a well-maintained open source project" a lot lately. =)
If they did all that, then I wouldn't need to fork the project, maintain any local patches/modifications, etc.
You're right that many github-hosted projects suffer from overly-brief documentation. Perhaps Github attracts a certain type of project maintainer at the moment..
Now groups is a different thing. How many of you mark messages that are spam? A few I am sure, but the point is the offending person is long gone and the message is already been posted. Harder problem unless you can wall off the source and control the users access.
I know some groups have promoted frequent posters to moderator status and this helps with the workload (though it sounds like the jQuery group already does this and it's still too much).
Spammers are more than happy to pay sweatshop wages to people who manually register and log in to site after site and post their spam until the account is hand-banned by a moderator.
What I really like is that I get mail telling me there's a moderated post, and I can approve simply by replaying to the meta-message.
Now, it is annoying that every so often users with approved posts are not correctly then set as auto-approve, so there's some occasional intervention needed, but Google groups has worked quite well for the 4 or so groups I look after.
It's not appropriate for just putting an idea out there and discussing it, or for that matter even having a discussion about answers to an individual question.