503 comments

[ 3.3 ms ] story [ 312 ms ] thread
XMPP federation with third-party services providers will no longer be supported starting June 26.

Fuck you, Google. Just fuck you...

Parent shouldn't be downvoted. Depression and/or rage are among the valid responses to this part of the news.
The writing on the wall for this has been up for a long time. Raging for something that has been so predictable is not reasonable, there has been plenty of time to plan for this contingency and act accordingly.
Calling this "rage" is probably overstating the situation a bit. Strong annoyance is probably closer to the truth. But you have to understand, I'm something of an ideologue when it comes to open standards, open source, etc. So this really sticks in my craw.
I wish someone at Google would explain their overarching strategy and target. This piecemeal approach just reeks of no coherent plan.
I doubt they have any coherent strategy in the messaging space.
Throw shit on the wall and see what sticks? Isn't that what pretty much everyone is doing?
Once it sticks and everyone is using it, then shut it down
It's sad but true. I can hardly trust any Google products any more because you never know when they're randomly going to give it the axe for no apparent reason.
They don't appear to have much of a coherent strategy in the product space.
How about coherent strategy within a company?
Well, it is not a big secret on many occasions people tell the outsiders Google would have multiple teams working on the same, if not, similar product, only to find just one will launch publicly, and then 6-24 months later the service will shut down.
If you remember whose Google's customers are--advertisers, not users--coherency is a lot easier to understand.

So it's really easy to predict Google. Just ask the following questions:

- can Google run ads to every user of the product? - can Google extract usage data to build a better user profile, in order to charge more for above ads?

The answer with regard to Talk or any interoperable service, is almost always no to both.

The incoherency we see is related to the fact that not every engineer or manager at Google is fully plugged into the above or sometimes they create a product that fails to answer the above with the hope they'll figure that out later. Sometimes the nature of the product or the fact that it's an open standard makes it impossible to evolve the product to get the questions right.

I loved that I could use my native Mac messaging app to communicate with colleagues in google chat. It was real-time and all messages were logged and available in gmail. I guess that's going away.

I agree. Where is Sundar? Larry? Sergueï?
A googler already summarized their strategy succinctly up-thread [1] as:

Google is a feudally organized collection of fiefdoms and this shows in their product strategy.

[1] https://news.ycombinator.com/item?id=13951918

yep, and neither Larry nor Sergey are strong, commanding personalities like Jobs was w/Apple - they're mega geeks, good at what they do, but at the level of the game they're playing, they need a firmer vision/hand to manage the direction of the company... and remain open at the same time instead of building walled gardens. They're slowly turning into Apple....
Seriously, Google owes me $50 for my obihai router I bought solely for free google integration. This is the problem with building anything on top of Gsuite -- they will screw you over at a random moment and deprecate whatever service you may have been relying on.
I think it'll still work, right? Google Voice is still supported.
Don't give them any ideas..
I think this relates to apps that utilise gtalk. Obihai is integrated with gVoice.

Is there something I'm missing?

OK, if they still support XMPP for Gvoice then I am an idiot and I take back my comment. There was a threat awhile back that they would remove XMPP support for Gvoice (and thus I would have to pay for a service to use with my obihai). But that seemed to have gone on the back burner for awhile.
>owes me

>solely for free google integration

Dude, you get what you pay for

exactly. thus I pay for Anveo...8 or whatever it is a month. Never have to deal w/Google.
I could probably afford to drink one fewer craft brew per month.
do it. they have so many features.. you get your own fax # too, plus you can fax whatever.. it's a total solution. You're right - we waste money on stupid stuff (not saying beer's dumb, love it myself) but won't pay a little for things that have value and aren't associated with google or advertising...
I'm going to take your suggestion! I was joking about google, they are pretty amazing. But I need more power, and if I can get it for a few bucks a month it's well worth it.
time to switch to matrix I guess
Google doesn't support the Matrix protocol either, unfortunately
But seeing as we can't trust Google to keep supporting their own stuff from one moment to the next, nor can expect their own services to interoperate with each other, perhaps we shouldn't particularly care which protocols Google supports.
I meant stop using google for communication for good, even if they'd supported it, they could drop the support any time
I'm sorry, I'm not really into chat protocols, but what does XMPP federation mean?

For what I understand I will be able to login via Pidgin into my Gmail account, but not from a custom domain, am I right?

Either way, GTalk was such a clean, centered and small IM experiencie +10 years ago, Hangouts is just a mess.

Federation is where different service providers can communicate with each other, like how email works. Meaning, you could login to Gmail (whether gmail.com or custom domain) and send messages to users with accounts on non-Gmail IM providers (practically speaking this meant hobbyist, corporate, or university IM server deployments; Google was the only major consumer IM provider to adopt a federated protocol).

The shutdown is unfortunate news because the only way IM is going to be as open as email is if two major consumer providers step up and do it. We had 1 and just needed 1 more. Now we have 0.

I believe that at one point a Google chat user could talk with a facebook chat user.
Federation means not having to reinvent the chat wheel for the umpteenth time.... Who was that quote from - physicists stand on the shoulder of giants... software and IT stand on each other's toes. Sad but true.
Indeed! Much different leadership back then:

Service choice is something you have with email and, for the most part, with your regular phone service today. [...] Unfortunately, the same is not true with many popular IM and VOIP networks today. [...] The Google Talk network supports open interoperability with hundreds of other communications service providers through a process known as federation.

The only benefit I see out of this is the removal of SMS support in the Hangouts Android app. I like my communication channels separate. I don't like Hangouts or Facebook Messenger wanting to control my cell phone messages (SMS).

This might be a naive question, but will chat within Gmail still work? Is that chat provided by Talk or Hangouts?

> I like my communication channels separate.

You could always keep them separate, if you wanted to. This feature was totally invisible to people who left it turned off.

I personally want to have all my messaging in one place, which used to be possible. Now everything is fragmented, and I constantly have to remember how each person in my life prefers to communicate, open the right app, find their name, and then message them. It's incredibly frustrating to have a worse 2017 messaging experience than I did in 2007 with my BlackBerry.

> This might be a naive question, but will chat within Gmail still work? Is that chat provided by Talk or Hangouts?

Chat in Gmail is Hangouts.

> Now everything is fragmented, and I constantly have to remember how each person in my life prefers to communicate, open the right app

I think this is true to an extent, but I think messaging isn't as simple "get this text to this person." Different communications channels have different contexts, implied importance, etc. Making them too unified papers over and confuses those social differences. For instance, it would be super weird if a coworker texted me instead of IM'd me about some everyday work thing.

We should solve that within the same protocol. Allow multiple named conversation streams for any combination of contacts with an interface to filter them (private, work, dating, family).

I also want priority of messages to let the receiver handle them differently (low/normal/high). Default ‘normal’ and changeable within reach, but quite invisible. Anyone can then let me write a ‘high’ priority message that makes noise in the middle of the night, or if I suspect they're resting and I want to send a cat picture I'll use ‘low’.

you're probably in the minority. When you look at the popularity of imessage and one of the big reasons people complain about allo is that there was no fallback.
I still use Gchat over Hangouts because it's nice seeing my contacts list sorted by who is online, as opposed being ordered by recency of last communication.

So many of my Gchat conversations have started because of the online status. I've hit up friends that I haven't talked to in over a year and had really great, synchronous, conversations because of it.

It sounds like that will continue to be possible — "Third-party XMPP clients will continue to work with Hangouts for 1-on-1 chats."
I remember there was a rather big outrage when Google first released Hangouts and it showed everyone as online all the time. Eventually, they made it so you only show-up as online if you've been in the window in the past 15 minutes. But they're still not ordering the contacts by online status. This is just one more of those "social" things that Google has never been able to grasp.
"online" as a concept has become a bit meaningless with mobile clients.
This is just the further erosion of manners, respect, and privacy that is so pervasive these days. No one is ever allowed any down time, we're always online and anytime is a good time to talk. To me this feels like a bad thing but I guess others don't see it that way.

People use to never call you after 9pm or during dinner time. If your phone rang during a meal you didn't answer it. If you didn't want to chat with people you didn't log into your IM client, if you wanted to chat with someone specifically there was a first class UI element for setting your status so you could tell others to leave you alone. Email and SMS were black holes where you only knew if your message was received via a response.

Now everyone is always online and it's never unacceptable to call, text, or IM someone. Read receipts are standard so you know if someone got your message. If someone's phone rings while you're eating, you're lucky if they apologize before answering, and you've won the lottery if they excuse themselves from the table.

It seems like we're all just sitting around waiting for an interruption to distract us from the life in front of us.

Idk... you damn kids need to get off my lawn.

Is your phone's Do Not Disturb feature broken?
My phone is always on silent and usually in another room. I have a first generation Mi Fit band that is setup with unique vibrations, colors, and quiet hours so that I can keep abreast of communications without persistent flashing notifications or interrupting activities.
You are allowed down time. If people don't respect/understand that, it's their problem, no yours.

I don't answer the phone when I eat or after 10pm. Same thing with text messages/emails. "Do not disturb" mode is there for a reason.

"Do not disturb" is there because people do not respect each other anymore.
(comment deleted)
> Now everyone is always online and it's never unacceptable to call, text, or IM someone

Eh, but you don't have to answer. I think the paradigm has shifted from a TCP-like approach where you're expected to respond in real time to a UDP-like approach where people fire off best-effort attempts to reach you and don't mind if they don't hear anything back.

At least, that's how it is among my peers. Reach out whenever, just don't expect a response.

>This is just the further erosion of manners, respect, and privacy that is so pervasive these days. No one is ever allowed any down time, we're always online and anytime is a good time to talk. To me this feels like a bad thing but I guess others don't see it that way.

I agree 100%, one of the only reasons why I prefer using Gchat over Hangouts and now I don't know what. Just because my Gmail is open 24x7 doesn't mean I am available to chat all the time to everyone.

Maybe I am wrong but I think this comes for people like you and me for whom desktop/PC is the primary utility over smartphone.

A couple of questions concerning this announcement.

1) With the removal of SMS from Hangouts on Android does anyone know if this affects SMS in Hangouts on ChromeOS? 2) Is retiring a euphemism for killing in terms of Google Labs additions or does this mean they will still work, but won't be made available to new users?

Thanks.

God damn it. I do not want to use the Google Hangouts experience. Talk is a simple, elegant chat application. That is all I require and all I desire from Google. And I do not want to be patronized about how I actually want more "integration" with google products.
Hangouts is solid proof that no one at Google uses the multiple account support. If they did, it wouldn't be so awful.

I click a hangout link for a work call and get "no one in the call". I have to add authuser=1 to the URL manually to see that it's actually that I don't have permission to it.

This. Happens all the time and then people say, "You never showed up." The authuser=1 tip is great. Thanks for sharing that!
At the bottom, in tiny font, there is a "change account" link that gives you the ability to switch that call to a different Google account.

It's tiny, you might have missed it...

Sure, but it's silly that it's required, and the "no one here" is hugely misleading.

Google knows which accounts I'm logged in as. They know which accounts have access to the meeting. If I'm logged in as one of these accounts, why not do it automatically for me?

This happens to me on GSuite all the time, I actually stopped using it and just call in a separate conference line. I would show up, usually first to the meeting, and it would just sit and say no one is around. Then I could logout, log back in and I'd see everyone else was suddenly there. I missed 2-3 meetings at my newest job because of this. This is the paid version of Google too, absolutely not acceptable. My old job had Office 365 and I long for it, it's just not as competitively priced for SMBs.
Hangouts is also solid proof that no-one at Google uses a non-Chrome browser on Android: trying to open a meeting hangout invite with anything other than Chrome as the default browser … opens a link to Google Play, not the Hangouts app.

I think this is because they use a different URL in meeting invites than the one registered for the Hangouts app.

It's been a bug for ages, and AFAICT they just don't care.

GTalk was an amazing simple-to-use application. Should I say it was simpler than Skype. I don't like hangouts at all.
simple, elegant, compact, cute emojis, and my preferred method of communication for more than a decade.
Their Chrome plugin was also wonderful. Keyword "was". It used to be a discreet window that persists across tabs, which you could minimize and expand from the lower right corner of your screen.

They killed it and now it's an awful behemoth google hangouts window. Really disappointing.

It's like designed for legally blind or touch screen and the green makes me want to vomit
Yes, I loved the old Hangouts extension. By far best experience of any IM app I've used (and that's a lot, unfortunately).
https://github.com/dhleong/hangr

is a really neat replacement. Unfortunately I'm not sure how much longer hangouts as a whole is going to last (didn't they say they were discontinuing it from consumer space?)

I just had an image of myself posting the headline:

"Google discontinues Gmail and Search"

as a way to kickstart a conversation about our reliance on services, and whether we can blame companies for not subsidizing services we like and so on.

But then I realized it would just go sideways into a discussion about click bait and appropriate headlines, so I am posting it as a comment.

It'll be around the same time as Amazon discontinues EC2
So in the next 5 years. Maybe sooner, considering HN's disappointment with their pricing, service offerings, and recent outages.
Yes, it won't happen.

But suppose, for whatever reason, they decided gmail was not economical and was not bringing in enough money.

Do you think they are obligated to keep it around because our lives are on there?

What is their obligation, morally, and what is just a "nice" thing to do?

> Do you think they are obligated to keep it around because our lives are on there?

> What is their obligation, morally, and what is just a "nice" thing to do?

I'd say their moral obligation, in a practical sense, is to let people extract their email from gmail before it's taken down. As of the discontinuing of Google Reader, that seems to be something they understand.

The Gmail part would go something like: Google discontinues Gmail and Google Inbox is the replacement! Then a year later, Google intros Mail from Google, a new webmail client that is like Inbox, but has more of the classic email interface UX that users expect. A year later, they announce Inbox is going away and everyone should move to Mail from Google, which they have rebranded "Gmail".
I am reminded of the scene with the Slurm Queen in Futurama talking about "reintroducing Slurm Classic", as a jab on Coke.

Realistically, if Google discontinued Gmail, people would move to other email providers. There's plenty of big ones.

BTW: Want to avoid being caught in that eventuality, but don't want to stop using Gmail? Buy yourself a personal domain name (for your last name or some such) on https://www.gandi.net/ and set up a redirect to your @gmail.com address. Your email is now john@smith instead of johnsmith@gmail. Congratulations, it looks a thousand times more professional, and you have it for life; if gmail dies, you can move providers without changing your email.

Except it doesn't look more professional because you could only get smithfamilyemails.com or something equally unmemorable.
3 letter non .com domains are your friend, plenty of those left for cheap
then all those signup forms with bad regular expressions can ruin your afternoon. I maintain an obscure .com only because the various gTLDs i could use regularly fail these naive validations. Some sites dont even properly handle ccTLDs
>and set up a redirect to your @gmail.com address.

What do you recommend for this?

I've been wanting to do this for some time, but that means I either need to:

* Use GMail for work, which does not seem tailored for personal used (it's paid, I can't easilly find the price and it seems a bit odd to use Google solution to avoid problems with Google)

* Send all mails to my personal server and setup a redirect there to GMail (which forces me to make sure that my server is always up, so I might as well use my own mail server)

Is there a better solution? I wish there was some way to define e-mail alias at the DNS level.

I use Gandi's native email redirects. If you use their DNS, you can set up to 1000 redirects from and to whichever email address you want, per domain.
Notice how Inbox has a "Gmail" link? Even Google appears to still concede that Inbox is missing sufficient features that some people need to be able to get back to Gmail from time to type (tried getting Inbox to stop putting things in your spam folder? Yeah, it's lacking a "Not spam" button)
I was going to say it didn't have a way to edit your filters, but it looks like that has appeared now. They're adding things bit by bit.
Reading startupdiscuss' response, I was saying to my self, no way Google discontinues Gmail, that's just silly. Then I read your reply and my heart rate went up. I've really got to look into alternative paid replacements for Google's services. I have my whole domain tied to my Google account, and that's making me feel uneasy now that I think about it.
In a den of lions when saying this, but companies that do that have too many engineers.

There's a "right" size for any project and going over it's hard to avoid over-thought, over-complicated, user-confusing, pedantic, semantic things.

This isn't the trite "too many cooks" argument. It's that the right kind of quick-and-dirty shortcutters who value leaving things alone produce better products; they use simpler paradigms and abstractions, have better combinable primitives, communicate purpose more effectively, and produce overall better tools.

The corollary is the stupid thing you can hack to do what you want is often better than the smart thing that is very carefully designed to do what you don't - because one of them you can technically use and the other you can't.

This is why earlier versions are frequently better than later ones and earlier products seem to do more, better, and faster. It's why smaller companies seem to produce better things, and counter-intuitively, products that take less time to finish (say a few days) seem to do better and gain more traction then the 6-month effort.

Some things really are profoundly complex, like databases and operating systems. Most things however, despite much we'd like to pretend, are not in that class of software.

And yes, these claims are cultural bomb-throwing. But the evidence for it is solid.

I think it's just as easily a sign that you might have too much or too little of a marketing department. Too much marketing influence and they create busy work for themselves like rebranding your existing products every year or so to keep them "fresh". Too little marketing influence and engineers are free to build unmarketable products without clear brands or focuses, and (closer to hand to this discussion) free to ignore the built-in good will and momentum of maintaining existing brands and products.

Sure, it's easy to blame engineers for focusing on new shiny ideas and only wanting to work in fresh codebases and "new" products, but it's just as much the fault of their management and the company not trying to adequately control its own brand(s) and how its users perceive it.

I'm contracting with Google. This is exactly the kind of argument I've been tying to make to my boss for over a year but to no avail. They come up with pet names for me to express how weird they think that type of engineering is.
"Lazy" like it's an insult when in fact it's the highest praise?
When Google Talk is discontinued, folks will gravitate to the next IM message like WhatsApp, Facebook messenger or even Twitter DM.

Gmail, well there are other client to use.

Search, umm, what can we do here? DuckGoGo?

I agree with you.

Do you think Google has a moral responsibility to provide us with tools and time to make the transition?

And, how good do the tools have to be? Just a text dump is okay, or should you be able to give it your credentials and it can do the transfer? And then who decides how many services it should make easy?

Are we being responsible now by not thinking about it?

(comment deleted)
A lot of my friends that are friends with me on FB for some reason prefer DMing me on Instagram... Such a weird trend, but it's honestly a thing amongst our group. Either that or Snapchat group chats.
Same here, weirdly enough. In order of most messages exchanged per service, in my friend circles it goes:

1. WhatsApp

2. GroupMe

3. Instagram DM

4. SMS

5. FB Messenger

6. Wickr

Twitter DM is bad. Messages don't appear "immediately" so you often end up with overlapping streams of conversation. The Android app's DM interface is so slow (it freezes for a long half-second every other word) that it's almost impossible to use (thankfully the mobile site works well enough). The best tactic if you're forced to use Twitter DM for an extended or even semi-real-time conversation is to move to a different IM client.
>XMPP federation with third-party services providers will no longer be supported starting June 26.

What does this actually mean?

XMPP Federation allows servers to talk to each other similar to how email works, and which is why XMPP addresses have always looked like email addresses. Up until June 26 you could (with some effort) run a server on your own domain and still chat with people on Google's XMPP servers.
What does this mean for the ability to search Hangouts conversations from the Gmail "chats" label? As far as I'm aware, there's no native search in Hangouts itself (e.g., https://hangouts.google.com ).
Here's what I don't get about Google.

A few years ago, they were one of the only players in the IM game, alongside MSN, AOL IM, ICQ... have you heard about any of these lately? Of course not, they're all dead.

Talk was built on top of XMPP, an open protocol, which helped its popularity as third party clients could connect to it. Talk was also built in to gmail, and at the time that was revolutionary: A fast and lightweight chat app, right in your browser. In (one of?) the most popular mail service providers at the time!

Google has failed to capitalize on any of that. They completely ignored Talk. But then when Facebook did the same thing, oh suddenly they had to compete. So they rebuilt Talk on top of a new protocol. This time, it's proprietary. This time, it's much slower in the browser. Oh and you lose half your contacts if you upgrade to it. But at least now it works on phones?

So they tried building this new closed chat ecosystem for no good reason, and they used their Android market share to do that. People didn't like it, still used Whatsapp, still used FB Messenger, still used Viber, and the now hundreds of alternatives there are, all incompatible with one another because everybody's gotta reinvent the wheel.

You know, I can get behind that XMPP wasn't up to the task - I tried dealing with XMPP myself and it's a frustrating piece of work. But the way Google has treated IM is appalling. Really backwards. They built a good product, then completely ignored it, then built another in an attempt to reinvent it and become more locked down, butchered the old one and are now losing everything. Who's making these decisions exactly?

Matrix is probably our best bet when it comes to open chat protocols, but it's honestly not mainstream ready. In the mean time, I use Discord (https://discordapp.com/) for essentially all my communications. I have completely moved off Talk, Hangouts, Skype and even most of IRC (which has frankly fallen way too far behind more recent comms tech, even as an open protocol). It's proprietary, but at least it gives me text+voice (+ soon video) and doesn't suck - and there is no open choice I can make at this point that is approachable enough that I can convert people to it.

Not just failed. Google betrayed XMPP. You can blame Eric Schmidt for it. He gave some lame speech about how other instant messaging services aren't playing fair, so Google should also become a walled garden. And that was it. They made Hangouts and stopped caring about federated IM idea.

It's a wonder we can send e-mails between many servers. Imagine someone like Eric Schmidt driving it. We'd be stuck with incompatible AOL and Compuserve forever.

And about XMPP shortcomings - sure, it's not perfect. And if Google thought they can do better, why didn't they propose some IM-next as an open federated or P2P protocol? Because "don't be evil" is off the table I suppose.

People tried proposing to use Discord for me, but I'm really not interested in another walled garden closed protocol, without FOSS clients and servers.

If ever there was a software example of killing the goose with the golden eggs.

Imagine if they thought like that about email?

Google Wave was supposed to replace email.

But actually they replaced email with gmail. Now you can't send email from your own server because it will be blacklisted on gmail by default and noone on gmail will receive these emails. So you have to use "cloud email providers". On Android they changed "Email" client to "Gmail" client.

> Now you can't send email from your own server because it will be blacklisted on gmail by default

That's not correct. I tried sending e-mails using my own local SMTP server, and it comes to Gmail just fine. Not that it's a good idea in general.

Why do you say it's not a good idea?
Managing your own server is in general pretty hard (for sending or receiving). Unless you want to invest a lot of time in security and so on. It's simply way overwhelming. And if you don't invest in security, you are inviting trouble.
Sure. I've been running my own mail server for almost 20 years, but you seemed to imply that the bare act of sending email from a personal server to Gmail was a bad idea for some reason.
I've been running my own email server since... forever. In 1985 it was uucp ! paths and sendmail. I got a "for real" domain just for email in 1993, still over uucp dialup. DSL always on in 1999. But it does become harder as time goes on.
On the list of computing tasks I do every day, running my email server is WAY down there on the list of difficulty. Email servers are probably even easier than web servers.
Which doesn't necessarily mean you take necessary steps to seriously protect it. Or may be it means you are an expert and all that is trivial for you. But in general it's clearly not.
I guess my point is that you need to know your audience. Here on HN, running your own server is not a rare skill.
Stop spreading FUD.

I've been running my own email server for a year without delivery issues to Gmail or Outlook/Office 365.

Just make sure your VPS IP is not blacklisted, and play the DKIM/etc game. You'll be fine.

It’s not that easy.

I’ve had proper DKIM, proper SPF, my IP wasn’t blacklisted anywhere.

But I still ended up in Spam.

So I had to send for about a year emails to friends every few days, replicating our discussions on WhatsApp etc (so they’d have organic content), and I’d ask them to mark them as "not spam".

Now I end up in their normal Inbox.

EDIT: Why the downvotes? Please comment instead what you think is wrong – or how else do you think I should get my personal domain trusted?

That is correct, the age of the domain and the "not spam" click matter a lot.

If you want a new domain and have little traffic (e.g. personal domain), you're fucked.

(comment deleted)
> EDIT: Why the downvotes? Please comment instead what you think is wrong – or how else you think I should get my personal domain trusted enough?

I don't understand the downvotes either. So I upvoted your comment to counteract them.

I've found myself doing that more and more lately. I'll upvote a comment I disagree with if it was made politely and in good faith, and it appears to be getting downvoted unfairly.

UPDATE: In case someone ever wants to know more details about why my server failed verification, here are checks for that domain I took when changing DKIM keys last (today):

http://i.imgur.com/9zqILRk.png

https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3akusc...

http://i.imgur.com/VIH4jlt.png

thanks for making me learn about mail-tester.com though you should probably obfuscate the email addresses in those images.
Don't worry — those email addresses in the images are public ones controlled by me.

I even have one of them in my HN profile if I recall correctly.

Thank you for this.

I plan to roll my own email server in the near future, due to legal and privacy concerns. As I understand it, in the US email archives aren't protected under the Fourth Amendment because they're asking for data from a third party, and the Fourth Amendment only applies to self-incrimination.

IANAL, but...

5th amendment is self-incrimination, 4th is unwarranted search and seizure.

I don't know the current law, but it used to be that e-mails left on a server for more than 6 months were treated like abandoned property, which has a lower-bar to clear for the 4th amendment. I seem to recall hearing that this was changed a while back though.

How the 4th amendment applies to any case is complicated though because it's very much a sliding scale of intrusiveness, with higher levels of intrusiveness requiring more cause for searching.

As I understand it, in the US email archives aren't protected under the Fourth Amendment

Currently, as I understand they are not. But, there is legislation [1] which proposes extending Fourth Amendment protection to e-mails and communication stored on your behalf on third party servers. I've already written to my Congressperson in support of that bill.

[1] https://www.congress.gov/bill/115th-congress/house-bill/387/

Yeah, people can still run their own mail servers without too much work. I've had one for a couple years but I ran into issues with Google tagging my mail as spam at first. It was a while back but I think they want domain owners to verify their ownership with Google's webmaster tools.[1]

My server worked better after adding a TXT record with google-site-verification + all the normal DKIM stuff.

EDIT: here's a better link https://support.google.com/a/answer/183895?hl=en

[1] https://support.google.com/webmasters/answer/35179?hl=en

The thing is, and this is the current state of email in general, aka not just google, using an email server other than from [big cloud], like say from my local ISP or a large shared hosting provider, you will run into issues. Usually fine, but every few months your (my) server's ip will randomly get blacklisted for a few days, and for a small biz you'll have customers occasionally not receive your email.

Random blacklisting maybe is less of an issue if you run your own server with a unique ip only you use, but for most people it's getting harder and harder not to use gmail/outlook/lycos.

So ya fuck spammers.

I built my own mail server, and use it for every day communication. If you do everything correctly (mainly setting up DKIM and SPF), you get pretty much trusted by gmail automatically. You might have to make sure your IP address isn't already blacklisted, though I didn't have that problem.
Might have to give it a month or so to settle down aswell and instruct users to mark email as non-spam for automated emails (which helps)
Same here. My personal mail server is running for 4 years. Never had problems being blacklisted by gmail.
I have been running my own mail server with no problem.
> Google Wave was supposed to replace email.

At least Wave was an open protocol including federation, even based on XMPP in fact. It maybe was the last thing before Google transitioned into a mostly walled garden.

> Now you can't send email from your own server because it will be blacklisted on gmail by default and noone on gmail will receive these emails.

I run a small mail server for some years now, no issues sending to gmail.

"On Android they changed "Email" client to "Gmail" client."

No, they didn't. Those are two separate apps. The Email one is part of AOSP. The Gmail one is part of Google Apps. Although many OEMs roll their own email client, as part of their custom skin.

On my Nexus 4, (Android 5.1.1), when I click on what used to be the email icon, I get a screen that says "Email has moved. The Gmail app now lets you view all of your email accounts." It has only one option (and a help link). That option is "TAKE ME TO THE GMAIL APP".

How do I get to the plain email client?

I run my own domain and mail server, no problems emailing major providers, including gmail.
> Imagine if they thought like that about email?

Exactly :) I updated my comment above.

What do you think the end-game for Google's various extensions for e-mail are?
...and people switched to slack.
(comment deleted)
After Gtalk was killed in Google Apps, everyone in my company switch to Slack. Hangouts is a botched POS. None of us want to use video.
... and end to end encryption would be a thing.

Open federated protocols with no obligation for providers to upgrade as the security landscape changes are a betrayal of user security and privacy of the highest order.

HTTPS is only becoming reasonable because Google can use its monopoly power and forced upgrade mechanism to bully website owners into adopting better practices. If we had a fragmented ecosystem of many open source browsers with user-consent upgrades and similar market share like the open web people wanted, certificate transparency would remain academic.

Personally, I consider the median advocate of open federated protocols to be more culpable for the wholesale violation of user privacy than the median NSA staffer.

No, closed ones are a betrayal. If someone doesn't upgrade, you can easily exclude them from communication and even build such logic in the protocol itself. It will pressure them to be up to date. I don't see an issue here.
You can't do that if the ones that refuse to upgrade are some of the bigger providers on the network.
You can if you care about security. That's exactly how the very same Google Talk was cut off from the rest of the XMPP universe, even before they decided to shut it down (they refused to support server to server encryption).
Isn't Hangouts mainly some kind of video chat? I've never used it because I'm not interested in video chat ...
No. It's a standard (and surprisingly shoddy) instant message service that also integrates video chat.
It does text and audio too. I mostly use it for text.
If you're looking for a future in federated chat: matrix.org. I think there's real potential in the protocol and Im slowing moving most of my chat activity there.
Matrix looks interesting, though I'm not sure why they use HTTP for transport. Sounds a bit counter intuitive, since HTTP was never built for proper duplex communication. XMPP (in the browser) had to use BOSH to work around that, then came WebSockets, WebRTC and so on.

Another intereting one is Tox.

Less problems with firewalls?
Perhaps this is the point we need legislation to require that IM services with above X users in the relevant jurisdiction must provide a full, open, protocol and enable interoperation via the published protocol? That's the only way I see the companies involved would move to enable cross-platform communications?

Aside:

>It's a wonder we can send e-mails between many servers.

I have a shared account on a small ISP, Microsoft won't receive our emails, even when replying to an email from a Microsoft user, even when user whitelisted. [It's probably the same with some other suppliers?] The only way to reply to people using Outlook/Hotmail/Live (I'm a user of hotmail for something like 16 years) is now to send via MS servers - basically had to make an account for this purpose.

The sending IPs aren't blacklisted with Senderscore (or other blacklists, I mention senderscore because they apparently provide domain reputation services for MS); some "related IPs" have medium reputation scores (senderscore 72).

Yeah, that's one of those problems which are hard to solve without global collaboration, and current major IM participants are too selfish and greedy to understand that concept.
I'm not a big fan of legislating technical standards, but in this case I think I could support it. IM has become a fundamental communication mechanism. Imagine how held back society would be if we had 50 different voice standards rather than the singular phone system we have now. IM needs to be forcefully standardized.
Don't we already have GSM and LTE, and who knows what else? And I'm not aware of any legislation for the use of those over others. As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.
Right, the point is that you don't have the right to use whatever frequencies you want because the usage of the frequency spectrum is highly regulated, and for good reasons.
> As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.

Point I ("I have the right to use the frequencies") directly depends on point II ("whatever format I want"). When applying for using a certain frequency band, you have to specify usage and purpose, and you can't just change your mind, i.e. you can't apply for GSM frequencies and say you will be running a GSM network, but then run the PavelLishinOverAir protocol.

>When applying for using a certain frequency band, you have to specify usage and purpose, and you can't just change your mind, i.e. you can't apply for GSM frequencies and say you will be running a GSM network, but then run the PavelLishinOverAir protocol.

While this is mostly true, in practice (at least on the 450-470 MHz band I work with), you can give fairly vague purposes, like "telemetry" or "SCADA", and it doesn't matter what vendor or what protocol you use. In fact, there is no section of the licence that says "I am going to use protocol X on equipment Y manufactured by vendor Z" or any part thereof (emissions designators are a different story).

I don't have any experience will cellular licences, though. Given that carriers frequently repurpose spectrum as technology changes, I can't imagine their licences say "GSM only" or anything like that. The band that used to be 3G GSM is being used for 4G LTE on my phone right now.

This likely differs a lot between countries (though there should be a common core due to ITU of what is being regulated how by whom); in my country the cellular frequencies were auctioned off for dozens of billions of euros to the cellular service providers, with strings attached, e.g. that the networks had to at least use and support 3G.

The same regulatory body (BNetzA) is also responsible for licensing telecommunications providers in general. This is probably the crunchpoint - if you're not a licensed telecommunication provider, it's simply illegal to offer telecommunication services.

Huh, I didn't know that. Guess it's time to pivot PavelLishinOverAir to become some sort of drone taco delivery service.
GSM and LTE use public goods (spectrum).

Spectrum is owned by the public, and regulated as spectrum is a limited resource. IP messaging isn't by any practical means a limited resource.

> As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.

No. Spectrum is regulated in terms of what technology you can use, and kind of usage is supported. You can't buy LTE spectrum and broadcast radio on top of it, and vice versa. You can't (in most cases) take 2G spectrum and deploy LTE on top of it.

> No. Spectrum is regulated in terms of what technology you can use, and kind of usage is supported.

Interesting! I was not aware of that. Thanks!

Also, bandwidth is a function of how much spectrum is available (for example between 1900MHz and 2000MHz there is a chunk of 100MHz) and the spectral efficiency (for example, 1 to 2 bits per Hz available).

For example:

If a carrier "owns" the block between 1900MHz and 1940MHz, and let's say LTE supports 2 bits/Hz using Frequency Division Duplexing (each direction, up and down, gets a chunk of spectrum), with 3 sectors per cellular tower, the carrier can support a total of 40x2x3 = 240Mbps per tower, or 120Mbps each way.

If the tower serves 1000 users (they serve far more in dense urban areas), each user will have 120kbps of capacity, and if they all use it at the same time, that's the speed they'll get. If there's a single user, the maximum down speed for that user would be 120/3 = 40Mbps.

> I'm not a big fan of legislating technical standards, but in this case I think I could support it

I feel another such example is IPv6. It could move much faster if it would have been mandatory already.

China has effectively mandated IPv6, and IPv6 certainly is popular in China (I think all mobile phones in PRC use IPv6, I may be wrong) - but let's not forget that China is promoting IPv6 because APNIC's allocated IPv4 space for China was comically undersized.
Nope. If mobile network uses ipv6 then people would have no problem accessing Facebook
You do realize how firewalls work, right? You have iptables and ip6tables. Sure you need entries for both, and sometimes ISPs forget, but it's easily implementable in all major routers and operating systems.
Firewall rules are not yet implementated on ipv6 network in China by ISP. It is a fact, and actually people are using ipv6 network as a means of bypassing the wall.
Nope. If mobile network uses ipv6 then people would have no problem accessing Facebook
Unfortunately this is not the fact. While some ISPs are experimenting with IPv6, the vast majority are not on that train. Indeed, IPv4 address space has practically run out in China already, but many ISPs are taking the nasty NAT-like approaches, Carrier-grade NAT [1] for example. And it is particularly common in mobile. So far, IPv6 is only (relatively) well accepted among universities (CERNET [2]).

[1] https://en.wikipedia.org/wiki/Carrier-grade_NAT [2] https://en.wikipedia.org/wiki/CERNET

And everything else would have slowed down because of resources (human and other) devoted to ipv6, and the complexity of ipv6.
Though I don't think the legislature even uses IM let alone understand it and understand it technically.
Keep it simple. All protocols for communication and data formats for storage must be published as open source or public domain. This is to prevent or reduce lockin, walled gardens, etc. No other change. They can still compete in numerous other ways.
Although this is an appealing idea, I don't think it is realistic. Lobbying would destroy any legislation almost immediately, and there would still be the issue of competing standards.

I think a more achievable version would be that any online service which maintains user data must provide a method for a user to retrieve that data in an open-source format. Some companies would deliberately obfuscate formats, such as calendar appointments as JSON rather than iCAL or another existing standard, but I think it would give users more control than they currently have.

"Lobbying would destroy any legislation almost immediately, and there would still be the issue of competing standards."

Oh, I agree. You're describing the effect of a corrupt government allowed to prevent or eliminate laws meant to protect its voters. I was talking about what law would be necessary if voters cared enough to push one.

And the result is OOXML, which is well known for being a trainwreck and which nobody - including Microsoft - has been able to implement properly, despite being an "open standard".
"And the result is OOXML"

The result of open-sourcing all data formats and protocols would obviously be all the data formats and protocols released as open-source. OOXML is not the only one in existence.

My point is that companies forced to do this would create abstruse formats, and then even misimplement them, to thwart those trying to interoperate.
Oh yeah. They'll probably try that. That will either be defeated with tons of sweat as it largely was in products like OpenOffice or the legal recommendation modified to attempt to deal with it. It might also be handled in the courts if the law includes language that specifically instructs to attempt simple, readable protocols to "facilitate interoperability." Obfuscations could be argued as the opposite of that in court.
That will either be defeated with tons of sweat as it largely was in products like OpenOffice

Then why do you need open standards? Every major proprietary IM protocol has been re-implemented by open source software, including Hangouts: https://bitbucket.org/EionRobb/purple-hangouts

It might also be handled in the courts if the law includes language that specifically instructs to attempt simple, readable protocols to "facilitate interoperability.

XMPP, the thing Google is being roasted for dropping support for, is often decried as being very complex and hard to implement correctly. If genuinely open protocols often end up like this, how can you possibly punish companies for writing complex, hard to implement protocols?

"Then why do you need open standards?"

"This is to prevent or reduce lockin, walled gardens, etc."

What I said. I'll add that it's hard to copyright API's to sue over compatibility when the API's and their implementation are released as open-source software.

"If genuinely open protocols often end up like this"

What are you talking about? There's all kinds of results from protocol design. You chose one of the worst ones. I was a PSYC fan over that if I had to choose an IRC replacement or open chat standard. A test for new ones might be trying to run it through LANGSEC's Nail, Cap n Proto or ZeroMQ to see if they handle it easily. Anyway, it's easier to do compatible implementations if you have source and can't be sued/imprisoned for imitating it.

Which would be obvious, flout the law, and allow for legal action to be taken.

The company itself has to use the format for live message exchange, have to publish a full working protocol document.

With OOXML Microsoft aren't required to be able to read & write a files correctly when it's written to the protocol. As I envisage it the requirement would be to open the protocol actually used "on the wire"; again with Word Microsoft don't use OOXML as primary.

I think it's workable except perhaps in requiring political will. It's an internet age equivalent of allowing third-party car parts manufacture.

Will it be obvious? How? Browsers often fail to implement open standards like CSS, even when they have nothing to gain, it's just that shit's hard.

Microsoft don't use OOXML as primary.

Sure they do, that's what .docx files are. They've been the default since Office 2007.

Microsoft was compelled to create and implement the standard so that governments can comply with their requirements to use "open standards". And they followed all the steps, including getting it past ISO, despite the mess that the format is. And so governments can now tick the "open formats" box when purchasing Office. It's all a sham, and I don't see how it'd be otherwise with this proposed law.

I don't think it needs legislation as such but could be worked from a licensing point of view.

Everyone moans about AGPL. Fine. Make something better that says if you're going to interoperate with a service, you better damn well actually interoperate. Make the license viral like GPL and hopefully when we all push in that direction, everyone needs to join up.

Korea mandated through legislation that all online shopping had to go through ActiveX controls, because that was the only way to guarantee secure encryption at the time. Now it's practically the least secure way to online shop and even Microsoft recommends not using ActiveX, but it's a huge amount of work to strip that infrastructure out and make everything work 'properly' so it's still an issue even today.

In short, legislating a solution is a workaround that can turn out to be a huge detriment not too long down the road. I'm no libertarian, but it seems like a bad idea to me.

That's a good point - especially important if someone's thinking about legislating a particular technology. On the other hand, maybe in this case it would be enough to legislate a requirement for openness of the protocol? I.e. whatever it is, if it has enough many users, it has to be fully available to third party integrations.
This feels reminiscent of net neutrality and also general competition laws. I wonder if there's something in that which could be drawn upon?
What about SMS? Everyone has it, it works, and is usually fairly instant. It's not very feature rich, but could there be an SMS 2.0 protocol that was backward compatible?
Sounds like a straw man, just because South Korea made a mistake doesn't mean legislating a standard is bad.
> IM needs to be forcefully standardized.

This is totally crazy to me. It would be like the government legislating when and where people have spoken conversations with each other.

No, it's not. What you're proposing would be like the government saying you can only use this specific service, which may be run by the government, to communicate with others. Standardizing IM is making is so that if you want to talk to your friend across the country you're able to using different applications but the same protocol. Forcefully standardized would be mandating any messaging app that's used in the country would have to follow the protocol. It's almost the same thing that happened to fax machines in the 70s and 80s before the Japanese government stepped in.
How would this look different to SMS, except that the protocol might be more feature rich.
It's not something needed. It's a plus, even if it influences society a lot, it's not something like food or healthcare, where if someone plays unfair people die. And it's not like net neutrality either, because you are not costrained by space (putting cables underground). People should be free to choose a service, and if the server want to stop providing the service or change the way it does ut, it should be free to do so. Anything else you can use contracts, or change provider, as long as there is net neutrality in place. And it's not something that messes with your health, so no justification for regulation aside from an arbitrary and unprovable "society being held back". Being unable to make something because of regulations, that's some real and provable holding back of innovation.
Except we still do have many different standards for the phone system. As an example the main codecs used for encoding voice today are μ-law (North America), A-law (Europe), GSM (2G/3G), AMR (3G/4G), iLBC (VoIP), G.729 (VoIP).

That is just for voice, now imagine you are a phone company and want to send a call to another phone company. There must be so many different things you need to cooperate on to make it happen - billing, switching, physical connections, numbering, codecs, etc. Yes the actual protocols may be standardised, but the main part is cooperation.

It's not standards we need, it's companies that are willing to cooperate and work together. None of the big tech companies want to do that now though. Everyone wants their own walled garden so they can say they have more users that X competitor.

(I'd argue that XMPP is good enough, or with a few extensions could be made good enough, for a company who really wants to push open IM)

And what if they don't publish the protocol? Do we block downloads of their client?

Even if we assume they are American companies, and we can sue them if they violate the rule; how on EARTH would we define an IM protocol broadly enough to avoid an easy avoidance of the law, but narrowly enough to not basically block all proprietary protocols?

Good questions.

If a company doesn't publish the protocol in use within, let's say 3 months of a valid request by a third-party then they'd be liable to a fine, perhaps something like $1 per day per registered user, plus costs of government and registered third parties in pursuing the interoperability request.

Small companies might escape, but if Facebook tried it they can't really run away. You'd need to be able to issue notices to ISPs to block those convicted of breaking the law, such orders are already part of UK law (against torrent discovery websites for example). Again you'd get some leakage but big providers like Microsoft couldn't feasibly expect enough customers to move to VPNs to bypass such a block.

IM client definition could be hard but in practice you'd have clauses requiring a judge to decide if the traffic exchange amounted to an IM communication. I see no problem with having a broad definition, if the definition were narrow then it would just get worked around. The definition would probably include email as a subset.

Were there other person-to-person communications protocols you are concerned would fall in the scope that definitely shouldn't be open?

I can think of a few... one example that comes to mind is a video game. The communication protocol to synchronize the game could fall in scope, and I think video game companies have the right to have their own proprietary game network protocol.
> You'd need to be able to issue notices to ISPs to block those convicted of breaking the law, such orders are already part of UK law (against torrent discovery websites for example)

And most of us in the US hope we never lose the right to open access in the way citizens of the UK have. I can't think of a worse thing to happen to the internet. The UK has proven that once you allow a government to censor the content of the internet they will aggressively and rapidly expand that power to censor and control.

Are you implementing spf?
I have a personal mailserver box in a cheap colo, and gmail won't put messages from me to my gmail-using friends in their inbox. It's shameful what email has become.
World you be happy to put the offenders in prison? Or if they didn't pay fines, to show up and collect?
i kinda doubt anyone sees protocol violation as a criminal offense.

it's wild to me how quickly "we should find a market-external pressure source for forcing interoperability given the profit motive seemingly guarantees lack of federation because it relies on competition as its motive force" becomes "we must frogmarch innovators to the death pits"

The correct way to solve this problem is through raising awareness of privacy issues and supporting efforts to create competitive and attractive open source communication protocols, not legislation enforced by a prison/military industrial complex. What you're suggesting is to hand the State further control over our choice in communications platforms and trust them to never abuse it.
I have been helping out a charity that uses a small web hosting company for email.

Any emails they send to Gmail automatically go into the spam folder.

Why? I reckon it is because there is no SPF configured, no DKIM configured, etc.

We could try to get that small web hosting company to configure all that.

Instead, the plan is to move to G Suite for Nonprofits.

I run my own e-mail server too and constantly run into issues:

http://penguindreams.org/blog/how-google-and-microsoft-made-...

If I send a new person an e-mail, I have to contact them on Facebook and tell them to check their spam folder. Fuck Google and Microsoft and their terrible e-mail filtering. Just because people are so dumb they will open any ransomware sent to them doesn't mean you drop all e-mail you don't trust with no way to account for it.

How long have you been sending mail from your latest IP Address?

I run some websites that have their own mail servers. One thing you didn't mention which can help is to be sure to use an email signature. Basically short emails are more likely to be flagged as spam, especially if there's an attachment.

In use 12 years, only ever used to reply (except when messaging businesses), never done mailings. Of course there is a footer with bricks & mortar address, phone, unsubscribe link.

Had problems with hotmail before, it's just associated IPs being factored in too strongly. Same ISP has an IP address that spam has been sent from; support responds with won't fix. In their favour at least they responded.

Domain has SPF but not DKIM.

Thankfully no other mailbox provider blocks us. We could pay to be added to the third-party "friendly" domains list of course ...

I'm surprised they wouldn't whitelist our domain but leave it on a tight leash.

Why don't you properly configure DKIM and SPF? Then you won't have this problem.
From the fine article he linked: SPF, DKIM and DMARC are all domain verification systems for validating e-mail’s origin to prevent spam. I have all three records set in DNS records for all the domains I send e-mail from, verified they were correct using testing tools, and I still get flagged as spam.
Google uses a lot of signals. The article above is on http, I saw a drop in spam flagging when the web host on my email domain was https only.
Historically giving the government more power to legislate the structure and content of digital systems has been disastrous.
They have already mail email incompatible. They haven't changed protocol but best of luck if you run your own email server.
The world made it impossible to roll your own server, not Google. The proliferation of spammers in email has necessitated a cottage industry of security and filtering and all manner of mechanisms, automated and otherwise, to deal with the avalanche of bullshit streaming through that protocol.

Google's the blame party for a lot of things, but not the loss of email freedom.

I don't know a whole lot about XMPP, but since it's federated, wouldn't it have the same avalanche? It seems like your choices are to use something interoperable and get spammed, or use a walled garden and you don't get spammed because the gardener has pitchforks.
> I don't know a whole lot about XMPP, but since it's federated, wouldn't it have the same avalanche?

It absolutely did. Everyone decrying how XMPP federation was killed conveniently forgets how useful it was to spammers.

Since this is IM the most you could do would be to generate a lot of requests to put the spammer on someone's roster list. In the unlikely event that a particular rogue XMPP managed to convince anyone to let them add some bogus users to their roster before it was blacklisted, it would take literally seconds to resolve the issue by removing them from your roster. So there isn't really any incentive to do spam over something like XMPP.

I have been on the federated XMPP network pretty much as long as it has existed and I have not gotten a single spam attempt. I guess I could post my jabber ID somewhere to make it more sporting (they need that to even try) but I really don't want to add random people to my roster. Email is intrinsically different that way.

This is not true. There are email standards out there to help prevent spam. But right now Google simply ignores all this even if everything is setup correctly. I think Google has a responsibility in helping me debug _why_ an email is spam and give tools to fix it. Currently, there is 0 information, 0 support. The webmaster tools simply say everything is awesome and dandy. And you can't do anything else.

Here is today' world work flow: 1. Buy a domain

2. Have to use Google because your mail will end up in spam with any provider (unless it's outlook or some special provider).

I've never heard of Google being particularly helpful in debugging literally anything. According to their "support", Google software is perfect and if it's not working for you you're doing something wrong.
You realize that if they did offer that support, every spammer in the world would be using it to make sure their spam gets through.
I got good mileage with the mail server on cloudron. Also, mail-tester.com is awesome to check spammyness.
Yes! Mail-tester is the best checker I've seen in a very long time. I'm amazed it's still free. Shush.
It was a product driven decision. The number of active federations was absolutely tiny and the number of messages sent over those links even tinier. The amount of XMPP spam was catastrophic, and for most users the only knowledge they'd ever have of XMPP was when they got a friend request from buy.herbal.viagra@cheap-online-meds.com. XMPP federation was a great idea, but it never had adoption, and at some point Google had to decide between UX and software purity - With one side having a clear high cost and unclear benefits.

(I was a Google SRE, sitting near the talk team. Everyone wanted XMPP to succeed, but the numbers didn't even come close to working out)

Product driven or not, that decision was backward, not forward directed. And as I said, if they thought XMPP was deficient and couldn't address spam or what not, where is their better federated and open alternative? They never made one.
Why spend time building a federated system when nobody was using the last federated system?

Ultimately most users just don't care about federation. Them's the facts. That's why everyone takes PayPal and only nerds use bitcoin.

Slack vs IRC, too. Twitter vs GNU Social. I'm sure there are more.

Firstly not nobody. Secondly, how can you grow it if instead of moving it forward, they roll it back? That's the opposite of progress.

> Ultimately most users just don't care about federation. Them's the facts.

Not really. Everyone is annoyed by this issue. But there isn't much they can do. People need to register on N different services and use N different clients to communicate with users of those networks when they need to. Do you think they appreciate this mess or find it highly convenient? But they don't have any other option (except may be not communicating with some of them at all).

People care about convenience, not federation. Google Talk and Facebook Messenger both supported it for multiple years and the number of people who used it rounds to zero. This is where we ended up. There was time for it, just no demand.
Launch a terrific, convenient new email service that doesn't federate and watch how quickly users will choose federation over convenience.
They did, it's called Facebook.

Slack if you don't like that example.

Those are IM services.
That work asynchronously even when I'm not online, that I can check later, that have functionally replaced almost all of the communication that used to happen over email...

It's technically and functionally identical to non-federated email.

Slack doesn't replace email. That's a silly marketing slogan. Imagine for a moment if it really did. What if Slack became so popular that everyone stopped using email? That would be horrifying. You wouldn't be able to communicate with anyone without using Slack because it doesn't federate with any other service. Every website would have to integrate with Slack for user account registrations. Etc. Decentralized federated communications is the backbone of the internet. The fact that IM services have never broadly adopted federation dramatically cripples the potential of the medium with horrible fragmentation and balkanization.
...and yet, here we are, where most people who wish to contact me will send me a message on facebook before they'll send me an email.

It's not fragmentation if there is a single one that the whole planet uses.

You're assuming here that federation = progress. That's the source of the disagreement.
Federation is clearly progress. Balkanized non interoperaiblility is not.
If you notice serious problems with federated messaging, it's an entirely valid choice to abandon federation instead of trying to make federation work. There's no evidence that a system that's (a) federated, (b) open to everyone and (c) non-spammy is possible in the first place; maybe it is, but even more likely you have to choose two out of three at best.
I don't see anyone repeating this fallacy w/regards to email.

The only difference between email and federated IM is email started out (mostly) federated and IM systems started out (mostly) not.

Only inertia is what drives this skepticism of federation in IM.

Just as there is no technical reason GateKeeper couldn't succeed on iOS just as it does on macOS, there is no technical reason federation couldn't succeed in IM as it does with email.

Just perceptions have to change.

> XMPP federation was a great idea, but it never had adoption,

And your alternative @ google was yet another closed source protocol that doesn't work with any other closed source protocol(obviously, because they are closed source). Now imagine if the internet worked that way, Google wouldn't be where it is today.

Surely the right answer would have been to enable users to report spam invites, and to kill spam-sending domains. Kinda like what is already done for email. Could probably repurpose a lot of the existing infrastructure.
It was done. Domains are cheap. User patience is not.
Why Google felt betrayed by Microsoft:

Larry Page commenting about it at I/O 2013: https://www.youtube.com/watch?v=9pmPa_KxsAM&t=2h54m20s

Strategist commenting about it: http://www.ucstrategies.com/unified-communications-strategie...

Microsoft's one-way integration was them not exposing presence and typing notifications to services they federated with (but they were "taking" that data from others).

Larry Page says there he would like to see more open standards used, and he is sad the industry isn't moving in that direction. So why isn't any of their new IM technology an open standard?
Because being constrained by existing standards, rather than being free to make the changes to the protocol that you think will make the product better, is a huge trade-off. The benefit is that your users get that interoperability with e.g. MSN Messenger or Skype or whatever.

However, if no one else is doing that, if no one else is playing fair (like Microsoft accepting things like typing or presence notifications but not sending them in return), then it turns out you're being constrained, and possibly doing additional work to add features, and the person benefitting the most is someone else's users and not your own.

Someone has to start doing it, otherwise you'll get the catch 22. So "others aren't doing it" is not an excuse.
But they did start doing it with XMPP and kept it that way for a while.
So. They had to persist, and address deficiencies too. That's the only way to break the cycle.
Then I assume their standards and implementations are completely open, and they allow third parties to federate?
I really doubt that Google were constrained for technical reasons. XMPP is completely extensible. I've worked with it for over a decade and there's absolutely no technical reason why Google couldn't have extended the XMPP specification and put out their own XEPs (XMPP Extension Protocol) which would have given them all the functionality and features they needed.

XMPP's only major downside at the time was mobile battery life due to XMPPs limitations over cellular. This could easily have been resolved by using a non-XMPP accepted handshake (as WhatsApp did) or by using a completely different protocol for mobile as Facebook did for mobile (MQTT).

Google decided to go for a walled garden approach as a competitive advantage. There's no technical reason for it.

Because their old one was an open standard, and they were effectively the only one open one (and the last open one standing). So I find it totally reasonable that they stopped caring about it (and probably would again if another major player did).
I find it a very lame excuse. If they care about it, it shouldn't depend on others. In the end, it shows they didn't care enough.
Probably costs a bunch for not that much convenience though. Tragedy of the commons in action.
That's the case with most of interoperability / standardization efforts. But without it, the whole situation ends up being the current mess.
I disagree. I can understand why you think so, but an analysis of the situation will point out something different.

Google got talk working almost by accident. And it was the most feature-complete chat app for years, at least. And then, they abandoned it for years. This isn't to say it had many features, it had few. But it worked well, was integrated with gmail, was compatible with XMPP and geek's IMs, ...

After that, they killed all the compatibility before making an app that was yet again a lot more feature complete : hangouts. And hangouts got the best features for quite a while.

And then Hangouts started wildly bashing it's weight around. Forcibly taking SMS was one thing. Refusing federation. Forcing Google+ account and lots of extra info. And so on, and so forth. Hangouts was a good app, but created a lot of ill will in the process.

And then it was seemingly abandoned. Feature frozen, with the excuse that all these features had resulted in an extremely difficult to maintain app that they couldn't add features to. Chatting without having a gmail being a big one (whatsapp allowing you to chat with "everyone in your phonebook"), status, reliability (and showing clearly and timely when it isn't/cannot be reliable because disconnected, not 3h after connectivity gets restored), video call quality, adding non-gmail users or just screens to video chats, not allowing bot interaction, apps in the video chat, ...). And this lasted for years.

Inside of China, some chat apps demonstrate how chat apps can be monetized in a way that users appreciate : allow chatting with companies and give those companies the ability to show interfaces for transactions. E.g. buy a coffee. Line at starbucks ? Open up whatsapp, because you're in starbucks it shows starbucks as a contact (you can add it permanently) and there's a button "order coffee". Select what kind of buvaranicpoppafrappadongieccino you want, you pay through the app, and you've just skipped the queue. Next time you do it 5 minutes before arriving. App takes cut of transaction value (just like credit card payment does).

Next, other apps turn up. Whatsapp, Lyne, ... and so forth. And they caught up with Hangouts. Surpassing it in some ways, behind in others. Mostly they're superior in letting you find the people you can chat with. And then they passed it by. And then they left it far, far behind. It's not (yet) the case that they truly dominate hangouts in features, but it's getting close.

And lo and behold : people switched to the (sorry) better apps. I'm not entirely sure why anybody is either surprised or complaining. These changes look a lot like they're making it worse, not better, but we should give them the benefit of the doubt.

Microsoft does support full federation. As long as you federate Skype servers. I know that's a bit lame, but since they bundle most of their products with most of their other products in licensing, no one in business seems to care.

Google doesn't do even that, as they don't offer anything on-site (anymore). They have seriously failed on business markets. I don't know about consumer market, because I mostly just send dick pics for the giggles to random people I don't know.

That's the big thing about SaaS, mobile and Cloud.

I don't get that people complain about Microsoft software being closed, and anti-competitive ... well any cloud software is more closed and anticompetitive than Microsoft ever was on it's worst days : it's not that you can't get good documentation that the application you're using uses ... you can't even get at the data at all, in most cases.

Google is the best at this out of all the cloud players, but it's still abysmal. And yes, you can download exports. And there's Google takeout. Is the data there all the data Google apps use to work ? No. Quite simply, no, it isn't. There's plenty more they won't ever give to you. And that's ignoring the fact that there's 1000s of developers making apps for Google and about 5 working on takeout. Why should Google be forced ? Well, that's the standard we're applying to microsoft, isn't it ?

And this is by far the most flexible cloud provider exporting data. They're not being jerks. I don't even think they're trying to be anticompetitive. Nevertheless, they're far more anticompetitive than Microsoft.

And of course, conveniently, the DRM-breaking is now a crime on things like iTunes and Hulu and ... because it involves actually creating access into someone else's computer system or at least changing their code (even though you're paying for it to do things and only getting it to let you exercise rights granted to you by law, like copying a movie). And the result is the same. iTunes, Hulu, even Play are ridiculously expensive, idiotically restrictive (buy movie on iTunes for a trip, go to Europe, log in to hotel wifi ... no movies. But no worries ! I copied that movie onto my device first. Nope, it won't play. WTF).

Same thing goes for app stores. There Apple is the big instigator of forced-incompatibility (or otherwise: your app won't run without Apple's permission hardware). Remember the flood of articles scaring people that microsoft might do this ? They're still at it, with the BIOSes that supposedly won't run anything but windows.

The dreaded "the right to read" scenario ( https://www.gnu.org/philosophy/right-to-read.en.html ) is far, far closer on the cloud and mobile than it is on your own machines.

Thanks.

(Talking about web and native APIs)

> And I think that we've really invested a lot in the open standards behind all that

> I've been personally quite sad at the industry's behavior around all these things

> If you just take something as simple as instant messaging

> We've kind of had an offer forever that we'll interoperate on instant messagoing

> I think just this week Microsoft took advantage of that by interoperating with us but not doing the reverse

> Which is really sad, right?

> And that's not the way to make progress

They gave up on Federated XMPP and never implemented video using XMPP. If they hated it so much, they could have proposed open changes. Now we're in a terrible IM landscape. No federation at all. E-mail is federated, but is been rendered terribly unreliable simply due to spam filters.
Another example is iMessage which is very popular amongst the Apple fans. And Snapchat essentially is another great mobile to mobile coms app people don't quite get yet, but it's more powerful and entertaining than "Hangouts"
> But at least now it works on phones?

This is actually really, really important to people.

You are saying it like if XMPP never worked on phones.
It didn't work well.

But who cares?

Even if there were, hypothetically, no possible way to make XMPP work, and Google really needed a custom protocol, they could still have released public documentation for that protocol, and open source client and/or server code.

Indeed, that's sort of what they did in 2009 by creating Jingle, an XMPP extension for voice chat, and releasing libjingle. But no such luck for the new stuff.

I could understand giving up federation, maybe, since it creates a spam problem. Though for Google to throw up their hands on spam is a bit ironic, given that Hangouts sits inside Gmail, which has world-class spam filtering. But federation was never really necessary: most XMPP clients support multiple accounts, and it's not like there were a lot of huge XMPP services - only one, Google. You just add that as an account in addition to whatever smaller-scale/organizational XMPP servers you use. (You know, the thing people used to use before Slack.)

Similarly, most XMPP clients support multiple protocols, so for Hangouts to use a custom protocol is not the end of the world. Annoying, but if the benefits are clear enough… it's fine.

But no. Like with most messaging services these days, Hangouts' protocol isn't documented, and nobody has bothered to reverse engineer it. So you're stuck with the official clients. Maybe not the end of the world, since unlike the bad old days of Windows-only clients that left Linux users in the cold, today any desktop OS can use the web client. (And non-top-two mobile OSes don't have enough market share for people to care about their not being supported, and many of them can run Android apps.)

Not the end of the world, unless you want all your messaging services in one place - people used to see a lot of benefit in that, and there are a lot more services now.

Or you don't like the official design. Or you don't like bloated web clients. Or you prefer free software.

Sigh.

XMPP doesn't work well on phones in part because Google, Facebook, Apple, etc. didn't want it to work well on phones because they wanted their own proprietary messaging services.
> Gmail, which has world-class spam filtering.

If by world class you mean huge amounts of false positives and destroying the ability for people to send messages reliably between one another, then yes. It's totally world class .. a world class failure.

As they say, Works For Me. Admittedly, I don't know how many important emails I've lost over the years, but I don't remember ever being contacted elsewhere about an email that turned out to be in spam. I have a bad memory so I might be forgetting some instance, but if so it must not have been terribly important, or I'd remember it… On the flipside, there have almost never been any false negatives. I remember being quite annoyed a while back when there was a brief period where a trickle of spam got through. Other than that, I've been spam-free.

(I also remember being IMed by spambots using XMPP federation a few times.)

This does come at the cost of letting Google see all my email. I'm not happy about that, but I'm skeptical that a fully encrypted solution could provide filtering as effective. Certainly a self-hosted solution cannot.

Are there any objective looks at this? I know personally I manually check the spam folder, and out of the hundreds of messages a month that end up there I've never seen anything important get mis-categorised. A few (<<1%) mass emails end up there inadvertently, but nothing that I would notice if I didn't check.
standard complaint was that it was a battery killer.
As far as I know, XMPP is still being developed further. The guy who wrote the Android Conversations (.im?) app is on HN and says it is a serious option for phones now.

(Except it costs money so most of my WhatsApp contacts will not switch.)

Very true! Although I think op meant it tongue in cheek. The joke being that the old version could just as well have run on phones, had they tailored the app for it, instead.

In fact, that's precisely what they did. The old talk app for android worked fine, even before they terminated federation.

I don't mean to diminish the importance of working on phones, however interoperability (at least as a third party) is also important and "not losing contacts" is pretty much at the top of the priority list.

Hangouts failed both.

Google had (no longer has, IMO) the clout to design and drive a solid, open source messaging protocol with a reference implementation, sitting in the inbox of hundreds of millions of users and on hundreds of millions of phones. Instead, they created a slow, mediocre, closed source product that didn't interact well with their own video/voice calling system and fractured their own userbase between Hangouts, Google Talk, Google Voice and now even Allo.

I forgot about Allo. Didn't they bring back the Google voice app? Yea it makes no sense at all.
Yes, if anything, some of us stopped using Hangouts not because it supported phones, but because it didn't support desktop machines.

(No, being forced to run chrome just to see chats in a crappy corner window isn't experience on par with iMessage / Telegram / Viber).

Don't forget Allo in the mix, from Google, that could replace Hangouts! Take a product people use, and replace it with something people don't, then replace that with something that people also don't use.
I tried Allo once, which was enough.

It was actually fascinating, because it was among the most viscerally negative reactions I've ever had to a piece of software.

It reminded me of that trick where you play back someone's speech to them in a very slight delay and it becomes very difficult for them to speak, only in this case with my inner monologue while trying to participate in a conversation.

Really impressively disturbing and uncomfortable. An amazing feat, but not something I will ever use.

Isn't Allo video only? Hangouts is first-text, with optional video.
Duo is video-only. Allo is text-only
This has got me thinking...

If the same GTalk was available on mobile, would I have installed WhatsApp?

So, what you don't get is why using an open chat protocol didn't turn into market leadership? Or...
Google's failure to turn Talk into a market leader has absolutely nothing to do with Talk being built on top of an open source protocol.
Everything has consolidated, so it's all about walling off users to increase friction between the Google/Facebook/Microsoft/Apple universes.
My gf gave in and bought an iPhone 7. I haven't owned an iPhone since 2009.... iMessage is pretty neat... But still, I'm sticking with Android.
Yep, and I wish a bunch of HNers with lots of time would spin up an OSS Google Apps (for most of the apps at least) replacement. Then people could do one-click installs on AWS or their own infrastructure, and the like...
Sadly Facebook also used to do XMPP which is annoying as well. Used to be you could login to either one from any XMPP client, this means Pidgin, which most people used to use for MSN Messenger, which I miss so much when I think of what has happened to Skype. Even though MSN Messenger had it's own weird flaws (like locking people out of their own accounts because people would force login attempts to force lock your account so you couldn't login anymore). Now Skype lets anyone grab your IP and pull it into a booter. The gaming community is full of skids.

I absolutely agree with you though, Discord is one product done right, runs everywhere, and everyone I ask to "join my discord" I can't even get rid of them once they're on it, and if I stop going on they nag at me. I secretly wish Discord had been based out of some open standard because it is well done.

> I secretly wish Discord had been based out of some open standard because it is well done.

I want to highlight that this is not realistically possible for anyone, not right now anyway.

When Discord was created, XMPP was the only option if you wanted to adopt an open standard, and it was absolutely not up to the task. Matrix was also an option but far too new.

Discord is a startup, and is run as a business (just like Slack). If you were at the head of a startup in the messaging space, and there were no reasonable open source options for messaging, you'd design your own. This is a hard process; making it an open protocol from the get-go removes a lot of your freedom.

I used to say (even here on HN) that Google was our best hope at creating a modern open source messaging protocol, by improving Hangouts and making it open. They have utterly failed.

I am confident Discord is our next best hope... assuming they succeed. Otherwise, it's back to waiting for Matrix to finish playing catchup.

>making it an open protocol from the get-go removes a lot of your freedom //

Can you expand on that. Are Discord using someone else's closed protocol then in preference to their own (which they could open) or an already open one?

Or did you mean "freedom" as in "ability to use lock-in", or something else?

They're using their own protocol. By freedom, I meant ability to iterate fast, break compatibility, not have to support unwanted clients etc.

It's something you care about less once you're big, but it matters a lot when you're starting.

Presumably though even if you're iterating hard and breaking stuff, once you're out of alpha you're not going to introduce many backwards-compatibility-breaking protocol changes? I can see doing that for v0.1 to v0.2 and maybe from v1.9.x to v2.0 but nothing about making those changes seems incompatible with sharing the details of final protocols used in your production level clients.
Take a very real example: Discord is working on video chat right now. As part of that work, it's not unthinkable they'd need to severely rework the voice/text protocols because they want to bring all of them in line with one another, reduce their tech debt or what have you.

If it's an open standard, they need to check in with everyone, document the move, potentially have to explain it or depending on popularity won't even be able to justify doing it. And suddenly, you see they're losing their competitive advantage for the sake of pleasing a few people on HN.

If it's a closed standard, they do whatever they want, don't have to justify or explain it to anybody. They can turn the protocol into fairy dust, run two versions of it for a while, and require a client upgrade if you want video chat.

Now, a few years down the line once Discord is established, has a solid business model and the protocol is clearly not changing anymore, then we can talk about making it open, allowing third party clients and we can really seriously start bugging them about it.

I very much want this to happen but Discord does need to be successful first. You'll get nowhere by bugging a startup to spend time on what could potentially kill the business.

>If it's an open standard, they need to check in with everyone //

You're​ missing the point I think (or I got cross-threaded). It's Discord's standard, making it an open standard just means the protocol details need to be public such that a third-party client that uses their protocol can send and receive messages to & from their client(s).

The people to be pleased are not just HN readers but everyone who uses an IM system as each system could, if desired, then speak direct to others. In theory it wouldn't matter that I have Facebook and you have Google, I could still message you. If the market works effectively the best protocols could then win, as could the best UX, even if they came from different companies.

If I were drafting a law on this then it would have an exclusion for SMEs or new companies, it would as you say be an unnecessary regulatory burden to require protocol openness prior to establishment of a service. We're looking at million user plus systems.

and it was absolutely not up to the task.

In what way? The chief complaint I've heard about XMPP over the last couple of years is about excessive power consumption on mobile. And my own experience is that those concerns are highly over-stated.

My anecdotal experience is also that all the purpose built closed protocols like Facebook Messenger and Slack both seem to use way more power consumption "in idle" (no message traffic) than I recall ever dealing with for an XMPP client...

I've never seen users actually complain directly about an app's "excessive power consumption". Can we all admit that one has always been an excuse?

The only other complaint I've heard about XMPP was that it was too extensible. There wasn't enough mandatory features in the base protocol to expect good clients for fancy things and there were too many extensions to need to follow to write a good client if you tried to track "fancy thing du jour"... which is as much a feature as a bug (and the whole reason for that X in XMPP).

Has nothing to do with that. Put yourself in the shoes of the people behind a new messaging platform that needs a competitive advantage. Using XMPP is a massive technical restriction, and would give almost no benefits (Discord gets to be opinionated on how servers and channels are organized).

Discord's product isn't a hosting service, their product is the quality of the service as well. This quality would be impossible with XMPP as it exists today, within the constraints of a startup. Remember, they do voice and they want to do video as well. Google's own efforts on Jingle failed due to performance.

As for mobile, it's not just power consumption, it's connection lifecycle. XMPP last I heard had extremely poor support for lossy connectivity.

Has nothing to do with what?

Again, power consumption/connection lifecycle isn't directly a competitive advantage because your user doesn't care so long as it works. (Slack and FB Messenger seem to have terrible power consumption and connection lifecycles for mobile but also seem to be doing just fine.)

(Not to mention those are also things that could be fixed in an open standard, if people cared to contribute that effort. The unfortunate reality is that isn't a competitive advantage either.)

The competitive advantage is to find excuses to dismiss open standards, whether the arguments are technically or factually correct is another matter, and build walled gardens.

But more crucially to my previous point, being technically superior isn't really a competitive advantage if a user doesn't notice it and in fact, can slow one from getting first mover advantage by shipping something/anything faster and sooner. I really don't think that on a power consumption or connection lifecycle standpoint any of the closed source protocols are really all that much better, and we have mostly nothing but anecdotes to trade on that question because the companies want to maintain their secret sauces.

Facebook Messenger used to be an egregious offender here and chewed through battery life. I uninstalled it two years ago and that helped my phone's battery life noticeably.
React native is an 'open standard' :p
fun fact, ICQ is doing just fine. They just stopped bothering staying in the English speaking game, it's now mostly a Russian thing. Doing quite well, still under active support and development.
"Advanced" runet users dropped ICQ for telegram long time ago
Honestly, what I think we're seeing is evidence of an existential crisis in some quarters in Google. There's a lot of people who still believe in "do no evil" and who furthermore believe that part and parcel of that is not building walled gardens. Then you have other people - including Larry - who think walled gardens are the only solution. The PM for RTC at Google wrote a long email about how basically interoperability is dead, XMPP is dead, federation is dead, and every attempt by Google to build open standards has been effectively exploited by competitors.

So a lot of influential management thinks going full FB is the way to go. But a lot of other people at Google think open standards are still the way of the future, or can be if a company like Google is willing to champion them.

The result of all of this is a complete and utter dumpster fire from a product perspective. It's the classic "if we're not pulling together then we're rowing against each other" problem. I think Google would be able to execute just fine if they could settle on what they want to achieve. Instead we have this.

> You know, I can get behind that XMPP wasn't up to the task - I tried dealing with XMPP myself and it's a frustrating piece of work.

XMPP is weird, true; it relies on opening an XML document and sending fragments of XML for the duration of the session and only closing the document at session completion. Very odd. But it performs extremely well. And once you get your head wrapped around the constraints, it's not so bad.

Talk was built on top of XMPP, an open protocol, which helped its popularity as third party clients could connect to it.

By the time google talk came around, third party clients were easily connecting to all the other major IM services. I don't think the open protocol made any difference to its popularity.

Talk didn't have its own client outside of the Gmail panel when it launched. It would have been much harder for Google to bootstrap at the time, if they hadn't been able to just point people to existing clients that supported XMPP. Third party clients made Talk useful faster (thus popularizing it). Sure, several of the third party clients were pretty good at reverse engineering weird protocols while also supporting open ones, but there's no guarantee they would have bothered with Talk, especially if Talk existed only in a web browser.
Talk didn't have its own client outside of the Gmail panel when it launched.

Didn't it? I think it had a client from the get go

https://en.wikinews.org/wiki/Google_Talk_released_in_beta_fo...

Interesting, I seem to recall an entirely different order of release. Blame it on personal bias (was already using XMPP at the time of Talk's release) and/or aging memory, I suppose.
I remembered it exactly the other way round and also wasn't sure. It's curious/disturbing that events barely over a decade past are difficult to google. 'What was the first gtalk client'. 'When was gtalk integrated into gmail'. 'When did gtalk get an OS X client'.
Regarding IRC, how do you feel about ircv3 [1]?

[1] http://ircv3.org

I have been following it closely but it feels like too little, too late.

IRC is just not the protocol of the future. There's far higher chances for Matrix to become popular or discord to be open sourced.

Are there any IRCv3 implementations? Is freenode planning on running it?

I'd love a totally OSS alternative to Slack. Maybe we'll see it with the recent Gitter acquisition?

Many popular ircd's support different IRCv3 specs. Freenode, Rizon, etc all support different specs/capabilities. You can see a list here: http://ircv3.net/support/networks.html

(Warning: bias/plug; I contribute to Lounge)

In terms of OSS alternative to slack, I recommend you check out Lounge, a self-hosted JavaScript web client (think: selfhosted IRC version of Slack, minus/plus a few features)

https://github.com/thelounge/lounge

Slack, minus the ability to gracefully run in any sort of long term roaming context without babysitting. There's no decent reconnecting logic on either client or server, you just have to notice it's gone quiet, and then figure out the undocumented way to restore it without nuking your settings. Only figured out by accident "/connect" without parameters is valid.

I tried to fix it, but the client is a pile of terrible code and needs a complete rewrite.

Well, if you get disconnected and don't get reconnected, the server tab mentions to try `/connect`.

Additionally, if you want to rewrite the client, you're welcome to start and open a PR - it's a community-run project. I've had Lounge running for months now without issue. Contributions are always welcome.

Mattermost is an existing OSS alternative to slack. It's terrible.

Gitter is more likely to be good but to be completely honest, it will take a lot of work to not be mediocre.

Freenode is part of the IRCv3 working group and I believe uses it internally somehow. IRCCloud is the largest IRCv3 implementation, they are fully IRCv3 compatible afaik.

Hi @scrollaway,

Thanks for your feedback on Mattermost. Could you share more on what you'd like to see changed?

For folks who haven't seen Mattermost, here's a demo video: https://www.youtube.com/watch?v=AKqHWqrAgpk&t=1s

Also, I noticed from your HN profile links you're a UX designer.

We're actually working on our new generation of mobile apps in React Native (https://github.com/mattermost/mattermost-mobile).

If you're open to critiquing some of the screenshots from early builds, or even participating in the design discussion, we'd welcome your involvement.

Also, we'd welcome you to join our community server at https://pre-release.mattermost.com/

I'm on there as it33 if you want to DM me to share more of your thoughts?

PS: Regarding IRC, Mattermost connects to IRC, XMPP, Slack, HipChat, Matrix and other systems via Matterbridge: https://github.com/42wim/matterbridge

Hi Ian, good to see you here. I really shouldn't have been so harsh :) Mattermost isn't terrible as a product. The issue is that it's not going to replace Slack because, although it's FOSS, it's not an alternative to Slack. Let me elaborate:

Very few people know about Mattermost right now unless they use or used Gitlab. Whenever I hear Mattermost brought up, it's associated with Gitlab, which has a reputation of being "the slower, less-good GitHub" (and I love gitlab, don't get me wrong, but I agree with that reputation). Therefore, Mattermost, at least in my circles, has a reputation of being "the less good Slack".

And the main issue (unless I'm misunderstanding, but I did go through your pricing page again to double check) is that you do not offer SaaS except to enterprises. Which is super backwards. You're not going to be able to gather mindshare if you require people to set up your product themselves to even just try them out. Nearly none of your potential users have the expertise to do that, and those that do aren't going to want to do it unless they know what they're getting into beforehand. The only remaining ones that already do are Gitlab users, because they've had it set up for them.

Github, Travis, Slack, Discord, Sentry, half of the integrations you find on GH, what do they have in common? They're hosted and free to try. Make a free hosted version, make it very limited if you have to and promote upgrades/self-hosting, and then you have a shot at getting solid mindshare.

On a lesser note, since you asked for feedback, here's a couple more pieces of advice:

- The demo video you linked me is great. I see Mattermost has changed quite a bit since I last tried it, for the better. So I visited mattermost.com to try and get another look and it's super barebones on screenshots, I found no videos, there's no demo. After browsing the site, I have no feel for what mattermost looks like. And you don't link to your community server anywhere I could find, not even on your contact page! - Consider making public Mattermost instances optionally readable without an account - read only, and enable that for your community server!. This would differenciate you from Slack and Discord and would improve the situation above. I realize it's a lot of work. - On pricing again, it's irresponsible to put 2FA behind Enterprise only. Please don't do that. Push notifications could also stand to be in the free version, imho.

Cheers, hope that helps. My email is on my profile if you want to talk about this further.

Hi Jerome, highly appreciate your feedback. I can see where you're coming from.

You're 100% correct on website, we need an upgrade. We have someone starting soon to help and we'll show them this thread.

A couple of notes:

1. Push notifications are free. Paid is if you want to use Mattermost key for encryption (we choose not to encrypt things for people we don't know, but we document how they can do it themselves): https://docs.mattermost.com/overview/faq.html#are-push-notif...

2. Our intended users are organizations that need to deploy communication behind their firewalls (no 2fa from us needed) and can't use SaaS, e.g. manufacturing, banking, healthcare, public sector, security companies, companies with very strict non-disclosure agreements, etc.

Logos on our website are examples: https://about.mattermost.com/

PS: I read about HearthSim, you're running a really cool company. If you're ever in San Francisco for GDC, perhaps we could meet in person.

> Push notifications are free.

Might want to clarify that on your pricing page then; I thought the free version simply didn't have push notifications since they weren't in there.

> If you're ever in San Francisco for GDC, perhaps we could meet in person.

You bet. I'm usually in Anaheim for BlizzCon which is a safer bet, I don't normally do GDC. Feel free to shoot me an email if you're around :)

Huge fan of Matrix, really hope it takes off. They've got the right idea and it plugs into other things like IRC very cleanly.
(comment deleted)
I don't disagree with your points but your timeline fudges the timescale quite a bit. I've researched this [1].

Facebook Chat went live 2008-04-06. Google didn't view them as a threat in messaging but a threat in social networking; they thought they had messaging in the bag with their 2008-09-23 release of Android 1.0, and their 2008-11-11 update to Google Talk which brought voice and video calling, and their 2009-03-11 acquisition of GrandCentral, which was soon rebranded an invite-only Google Voice. But Facebook kept growing and growing and it had an integrated chat on a website where people went to spend their time, instead of Gmail, where they went to manage email.

To combat Facebook on social networking, Google launched Buzz with aggressive auto-opt-in on 2010-02-09. Buzz fizzled and attracted controversy for its aggressive piggybacking on Gmail, so Google tried again with Google Plus on 2011-06-28. That was a better effort, and it included the features "+Messenger", a text chat, and the video chat "+Hangouts". By this point Facebook had more than 700 million active users, and won messaging handily; its lead was cemented by the acquisition of WhatsApp on 2014-02-19, as Google continued to flail about.

In a post of mine last year [2] (which includes an older revision of the timeline linked in [1]), I speculate that it was Facebook Chat that killed the mid-2000s chat networks of old like AIM, Yahoo Messenger, and WLM, rather than Google Talk or any particular missteps of those incumbent chat networks. For example, I was surprised to learn that AIM was present in the iOS App Store at launch -- of course, there were no push notifications at the time -- not until 2009-06.

[1] https://news.ycombinator.com/item?id=13465483

[2] https://news.ycombinator.com/item?id=11114518

> there is no open choice I can make at this point that is approachable enough that I can convert people to it.

https://meet.jit.si/ works just in the browser. What's the problem?

I will answer what you don't get.

Just because you use Google's stuff all around doesn't mean Google were "one of the only players...."

Google was a distant third in IM behind MS and Yahoo.

I've never seen a Zebra doesn't mean the Zebra doesn't exist.

> So they rebuilt Talk on top of a new protocol. This time, it's proprietary. This time, it's much slower in the browser. Oh and you lose half your contacts if you upgrade to it. But at least now it works on phones?

Talk had a full-featured native phone app way back in Android 2.1 with multiple device support, which Allo still doesn't have today. I used it religiously on my Nexus One.

With that said, despite being a poor product I rely on Hangouts daily because I can make free WiFi calls with Voice. I can't afford cell service and it lets me stay connected to my family.
everyone get and will get IM wrong.

communication is taboo for ads. and nothing on the Internet can have a fee. see the incompatibility?

Facebook is spending on IM because it bought the first IM that dissociated itself from the Internet. every whatsapp user see it as a sms and phone company alternative.

everyone else that live by ads (Google, yahoo, etc) will try their hand at IM, realize they can't use it for ads or ad targeting, and will scratch it.

I don't understand what you don't get, this is how google works. Google does things like that constantly: buying popular softwares to kill them, make services such as google reader and kill them, etc.

Google has its own agenda and making an open and distributed internet that respect people privacy or wishes is not part of it.

I'm just going to type LOL and see what happens

All the context for the LOL has been provided with great previous discussions that have happened on HN on many/frequent Google kills

So here it goes...

L-O-L

Aka their best chat client. Instead of having one great chat platform they now have several not so great clients.
Allo, Duo, Android Messages, Hangouts, Gtalk. Oh you have 3 Gmail's because you've been an Android YouTube and Google Apps user since 2010? Good luck with that nightmare, we'll get all your personal and business contacts all jumbled together!
I just figured out I could access my hotmail inbox from outlook.com using my Microsoft Account credentials associated with an email address setup through Google Apps using a custom domain.

It was full of spam.

They should just keep a constant URL that will redirect you to whatever is their current messaging/calling product of the day. Jeez.
I'm pretty upset at Google about how they handled IM. It basically cost me more than a half of my contacts (who use Google), since first XMPP federation with Google Talk stopped working when Google refused to support server to server encryption, and now it's being dropped altogether.

So much for moving things forward, instead of falling into the dark ages of incompatible balkanized instance messaging services.

That decision basically killed instant messaging for me. I miss it, but there just isn't any critical mass anymore. I could run half a dozen different messaging apps, each one reaching some subset of my friends, most of whom are never available anyway, but the old situation where it was easy to keep in touch with people via chat just doesn't exist anymore.
> That decision basically killed instant messaging for me.

I feel the same way.

I think instant messaging also worked better in the PC era, when you knew people were active, "away" or altogether offline.

The messages I want to send or receive with a phone notification are a lot different from the ones people would exchange in the old days of IM.

I use pidgin, how does this affect me?
If you use Pidgin to connect to an XMPP federated service, then that won't work any more.

If like basically everyone relevant, you use Pidgin to directly chat with Hangouts users and to hang out on IRC (the federated protocol that actually succeeded in the marketplace), then you don't care.

I genuinely don't understand the outrage in support of XMPP. It didn't work out guys. IRC tuns out to be all anyone actually wanted, and to the extent that's not true, it would have been eaten by Slack anyway.

You didn't see this part in the article?

> "Third-party XMPP clients will continue to work with Hangouts for 1-on-1 chats."

As part of our continued effort to focus Google+ around shared interests, we’re retiring two legacy Google+ features in Gmail: the ability to email Google+ profiles and the use of Google+ Circles. (Expected: No earlier than April 24, 2017)

Whaaaat? Emailing profiles was the most useful part of g+

Good. They should kill every chat app except for Hangouts and merge functionality.

They're losing a lot of points with customers with the "have 10 teams make 10 apps that do the same thing and slowly kill off support before discontinuing" model.

This is for G Suite business customers. They don't explictly say if this applies to regular Google users.
This is more likely regarding the fact that Google is obligated to warn their paying customers, and not obligated to warn their products. If these services didn't tie to G-Suite, there'd likely be no warning at all.
This blog post mentions following messenger technologies (Google has even more not mentioned here):

- Google Talk

- Hangouts Meet

- Hangouts Chat

- Android Messages (something new, never heard of it)

(And non-google messenger technologies: SMS, RCS).

How user is supposed to navigate all these? Given that all these things are inferior to even Facebook and is ICQ-grade.

Android messages is the (renamed) default SMS app for Android. It's actually fantastic, beautifully designed, responsive and lightweight.
But Facebook Messenger now wants to eat Snapchat's lunch and is offering Snapchat functionality. Great, that annoying cat lady who I'm "friends" with now can spam my Messenger app with pics of her cat. And I shared a picture with a friend and Messenger suggested I posted it to it's NotSnapChat. It's a single button inside the chat bubble, which makes me wonder if the publishing happens directly when I push that button, and who'll be the next Anthony Weiner after he accidentally presses that button...
I've thought about this, too. All the chat and social network tools are constantly shifting their UIs and naturally, I guess, trying to highlight new features, which on a tiny phone screen is equivalent to making them easy to accidentally click.

The worst I've done is accidentally send a Facebook "thumbs up" in an inappropriate context, but it's easy to imagine how you could do much worse.

I downloaded Facebook Messenger Lite and highly recommend it. It strips out everything except... well... Facebook Messenger. I can message people, and there's none of the extra UI crap, including their Facebook branded Snapchat copy (not to be confused with their Instagram or Whatsapp branded Snapchat copies).

Officially it's only supported in emerging markets, but you can download it from apkmirror and it'll install just fine.

just use Messenger Lite and same goes for Skype Lite
Hangouts Meet and Hangouts Chat are part of Hangouts realignment to be enterprise-focussed, and are, IIRC, only available as part of GSuite. Users don't choose them, admins do.

Android Messages is an SMS-only app; your device vendor probably has their own default one, but if not (or if you don't like it) you can use the Google one. (It's not that new, it's been around since about the time they announced the new direction for Hangouts, long before they implemented the new enterprise-focussed Hangouts offerings.)

And GTalk is being buried, so isnt really an issue.

In my experience users don't know or care to know the name of the product they're using beyond 'google'. If the app has their friends and contacts in it and people respond to messages they'll use it.

If I ask someone what app or service they used to send me a message I didn't see, they can't really tell me.

Increasingly they don't even know if they sent a text message or something else entirely.

If you're a Project Fi user like myself, you have all access to all of these technologies. All you need to do is pressure your friends into joining Project Fi and they'll be able to chat with you too!
I don't know why, but I have this Deja Vu feeling...
Android Messages is apparently [1] the renamed "Messenger", the Google one (the internal android APK names are the same), which was probably confused with "Messenger" the Facebook one, likely to Google's detriment and Facebook's gain. Renaming it probably salvages some of that confusion.

Also, Hangouts "the original" (aka. the Hangouts App, Chrome Extension, Chrome App, Gmail Widget, and the standalone website) -- most of which will probably live on. The extra irony is that the same time Hangouts App users are being warned to move to a different app for mobile text messages, GTalk users are being told to switch to Hangouts, while Google Voice users are being assured that their Google Voice texts (and presumably calls) will keep working within the Hangouts App, even though a new Google Voice App was recently released that mysteriously added chat.

Then there's Allo, Duo...

[1] http://www.theverge.com/2017/2/24/14721602/android-messages-...

I switched back to SMS/MMS, it works every time and never changes.
I switched back to SMS/MMS, it works every time and never changes.
Remember the first law of Google services: if it's not useful to sell you better to advertisers it's not useful.
At this point I have no idea which Google chat client is which. A few years ago I remember having Google Talk on my android phone. Then after some random update the application disappeared from my phone and was replaced with hangouts. Now is Google Talk the same as GChat? Was Hangouts anything more than just a new branding for Google Talk? I have no idea. And what's up with this Google Allo thing? Again, it seems like Google seriously dropped the ball with branding and communicating wtf was going on. Most of my messages has moved over to FB Messenger or Whatsapp/Wechat and Google only has itself to blame for that.
I'd say google has entered into a post modern era of chat. I don't think there's anything they could announce that would actually mean something to the generally public, there's too many overlapping brands with overlapping functionality.
They're like the 1980's GM of chat products right now: too many overlapping products cannibalizing their own userbase.
I mostly agree, but there is one thing that end-users are dying for. Truly federated chat services and open standards. Its the only thing that anybody could do differently to really stand out above the rest.
All I know is that every now and again I notice a bunch of notifications of missed conversations in Gmail, sorry Inbox, or maybe it was in Gmail, and once again swear and tell people it's a bad idea to try to get ahold of me that way because odds are it will take ages before I see it.

I'll never trust a Google messaging app because of that mess, and frankly I'm getting increasingly frustrated with Gmail/Inbox too and will probably move off it soon.

I agree 100%. I'm frankly still not certain if Google talk and G-chat are even the same thing. I've used G-chat for a long time, but don't remember the name being changed.

Also, regarding branding: even overlooking the shell-game Google is playing with regards to moving features around (SMS => Voice, SMS => Hangouts, SMS => ?), the branding was confusing from the outset:

- Google Chat (GChat) - Google Talk - Google Voice

These might as well be synonyms.

Seriously. I somehow ended up using all of them, and I honestly don't know what's going on anymore. I thought my Talk got merged into Hangouts, if I start a chat in Gmail it's sometimes findable in my iOS hangouts app, and sometimes it's not. A while ago the Voice team asked me to merge Voice with Hangouts, which I did because clearly the service was dead. This seemed like a plan... your SMS chats are just chats in Hangouts and in addition to calling other Hangouts users you can call phone numbers, but we're keeping the Voice app & site around for legacy users. Fine. But in January they released an upgrade to the Voice apps with this gem:

If you currently use Hangouts for your Google Voice communication, there’s no need to change to the new apps, but you might want to try them out as we continue to bring new improvements.

W.T.F? Don't bring me into your internal office politics...

Then there's Allo because with all the above they made it too complicated to chat with someone or something. Also Duo, because video calling was completely missing from the offerings above (it wasn't).

I just use hangouts on all platforms, and it seems to work well. Although I did have to install "hangouts dialer" on android to make phones calls from Hangouts. Not sure if that's still required.

Google: please just leave hangouts alone!

It's so disorganized you wouldn't believe it if someone told you this is how a company actually approached IM, and not just any company, but the biggest internet company in the world. I literally have no idea what I'm supposed to be using now. Can I get google hangouts messages (is that even a thing?) and sms in the same app now somehow?

I'd love to be a fly on the wall where these decisions are being made, the powerless non-autistic people in the room must be simply incredulous.

> ... the powerless non-autistic people in the room must be simply incredulous.

This doesn't belong on HN, so please omit these sorts of things from comments.

Google is a feudally organized collection of fiefdoms and this shows in their product strategy.
Really makes me wonder their long term survivability being that way. It doesn't seem to be working well for them.
Part of the confusion is that they kept around the old clients and protocols. There is also a lot of confusion between the different apps and the platform.

Google Talk is the original XMPP based chat system and was known informally as Gchat or Gtalk. There were web and desktop clients. Hangouts is the "new" proprietary messaging system from 2013. Google Talk and Hangouts are integrated at least for text messages but Hangouts adds lots of features like group video chat. Google Talk provides XMPP protocol access for federation and third-party clients.

Hangouts app on Android could do Hangouts, local SMS, and Google Voice SMS.

I find it interesting that there is less confusion about Apple Messages even though it was renamed from iChat to iMessages to Messages. The app also supports both SMS and iMessage protocol. In addition, iChat supported XMPP and third party clients until was shut down.

> find it interesting that there is less confusion about Apple Messages

The reason people aren't confused is because Apple just renamed the app on people's iPhones but didn't screw anything up for them. They were never forced to switch from one app to another app, never had multiple apps that did roughly the same thing with competing standards, and they didn't migrate users from one platform to another losing data in the process.

Neither did Google. The Android app for Hangouts is com.google.android.talk
Don't forget Messages!
And is that the same as or different from "Google Messenger" (an SMS app)?

For bonus confusion, the Google Messenger icon just says "Messenger" and is nearly indistinguishable from Facebook Messenger, a completely different system. Better keep track of which blue speech bubble is which.

It seems Messenger was just renamed Messages, probably for this very reason.
Yep, and I wish a bunch of HNers with lots of time would spin up an OSS Google Apps (for most of the apps at least) replacement. Then people could do one-click installs on AWS or their own infrastructure, and the like... (Yes, I know Zoho exists, but it isn't a close enough fit imo)
All I know at this point is that I'll keep using Hangouts and when Google eventually kills it, I'll never use another Google messenger again.
just curious, why would anyone outside China use censoring spyware like Wechat?
(comment deleted)
I feel like Google has a bunch of teams working independently on things like this and somehow don't have to check with some central assurance party before releasing their competing products.
Can somebody please explain to me Google's messaging strategy? As a Voice, Talk, Hangouts, SMS user for what feels like the last decade, I am so incredibly confused that I want to walk into Google and fire all 18 of the 19 messaging teams. That way at least we'll have one singular group to yell at when things go wrong.

Here's the Google Messaging landscape as far as I know it:

- Google Talk in Gmail: I think this is what my work account has, and what they're shutting down: http://imgur.com/Z0v6Zjq This one is pretty useful because you can initiate a new Hangouts Video call right from this interface, as well as open up the Google Talk Voice POTS Dialer (?)

- Google Hangouts Messaging in Gmail: I think this what my personal account has, and perhaps what's sticking around? http://imgur.com/dBf7me2 This interface clearly got upgraded a bit, but removed the ability to create a new Hangouts Video call unless you're initiating it with a specific person (or I couldn't find the place to do it)

- Google Hangouts for Android: This is a dual Hangouts and SMS client that supposedly will send messages with the two services above ^, plus over SMS to your carrier contacts. This is also being discontinued?

- Android Messaging: Sounds like a new app that's coming out which will replace the SMS side of the above existing Hangouts app

- Hangouts Chat: This is the new app that's going to replace the IP-based messaging feature of the current Google Hangouts for Android, right?

- Hangouts Meet: This is the new app that's going to replace the video and screensharing portion of Hangouts for Android it sounds like. Why these two are now separate apps is beyond me. But it sounds like these two will interoperate with Google Hangouts in Gmail (the second screenshot), and the first one is disappearing and/or being converted into the second. OK, I think I'm staring to figure this out...

- Google Voice: Oh don't get me started on Voice. I've been a Voice user forever, and we finally got an upgrade! Which made no sense, because I thought they were killing it? But now they're not? So now Voice is purely for SMS-based communication, and would be an alternative to Android Messaging, just as it always was (with its new web client as well). But if you were one of the unlucky people who chose to integrate your GVoice SMS experience into Hangouts for Android, I guess you can still use that? Supposedly that's still supported, but I didn't, and I fear for whoever ends up with that experience...

- Hangouts Dialer for Android: This is / was the POTS dialer that integrated with Hangouts, and allowed you to dial any POTS number over VoIP. AND it lightly integrates with Google Voice, because it uses your GVoice number and balance for international calls. BONUS: It can also /receive/ inbound calls that are placed inbound to your GVoice number, OR from Google Talk, because "Google Talk" was considered a "forwarding phone" as as Voice was concerned. Is that part going away? Can you "call" people over GTalk now? Sounds like no, but you can still "Hangouts Meet" with them, and if you have the Android client, it will ring your phone? I guess that makes sense... What about inbound calls to your GVoice number and being able to answer them over VoIP? That works today, and it'd be a real bummer if it went a way.

- Google Allo: Hey remember this? Probably not even though it's only less than a year old. This is an entirely separate walled garden of a messaging app that allows you to Allo (text chat) other Allo users. But it also DOES have an SMS bridge (surprise!), and you can Allo non-Allo users, however the receiving SMS end is still a little questionable and is more meant as a user-acquisition funnel than a transparent GVoice-like experience...

> Screw Google and whoever is running their messaging division. Except I think that's the problem: nobody is.

Nobody is. AFAIK, they're basically all from different divisions of the company, and that's why it's a total clusterfuck.

I always thought the reason why there is a CEO is to avoid such clusterfuck.
And the short-lived Google+ Messenger
Firing 18 of the 19 messaging teams will just make 36 new ones sprout up.
Gtalk.exe should be on this list... Man I miss desktop IM apps...
Thank you. I think this was the most complete post I've seen of this and largely lines up with what I know of it as well.

The worst part is they have tons of really great things, but rather than improving and working on getting more users they keep starting over.

I feel like at one point they were working towards an experience where you could:

* text, voice or video chat (and flip between them in any conversation at any point)

* in 1-1 or in groups

* integrate with PSTN voice and SMS (to the extent possible)

* access across several platforms (with a seamless experience if you move from one device to another midway) including native PC, web-based, and Android apps

They got like 80% of the way there -- then just gave up, and everything they are doing now is just on its own randomly wandering path with no cohesive vision.

so now that you pretty much described Skype why people insist to keep using Google products?
They are shooting in the dark hoping that some idea catch.
Even if you ranted on this topic 2x as long I'd upvote you several times if I could. You couldn't have explained it better even if you tried - the singular, overarching point being - lack of vision, lack of leadership, lack of interoperability and openness... and just pure greed, or rather, vanity. They've long departed from their "do no evil" days and there's just no coming back.
For them its a very smart move, if its a non-core part of the business that is not generating revenue, it makes sense.

From a user perspective, it is a bad play. Just like Google Reader, there were users that relied on the software. While it may save them money, it hurts them in other ways.

"Third-party XMPP clients will continue to work with Hangouts for 1-on-1 chats."

For me, this means it basically still exists and my user experience is the same.

Somehow this 1-on-1 chat has stopped working circa 2013 for me. People using google still show up as online in my contact list but do not ever receive my messages. On the other hand I do not appear in their contact list anymore.
> we’ve evolved Hangouts to focus on two new experiences

> Google Talk launched in 2005 as a simple chat experience

To quote Inigo Montoya, 'You keep using that word. I do not think it means what you think it means.'

> As part of our continued effort to focus Google+ around shared interests, we’re retiring two legacy Google+ features in Gmail: the ability to email Google+ profiles and the use of Google+ Circles.

Sad to see Circles die — that's what was compelling about Google+. I don't even know what it is now.

Why's this getting downvotes? I think that calling software & features 'experiences' rather than 'software' and 'features' is a symptom of precisely what's wrong with Google's current direction.