Another useful feature, interesting article to look how it works under the hood.
And again, i wonder how a tiny team can push such great and useful features into such amazing UI. And then I'm looking at other alternatives, from naked WhatsApp over laggy wechat to horrible UX in signal.
What's the reason for telegram amazing performance and features?
Telegram is pretty impressive when you take into account the number of (well implemented) features that they managed to shove in the app without breaking ergonomics and performance.
At this point, I just think the answer is top notch developers and designers with an impressive alignment about what the product should be.
I know no other free application with such an amazing usability. There must be some shitload of money behind them, for sure.
Not quite, Discord's conceptual design is much messier due to its monetization attempts. Plus nowadays it isn't nearly as fast as it was a couple years ago.
From what I gathered: a boatload of money from the rich owner for development and hosting, plus not caring about encryption in any way. Don't need to bother with tricky distributed device syncing protocols if your servers simply store everything and can index the data at leisure for fast global search etc.
It's still impressive, though. It's not as if Facebook's chat works this well and that isn't (wasn't?) encrypted either and they've got an even bigger pile of cash. It's also a lot older though, just look at the evolution from IRC to MSN to Facebook to Telegram: each step it gets better. Maybe that's how that difference can be explained? Anyway, between normal messengers like Signal/Whatsapp/Wire/etc. and Telegram, the main difference is not caring about privacy. (To be clear, I don't mean that whatsapp is a privacy-conscious messenger as they will collect what they want behind the scenes, but on the client side they have to care about keeping the server "untrusted" and that will slow everything down a lot.)
People tend to forget that Whatsapp encryption was introduced years after Telegram’s secret chats. Now they are continuously bashed for no encryption by default, but their original proposition was “messenger done right” as compared to unencrypted Whatsapp and FB.
I'm well aware of that and I don't have WhatsApp because of its privacy problems (much to the annoyance of some family members, but I imagine it saved me from a lot of printer queries, and if they aren't willing to switch then why should I).
What I think you're underappreciating is that Telegram was built on the premise of privacy and they've neglected that from day one. It was better than the status quo on day one (at release), but have fallen further and further behind ever since.
Telegram was launched when Whatsapp sent messages without transport encryption over port 443 (this was fun on public wifi!) and got big when Whatsapp was bought by Facebook because Telegram was independent and had proper end to end encryption (with air quotes around "proper" if you like; it wasn't OTR-grade but at least it was better than the status quo). Since then, nothing happened whatsoever on the privacy front. Even the legal compliance is a complete joke (gdpr data exports only work in a few clients and not very well at that), but more importantly to me, they need gdpr data exports because the servers still know everything. Whatsapp moved on, Signal got a lot more mature, Wire has also come onto the playing field, and Telegram has done nothing since before Snowden told us to turn on https basically.
Not just UX, also availability. I simply cannot use encrypted chats in the Telegram desktop client that I use the most and runs on the device that can actually get security updates while being fully owned by me (I, ahem, "rooted" my laptop and it can still install system updates without any hassle, it's amazing). This is why I'm slowly moving more and more friends to Signal but their UX is also very mediocre by comparison.
I'm using Matrix with two friends and it's just a world of pain. Can never find any messages because search is not implemented on web or android (only ios seems to have it or an electron desktop client... no thanks to both), random issues with encrypted chats (it all works great if you leave e2ee off! Though it got a lot better already since a big update ~1.5 years ago) such as unavailable message keys, random bugs, and too complicated for use with non-techies (so not an option for my mom). What helps is having alternative contact channels for when a message fails to decrypt or the custom homeserver is down again.
There has been a lot of work on it but the smart thing to do would have been to copy Wire or Signal and build on an existing thing if you don't have the manpower to start from scratch and want to be a mainstream alternative.
I'd rather recommend Threema (best UX, not so great in features or encryption tech), Signal (network effect, best privacy, reasonable UX), or Wire (like Signal but slightly better desktop experience).
I have been testing it for about 3 years now (FOSDEM 2019 made me interested again)
I started pulling people to it only a few months ago when I thought it was good enough (includes almost no encryption issues happening in all day usage)
My mom uses it as well. It works fine. I had to install the app though, like all the other apps.
The reasons why I found Matrix interesting, were:
- Telegram like syncing messages across all devices while adding a new device is as simple as setting up WhatsApp Web
- It's a standard for interoperability - I got sick of telling others what to use and being told so by the network effect
Copying Wire or Signal just wouldn't work for technical reasons. I also wouldn't want a copy.
I got all of your recommendations over time (including buying Threema with all my friends), but they just haven't been good enough for us and me.
In practice, no one cares about Electron. It seldomly VScode or Teams are used at work
Of course, I find that to be a given these days. When people talk about encrypted chats in 2021, it's not about transport encryption... installing let's encrypt for your api is the easy part.
Actually Pavel Durov made some very rough accusations against Signal and other "secure" apps for using the standard encryption method. For what he says, he doesn't trust others encryption protocols because he believes NSA made some pretty well hidden backdoors in them to easily decrypt them.
Meanwhile I'm just here thinking, what's the matter if you're messager is super duper safe if your device OS running them is plagged with backdoors?
Kleptographic backdoors in the Signal protocol is FUD. I work as a security consultant, which is definitely not a cryptographer, but I do get around and "NSA made some pretty well hidden backdoors in them to easily decrypt them" is definitely false.
> what [if your] mess[en]ger is super duper safe if your device OS running them is plag[u]ed with backdoors
This is more relevant. Bugdoors more than backdoors, or perhaps just stupid bugs and enough budget to find them, but yeah basically that's how messages are decrypted these days (e.g. NSO group).
Wouldn't it be a good idea to use two chained encryption methods? AES+whatever Telegram is using? This would be resistant against a AES backdoor and against bugs/backdoor in the Telegram encryption method. Similar to TrueCrypt where you can select AES+Serpent+Twofish to encrypt your files.
That would work, but historically this hasn't proven necessary and double encryption means double the CPU power (or more, if one of them has CPU extensions and the other encryption method does not). Doing group calls in Wire is already very taxing on nowadays' energy-efficient laptops or not-high-end phones.
For example, people have been using Russian GOST algorithms as a hedge against the USA stuff, but it's falling out of style because it just hasn't proven necessary in the decades since the AES and SHA families came into existence. Any bugs we found, for example in SHA-1, affected everyone equally and did not create a backdoor as with Dual-EC (in which flaws were identified before release and which went unaddressed, very much unlike other common algorithms).
Idk about its security, but proxies' security is much low that owner of the proxy (Domain of the proxy) can see your messages sending while using his proxy, and it backups your messages.
For me, I usually use telegram, it's better and more comfortable than whatsapp.
I dont judge about its general security, but the proxies security is shit as possible.
btw I live in Iran and I have only two ways to connect to telegram, VPNs or Proxies.
But with these conditions, I rather using telegram instead of shitty whatsapp
End to end encryption seems like an endless pain in the ass. So many expected features like server side searching and link embeds become basically impossible to do in a secure way.
The problem with having your own server instead (then you can just do transport encryption and call it a day) is that everyone needs to be on your server. Or go decentralized, but that has its own host of complications that may or may not be easier than end to end encryption.
Also note that it's not impossible. Wire and Signal have come a very long way already, it's just the front-end (UX) that they're lacking on really. Telegram has features like a video editor and user-filtered search, but those are all client-side things and have nothing to do with e2e encryption. All the things that do (sending messages, emoji reactions, group chats, group video calls, etc.) are already implemented by both apps.
> Another useful feature, interesting article to look how it works under the hood.
I think you might be missing the point of the article, which is that they’re misusing Google APIs to avoid paying for the proper way of doing things.
This feature will break as soon as Google throws up a captcha, because these endpoints aren’t intended to be used programmatically.
> And again, i wonder how a tiny team can push such great and useful features into such amazing UI.
The translate functionality comes from Google. They’re just sending messages to Google and getting a result back, at least until Google detects the API misuse and breaks the response.
I guess, given its popularity, Google won't kick Telegram off the store for obfuscating the URL and using an unauthorised (?) API endpoint but I imagine this will get them in some sort of trouble.
I think they could do it. In Germany, Telegram is often cited by media as a platform for (illegal) right-wing, antivax and hatespeech. Some politicians openly demand to go ofter Telegram and/or block it. So google could kill two birds with one stone here. At least remove it from the Play Store in some countries.
Telegram is basically opposed to any kind of censorship which is IMHO a good thing. Western politicians are outraged as hatespeech, murder threats, right-wing paroles etc. are openly distributed through Telegram and demand regulation/takedown. At the same time, when protesters in Belarus or HongKong use Telgram to coordinate and those Governments demand takedown, the West screams oppression and demands freedom of speech.
In general I am pro Telegram, as I think any democracy needs to have censorship-free communication for whistleblowers etc. and to prevent attacks against democracy itself. Even if this means we have to live with stupid/illegal opinions being expressed too.
> I imagine this will get them in some sort of trouble.
I'm not sure about this.
I bet Google is happy to collect the text data of up to 500 million users with zero restrictions from Telegram's end on how the data is used. I'm not a lawyer, but my hunch is that Google's data privacy policy applies to the official, premium service: https://cloud.google.com/terms/data-processing-terms
Google might make the determination that they'll get more value from allowing Telegram to abuse the unofficial API. However, they might face some angry customers who are paying a premium to use the official API now that this loophole has been published.
I think it's possible construct to construct a (very weak) argument for the random user agent rotation, but why split the spring if not to avoid being flagged.
On the other hand, I find it hard to believe that Telegram would risk a Play Store ToS violation, given how many tens of millions of users use the app.
Pretty sure at the point you have over a billion(!) installs, even Google affords some leniency towards its Play Store policies. Or at least we are about to find out anyway..
Meanwhile, indie developers with smaller user base are subject to unappealable automated decisions.
Isn't Google going to move to always having the user agent be the same anyhow? They've already decided to break that contract with the tech community, so I don't see that they have much room to talk there.
That contract was torn up a long time ago. Or do you really browse the web with "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"?
I'm not sure if Google will start flagging the IP addresses of the users because of each request having a different agent. That would render normal Translate unworkable for them too!
It allows Telegram users to hide in plain-sight, within the noise of other Google Translate web users.
I'm pretty sure that using the official pre-built java SDK, as suggested by the author, would allow Google to cluster the content of Telegram users (since app-specific id/token should be sent).
Other than that, a great read and kudos to the author for shedding light on it.
I think Google can still cluster Telegram users pretty easily, especially now that that the method is in the open.
Yes, Telegram fakes the user-agent, but the rest of the request still looks very different from a request an actual browser would do. (No referrer, missing headers, different connection pooling behaviour, possibly different TLS and HTTP2 behaviour, etc).
So if Google is doing any detection for browser vs non-browser requests, those requests should show up as suspicious.
On one hand, it's quite asshole-ish. On the other, google is serving broken frontends to their services and charge ridiculous prices on their API's. When I tried to make a third party search using google engine, I've exhausted the limit in less than an hour. It'd cost me like $40/mo to get what I get for free using their crappy frontend.
Like telegram did with the translate api, there is also a way to have an unlimited api for search results. You have to find one of the old mobile pages of google.
> On the other, google is serving broken frontends to their services and charge ridiculous prices on their API's.
How does that make this okay? Nobody is entitled to get a company’s services for free just because you think their price is too high or their front ends aren’t built to your liking.
You could use the "turnabout is fair play argument". If you publish a web page, and don't specifically block google, they scrape your content, and use it for their own purposes. And even use it for "rich snippets", products other than search, etc. You're basically doing the same to them...using their content for your own purposes until they specifically block you.
Disagree. The web is clearly architected such that publishing a webpage makes it public and crawlable. You don’t “block Google”, you specify that the site is not for crawling in robots.txt according to well-known standards. This is all basically the contract of the internet and it shouldn’t be surprising to anyone.
Google specifically does not publish their API for free consumption by other companies, yet that’s what’s happening here anyway. The company is also using specific tricks to circumvent detection of the behavior.
In your analogy, this would be like a crawler ignoring robots.txt and then scraping the content for their own website with zero attribution to the source, which is nothing like Google indexing your site with full attribution and driving traffic to it for you.
Regardless, “turnabout is fair play” is unequivocally not a legally or even ethically acceptable standard, so that argument wouldn’t actually hold up anywhere anyway.
`noindex` was a Google-specific rule that was never officially documented nor supported. I think they were perfectly entitled to withdraw support for it, especially considering there are alternatives.
I don’t understand your argument. There is no actual “publishing” of web sites or APIs on the web. You simply make something available at a URL, and it’s up to anyone else to discover that URL. In this regard, your personal web site is no different than this Google Translate web API.
"Nobody is entitled to get a company’s services for free just because you think their price is too high or their front ends aren’t built to your liking." Tell this to Google!
Someone deleted an interesting comment about adversarial interoperability [0]
I’d love to see and give money to a project to create and maintain easy to use and stable “adversarial interoperability” APIs for as many services and products as possible.
Perhaps companies and projects would not often use these directly because of the risks (hopefully some would, though!) but individuals could drop the library or the URL to a server hosting it into their apps to gain extra features.
If standardised, whole open source apps could be built around them that allow querying and analysis of data from services and aggregating and automating using the services including optimising prices, taking advantage of offers, and using undocumented APIs to the users advantage.
Maybe something architected and incentivised like https://thegraph.com/ for adversarial intercom and undocumented APIs. Building as a network of nodes and funding with crypto would make it harder to attack and take down.
"pip3 install woob" succeeds but e.g. "woob config-qt" returns "not a woob command". Are the -qt versions not available via pip? I have python3-pyqt5 installed.
So they finally gave in to the complaints about the name (https://lists.symlink.me/pipermail/weboob/2021-February/0016...). woob is excellent but unfortunately its very existence means there is a problem. Hopefully they will gain visibility and problematic services will adopt standard protocols
This is nice, and feature rich. Its available in Homebrew, where I was able to try it out (if I ran Nix I'd been able to use nix-shell for such). However, its France and French oriented. Not the world-wide interface I expected it to be. Bands for example, only contains a metal database. Recipes, 5 out of 6 entries is French. Travel, I don't see German or Dutch or UK or Belgian public transport.
> Perhaps companies and projects would not often use these directly because of the risks
If you provide a solution to someone’s problem, it will be used all over the place.
The biggest companies won’t use these things, but plenty of smaller companies and individual programmers without oversight would use them without a second thought, at least until they’re caught.
Telegram is a relatively large business and here they are abusing an API exactly like you suggest.
Along those lines: maybe we could use a middleware pattern for APIs, frameworks, etc where the interface/package would be built as a layer above two or more services.
That way the developer could switch between them at any time, or even failover automatically.
So for example, rather than going onto GitHub to download an SDK for something like Mailgun, you'd download a middleware framework built on Mailgun and Sendgrid.
This pattern could be used to identify vulnerabilities in software at the conceptual level, by helping developers to avoid marrying their code to individual providers like AWS. Some mission critical software could even be certified as using all adversarial interoperability frameworks.
It could even help third party services pull themselves up by their bootstraps, if they get added to one of these middlewares. An instant user base without having to rely on marketing or word of mouth.
And could be used to identify monopolies when there's no middleware for a service.
I was advocating for this for years but gave up when I talked to someone at Pager Duty that was just straight out against it. For almost irrational reasons. I kept trying to make the case that for a company like their that literally can't be taken offline without risking huge chunks of the internet why wouldn't they want to run their whole business on an abstracted away platform to, say, switch to Windows if linux had a 0day, etc.
> Too many moving pieces. Too much work.
It still boggles my mind that this is the way we do things. "Patch fast!" Ok cool. That's going to keep working out forever.
I would love to have applications that are more tools than products and weave together these APIs and middlewares, both in querying and visualising, combining, enriching, analysing, filtering etc. data and also taking the output and actioning it.
With the amount of data and services that are now available online we should have superhuman capabilities but the productisation of the internet has left us stuck in company run silos fighting “user journeys”, undocumented APIs and EULAs.
youtube-dl does this for video sites, it works on more than just YouTube despite its name.
teller.io does similar for banks.
I would love to see a coordinated effort along the same lines for things with non public APIs. It is however a huge ask as internal APIs are unstable and constantly changing/actively working against things like this which is a huge amount of work to keep up with.
> Someone deleted an interesting comment about adversarial interoperability
That was me. The comment was rapidly amassing upvotes, which made me feel that not only was the comment preaching to the choir but it had the potential to derail the discussion around the actual issue of Telegram's use of Google's Translate APIs.
I wonder how useful this, considering how Telegram conversations are unencrypted by default. If they were to change this default, now _that_ would be something.
This can't work for long. Translate is a profit center for google, and this also shows others that they can disregard google's monetization model for translate.
Commercial use of those APIs is common, despite translate being pretty expensive. Also, GCP current leadership is so hell bent on nickel-and-diming their customers, and their compensation packages are so dependent on value share growth, that they simply can't afford anyone openly violating their pricing models. Especially a popular app. My guess is this will be down within the first week of January.
I'm curious what techniques they will use to differentiate between Telegram and non-Telegram users. If I were them, I'd simply use my power leverage and threaten them to remove the app from the Play store unless they remove/fix the offending code - it's much simpler than an eternal mouse-and-cat game, with possible collateral damage.
Given that Google runs ReCapcha - which is almost certainly the world's most widely used browser fingerprinting system - they have ample experience with is-it-a-real-browser cat and mouse games.
In this case, Telegram may just display the captcha and let the user solve it.
That's likely also why Telegram doesn't proxy every translation request over their server: so that it is users individually requesting small number of translations, from their phone, getting around quota of free APIs "naturally".
Honestly those are so easy to circumvent. I built translation functionality into an IRC bot in the past and it was so easy to avoid recaptcha if you did stuff in a clean way.
I think they will look for a legal solution. I doubt they will change the API; my guess it it exists to support some other services google sells -- and adding heuristics to detect freeloaders may impact legitimate customers. Nope, they will get a cease and desist letter, and if they happen to use any other GCP services, or rely on Google to do business, this will likely be pretty persuasive.
Ah, you're suggesting Google uses its position on the Play Store to fend of abusers of an unrelated service, that happens to be owned by Google as well? I don't think that will work in Google's favor when they try to defend any anti-trust cases slung their way, or being active right now.
Google has many other subtle ways that garner less attention to ward of any misuses. Subtly downrank in the search index, throttle any traffic heading their way, suggest other applications in the play store on top, accidentelly flag Telegram as a "risky" platform, etc.
I see what you mean but surely there must be precedents of apps being pulled from the Play Store because they abused 3rd party APIs, be it Google's or somebody else's?
I mean why even bother obfuscating the URL otherwise, surely the expected that it could be caught in the review process.
I don't think you're serious as most of these ideas aren't that novel / intricate, but in general I'd rather publicise these ideas so they're general knowledge and everything would be aware of the absurd amount of power Google levies.
I'm pretty sure Telegram is outside of US jurisdiction and can trivially respond with a middle finger.
Google's only real option here is to either engage in cat & mouse trying to block this usage or threaten a Play Store removal which comes with its own drawbacks (Telegram has significant marketshare).
Ummm, the UK has legal system too. I sure would love it if we could "trivially respond with a middle finger" to legal issues. This isn't Russia we are talking about here.
Off topic, but this is a code smell for me: [(int) Math.round(Math.random() * (userAgents.length - 1))]); This leads to a lower probability of selecting the 0th and the last items in the array.
floor is unnecessary. The cast does it for you. The problem is that a random double is not going to be evenly distributed into length equal parts unless length is a power of 2.
Quite a lot of libraries exist to do this. But doing this in an app with a large user base looks offensive. Solution would be for some decent open source translation APIs to appear.
Any ideas which ones are the best? I would quite like to use a good open-source api, am happy to pay for my own compute. (I've just used googles before because it was handy. But I wouldn't use it the way telegram have done.)
Huggingface offers a few models that are pre-trained [0], OpenNMT is a good framework [1] as is MarianMT [2]. Many of the best MarianMT models have been ported to HuggingFace.
If you don't want to self host, huggingface offers these APIs at a cost.
Is there any proof in this post that Telegram actually circumvent Google Translate API ? Because it is also possible that Google told Telegram to use the method explained in the article.
Why would they concatenate the API url string then? The only reason I see is to avoid detection when their app is scanned when submitted to the PlayStore.
Yup. And those guys will get bonus points for actively evading detection. IANAL, but its one thing to use an open api and claim ignorance, and quite another to intentionally defeat access restrictions/rate limits. Stupid thing to do.
They are bound to get in trouble with Google for this, but they can’t easily pull the feature. They can’t just be like „oh you’ve had translate for two weeks now, but now we can’t pay for it, so it’s gone.“
What is the long term thinking behind this? Or is this just developers and management not communicating?
Telegram knows how to play the PR game. It’s going to be a bit more elegant than that, with a “we will rise from the ashes” tone. They know what they’re doing, and I doubt they had any intention of getting away with this.
Yeah I'm a bit shocked honestly that it made it into an application as widely used as Telegram. It's bound to be detected eventually and the feature will suddenly break. Such a strange software engineering decision.
They can't even plausibly pretend that they didn't know and it's all a big misunderstanding given the lengths they went to obfuscate it in the code.
This seems like one of those features that leadership demanded with a “just make it happen” decree and no budget for API calls.
Then some developers facing a deadline cobbled together something that “just made it happen” so they could kick the can down the road with something that worked, ideally long enough to collect their bonuses and find a new job so it becomes someone else’s problem.
Or maybe Telegram the company just likes to abuse other people’s things and see how long they can get away with bad behavior. Who knows.
Telegram problem is that unlike WhatsApp they don't have a billion dollar corporation backing them. I wouldn't be surprised if they were bleeding money.
> They are bound to get in trouble with Google for this
From first look, I don't think they are. Telegram gets a new feature, Google gets more data to mine. It's a win-win. I just hope they'll be clear with their users about sending data to Google.
This reminds me of my own usage of Google Translate's speech synthesis API in a chat bot way back. It was as easy as sending a GET request to https://translate.google.com/translate_tts. People loved it.
Of course, my use case was neither commercial nor large scale.
Not everyone can afford to use these APIs or are able to get permission to use the external APIs. The internal APIs are very good for people who don't mind taking the risk of using a potentially unstable API.
Even if they would have TOS for translate, pretty sure that’s unenforceable. Not unless hiding that page behind a paywall, or requiring a google account. Merely visiting a publicly available web page doesn’t create contractual relationship between end user and web server owner.
I've spent some time looking into this and found very little. The closest I could get is that if you are a Google Cloud customer, it forbids you from reverse engineering other Google APIs that are not documented as supported, which would probably include this use case. No idea if they're a Google Cloud customer though.
> No idea if they're a Google Cloud customer though.
Even if they are, I have doubts that’s enforceable either. One doesn’t need to reverse engineer an API to consume that API, and it’s hard to find out who did the reverse engineering. The reverse engineering might be accomplished by someone else who’s not a Google’s Cloud customer, like an unrelated person answering a question on stackoverflow.com.
Personally I have found DeepL to be more accurate in the languages they support than Google Translate. They used to support much less, but for the languages it did, it was pretty great. It supports way more now though!
I am unsure if the support for those languages are better vs. Google Translate, but the small set of languages it used to support a year ago or something is definitely better. I remember French and Spanish being way better. DeepL's Polish is not that great, that I can say for certain! Not sure if better than Google Translate's though.
Not really. Try to have discussions using DeepL and Google Translate. Ask native speakers which one was more accurate and whatnot.
I do not know French, but DeepL allowed me to speak to someone using the language, and apparently at some point some people thought I was a native speaker!
Contrary to Turkish, Estonian, Latvian and Hungarian are all EU official languages. That means that all EU legislation is legally required to be translated to these languages and it is readily available in all these languages for free to train the AI model for translating to those languages.
Indeed, also see linguee.com which uses (mainly) official EU documents to feed an enormous amount of word and phrase translations. Beautiful site that I've been using a long time - and only learn this minute that they are in fact also owned by deepL.
If it's public and available and Telegram didn't sign a contract or EULA which forbids using this endpoint, it's legitimate.
The cons is that Google can break the endpoint at any time.
That said, I wrote a similar hack for a Wordpress plugin 10 years ago which automatically generated articles in multiple languages (think SEO farming).
I just looked at what their frontend app was calling and used it.
I run a business which needs to run programmatically searches on different search engines and using official APIs is out of the question.
I didn't even check what their API offering is.
The only search engine that seriously try to prevents scraper is Yandex which asks for a lot of captchas but I've built a system to allow humans to just process captchas and give back to the code the result.
If you use a service by someone else then you implicitly agree to the terms. I agree that browsewrap style ToS are bullshit where merely landing on a website supposedly binds you to whatever terms they wrote, but if you actually go out of your way to integrate someone's API into your code you are definitely agreeing to the terms!
How so? A contract requires a "meeting of the minds". If one of the minds doesn't agree to (or even see) the contract, then how could it be enforceable?
This might not be so clear. If you think about google crawling your site without you knowing it - technically its same unprotected request/response as calling API. If i had TOS that prohibits crawling of the site (instead of robots.txt) does it mean that google is breaking my TOS? Seems like the solution is clearly for me to protect my site/api - not other way around.
Now if I add to all my backend response headers ”x-terms: you must agree usage terms: https://xn--blah-jb7a would it make it explicit enough, also for the ignorant engineer with “But this API just works” attitude ? It should be a industry standard really.
I see few possible issues:
1) If this is some kind of hack to reduce the cost this means that Google can pull the plug to this at any given time
2) How many users are aware that this means that the content is sent to Google? Yes, there is a warning on the screen where you turn one the option but will the users see it?
This is an interesting data point for folks designing demo pages. The API was discovered by playing with the demo page [1] according to a link from a link in the article
292 comments
[ 0.27 ms ] story [ 301 ms ] threadAnd again, i wonder how a tiny team can push such great and useful features into such amazing UI. And then I'm looking at other alternatives, from naked WhatsApp over laggy wechat to horrible UX in signal.
What's the reason for telegram amazing performance and features?
Wouldn't a better question be "what's the reason for other apps having crappy performance and subpar features"?
At this point, I just think the answer is top notch developers and designers with an impressive alignment about what the product should be.
I know no other free application with such an amazing usability. There must be some shitload of money behind them, for sure.
It's still impressive, though. It's not as if Facebook's chat works this well and that isn't (wasn't?) encrypted either and they've got an even bigger pile of cash. It's also a lot older though, just look at the evolution from IRC to MSN to Facebook to Telegram: each step it gets better. Maybe that's how that difference can be explained? Anyway, between normal messengers like Signal/Whatsapp/Wire/etc. and Telegram, the main difference is not caring about privacy. (To be clear, I don't mean that whatsapp is a privacy-conscious messenger as they will collect what they want behind the scenes, but on the client side they have to care about keeping the server "untrusted" and that will slow everything down a lot.)
What I think you're underappreciating is that Telegram was built on the premise of privacy and they've neglected that from day one. It was better than the status quo on day one (at release), but have fallen further and further behind ever since.
Telegram was launched when Whatsapp sent messages without transport encryption over port 443 (this was fun on public wifi!) and got big when Whatsapp was bought by Facebook because Telegram was independent and had proper end to end encryption (with air quotes around "proper" if you like; it wasn't OTR-grade but at least it was better than the status quo). Since then, nothing happened whatsoever on the privacy front. Even the legal compliance is a complete joke (gdpr data exports only work in a few clients and not very well at that), but more importantly to me, they need gdpr data exports because the servers still know everything. Whatsapp moved on, Signal got a lot more mature, Wire has also come onto the playing field, and Telegram has done nothing since before Snowden told us to turn on https basically.
Telegram is still at "open a secret chat" with worst UX
But definitely
I switched to Matrix though
There has been a lot of work on it but the smart thing to do would have been to copy Wire or Signal and build on an existing thing if you don't have the manpower to start from scratch and want to be a mainstream alternative.
I'd rather recommend Threema (best UX, not so great in features or encryption tech), Signal (network effect, best privacy, reasonable UX), or Wire (like Signal but slightly better desktop experience).
I started pulling people to it only a few months ago when I thought it was good enough (includes almost no encryption issues happening in all day usage)
My mom uses it as well. It works fine. I had to install the app though, like all the other apps.
The reasons why I found Matrix interesting, were:
- Telegram like syncing messages across all devices while adding a new device is as simple as setting up WhatsApp Web
- It's a standard for interoperability - I got sick of telling others what to use and being told so by the network effect
Copying Wire or Signal just wouldn't work for technical reasons. I also wouldn't want a copy.
I got all of your recommendations over time (including buying Threema with all my friends), but they just haven't been good enough for us and me. In practice, no one cares about Electron. It seldomly VScode or Teams are used at work
That's hyperbole and I think you know it. It's not like the messages are sent unencrypted for anyone to sniff with minimal effort.
https://telegram.org/faq#q-so-how-do-you-encrypt-data
Meanwhile I'm just here thinking, what's the matter if you're messager is super duper safe if your device OS running them is plagged with backdoors?
> what [if your] mess[en]ger is super duper safe if your device OS running them is plag[u]ed with backdoors
This is more relevant. Bugdoors more than backdoors, or perhaps just stupid bugs and enough budget to find them, but yeah basically that's how messages are decrypted these days (e.g. NSO group).
For example, people have been using Russian GOST algorithms as a hedge against the USA stuff, but it's falling out of style because it just hasn't proven necessary in the decades since the AES and SHA families came into existence. Any bugs we found, for example in SHA-1, affected everyone equally and did not create a backdoor as with Dual-EC (in which flaws were identified before release and which went unaddressed, very much unlike other common algorithms).
Also note that it's not impossible. Wire and Signal have come a very long way already, it's just the front-end (UX) that they're lacking on really. Telegram has features like a video editor and user-filtered search, but those are all client-side things and have nothing to do with e2e encryption. All the things that do (sending messages, emoji reactions, group chats, group video calls, etc.) are already implemented by both apps.
I think you might be missing the point of the article, which is that they’re misusing Google APIs to avoid paying for the proper way of doing things.
This feature will break as soon as Google throws up a captcha, because these endpoints aren’t intended to be used programmatically.
> And again, i wonder how a tiny team can push such great and useful features into such amazing UI.
The translate functionality comes from Google. They’re just sending messages to Google and getting a result back, at least until Google detects the API misuse and breaks the response.
Although at the same time half my friends use it at our (critical) communication platform. So it's not in our best interests.
In general I am pro Telegram, as I think any democracy needs to have censorship-free communication for whistleblowers etc. and to prevent attacks against democracy itself. Even if this means we have to live with stupid/illegal opinions being expressed too.
I'm not sure about this.
I bet Google is happy to collect the text data of up to 500 million users with zero restrictions from Telegram's end on how the data is used. I'm not a lawyer, but my hunch is that Google's data privacy policy applies to the official, premium service: https://cloud.google.com/terms/data-processing-terms
Google might make the determination that they'll get more value from allowing Telegram to abuse the unofficial API. However, they might face some angry customers who are paying a premium to use the official API now that this loophole has been published.
On the other hand, I find it hard to believe that Telegram would risk a Play Store ToS violation, given how many tens of millions of users use the app.
Meanwhile, indie developers with smaller user base are subject to unappealable automated decisions.
https://imgur.com/a/7UIFLxT
It allows Telegram users to hide in plain-sight, within the noise of other Google Translate web users.
I'm pretty sure that using the official pre-built java SDK, as suggested by the author, would allow Google to cluster the content of Telegram users (since app-specific id/token should be sent).
Other than that, a great read and kudos to the author for shedding light on it.
Edit: typo.
Yes, Telegram fakes the user-agent, but the rest of the request still looks very different from a request an actual browser would do. (No referrer, missing headers, different connection pooling behaviour, possibly different TLS and HTTP2 behaviour, etc).
So if Google is doing any detection for browser vs non-browser requests, those requests should show up as suspicious.
On the contrary - it's the most stupid thing to do. The only result will be their users wondering soon why this function is broken.
How does that make this okay? Nobody is entitled to get a company’s services for free just because you think their price is too high or their front ends aren’t built to your liking.
Google specifically does not publish their API for free consumption by other companies, yet that’s what’s happening here anyway. The company is also using specific tricks to circumvent detection of the behavior.
In your analogy, this would be like a crawler ignoring robots.txt and then scraping the content for their own website with zero attribution to the source, which is nothing like Google indexing your site with full attribution and driving traffic to it for you.
Regardless, “turnabout is fair play” is unequivocally not a legally or even ethically acceptable standard, so that argument wouldn’t actually hold up anywhere anyway.
I did mention rich snippets.
Google ignores the noindex directive in robots.txt now. You're supposed to put it in your HTTP response headers or HTML meta tags...
https://developers.google.com/search/blog/2019/07/a-note-on-...
I’d love to see and give money to a project to create and maintain easy to use and stable “adversarial interoperability” APIs for as many services and products as possible.
Perhaps companies and projects would not often use these directly because of the risks (hopefully some would, though!) but individuals could drop the library or the URL to a server hosting it into their apps to gain extra features.
If standardised, whole open source apps could be built around them that allow querying and analysis of data from services and aggregating and automating using the services including optimising prices, taking advantage of offers, and using undocumented APIs to the users advantage.
Maybe something architected and incentivised like https://thegraph.com/ for adversarial intercom and undocumented APIs. Building as a network of nodes and funding with crypto would make it harder to attack and take down.
[0] https://www.eff.org/deeplinks/2019/10/adversarial-interopera...
"pip3 install woob" succeeds but e.g. "woob config-qt" returns "not a woob command". Are the -qt versions not available via pip? I have python3-pyqt5 installed.
pip install woob-qt
https://gitlab.com/woob/woob-qt/-/blob/master/INSTALL
If you provide a solution to someone’s problem, it will be used all over the place.
The biggest companies won’t use these things, but plenty of smaller companies and individual programmers without oversight would use them without a second thought, at least until they’re caught.
Telegram is a relatively large business and here they are abusing an API exactly like you suggest.
Along those lines: maybe we could use a middleware pattern for APIs, frameworks, etc where the interface/package would be built as a layer above two or more services.
That way the developer could switch between them at any time, or even failover automatically.
So for example, rather than going onto GitHub to download an SDK for something like Mailgun, you'd download a middleware framework built on Mailgun and Sendgrid.
This pattern could be used to identify vulnerabilities in software at the conceptual level, by helping developers to avoid marrying their code to individual providers like AWS. Some mission critical software could even be certified as using all adversarial interoperability frameworks.
It could even help third party services pull themselves up by their bootstraps, if they get added to one of these middlewares. An instant user base without having to rely on marketing or word of mouth.
And could be used to identify monopolies when there's no middleware for a service.
> Too many moving pieces. Too much work.
It still boggles my mind that this is the way we do things. "Patch fast!" Ok cool. That's going to keep working out forever.
I would love to have applications that are more tools than products and weave together these APIs and middlewares, both in querying and visualising, combining, enriching, analysing, filtering etc. data and also taking the output and actioning it.
With the amount of data and services that are now available online we should have superhuman capabilities but the productisation of the internet has left us stuck in company run silos fighting “user journeys”, undocumented APIs and EULAs.
teller.io does similar for banks.
I would love to see a coordinated effort along the same lines for things with non public APIs. It is however a huge ask as internal APIs are unstable and constantly changing/actively working against things like this which is a huge amount of work to keep up with.
That was me. The comment was rapidly amassing upvotes, which made me feel that not only was the comment preaching to the choir but it had the potential to derail the discussion around the actual issue of Telegram's use of Google's Translate APIs.
Well, the plain text, not the IP, but that should be implicit with how web services work.
Commercial use of those APIs is common, despite translate being pretty expensive. Also, GCP current leadership is so hell bent on nickel-and-diming their customers, and their compensation packages are so dependent on value share growth, that they simply can't afford anyone openly violating their pricing models. Especially a popular app. My guess is this will be down within the first week of January.
That's likely also why Telegram doesn't proxy every translation request over their server: so that it is users individually requesting small number of translations, from their phone, getting around quota of free APIs "naturally".
[0] https://dcurt.is/apple-card-can-disable-your-icloud-account
I mean why even bother obfuscating the URL otherwise, surely the expected that it could be caught in the review process.
A cease-and-desist letter from Google legal tends to work pretty well as a technique in these cases.
Google's only real option here is to either engage in cat & mouse trying to block this usage or threaten a Play Store removal which comes with its own drawbacks (Telegram has significant marketshare).
It is now blocked, always responds 403, maybe tweaking some request parameters can make it work again.
Edit: if you want to try it out the parameters I used were:
- container: focus (there are other values I cannot find anymore)
- url: urlencoded URL of the image to be resized
- resize_w: width in px
- resize_h: height in px
https://jsfiddle.net/depxb0gn/2/
From what I remember, there was some minor rate-limiting that I hit once or twice while using it, which complicated things a little.
> The java.lang.Math.random() method returns a pseudorandom double type number greater than or equal to 0.0 and less than 1.0.
Huggingface offers a few models that are pre-trained [0], OpenNMT is a good framework [1] as is MarianMT [2]. Many of the best MarianMT models have been ported to HuggingFace.
If you don't want to self host, huggingface offers these APIs at a cost.
[0]: https://huggingface.co/models?pipeline_tag=translation [1]: https://opennmt.net/ [2]: https://marian-nmt.github.io/
They are bound to get in trouble with Google for this, but they can’t easily pull the feature. They can’t just be like „oh you’ve had translate for two weeks now, but now we can’t pay for it, so it’s gone.“
What is the long term thinking behind this? Or is this just developers and management not communicating?
…but with more elegant phrasing.
blah blah blah on 31st of February 1970 Google unanimously decided to terminate our access to their Translate API blah blah blah blah
They can't even plausibly pretend that they didn't know and it's all a big misunderstanding given the lengths they went to obfuscate it in the code.
Then some developers facing a deadline cobbled together something that “just made it happen” so they could kick the can down the road with something that worked, ideally long enough to collect their bonuses and find a new job so it becomes someone else’s problem.
Or maybe Telegram the company just likes to abuse other people’s things and see how long they can get away with bad behavior. Who knows.
From first look, I don't think they are. Telegram gets a new feature, Google gets more data to mine. It's a win-win. I just hope they'll be clear with their users about sending data to Google.
Of course, my use case was neither commercial nor large scale.
If they don't want to pay, they should be using a free open source alternative like https://github.com/LibreTranslate/LibreTranslate
I'd recommend pizza and beer at tech events instead (albeit the nutritional content of your diet could be more important than free food).
Even if they would have TOS for translate, pretty sure that’s unenforceable. Not unless hiding that page behind a paywall, or requiring a google account. Merely visiting a publicly available web page doesn’t create contractual relationship between end user and web server owner.
Even if they are, I have doubts that’s enforceable either. One doesn’t need to reverse engineer an API to consume that API, and it’s hard to find out who did the reverse engineering. The reverse engineering might be accomplished by someone else who’s not a Google’s Cloud customer, like an unrelated person answering a question on stackoverflow.com.
Yet tiny European languages like Latvian are supported, as are very difficult translation targets such as Estonian and Hungarian.
My hopes are dashed every time they add another tiny European language and Turkish remains off the table. :-(
> do you have some metrics you can share?
Not really. Try to have discussions using DeepL and Google Translate. Ask native speakers which one was more accurate and whatnot.
I do not know French, but DeepL allowed me to speak to someone using the language, and apparently at some point some people thought I was a native speaker!
I’ve been aware of this loophole for years, it’s nothing new and is widely known.
I just chose to build our business using legitimate above-board APIs and services, as should Telegram.
The cons is that Google can break the endpoint at any time.
That said, I wrote a similar hack for a Wordpress plugin 10 years ago which automatically generated articles in multiple languages (think SEO farming).
I just looked at what their frontend app was calling and used it.
I run a business which needs to run programmatically searches on different search engines and using official APIs is out of the question. I didn't even check what their API offering is.
The only search engine that seriously try to prevents scraper is Yandex which asks for a lot of captchas but I've built a system to allow humans to just process captchas and give back to the code the result.
Google will most likely just fingerprint these calls and block them serverside, I'm guessing.
[1] https://weblog.west-wind.com/posts/2011/aug/06/translating-w...