Love it. A few q's: all traffic on your servers is plain open? "The phone recieves this stream at 3 messages per second", how about upstream? How much data (kb) fits on each message?
Or you could text them directly. This scenario is only useful in the case where you have SMS but no internet, and your intended recipient has internet but no SMS.
EDIT: Also, as someone pointed out above, Twitter already has an SMS interface.
They're just a terrible example cause they're the one service you might already have access to that way. Twitter's only got 232 million users so anything that hinges on them is immediately not as useful as something that makes more of the internet - fb, email etc - available to more people, especially from someone else's device.
It's becoming common for Twitter to used for updates and coordination in disaster zones. And I guess if I wanted to let my friends know I was safe in such a situation, and didn't want to text them all, Facebook would be a good choice.
It appears the HTML of the requested page is returned to the requesting phone via SMS, which could be many many messages if you are loading say, a wikipedia page.
I know that in the past in US and Canada carriers billed incoming SMS, but I thought that alongside with the new age of messaging apps like WhatsApp and iMessage they dropped this nonsense.
In Europe even when in Roaming there is no cost for incoming SMS.
On most plans offered in the US nowadays texting is included. All of the T-Mobile, AT&T, Verizon, Straight Talk, Republic, etc plans have unlimited texting included.
Right, but with the GSM 7-bit alphabet they could easily expand their index table to 127 entries rather than Base64's 64, netting them much greater efficiency.
You can send anything in the GSM 7-bit alphabet, which has 128 characters rather than Base64's 64-entry index table.
They could gain a lot of efficiency by using a 127-entry index table (or 128 if the ESC character doesn't need to be avoided - I don't know how the Android SMS APIs handle it) rather than a 64-entry one.
Yes, Base64 encoding will expand a payload by 33%. However, SMS limits the character set you can send easily, with the most common mode of transit being the 7 bit SMS character set. With the SMS character set having an escape character that can get in the way of doing a direct encoding, I imagine they went with Base64 because there were already libraries for it, minimizing the work they had to do at this stage.
Interesting approach. I wonder if this HTTP-specific approach is better or worse than coming up with some form of generalized TCP-over-SMS stack (with accompanying server-side router software, of course).
Reminds me, though, of Georgia Weidman's Smartphone Pentest Framework[1] which provides tools to hack a target smartphone and from there create a data connection via SMS via which you can then access a firewalled corporate network using SMS over the cell network as your tunnel in. I've seen it demoed, and it's frankly incredible.
In germany unlimited sms has only been a thing since ip based messengers like WhatsApp took over messaging and nobody is using SMS anymore, before that SMS were quite expensive after you hit the budget of your contract (typically about 50-100 SMS). Now, unlimited SMS is the norm and things like this browser get interesting. But even if it would be a great working alternative, it will only work until carriers adapt. I really want unlimited data...currently i get 750MB with a contract that costs EUR 50.
Would this allow users to circumvent internet censorship? It seems like that as long as the main server was outside of the censored zone, SMS traffic would be able to get information in under the radar.
Could you achieve a more high throughput version over voice? I wonder, after compression what the data rate ends up being. I'm reading gzip can get 2:1-4:1 with text. So about a kb/s?
that's what error correcting codes are for :)
If you design things to play nicely with the compression algorithm that's used by trying to stay within normal human vocal range and stuff, might be okay.
Before this fancy GPRS thing, we used to have something called CSD (circuit-switched data), where your phone could use a raw GSM voice circuit to send data instead of digital audio.
I think we lost that feature in the transition to 3G.
True, but if you were somewhere where data was unavailable or you didn't want to use a local data connection, it might be a nice way to force a map app to update to your current location, get an email, send your location to somebody, or answer a chat message.
I may be wrong but aren't we going backwards here...
data over voice=dialup right? 56kb/s ideal? Still possibly useful but this just reminds me of the wep browser days on my old nokia
56kbps over landlines. That works because the landline network, when it is trunked, is compressed into 56kbps or 64kbps channels using methods the 56kbps modems were explicitly designed to take maximal advantage of.
Over a cell phone connection you will get much less. GSM data modems gets to bypass the voice codec, and are still limited to 9600 bps. An acoustic modem over a GSM connection will get less than that, if it will connect at all: A full data rate GSM code is compressing the audio down to about 12.2kbps using codecs geared towards reproducing voice as well as possible, which is going to be brutal to a regular modem and far more wasteful than GSM data.
It's unfortunately pretty slow. I mean, when you factor in the overhead for text messages, it adds up. We haven't tested it enough but it definitely won't match 4g. We also are planning to switch to a better compression algorithm.
I'm guesstimating you'll find an upper bound, assuming "friendly" telcos that don't start rate limiting, or dropping SMSs, at no more than about 6kbps. Quite likely less.
GSM data (which is normally possible anywhere where you can get a GSM voice connection) is 9600bps and uses the full, raw GSM data/voice channel. You'd not be able to exceed the channel bandwidth. Then you have the SMS overhead. And you're unlikely to manage to max out the channel.
I had a browser (WAP) on my Nokia 7110 that supported browsing via a SMS bearer; 1999. I tried it once and got billed up the wazoo, SMS was expensive back in those days.
There was also WIG. I was writing WAP sites in '99, incurring pretty large bills while developing and testing. The company had a very close relationship with the mobile company so fortunately we just got them written off.
They're using Twilio. Each SMS they send costs them $0.0075.
However, simply stripping the junk out of web pages for mobile delivery has real promise. Just put all third-party content on "load on user request only", like mail attachments.
That would be pretty cool for a normal web browser too.
EDIT: I'm also part of the team, that comment looks really strange out of context.
We don't have specific monetization plans yet, but we're probably going to do some sort of donation system, or maybe ads (through wifi). No idea though, it is pretty expensive.
Wow, that gets expensive very quickly especially at 3 messages/sec.
Assuming the average user gets only one burst of three messages per query, uses it only twice a day for 200 days a year and the service gets only 10,000 users signed up, the costs just to send the texts for the next year alone would be $90,000.
Seems like it will be hard to scale but I agree that this is a really cool way to get connectivity! Best of luck.
I don't know, next to paying $35/mo for a cheapo cell with unlimited texts and a USB gateway to SMS, you're saying that for moderate users they would need to pay about $9/year to get their internet access on top of that.
Seems reasonable to think that you might pay $5-9/mo on a budget for a limited access to internet such as this; now it sounds like potentially economically viable to even have an employee or two after paying for all the texts.
The price goes down with volume. They should also look at alternative providers. Nexmo and Plivo both offer free inbound SMS and have APIs similar to Twilio's.
Wow. I knew it was cheap - that makes sense in light of what you're saying. Though there is a minor sting: "mobile networks charge each other interconnect fees of at least US$0.04 when connecting between different phone networks" [1]
To the operator, not the provider, Twilio in this case. Twilio doesn't send them for free. Operators charge for access to them and you have to pay, if you want to send SMS.
They also aren't entirely free to the operators themselves - once you start counting them and charging for them, there are administrative costs; then there are costs related to interconnections with other operators, and costs related to maintaining their own SMSCs (SMS centers). The only thing in this whole mechanism that the operator gets for free (actually not free since they have to pay for the network and base stations) is the delivery channel which is not insignificant but it is also just one part of many.
SMS isn't going to get good bandwidth, given the path it uses over the mobile network.
But I've thought about building HTTP-over-WhatsApp. Mobile providers here offer prepaid contracts where you don't pay for WhatsApp access, regardless of your balance or traffic remaining.
I don't see this could EVER be economical. Even wholesale SMS (not Twilio/Plivo/Nexmo/Etc) can still cost $0.001 / SMS. There is simply no way they can operate the proxy/gateway profitably.
Unlimited SMS is common in the US too: But that's not really "unlimited" like you'd want to run a service of this scale. 3 messages/second is way too small. You could only host a single client per relay that way. The right way to do this is to not involve an SMS plan, and connect to a proper SMS gateway, which won't be "unlimited" at all.
If you (on the server side) connect via a mobile modem, yes, and probably violate the TOS. Otherwise, typical prices for bulk SMS in the UK is in the 2p-4p range per message (with higher risk of delays and failure to deliver towards the lower end of that scale). I'm sure it's possible to get below that if you have really huge volumes, but those prices are fairly typical in volumes up to millions of messages per month.
I wonder if they could do some intelligent parsing of the content and just return a plain text version of what the user requests. It wouldn't work for everything, but for things where there are articles with a defined "content" it shouldn't be too hard.
In fact, isn't that what the "read it later" type apps do? I'll bet there is a free js library that already does the detection or you.
Is it a web browser? If so, then I don't see the point. Why not simply write some kind of driver that does IP over SMS; then any app that uses the net could connect that way.
(And if it isn't a web browser, then why "browser"?)
Well, I'm sure this was a nice achievement, and no offense intended, but that doesn't make sense. You were limited to 36 hours, and therefore, instead of simply writing a driver, you wrote a driver and and entire web browser sitting on top of it?
But we might take it as a negative comment on the modularity or ease of configurability of the environment supported by Android (or, more generally, by Linux).
Surely they've reused a browser component rather than writing one from scratch. And they do more on the server than just pipe the full HTTP responses back, they strip CSS, images, JS etc. and minify the HTML. I think it's pretty cool!
Coming up with new ways of compressing data to travel down legacy or fundamentally anti-functional data channels (i.e. low bandwidth, low quality of data, high latency, preprocessing that destroys the majority of the experiencial data) is always a fun hack, but is never a marketable strategy for a few reasons.
- It bets on technology going backwards rather than forwards (i.e. will have cheap android phones with cheap data plans before you can scale it)
- It doesn't provide much more than novelty self promotion. "Hey i made a webpage deliver via SMS"
- Its usually more expensive and time-consuming to produce what almost always becomes an inherently inferior product.
Now, let's take those 3 points and consider a 3rd world country with interwebz-censorship or very poor infrastructure and no forseeable future where it'll improve to anywhere near what most first-world countries have. Cellphone coverage is the easiest thing to setup in those kinds of environments.
Do you have any examples of such countries where at the same time SMS will still work and be cheap enough to be viable?
Even most countries in sub-Saharan Africa has at least GPRS - most of them have 3G. Some of them are deploying LTE. Certainly there will be areas where for some time still there will still be limited data access, but the question is how many of those areas actually has functioning cell coverage.
THANK YOU for this very lucid post!!! People who have access to these services almost certainly have access to a data plan and would view it as a novelty rather than a means of communication. I learned this the hard way by having done a very similar project a year ago, but then basically lost interest as soon as I got a smartphone.
It's open-sores and free to set up and use (if you have a text plan) It would be pretty easy to turn it into a sms terminal which might be negligibly more useful (using a PTY with the kbd is easier than using an X Server). I guess the OP's project and mine would be perfectly equivalent if anyone's written a text-only, line-based browser, but I stopped short of this. Maybe someone else will make one? (:
My biggest takeaway from the whole project was how easy it is to fall into the trap of doing something just to be able to pat yourself on the back. Real altruism isn't this cheap.
164 comments
[ 2.7 ms ] story [ 201 ms ] threadEDIT: Also, as someone pointed out above, Twitter already has an SMS interface.
http://en.wikipedia.org/wiki/GSM_03.38
They could gain a lot of efficiency by using a 127-entry index table (or 128 if the ESC character doesn't need to be avoided - I don't know how the Android SMS APIs handle it) rather than a 64-entry one.
http://tools.ietf.org/html/rfc1149
Reminds me, though, of Georgia Weidman's Smartphone Pentest Framework[1] which provides tools to hack a target smartphone and from there create a data connection via SMS via which you can then access a firewalled corporate network using SMS over the cell network as your tunnel in. I've seen it demoed, and it's frankly incredible.
[1] https://github.com/georgiaw/Smartphone-Pentest-Framework
The issue is that it quickly ends up costing about the same amount as a data connection would in the first place.
Unfortunately, voice calling uses lossy compression, so your data rate would be somewhat limited as you struggle to be 'heard' over that.
That by itself could get you up to about 0.5kbps for text with decent compression, with no other effort.
I think we lost that feature in the transition to 3G.
Yeap, except voice is now carried over a digital channel, so it'd be digital-over-voice-over-digital :)
Over a cell phone connection you will get much less. GSM data modems gets to bypass the voice codec, and are still limited to 9600 bps. An acoustic modem over a GSM connection will get less than that, if it will connect at all: A full data rate GSM code is compressing the audio down to about 12.2kbps using codecs geared towards reproducing voice as well as possible, which is going to be brutal to a regular modem and far more wasteful than GSM data.
I'm guesstimating you'll find an upper bound, assuming "friendly" telcos that don't start rate limiting, or dropping SMSs, at no more than about 6kbps. Quite likely less.
GSM data (which is normally possible anywhere where you can get a GSM voice connection) is 9600bps and uses the full, raw GSM data/voice channel. You'd not be able to exceed the channel bandwidth. Then you have the SMS overhead. And you're unlikely to manage to max out the channel.
Hell, this might even be faster than loading stuff over WiFi.
However, simply stripping the junk out of web pages for mobile delivery has real promise. Just put all third-party content on "load on user request only", like mail attachments.
That would be pretty cool for a normal web browser too.
EDIT: I'm also part of the team, that comment looks really strange out of context.
We don't have specific monetization plans yet, but we're probably going to do some sort of donation system, or maybe ads (through wifi). No idea though, it is pretty expensive.
Assuming the average user gets only one burst of three messages per query, uses it only twice a day for 200 days a year and the service gets only 10,000 users signed up, the costs just to send the texts for the next year alone would be $90,000.
Seems like it will be hard to scale but I agree that this is a really cool way to get connectivity! Best of luck.
Seems reasonable to think that you might pay $5-9/mo on a budget for a limited access to internet such as this; now it sounds like potentially economically viable to even have an employee or two after paying for all the texts.
If they can just get those SMS costs down it would be much easier to scale.
[1] https://en.wikipedia.org/wiki/Short_Message_Service
They also aren't entirely free to the operators themselves - once you start counting them and charging for them, there are administrative costs; then there are costs related to interconnections with other operators, and costs related to maintaining their own SMSCs (SMS centers). The only thing in this whole mechanism that the operator gets for free (actually not free since they have to pay for the network and base stations) is the delivery channel which is not insignificant but it is also just one part of many.
Still, major admiration from me for getting that to work!
But I've thought about building HTTP-over-WhatsApp. Mobile providers here offer prepaid contracts where you don't pay for WhatsApp access, regardless of your balance or traffic remaining.
US pricing for texts seems much more extortionate, for some reason..
In fact, isn't that what the "read it later" type apps do? I'll bet there is a free js library that already does the detection or you.
Is it a web browser? If so, then I don't see the point. Why not simply write some kind of driver that does IP over SMS; then any app that uses the net could connect that way.
(And if it isn't a web browser, then why "browser"?)
But we might take it as a negative comment on the modularity or ease of configurability of the environment supported by Android (or, more generally, by Linux).
- It bets on technology going backwards rather than forwards (i.e. will have cheap android phones with cheap data plans before you can scale it)
- It doesn't provide much more than novelty self promotion. "Hey i made a webpage deliver via SMS"
- Its usually more expensive and time-consuming to produce what almost always becomes an inherently inferior product.
Even most countries in sub-Saharan Africa has at least GPRS - most of them have 3G. Some of them are deploying LTE. Certainly there will be areas where for some time still there will still be limited data access, but the question is how many of those areas actually has functioning cell coverage.
Here is my self-promotion:
* http://nerdhow.net/uzblsms-a-way-to-browse-the-web-via-text-...
* and of course I had it on Hacker News: https://news.ycombinator.com/item?id=5835365
It's open-sores and free to set up and use (if you have a text plan) It would be pretty easy to turn it into a sms terminal which might be negligibly more useful (using a PTY with the kbd is easier than using an X Server). I guess the OP's project and mine would be perfectly equivalent if anyone's written a text-only, line-based browser, but I stopped short of this. Maybe someone else will make one? (:
My biggest takeaway from the whole project was how easy it is to fall into the trap of doing something just to be able to pat yourself on the back. Real altruism isn't this cheap.
By far my favorite unintentional turn of phrase I have seen today :)
Would be an optimal way of encoding the return.