Although not a dedicated hardware device, Kiwix makes it very easy to browse Wikipedia offline on both desktop and mobile devices: https://kiwix.org/en/applications/
I feel cool having a "doomsday" phone with offline maps, wikipedia and a few other references in kiwix, music, a few weeks worth of podcasts... and curious to know what else one should have.
Now if apple could just shoot me a few kb of news/weather updates by pointing it at one of their satellites, we'd be really cooking.
(Other than a solar panel and I got a cigarette lighter adapter with clips at the end so I can charge off a car battery without turning on all the accessories)
I wonder why there's no P2P crowdsourced news and weather network. Even stuff like the Citizen app gets ignored mostly.
I think the most useful offline reference is various rope knots. Mostly because I'm not a prepper and they're one of the few old fashioned skills that doesn't seem to be any less useful now than 100 years ago. Knots aren't just still usable, they're still the common, generally preferred way.
Almost everything low tech is, for most people, something pretty cool, maybe even something we think we probably should learn, but not something that generally comes up in real life for most, like ropework seems to, at least for me.
Wiki2Touch is about 9 GB for en-wiki as of 2012, and about 14 GB now.
I've been working on a Wiki2Touch reader for more modern iPhones. I've got the BZ2 decompression working, now I'm just cleaning up the markup to HTML parser in JavaScript.
If you're interested, please email me and I'll send you the code! You can install it for a week on a non-jailbroken device using XCode self-signing, and if you're jailbroken you can install the Immortal tweak to make it work forever.
If you have a developer subscription and want to put it on the App Store, I'm be very happy to let you do that. I just don't want to pay $100 for the privilege of running my own code.
At a first glance, the Wikimedia XML dump does not look substantially different from what Kiwix/ZIM does with compressed HTML: They're both compressed (bz2 for the Wikimedia dump, zstd or LZMA for Kiwix/ZIM), and both compress multiple files at once, so inter-file redundancy should hopefully be significantly reduced.
HTML seems a bit more verbose than the Mediawiki syntax (plus the XML header for each article), but I'd be surprised if that actually accounted for a 3x difference in size.
Then again, Kiwix seems to have experimented with shared dictionary brotli compression, which supposedly yields an >2x improvement: https://github.com/openzim/libzim/issues/144
I wonder if their current zstd implementation also uses shared dictionaries. If not, that might just be the reason: If ZIM compression chunks are much smaller than the bz2 streams of the Wikimedia dumps, there would still be a lot of redundancy between chunks.
> "The last official WikiReader image was released by Pandigital in 2011. However, there exists an active fork [...]"
> "The latest update was released by this community in June 2021."
Nice to see that a community-backed project can outlive the official one for such a long time and keep it alive.
Shows how good open source can be.
This is excellent, I looked for something like this a few years back to explore what a minimalistic connected computing stack could be, and the only options were dead kickstarters or hacking something from blackberries or nokia c3s with hit or miss results.
Thank you for the link, bookmarked! Might explore this again some time soon.
And the various Gameboy clones like the Odroid Go.
I've been really wanting to do something with this kind of thing for a while... but I'm not really sure what, just that I want it to work with ESPHome somehow.
26 comments
[ 1.5 ms ] story [ 75.0 ms ] thread> Updated on Oct 22, 2021
https://github.com/kiwix/kiwix-android
last commit 9 hours ago
EDIT: Oh I see the F-Droid one wasn't updated in a while https://github.com/kiwix/kiwix-android/issues/3376 https://github.com/kiwix/kiwix-android/issues/3460
the newest one was built yesterday, this one should probably work
Now if apple could just shoot me a few kb of news/weather updates by pointing it at one of their satellites, we'd be really cooking.
(Other than a solar panel and I got a cigarette lighter adapter with clips at the end so I can charge off a car battery without turning on all the accessories)
I think the most useful offline reference is various rope knots. Mostly because I'm not a prepper and they're one of the few old fashioned skills that doesn't seem to be any less useful now than 100 years ago. Knots aren't just still usable, they're still the common, generally preferred way.
Almost everything low tech is, for most people, something pretty cool, maybe even something we think we probably should learn, but not something that generally comes up in real life for most, like ropework seems to, at least for me.
I made a one-page reference of my top favorites but someone can probably do better: https://www.reddit.com/r/knots/comments/16zcsxi/i_made_an_a5...
Kiwix is over 100 GB for the English Wikipedia. https://dumps.wikimedia.org/other/kiwix/zim/wikipedia/
wikipedia_en_all_maxi_2023-10.zim 31-Oct-2023 07:37 103214717026
http://www.haukap.com/wiki2touch/
Wiki2Touch is about 9 GB for en-wiki as of 2012, and about 14 GB now.
I've been working on a Wiki2Touch reader for more modern iPhones. I've got the BZ2 decompression working, now I'm just cleaning up the markup to HTML parser in JavaScript.
If you're interested, please email me and I'll send you the code! You can install it for a week on a non-jailbroken device using XCode self-signing, and if you're jailbroken you can install the Immortal tweak to make it work forever.
If you have a developer subscription and want to put it on the App Store, I'm be very happy to let you do that. I just don't want to pay $100 for the privilege of running my own code.
What date is that 14GB version from? And is it really not filtered at all?
The XML, which is larger, is now 21 GB. How did you get 60GB for the non-image English file?
https://dumps.wikimedia.org/enwiki/latest/
enwiki-latest-pages-articles.xml.bz2 02-Dec-2023 02:38 21557219519
At a first glance, the Wikimedia XML dump does not look substantially different from what Kiwix/ZIM does with compressed HTML: They're both compressed (bz2 for the Wikimedia dump, zstd or LZMA for Kiwix/ZIM), and both compress multiple files at once, so inter-file redundancy should hopefully be significantly reduced.
HTML seems a bit more verbose than the Mediawiki syntax (plus the XML header for each article), but I'd be surprised if that actually accounted for a 3x difference in size.
Then again, Kiwix seems to have experimented with shared dictionary brotli compression, which supposedly yields an >2x improvement: https://github.com/openzim/libzim/issues/144
I wonder if their current zstd implementation also uses shared dictionaries. If not, that might just be the reason: If ZIM compression chunks are much smaller than the bz2 streams of the Wikimedia dumps, there would still be a lot of redundancy between chunks.
I remember getting one of these from woot-dot-com (iirc?) back in The Day. My dad really liked it.
Nice to see that a community-backed project can outlive the official one for such a long time and keep it alive. Shows how good open source can be.
Kind of a dealbreaker isnt it?
Humanity’s knowledge in a box, or as the Internet Box as the IT Crowd called it.
i think there might be room for another class of less-powerful, offline-first but still connectable, app-capable, long battery life devices.
Thank you for the link, bookmarked! Might explore this again some time soon.
And the various Gameboy clones like the Odroid Go.
I've been really wanting to do something with this kind of thing for a while... but I'm not really sure what, just that I want it to work with ESPHome somehow.