Ask HN: How to keep tech running in the apocalypse?

74 points by armagon ↗ HN
The gist of this question is, "how does one prep to keep technology operable?"

Imagine a disaster causes widespread collapse, where resources are no longer easily available, power and internet access is erratic or non-existent, and then things calm down enough to where the necessities of life are available and society tries to resume functioning.

Now, suppose you'd like to make computers work. What sort of things would you need?Power, obviously, or a way to charge batteries. What else?

What if you wanted to have communication with other people, or perhaps data communication? Would you want packet radio? HAM gear?

What if you want to repair equipment? I don't know if the computers in cars "break", for example [not that I imagine fuel being available]. I wonder about what sort of needs there might be to repurpose gear.

What if you need to keep an application running? Maybe you have a mesh network and people want a web app for some purpose. But, without documentation, it'd be incredibly tedious.

I can't help but think I've seen a linux distro designed for this scenario, where you have binaries, and all the code, and all the toolchains, and all the documentation.

The right answer might be "don't bother", but I'm still curious.

52 comments

[ 2.0 ms ] story [ 68.0 ms ] thread
The needs of an apocalyptic world would not be the same as the needs today.

I’m guessing repair knowledge would be more important in the short to medium term.

Outside of that I think you need to work your way back up the tech chain starting with the wheel, leapfrogging as much as you can. But you don’t want to leapfrog over early tech like industrial machines, transportation, etc. and start working on web apps too early.

I started thinking about this a bit after Sept 11th. Technology was the only think I knew, so I decided to diversify my skill-set. I started by reading Phil Gingrey books and building a metalcasting foundry as step 1. From there, I built a crude generator based on a lawnmower motor, a pulley and an automobile Alternator. This coupled with an inverter provided power for as long as I could fuel the motor.

Securing fuel for the motor, let me down the dark side of thought which is either protection or "just take it". I bought guns and learned how to use them (safely!).

Of course, later I learned about gardening and self-sufficiency and being off grid. While I've never been off-grid, I started to do thought exercises and preparation. I was never a "prepper" per se, but I learned the basics because the early 2000s created a lot of global uncertainty.

Circling back to tech, I started to learn morse code so I could HAM.

Power + communication + protection and food are the basics (not necessarily in that order).

I have some survival books and Wikipedia downloaded on a disk in a faraday box, but I'm not sure how long I'd be able to access it - it largely depends on the type of event.

A significant power outage is all it would take to simulate technology life in a disaster scenario. My servers/homelab wouldn't be of much use in a disaster, but being able to read would be tremendously useful and valuable.

> Securing fuel for the motor

Use a diesel motor and grow vegetable oil. Or culture the right kind of algae.

You can also build a wood gasifier and run a diesel engine/tractor off wood fumes.
That's impressive that you actually did something.

Generating power with a lawn mower? Would never have thought of that.

Small motors have a variety of uses in this kind of scenario, fuel limited of course. They can provide electric power, but also power your mills and other machinery. They're also (relatively) easy to maintain and repair, not a skill everyone has, but a skill nearly everyone can learn.

Of course, since it is fuel limited you'd want watermills and windmills, along with other manual (human or animal) powered systems, for driving machinery most of the time. Now, if you can get a diesel system going then you have the possibility of using biodiesel which would alleviate some parts of the fuel problem, assuming you have a source to produce the biofuels from.

I may have posted this before, but here are pictures of the generator[1] and furnace[2].

As a bonus, I completely forgot to mention my homebrew ethanol still[3]. I applied for a small fuel producers license from the ATF/TTB which required me to denature it so it could only be used for fuel purposes.

[1] https://chrisbergeron.com/2008/08/24/homebrew_generator/

[2] https://chrisbergeron.com/2008/01/21/metalcasting_furnace/

[3] https://chrisbergeron.com/2007/08/07/ethanol_still/

It all depends on which specific fringe scenario (apocalypse scenario) you are prepping for.

Since you mention computers in cars breaking, I am guessing your preferred flavor is nuclear winter.

Check out this study by the EMP Commission.[0]

Also I am having trouble imagining the average Joe in a post apocalyptic scenario wanting a web app for anything.

[0] http://www.empcommission.org/docs/A2473-EMP_Commission-7MB.p...

It's not enough to have a bunch of ebooks on a drive and a pile of canned goods.

You need to get the ebooks and food to people who need them.

To that end, with most everyone having a mobile device, a wifi hotspot where people can leave text messages and grab books is useful.

That actually makes a whole lot of sense -- might be the killer app; a bulletin board that works on your phone. Assuming people had enough power to charge their phones, and assuming the internet at large was unavailable, this would be brilliant.
"A bulletin board that works on your phone" is actually a decent summary of gossip protocol based p2p systems like Secure Scuttlebutt (https://scuttlebutt.nz). Data can be transferred when devices are in near enough proximity to each other (share a network, bluetooth, whatever). Messages are cryptographically signed (and possibly encrypted) so you could use this as a way of maintaining (albeit slowly) communications across a distance so long as there is still travel between communities.
"repair" means "stock a bunch of replacement parts", where any repair is possible in electronics. many people replaced capacitors on motherboards, but very few can make a capacitor from aluminum foil and tape or whatever's to hand that will do the job of the $0.50 commercial one.

Presuming hardware's running and communications are just spotty; then having your own local archive of sources is a good idea. mirror gutenburg and the wikipedia dump and stuff like that as well as the full repositories of some of your favorite distributions (source packages too!)

As another option, you can also obtain the knowledge and tools to cannibalize other electronics to get the replacement components you need. Might be more sustainable since you never know which components you might need, and you can't reasonably stock all of them.
> suppose you'd like to make computers work

Specifically why? Paint me a scenario.

to check twitter to see if anyone found a source of drinking water. ;)
this is not how you find a water. You have to know your local area.
Maybe power is available, internet is not, and you are running a factory -- perhaps a cannery or perhaps making textiles. Maybe the factory existed before the disaster, and there's enough demand to run it if you can, but you no longer have (or never had) pre-computerized technology to run it -- so you need to make your existing computers work to run things.
The internet is the least of your problems. The key industrial automation equipment (PLCs and IPCs) is connected directly to your machines. Higher-level control and planning functions like MES and SCADA are usually running from your server closet. You might not have access to your ERP or cloud MES system, but you can keep the lines running without them for a while. Plenty of factories have weak or flaky internet, so operating during an internet outage is hardly a novel situation.

Your bigger problem is the supply chain. After the apocalypse, I doubt you'll have an easy time sourcing cans, bulk product, spare parts, and sanitizer to run your factory. After all, each of those inputs to your process have their own complex supply chain too. And no, you're not really going to be able to scavenge these parts of manufacture them yourself, since they all have fairly fine tolerances.

In reality, I'd expect that an apocalypse will cause a reversion to a more subsistence-style lifestyle, where production and consumption are largely local and supply chains are very simple, if present at all. Computers will not help with this. I can't imagine keeping our current production processes running without all of the things required to support them.

most likely all the usable raw materials would have been scavenged, or melted, or ruined by the weather, or lacking any of the sundry bits of support liquids/solvents/grease necessary to make it run

say you do find a working factory with everything in it only waiting to be turned on it'll likely have manuals in some file cabinet and won't need an external computer

But yea, it's trivial to paint a scenario that makes a working computer useless, however it'll be difficult to keep one running. So, best of luck!

I'd really want access to essential sources such as wikipedia, a whole bunch of survival books, manuals, governments stuff, patents etc.

Access to internet-level knowledge in a pocket-sized device when the internet is gone is going to be a massive advantage.

Could go with a specialised solar-powered e-ink device for power concerns.

Access to internet-level knowledge in a pocket-sized device when the internet is gone is going to be a massive advantage.

In the 1950s, such a thing was created by US Civil Defense. It was a microfiche library of how-to books with viewers that worked in sunlight, and major fallout shelters were stocked with it. If anyone knows the whereabouts of a copy, please post. Not pocket size, but liftable.

Kiwix for offline wikipedia with the wikipedia "all maxi" package (87G for English) comes close.

https://wiki.kiwix.org/wiki/Content

Also a mirror of scihub might be useful at a civilization level, but that's more like 70TB.

Someone should create a 'foundation' project, with mirrors of essential collections, eg wikipedia, scihub, digital art collections, gutenburg, etc.

Good question. Relatedly, could a cyberwar with Russia or China take down our grid? A cyberwar almost seems inevitable at this point. And would the answer in regards to keeping tech running be different for apocalypse vs cyberwar?
reliable electricity generation, transmission, and storage would be step one.
I suppose it depends on the nature of the 'apocalypse' and how much of existing infrastructure is lost. Do you want to regard the last few decades as a slow-rolling apocalypse shearing away bits of civilization over time?
No, I hadn't seen things that way. It is worrisome that we've lost most of our manufacturing capabilities in the West. Would make any sort of rebuilding difficult.

I've just had it underscored to me recently that the world can change dramatically and permanently in minutes, and was wondering what I could do to be prepared for the worst (while hoping for the best).

I'd say a Raspberry Pi 400 (or several) and a shoebox packed full of USB drives and other flash storage. I would also use read-only optical disks for redundant copies (along with a couple of USB optical drives). I would also get a USB-powerable travel router.

Basically, the massive prevalence of USB over the past couple decades is your friend.

I would store local copies of the standard fare like Project Gutenberg, Wikipedia, instruction/repair manuals, and textbooks (especially math and science ones). For particularly important stuff, like survival guides, I'd even look at paper copies. Perhaps even laminate a few maps and navigational charts.

I'd also try to have several copies of complete Linux package repositories or at least all of the packages that I know I'll need. If/when backbone infrastructure starts crumbling, it'll cascade. Even downloading a text-based browser could prove an impossible task.

Even just having a hand radio could prove vital. Just be aware of how you might power it (and any audio equipment it might need).

I think there needs to be a good "distribution" of all modern open software in a way that it could be developed offline. We can download Wikipedia and Project Guttenberg and have access to a huge amount of human knowledge, but there is no good similar archive for software. If I currently had a copy of popular repos on GitHub for example and needed to change and build some random important piece of software for helping rebuild society without the rest of internet, that would be hard for a lot of projects. You would probably need a copy of all of Linux package repos, npm, pypy, dockerhub etc to be able to edit any piece of software you might need to edit but then you would also have make changes in different places for things to work offline etc. I think things like content addressability, reproducible builds, nix, and ipfs are getting us closer. Hopefully soon we will be able to make a usable "dump" of all modern open source software and have it be able to just work for development.
Imagine if people forgot to have a copy of stackoverflow.com in the apocalypse. I guess that is what would make it the apocalypse.
Macro: that's why the military is over funded. They are the "prep" to keep everything operable.

Micro: ham radio

You might like to try reading The Knowledge: How To Rebuild Our World After An Apocalypse. It's a very interesting book, and basically works its way through from the simplest agriculture to reinventing computers

https://www.amazon.co.uk/Knowledge-Rebuild-World-After-Apoca...

Yeah, I came here to say this.

In particular, I remember a bit about how to take an alternator from a car, use a river's current and a paddle-wheel to spin it, and tada: you've made a hydro-electric generator.

Each one provides a small but consistent amount of electricity, so if you need more power, find another alternator.

A very practical problem right now: keeping US tech running if Russia is trying hard to take it down. Everybody in network operations needs to be thinking real hard about that.

Remember when Maersk, the shipping line, shut down for over a week due to a cyber attack? The pipeline shutdown? Several hospital shutdowns. Expect that all at once.

Internet enabling power grid protection relays now looks like a big mistake.

There is a Scandinavian tech guy who has a whole site/blog around this concept -- I can't seem to track it down though. The info was really good, so hopefully someone on HN knows what I'm talking about. His site has a great 90's-style design, and he has an epic beard I think. :)
the technology you need after the apocalypse is called "fire". and perhaps some rocks to hunt your food
Bows and arrows. Easy to make if you have a steady hand, a blade, and a hatchet. The strings are the hard part to make on the go, but doable.
A shovel, hoe, ax, wheels, maybe a lathe.

Actually, the biggest technology you'd need--and one that's probably irreparably lost in the United States, is a community.

Not if you're Amish. :)
No one person in the world is capable of making a pencil from scratch! [1]

Given that, i would think the big factor in building anything post-apocalypse is communication. So i guess my answer would be paper and pens. Hard paper copies of as much information as possible and blank paper to write letters.

Computers are the end product in a long long tech tree. So the question just seems sort of non-sensical to me. Like if we get to the point where we rebuild society enough to even consider computers, we have already solved food, shelter, heating, manufacturing, mining etc. At that point its mostly a matter of information - just archiving as much info as possible on stuff like silicon manufacturing, linux source code, electronics etc. would be enough.

[1] https://www.econlib.org/library/Essays/rdPncl.html?chapter_n...

In many post-apocalyptic stories, people are using whatever vehicles they can find for as long as they can last. The point of the question is similar -- how can you use whatever computers you can power for as long as they last? I'm not talking about rebuilding fab plants and stuff first; I'm talking about using the millions/billions of devices that are still around and working, and making them do something useful.
Copies.

Have any simple-ish tech stack, and have many copies of the SAME thing.

ie: Get N-ipads, N-rasperrys, etc.

Then N copies of whatever you need to power, connect, protect, etc (cables, routers, ups, batteries, etc).

Then N copies of the data in N storages

--

This is probably the best course: REPAIR anything will be near impossible.

What’s the right value of N? That’s the tricky part.
Estimated based on MTBF (mean time between failure), and expected duration of the apocalypse. If it's truly apocalyptic (nuclear holocaust, not a large scale but conventional war), then the expected duration is your lifetime.

Let's say you only want one of the device operational at a time, then you want:

  Spares = Apocalypse Duration / MTBF
This is under the assumption that you will never get any more spares for the duration of the apocalypse. So if your iPad breaks once a year on average, and you expect civilization to have reached basic recovery levels to give you an alternative or replacement for your iPad in 10 years, you want 10 spares (well, I guess 9 in this case, stupid off-by-ones).

If you have N gas generators operating (or that you cycle through to extend their useful lifetimes), you will want:

  Spares = Apocalypse Duration * N / MTBF
An iPad battery, even used and recharged daily, should last you several years. Let's say 3 for the sake of the calculations and you only need one iPad. Then you want:

  Spare Batteries = Apocalypse Duration / 3
You may obtain those spares by getting spare iPads. Of course, you also have to consider spoilage. A battery sitting on the shelf for 3 or 6 or 9 years will have some chance of becoming a useless battery. This would force you into a rotation of the devices, charging them less frequently and finding a way to synchronize data between them.
I Will add plenty of AP, routers and UTP cable.

They were projects like decentralized WiFi networks, mostly on cities (I remember one in my city), maybe people need to remember them

We need a real life foundation.