Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM?
This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
It already is a local archive that can be searched independently, but the LLM does RAG and is, according to them, 95% accurate. It can be useful if you don’t remember where to find a source/what words to search for, but even a durable device with Kiwix would be useless in a a lot of survival situations people imagine.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
I like this "both / and" approach. Restructure this project to have the LLM source from local Kiwix data you've selected to make available offline, rather than a source of data you're unfamiliar with.
I think that for situations like this we should invent a device that can hold information for an arbitrary amount of time even without electricity. Maybe we could call it "BOOK" or something like that.
This isn't clever or insightful. You can of course condense down critical information into a survival handbook, but in existential crises you don't necessarily know in advance what you'll need to know. Wikipedia (as a benchmark repository of knowledge) is ~5 million pages and would require a decent-size room to store if printed out. By contrast, a laptop + solar panel can fit in a backpack. SD cards/USB sticks with the capacity to store all of Wikipedia are a few $ each.
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
Why not both? After the collapse of civilization, as long as you can maintain the computer running, you could educate the next generation as soon as they could write. Maybe they will bypass the local agent as they learn their way into the library, to save energy.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
Survival implies that energy would definitely be a very limited resource. Using it to replace knowledge is just... I can't describe it even. Why not just learn it on our own and keep a small guidebook in your backpack. Our overrelience on computers and now LLMs, even for basic life-saving knowledge is simply our way of going the way of the dodo... I use frontier models at work, I definitely get more done. However I will never use it as a replacement of my own knowledge.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
For one-off questions, a local archive makes sense.
This is more for elaborate projects where you don't know precisely what you are looking for. Questions I can think of in a survival scenario:
- my solar panel connector is broken. We only have 8 hr battery left. Can you tell me how to DIY repair it with the following parts
- we want to receive emergency FM broadcasts from the state. Give me steps on how to receive it on Flipper Zero with this breakout board I found in my spare parts
- with the dewalt drill and these parts, can you tell me how to build an outboard motor for the canoe. We need to get outta here before the flood goes up further
Love this. And as a trend, I'd love to see more local/small task-specific models/agents. Experts at helping you with that one thing, leaving the everything-and-the-kitchen-sink to the frontier models/agents
With all due respect: why does this need an LLM, and this much data? Surely a "classic" knowledge base with a more traditional search bar, running on top of the same data this LLM was trained on would be way smaller, and actually as trustworthy as the source material it surfaces for you?
There are a fair number of people who seem to believe that LLMs contain "world knowledge" and are blissfully unaware of how inaccurate an ungrounded chatbot can be. They ask it questions, it gives them confident and plausible (sometimes even correct!) answers, and they leap to the conclusion that these are useful tools for end-of-civilization knowledge bases. I fought this battle countless times on HN and Reddit, generally unsuccessfully.
This project at least attempts to ground its answers with (presumably) human-vetted sources. That will make it much, MUCH more accurate and truthful than a bare model. But there is still a non-zero chance for it to hallucinate answers, especially when posed a question that it doesn't have any source material in its database.
As for why bother hooking up an LLM instead of a simple search engine, search engines don't synthesize data from multiple sources into a coherent answer for you, but LLMs are (mostly) great at that.
That said, while I like this pattern (LLM + knowledge base) for lots of things, this isn't necessarily something I'd trust with my own (or my family's) survival.
On the setup process: "You only click for about 5 minutes; the rest is downloading." On the energy consumption: "A small generator or power station runs it easily"
Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.
Back in 2018 I was working on a project for educating schools in certain parts of Africa and Middle East, on AI. The issue was it had to be air-gapped (no internet guarantees) and small/portable. I ended up downloading Wikipedia (not all of it), and then created a lightweight inverted index (for search) and TextRank (for summaries). The whole thing was on a Raspberry Pi, and installed in a small plastic enclosure. We had many of these and could easily ship them. There was an "AI Learning Module" as well we built, that started with Scratch, and then moved into Python. But if I was to do it all over today, I would probably add a tiny LLM and maybe a small ASR and a TTS model too, and you basically have a completely self-contained AI knowledge assistant in a cigarette box. The point being, you basically also have a "survival AI" you can take with you. Of course you still need electricity...
If you're actually offline and need information in a 'survival' situation, I'm not so sure the hallucinatory nature of an LLM is the based way to distribute a repository of information. Additionally, there's no need to fit an LLM entirely in memory.
In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.
> Mac or PC with 16–31 GB | IBM Granite 4.2 8B | 14 of 22 right | 6 GB
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
The advantage of a turn-by-turn flow (honestly the chat interface was genius) is that with a little human intervention you can get much better results out of even silly models. The model of choice is strange, though. Not a particularly good one.
Either it depends a lot on the model, or I’m bad at chat, or both, because in my experience once a chat starts going off the rails it’s pretty hard to correct the initial misconception. I find it works better to backtrack to just before the misconception and edit my previous response to head it off at the pass.
Definitely agreed that it's better to not have some text in the context than to have some that needs to be opposed. I suppose I meant steering prompts are a non-obvious innovation to LLM interaction.
People apparently like Granite for RAG specifically. I'm not aware of any evaluations with numbers though, so I'm not sure if it's a real difference or just vibes.
I just use llama cpp , it's light weight, I think you create your own integrations, with a larger model so llama cpp can interract with scripts that would browse the web, control your computer etc... propably will have to download some pip packages
I would suggest Hammunition, https://github.com/Renegade-Penguin/Hammunition with install docs at https://renegade-penguin.github.io/Hammunition/ . Hammunition offers much more info than Nomad for SHTF type emergencies. Hammunition uses KIWIX tools instead of an LLM to search the reference sources that you downloaded (eg. Wikipedia, etc.). You can install it on a Linux distro on a 500 GB external SSD drive that you can pick up for $100 at Microcenter. It is geared toward people interested in ham radio, but you can listen only on the ham radio bands without a ham license.
It's disappointing that Gemma E4B seems to perform poorly on this (42% accuracy according to the GitHub). It would be nice if a $25 phone could be loaded up with this and stuffed in a EMP proof metal box with a small solar charger. Maybe a finetune or Gemma 5 will get us there.
No thanks, I'll use a generic model along with trusted sources.
Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
The $100 merely serves as a gatekeeper to prevent people and non-devs to create unlimited developer accounts. It used to be $500 a decade or two ago, and it was worth it even back then, until Apple realized they should probably just give it away almost for free.
> Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
This is a great example of how Apple's "scare warnings" completely work to mislead folks about what they are actually warning you about.
This topic is bringing out the hallucination-panic crowd.
It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice.
Use the LLM as search technology: string match -> semantic search -> search intelligence.
This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation.
Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first.
Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.
A great strength of LLMs is how they provide useful answers even if the prompt has glaring spelling or grammatical mistakes. If your prompt is not great, as you said, they will reply in a way that helps you identify that flaws, and learn how to steer the model better.
They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.
The readme on Github does say the system produces hallucinated sources https://github.com/anonnerdfrenzy/survival-llm#how-the-model..., but really you're correct, for this kind of highly targeted use case the LLM's summary could probably be ignored entirely and the system could work extremely well even just as a search engine: index all of the content up front, chunked by paragraph and maybe enriched with a HyDE step, have the LLM facilitate running a handful of hybrid keyword/semantic searches based on the user's question and reranking the results, and show the exact pieces of text in the corpus that are most responsive to their question. No summary needed.
Query expansion and reranking are both valuable and things that even small LLMs are pretty good at. People are broadly not great at writing search queries, so a system that takes a poorly phrased question, turns it into several different actual search queries and then merges and sorts the results by relevance can have a big impact. All the better if the AI can ask the user some clarifying questions.
Nevermind hallucinations, the donkey-work is important; why bypass the hours of research, which trains our in-built LLM and can lead to new realizations, and get the carrot without much effort? This LLM stuff feels like a candybar; the anticipation of the magic feels good, the thing tastes good while you're eating it, and everything thereafter is shit, but we easily forget, and reach for the next candybar.
I appreciated Blindboy Boatclub's take in his podcast episode "Artificial intelligence is Disgusting and it will never replace Artists" from 2026 March 17. I also appreciate Electrical Engineer & Computer Scientist Timnit Gebru's perspective on these stochastic parrots.
While intraspecies diversity confers, up to a point, a survival benefit, consider the long-term consequences of playing this particular game.
We don't have to eat every candybar the rich mostly-guys ply us with.
The premise in this case is to get answers quickly in an emergency situation. You don’t want to tell the person with a burn injury to wait a bit since you always wanted to learn how burns are treated.
In a true emergency, there are often many cases where you need not just information retrieval, but "thinking and reasoning" to deal with unprecedented situations, right? Is it technically possible to balance lightweight local performance with thinking capability, while avoiding the risk of hallucinations in life-or-death instructions?
The summarization is the main issue here. A fuzzy search is far less controversial. The LLM companies have done a good job synonymizing "AI" with their products however.
Multiple times yesterday I caught Claude giving slightly incorrect information about things it had read in docs and source code. Its info was accurate enough to be plausible, but it was still wrong. And when I pointed that out, it agreed that it had overstated or made statements without verifying them.
It would be terrible to be doing an emergency medical procedure and have an AI backtrack on something it had told you to do.
The AI should have a tool that it can use to quote an underlying document. The AI itself would not be responsible for filling the quote; that would be done by the interface which deterministically quotes the requested section.
I like using Gemini Notebooks, but this is one of my current frustrations. At least with the current UI, I can see the document getting quoted, but not the actual quote to verify what it's saying or to see the context of the quote. So I'm forced to find the content manually or just trust the LLM got it right.
There really needs to be some kind of "double check" mechanism built into these things instead of putting 100% of that on the consumer. Humans simply don't have the mental capacity to double check the amount of data coming out of LLMs.
The "double check" mechanism for medications is: We tell people not to take medical advice from anyone who is not trained in medicine.
This is not a new problem. The internet has been full of nonsense since the 1990s (at least). Gossip and malicious rumors are as old as language.
It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort). We should train people to detect and ignore nonsense when they hear it.
This advice is inadequate and impractical. Most people don't have access to doctors who spend enough time on them to really get to the bottom of all issues.
In most countries, there are simply too many patients per doctor, meaning it's a conveyor belt of "good enough" 5 minute appointments.
People will turn to other modes of discussing and analyzing their problems if they go to professionals repeatedly and the problem still isn't solved. Just telling them not to do so or to ignore 100% of it isn't going to solve that.
> It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort).
Seems like a big statement. Why is it not realistic? People are already doing it manually by having one LLM fact check output from another.
The difference is back then we had the adage "don't believe everything you read online" that helped adjust our priors. Today, however, these companies are specifically marketing themselves as the source of ground truth. Your boss, or your school that a generation ago banned wikipedia, is now demanding you to use these tools and make the work dependent on them.
It’s fine and safe, it just feels a bit silly. Yes, I could have an LLM read a ton of stuff and generate a list of suggestions on the fly.
Or I could have an honest to god farmer make that list now, ahead of time. Regular search could absolutely surface that.
But also, as someone who does hydroponics, you’re going to end up reading the papers anyways unless you’ve done this a lot. It’s not easy to pick out something like low iron from too little light.
Agreed. Especially for something like this, in which many imaginable situations would require the ability to run on Linux rather than MacOS or Windows, pretty strange.
Download a good fast local model e.g. Gemma-4-12B (small machine) or Qwen3.6-35B-A3B (larger memory). Or a Nemotron model, if you prefer that.
Then you'll have both your backup/emergency assistant that runs on your own computer and be gaining skills in a common, frequently-updated, well-supported program.
This will come in handy after AI takes over the world.
But kidding aside, this is an interesting reveal on the ease with which a custom LLM can be spun up to fit a particular niche application, especially for systems not meant to host an LLM.
182 comments
[ 0.29 ms ] story [ 15.7 ms ] threadThis seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
If there's any place to get an answer it's here.
Bravo my friend. And I mean that in a good way. Love me some diversity.
You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.
Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.
I work on compilers, kernels, and operating systems using exclusively consumer GPUs.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
One of these situations prohibits the other.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
This is more for elaborate projects where you don't know precisely what you are looking for. Questions I can think of in a survival scenario:
- my solar panel connector is broken. We only have 8 hr battery left. Can you tell me how to DIY repair it with the following parts
- we want to receive emergency FM broadcasts from the state. Give me steps on how to receive it on Flipper Zero with this breakout board I found in my spare parts
- with the dewalt drill and these parts, can you tell me how to build an outboard motor for the canoe. We need to get outta here before the flood goes up further
This project at least attempts to ground its answers with (presumably) human-vetted sources. That will make it much, MUCH more accurate and truthful than a bare model. But there is still a non-zero chance for it to hallucinate answers, especially when posed a question that it doesn't have any source material in its database.
As for why bother hooking up an LLM instead of a simple search engine, search engines don't synthesize data from multiple sources into a coherent answer for you, but LLMs are (mostly) great at that.
That said, while I like this pattern (LLM + knowledge base) for lots of things, this isn't necessarily something I'd trust with my own (or my family's) survival.
Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.
In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Honestly, just buy some books.
I found Project Nomad (https://projectnomad.us) which looks more polished. Any other suggestions?
Helps you download a local LLM and offline Wikipedia. Probably easier to just ask your favorite agent to glue something together yourself these days.
Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
This is a great example of how Apple's "scare warnings" completely work to mislead folks about what they are actually warning you about.
The source (data collection) seems the most important feature here worth reusing.
It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice.
Use the LLM as search technology: string match -> semantic search -> search intelligence.
This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation.
Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first.
Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.
They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.
I tried using dictation but I really disliked speaking to it.
You can write a really messed up sentence and it almost always worked out what I meant.
Quite amazing actually
It would be terrible to be doing an emergency medical procedure and have an AI backtrack on something it had told you to do.
There really needs to be some kind of "double check" mechanism built into these things instead of putting 100% of that on the consumer. Humans simply don't have the mental capacity to double check the amount of data coming out of LLMs.
This is not a new problem. The internet has been full of nonsense since the 1990s (at least). Gossip and malicious rumors are as old as language.
It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort). We should train people to detect and ignore nonsense when they hear it.
In most countries, there are simply too many patients per doctor, meaning it's a conveyor belt of "good enough" 5 minute appointments.
People will turn to other modes of discussing and analyzing their problems if they go to professionals repeatedly and the problem still isn't solved. Just telling them not to do so or to ignore 100% of it isn't going to solve that.
> It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort).
Seems like a big statement. Why is it not realistic? People are already doing it manually by having one LLM fact check output from another.
Or I could have an honest to god farmer make that list now, ahead of time. Regular search could absolutely surface that.
But also, as someone who does hydroponics, you’re going to end up reading the papers anyways unless you’ve done this a lot. It’s not easy to pick out something like low iron from too little light.
From the command line: unsloth studio
Browse to the URL it gives you (eg http://127.0.0.1:8888/)
Download a good fast local model e.g. Gemma-4-12B (small machine) or Qwen3.6-35B-A3B (larger memory). Or a Nemotron model, if you prefer that.
Then you'll have both your backup/emergency assistant that runs on your own computer and be gaining skills in a common, frequently-updated, well-supported program.
But kidding aside, this is an interesting reveal on the ease with which a custom LLM can be spun up to fit a particular niche application, especially for systems not meant to host an LLM.