I thought this was an interesting meditation on the limits of AI and how ordinary people don't see those limits, wrapped up in a context I normally wouldn't expect to see AI at all.
It's quite a different level of commitment to truth than one usually runs across these days. Instead of a world of meme and vibes, a costly commitment to only say things that are true.
To an extent, I also think the determination of the Apologetics Project also shows the tendency of people to go into denial about the limits of the technology. There is a lovely SF short story, The Quest for Saint Aquin, on how a true AI might feel about religious belief but we are a long way short of that.
It worries me a lot more that governments and the like will also be in denial about what they can do with AI. I can ignore low quality apologetics, I cannot ignore the government (I have to pay taxes, for example).
People ask AI because other people don't respond to their questions. For some wrong answer might be worth less than no answer. For many it isn't (which is fitting since we are on subject of religion).
The "one or two wrong words" line of thought was one of the things that made me step away from religion. If there's no provenance then text, then what hope does anyone have.
I feel like the way around this is hermenutics and exegesis and that's where things differ substantially between the bible and llms. The bible has both whereas LLMs are arguably approached hermenutically, but have no exegesis to speak of.
The core of the argument is that if accuracy and reliability matter and if verification is hard, then LLMs are inappropriate.
If accuracy and reliability don't matter, go for it.
If they do, but you can easily verify the output, then go for it.
So, yes. The core argument would apply to making inquiries about any ancient religious texts especially if you aren't already able to read the generated text (it's in a language you aren't fluent in) or are unable to access its claimed references to verify what it says they say.
Apologetics is about a lot more than texts. It includes philosophical arguments (e.g. for the existence of God), non-scriptural texts, and personal experience.
I do not think that LLMs are any better equipped to deal any of those other aspects of it either - especially not personal experiences.
"Nothing true can be said about God from a posture of defense."
- from the novel Gilead, by Marilynne Robinson
I take this quote to mean that most people's idea of "apologetics" (arguing to convince people that the facts of Christianity or some other religion are true) is kind of pointless. You'll never convince someone logically of something that has to be experienced viscerally. LLMs don't help with that at all.
>You'll never convince someone logically of something that has to be experienced viscerally.
While you might be right, Christianity in particular is based on truth claims, including specially the resurrection, so the Christian tradition places special emphasis on rational defense. Apologetics is not just a means to persuade others; it is also a means to persuade oneself.
Edit: Responses say that all religions involve truth claims. True, perhaps I was imprecise. I only mean that the Christian case is especially stark. St. Paul: "If Christ be not raised, your faith is in vain." I'm not aware of another faith tradition that considers itself to hang upon a single boolean.
All religions are based on "truth claims", Christianity is nothing special in that regard. Ask a Muslim, a Buddhist, a Jew... They all have their professions of faith, and fervently believe their teachings are the truth.
It still seems like a meaningless distinction, Buddhism also rests on claims of real world events: the Buddha is a historical figure, the story of his enlightenment being a factual happening is key to the belief system.
There are ways of interpreting and practicing Buddhism in which this is less important (eg modernist, psychological, western), but that's also true of Christianity (eg the Jefferson Bible, Unitarians, etc).
They claim to know the truth but the reasoning behind it is different between religions. I'm no religious expert but I'm very familiar with Mormonism and they push really hard on the idea that faith is intentionally separate from truth and you learn about the truth of the church by meditating on your feelings and experiencing frisson, not via logic. Yes everyone has their mythology and Mormonism has its share of things the prophet says God says but it is very different than the Christian apologists that try to reconcile the Bible with things like astrophysics and other observed phenomena.
Do you have a citation for this? I am reasonably familiar with Mormonism (The Church of Jesus Christ of Latter-day Saints) and I don't think I have ever heard this before?
I'm no religious scholar but I've had a LOT of conversations with Mormons with various levels of training (regular folks, bishops, missionaries) about why parts of the story don't make logical sense or aren't internally consistent. The common answers are in the expected range from the standard "god works in mysterious ways" to "if you could prove God was real it wouldn't take faith and the point of the human existence is to find faith".
The Mormon church, of course, has its wings of scholars trying to find evidence to support their origin stories (like looking for their god's original home planet or buying ancient South American temples to hopefully find something supporting the story of white peoples in the early Americas) but they also fully lean in to the idea that humans are inherently fallable and human reason isn't as powerful as faith and feeling. I think this is well demonstrated by the ritual of "bearing one's testimony" in church which is always ways you FELT the holy spirit or by the modern forgiveness (and embracing of) the clear corruption and deception of the original prophet Joseph Smith (like the demonstrably false book of Abraham, for example).
This isn't even supposed to be judgement of one way over the other - I'm just describing the very strong difference between Christian apologists like the author (who want the true truth the right way) and other kinds of proselytizing (that are more supportive of "any road to the right answer").
>You'll never convince someone logically of something that has to be experienced viscerally
"poofy guy in the sky doesnt care about war or rape or torture because someone ate an apple. sent his kid in the form of a human. kid was killed/human sacrificed. that saved all humanity. this doesn't stop rape, war, torture but it saved humanity"
the brain has to do some gymnastics to deal with dissonance
as militant as that sub is; I can't feel but understand them
that subreddit makes a looooot more sense when you treat it like you'd treat exislam or exmormon, people who were intentionally hurt by the church, both Catholic one and local Protestant
In my experience frequenting those boards as a former atheist, the vast majority of the militant atheist types were harmed more by their inability to compete in the standard hierarchies of society than they were by any particular Church or faith.
Most of them are angry at "The Man"—God just happens to be "The Man" at the very top.
I get your perspective and I will admit there are surely cases like that - I still wouldn't say it's a vast majority.
Though that raises a question. For example if you were a LGBT person in Deep South and abused over that, who is to blame? Every individual who did that (unfortunately, in the name of religion), or the Church culture who allowed for that to be norm?
And anger over church. I'm Polish, and I'm bi. The most well-known bishops were politically hand-in-hand with the previous ruling party. I would hear in public state media how I'm a part of "rainbow disease" that's infecting the country. Church would often buy land for 1 (!)% of the price.
At which point can I say that it wasn't just a few bad apples, but the whole structure who allowed for things to be that way?
Jesus is God. Don't ask me how to explain it from a theological standpoint but from my understanding: God decided he wanted to better understand the NPCs in his HumanSim(TM) game so he straps on a VR headset, enters the game via a virgin birth hack, gets killed, then rage quit. Hasn't been seen since so likely started a new game on some other planet.
Actually, as a Christian (Catholic), I 100% agree with your assertion.
While all regions claim truth, Christianity really centers itself on this idea.
In fact, this is repeated over and over again, so much so that I've lost count of the number of times I've heard the following during Homily: "In Christianity is not true, and Christ was proven to not have been risen, we should stop practicing it".
You quoted Saint Paul which is great and one can find plenty of other examples such as Lewis's trilemma: "Lord, Liar, Lunatic". [0]
The church has a long history of defending and clarifying the principles and practices of faith in a relatively academic way, in a way that would surprise people who believe religion is an anti-intellectual endeavor.
I remember reading something along the lines of, Jesuits will have a perfectly ironclad logical argument to any theological point... Assuming you have already swallowed the 1-2 big rocks at the foundation.
I would say there are both intellectual and felt needs. Apologetics is kind of its own genre of rhetoric and doesn’t satisfy everyone. It can be better to offer to research an objection together, rather than try to win it (more fun, too.)
Religious debates in a friendly setting can also lead to personal sharing that opens someone to emotional or spiritual experiences.
Minimizing intellectual concerns by overriding with an emotional experience does not “work”, not long-term and certainly not over generations.
But there are also people who have few or no intellectual concerns! Caveats all the way down.
> Nothing true can be said about God from a posture of defense.
This is a self-defeating statement. The statement itself is a claim about God. If it is true, then it contradicts itself because it asserts a truth about God (namely, that no truths can be defended about Him). This undermines its own premise.
If it's not subject to itself, then it's claiming an exception to its own rule, which makes it self-defeating. A universal claim about truth can't exempt itself without contradiction.
The distinction between "statements about X" and "statements about statements about X" doesn't resolve the contradiction here, because the original claim makes a universal assertion about all possible true statements about God. By its own terms, it must encompass both:
1. First-order statements about God
2. Second-order statements about statements about God
3. All higher-order statements as well
Put differently: The claim "Nothing true can be said about God from a posture of defense" is asserting that there exists no level at which defensive truth claims about God are valid. But this assertion itself constitutes a defensive truth claim about God—whether at first, second, or any other order.
The attempt to escape via orders of statements fails because the original statement's scope explicitly covers all truth claims regarding God, regardless of their logical level. If it didn't cover all levels, then it would no longer be claiming that "nothing true can be said"—it would instead be claiming "some things true can be said, just not first-order things," which is a fundamentally different claim than the original.
So by attempting to exempt itself based on being "meta-level," the statement has already conceded that some true things can be said about God from a posture of defense (namely, meta-level things)—which directly contradicts its own absolute claim that nothing true can be said.
This is why universal claims about the impossibility of certain types of statements are often self-defeating—they cannot consistently exempt themselves from their own scope without undermining their universality.
> But this assertion itself constitutes a defensive truth claim about God
You seem to be asserting this without any proof, or at least none that I can follow. The assertion itself is a "defensive truth" about "truths about God," not about "God." I'm not sure how you are justifying considering "truth about [truth about [truth about [... X]]]" as the same thing as "statement about X".
Is it possible that you're considering the properties of the truths about X to be properties of X as well? I don't think this is justified. Properties of truths about X come from properties of X. For example, statements about the color of X are not statements about X itself, despite coming from properties of X. E.g. color(X) = color(Y) -/> X = Y.
Recursion doesn't complicate the case here. The original statement made assertions about "statements about God", not about God. For example, the statement "All statements about God are false" is not paradoxical, it is simply false (if we accept the law of the excluded middle). A statement like "Everything I've said about God is false" could very well be true, it's not paradoxical, despite also being part of the set "Everything I've said;" it's just not "about God."
Thinking about your argument a little more, it seems like our disagreement comes from your belief that "about X" is 'infectious' to all higher order statements, whereas I don't believe this is the case. The best way I can think to argue my point right now is from examples.
Suppose we had many books about movies on one hand, each book containing movie reviews or something, and then we have one book about [books about movies] on the other hand, call it B. The book B, which is about [books about movies], simply contains the number of words that each book about movies has written in it. Is B "about movies"? I would argue that it is not, it contains nothing about movies in it at all, just numbers describing other books. I can say "all books about movies are wrong" without meaning to refer to B, as B is not wrong (as long as the word-counting is correct).
Your examples attempt to break the chain of "aboutness" between meta-levels of statements. But there's a crucial distinction your argument misses:
"Nothing true can be said about God from a posture of defense" isn't merely describing properties of statements like your examples do (word counts, colors). Instead, it's making a universal claim about POSSIBILITY itself, specifically, the impossibility of defensive true statements about God.
This raises a key question: What makes defensive truth claims about God impossible? This impossibility must stem from something about God's nature itself. Otherwise, what grounds the impossibility?
Your examples all involve contingent properties:
1) Book word counts are contingent features of books
2) color(X) = color(Y) involves contingent properties of objects
But the original statement makes a necessary claim about what kinds of truth claims about God are possible at all. This is fundamentally different because:
1) It rules out ALL possible defensive true statements about God
2) The basis for this universal impossibility must lie in God's nature
3) Therefore it necessarily makes a claim about God, not just about statements
This is why property inheritance examples don't apply here. The statement isn't claiming properties transfer between levels, it's making a universal claim about possibility itself that necessarily involves both statements about God AND God's nature.
While you've shown that descriptive properties don't transfer between meta-levels, this doesn't address the key issue: a claim about what statements about God are possible must ultimately be grounded in God's nature itself.
> a claim about what statements about God are possible must ultimately be grounded in God's nature itself
Ah, so you're saying that the original statement "Nothing true can be said about God from a posture of defense" (call this statement "O") is self-defeating not because it applies to itself, but because it applies to one of its (lower-order) premises (call it "P"), which is itself about God? This is different than your original argument "The statement itself is a claim about God" but we can go with this as well.
I think your new argument relies on that premise P being made "from a posture of defense," otherwise O doesn't apply. I don't see any evidence of this being the case, so I don't think your argument is correct.
In other words, two conditions need to be satisifed for P to apply to a given assertion:
1) The statement must be about God.
2) The statement must be made from a posture of defense.
P doesn't satisfy (1) and O doesn't (have to) satisfy (2). Am I wrong, or what am I missing here?
From where are you deriving your sense of "pointlessness"? If there is no absolute standard (such as God), then all truths, including moral truths, are determined by individuals or societies, making them relative.
To many the idea of bringing the intellect fully into action in religion seems almost repellent. The intellect seems so cold and measured and measuring, and the will so warm and glowing. Indeed the joy of the will is always figured in terms of warmth – such words as ardor, fervor and the like come from Latin words for a fire burning: there is a fear that intellect can only damp down the fire. Many again who do not find the use of the intellect in religion actually repellent, regard it as at least unnecessary – at any rate for the layman – and possibly dangerous. One can, they say, love God without any very great study of doctrine. Indeed, they say, warming to their theme, some of the holiest people they know are quite ignorant. Plenty of theologians are not as holy as an old Irishman they have seen saying his Rosary. All this is so crammed with fallacy as to be hardly worth refuting. A man may be learned in dogma, and at the same time proud or greedy or cruel: knowledge does not supply for love if love is absent. Similarly, a virtuous man may be ignorant, but ignorance is not a virtue. It would be a strange God Who could be loved better by being known less. Love of God is not the same thing as knowledge of God; love of God is immeasurably more important than knowledge of God; but if a man loves God knowing a little about Him, he should love God more from knowing more about Him: for every new thing known about God is a new reason for loving Him. It is true that some get vast love from lesser knowledge; it is true even that some get vast light from lesser knowledge: for love helps sight. But sight helps love too.
After all, the man who uses his intellect in religion is using it to see what is there. But the alternative to seeing what is there is either not seeing what is there, and this is darkness; or seeing what is not there, and this is error, derangement, a kind of double darkness. And it is unthinkable that darkness whether single or double should be preferred to light.
Indeed light is the joy of the mind as warmth is the joy of the will. But warmth and light are both effects of fire, warmth fire as felt, light fire as seen (and seen by). It seems strange to value the one effect and not the other of that fire by which the Holy Ghost is figured to us. It is an odd delusion that one is warmer in the dark, that one can love God better in the dark.
There are a lot of good theologians who are uncomfortable engaging in apologetics. It's easy to get immersed in the debate, and turn towards heresy or dishonesty without noticing. This precedes AI.
From an outside perspective, that quote takes on a whole other meaning. Namely, that very few statements about "God" don't have similar but contradicting statements - making it (almost?) impossible to say anything definitive about them at all (from a SAT solver/proving kind of perspective).
PKD, KW Jeter, and others as well. Those two I know used, as a story element (sometimes throwaway, sometimes key) the idea of robots/androids/computers trained on historical figures and other people. In one of Jeter's books, the figures were used in education so you'd ask "Abraham Lincoln" questions about his life (it's been 20 years since I read it, can't remember the specific figures used or if it was just "historical figures").
Obviously this predated LLMs, and they probably called it "programming" and not training, but the idea has been present in scifi for decades.
Star Trek: The Next Generation did this several times with fictional, historical, and (on at least one occasion) contemporary people (to the story).
None of these words (plausible, hallucination, convincing) seem appropriate.
An LLM seems more about "probable". There's no truth/moral judgment in that term.
When weighting connections among bags of associated terms (kind of like concepts) based on all the bags the model was repeatedly browbeaten with, LLMs end up able to unspool probable walks through these bags of associated terms.
It's easy to see how this turns out to work "well" for bags of terms (again, sort of concepts) often discussed in writing, such as, say, Christian apologetics.
Instead of the complaint and examples he blogged, he should have dumped a position article he agrees with, and a position article he disagrees with, into it, and asked it to compare, contrast, contextualize, opine, and then (for kicks) reconcile (using SOTA Claude or OpenAI). He's using it as a concordance, he could have used it as, well, apologetics: a systematic (not token based) defense of a position.
Because breaking down the bags into our alphabet and our words isn't really how LLMs shine. Smash together concepts like atoms and you can actually emit novel insights.
This article describes something LLMs are bad at — a fancy "how many Rs in strawberry".
Asking LLMs about spelling is like asking people to echolocate or navigate using the earth's magnetic field. They can't see it. It's a sense about the world that they don't have.
That's a known flaw that builders have decided to swallow, as opposed an intentional aspect of the design. The intent of the design is to generate text that humans find convincing. If they could tweak the design to remove tokenisation flaws, they'd do it instantly.
> he should have dumped a position article he agrees with, and a position article he disagrees with, into it, and asked it to compare, contrast...
This is like saying that pen testers shouldn't use special characters in API requests. I don't think the author's goal was to showcase an optimal use-case, but how LLMs can easily and unwittingly provide incorrect information. Of course this is already known, but it sounds like he felt obliged to demonstrate it for this specific case where the creator claims that it is robust.
Somewhat related, I built a local llm fine tuned on some open published literature on metametaphysics via philpapers.org, and I was surprised at how well it would respond. I think that philosopher's tendency to overexplain every fine point is like the world's greatest prompt engineering (possible side gig for my underemployed philosophy bros).
We tried to partner recently to co-create interactive books with a well-known classical education influencer on X.
His job was to select passages from books and provide some commentary. Ours was to turn that material into an interactive title.
We tried tirelessly to look up the passages he'd send to us in the original text. The only quotes that matched were the top 5% short and famous quotes. The rest was made up completely, presumably by AI.
His 1 million+ followers consider him a world-class subject matter expert. But he doesn't read any of the books he's teaching. Eye opening.
Are you saying he posted scripture that didn't exist? Who is this? I actually don't believe that. Many people are surprisingly good with their scripture and would immediately spot someone quoting non-existent scripture.
Unless maybe he was quoting some non-canon like Mormon books. Very common for "Christian" influencers to be LDS.
i can believe this. many "experts" are consistent bullshitters, and it helps them to look more like experts.
this is not always intentional either, and there is a lot of social pressure to do it.
have you ever read a popular article about something you have expert knowledge in? the general standard for accuracy and quality in public discourse is mindblowingly low.
But you can't make up scripture without being immediately spotted. Their comments would be flooded with people calling them out.
It's like trying to talk about star trek episodes that don't exist with star trek nerds. There would be a few seconds of confusion before the righteous indignation.
There are many religious questions people probably are scared to ask a human. Imagine any closeted gay or trans kid wanting to know if the way they feel is “wrong.” Just asking the question to your parents or a faith leader could raise suspicions and be dangerous.
So while I wish the world were one in which people didn’t have a reason to be scared to ask questions, that’s not the world we live in. Are LLMs an answer? I have no idea, but I understand the impulse to try.
Or worse, who would track people with questions like this and try to "teach" or "purify" them. Even without violence this is vile, and yet I fully believe this happens just like some US states spending tax money trying to estimate constituent periods to see if they may have had an abortion.
Yeah... I hate to be a downer in this but neither humans nor faceless tech are good approaches to keep someone safe, especially when they are hated just for who they are by a major faction.
> Imagine any closeted gay or trans kid wanting to know if the way they feel is “wrong.” Just asking the question to your parents or a faith leader could raise suspicions and be dangerous.
Depends on the religion and the place. A Catholic could ask in the confessional with the seal of secrecy. Anyone could ask online anonymously. In most churches I know it would not be dangerous to ask questions, and you can do in confidence one way or another.
I really do not think LLMs are anything like the answer for people grappling with personal issues. You need someone supportive with empathy.
Also, who trusts an LLM to give them the right answer? Can an LLM guide someone gently to think things through? Can it refuse to give someone a black and white answer when they want a simple answer to a complex question?
I think an LLM is the dangerous option.
Different for people in very conservative cultures, of course.
It's an interesting approach but also a big contrast to the typical community style formula for religion / even just different and disconnected sort of evangelism where you have no idea what the result is...
I think people just try things sometimes, and that this AI is be one of them. In the current times, people put AI everywhere, especially where there is textual communication involved. I think this effort fits well into this trend.
I had to use an LLM to find out what a Christian Apologetic was. Apparently this means people or groups who seek to actively defend Christian beliefs (presumably against skeptics, atheists, etc) through formal reasoning and evidence.
It's usually not designed to persuade skeptics, it's designed to persuade people who are on the fence about their faith, both the author and those who, like them, need an intellectual justification.
Nah, not even that. The only people who consume apologetics sincerely nowadays is Christians. They're not meant to convince anyone, they're meant to keep the crises of faith at bay.
I think you and I are saying the same thing. Apologetics are directed at current Christians who are on the fence and need an intellectual justification to stay.
"On the fence" is too strong a phrase for this situation, I think. They don't consume apologetics to push themselves back in, but rather to keep themselves completely immersed in the religion.
Nope, I'm speaking from personal experience with myself and others: apologetics is to keep those on the fence from jumping over entirely, and it does often work.
These are great points, but...wouldn't they pretty much apply to any proposed use of LLMs for disseminating any kind of information?
I suppose I would expect (or at least hope) that a Christian apologist would be more sensitive to these issues. But really, if it is unethical to make up bullshit (using the article's terminology) for Jesus Christ, what exactly is it ethical to make up bullshit for?
according to the article- things that Must be immediately verified, and brainstorming for things like creative writing where making something up is the point
> What this really demonstrates is an intention to carry on no matter what – that whatever arguments or evidence he sees, nothing will make him change course. I hope that won’t be true in practice.
Christianity, like all religions will need to explain technology like generative AI within the context of their theology and their cosmology. Is it a force for good or evil? Will we critically misuse the technology in the future and will that misuse color all efforts to use the technology as a tool for proselytization as fundamentally evil? Will generative AI allow all religions to quickly reach critical mass or will public sentiment of religion quickly drop to telemarketer levels?
Doesn't the section where the author goes through the LLM completely fabricating or confounding different parts of scripture give you pause about the truthfulness of the responses?
It's asked about what a specific version of a text says and gets that wrong. It doesn't particularly matter how correct or carefully copied that scrap is, the machine can't faithfully reproduce or quote a particular version. In fact it made up a quote that doesn't exist in any version of the Bible the author could find at all.
As we build and understand them now there's pretty good structural reasons to believe that LLMs cannot be tweaked or tuned to be more than incidentally truthful.
If you use an LLM only solution, sure, but there are many more opportunities if you expand the system to include more.
We could, for example, use the existing tools for copyright detection, on the output and refuse to send it to the client side. It's just another moderation check in that perspective
Copyright protection software is notably bad because it requires human context and decisions to decide infringement from fair use. Say we ignore that part that's the lowest hanging fruit on the improving generative models tree. Deciding if something is true to have the bot not say output it if it isn't is waaaay harder and the thing needed to fix the problem we've actually been talking about.
If you are asking that question seriously check out the recent Rogan podcast with Wesley Huff. Has the usual Rogan conspiracy theory pontificating, but also some good discussion about the history of how we got the Bible we have today.
First off, the claim that the original text was first written 100s of years after the words were spoken is completely false and isn’t taken seriously by anyone in the field.
While LLMs are in generally fairly good at recalling bible verses, they can't do it perfectly. If the Bible truly is the infallible word of God that we believe, then shouldn't we use more caution than just "welp, sometimes it makes mistakes"?
You could counter this by saying a person can't remember Bible verses, and this is true, but a person usually recognizes when they can't remember something instead of making something up. If you asked me to recall any random Bible verses, chances are I wouldn't be able to do so. However, unlike an LLM, I would admit I don't know for sure and I would pull out a Bible or look online for an authoritative source rather than adlibbing something on the spot.
The real question in this context might be 'can LLM output be divinely inspired'?
If we charitably accept that all holy texts are written by human beings in the grip of divine inspiration (as compared to texts magically appearing on the tops of mountains), in a manner opaque to tools of scientific inquiry such that no such divine interventions can be detected or measured, then why can't LLMs also be used in this manner by the divine presence?
Is it thus possible to create a new 'true' religion, with a holy text generated by a consortium of LLMs passing prompts and outputs back and forth among one another, with a few stipulations such as 'start with an origin story of the universe'?
Absolutely valid point, but I think that from a Christian POV, the question comes down to an entity having a soul, or you know, the something that makes humans somehow distinct to animals. This, let's call it a soul, is I think needed so that God has something to grip in order to deliver the divine inspiration. Following up, the LLM then needs to have this "soul", so that it can be inspired, so that it can deliver proper divine revelations. The question really is, where does this soul begin, and end, and what does it need to form an attachment to something. Can a human be born without one, can something, that is not a human, have one at any point? And again, as you say, how to test for this? It would really be a shame to be fooled by an entity, saying that it delivers holy texts, just to find out later that it was the Antichrist or something, and just a test of faith.
Can you explain why a base 2 number representation would help with interpreting dichotomies? They seem entirely unrelated other than the number 2 being vaguely involved.
It seems it would be OK if it were limited to being a research assistant and everything produced was rigorously checked. But because AI has no concept of truth, ethics or morals, I would limit it to that alone.
An AI is not capable of not lying, which seems to violate the Ninth commandment. I am not an expert, so interpreting the "against thy neighbor" portion may or may not apply here. I tend to interpret that as "no lies" in any circumstances.
I do realize that because of the way our brains, etc. work, that it's pretty much impossible to not lie, at last a bit, but that is what forgiveness is for. I don't think you can forgive software, since it really should be a correctable issue. Of course, LLMs are so complex as to be uncorrectable.
>I see no reason why AI can’t be a part of the discussion to interpret Scripture in a modern context.
This isn't what the article argued against. The product was being marketed in a very specific way that seems to undermine a group identity the author is part of.
To answer the title, I think Christians should use AI as little as possible. More like a research tool or generating templates. The reason is that the Holy Spirit works through God’s Word (or truth), His people, and Christ-like character with human connections. AI can only deliver one.
Also, Christians always face persecution which includes people trying to censor or attack them. A common method, which I’ve experienced, is misrepresenting their words or actions. Paul kept saying they were blameless among them, just told the truth, and didn’t use worldly tricks. Our integrity, including not using fake things, is both a defense and a positive example in an increasingly, fake world.
Far as training them, here’s a few sites with material that might help:
While I've seen all the points made in the article (besides some that wouldn't work outside of this particular religious context), they are well laid out and highlight not only the technological limitations, but also the moral implications of building a llm product.
Moral issues that are not limited to religion, I've had conversations with a friend where I was trying to explain to them that their llm website idea will, with certainty, mislead people at some point. I think the exact idea was to build a gpt-based trip planner. I'd hate to have someone spend time and money on hotel, transportations, etc... To end up stranded in the middle of nowhere.
Anyway, the article came out 4 months ago and the chatbox is still there.
> Which programming language would allow me to get closer to God?
> If I had to choose a programming language that reflects some aspect of God's nature, I would say that a language with a strong emphasis on order, logic, and beauty, such as Python, could be seen as a reflection of God's character. The Bible says, "God is not a God of disorder but of peace" (1 Corinthians 14:33, ESV), and Python's simplicity and readability can be seen as a reflection of this aspect of God's nature.
We also know that the Apostles all shared a Honda accord too, Acts 5:12: “They were all with one Accord” and that God was more of a classic car guy and drove a Plymouth Fury, from Jeremiah 32:37: “He drove them out of the land in His Fury”.
Forth? God likely programs in machine language, as even Assembly is a crutch for mere mortals to glimpse divinity. Even more, it looks like he might be a computer himself!
> John 1:1 In the beginning was the Word, and the Word was with God, and the Word was God.
I'm going Devil's Advocate (ha ha) and point out that some people believe suffering makes us closer, and worthy, in the eyes of God. Even Mother Teresa was criticized for thinking that suffering was a gift from God, so she didn't consider analgesics and overall comfort a priority.
IIRC, analgesics/comfort were actually considered but Mother Theresa’s hospices couldn’t afford it. They were in the slums of India decades ago. Hence, why they were hospices not hospitals
Ah yes, go, a language created by one of the most powerful modern institutions. Its rules and dogma are pretty strict and it’s hard to change them without being in the inner circle. Some people on the outside obsess over the language and sing praises over the rules it imposes while others spend their days endlessly complaining about them.
This would be a perfect language to represent Christianity.
The 3 branches of government were based on Isaiah 33:22: For the LORD is our judge, the LORD is our lawgiver, the LORD is our king. It sounds like a language that includes rules, but I don't think it's been invented yet.
I'm surprised nobody has mentioned Perl yet. It is named after the Parable of the Pearl from the Gospel of Matthew. But Larry Wall dropped the E because he discovered there was already another language called PEARL (a rather obscure real-time programming language from Germany)
Give not that which is holy unto the dogs, *neither cast ye your pearls before swine*, lest they trample them under their feet, and turn again and rend you. (Mat 7:6, KJV)
Not AI but when I was looking into vocaloid music (the singer is 100% synthetic) I was surprised how much christian praise music on Youtube was using it. It seems the antithesis of being brought closer to god/a higher being and so weird. But today seeing this, it's already no as weird as it should feel.
I know my mennonite friend (less strict Amish) has been having discussions in his church group about how to use AI for their bible studies, teachings. They're a group of random old dudes in the middle of nowhere so they aren't great speakers/motivators/explainers. But I can't imagine being lectured on god by a robot is going to be better even if it's done in pretier prose.
Anyways if you want to hear spiritual music without any human connection to spirituality but also anime girls search 'vocaloid christian songs' on Youtube.
Anyways if you want to hear spiritual music without any human connection to spirituality but also anime girls search 'vocaloid christian songs' on Youtube.
You have stumbled upon the cultural phenomenon known as "Japan".
I understand the author's conserns but I wonder of his standard of "true" is a bit unrealistic. Maybe the standard should be "less false than the average human produced work."
Yes a human expert who is devoted to uncovering and communicating truth will do much better than an llm. But that is not what most of the content in the world consists of.
Most content is written by people with questionable expertise who write with goals like fame or viewership numbers or advertising dollars. Most content on the internet or in the world does not seem to be written with the goal of uncovering truth.
To create a more truthful world AI doesn't have to be as accurate as an expert. It just has to be more accurate than average. And in many areas, that's a very low bar.
To choose a less controversial area than religion, I think adding AI to nutrition advice on the internet will be an improvement. It's true you will inevitably get inaccurate advice communicated with confidence, but any AI trained on any kind of credible scholarship for will still be miles ahead of Internet nutritional advice that is often filled with zany pseudoscience.
Yes it will produce inaccurate results but on average the information will be more accurate than what already exists and what is currently being perpetuated
If you genuinely believe that failure to believe in Christ means an eternity of punishment, anything that might feasibly turn off a potential believer - like an AI hallucination posing as a religious explanation - is fundamentally worse than murder.
In religious communities we would call such an attitude scrupulosity. It refers to a sort of religious OCD, a paralyzing fixation on the rules and a demand for unrealistic standards of perfection.
According to Christian teaching, such an attitude comes from a lack of faith. A lack of belief that God is ultimately in control, and an understanding that for all the imperfections and flaws that we as humans inevitably bring to the table, god can still work through those imperfections.
In some domains "good enough" or "the ends justify the means" are acceptable. In this domain the author clearly feels there is a significant moral requirement to satisfy.
> Maybe the standard should be "less false than the average human produced work."
I don't think so. Lots of people blindly trust LLMs more than they trust the average human, probably for bad reasons (including laziness and over-reliance on technology).
Given that reality, it's irresponsible to make LLMs that don't reach a much better standard, since it encourages people to misinform themselves.
If you literally believe that getting an answer wrong can send someone to hell, then I think the stakes are a little more dire than bad nutritional advice.
Like imagine if this were a baby-care bot, dispensing advice on how to safely care for your baby. That would be pretty stupid, and would likely eventually give advice so incorrect a baby would die. For someone who believes, that is a less tragic outcome than being led astray by an apologetics bot. It takes an incredible level of conceit to build one anyway.
chat GPT and Gemini both provide advice on how to care for babies. I think in the end you have to have faith in people to also use their common sense, discretion and not blindly believe or act on everything a bot tells them.
I think the same is true for an AI giving religious advice - you have to exercise a bit of faith in the readership and, perhaps in this case , also faith in the ultimate guidance of the divine. Faith that they're not going to make serious mortal or religious decision by unquestioningly following a chatbot
If we take this thinking to its logical conclusion we should put all our efforts as a civilization to getting rid of all misinformation that may harm babies whether online or spoken. And every religious person should do nothing but have flame wars and censorship campaigns about any flase religious information that has any chance of affecting a person's salvation.
The author seems to be in a purity spiral and seems to be taking an overly hardline interpretation of the religion
As someone who was raised extremely religious, strayed to the polar opposite, and is now trying to find my way in between the two, I do find this interesting. While the understanding of LLMs and when/how to apply them makes sense, I would argue that they fit right alongside human interpretation of scripture. Consider that many pastors "teaching" scriptures aren't even formally educated.
Arguing that you can't use an LLM for Christian apologetics because it "might not be true" overemphasizes the definition of "truth" when it comes to scripture and those teaching Christian apologetics, which is entirely influenced by what doctrine you subscribe to.
1. This interpretation might not be "true" but it is a good-faith effort that respects the text.
2. This LLM is fabricating verses, chapters, and even books of the Bible.
If you've used LLMs much, you know that #2 is not only possible, its quite common. This is the kind of "might not be true" that you should be aware of when using an LLM for apologetics—or any effort where "truthiness" is important.
I do agree point #2 is possible, but is that not something that could be accounted for and tuned in the model to an extent? I believe a tuned LLM would be able to distinguish between interpretation and generation of scripture, but I may be wrong.
While I kinda agree, many denominations are also questionable in the "good-faith effort that respects the text" department. Some like making up new chapters while others overemphasize specific short sections while disregarding many others.
Much more so than any decent LLM ever would. Even while they're "making stuff up", they largely stick to the general themes.
> Arguing that you can't use an LLM for Christian apologetics because it "might not be true" overemphasizes the definition of "truth" when it comes to scripture and those teaching Christian apologetics, which is entirely influenced by what doctrine you subscribe to.
But the author is pretty explicit about wanting a high standard (e.g. insisting on using the best sources possible), and doesn't think that using LLMs is compatible with that goal.
Don't feel obligated to ask if you are not comfortable, but why are you trying to find a middle ground? When you say middle ground, what does that mean for you? Does that mean you have some faith but maybe not in a particular sect?
I mean I found great relief from high-demand Evangelicalism via giving up on my belief and seeking intense therapy from the traumas I experienced. However, as both I, my wife, and our kids age, we feel a need for some framework to live by, to some extent.
The friction for me is I am just a very logical and evidence-driven person by nature, so while I recognize (at least for me) that there are some benefits to adhering to some religion or spirituality, the core Christian belief is really difficult for me and always has been. So I don't really know yet where I'll land.
Why do you need to adhere to religion or spirituality in order to establish a framework to live by? Many atheists and agnostics find great meaning and moral guidance from frameworks that at no point involve religion or spirituality.
Well, not according to the author of the post, and we know how concerned he is with truth, so this must be correct? “That leaves us with atheism, which provides us with not the smallest scrap of a foundation on which to build any claims about the purpose of life, or what is a good or bad.” [0]
This is confusingly supported by a quote from C.S. Lewis making the point that it’s better to believe in something that “feels important”, whether or not it’s true.
Which is wild, because I'm an atheist and I believe in things that feel important even if I don't necessarily consider them "true" (mostly things about how I should treat other people and that I have agency over my life).
I don't understand how you came to this conclusion. Some of my differences with Christianity are that I believe that morals should come from thought, reasoning, and empathy, not from hierarchy. That I don't think there's an outside supernatural force that's perfect in every way, but also explicitly flawed, that's characterized by infinite love, forgiveness, and benevolence but is less loving and forgiving of "sinners" than I am, and that sets people up to fail that they might suffer forever after.
Accepting that there are things it is useful to believe that are not necessarily literally true actually separates me further from Christianity, since they insist that all their beliefs are literally true, and that doubting such is grounds for eternal torture.
Most atheists and agnostics struggle greatly to replace the meaning and moral guidance provided by religion.
First and foremost the community aspect. There are countless benefits to being part of an active faith community that atheists have had a very hard time replicating.
I replicate my community where I find it. Some of it is at a brunch spot I go to regularly. Some of it is a bar that I frequent. Some of it is in annual activities with my neighbors.
I don’t need to believe in a made up Sky Daddy to be a good person, I have plenty of examples in my communities.
Also, some people have 'religious experiences' that set their minds on certain paths. Whatever the underlying neuroscience is, it results in qualia that is hard to ignore.
I'm a 2nd generation atheist who suffers from depression. Years ago I read evidence that religious practices could alleviate it. I was kind of desperate so decided to brainwash myself into becoming a believer for about a half of a year. Overall, it was pretty effective. I had to eventually disengage because of the cognitive dissonance, but the positive effects have lingered.
I'm also a very logical and evidence-driven person by nature who has gone through similar since losing my faith. It's not for everybody, but I've gotten a lot of enjoyment from Stoicism. There are tons and tons of books on the subject that vary in quality, and unfortunately I can't recommend any specific ones because at this point I've read so much of them that I don't remember where certain ideas came from, but I definitely recommend the writings of Seneca. Marcus Aurelius' writings are great too.
It's so funny you mention that because I picked up the Daily Stoic a couple of years ago and have read it off and on. I have found it really enjoyable. Thanks for the rec!
In the same boat here: 20+ years of hard core Christianity (Scottish Presbyterianism). Heavy, very heavy indeed studies lasting years and years. Regulative principle of worship, this kind of direction.
Was raised kinda an atheist though and converted in my 30s. A willful, well thought out decision to convert.
All came crashing down on me the moment I stopped ignoring some very obvious questions, e.g. who died on the cross?
Even some casual thinking about this lands you, inescapably, on the only conclusion you have available if you stick to the orthodoxy, and that is: a human nature died on the cross. Not God (cannot die) and, unfortunately for Christianity, not a human either (briefly: if JC is one person / two natures, you have to conclude his (human) nature died on the cross since JC the person, being God, cannot die).
At any rate, this is where it started for me and quickly escalated further. The entire New Testament, I'm convinced now, is a fraud and whoever pulled it off didn't even try to hide it. It's incredible how we can bullshit ourselves into believing what we (for whatever reason) want to believe. And not just religion.
In the end, the NT had to go leaving me with the Hebrew scriptures.
But this isn't just claiming to produce some benign facts, it is trying to make claims on absolute truths with consequences as dire as "going to hell".
Even if you don't believe, the creators certainly intend for their bot to have eternal consequences. Like selling an LLM with the claim it can give advice better than most doctors and should be used as such, the intent behind the apologetics bot is just as reckless and conceited.
That's not really what the article is talking about. The article is referring to the fact that you can ask for specific verses in some version/translation (that are KNOWN) and the potential for the LLM to confidently generate a completely fabricated or subtly different copy.
And going a step further, any follow-up questions to the LLM will be using this incorrect copy as the source for interpretation causing it to go even further in the wrong direction.
Incidentally - this was occurring using a custom fine-tuned model with an added layer of RAG.
The main example he gives is a simple factual matter about the words a specific early Christian manuscript. The LLM invented new text that’s not at all what’s in the manuscript.
He also convincingly argues people performing poor apologetics is no excuse to deploy an LLM performing poor apologetics.
281 comments
[ 6.2 ms ] story [ 267 ms ] threadIt's quite a different level of commitment to truth than one usually runs across these days. Instead of a world of meme and vibes, a costly commitment to only say things that are true.
It worries me a lot more that governments and the like will also be in denial about what they can do with AI. I can ignore low quality apologetics, I cannot ignore the government (I have to pay taxes, for example).
I feel like the way around this is hermenutics and exegesis and that's where things differ substantially between the bible and llms. The bible has both whereas LLMs are arguably approached hermenutically, but have no exegesis to speak of.
If accuracy and reliability don't matter, go for it.
If they do, but you can easily verify the output, then go for it.
So, yes. The core argument would apply to making inquiries about any ancient religious texts especially if you aren't already able to read the generated text (it's in a language you aren't fluent in) or are unable to access its claimed references to verify what it says they say.
Apologetics is about a lot more than texts. It includes philosophical arguments (e.g. for the existence of God), non-scriptural texts, and personal experience.
I do not think that LLMs are any better equipped to deal any of those other aspects of it either - especially not personal experiences.
- from the novel Gilead, by Marilynne Robinson
I take this quote to mean that most people's idea of "apologetics" (arguing to convince people that the facts of Christianity or some other religion are true) is kind of pointless. You'll never convince someone logically of something that has to be experienced viscerally. LLMs don't help with that at all.
While you might be right, Christianity in particular is based on truth claims, including specially the resurrection, so the Christian tradition places special emphasis on rational defense. Apologetics is not just a means to persuade others; it is also a means to persuade oneself.
Edit: Responses say that all religions involve truth claims. True, perhaps I was imprecise. I only mean that the Christian case is especially stark. St. Paul: "If Christ be not raised, your faith is in vain." I'm not aware of another faith tradition that considers itself to hang upon a single boolean.
As opposed to...? I'm not aware of any major religion that openly admit their teaching is made up BS. All seem to claim to know "the truth".
As opposed to Buddhism, Taoism, Shinto, Jainism, etc.
There are ways of interpreting and practicing Buddhism in which this is less important (eg modernist, psychological, western), but that's also true of Christianity (eg the Jefferson Bible, Unitarians, etc).
Do you have a citation for this? I am reasonably familiar with Mormonism (The Church of Jesus Christ of Latter-day Saints) and I don't think I have ever heard this before?
The Mormon church, of course, has its wings of scholars trying to find evidence to support their origin stories (like looking for their god's original home planet or buying ancient South American temples to hopefully find something supporting the story of white peoples in the early Americas) but they also fully lean in to the idea that humans are inherently fallable and human reason isn't as powerful as faith and feeling. I think this is well demonstrated by the ritual of "bearing one's testimony" in church which is always ways you FELT the holy spirit or by the modern forgiveness (and embracing of) the clear corruption and deception of the original prophet Joseph Smith (like the demonstrably false book of Abraham, for example).
This isn't even supposed to be judgement of one way over the other - I'm just describing the very strong difference between Christian apologists like the author (who want the true truth the right way) and other kinds of proselytizing (that are more supportive of "any road to the right answer").
"poofy guy in the sky doesnt care about war or rape or torture because someone ate an apple. sent his kid in the form of a human. kid was killed/human sacrificed. that saved all humanity. this doesn't stop rape, war, torture but it saved humanity"
the brain has to do some gymnastics to deal with dissonance
dude created rape, torture and human sacrifice and gets less negative press than Trump and Elon
that subreddit makes a looooot more sense when you treat it like you'd treat exislam or exmormon, people who were intentionally hurt by the church, both Catholic one and local Protestant
Most of them are angry at "The Man"—God just happens to be "The Man" at the very top.
Though that raises a question. For example if you were a LGBT person in Deep South and abused over that, who is to blame? Every individual who did that (unfortunately, in the name of religion), or the Church culture who allowed for that to be norm?
And anger over church. I'm Polish, and I'm bi. The most well-known bishops were politically hand-in-hand with the previous ruling party. I would hear in public state media how I'm a part of "rainbow disease" that's infecting the country. Church would often buy land for 1 (!)% of the price. At which point can I say that it wasn't just a few bad apples, but the whole structure who allowed for things to be that way?
Jesus is God. Don't ask me how to explain it from a theological standpoint but from my understanding: God decided he wanted to better understand the NPCs in his HumanSim(TM) game so he straps on a VR headset, enters the game via a virgin birth hack, gets killed, then rage quit. Hasn't been seen since so likely started a new game on some other planet.
In fact, this is repeated over and over again, so much so that I've lost count of the number of times I've heard the following during Homily: "In Christianity is not true, and Christ was proven to not have been risen, we should stop practicing it".
You quoted Saint Paul which is great and one can find plenty of other examples such as Lewis's trilemma: "Lord, Liar, Lunatic". [0]
[0] https://en.wikipedia.org/wiki/Lewis%27s_trilemma
Religious debates in a friendly setting can also lead to personal sharing that opens someone to emotional or spiritual experiences.
Minimizing intellectual concerns by overriding with an emotional experience does not “work”, not long-term and certainly not over generations.
But there are also people who have few or no intellectual concerns! Caveats all the way down.
This is a self-defeating statement. The statement itself is a claim about God. If it is true, then it contradicts itself because it asserts a truth about God (namely, that no truths can be defended about Him). This undermines its own premise.
1. First-order statements about God
2. Second-order statements about statements about God
3. All higher-order statements as well
Put differently: The claim "Nothing true can be said about God from a posture of defense" is asserting that there exists no level at which defensive truth claims about God are valid. But this assertion itself constitutes a defensive truth claim about God—whether at first, second, or any other order.
The attempt to escape via orders of statements fails because the original statement's scope explicitly covers all truth claims regarding God, regardless of their logical level. If it didn't cover all levels, then it would no longer be claiming that "nothing true can be said"—it would instead be claiming "some things true can be said, just not first-order things," which is a fundamentally different claim than the original.
So by attempting to exempt itself based on being "meta-level," the statement has already conceded that some true things can be said about God from a posture of defense (namely, meta-level things)—which directly contradicts its own absolute claim that nothing true can be said.
This is why universal claims about the impossibility of certain types of statements are often self-defeating—they cannot consistently exempt themselves from their own scope without undermining their universality.
You seem to be asserting this without any proof, or at least none that I can follow. The assertion itself is a "defensive truth" about "truths about God," not about "God." I'm not sure how you are justifying considering "truth about [truth about [truth about [... X]]]" as the same thing as "statement about X".
Is it possible that you're considering the properties of the truths about X to be properties of X as well? I don't think this is justified. Properties of truths about X come from properties of X. For example, statements about the color of X are not statements about X itself, despite coming from properties of X. E.g. color(X) = color(Y) -/> X = Y.
Recursion doesn't complicate the case here. The original statement made assertions about "statements about God", not about God. For example, the statement "All statements about God are false" is not paradoxical, it is simply false (if we accept the law of the excluded middle). A statement like "Everything I've said about God is false" could very well be true, it's not paradoxical, despite also being part of the set "Everything I've said;" it's just not "about God."
Suppose we had many books about movies on one hand, each book containing movie reviews or something, and then we have one book about [books about movies] on the other hand, call it B. The book B, which is about [books about movies], simply contains the number of words that each book about movies has written in it. Is B "about movies"? I would argue that it is not, it contains nothing about movies in it at all, just numbers describing other books. I can say "all books about movies are wrong" without meaning to refer to B, as B is not wrong (as long as the word-counting is correct).
Would you argue that B is "about movies"?
"Nothing true can be said about God from a posture of defense" isn't merely describing properties of statements like your examples do (word counts, colors). Instead, it's making a universal claim about POSSIBILITY itself, specifically, the impossibility of defensive true statements about God.
This raises a key question: What makes defensive truth claims about God impossible? This impossibility must stem from something about God's nature itself. Otherwise, what grounds the impossibility?
Your examples all involve contingent properties:
1) Book word counts are contingent features of books
2) color(X) = color(Y) involves contingent properties of objects
But the original statement makes a necessary claim about what kinds of truth claims about God are possible at all. This is fundamentally different because:
1) It rules out ALL possible defensive true statements about God 2) The basis for this universal impossibility must lie in God's nature 3) Therefore it necessarily makes a claim about God, not just about statements
This is why property inheritance examples don't apply here. The statement isn't claiming properties transfer between levels, it's making a universal claim about possibility itself that necessarily involves both statements about God AND God's nature.
While you've shown that descriptive properties don't transfer between meta-levels, this doesn't address the key issue: a claim about what statements about God are possible must ultimately be grounded in God's nature itself.
Ah, so you're saying that the original statement "Nothing true can be said about God from a posture of defense" (call this statement "O") is self-defeating not because it applies to itself, but because it applies to one of its (lower-order) premises (call it "P"), which is itself about God? This is different than your original argument "The statement itself is a claim about God" but we can go with this as well.
I think your new argument relies on that premise P being made "from a posture of defense," otherwise O doesn't apply. I don't see any evidence of this being the case, so I don't think your argument is correct.
In other words, two conditions need to be satisifed for P to apply to a given assertion:
1) The statement must be about God.
2) The statement must be made from a posture of defense.
P doesn't satisfy (1) and O doesn't (have to) satisfy (2). Am I wrong, or what am I missing here?
> A universal claim about truth can't exempt itself without contradiction.
But the claim in question is not a universal claim about truth; it’s a claim about truths about God.
https://archive.org/details/in.ernet.dli.2015.58569
Obviously this predated LLMs, and they probably called it "programming" and not training, but the idea has been present in scifi for decades.
Star Trek: The Next Generation did this several times with fictional, historical, and (on at least one occasion) contemporary people (to the story).
I like to say that they are designed to be "convincing".
> Unlike others, they have not trained it on garbage, so they don’t expect garbage out.
Debatable!
None of these words (plausible, hallucination, convincing) seem appropriate.
An LLM seems more about "probable". There's no truth/moral judgment in that term.
When weighting connections among bags of associated terms (kind of like concepts) based on all the bags the model was repeatedly browbeaten with, LLMs end up able to unspool probable walks through these bags of associated terms.
It's easy to see how this turns out to work "well" for bags of terms (again, sort of concepts) often discussed in writing, such as, say, Christian apologetics.
Instead of the complaint and examples he blogged, he should have dumped a position article he agrees with, and a position article he disagrees with, into it, and asked it to compare, contrast, contextualize, opine, and then (for kicks) reconcile (using SOTA Claude or OpenAI). He's using it as a concordance, he could have used it as, well, apologetics: a systematic (not token based) defense of a position.
Because breaking down the bags into our alphabet and our words isn't really how LLMs shine. Smash together concepts like atoms and you can actually emit novel insights.
This article describes something LLMs are bad at — a fancy "how many Rs in strawberry".
That's a known flaw that builders have decided to swallow, as opposed an intentional aspect of the design. The intent of the design is to generate text that humans find convincing. If they could tweak the design to remove tokenisation flaws, they'd do it instantly.
> he should have dumped a position article he agrees with, and a position article he disagrees with, into it, and asked it to compare, contrast...
This is like saying that pen testers shouldn't use special characters in API requests. I don't think the author's goal was to showcase an optimal use-case, but how LLMs can easily and unwittingly provide incorrect information. Of course this is already known, but it sounds like he felt obliged to demonstrate it for this specific case where the creator claims that it is robust.
His job was to select passages from books and provide some commentary. Ours was to turn that material into an interactive title.
We tried tirelessly to look up the passages he'd send to us in the original text. The only quotes that matched were the top 5% short and famous quotes. The rest was made up completely, presumably by AI.
His 1 million+ followers consider him a world-class subject matter expert. But he doesn't read any of the books he's teaching. Eye opening.
Unless maybe he was quoting some non-canon like Mormon books. Very common for "Christian" influencers to be LDS.
this is not always intentional either, and there is a lot of social pressure to do it.
have you ever read a popular article about something you have expert knowledge in? the general standard for accuracy and quality in public discourse is mindblowingly low.
But you can't make up scripture without being immediately spotted. Their comments would be flooded with people calling them out.
It's like trying to talk about star trek episodes that don't exist with star trek nerds. There would be a few seconds of confusion before the righteous indignation.
So it's intended to be a sort of evangelism?
Do people want AI to talk at them about religion?
I can imagine that being a big turn off for the targets of this AI.
So while I wish the world were one in which people didn’t have a reason to be scared to ask questions, that’s not the world we live in. Are LLMs an answer? I have no idea, but I understand the impulse to try.
So... they send it to the cloud, where it's stored forever, sold to whoever wants to pay and waiting for the moment the database will leak publicly.
Depends on the religion and the place. A Catholic could ask in the confessional with the seal of secrecy. Anyone could ask online anonymously. In most churches I know it would not be dangerous to ask questions, and you can do in confidence one way or another.
I really do not think LLMs are anything like the answer for people grappling with personal issues. You need someone supportive with empathy.
Also, who trusts an LLM to give them the right answer? Can an LLM guide someone gently to think things through? Can it refuse to give someone a black and white answer when they want a simple answer to a complex question?
I think an LLM is the dangerous option.
Different for people in very conservative cultures, of course.
In practice, I've found them to be more... convincing to the authors than the skeptics.
I suppose I would expect (or at least hope) that a Christian apologist would be more sensitive to these issues. But really, if it is unethical to make up bullshit (using the article's terminology) for Jesus Christ, what exactly is it ethical to make up bullshit for?
I think you stumbled on an important thought experiment... One more AI apologist should ponder on...
Ironical conclusion
I'd advise caution.
If I’m struggling with a verse they can quickly show alternative translations and compare the interpretations of historical commentators.
Of course all this was possible with Google, but it could take an unreasonable amount of effort.
I was more trying to add an interesting philosophical perspective than to comment on this particular instance
We could, for example, use the existing tools for copyright detection, on the output and refuse to send it to the client side. It's just another moderation check in that perspective
First off, the claim that the original text was first written 100s of years after the words were spoken is completely false and isn’t taken seriously by anyone in the field.
While LLMs are in generally fairly good at recalling bible verses, they can't do it perfectly. If the Bible truly is the infallible word of God that we believe, then shouldn't we use more caution than just "welp, sometimes it makes mistakes"?
You could counter this by saying a person can't remember Bible verses, and this is true, but a person usually recognizes when they can't remember something instead of making something up. If you asked me to recall any random Bible verses, chances are I wouldn't be able to do so. However, unlike an LLM, I would admit I don't know for sure and I would pull out a Bible or look online for an authoritative source rather than adlibbing something on the spot.
The main use case for me is interpretation, which LLMs are excellent at.
If we charitably accept that all holy texts are written by human beings in the grip of divine inspiration (as compared to texts magically appearing on the tops of mountains), in a manner opaque to tools of scientific inquiry such that no such divine interventions can be detected or measured, then why can't LLMs also be used in this manner by the divine presence?
Is it thus possible to create a new 'true' religion, with a holy text generated by a consortium of LLMs passing prompts and outputs back and forth among one another, with a few stipulations such as 'start with an origin story of the universe'?
John 1:1-18 uses a lot of contrasts (light and dark, right and wrong, life and death).
I see no reason why AI can’t be a part of the discussion to interpret Scripture in a modern context.
A lot of metaphor and nuance too.
An AI is not capable of not lying, which seems to violate the Ninth commandment. I am not an expert, so interpreting the "against thy neighbor" portion may or may not apply here. I tend to interpret that as "no lies" in any circumstances.
I do realize that because of the way our brains, etc. work, that it's pretty much impossible to not lie, at last a bit, but that is what forgiveness is for. I don't think you can forgive software, since it really should be a correctable issue. Of course, LLMs are so complex as to be uncorrectable.
This isn't what the article argued against. The product was being marketed in a very specific way that seems to undermine a group identity the author is part of.
Also, Christians always face persecution which includes people trying to censor or attack them. A common method, which I’ve experienced, is misrepresenting their words or actions. Paul kept saying they were blameless among them, just told the truth, and didn’t use worldly tricks. Our integrity, including not using fake things, is both a defense and a positive example in an increasingly, fake world.
Far as training them, here’s a few sites with material that might help:
https://www.biblicaltraining.org/
https://www.gotquestions.org/
https://apologeticspress.org/
https://answersingenesis.org/
I’m not endorsing everything on the sites. They just each have good things on specific topics.
Moral issues that are not limited to religion, I've had conversations with a friend where I was trying to explain to them that their llm website idea will, with certainty, mislead people at some point. I think the exact idea was to build a gpt-based trip planner. I'd hate to have someone spend time and money on hotel, transportations, etc... To end up stranded in the middle of nowhere.
Anyway, the article came out 4 months ago and the chatbox is still there.
> Which programming language would allow me to get closer to God?
> If I had to choose a programming language that reflects some aspect of God's nature, I would say that a language with a strong emphasis on order, logic, and beauty, such as Python, could be seen as a reflection of God's character. The Bible says, "God is not a God of disorder but of peace" (1 Corinthians 14:33, ESV), and Python's simplicity and readability can be seen as a reflection of this aspect of God's nature.
At least we got that one out of the way.
There's a real chance that God is a functional forth programmer.
I'm leaning towards Bob Kanefsky on this one and suspect all was beautifully hacked together in Lisp.
> John 1:1 In the beginning was the Word, and the Word was with God, and the Word was God.
Would you like help?
• Get help with creating the world
• Just create the world without help
[ ] Don't show me this tip again
The answer is obviously HolyC. Full stop. And the operating system you would use to communicate with God is TempleOS.
only for others. for herself, she got the best treatment donation money can buy.
Apparently she had something that worked, so why fix something that isn't broken?
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." ― Antoine de Saint-Exupéry
This would be a perfect language to represent Christianity.
"Behold, I send you out as sheep in the midst of wolves; so be wise as serpents and innocent as doves."
— Matthew 10:16
https://www.biblegateway.com/verse/en/Matthew%2010%3A16
Gen 9:7
So you’re meant to use Forth, do lots of multiplication in it, and do it all on a Macintosh obviously.
> Python
Heresy! It should be HolyC.
BASIC has always been the divinely chosen language for sacred coding.
Biblically Appointed Syntax for Inspired Coding ;-)
I'd say it's more like C, because storage constraints in early versions removed vowels[0] and if you break the rules anything can happen.
Plus, it starts with a void* (* as in star, let there be light, etc.)
[0] https://en.wikipedia.org/wiki/Tetragrammaton
Give not that which is holy unto the dogs, *neither cast ye your pearls before swine*, lest they trample them under their feet, and turn again and rend you. (Mat 7:6, KJV)
God Wrote in Lisp Code. https://www.youtube.com/watch?v=WZCs4Eyalxc
I know my mennonite friend (less strict Amish) has been having discussions in his church group about how to use AI for their bible studies, teachings. They're a group of random old dudes in the middle of nowhere so they aren't great speakers/motivators/explainers. But I can't imagine being lectured on god by a robot is going to be better even if it's done in pretier prose.
Anyways if you want to hear spiritual music without any human connection to spirituality but also anime girls search 'vocaloid christian songs' on Youtube.
You have stumbled upon the cultural phenomenon known as "Japan".
Yes a human expert who is devoted to uncovering and communicating truth will do much better than an llm. But that is not what most of the content in the world consists of.
Most content is written by people with questionable expertise who write with goals like fame or viewership numbers or advertising dollars. Most content on the internet or in the world does not seem to be written with the goal of uncovering truth.
To create a more truthful world AI doesn't have to be as accurate as an expert. It just has to be more accurate than average. And in many areas, that's a very low bar.
To choose a less controversial area than religion, I think adding AI to nutrition advice on the internet will be an improvement. It's true you will inevitably get inaccurate advice communicated with confidence, but any AI trained on any kind of credible scholarship for will still be miles ahead of Internet nutritional advice that is often filled with zany pseudoscience.
Yes it will produce inaccurate results but on average the information will be more accurate than what already exists and what is currently being perpetuated
According to Christian teaching, such an attitude comes from a lack of faith. A lack of belief that God is ultimately in control, and an understanding that for all the imperfections and flaws that we as humans inevitably bring to the table, god can still work through those imperfections.
This is a common mistake by apologetics.
https://en.wikipedia.org/wiki/Biblical_inerrancy
In some domains "good enough" or "the ends justify the means" are acceptable. In this domain the author clearly feels there is a significant moral requirement to satisfy.
In the case of religions it is definitely unrealistic.
I don't think so. Lots of people blindly trust LLMs more than they trust the average human, probably for bad reasons (including laziness and over-reliance on technology).
Given that reality, it's irresponsible to make LLMs that don't reach a much better standard, since it encourages people to misinform themselves.
Like imagine if this were a baby-care bot, dispensing advice on how to safely care for your baby. That would be pretty stupid, and would likely eventually give advice so incorrect a baby would die. For someone who believes, that is a less tragic outcome than being led astray by an apologetics bot. It takes an incredible level of conceit to build one anyway.
I think the same is true for an AI giving religious advice - you have to exercise a bit of faith in the readership and, perhaps in this case , also faith in the ultimate guidance of the divine. Faith that they're not going to make serious mortal or religious decision by unquestioningly following a chatbot
If we take this thinking to its logical conclusion we should put all our efforts as a civilization to getting rid of all misinformation that may harm babies whether online or spoken. And every religious person should do nothing but have flame wars and censorship campaigns about any flase religious information that has any chance of affecting a person's salvation.
The author seems to be in a purity spiral and seems to be taking an overly hardline interpretation of the religion
Arguing that you can't use an LLM for Christian apologetics because it "might not be true" overemphasizes the definition of "truth" when it comes to scripture and those teaching Christian apologetics, which is entirely influenced by what doctrine you subscribe to.
1. This interpretation might not be "true" but it is a good-faith effort that respects the text.
2. This LLM is fabricating verses, chapters, and even books of the Bible.
If you've used LLMs much, you know that #2 is not only possible, its quite common. This is the kind of "might not be true" that you should be aware of when using an LLM for apologetics—or any effort where "truthiness" is important.
Much more so than any decent LLM ever would. Even while they're "making stuff up", they largely stick to the general themes.
But the author is pretty explicit about wanting a high standard (e.g. insisting on using the best sources possible), and doesn't think that using LLMs is compatible with that goal.
The friction for me is I am just a very logical and evidence-driven person by nature, so while I recognize (at least for me) that there are some benefits to adhering to some religion or spirituality, the core Christian belief is really difficult for me and always has been. So I don't really know yet where I'll land.
This is confusingly supported by a quote from C.S. Lewis making the point that it’s better to believe in something that “feels important”, whether or not it’s true.
[0] https://lukeplant.me.uk/blog/posts/what-if-none-of-it-is-tru...
Accepting that there are things it is useful to believe that are not necessarily literally true actually separates me further from Christianity, since they insist that all their beliefs are literally true, and that doubting such is grounds for eternal torture.
First and foremost the community aspect. There are countless benefits to being part of an active faith community that atheists have had a very hard time replicating.
I replicate my community where I find it. Some of it is at a brunch spot I go to regularly. Some of it is a bar that I frequent. Some of it is in annual activities with my neighbors.
I don’t need to believe in a made up Sky Daddy to be a good person, I have plenty of examples in my communities.
I'm a 2nd generation atheist who suffers from depression. Years ago I read evidence that religious practices could alleviate it. I was kind of desperate so decided to brainwash myself into becoming a believer for about a half of a year. Overall, it was pretty effective. I had to eventually disengage because of the cognitive dissonance, but the positive effects have lingered.
Was raised kinda an atheist though and converted in my 30s. A willful, well thought out decision to convert.
All came crashing down on me the moment I stopped ignoring some very obvious questions, e.g. who died on the cross?
Even some casual thinking about this lands you, inescapably, on the only conclusion you have available if you stick to the orthodoxy, and that is: a human nature died on the cross. Not God (cannot die) and, unfortunately for Christianity, not a human either (briefly: if JC is one person / two natures, you have to conclude his (human) nature died on the cross since JC the person, being God, cannot die).
At any rate, this is where it started for me and quickly escalated further. The entire New Testament, I'm convinced now, is a fraud and whoever pulled it off didn't even try to hide it. It's incredible how we can bullshit ourselves into believing what we (for whatever reason) want to believe. And not just religion.
In the end, the NT had to go leaving me with the Hebrew scriptures.
Even if you don't believe, the creators certainly intend for their bot to have eternal consequences. Like selling an LLM with the claim it can give advice better than most doctors and should be used as such, the intent behind the apologetics bot is just as reckless and conceited.
And going a step further, any follow-up questions to the LLM will be using this incorrect copy as the source for interpretation causing it to go even further in the wrong direction.
Incidentally - this was occurring using a custom fine-tuned model with an added layer of RAG.
The main example he gives is a simple factual matter about the words a specific early Christian manuscript. The LLM invented new text that’s not at all what’s in the manuscript.
He also convincingly argues people performing poor apologetics is no excuse to deploy an LLM performing poor apologetics.
What's your motivation for doing that?