The "cow in the field" example reminds me of two heuristics I like: am I right for the wrong reason?; am I wrong for the right reason?
Being right for the wrong reason is dangerous: it's not easy to spot, and it perpetuates false sense of security leaving "black swan events" unanticipated. This might occur during debugging as the article points out, or e.g. during A/B testing of a product.
Bring wrong for the right reason is just plain frustrating.
In the context of political forecasting, imagine that you are a defence chief who is faced with an unquantifiable external threat, as the US was by Russia during the Cold War. You can predict that this enemy is a very great threat, or you can say that it isn’t much of a threat, but the outcomes are asymmetric. If you say that the threat is a grave one, and strengthen your defences accordingly, then if the enemy attacks, you were clearly right to take the threat seriously. If the enemy doesn’t attack, you were still right, because you can say that the enemy only didn’t attack because of the action you took. On the other hand, if you dismiss the threat as insignificant, and the enemy attacks, then at best your career comes to a sudden and unpleasant end. So therefore, it is always right to over-emphasise the threats, and if you turn out to be wrong, you were wrong for the right reason.[1]
> always right to over-emphasise the threats, and if you turn out to be wrong, you were wrong for the right reason.
if the audience is not receptive to the concept of opportunity cost, then yes. Unfortunately, a majority of people over-estimate the need for security and thus, allow themselves to be fooled into believing that this over-emphasis, no matter the cost, is justified.
Nate Silver is widely seen as having forecast the election "wrong" because he said Trump had a 20% chance of winning. AIUI his position is that his model was right and 20% was an accurate probability.
Games are a place this comes of frequently. In poker you will often make the "right" (highest EV) play and still lose a given hand.
Making the highest odds plays (and being able to figure out what they are) over and over again regardless of how the individual hands turn out is how you win.
Obviously, you can also win while making the wrong plays by getting lucky (right for the wrong reasons). Evaluating your play based on the outcome of the hands (did you win or lose) rather than the plays you made with the information you had at the time is called Results Oriented Thinking: https://www.pokerdictionary.net/glossary/results-oriented-th... ... and it is a pernicious mistake (with wider applications than just poker :).
My compensation includes company stock. I sell that stock, and some of the proceeds end up in index funds. I do this because I want a diversified portfolio.
My employer's stock is up 28% over the last year, S&P 500 is down 6%.
I guess my goal going in was to reduce variance, so I wasn't "wrong" about anything. 36% more money would be nice, though. (I haven't changed my strategy as a result)
These cases seem to come up often (weekly?) in software development. I wonder how often they come up in other professions.
One common case is when you change or delete a comment, and suddenly something breaks. It couldn't have been the comment... but it was working fine before my edit... wasn't it?
Considering the fact that so much of programming is error finding, it's useful (and probably necessary) to have a solid heuristic for quickly determining causal relationships.
I lost quite a few hours trying to restore a feature after I made a commit, only to find out that it was broken for weeks already. Or worse, was not even implemented yet.
This gettier concept is new to me, but what's certainly not new is trying to wrap my head around errors in the code. I'm a relatively new developer and I've many times asked more seasoned coworkers about what they do with all the their thinking they perform and possibly code they write during a long error invalidation process. Say you try fixes b,c,d,e,...,z (some of which might be objective improvements, now more robust code) then you finally fix the bug by trying solution A. What do you now do with the code for attempts b through y, and more unclear, all the thoughts in your mind that went into those efforts? Just forehead slap and move on?
This hits close to me as a possible reason why I could never get good at solving geometry problems, solid geometry especially. Most problems would be trivial when one assumes specific preconditions, but my mind was always wandering around, looking at all potential sides of a problem and I could never solve anything. To quote the author from my particular pov:
a problem has multiple potential causes, and you have every reason to believe in one of them, even though another is secretly responsible.
Reminds me that I need to pick up a book and re-learn the damn thing. It really saddens me that I suck at geometry.
I think you should try looking at geometry more like a creation initially then a problem. In this manner you can see assumptions as just building up more simple worlds with those constraints. I have found this view helps when teaching geometry as it empowers the mind.
I had a philosophy lecture last year that included a lot of epistemology (Theory of Knowledge). We talked a fair bit about justified true beliefs, but Gettier only came up in a side note - the professor being more interested in skepticism and the responses thereto. Never would have dreamt of applying that lecture to programming, though.
The propensity for mistaking belief for facts certainly take daily hits as a software developer. "How come this simple thing isn't working? I thought of everything didn't I?". After a while you are forced to realize that belief isn't the same as reality.
It seems insights like this don't easily translate into other domains though, like relationships, dearly held political views etc. We prefer to think of them as based on facts, when in all probability they are merely beliefs fraught with assumptions.
Some people might be good at being skeptics in all areas, but I sense most share my own ineptitude here, the reason probably being that any such (incorrect) beliefs don't immediately come back and bite us, as in programming.
The funny thing about development is what's say 90% of the time you are convinced everything is correct and should be working and it's immensely frustrating because it's not, so you know you are wrong but are unable to offer yourself a more convincing theory and just get stuck until something clicks. But then there's that 10% of the time where you're actually right. And you don't know which one it's going to be. So you have to calm yourself down like "I know I think Im right about this but clearly Im not" but at the same time you have to hold onto that conviction because you're right damnit. Haha.
My favorite example of this is when someone says "Nobody knows X." As though your own ignorance is as good as everyone's ignorance.
Somebody might know X. And they might know X for all the right reasons. But they probably didn't tell you those reasons, and you probably wouldn't believe them or understand them if they did.
This is a lesson learned well from open-ended systems. An open-ended system is one where input is received to process, but the input is not well defined. The more accepted unknown input becomes the more open the system must be in its rules to process it. The results of processing are:
* expected output from a known input (intention)
* unexpected output from a known input (defect)
* expected output from an unknown input (serendipity)
* unexpected output from an unknown input (unintentional)
For example I maintain a parser and beautifier for many different languages and many different grammars of those languages. In some cases these languages are really multiple languages (or grammars) imposed upon each other and so the application code must recursively switch to different parsing schemes in the middle of the given input.
The more decisions you make in your application code the more complex it becomes and predicting complexity is hard. Since you cannot know of every combination of decisions necessary for every combination of input you do your best to impose super-isolation of tiny internal algorithms. This means you attempt to isolate decision criteria into separated atomic units and those separated atomic units must impose their decision criteria without regard for the various other atomic decision units. Provided well reasoned data structures this is less challenging than it sounds.
The goal in all of this is to eliminate unintentional results (see forth bullet point above). It is okay to be wrong, as wrong is a subjective quality, provided each of the atomic decision units are each operating correctly. When that is not enough you add further logic to reduce the interference of the various decision units upon each other. In the case of various external factors imposing interference you must ensure your application is isolated and testable apart from those external factors so that when such defects arise you can eliminate as much known criteria as rapidly as possible.
You will never be sure your open-ended system works as intended 100% of the time, but with enough test samples you can build confidence against a variety of unknown combinations.
Yes, i would still classify the software defective. It did not reject backwards explicitly. Just because by accident it works (i.e., the consumer of the output doesn't care) doesn't mean the defect has gone away.
To be not defective, the software has to explicitly reject input that it was not designed to handle.
Imagine if the software updated with some changes, and the unknown input now produces an incorrect output. Is the defect introduced with the changes? Or was the defect always there?
> To be not defective, the software has to explicitly reject input that it was not designed to handle.
In some cases that breaks forward capability. e.g. the case where there is an unknown XML tag. You could reject the tag or message. You'll end up rejecting all future change inputs.
If the whitelist of acceptable items is large, it may be acceptable to have a black list however if the above holds, you don't know what you don't know.
The middle ground may be explicitly flag/indicate/log that an unknown situation has been encountered, and 'handle' that by doing something useful (continuing to work without crashing, preventing known "unknown" data from being processed silently, etc). It may not help with forward compat entirely, but it would be explicitly known (and I'd think would be somewhat easier to modify/extend for known unknowns in the future).
When you come around a curve near sunrise or sunset, you may suddenly encounter visual input that overwhelms your sensors. The sun is blinding you. It might overwhelm infrared sensors, too.
If you have alternate sensors, you should trust them them more, and camera systems less.
If you have a sunshade, you should deploy that.
If it is raining, or partially cloudy, the situation may change rapidly.
And perhaps you should slow down, but if you slow down too fast, other vehicles might not be able to avoid you.
Or simply acknowledge that your initial specs didn't cover enough, update the specs, test the "new" functionality, and call it a feature in the release notes.
I've been there, painfully. On my last day on a job, some code I wrote went into production, and the databases started filling up rapidly, to the point where the whole system would halt in a few hours.
Turned out the bug had been latent in the code for 5+ years, predating me. Its data consumption had never been observed before because it was swamped by other data consumption. Changing the architecture to remove that other data brought it to the foreground.
(fwiw, the bug was caused by the difference between 0 and null as a C pointer!)
My application is a code beautifier that receives code samples as input. I cannot know of every possible combination of code samples. This does not necessarily mean the application is defective. A defective state is unwanted output.
Another way to think about this is that the more open a system is the more useful and risky it is. It is useful because it can do more while tolerating less terse requirements upon the user. It increases risk because there is more to test and less certainty the user will get what they want. Part of that risk is that its hard to guess at what users want as sometimes the users aren't even sure of what they want.
No, it doesn't. It relies on explicit rules (old school) or statistical inference (new school).
There's a difference between "breaking" unknown input - i.e. non-computable within the system as it stands - and "working" unknown input, which is within expected working parameters.
The latter is business as usual for computing systems.
The former should have a handler that tries to minimise the costs of making a mistake - either by ignoring the input, or failing safe, or with some other controlled response.
It may not do this perfectly, but not attempting to do at all it is a serious design failure.
There's a saying that when people figure out how to make a computer do something well, that it's no longer in the field of AI. I'd say there's some truth in this, in that for many problems we have solved well (e.g. playing chess), the intelligence is not that of the machine, but of the programmer.
I think that in order for a machine to genuinely be intelligent, it must be capable of original thought, and thus unknown input. Known doesn't necessarily mean specifically considered, but that it could be captured by a known definition. As an example, we can easily define all valid chess moves and checkmates, but we can't define the set of images that look like faces.
I learned this word, but I'm scared and wondered by it every time. I think it fits here. There is two worlds: believed and true. And when they merge it's called "peripeteia"
I read the Wikipedia's entry. It seems that it could be better explained as a "plot twist": when your confidence in an outcome is overturned, for better or worse.
I think it helps to look at the mind as a probabilistic survival engine than some truth engine.
If there appears to be a cow in a random field the odds are extremely low that someone put a papier mache cow there. If there’s something that has 50 % chance of being a snake you panic and run because that’s a 50 % chance of dying.
In the case of the authors bug yes the change he introduced had a good probability of being the cause. However he could have increased the probability by going back over commits and confirming that his exact commit introduced the bug. Now the probability goes even higher. But it could still be machine specific a cosmic ray or whatever but the odds are over whelmingly low.
In practice causal reasoning also works in a probabilistic fashion.
I have a simple model saying that if a planes engine is on then it’s flying. Its a single step probability and so it’s not very accurate in the real world.
I do a bunch of experiments and say a plane is flying if the engine is on and air is going over it’s wings faster than a certain speed.
Now we have two correlations connected by a causal model that works in many other cases. Hence the probability of it being correct rises.
But at the same time we should never mistake direct correlation for causality. But in daily life it’s “good enough”.
I like the term "probabilistic survival engine". It explains a lot of phenomena in social trends, religion, politics, and even the practice of science.
The first time I heard to a brain referred to in this way it was in the scifi book Blindsight by Peter Watts. The book is pretty solid and deals with a lot interesting ideas from cognitive science.
I'm sad that I had to read so far down the thread to find this comment, which beautifully cuts right through the argument. The problem is the idea of absolute knowledge -- knowledge is probabilistic and contextual. If I'm driving by the field in a car, I may be duped by the cow, but my perception that I'm surrounded by pasture land is more than adequate to support the activity of driving. If on the other hand, I'm Farmer Jones and I'm searching my pasture for Susie the cow, well, I'm not going to be taken in by a statue of a cow for more than a split second anyway.
I think what you're saying is correct when it comes to perception with our senses, the kind of knowledge we share with the other higher animals. However, there is a higher level of knowledge that is unique to humans, which is rational in nature and can come up with certain knowledge. We can know certain mathematical and logical truths with certainty, such as 2 + 2 = 4 and that A and not A cannot both be true. We may not get an accurate count of cows in the field because one is hidden from view, but we can know with certainty that if we have X cows in field A and Y cows in field B, we have a total of X + Y cows in the two fields.
I agree with your main points, but I think it's worth pointing out that even things like 2 + 2 = 4 and X + Y cows in fields A and B, all rest on certain definitions and sets of deductions, many of which go back down to basic axioms which have to be assumed (e.g. for arithmetic, see Peano axioms as an example: https://en.wikipedia.org/wiki/Peano_axioms).
- We have a definition of what a cow is, and we know that cows are discrete/physical objects, and have relatively fixed locations (i.e. that they are not like an electron cloud with a probabilistic location).
- We assume that fields A and B in your hypothetical have clear, non-overlapping boundaries.
- We assume that we are working in a fairly normal universe with a fairly standard model of physics, and that due to the way time works in this universe, a cow cannot simultaneously be located in both fields A and B.
- ...
- (this could get really pedantic and go on forever)
The point is, even the things "we can know with certainty", are only as certain as the framework of observations/deductions/axioms/etc. that they rest upon. Almost nothing is certain on its own, without any further layers of reasoning behind it.
Denying knowledge makes your arguments work, but knowledge does exist and is a real thing. Are you sitting on a chair? The answer isn't probabilistic, you KNOW the answer. What's your name? Have you ever been to the moon?
This is an excellent comment - I'm just disappointed you managed to say it all without the word "Bayesian". The base rate for paper mache cows in fields is very low - one is perfectly justified in assigning a decent probability that a field contains a cow, if one sees a cow-shaped object in it. If you are in a part of the world that has a lot of cows in fields, you will presumably assign an even higher probability. You might even say you're "sure" there's a cow in the field, and act as such for everyday purposes. But don't be fooled - you're not really sure. If someone offers to bet you ten dollars versus your life that there is a cow in the field, you'll likely not take the bet.
It seems that the philosophers were grasping towards a definition of "know" that encapsulated the idea of assigning 100% probability to something, after incorporating all the evidence. From a Bayesian standpoint, this is impossible. You can never be 100% certain of anything. To "know" something in the sense of "a justified, true belief" is impossible, because a belief is never both 100% and justified.
(Note that it is entirely possible to see the paper mache cow and conclude that it is likely only paper mache and that there is not a real cow in the field. Is this belief "justified"?)
I've thought a bit more about it, and concluded that while the above is a neat answer, it doesn't explain the question, and [thewarrior]'s remarks were nearer the mark on that. So here goes.
It's tempting to think of "knowledge" as some relationship between the mind and a single fact. But when we use the word "knowledge", what we actually mean is "an accurate world model" - a set of beliefs. This is the disconnect that Gettier cases are designed to expose - they construct scenarios where someone's mental model is inaccurate or incomplete, yet by sheer luck produce a single, cherry-picked correct prediction. We are uncomfortable calling these correct predictions "knowledge" because as soon as you start probing the rest of the mental model, it falls apart. Sure, they think there's a cow in the field, and there really is one. Ask them any more questions about the scenario though ("what color is the cow?") and they'll give wrong answers.
From this perspective, "knowledge" as a "justified, true belief" is a perfectly coherent concept - the problem lies with the inadequacy of the word "justified" to describe the output of a complex decision procedure that incorporates many beliefs about the world, such that it could be expected to yield many other correct predictions in addition to the one in question, up to some arbitrary threshold.
A thought experiment - suppose you tell the observer that the cow they see is made of paper mache. They no longer believe there is a cow in the field. Intuitively, has their knowledge increased or decreased?
An unforeseen issue with probabilistic assessments is the model of the probability space is probably inaccurate.
For example, Nassim Taleb has an argument of IQ being a single-dimentional assessment to a multi-dimensional domain.
I think it's more practical to have a possibility space (where unknown unknowns is a possibility). This removes the need to assess probabilities (which will probably be incorrect) while having being able to per-mutate through the list of possibilities. One can also do logical deductions, based on the possibility space, to assess possible strategies to explore/solve the issues at hand.
To me this seems unhelpful. I'd say there is no "knowledge"; there's only belief. And if you defined knowledge as "justified true belief" then you couldn't apply the definition in practice in the real world because you don't know when something is true. But that's philosophy for you: fun (for some people) but not useful.
You can apply that to the real world if you then add social processes and see "knowledge" or "truth" as shared belief between a chosen set of people (or all of humanity). Then you can go down the whole rabbit hole of beliefe aggregation and voting theory.
Not good enough, point to that which knows. Where is it? Who knows it? If there is a knower to that, where is it.. Keep going with this investigation and realize there is no knower, only knowing.
Eastern philosophy has nailed this thousands of years ago and we westerners are up to this day totally in the dark. We actively treat the I as a concrete object that really exists as an entity. It does not hold any closer examination and evaporates entirely the closer it is questioned.
I know my name just like I make a cup of tea. It's not you who makes my cup of tea, right, so who is it? Well, me—the referent of my name, this person right here.
That's just part of how our language works. It doesn't seem to matter whether I am a "concrete object" or some swirly pattern of becoming or indeed even an illusion! The English word "I" does not refer to an eternal soul or "atman."
If you stare long enough at an ice cream you'll have the marvelous insight that in reality there is no concrete ice cream entity, not least because it melts. Yet people don't go around saying "wake up, there are no ice creams!" Why is that?
You may be right about knowledge, (relativstic quantum
information
theory leans in your favor, hi Ron Garrett!)
but your dreprectatory definition of knowledge and philosophy covers most humans who have ever existed, not just self titled philosophers.
Anyway your definition is wanting. A religious scientist has two kinds of clearly different beliefs: faith and knowledge. A mathematician has the same two kinds, under different names: axioms and deductions.
Saying that axioms are the same as deductions is a radical claim.
You assert that there is no knowledge under the expectation that it is true, then you go on and justify it. So it seems you know that there is no knowledge, a contradiction.
Why deny the word has meaning, just because you can't distill it down to a concise explanation? The meaning of a word can be arbitrarily complex. Knowledge can exist, even if you can't define it, because meaning is not determined by definition. Definitions are simply a mechanism for coordinating understanding, not for demonstrating its existence.
Besides, any argument against the existence of knowledge could be used against belief. Play "taboo" with the subject and don't confine yourself to using ancient terminology to describe the world and these pointless linguistic problems melt away.
We do know when some things are true, and when observing the world we can talk about truth under certain assumptions. As in “it’s True the sun didn’t explode yesterday, assuming we aren’t all living in a simulated reality run by aliens in a different solar system billions of years from now the exact day after the sun exploded, or similar possible scenarios”
And while everyone loves to run in circles around the argument “but how can you know with certainty” the fact is that I am as certain that this assumption holds as I am that it provides no value at all to continually question if reality is really real, you’d have to take that as an assumption to ever have any kind of value adding discussion.
The people who insist we can’t know if anything is definately true must agree that they can’t know if that assertion is definitely true, so they sort of kill their own argument axiomatically.
>A philosopher might say that these aren’t bona fide Gettier cases. True gettiers are rare.
I beg to differ. Besides the examples in programming the author gave, I can very easily think of examples in medicine, police work (e.g. regarding suspects), accounting, and so on...
In science, experiments with a good control is how this problem is removed. And repetition is how to control for coincidences and outlier measurements.
Software engineering has many of these Gettier cases, because most software engineers do not follow the scientific method when investigating a problem!
That sounds to me like (1) not a clear example of a Gettier case and (2) something that would, in fact, be rare.
#1 because one of the reasons for your believing X has cancer is that "he has the symptoms", which (if I'm understanding your example correctly) is in fact a consequence of the cancer he actually has; so, at least to some extent, your belief is causally connected to the true thing you believe in just the way it's not meant to be in a Gettier case.
#2 because (so far as I know) it's not at all common to have both (a) cancer in an organ that doesn't show up in your X-ray (or MRI or whatever) and (b) a cyst in the same organ that looks just like cancer in the X-ray/MRI/whatever. I'm not a doctor, but my guess is that this is very unusual indeed.
So this isn't a very convincing example of how clear-cut Gettier cases aren't rare: it's neither a clear-cut Gettier case nor something that happens at all often.
In the general case it's "I believe X based on signs that would imply X correctly, and I happen to be correct that X holds, but I misread the signs I used to come to the conclusion".
I don't think this is rare -- including in the version of my example.
The only reason there's arguing that it's not a "clear cut case" is that I mentioned seeing "symptoms". Ignore the symptoms I mentioned, as they are a red herring, e.g. seeing the mark could cause the belief alone.
Other than that, it's a belief (1), that's justified (2), and true (3) -- while being accidentally justified.
Consider the case of a policeman that things someone is dangerous because they think they seen a gun on them. So they shoot first, and lo and behold, the suspect did have a gun on them -- but what the policeman seen was just a cellphone or something bulky under their jacket.
Or the spouse that thinks their spouse is having an affair because they see a hickey. Their spouse indeed has an affair (and even has a hickey on the other side of the neck), but what their spouse saw was just some small bruise caused by something else.
Or, to stick with the theme, figuring domestic abuse, and the victim suffers that indeed, but your guess is based on a bruise they had from an actual fall.
As in, when u write "I can very easily think of examples in medicine, police work (e.g. regarding suspects), accounting, and so on..." but you don't give any, the natural tendency is not to believe you.
That would be the "natural tendency" if I was describing something mysterious and rare that few can fathom. Whereas to me those situations don't seem really as far-fetched or hard for someone to come up on their own.
I didn't write "one can easily" to imply I have some special talent to imagine such situations (and thus had motive to leave examples off to hide the fact that I don't).
I wrote it because I really do believe one can easily find such examples, and wasn't it even worthy to go into details (since I mentioned medicine, police work, etc, I thought the cases I implied where pretty clear too).
In any case, I gave 3 examples in a comment above.
This are called false positives and are a normal occurrence in the life of a software developer. That's why when testing the root-cause you should test for a false positive as well.
I don't think the example of checking in someone else's bug is a very good one. If I checked in something that had code from other people I would see it very easily, and if I did not think there was any way my code should have affected the autocomplete then I would assume the code I checked in which was not mine broke the autocomplete.
Matching it to the example of the papier mache cow doesn't really work because the papier mache cow hides the real cow but it is very easy to see that your code was also checked in with other people's code.
There's a much closer analogy from software development to the cow story. The cow story is confusing because the cow that you see (A) is fake, but the real one (B) you don't know about. So your belief is not a justified true belief because although the real cow exists, the one your knowledge refers (A) to isn't the real one (B).
An intertemporal variant of this is race conditions. There have been lots of problems of the form "(1) check that /tmp/foo does not exist (2) overwrite /tmp/foo"; an attacker can drop a symlink in between those and overwrite /etc/password. The file that you checked for is not the same file as you wrote to, it just has the same name. This is an important distinction between name-based and handle-based systems.
If the real cow (B) was not present, your belief that there was a cow in the field would be justified but not true. Seeing a the fake cow (A) justifies your belief that there's a cow in the field. Adding a real cow (B) that you can't see doesn't remove the justification.
I actually disagree with the Gettier thought experiment and don’t believe it demonstrates anything interesting.
When you see the cow (but it’s really a convincing model), then in your mind, there should be some probability assigned to a variety of outcomes. The main one would be a cow, another one might be that you’re hallucinating, and so on down the list, and somewhere the outcome of cow-like model would be there.
From that point you can go in at least two directions, one would be something like a Turing test of the fake cow... beyond a certain point it’s a matter of semantics as to whether it’s a real cow or not, or you could say that your “justified true belief” had to apply to the total state of the field. If you believed there was both a cow model and a cow behind it, that woukd be justified, but the existence of the cow behind the model would not justify incorrect belief that the model was a real cow, in the sense of not admitting uncertainty over the things you see.
Your model of thinking is something like idealized bayesian one, not like human. If your mind decided that something is true with sufficient probability, it then fails to see alternative explanations. This "something" will be a reality to you, not just a belief about reality. But it is not the all. Your mind have some ideas associated with cow you've seen, and those ideas also becomes beliefs of yours. Implicit beliefs, you are probably not aware of them, you could notice this implicit beliefs only if they will begin to contradict evidence in a sharp way. For example, you can implicitly decide that this is a soft tempered cow, that would like to lick you in a face, if you come near. It was not important when you saw the model of a cow, so you didn't become consiously aware of your idea of a cow temper. But it was planted in you idea of reality. Maybe you would have an intention to come near the cow to be licked in the face, and you might be unaware of this intention. Human mind can easily manage such tricks.
And it leads to a funny thing. You saw the model of a cow, and it make you believe that there is a cow in the field and that you saw a cow. Then you could find a heap of poo, and you will strengten your beliefs futher. You might find a lot of evidence and it all will be explained under assumption that you saw a cow. And this evidence will strenghten your belief that you will be licked in the face, when you come near the cow.
But you didn't saw the cow that made this heap of poo. The real cow is pitch black, with a horns of gigantic size and they are really sharp. The real cow has red glowing eyes and it is going to kill you. But before you see the real cow itself, all the evidence that would point that there is a cow would also reinforce the idea of soft tempered black and white cow. The longer you manage to keep youself oblivious to real cow traits, the more surprised you will become when you find the real cow.
> From that point you can go in at least two directions, one would be something like a Turing test of the fake cow... beyond a certain point it’s a matter of semantics as to whether it’s a real cow or not, or you could say that your “justified true belief” had to apply to the total state of the field. If you believed there was both a cow model and a cow behind it, that woukd be justified, but the existence of the cow behind the model would not justify incorrect belief that the model was a real cow, in the sense of not admitting uncertainty over the things you see.
You're replacing the model it was criticizing with a different model and then saying that it doesn't say anything interesting about your model, so it's not interesting. It's not an argument that knowledge isn't possible, it was an argument against the traditional definition of knowledge as it was almost universally understood at the time.
The first example is exactly the reason why I hate rebases and prefer merges and complicated history instead. It may be more complicated, but it doesn't swipe problems under the rug.
>> He called them “gettiers” for short. So we used to talk about gettiers all the time, no doubt in part just because it felt clever to talk about them, but also because when you’re a programmer, you run into things that feel like Gettier cases with unusual frequency.
Sometimes I think that is what philosophers are doing - feeling clever - perhaps as a defense against some negative inner problem (psychology is an outgrowth of philosophy after all). The whole cow story stinks of telling someone "you're right, but you're also WRONG! Your perception of reality is BROKEN!". To me knowledge is simply having a model of the world that can be used to make useful predictions and communicate (and some other things). Aside from that, it doesn't matter if your model is "grounded in reality" until it fails to work for you, at which time it can be helpful to realize your knowledge (model) needs adjustment.
One way to resolving the authors first software issue would be to check a diff between what he committed and the previous production revision - this would quickly uncover the changes he "didn't make". This is an old lesson for me - what I changed may not be limited to what I think I changed. It's a lesson in "trust but verify". There are any number of ways to view it, but in the end we only care about ways that lead to desired outcomes weather they're "right" or not.
On a related note, I've found that software is one of the only places where there is a "ground truth" that can be examined and understood in every detail. It's completely deterministic (given a set of common assumptions). I've found the real world - and people in particular - to not be like that at all.
Exactly, one ontology of knowledge is: pointers (names), categories (matchers/recognizers), models (descriptions of systems).
The first two have all the problems philosophers talk about. But the last one does not. Not even underdeterminism, unless the system of the model is fundamental or fades into history or is a "wicked" problem.
> Sometimes I think that is what philosophers are doing - feeling clever - perhaps as a defense against some negative inner problem (psychology is an outgrowth of philosophy after all).
All science is an outgrowth of philosophy.
It's very frustrating when people look the obviously trivial and sometimes silly examples that philosophers use to elucidate a problem, and take it to mean that they are interested in trivial and silly things. Being right for the wrong reasons is a common and difficult problem, and some if the solutions to it a really insightful and powerful ideas.
> Aside from that, it doesn't matter if your model is "grounded in reality" until it fails to work for you, at which time it can be helpful to realize your knowledge (model) needs adjustment.
It might matter a great deal if your model is not grounded in reality - there are situations where that can kill you. It also seems like one of the fundamental aims of science, to have theories fail less often.
Note most gettier cases are plays on our intuitions:
What feels like a pointer is actually a category. That is, it feels like it points to one, but it points to many. Like both examples given here: https://en.wikipedia.org/wiki/Gettier_problem .
I can imagine a future where what's true generally describes itself (like terraform on drugs for software :p) Imagine software that is fully self descriptive and would no longer require engineers to individually interpret what's happening because the software would tell us. The system would be a graph of every single component and all possible connections between them, and all variants of each component and state that it could be in. When we introduce a change we would be aware with perfect information about the affect to the states and the paths between them.
In the example the Mental Model was at a level too shallow, it should have only affected the paths between the autofocus and the user. But the bug necessitated a larger mental model (the author was considering too small subsection of the graph).
I'd hope in the future we could reach a state where the program could have detected that the frame refactor would have an affect on the autofocus and all other components instead of being an implementation detail.
This reminds me of a recent event at WrestleKingdom 13, a Japanese professional wrestling event where, as you might imagine, pretty much everything is planned and choreographed ahead of time.
In the opening match, Kota Ibushi suffered a concussion. Some doctors came out, carried him out on a stretcher, and took him to the back. As it turns out, this was all planned. The doctors were fake, and this course of events was determined ahead of time. But coincidentally, Ibushi _actually_ suffered a real-life concussion in the match.
Wrestling always has an interesting relationship with reality.
184 comments
[ 4.2 ms ] story [ 217 ms ] threadBeing right for the wrong reason is dangerous: it's not easy to spot, and it perpetuates false sense of security leaving "black swan events" unanticipated. This might occur during debugging as the article points out, or e.g. during A/B testing of a product.
Bring wrong for the right reason is just plain frustrating.
what's an example of being wrong for the right reason? I can't think of any cases where this happens...
[1] https://wiseinvestment.co.uk/news/antiques-roadshow-tony-yar...
if the audience is not receptive to the concept of opportunity cost, then yes. Unfortunately, a majority of people over-estimate the need for security and thus, allow themselves to be fooled into believing that this over-emphasis, no matter the cost, is justified.
Just look at the TSA!
Making the highest odds plays (and being able to figure out what they are) over and over again regardless of how the individual hands turn out is how you win.
Obviously, you can also win while making the wrong plays by getting lucky (right for the wrong reasons). Evaluating your play based on the outcome of the hands (did you win or lose) rather than the plays you made with the information you had at the time is called Results Oriented Thinking: https://www.pokerdictionary.net/glossary/results-oriented-th... ... and it is a pernicious mistake (with wider applications than just poker :).
My employer's stock is up 28% over the last year, S&P 500 is down 6%.
I guess my goal going in was to reduce variance, so I wasn't "wrong" about anything. 36% more money would be nice, though. (I haven't changed my strategy as a result)
One common case is when you change or delete a comment, and suddenly something breaks. It couldn't have been the comment... but it was working fine before my edit... wasn't it?
[0]: https://en.m.wikipedia.org/wiki/Heisenbug
It's amazing how that just keeps on happening.
Russian geometry problems, as I remember them, required creative thinking, which seem to be exactly the opposite to what you are describing.
It seems insights like this don't easily translate into other domains though, like relationships, dearly held political views etc. We prefer to think of them as based on facts, when in all probability they are merely beliefs fraught with assumptions.
Some people might be good at being skeptics in all areas, but I sense most share my own ineptitude here, the reason probably being that any such (incorrect) beliefs don't immediately come back and bite us, as in programming.
Reading this made my day.
This is what the technique of Rubber Duck Debugging helps with. I wonder if you could translate it to other domains?
I think that can be expanded to the whole human race.
Somebody might know X. And they might know X for all the right reasons. But they probably didn't tell you those reasons, and you probably wouldn't believe them or understand them if they did.
* expected output from a known input (intention)
* unexpected output from a known input (defect)
* expected output from an unknown input (serendipity)
* unexpected output from an unknown input (unintentional)
For example I maintain a parser and beautifier for many different languages and many different grammars of those languages. In some cases these languages are really multiple languages (or grammars) imposed upon each other and so the application code must recursively switch to different parsing schemes in the middle of the given input.
The more decisions you make in your application code the more complex it becomes and predicting complexity is hard. Since you cannot know of every combination of decisions necessary for every combination of input you do your best to impose super-isolation of tiny internal algorithms. This means you attempt to isolate decision criteria into separated atomic units and those separated atomic units must impose their decision criteria without regard for the various other atomic decision units. Provided well reasoned data structures this is less challenging than it sounds.
The goal in all of this is to eliminate unintentional results (see forth bullet point above). It is okay to be wrong, as wrong is a subjective quality, provided each of the atomic decision units are each operating correctly. When that is not enough you add further logic to reduce the interference of the various decision units upon each other. In the case of various external factors imposing interference you must ensure your application is isolated and testable apart from those external factors so that when such defects arise you can eliminate as much known criteria as rapidly as possible.
You will never be sure your open-ended system works as intended 100% of the time, but with enough test samples you can build confidence against a variety of unknown combinations.
An unknown input producing correct results is still a problem - the unknown input is the problem.
Therefore, i postitulate that anytime an unknown input is possible, the software is defective.
You expect that people will move forward, left, or right.
You didn't expect people to try moving backward.
People start moving backward, but the software happens to do the right thing due to how it was written.
Is the software defective because of your missed expectation?
To be not defective, the software has to explicitly reject input that it was not designed to handle.
Imagine if the software updated with some changes, and the unknown input now produces an incorrect output. Is the defect introduced with the changes? Or was the defect always there?
In some cases that breaks forward capability. e.g. the case where there is an unknown XML tag. You could reject the tag or message. You'll end up rejecting all future change inputs.
If the whitelist of acceptable items is large, it may be acceptable to have a black list however if the above holds, you don't know what you don't know.
If you have alternate sensors, you should trust them them more, and camera systems less.
If you have a sunshade, you should deploy that.
If it is raining, or partially cloudy, the situation may change rapidly.
And perhaps you should slow down, but if you slow down too fast, other vehicles might not be able to avoid you.
It's not professional to design systems that rely on luck.
"Let's ignore this edge case and hope we get lucky" is not something you want to see in a software specification.
Turned out the bug had been latent in the code for 5+ years, predating me. Its data consumption had never been observed before because it was swamped by other data consumption. Changing the architecture to remove that other data brought it to the foreground.
(fwiw, the bug was caused by the difference between 0 and null as a C pointer!)
Another way to think about this is that the more open a system is the more useful and risky it is. It is useful because it can do more while tolerating less terse requirements upon the user. It increases risk because there is more to test and less certainty the user will get what they want. Part of that risk is that its hard to guess at what users want as sometimes the users aren't even sure of what they want.
There's a difference between "breaking" unknown input - i.e. non-computable within the system as it stands - and "working" unknown input, which is within expected working parameters.
The latter is business as usual for computing systems.
The former should have a handler that tries to minimise the costs of making a mistake - either by ignoring the input, or failing safe, or with some other controlled response.
It may not do this perfectly, but not attempting to do at all it is a serious design failure.
There's a saying that when people figure out how to make a computer do something well, that it's no longer in the field of AI. I'd say there's some truth in this, in that for many problems we have solved well (e.g. playing chess), the intelligence is not that of the machine, but of the programmer.
I think that in order for a machine to genuinely be intelligent, it must be capable of original thought, and thus unknown input. Known doesn't necessarily mean specifically considered, but that it could be captured by a known definition. As an example, we can easily define all valid chess moves and checkmates, but we can't define the set of images that look like faces.
If there appears to be a cow in a random field the odds are extremely low that someone put a papier mache cow there. If there’s something that has 50 % chance of being a snake you panic and run because that’s a 50 % chance of dying.
In the case of the authors bug yes the change he introduced had a good probability of being the cause. However he could have increased the probability by going back over commits and confirming that his exact commit introduced the bug. Now the probability goes even higher. But it could still be machine specific a cosmic ray or whatever but the odds are over whelmingly low.
In practice causal reasoning also works in a probabilistic fashion.
I have a simple model saying that if a planes engine is on then it’s flying. Its a single step probability and so it’s not very accurate in the real world.
I do a bunch of experiments and say a plane is flying if the engine is on and air is going over it’s wings faster than a certain speed.
Now we have two correlations connected by a causal model that works in many other cases. Hence the probability of it being correct rises.
But at the same time we should never mistake direct correlation for causality. But in daily life it’s “good enough”.
- We have a definition of what a cow is, and we know that cows are discrete/physical objects, and have relatively fixed locations (i.e. that they are not like an electron cloud with a probabilistic location).
- We assume that fields A and B in your hypothetical have clear, non-overlapping boundaries.
- We assume that we are working in a fairly normal universe with a fairly standard model of physics, and that due to the way time works in this universe, a cow cannot simultaneously be located in both fields A and B.
- ...
- (this could get really pedantic and go on forever)
The point is, even the things "we can know with certainty", are only as certain as the framework of observations/deductions/axioms/etc. that they rest upon. Almost nothing is certain on its own, without any further layers of reasoning behind it.
I see OP's comment not as denying knowledge, but rather as clarifying what is meant when people speak to each other of knowledge.
It seems that the philosophers were grasping towards a definition of "know" that encapsulated the idea of assigning 100% probability to something, after incorporating all the evidence. From a Bayesian standpoint, this is impossible. You can never be 100% certain of anything. To "know" something in the sense of "a justified, true belief" is impossible, because a belief is never both 100% and justified.
(Note that it is entirely possible to see the paper mache cow and conclude that it is likely only paper mache and that there is not a real cow in the field. Is this belief "justified"?)
It's tempting to think of "knowledge" as some relationship between the mind and a single fact. But when we use the word "knowledge", what we actually mean is "an accurate world model" - a set of beliefs. This is the disconnect that Gettier cases are designed to expose - they construct scenarios where someone's mental model is inaccurate or incomplete, yet by sheer luck produce a single, cherry-picked correct prediction. We are uncomfortable calling these correct predictions "knowledge" because as soon as you start probing the rest of the mental model, it falls apart. Sure, they think there's a cow in the field, and there really is one. Ask them any more questions about the scenario though ("what color is the cow?") and they'll give wrong answers.
From this perspective, "knowledge" as a "justified, true belief" is a perfectly coherent concept - the problem lies with the inadequacy of the word "justified" to describe the output of a complex decision procedure that incorporates many beliefs about the world, such that it could be expected to yield many other correct predictions in addition to the one in question, up to some arbitrary threshold.
A thought experiment - suppose you tell the observer that the cow they see is made of paper mache. They no longer believe there is a cow in the field. Intuitively, has their knowledge increased or decreased?
For example, Nassim Taleb has an argument of IQ being a single-dimentional assessment to a multi-dimensional domain.
I think it's more practical to have a possibility space (where unknown unknowns is a possibility). This removes the need to assess probabilities (which will probably be incorrect) while having being able to per-mutate through the list of possibilities. One can also do logical deductions, based on the possibility space, to assess possible strategies to explore/solve the issues at hand.
You can apply that to the real world if you then add social processes and see "knowledge" or "truth" as shared belief between a chosen set of people (or all of humanity). Then you can go down the whole rabbit hole of beliefe aggregation and voting theory.
So much philosophy is playing with words.
Eastern philosophy has nailed this thousands of years ago and we westerners are up to this day totally in the dark. We actively treat the I as a concrete object that really exists as an entity. It does not hold any closer examination and evaporates entirely the closer it is questioned.
That's just part of how our language works. It doesn't seem to matter whether I am a "concrete object" or some swirly pattern of becoming or indeed even an illusion! The English word "I" does not refer to an eternal soul or "atman."
If you stare long enough at an ice cream you'll have the marvelous insight that in reality there is no concrete ice cream entity, not least because it melts. Yet people don't go around saying "wake up, there are no ice creams!" Why is that?
Anyway your definition is wanting. A religious scientist has two kinds of clearly different beliefs: faith and knowledge. A mathematician has the same two kinds, under different names: axioms and deductions.
Saying that axioms are the same as deductions is a radical claim.
Why deny the word has meaning, just because you can't distill it down to a concise explanation? The meaning of a word can be arbitrarily complex. Knowledge can exist, even if you can't define it, because meaning is not determined by definition. Definitions are simply a mechanism for coordinating understanding, not for demonstrating its existence.
Besides, any argument against the existence of knowledge could be used against belief. Play "taboo" with the subject and don't confine yourself to using ancient terminology to describe the world and these pointless linguistic problems melt away.
And while everyone loves to run in circles around the argument “but how can you know with certainty” the fact is that I am as certain that this assumption holds as I am that it provides no value at all to continually question if reality is really real, you’d have to take that as an assumption to ever have any kind of value adding discussion.
The people who insist we can’t know if anything is definately true must agree that they can’t know if that assertion is definitely true, so they sort of kill their own argument axiomatically.
I beg to differ. Besides the examples in programming the author gave, I can very easily think of examples in medicine, police work (e.g. regarding suspects), accounting, and so on...
Software engineering has many of these Gettier cases, because most software engineers do not follow the scientific method when investigating a problem!
You believe X has cancer because he has the symptoms and you can see an offending black spot on their X-ray.
The lab results say the black spot was just a cyst but X indeed has cancer in the same organ.
#1 because one of the reasons for your believing X has cancer is that "he has the symptoms", which (if I'm understanding your example correctly) is in fact a consequence of the cancer he actually has; so, at least to some extent, your belief is causally connected to the true thing you believe in just the way it's not meant to be in a Gettier case.
#2 because (so far as I know) it's not at all common to have both (a) cancer in an organ that doesn't show up in your X-ray (or MRI or whatever) and (b) a cyst in the same organ that looks just like cancer in the X-ray/MRI/whatever. I'm not a doctor, but my guess is that this is very unusual indeed.
So this isn't a very convincing example of how clear-cut Gettier cases aren't rare: it's neither a clear-cut Gettier case nor something that happens at all often.
I don't think this is rare -- including in the version of my example.
The only reason there's arguing that it's not a "clear cut case" is that I mentioned seeing "symptoms". Ignore the symptoms I mentioned, as they are a red herring, e.g. seeing the mark could cause the belief alone.
Other than that, it's a belief (1), that's justified (2), and true (3) -- while being accidentally justified.
Consider the case of a policeman that things someone is dangerous because they think they seen a gun on them. So they shoot first, and lo and behold, the suspect did have a gun on them -- but what the policeman seen was just a cellphone or something bulky under their jacket.
Or the spouse that thinks their spouse is having an affair because they see a hickey. Their spouse indeed has an affair (and even has a hickey on the other side of the neck), but what their spouse saw was just some small bruise caused by something else.
Or, to stick with the theme, figuring domestic abuse, and the victim suffers that indeed, but your guess is based on a bruise they had from an actual fall.
I didn't write "one can easily" to imply I have some special talent to imagine such situations (and thus had motive to leave examples off to hide the fact that I don't).
I wrote it because I really do believe one can easily find such examples, and wasn't it even worthy to go into details (since I mentioned medicine, police work, etc, I thought the cases I implied where pretty clear too).
In any case, I gave 3 examples in a comment above.
Matching it to the example of the papier mache cow doesn't really work because the papier mache cow hides the real cow but it is very easy to see that your code was also checked in with other people's code.
An intertemporal variant of this is race conditions. There have been lots of problems of the form "(1) check that /tmp/foo does not exist (2) overwrite /tmp/foo"; an attacker can drop a symlink in between those and overwrite /etc/password. The file that you checked for is not the same file as you wrote to, it just has the same name. This is an important distinction between name-based and handle-based systems.
When you see the cow (but it’s really a convincing model), then in your mind, there should be some probability assigned to a variety of outcomes. The main one would be a cow, another one might be that you’re hallucinating, and so on down the list, and somewhere the outcome of cow-like model would be there.
From that point you can go in at least two directions, one would be something like a Turing test of the fake cow... beyond a certain point it’s a matter of semantics as to whether it’s a real cow or not, or you could say that your “justified true belief” had to apply to the total state of the field. If you believed there was both a cow model and a cow behind it, that woukd be justified, but the existence of the cow behind the model would not justify incorrect belief that the model was a real cow, in the sense of not admitting uncertainty over the things you see.
And it leads to a funny thing. You saw the model of a cow, and it make you believe that there is a cow in the field and that you saw a cow. Then you could find a heap of poo, and you will strengten your beliefs futher. You might find a lot of evidence and it all will be explained under assumption that you saw a cow. And this evidence will strenghten your belief that you will be licked in the face, when you come near the cow.
But you didn't saw the cow that made this heap of poo. The real cow is pitch black, with a horns of gigantic size and they are really sharp. The real cow has red glowing eyes and it is going to kill you. But before you see the real cow itself, all the evidence that would point that there is a cow would also reinforce the idea of soft tempered black and white cow. The longer you manage to keep youself oblivious to real cow traits, the more surprised you will become when you find the real cow.
You're replacing the model it was criticizing with a different model and then saying that it doesn't say anything interesting about your model, so it's not interesting. It's not an argument that knowledge isn't possible, it was an argument against the traditional definition of knowledge as it was almost universally understood at the time.
Nothing is absolute.
Example:
1=1 is something i know is true cause i know the rules of mathematics. There is no absolute truth to that.
[1] In America anyways.
Sometimes I think that is what philosophers are doing - feeling clever - perhaps as a defense against some negative inner problem (psychology is an outgrowth of philosophy after all). The whole cow story stinks of telling someone "you're right, but you're also WRONG! Your perception of reality is BROKEN!". To me knowledge is simply having a model of the world that can be used to make useful predictions and communicate (and some other things). Aside from that, it doesn't matter if your model is "grounded in reality" until it fails to work for you, at which time it can be helpful to realize your knowledge (model) needs adjustment.
One way to resolving the authors first software issue would be to check a diff between what he committed and the previous production revision - this would quickly uncover the changes he "didn't make". This is an old lesson for me - what I changed may not be limited to what I think I changed. It's a lesson in "trust but verify". There are any number of ways to view it, but in the end we only care about ways that lead to desired outcomes weather they're "right" or not.
On a related note, I've found that software is one of the only places where there is a "ground truth" that can be examined and understood in every detail. It's completely deterministic (given a set of common assumptions). I've found the real world - and people in particular - to not be like that at all.
The first two have all the problems philosophers talk about. But the last one does not. Not even underdeterminism, unless the system of the model is fundamental or fades into history or is a "wicked" problem.
All science is an outgrowth of philosophy.
It's very frustrating when people look the obviously trivial and sometimes silly examples that philosophers use to elucidate a problem, and take it to mean that they are interested in trivial and silly things. Being right for the wrong reasons is a common and difficult problem, and some if the solutions to it a really insightful and powerful ideas.
> Aside from that, it doesn't matter if your model is "grounded in reality" until it fails to work for you, at which time it can be helpful to realize your knowledge (model) needs adjustment.
It might matter a great deal if your model is not grounded in reality - there are situations where that can kill you. It also seems like one of the fundamental aims of science, to have theories fail less often.
What feels like a pointer is actually a category. That is, it feels like it points to one, but it points to many. Like both examples given here: https://en.wikipedia.org/wiki/Gettier_problem .
In the example the Mental Model was at a level too shallow, it should have only affected the paths between the autofocus and the user. But the bug necessitated a larger mental model (the author was considering too small subsection of the graph).
I'd hope in the future we could reach a state where the program could have detected that the frame refactor would have an affect on the autofocus and all other components instead of being an implementation detail.
In the opening match, Kota Ibushi suffered a concussion. Some doctors came out, carried him out on a stretcher, and took him to the back. As it turns out, this was all planned. The doctors were fake, and this course of events was determined ahead of time. But coincidentally, Ibushi _actually_ suffered a real-life concussion in the match.
Wrestling always has an interesting relationship with reality.