A more sane* way to implement the single lookup behavior is:
def values_in(keys, d):
ret = {}
for key in keys:
try:
ret[key] = d[key]
except KeyError:
pass
return ret
which works much more quickly in cases where keys is nearly all of d.keys, and doesn't force you to reinvent monads in Python, which is arguably unsuited for them.
* Sanity values may vary for people who haven't gotten into the habit of using Python exceptions for little things.
The article is looking at returning a list, while your option returns a dict, but that's an easy fix. And even for an Option-like approach, the article tries way too hard: Option can be viewed as a constrained cardinality list, and regular lists work just fine to implement them.
from itertools import chain
def getOpt(d, k):
return [d[key]] if key in d else []
def values_in(keys, d):
return list(chain(*[ getOpt(d,key)
for key in keys ]))
Author of the article here. The Python solution using lists-as-options is actually even simpler:
def values_in(keys, d):
return [x for key in keys for x in getOpt(d,key)]
Using lists as options is a neat trick I hadn't thought of when writing the article. Unfortunately, no dynamic language uses this idiom (or really any other non-exception option-like patterns) natively.
> The Python solution using lists-as-options is actually even simpler:
Good point. For some reason, I have this giant python blind spot for comprehensions where the sources aren't orthogonal -- I miss them in the first pass all the time; I normally notice it fairly quickly in code I'm not posting on a discussion forum...
> Unfortunately, no dynamic language uses this idiom (or really any other non-exception option-like patterns) natively.
Yeah, to use it in your own code in most popular dynamic languages, you have to wrap library code that works differently. The good thing is that most popular dynamic libraries have good (largely functionally-inspired) tools for working with lists and other iterables that the payoff is pretty good. (Of course, with duck typing in most popular dynamic languages, one thing you could do to be more explicit is to combine the Option-as-class and Option-as-list approach, and make Option a class that implemented the interface used by immutable collections.)
Actually, at least in PHP, I find myself using arrays and other iterables as pseudo-Options quite a lot; eg. when checking the existence of a DB value, xpath element, etc.
// Using if/then/else
function foo($x) {
$results = look_up_something($x);
if (count($results) > 0) {
$val = $results[0];
return do_something($val);
}
}
// Same as above, but with a foreach loop
function foo($x) {
foreach (look_up_something($x) as $val) {
return do_something($val);
}
}
Of course, the nice thing about Options (other than being able to distinguish between "None", "Some None", "Some (Some None)", etc.) is the availability of combinators to lift and compose non-Option functions with optional functions. Sometimes we can map and fold ("reduce"), but sometimes we can't (eg. PHP's "array_map" doesn't work on iterable objects, which many APIs use in lieu of arrays).
That version (and the one in the article) are somewhat confusing to parse and detract from the point of the article.
This seems more "Pythonic":
def lookup(d, key):
return Just(d[key]) if key in d else None
def values_in(keys, d):
values = []
for k in keys:
x = lookup(d, k)
if x:
values.append(x.value)
return values
If you really want a one-liner, I think this is easier to understand:
def values_in(keys, d):
return [x.value for x in (lookup(d, k) for k in keys) if x]
Well, considering that this is an indirection on top of the `in` operator which will slow the whole thing down, of course they're not typically implemented in idiomatic solutions in Python.
The obvious being said, you can also use functions and lambdas to get around having to create custom classes or abuse lists.
def value(x):
return lambda: x
def values_in(keys, d):
return [x() for x in (value(d[key]) if key in d else None for key in keys) if x]
def get_maybe(d, k):
return value(d[k]) if k in d else None
def values_maybe(keys, d):
""" If you want the caller to be responsible for handling the maybe case """
return [get_maybe(d, k) for k in keys]
For this case and similar cases where the API provides a "supply your own default value for the failure case" option, this kind of "unique one-time null marker" is actually a very good approach that addresses much of the problem with standard null values.
Its not as composable as Option- or list-based approaches, though. (OTOH, you can use it to implement those approaches, and its cleaner for that than the way I did that elsewhere in the thread.)
Null, empty string, empty value, Undefined etc. are the biggest causes of crashes,edge case bugs and a big time sink programming and data handling. Wish languages had better support to handle straightforward cases instead of just crashing.
Crashing is ok. The problem is that it usually doesn't crash in the right spot. A null is generated somewhere else and propagated around for a while and the crash happens later in a unrelated area. But it's even worse if it doesn't crash at all.
This is close to saying that lack of oxygen is the number one killer in drownings. That is, when you are in that situation there is a lot going wrong.
So, sure, you got rid of null from the language. Does your program handle the case where it was handed a negative number correctly? All positive values? All unicode?
Consider, in the Ariane Rocket incident, they were using a dependently typed language. Didn't magically prevent errors in assembly.
So, fine, we'll make our programs where they handle all cases... Rapid prototyping probably just flew out the window. As did worrying about actual honest to goodness use cases.
A great quote by Knuth I saw once was where he acknowledged that tex would choke on mile wide documents. But who cares?
> So, sure, you got rid of null from the language. Does your program handle the case where it was handed a negative number correctly? All positive values? All unicode?
Aren't you just moving the goal post? You can play the 'OK so you can eliminate that from the equation, but can you ...' indefinitely. Well, perhaps only until you get to the question of cosmic rays.
> So, fine, we'll make our programs where they handle all cases... Rapid prototyping probably just flew out the window. As did worrying about actual honest to goodness use cases.
The topic is a few special cases, like null, not every conceivable case. If things like null and undefined cause the majority of crashes, and they are simple to handle with a 'proper' language (and they are), then you already have gotten a lot of bang for your buck.
For rapid prototyping: if handling null cases is too much, use a function that unwraps nullable values directly, and if they are null, crashes (and no, this is not the same as a null pointer exception). Then go back and clean up the program when you are not prototyping quite as rapidly anymore.
That is my point, that this is really just moving some goal posts. So, we know that many programs didn't make it across the line with null as an option. Will they really do much better without it? I'm not convinced it is a sure thing.
So, for example, a place where null comes up. When you create a file. Programs that didn't check just threw up on an exception now. It is just as likely that without that option, the code would have just checked for a value, or thrown an error. So... what changed?
Maybe they don't throw an exception immediately. Maybe that null value gets stored somewhere, and you don't hit it until three hours later, when the system goes to save data to disk. Now your exception gets thrown far away from where the error occurred, both in terms of code and actual time.
(For the record, I'm assuming an interface such as:
fileOpen -> FileHandle
Where the FileHandle object can be null, and it being null is used to indicate that the call to fileOpen failed.)
Couldn't that happen with an optional type, as well? Especially if you let it get inferred. Only when you go to use it would you realize you have to map over the value. And then you are back in the world of messed up.
It depends on the language, and the API of the libraries you use. If we have a Haskell-like Maybe type, you should be able to create functions that refuse to take a Maybe. That would force you to determine soon after the fileOpen call: did it succeed or not?
To be clear, this would involve the static type system for your language forcing certain things to happen, or not compiling. The sequence:
1. fileOpen returned a Maybe<FileObject>
2. You want to store the FileObject somewhere, and it expects an actual FileObject.
3. Maybe<FileObject> is not itself a FileObject, so you have to figure out: is it Nothing, or a FileObject?
4. You have to handle the Nothing case (language forces you to, even if you do nothing about it - you explicitly decided to do nothing).
5. If it is indeed a FileObject, you store it for later use, and you are guaranteed it is, indeed, a FileObject.
Right. This makes sense. Though, I would have thought the lazy semantics of Haskel would trip you up here. Ultimately doesn't matter, my main argument is that I see people proliferate "Optional" values all over the bloody place. So, I would expect to find that they just put the optional ref up and used it later when necessary.
And I should note that I don't necessarily dislike the optional types and such. I just also don't have the major disdain for null that the internet seems to send through a massive echo chamber nightly.
The problem with languages that have null is that they don't let you write non-nullable types. That means any variable at any time could be null. You don't have the option of guaranteeing something won't be null.
> The problem with languages that have null is that they don't let you write non-nullable types.
That's a problem with Java (because it doesn't let you do that) and a similar problem exists with dynamic languages (because types aren't attributes of variables, so any variable could be null -- or any other value, null isn't particularly exceptional in dynamic languages for being a source of this kind of problem), but its not a problem "with languages that have null", as there are statically typed languages that have null but distinguish nullable and non-nullable types (C# and other static .NET languages are probably the most well-known examples, though there are others.)
C# only kind of gives you this. When I said "the problem with languages that have null", that's really what I meant. There definitely could be a static language that let you choose if something is nullable or not. But really, at that point it's just sugar for an option type.
C# is absolutely a static language that lets choose if something is nullable or not, there is no "kind of" about it.
> But really, at that point it's just sugar for an option type.
No, having a the free choice not use nullable types doesn't make the languages nullable types into sugar for option types. Nullable types don't compose the way option types do -- the issue in the article this thread is about isn't about whether or not you have the choice for nullable types, its about the fact they don't compose so you can distinguish a function returning a null as a substantive value because the thing you tried to store was also a nullable type and had a null value (Just None, in a world with Options instead of nullable types) or where the null is a result of the lookup failing (None, in a world with Options instead of nullable types.)
The deficiency of nullable types in C# compared to Options isn't the lack of choice of writing non-nullable types -- as you certainly have that choice for any data you need to represent -- its that nullabe types don't compose the way Options do.
As far as I understand, the only way to get a type that's not nullable is to make a struct. How do you specify that a function can take a non-nullable string?
> As far as I understand, the only way to get a type that's not nullable is to make a struct.
Well, any value type -- but, yes, a custom type has to be a struct to not be nullable.
> How do you specify that a function can take a non-nullable string?
"string" in C# is the name of a particular reference (and hence, nullable) type. If you want a function that takes a non-nullable sequence of characters, that's quite possible, but the type of its argument won't be string. (The fact that there is no such type predefined and the entire .NET framework library is built around the assumption that if you want a sequence of characters you are going to use the built-in, reference -- and therefore nullable -- string type is an issue with the cost, in programmer-time, of doing this for any substantial use, but its not a "the language doesn't let you have a non-nullable type for any kind of data you want" issue.)
Having worked on some projects that have more bloody optional variables than I can shake a stick at, I wouldn't claim there are no drawbacks. The cognitive overhead of some of them is bloody heavy. Especially if there is poor judgement on when a parameter is optional or not.
I prefer to architect the code such that it is only possibly null in one section of the code. For the rest, it should never be null. Pretty much period.
Granted, this can certainly be done with optional types. So, I'm not ultimately against them. I just think they are oversold quite a lot.
It's not just that this "can be done" with optional types - this is precisely what optional types are for. And they help make sure you're never mistaken about that "pretty much". Like any tool, people can misuse it (propagating them everywhere would be an example of this, partial extraction functions would be another).
Right, that is essentially my point. I do still hold more pessimism than I probably should, but by and large my main objection is just to the nature with which optional types are sold. They are almost like the vitamins of programming. Great for when they are really needed, probably wasting a ton of time for where many people that just found them put them to use.
> I would have thought the lazy semantics of Haskel would trip you up here.
Laziness doesn't come into it, since you won't even manage to compile your code, let alone run it, if you try to use a "Maybe FileHandle" when a "FileHandle" is expected.
With that said, laziness does cause problems related to file handles, but those have nothing to do with unhandled failures. The problem is that files are a limited resource which we should release as soon as possible (just like memory, sockets, etc.). Lazy evaluation is at odds with this, since it essentially defers function calls to the last possible moment. One way of thinking about it is that laziness is "call-by-need", but we never really need the return values of functions like "closeFile".
Right, I would have thought you could get tripped up by getting a FileHandle that is invalid, but you wouldn't know it till you get to the point that you use it. Though, I am clearly not familiar with IO API design in Haskel. :)
If something returns Maybe FileHandle, and it can't provide you a valid FileHandle, it'll provide Nothing (and if it can provide a valid FileHandle, it will provide "Just" that filehandle).
Edited to add:
Note that this can be the case in Haskell when you're relying on exceptions for error handling. That's the opposite approach from option types, though.
In my case, I thought the laziness could be more crazy, where you created a function that would evaluate to a FileHandle, but until you actually evaluate that function, you don't have a FileHandle.
So, depending on just how "lazy" the program truly is, sure you passed it something that unwraps a Maybe, but only once you actually go to use it. That make sense?
Consider this fun bit of nonsense scala
def foo(x:=>String): Unit = {
println("Hello")
println(x)
}
If I call this as "foo(None.get)", then it will print "Hello" before barfing. In fact, if I didn't evaluate x, it wouldn't even barf. I had thought the laziness of Haskel was that ++. (Meaning I thought if I stored x off to another also lazy reference, it wouldn't barf right away.)
To make that a little more clear of what I meant, some even more amusing scala.
class Foo(x: => String) {
def happy = println("Doing good.")
def sad = println(x)
}
If I create something with "new Foo(None.get)", I can call happy on this object all day long. If I ever call sad, I get an exception.
The relevant gotcha is when doing lazy IO in Haskell. The general answer is "don't do that" - there are a number of approaches for how. (Lazy IO is fine and quite convenient for things of roughly the scale of a shell script. It's not a good fit to bigger things.)
I'm not sure I see how this changes much. My point is that with laziness, your explosion doesn't happen right where you unwrap a Maybe in the code. It happens where you evaluate the unwrapping.
Apologies for the ninja edit above, but this scala seems to show it isn't just IO that is at danger here.
class Foo(x : =>String) {
def happy = println("I'm good.")
lazy val ack = x
def sad = println(ack)
}
Again, if I do "new Foo(None.get)", I can call happy as much as I want, but "sad" will blow up.
Now, I do not and could not claim that this sort of nonsense would have proliferated with the same abandon as null. I am claiming that this is not that unlike a null pointer exception, though.
I think the equivalent to None.get is fromJust Nothing. And yes, it's the equivalent of an unchecked nullable. The Haskell community generally frowns on fromJust (along with other unnecessarily partial functions). The better approach is to either pattern match:
case maybe_string of
Just string -> newFoo string
Nothing -> defaultFoo
Or to use one of the relevant higher-order functions:
Yeah, exception resolution is pretty weird in Haskell due to laziness. In some sense, you end up thinking of exceptions as being buried in values in the language instead of exposed by computation—this is your null-value landmine problem exactly
This is also why the Control.Exception module exposes `evaluate`, which embeds a value in an IO computation and thus resolves this problem... If you remember to use it.
Fortunately, in idiomatic Haskell exceptions are considered to be an expert feature (basically I see them in concurrent IO or resource management code only) and partial functions—usually introduced by incomplete pattern matching like `None.get` does—are veboten.
Rather than being specific to IO, I think you're missing an understanding of algebraic data types, and their use in strongly and statically typed languages. The wikipedia entry is a decent place to start: http://en.wikipedia.org/wiki/Algebraic_data_type
Actually, I think I'm just having more of this conversation in my head than I am in this forum. For better and worse. :)
My point here was more that just because you have a file handle doesn't mean you have a valid place to write data. Making it optional would possibly prevent some bugs, true. However, it doesn't really help as soon as you have a filehandle to a full filesystem, for example. (Or, well, any other problem that usually happens to cause grief with the filesystems. You got a file, but by the time you went to use it the system was full and you couldn't, etc.)
Regardless, I should have been clearer on many points (and I fully accept I was flat out wrong on at least a few :) ). I do think null pointers are bad. I also know with proper static analysis tools, you don't need a whole new type system to make things better. Unless you are considering tools such as Coverity and friends some form of type system.
I don't pretend my understanding is the most common understanding. I think it squares better with the state of research on these things than the most common understanding, though. I could absolutely be mistaken - I am more a practitioner than theoretician.
It makes sense that the theories behind them are the same. The distinction I make is whether or not the type is directly conveyed in the language. I don't know if that is a false distinction or not.
I think that is mostly a false distinction. Types that are not directly conveyed in the language (and by that I assume you mean "that the compiler is aware of" and not "that are syntactically spelled out all over" - inference accounting for most of that difference) obviously cannot be used to specify semantics.
I think the only other differences stem ultimately from standardization. Tooling could exploit any type system a program is known to follow. Likewise, compiler optimizations, &c, &c.
Tooling can also go beyond types, though. Interleaved calls to different types, for example, can be found through tooling. Types can't really do that.
Well, I realize that is false. But the hoops all of the examples I have seen that types have to jump through to accomplish that aren't exactly pleasant.
So, maybe my preference is just that tooling can give you many of the benefits of more advanced type systems, without having to have all of the boilerplate of it everywhere in the code.
At any rate, I should say thank you very much for keeping this dialog going. Been a blast!
By "tooling" there, I wasn't referring to static analysis. I meant things like type-directed name resolution, type-directed completion, Agda's "holes", and so on. Strictly, these do not rely on the type system being "a part of" the language, but they do rely on some degree of standardization on a particular type system for the tool to work (or work well).
"Interleaved calls to different types, for example, can be found through tooling."
I actually don't understand exactly what you mean here. You might be surprised what even pretty rudimentary type systems can find if you leverage 'em right, though - as I mentioned in that other thread, I track which threads access what resources, and which functions live on which threads, with C's type system unadorned by additional checks (I do use additional static analysis, it just is entirely unnecessary to catch this one).
"Well, I realize that is false. But the hoops all of the examples I have seen that types have to jump through to accomplish that aren't exactly pleasant.
"So, maybe my preference is just that tooling can give you many of the benefits of more advanced type systems, without having to have all of the boilerplate of it everywhere in the code."
Ironically, I think my C code has more annotations for my static analysis tool-of-choice than the C types. Certainly there are a lot of poor type systems, and a lot of languages that expose particular type systems poorly. Boilerplate type annotations are mostly unnecessary in languages with type inference.
As I've said elsewhere, though, your tooling may be "more advanced type systems". Splint, for instance, can enforce linear typing - something that is not a part of the type system laid out in the C standard (or that enforced by typical C compilers, even with warnings).
"At any rate, I should say thank you very much for keeping this dialog going. Been a blast!"
Laziness is a run time question. Nullability is a compile time question. Nullness, given nullability, is a run time question. Laziness in no way enters into this.
See my response upthread https://news.ycombinator.com/item?id=8007555. Basically, I was under the impression that it wouldn't actually unwrap the Maybe until you went to actually use it. I think I was just holding the laziness a little too magically. :)
I'm confused, then. Sounds like it can have obscure, "doesn't bite you immediately and appears far away from where the problem actually is," properties that were attributed to null.
That is, at the point that a Maybe String statically changes to a String is not necessarily where the program will explode. A lot like a null assign. (Again, in a lazy language. Unless I'm mistaken on this subthread, the point was simply that you can still have that level of WTFery with Maybe that you can have with a null.)
Maybe doesn't have to be involved at all. The point where the exception is raised is the point when the relevant thunk is forced. It is certainly the case that this can be problematic. This is an artifact of the intersection of laziness and IO, and there are methodological approaches (helped very much by libraries) to it, but it is a similar problem to nulls. Note that there are plenty of languages that are strict and provide an optional type - including Idris, I think, which is Haskell-like, dependently typed, and strictly evaluated.
Right, this particular subthread was started by my saying that the laziness of Haskel could trip you up on the conversion from a Maybe x to an x. Which seems to be true, from what we are saying now.
Now, rereading the beginning, the reference was to a "Haskel like" type. So, I think I was definitely off the rails on this. I do feel I learned something going down this particular rabbit hole, though. Thanks!
This takes a default output to use if you're given Nothing, a function to apply to the input and generate an output if you're given Just an input, and Maybe the input, and gives you the resulting output. As I said this is often more useful than "and use this default", because quite often the value you want to generate when it's Nothing is outside the image of the input under the function in question. For instance:
maybe "no number" show maybe_integer
There's no number I can pass to show to get "no number", maybe lets me very conveniently avoid trying to force one.
That is my point, that this is really just moving some goal posts.
Yes, they will. You could make the same statement about crime: "We'll never get rid of all crime, so why bother having police at all? Having police is just moving some goal posts." Does that make it clear enough how silly this sort of reasoning is?
Note that I am not actively arguing against adding some of these solutions. I just don't have an idyllic vision of a world where we have better software simply because null doesn't exist. Specifically, I'm not convinced "all of the null pointer bugs" would have simply been erased and replaced with good working software. Rather, I tend to think they would have just been replaced with similar bugs.
> Specifically, I'm not convinced "all of the null pointer bugs" would have simply been erased and replaced with good working software. Rather, I tend to think they would have just been replaced with similar bugs.
This reads to me like eliminating null pointer bugs would just create a void that something else would fill? I don't see how that would work, at all. Is eliminating a null pointer exception going to introduce a divide by zero exception/bug? Is it going to introduce program logic bug? ...
If you're thinking about unwrapping a nullable value and throwing an exception if it really is null; I don't think of that as a conceptual null pointer exception, at least not in the usual sense. It's a deliberate decision along the lines of "I know [because of the control flow up to this point] that this value can not be null" or "if it is null, just throw an exception". It is NOT "this value can never be null... I think... well let's check the documentation, OK I'll assume that it isn't being misleading (documentation can easily 'lie' about a value being null in a nullable-by-default language, since non-nullable is not expressible in the language).
In a language with nullable-always, you can always pass a value (probably reference, but still) as null. In a language with explicitly nullable types, this is impossible. That's why unwrap functions throw an exception if the value is null: there is no other way to handle that case in a typed manner (well, I guess you also could loop forever...). In a nullable-by-default language? It is always fine to pass null. And it will not necessarily fail if a null is passed that shouldn't have been passed: it will only error when the reference is (tried to be) dereferenced.
It may introduce a program logic bug. Or arguably, it may have already been a program logic bug that remains a program logic bug. I would be tremendously surprised if this was close to 100% of the time, though, and even at 50% we're putting a good dent in our bugs (with, of course, plenty remaining, but that's no reason to forgo this).
My main hypothesis, which I of course never said, is that current static analysis tools should be able to catch the majority of these cases anyway. Especially the ones that matter.
That is, a type system is not the only tool we have in static analysis. So, I'm not completely convinced that moving to a type system is the best way to go.
Granted, it large part, it is clear it doesn't matter "what I think." Statically typed languages have a massive amount of support.
With the isomorphism between proofs and types, most of the stronger guarantees that any static analysis tool can give you are in some sense types. General "smell" type linting stuff, perhaps not.
Makes sense, and I'd imagine that in large part that the theory behind them is the same. The practice of the two is different, though.
Specifically, majority of the static typing regimes seem to demand a change in how software is written. Whereas most of the static analysis seem to simply provide stronger guarantees on what you are already writing. That is a huge difference.
I think this is an artifact of the code you find yourself wanting to write relying on guarantees closer to the type system the static analysis tool applies (implicitly or explicitly). Some of this probably varies by domain, some by language, some by familiarity with particular type systems, and some simply by individual.
I balk at the faster claim. Specifically for existing programs. It seems that it should be far faster to run a new analysis tool over an existing codebase than it would be to rewrite said codebase in a new statically typed language.
> So, for example, a place where null comes up. When you create a file. Programs that didn't check just threw up on an exception now. It is just as likely that without that option, the code would have just checked for a value, or thrown an error. So... what changed?
What changed is that you are unable to propagate a null-that-shouldn't-be-null value to the rest of the program.
If you have a value V that might be null, and you are supposed to return that value itself (not wrapped in an Option or anything like that, ie a 'nullable type'), all you can really do is: actually return a value of type V, or throw an exception. That's it! (To throw an exception is also something that can be done at any time in most statically typed languages, for any reason whatsoever. Throwing exceptions is usually not a question of the type correctness of a program (maybe it is in the case of checked exceptions?).) You can not give null as an output: you are supposed to give V as the output, not a nullable V. So you know exactly where in your program you are doing something 'unsafely', i.e. where you are not rigorously handling every case of the value of some type. Though 'unsafe' is a bit misleading, since you may very well intend to just throw an exception in the (perhaps deliberately possible) case that a value is null.
The Ariane incident shows that if you combine misuse of type checks and practice poor software engineering methodologies you can get failure, not that proper use of the type system wouldn't have prevented it.
The first value is the stored value in the hash table. If there is no key, the first value is NIL. The second value indicates whether the key is present in the hash table.
I list Common Lisp & its behavior in Appendix C at the end of the article. Multiple-return-value is definitely better than just returning NIL, but I still dislike it; it doesn't actively prevent you from confusing NIL with absence, so "the obvious way to do it" can still be the wrong way. A good CL programmer won't fall into this trap, but the article is in large part about how some languages/interfaces default you into good programming by requiring you to distinguish success from failure, and others don't.
Another way of seeing the same thing is: Type-theoretically speaking, MRV is using a product type (two values) to emulate a sum type (one value of two possible forms); why not just use a sum type?
The optimistic type theoretician says that since dependent products are sums then it's perfectly fine to use a product to emulate a sum... except since the dependence in the projections isn't reflected in the type system it's totally up to the programmer to ensure that the providence is maintained.
> In Lua, it's impossible to have a dictionary that maps a key to nil: setting a key to nil removes it from the dictionary!
Wait is he really trying to say, "Setting a VALUE to nil removes the entry from the dictionary!"? If so, that sounds terrible.
I can imagine barking up the wrong tree wondering why my map is missing fields when the real problem is an object I passed into it is unexpectedly nil.
Yes, that is what I meant. And yeah, it is weird behavior. (I think of "d[k] = v" as "setting the key k to the value v", so that's why I said "setting a key". Perhaps "binding a key" would be clearer?)
Yes, that's true. It does have drawbacks, but overall I think it is better than say, JavaScript. In JavaScript, a key can be not present in an object, be undefined, be null, be set to some other value, or be present in the prototype. The difference between these can cause very subtle bugs.
//Let's define some object
var obj = {a: true, b: true, c: true, d: true, hasOwnProperty: true}
//Now let's make them falsy in different ways
obj.a = undefined; //set to undefined
obj.b = null; //set to null
obj.c = false; //set to false
delete obj.d; //delete key
delete obj.hasOwnProperty //delete key, but present in prototype
console.log(obj.a) //--> undefined
console.log(obj.b) //--> null
console.log(obj.c) //--> false
console.log(obj.d) //--> undefined
console.log(obj.hasOwnProperty) //--> [Function]
console.log('a' in obj) //--> true
console.log('b' in obj) //--> true
console.log('c' in obj) //--> true
console.log('d' in obj) //--> false
console.log('hasOwnProperty' in obj) //--> true
console.log(obj.a == null) //--> true
console.log(obj.b == null) //--> true
console.log(obj.c == null) //--> false
console.log(obj.d == null) //--> true
console.log(obj.hasOwnProperty == null) //--> false
for(var key in obj) {
console.log(key) //--> prints a, b, c
}
In Lua, the only data structure is a hashtable, which maps any non-nil value to any other non-nil value. It can also be set in the objects __index metamethod/table
local tab = {a = true, b = true, c = true}
-- by default there are none, but we can defined a fallback metatable similar to a prototype method like js's obj.prototype.hasOwnProperty
setmetatable(tab, {__index={c='meta c', d='meta d'}})
-- if it is important that a key is explicitly set to false, do so, otherwise simply remove it
tab.a = false --set to false
tab.b = nil --set to nil
tab.c = nil --set to nil, present in fallback table
tab.d = false --set to false, present in fallback table
print(tab.a) --false
print(tab.b) --nil
print(tab.c) --meta c
print(tab.d) --false
The rules for Lua are simple: Look for the key in the table, if it's not there, check the fallback defined in the metatable, if not, then return nil.
This is especially important in Lua, because any value can be a key, you would have to be sure to delete the key otherwise it could not be garbage collected.
local tab1 = {1,2,3,4,5}
local tab2 = {}
tab2[tab1] = true
for k,v in pairs(tab2) do
tab[k] = nil
end
If this worked like JavaScript, then tab2[tab1] would return nil, but tab1 could not be garbage collected until tab2 is. Because JavaScript objects can only have strings for keys, this is less of an issue.
Furthermore, I've found that absent keys degrading to nil is actually a very useful feature that makes a lot of code really composable and simple. I can't think of any Clojure code I've written recently where a value of nil actually should be distinguished from an absent value - I just write my code so that semantically they mean the same thing, because this makes the code much easier to compose and much easier to think about.
88 comments
[ 1.9 ms ] story [ 168 ms ] threadhttp://www.oracle.com/technetwork/articles/java/java8-option...
* Sanity values may vary for people who haven't gotten into the habit of using Python exceptions for little things.
Good point. For some reason, I have this giant python blind spot for comprehensions where the sources aren't orthogonal -- I miss them in the first pass all the time; I normally notice it fairly quickly in code I'm not posting on a discussion forum...
> Unfortunately, no dynamic language uses this idiom (or really any other non-exception option-like patterns) natively.
Yeah, to use it in your own code in most popular dynamic languages, you have to wrap library code that works differently. The good thing is that most popular dynamic libraries have good (largely functionally-inspired) tools for working with lists and other iterables that the payoff is pretty good. (Of course, with duck typing in most popular dynamic languages, one thing you could do to be more explicit is to combine the Option-as-class and Option-as-list approach, and make Option a class that implemented the interface used by immutable collections.)
This seems more "Pythonic":
If you really want a one-liner, I think this is easier to understand:The obvious being said, you can also use functions and lambdas to get around having to create custom classes or abuse lists.
Its not as composable as Option- or list-based approaches, though. (OTOH, you can use it to implement those approaches, and its cleaner for that than the way I did that elsewhere in the thread.)
So, sure, you got rid of null from the language. Does your program handle the case where it was handed a negative number correctly? All positive values? All unicode?
Consider, in the Ariane Rocket incident, they were using a dependently typed language. Didn't magically prevent errors in assembly.
So, fine, we'll make our programs where they handle all cases... Rapid prototyping probably just flew out the window. As did worrying about actual honest to goodness use cases.
A great quote by Knuth I saw once was where he acknowledged that tex would choke on mile wide documents. But who cares?
Aren't you just moving the goal post? You can play the 'OK so you can eliminate that from the equation, but can you ...' indefinitely. Well, perhaps only until you get to the question of cosmic rays.
> So, fine, we'll make our programs where they handle all cases... Rapid prototyping probably just flew out the window. As did worrying about actual honest to goodness use cases.
The topic is a few special cases, like null, not every conceivable case. If things like null and undefined cause the majority of crashes, and they are simple to handle with a 'proper' language (and they are), then you already have gotten a lot of bang for your buck.
For rapid prototyping: if handling null cases is too much, use a function that unwraps nullable values directly, and if they are null, crashes (and no, this is not the same as a null pointer exception). Then go back and clean up the program when you are not prototyping quite as rapidly anymore.
So, for example, a place where null comes up. When you create a file. Programs that didn't check just threw up on an exception now. It is just as likely that without that option, the code would have just checked for a value, or thrown an error. So... what changed?
(For the record, I'm assuming an interface such as:
Where the FileHandle object can be null, and it being null is used to indicate that the call to fileOpen failed.)To be clear, this would involve the static type system for your language forcing certain things to happen, or not compiling. The sequence:
1. fileOpen returned a Maybe<FileObject>
2. You want to store the FileObject somewhere, and it expects an actual FileObject.
3. Maybe<FileObject> is not itself a FileObject, so you have to figure out: is it Nothing, or a FileObject?
4. You have to handle the Nothing case (language forces you to, even if you do nothing about it - you explicitly decided to do nothing).
5. If it is indeed a FileObject, you store it for later use, and you are guaranteed it is, indeed, a FileObject.
And I should note that I don't necessarily dislike the optional types and such. I just also don't have the major disdain for null that the internet seems to send through a massive echo chamber nightly.
That's a problem with Java (because it doesn't let you do that) and a similar problem exists with dynamic languages (because types aren't attributes of variables, so any variable could be null -- or any other value, null isn't particularly exceptional in dynamic languages for being a source of this kind of problem), but its not a problem "with languages that have null", as there are statically typed languages that have null but distinguish nullable and non-nullable types (C# and other static .NET languages are probably the most well-known examples, though there are others.)
> But really, at that point it's just sugar for an option type.
No, having a the free choice not use nullable types doesn't make the languages nullable types into sugar for option types. Nullable types don't compose the way option types do -- the issue in the article this thread is about isn't about whether or not you have the choice for nullable types, its about the fact they don't compose so you can distinguish a function returning a null as a substantive value because the thing you tried to store was also a nullable type and had a null value (Just None, in a world with Options instead of nullable types) or where the null is a result of the lookup failing (None, in a world with Options instead of nullable types.)
The deficiency of nullable types in C# compared to Options isn't the lack of choice of writing non-nullable types -- as you certainly have that choice for any data you need to represent -- its that nullabe types don't compose the way Options do.
Well, any value type -- but, yes, a custom type has to be a struct to not be nullable.
> How do you specify that a function can take a non-nullable string?
"string" in C# is the name of a particular reference (and hence, nullable) type. If you want a function that takes a non-nullable sequence of characters, that's quite possible, but the type of its argument won't be string. (The fact that there is no such type predefined and the entire .NET framework library is built around the assumption that if you want a sequence of characters you are going to use the built-in, reference -- and therefore nullable -- string type is an issue with the cost, in programmer-time, of doing this for any substantial use, but its not a "the language doesn't let you have a non-nullable type for any kind of data you want" issue.)
The disdain is probably (allow me to project) because it is a totally unnecessary problem which can be solved easily with virtually no drawbacks.
Granted, this can certainly be done with optional types. So, I'm not ultimately against them. I just think they are oversold quite a lot.
Laziness doesn't come into it, since you won't even manage to compile your code, let alone run it, if you try to use a "Maybe FileHandle" when a "FileHandle" is expected.
With that said, laziness does cause problems related to file handles, but those have nothing to do with unhandled failures. The problem is that files are a limited resource which we should release as soon as possible (just like memory, sockets, etc.). Lazy evaluation is at odds with this, since it essentially defers function calls to the last possible moment. One way of thinking about it is that laziness is "call-by-need", but we never really need the return values of functions like "closeFile".
Edited to add:
Note that this can be the case in Haskell when you're relying on exceptions for error handling. That's the opposite approach from option types, though.
So, depending on just how "lazy" the program truly is, sure you passed it something that unwraps a Maybe, but only once you actually go to use it. That make sense?
Consider this fun bit of nonsense scala
If I call this as "foo(None.get)", then it will print "Hello" before barfing. In fact, if I didn't evaluate x, it wouldn't even barf. I had thought the laziness of Haskel was that ++. (Meaning I thought if I stored x off to another also lazy reference, it wouldn't barf right away.)To make that a little more clear of what I meant, some even more amusing scala.
If I create something with "new Foo(None.get)", I can call happy on this object all day long. If I ever call sad, I get an exception.Apologies for the ninja edit above, but this scala seems to show it isn't just IO that is at danger here.
Again, if I do "new Foo(None.get)", I can call happy as much as I want, but "sad" will blow up.Now, I do not and could not claim that this sort of nonsense would have proliferated with the same abandon as null. I am claiming that this is not that unlike a null pointer exception, though.
http://research.microsoft.com/en-us/um/people/simonpj/papers...
This is also why the Control.Exception module exposes `evaluate`, which embeds a value in an IO computation and thus resolves this problem... If you remember to use it.
http://hackage.haskell.org/package/base-4.7.0.0/docs/Control...
Fortunately, in idiomatic Haskell exceptions are considered to be an expert feature (basically I see them in concurrent IO or resource management code only) and partial functions—usually introduced by incomplete pattern matching like `None.get` does—are veboten.
It's not perfect, but -Wall will help with that.
My point here was more that just because you have a file handle doesn't mean you have a valid place to write data. Making it optional would possibly prevent some bugs, true. However, it doesn't really help as soon as you have a filehandle to a full filesystem, for example. (Or, well, any other problem that usually happens to cause grief with the filesystems. You got a file, but by the time you went to use it the system was full and you couldn't, etc.)
Regardless, I should have been clearer on many points (and I fully accept I was flat out wrong on at least a few :) ). I do think null pointers are bad. I also know with proper static analysis tools, you don't need a whole new type system to make things better. Unless you are considering tools such as Coverity and friends some form of type system.
I would, for the record.
There was a thread some days ago (https://news.ycombinator.com/item?id=7916554) where I described some hackary I used to get particular guarantees out of C's type system. My response in https://news.ycombinator.com/item?id=7918423 seems tangentially relevant here. Type systems aren't (only/necessarily) features of a language.
I think the only other differences stem ultimately from standardization. Tooling could exploit any type system a program is known to follow. Likewise, compiler optimizations, &c, &c.
Well, I realize that is false. But the hoops all of the examples I have seen that types have to jump through to accomplish that aren't exactly pleasant.
So, maybe my preference is just that tooling can give you many of the benefits of more advanced type systems, without having to have all of the boilerplate of it everywhere in the code.
At any rate, I should say thank you very much for keeping this dialog going. Been a blast!
By "tooling" there, I wasn't referring to static analysis. I meant things like type-directed name resolution, type-directed completion, Agda's "holes", and so on. Strictly, these do not rely on the type system being "a part of" the language, but they do rely on some degree of standardization on a particular type system for the tool to work (or work well).
"Interleaved calls to different types, for example, can be found through tooling."
I actually don't understand exactly what you mean here. You might be surprised what even pretty rudimentary type systems can find if you leverage 'em right, though - as I mentioned in that other thread, I track which threads access what resources, and which functions live on which threads, with C's type system unadorned by additional checks (I do use additional static analysis, it just is entirely unnecessary to catch this one).
"Well, I realize that is false. But the hoops all of the examples I have seen that types have to jump through to accomplish that aren't exactly pleasant.
"So, maybe my preference is just that tooling can give you many of the benefits of more advanced type systems, without having to have all of the boilerplate of it everywhere in the code."
Ironically, I think my C code has more annotations for my static analysis tool-of-choice than the C types. Certainly there are a lot of poor type systems, and a lot of languages that expose particular type systems poorly. Boilerplate type annotations are mostly unnecessary in languages with type inference.
As I've said elsewhere, though, your tooling may be "more advanced type systems". Splint, for instance, can enforce linear typing - something that is not a part of the type system laid out in the C standard (or that enforced by typical C compilers, even with warnings).
"At any rate, I should say thank you very much for keeping this dialog going. Been a blast!"
Yes, it's turned out great :)
That is, at the point that a Maybe String statically changes to a String is not necessarily where the program will explode. A lot like a null assign. (Again, in a lazy language. Unless I'm mistaken on this subthread, the point was simply that you can still have that level of WTFery with Maybe that you can have with a null.)
Now, rereading the beginning, the reference was to a "Haskel like" type. So, I think I was definitely off the rails on this. I do feel I learned something going down this particular rabbit hole, though. Thanks!
There is a function, fromMaybe, which takes a Maybe x and gives you an x... but it also takes a default!
Often even more useful is the maybe function:
This takes a default output to use if you're given Nothing, a function to apply to the input and generate an output if you're given Just an input, and Maybe the input, and gives you the resulting output. As I said this is often more useful than "and use this default", because quite often the value you want to generate when it's Nothing is outside the image of the input under the function in question. For instance: There's no number I can pass to show to get "no number", maybe lets me very conveniently avoid trying to force one.Yes, they will. You could make the same statement about crime: "We'll never get rid of all crime, so why bother having police at all? Having police is just moving some goal posts." Does that make it clear enough how silly this sort of reasoning is?
This reads to me like eliminating null pointer bugs would just create a void that something else would fill? I don't see how that would work, at all. Is eliminating a null pointer exception going to introduce a divide by zero exception/bug? Is it going to introduce program logic bug? ...
If you're thinking about unwrapping a nullable value and throwing an exception if it really is null; I don't think of that as a conceptual null pointer exception, at least not in the usual sense. It's a deliberate decision along the lines of "I know [because of the control flow up to this point] that this value can not be null" or "if it is null, just throw an exception". It is NOT "this value can never be null... I think... well let's check the documentation, OK I'll assume that it isn't being misleading (documentation can easily 'lie' about a value being null in a nullable-by-default language, since non-nullable is not expressible in the language).
In a language with nullable-always, you can always pass a value (probably reference, but still) as null. In a language with explicitly nullable types, this is impossible. That's why unwrap functions throw an exception if the value is null: there is no other way to handle that case in a typed manner (well, I guess you also could loop forever...). In a nullable-by-default language? It is always fine to pass null. And it will not necessarily fail if a null is passed that shouldn't have been passed: it will only error when the reference is (tried to be) dereferenced.
That is, a type system is not the only tool we have in static analysis. So, I'm not completely convinced that moving to a type system is the best way to go.
Granted, it large part, it is clear it doesn't matter "what I think." Statically typed languages have a massive amount of support.
Specifically, majority of the static typing regimes seem to demand a change in how software is written. Whereas most of the static analysis seem to simply provide stronger guarantees on what you are already writing. That is a huge difference.
It's not the only tool but it's one of the better ones. It's faster, more reliable and more well-researched than almost any other tool.
What changed is that you are unable to propagate a null-that-shouldn't-be-null value to the rest of the program.
If you have a value V that might be null, and you are supposed to return that value itself (not wrapped in an Option or anything like that, ie a 'nullable type'), all you can really do is: actually return a value of type V, or throw an exception. That's it! (To throw an exception is also something that can be done at any time in most statically typed languages, for any reason whatsoever. Throwing exceptions is usually not a question of the type correctness of a program (maybe it is in the case of checked exceptions?).) You can not give null as an output: you are supposed to give V as the output, not a nullable V. So you know exactly where in your program you are doing something 'unsafely', i.e. where you are not rigorously handling every case of the value of some type. Though 'unsafe' is a bit misleading, since you may very well intend to just throw an exception in the (perhaps deliberately possible) case that a value is null.
The actual code that caused the overflow can be read here[^1] and the report on the overflow can be read here[^2].
[^1]: http://en.wikipedia.org/wiki/Cluster_%28spacecraft%29#Arithm...
[^2]: http://sunnyday.mit.edu/nasa-class/Ariane5-report.html
I can only guess that I got confused over reading about Agda at some point. Again, apologies.
Another way of seeing the same thing is: Type-theoretically speaking, MRV is using a product type (two values) to emulate a sum type (one value of two possible forms); why not just use a sum type?
Further, in idiomatic Erlang, you don’t bother with a branch for each case of the option type; you just do the following:
Effectively, that's a type assertion: if maps:find/1 responds with error instead of {ok, _}, that’s a process crash.Wait is he really trying to say, "Setting a VALUE to nil removes the entry from the dictionary!"? If so, that sounds terrible.
I can imagine barking up the wrong tree wondering why my map is missing fields when the real problem is an object I passed into it is unexpectedly nil.
This is especially important in Lua, because any value can be a key, you would have to be sure to delete the key otherwise it could not be garbage collected. local tab1 = {1,2,3,4,5} local tab2 = {} tab2[tab1] = true for k,v in pairs(tab2) do tab[k] = nil end
If this worked like JavaScript, then tab2[tab1] would return nil, but tab1 could not be garbage collected until tab2 is. Because JavaScript objects can only have strings for keys, this is less of an issue.