The problem is that you don't get guarantees that something is not null; you can still do Optional<T> myOptional = null. Second, you're not required to handle the not-present case. Third, if you attempt to get the value of an optional (using .get()) when it is not present, you get an (unchecked, so no requirement to catch) exception - basically replacing NullPointerException with NoSuchElementException.
The only value Optional has over regular values is that the developer has to assume the value can be missing - but the same was true for nullable properties.
At least one of the values of Optional is to communicate in the API that a value can be missing. The article mentions that:
A programmer looking at the code for the first time will know, just by looking at the return type, "Hey, this method may not return a record! I'll have to handle that scenario."
Making this clear to users of an API is something that nullable properties can not do. The user has to deliberately ignore the fact that the value might be missing by calling `.get()`.
This looks like a advantage to me, so to say that Option does not solve ANYTHING seems incorrect. It certainly does not solve EVERYTHING.
> A programmer looking at the code for the first time will know, just by looking at the return type, "Hey, this method may not return a record! I'll have to handle that scenario."
Well, that's basically the same thing with any nullable return then. Wether I return T or Optional<T>, there might be no record.
Even worse, when I return T, you only have 2 cases : I return something, or I return null.
With optional, you have 3 cases to check : I returned Something, I returned Optional.of(null), or I returned null.
You should never have to check that the Optional itself is null. While that is allowed by the language, it is such an obvious error that static analysis tools could easily detect it.
Then it shouldn't be allowed by the language, and that's the core issue. Until the compiler enforces that Optionals aren't null they're not fit for most purposes.
I guess you are either saying (A) to treat every return type as possibly null or (B) to use your judgment when doing null checks of return types.
In case of (A):
Do you want to write null-checks for methods like String.toUpperCase() or for user.getLastName()? That would lead to a lot of bloated, dead code. Also do you want to write error handling for all these cases you know can never happen?
In case of (B):
NullPointerExceptions are most likely when the user of an API did not expect he might get a null value. For instance user.getDepartment -> null. "What? An employee does not have to have a department?!" In this case you force users of an API to guess if they might reive a null return value. And they might guess wrong, which either leads to dead code or NPEs. Using Optional as a return type makes the difference explicit.
I would say that every return type that can be NULL should be checked for being NULL. In C++, this is possible, because you most functions are return-by-value or return-by-reference. This leaves only return-by-pointer that need to be explicitly checked. In Java, all functions that do not return primitives are return-by-pointer, and are therefore nullable. This, as you mentioned, makes it infeasible to apply null-checks on every function return.
A consistent API written with Optional should make it such that if an Optional is returned, then a null value could have been returned. If an Optional is not returned (including for references), then you can assume that null will not be returned. This is a contract that can be documented as part of the API documentation.
To me, this is much clearer than having to guess on every call and I would prefer this use of Optional or not in an API to an API without Optional and having to check everything.
That hasn't been a problem for me in practice. It's pretty clear that a reference to Optional should never be null. It can be statically or dynamically checked with @Nonnull or a precondition.
> but the same was true for nullable properties.
It's inconvenient to have to treat everything as possibly null. It's a lot nicer to create the convention, within one's codebase, that references are always nonnull and only Optional are nullable.
It has for me. Functions have multiple return points, some of them returning "null". Lead developer insists we use Optional<> to clarify that these functions can return null. People go over the function, replacing each instance of "return null" with "return Optional.empty()", but forget some instances. Now we have Optionals that are sometimes null.
But that's a bug in the library (or code you're calling into), not a bug in how you're using it. That's the main difference in my opinion.
Optional allows the code to say "I never intend to return null, only an Optional that may be nothing". In theory, a library/module/whatever that's been converted to use Optional should not have any calls that do return null.
In theory, its no different than a C API that can return a size or -1 for failure. Sure, it could return -2 because the language allows it, but you shouldn't need to check for that in your code that uses it.
If you are in the midst of an upgrade like this, then you might want to program defensively. In this case I would do that by applying annotations like FindBugs' @CheckForNull or @Nonnull or @Nullable. Across a trust boundary like an interface, it's possible to establish a convention like: every reference shall have one of those. You might be able to write a CheckStyle rule that verifies that every reference either has an annotation or is an Optional. Last but not least, assertions can be tremendously useful. Add Objects.requireNonNull() checks liberally, especially anywhere you pass a reference from one place to another without using it.
By applying these techniques you can quickly create a "demilitarized zone" for null, allowing you to use Optional confidently on the other side. Errors are caught quickly and locally by analysis and assertions. In a large codebase it's not an all at once upgrade, unless you refactor checked by static analysis, but you can be confident in the parts you've upgraded.
Right, but the difference in reality is that developers don't check every reference to see if it's null. The difference between a raw reference and optional is that with optional the author of the interface has explicitly signalled 'you really ought to check this'. Optional provides the ability to work with better conventions.
Sure, this doesn't live up to the actual guarantees provided by other languages, but that's not the same as being useless.
The fact that you can get a null instance of Optional is not Java specific - Scala has the same issue, for example. Saying that Optional does not sort any problem because there is no way to enforce it cannot be null is just wrong - it sorts plenty of problems, just not all of them (you didn't mention the case of an optional that contains null, which is unpleasant as well).
You can indeed call .get - so can you in Scala (.get) or Haskell (fromJust) with exactly the same semantics. What do you propose for the case where you know that there is a value in there but can't make the type system aware of that fact (you've called isPresent beforehand)? pattern matching and fail in the empty case? How is that different from calling get? Is it because of the fact that the exception is unchecked? In that case, why not throw a more specific, checked exception through orElseGet? And if you know that there is data to be had, isn't that exactly why we have unchecked exceptions in the first place?
You absolutely are required to handle the not-present case - you can't treat an Optional<T> an a T, and someone will need to decide what to do at some point. Either the caller (ie you return an Optional yourself after mapping into it) or you. There's a wide range of ways you can do that, one of which being "I want to fail if there is no data" - that's either .get if you don't need a checked exception or .orElseGet if you need something more constrained. You can also decide to use a default value, or o different code path entirely...
Optional has issues - it can contain reference to a mutable variable, for exemple, or you might find yourself with a null optional or an optional that contains null. It's not perfect. It's also a lot better than the alternative, null, as it lets you communicate some of your constraints to the compiler and allow it to catch certain kinds of errors at compile-time rather than at runtime. Rather like the rest of the Java type system, in fact - it's far from perfect, it doesn't catch all type errors at compile time, but it's (arguably) better than a fully dynamic one.
Definitely. For me, the main use case of Optional is to provide a way for programmers to express the semantic fact that "there is no such object" without resorting to null.
Doing this leads to checking if (obj == null) for dealing with the semantic case that the object is not present, but other kinds of failure may cause obj to be null, but should be treated differently (for being a lower-level issue, for example).
So, Optional<T> is basically a way to differentiate between different types of nullity.
The lack of pattern matching doesn't make it useless. The existence of Optional reminds the programmer to check whether the value is present, and the type system does enforce this; you can't accidentally treat an Optional as a reference of the same type. The type system does help us remember to handle things; it is a reminder enforced by the type system that's easy to examine during code review.
Yes, you can write get() and risk an exception. So don't do that. I'm still glad get() is present because there are times when the human knows the value will always be present, but sometimes in a way that can't be encoded in the type system. Alternatively, sometimes an absent value is an error and get() is an acceptable way to check-and-throw.
I agree with the suggestion that static analysis can be useful, though many users had that already with FindBugs or Fortify. If we're talking about annotations based analysis, then it only goes so far. Optional and streams provide useful new tools for documenting intentions from my perspective. Another reason I enjoy the type is because it makes it a little more reasonable to say that, at least in new code, all references are nonnull references by default, since they'd be Optional otherwise.
It's a curious feature, the way you've described it. I have virtually no experience with Java so I'm looking at this from a totally foreign perspective. The authors considers it an ineffective feature because you still have to think about null checks, whereas you consider it an effective feature because it forces you to think about the problem instead of ignoring it.
I believe the point @jontro is making is that unsafe programming behavior is unsafe without regard to the language's constraints. Option simply makes it more difficult to behave unsafely. Of course a developer can still call .get() when the presence of a value is unknown, but the Option type lets them know they are being careless in doing so.
No, the author considers it ineffective because you are not forced to think about nulls at compile time. Of course, at run time they will explode in your face, so you still have to consider them eventually.
@jontro is saying that proper Optionals make it impossible (or at least, more difficult) to forget about the problem at compile time.
Wouldn't it be better to have a non-nullable type, akin to C++'s return by value? That way, instead of forcing the programmer to think, you are removing the problem that they would need to think about.
In other languages that use Option types more natively, like Scala, anything that isn't optional is assumed to just not be null: The compiler doesn't guarantee it (it's not as if it can, we are dealing with a JVM here) but it works in practice. I've not seen a NullPointerException in years.
Option has other advantages, like its compositional qualities (thanks to Option being a monad). So imagine the following code
If the function returns None if it receives either None or a string longer than 10 characters, and if we got Some("short "), foo will return Some("short More data"). The alternatives without an option type, involve at least a couple of branching statements, even if you have a non nullable type, to convert from nullable to not nullable, while Option does it all for us.
> I've not seen a NullPointerException in years [in Scala]
This sounds odd. Does your code never interface with Java libraries and frameworks? I envy you! It's all too easy to forget to wrap a suspect value from Java-land with an Option(..). And cumbersome even if you do remember.
And then, there's the code written by Scala novices who just love using nulls. Yes, yes, "code reviews", we all know they are always practiced all the time :/
The Scala landscape is very, very quickly creating their own libraries for similar popular Java libraries/frameworks. It is entirely possible to create an application, web or otherwise, that uses no Java libraries at all (save for the sbt dependency tree).
As for "Scala novices who just love using nulls", if you're seeing that, then the coder in question missed day 1 of Scala training, which is always "if you are writing the word null, you are doing something wrong" (right up there with "if you are writing the word var, you are probably doing something wrong").
Maybe, but if true that's still in the future. The OP claimed they hadn't seen a NPE "in years", which simply doesn't match my own experience.
As for novices: yes, novices tend to do that. They tend to miss, misremember or fail to put into practice what they should have learned on day 1. I've also seen senior devs who were not exactly newbies (in general) still using nulls because they don't grok Scala; the famous "you can program Fortran in any language". The reality of software development is that you have to deal with all these people.
Whilst non-nullable references are useful, that isn't what Option is trying to solve. The problem with null references is that when you're handed a reference to an object of type T - the type system is saying to you: "This is the contract I will honour". Except it doesn't when the reference is null.
However the functionality of a null reference type is useful, because it allows a function to return 'no value'. So the goal of an Option type should be to expose the contract for T when there's a value and remove the contract when there's no-value.
The programmer is then explicitly forced to acknowledge the two states that the value could be.
Shameless plug - My C# implementation of Option which is possibly the most complete example out there (at least in C# world):
>Yes, you can write get() and risk an exception. So don't do that.
Yes, you can dereference a null pointer and risk an exception. So don't do that.
See where this is going?
The problem with Java's optional is that it is by all appearances (in name and high-level description), a general purpose optional, albeit with a ton of gotchas that will result in people saying "Just don't do that."
Java's optional is not a general purpose option type. In fact, its intended use-case is very specific, according to a rather obscure SO answer by Brian Goetz: http://stackoverflow.com/a/26328555/547365
This is, of course, just tribal knowledge, meaning people are going to be doing shit with optional that they shouldn't, and it's going to be messy.
>>Yes, you can write get() and risk an exception. So don't do that.
>Yes, you can dereference a null pointer and risk an exception. So don't do that.
Except these aren't at all the same. With a reference, you have no good way of telling whether your reference IS nullable. With Optional, it's ALWAYS optional.
Therefore it is trivial to a) train developer habits to always check before fetch, and b) enforce an isPresent check with static checks.
Sure pattern decomposition is nicer, but that doesn't mean that Java's implementation "defeats the purpose entirely" like the author suggests. I'd love everything to be checked statically, but if that can't be done, I still like strong conventions which make the code clearer and safer.
>Except these aren't at all the same. With a reference, you have no good way of telling whether your reference IS nullable.
As the article mentions, we have annotations to do just that.
>Therefore it is trivial to a) train developer habits to always check before fetch, and b) enforce an isPresent check with static checks.
Again, if we're using static analysis to verify the correct use of Optional, why are we using optional at all? Why wouldn't we use static checks to verify null references can't be dereferenced, instead of hypothetical yet-to-be-implemented static checks that may not be implemented anyway? See https://github.com/findbugsproject/findbugs/issues/56
>>Except these aren't at all the same. With a reference, you have no good way of telling whether your reference IS nullable.
>As the article mentions, we have annotations to do just that.
Your mileage may vary, but my experience with nullable/nonnull annotations is that because they are not used across the board, you are either left with a ton of false positives or a ton of false negatives. The fact that there hasn't really been a great standardization in the core language is probably evidence of that (yes, JSR305 exists, but the JDK doesn't enforce its constraints by default).
With Optional, it's a clean slate. You can simply say "a reference to an Optional shall never be null" and you could probably enable such an enforcement for an existing project. Of course it's not going to be 100% while it's still a normal reference type, but catching 99% of the cases is a really big improvement!
No, annotations do not accomplish the same thing in practice. Annotations function properly only when the code was written with annotations, and where the code and all callers are checked against the annotations. Unfortunately, annotations miss the mark in some practical cases:
1) A reference is @NonNull in my code, and I pass it to a component that accepts a regular reference as input. The annotation semantic is simply lost, and I may not even realize it as I'm writing that code. By comparison, if I need to pass an Optional<T> reference where a T reference is needed, then the type system reminds me to consider the present and absent cases. Optional does not allow the programmer to forget.
2) The fact that a component or library was written with annotations does not mean that all usage of the library will properly check the annotations. Sometimes people forget, or sometimes they interact with a component in a dynamic way like through Spring and the semantics get lost. An Optional makes this impossible to forget. You can confuse a @Nullable T with a T, or forget to check upon conversion, but there is no failure case that causes an Optional<T> to turn into a T without handling the present and absent cases.
More choice is better than less choice. You could design an Optional type to omit the get() method. That's less useful than what we have now. Like goto, it has rare but legitimate uses, and it's easy to avoid for the "right way" in the average case (don't use it, flag it during code review, and document the reason why when it's being used).
> With a reference, you have no good way of telling whether your reference IS nullable.
You do, a reference is ALWAYS nullable/optional. Just because people might expect a certain reference never to be null does not change that. You could equally as well as semantically expect an Optional never to be "optional" (it is just a matter of perspective). People who are too lazy to check for null might be too lazy to check isPresent() and both will give you, unsurprisingly, an exception.
java.util.Optional is really just yet another layer added to support laziness in software developers. To put it bluntly, write better manuals/JavaDocs if necessary and, most importantly, read them.
If you ever have actual success getting developers to obey documented input/output constraints without actually enforcing them with a static or runtime check, you could probably become a highly-paid coach. But I'm skeptical. I've never seen it happen, and most good libraries I've used (e.g. Guava) always follow up preconditions with a runtime check. Statically enforceable pre/postconditions shouldn't even have to be documented because the signature is the documentation.
I think the main use of Optional is in a codebase which doesn't _ever_ allow "null".
If every method call either returns an actual object then you can get rid of null checks, and not bother doing any checks on method returns that don't return an Optional.
some have already commented here that author maybe complains too much (e.g., calling .get() on an Option<T> is a deliberate move, meaning that the developer explicitly took on the risk of that exception. that's very different than implicitly assuming that a regular ref is not null, using it and getting bitten in the ass.). but this one really hurts:
"The last flaw with Java's implementation is a bit ironic — It's possible for the optional reference itself to be null."
This is another off-base point, in my opinion. Sure, it's possible for it to be null, and that isn't great - but the vast majority of null problems come from the user assuming the interface won't give you a null. Something returning an Optional should never return null, so the ambiguity is resolved. Sure, it still could as a bug, but that's actually a pretty rare case. I don't see it being a real problem.
that makes sense. i think an important question here is - what does "return null" in a method which returns Optional<T> do? will it return a null Optional<T>, or will it return a non-null Optional<T> which is empty?
sorry to rub it in, but Java needs value types. this whole issue, however maybe irrelevant, does not exist in C#.
The point is that wherever that happens, it's an obvious flaw with the method returning the null, and can also be caught by static analysis. It means that in general, the problem is avoided.
Yes, it's not ideal, it's still a flaw that should be caught at compile-time, but it's not a big real-world problem, as returning null by accident instead of an option is a pretty rare bug.
The much more common bug (you didn't realize no return value was a case), the Option is a good counter to.
No, it's not a perfect solution, but it's not useless either.
i understand, and agree. i would personally, though, avoid adding it, and add attributes for the same purpose instead. just as the article suggests... it achieves the same without introducing a new class, thus also remaining backwards compatible.
Exactly. A null Optional reference is such an obvious bug, that it can easily be detected by simple analysis tools. OTOH, a null is often an accepted (if derided) method of indicating that a value simply doesn't exist, so it's much more difficult to detect when a null will cause a problem.
That the reference can be null is hardly the fault of the implementation of Option in Java,unless people expect individual classes to modify the type system.
I feel get is acceptable style if you actually want to fail when there is no data to be had.
If you don't have a default value to provide, nor an alternate code path other than "sod this, I'm bailing", get is ok. Not quite as good as using a more specific exception through orElseGet, but not bad, exactly.
Also, if you have just called isPresent (and assume your data to be immutable), then get is ok - the alternative being to manually throw something like new IllegalStateException("the impossible has happened!"), which is not that much more useful, and is a pain to have to write all the time when you know it will not get called.
>Using get() is just bad style and so is returning null where you could return Collections.emptyList().
I agree that you should never return null, but if get() is bad style I think you should send out that memo because I haven't yet seen codebases that agreed.
I've included nice as it had a different syntax for optionals (?Type instead of Type?) which I found interesting. I was interested in Nice back in the day.
Welcome to Scala version 2.11.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45).
scala> var h:Option[String] = "Hello".some
<console>:7: error: value some is not a member of String
var h:Option[String] = "Hello".some
Sorry, as the other comment mentioned this is Scalaz as it makes code IMHO a little more readable. The example was more about the h:Option[String] part.
I don't understand what's wrong with get() method. Swift has built-in null-safety and it has "!" operator. Kotlin has it. Even Haskell has fromJust function which does exactly the same.
May be name "get()" is too short and innocent-looking.
Partial functions such as fromJust are generally frowned upon in the Haskell community. There was a relatively recent proposal to deprecate and then remove fromJust:
Is he? The point is that in Java you are still able to treat x unsafely, while languages with stronger typing do not. E.g. in Haskell, if a function returns a Maybe a, it will always be a Just a or Nothing value. Moreover, such languages allow you to make non-exhaustive matching against all constructors a compiler error.
tl;dr: Haskell, Rust, et al. put the burden on the compiler. Java puts the burden to ensure safety on the programmer. (As can be witnessed in your snippet.)
As of Java 8, Java has pluggable type systems[1], some of them are more advanced than what Haskell provides (like true intersection unit types), some effect types (locks etc.) and more. So Java puts the burden on some very advanced -- but optional -- type systems.
That's still more than Haskell and Rust developers in the field... ;) In any case, as Checker matures and becomes more accessible, it will be marketed more and I expect higher adoption.
>tl;dr: Haskell, Rust, et al. put the burden on the compiler. Java puts the burden to ensure safety on the programmer. (As can be witnessed in your snippet.)
Actually, (much to my disappointment as I'm just learning rust) you can just take an Option or Result and .unwrap() and the compiler won't complain at you for not checking it. For such a "safe" strongly-typed language, I'm surprised that so much new rust code does this when:
* the developer believes the Option/Result can never be None/Err (oh come on, just wait until you refactor your code a bit one day and miss a spot)
* the code is example code (I've had little luck finding "good" examples sprinkled about github)
* the developer is lazy (see above)
Allowing unsafe unwraps defeats a core purpose of rust, and the unwrap method shouldn't even exist in the API.
Allowing unsafe unwraps defeats a core purpose of rust,
Definitely, but you can also still do this in Haskell (fromJust). But it's better than nullable types since you explicitly have call an unsafe method. (Assuming that you have set non-exhaustive pattern matching to be a warning/error.)
I completely agree. unwrap() is a huge mistake in Rust. There are brilliant ideas implemented in Rust (primarily the borrow checker) but their advantages seem to be wiped out by the kludge of unwrap.
Care to explain why? Whatever unwrap provides can be done with a match+panic. Which is more explicit, but that's just splitting hairs -- unwrap is pretty explicit anyway.
I understand the anger but at the same time I don't.
Overall unwrap() isn't good. Really it should be replaced by TODO WRITE ERROR HANDLING in 90% of its cases. That being said there are still good uses for it.
The main _safe_ use for unwrap() I find is when dealing with an iterator of Results/Options (which seems to happen a lot). The pattern:
.filter( |x| x.is_some() )
.map( |x| x.unwrap() )
Is safe, and I really don't know a more eloquent method of handling these operations.
In Swift you can solve this with a filterNone() method with an explicit type signature of [T?] -> [T], I assume that could also be implemented in Rust?
> Really it should be replaced by TODO WRITE ERROR HANDLING in 90% of its cases.
That's what the Rust community reads it as, mostly.
I've seen it occur very infrequently in libraries (aside from example/benchmark/test/mocking code, of course). When it does its usually for really out of whack errors like poisoned mutexes, crashed threads, etc (mind you, these are for unwrapping error values, not options, but it's almost the same thing). Or for cases where it's known to be valid due to invariants that can't be expressed in the normal way.
I did a quick grep of hyper, and the unwraps are almost all in "dev" code (tests/benchmarks/etc). The cases where they aren't are where there was an `is_some()` earlier (and due to some reasons it couldn't be restructured as a match, perhaps just because of rightward drift), or the "invariants" thing -- i.e. it often calls `<some url>.serialize_path().unwrap()`, which is OK, because hyper only ever deals with relative schemes (e.g. `http://` and `https://`, not `data:` or `mailto:` -- the latter two don't have paths), which is checked in the constructors, so this should NEVER panic. I didn't go through all the code, but in the main HTTP and HTTP2 code serialize_path was the only offender, which was one that was justified.
I see it more often in applications. In the following cases:
- Example applications in blog posts (because you want to explain something else, not spend too much time on error handling)
- Slightly less out of whack errors which are weird enough that they should close the app
- Normal errors which should close the app (e.g. "insufficient permissions" for a command line util, or a port-opening error on a network app). unwrap() prints out the error message so this is a very rudimentary form of error handling anyway.
- Laziness, or often with a TODO. Not too common.
So, it's mostly used in justifiable ways, and that's why it exists in the stdlib. It doesn't "wipe out" anything that the borrow checker provides (borrowchk and exhaustive match are two different things), and while it's a kludge if used often there are legitimate reasons to use it; enough reasons to have it in the stdlib.
> I really don't know a more eloquent method of handling these operations.
.filter_map(|x| foo(x))
:)
(but yes, in general there are situations in which you know an Option is unwrappable, e.g. the serialize example above)
Note that anytime you see a closure in any language that just takes a single argument, calls a single function with that argument, and returns the value of that function, then you can eliminate the closure entirely by just passing the function instead.
So `filter_map(|x| foo(x))` can be written more simply as just `filter_map(foo)`.
The number of times I've had this happen has just lead to me giving up on simply just passing the lambda as a variable instead of wrapping it in another.
If we didn't provide it, someone would write it. (Actually, I suspect lots of libraries would just have it.) You can't force people to write good code.
This presupposes that the pattern that "unwrap" wraps is always a bad thing anyway, which it isn't. Sometimes the right thing to do is to panic your program if None is present. And sometimes the type system isn't sophisticated enough to express an invariant, and you need to use "unwrap" to work around it.
By the way, the borrow checker has nothing to do with unwrap().
This is a dramatic overreaction. `unwrap` is a convenience method whose trivially-obvious implementation is four lines long. Were it not provided by the stdlib then we'd have a preponderance of bespoke implementations in the ecosystem all with different names, which would make this pattern much harder to distinguish and discourage.
(Not to mention that `unwrap` in no way subverts memory safety, which makes mentioning it in the same breath as the borrow checker puzzling.)
Just to clarify since `unsafe` is a special term in Rust: `unwrap` still checks for None and panics, it doesn't blindly assume it's valid.
That said, I think you're too dismissive of "can never fail" assumptions. There's tons of places where something is optional in general, but based on various invariants you know it won't be. For instance, if you successfully acquire the first element in an array, you know you can get the last.
If you legitimately believe that Option is always Some, I don't understand what you think you would otherwise put there? Gonna have your function return an Err if an internal invariant is broken? Gonna have callers actually check for that?
>Gonna have your function return an Err if an internal invariant is broken?
Absolutely you should return an error. Whether the caller wants to panic or handle it or print unicorns should be left up to the caller, not your function. Functions should not be expected to tear down the thread in case of an error. Nothing that panics should belong anywhere in exported code
We specifically recommend against providing panicing and non-panicking. This causes horrible combinatoric explosions. There's very few exceptions to this (array indexing being the major exception). We do of course recommend non-panicking, but there's a few cases where the burden of checking for errors is too high (indexing, refcell).
The problem is that a function cannot know whether or not the caller can recover from a particular error, so there's no point in making that distinction in the first place.
It's part of your API design, if they are recoverable or not. You should lean towards recoverable: if the caller can't recover, they can always do something with it. But some kinds of problems are non-recoverable.
Say you have a library which has some kind of internal state, and the consistency of said internal state is important for memory safety, as other operations rely on it.
Using assert! to verify that your state is consistent is useful, in case you have a bug. But since it's all internal, it's not something that the caller can really recover from, either. It's not their fault, it's yours.
Well here's my issue with this explanation that I've heard a lot: the caller may or may not be able to recover from it, but the caller may still be doing something with the error. Maybe the caller doesn't want to expose the error to its caller, or maybe the caller wants to build a new message to explain the cause of your error. But to panic! takes away a lot of options for the caller.
Worst of all, it leads to untestable code, because the calling test case can't properly check the error against some known result.
This seems a bit too unpragmatic. Would you also require the user to explicitly handle:
* Index out of bounds on every array op
* Integer overflow on every arithmetic op
* OOM on every allocating op
Maybe if you're writing an ironclad RTOS for a critical system without any good redundancy? Otherwise these are such pervasive operations that most have accepted that they're not worth handling everywhere they happen. Requiring that really dilutes the value/meaning of errors.
Given this signature:
fn do_thing() -> Result<T, E>
There's a clear signal that there's legitimate error conditions that you probably want to think about today. If every function that accessed arrays, worked with integers, or allocated memory returned a Result, it would border on meaningless. It would be like if there was no Throwable/Exception distinction in Java. Everything would `throw Exception`, eliminating the value of even noting that something can fail.
I think the only way such a system could be tolerable is if the language in question had really good dependent type support (but that wouldn't handle the OOM issue -- which you can't even reliably handle on some systems).
Unwinding/crashing is valuable in a truly robust system, because it needs to handle crashes anyway. Might as well punt obscure problems to that level of the reliability system. (This is basically the basis of Erlang's task system, AFAIK)
>Requiring that really dilutes the value/meaning of errors.
No. You have already diluted the meaning of errors, and you want them elevated to _your_ standard.
>Index out of bounds on every array op
These are removed if you build a rust program with --release.
>Integer overflow on every arithmetic op
Add the Wrapping class if you expect overflow. Overflow _shouldnt_ normally happen on an Integer operation. It is a hardware error when it happens, and can cause massive pain-in-ass bugs when it happens unexpectedly.
I'd rather get errors when it does happen, rather then find out 6 months into a production run.
Rust doesn't remove bounds checks in --release. It's wrapping that gets turned on in release. I'm not sure why you're distinguishing overflow as truly exceptional, as opposed to any other "this should never happen" error?
Also I don't think many C libraries that allocate expose that as a failure condition (I've certainly seen some which don't even check!)
> you can just take an Option or Result and .unwrap() and the compiler won't complain at you for not checking it.
Explicit vs implicit. In Rust/Haskell you are forced to do _something_ about the nullability. If you use unwrap/expect/fromJust you are explicitly acknowledging that you want it to panic if it fails.
On the other hand, in Java, you can get an NPE where you didn't expect it because nulls move around easily and there's nothing in the type system to prevent that. That's the difference.
> I'm surprised that so much new rust code does this
Also, in my experience, unwrap() in Rust is mostly confined to irrecoverable errors in applications, and example blog posts (where the focus is on getting something done and not worrying about good error handling).
> defeats a core purpose of rust
What purpose? Whatever unwrap provides is still possible via a match+panic. Which is _more_ explicit, but they're both still explicit.
> If you use unwrap/expect/fromJust you are explicitly acknowledging that you want it to panic if it fails.
If you use Optional.get() without Optional.isPresent(), how is that any different? It's not as nice as pattern decomposition, but it's still fundamentally the same, and on top of that, transform functions are provided so that most of the time you can do null-safe operations.
It can be, and it's annoying, but that's still a step forward. It's trivial for a good static checker to treat all Optional references as if they were annotated with your favorite @Nonnull annotation.
>Actually, (much to my disappointment as I'm just learning rust) you can just take an Option or Result and .unwrap() and the compiler won't complain at you
unwrap() is a convenience method that still uses exhaustive pattern matching and will panic if no value is present. The developer is making a conscious decision to panic in a not-present scenario, and on a case-by-case basis. The compiler is still doing it's job. This is different than java, where a get() on a non-present value is a runtime violation.
The distinction is there but the runtime behavior is not remarkably different. This feels more like a philosophical argument with no real purpose, to be honest.
If you don't think it's an important distinction then your real beef is that the developer can choose to panic in certain scenarios, which of course is silly when put in context.
The point of the new Optional type in Java isn't to prevent NPEs, it's to make using the new Streams API cleaner. Since you can now flow through optional values in a stream you can ignore whether the stream contains optionals or not and only deal with them at the end. It's certainly possible that in a future version of Java the optional type might be paired with something like pattern matching to prevent NPEs with some syntax sugar, but that's not the reason that the Optional type exists today.
Not entirely true from what little I know about Haskell. Haskell might put MORE of the burden on the compiler but it doesn't put ALL of it there. The following generates a runtime exception. (not a compile time type error)
Close, but not quite there. You should leap over tall buildings to make sure that you never have to set y to null.
In my code, I'm only dealing with null if a API gives me one. At that point, I take some sort of action to make absolutely sure I'm not passing it along to anyone else. This means using exceptions, the null object pattern, or Optional without a need to dereference. The goal is to prevent a proliferation of references that could be null and therefore have to be checked.
I agree. Having programmed in java for the past 15 years, I can attest that NullPointerExceptions are a constant source of errors. The main benefit I've experienced with Optional is that it documents the fact that a method might return a null value. Without this, the client has to guess whether or not to add defensive null checks.
It does provide some type-level information, but if you're already writing documentation, the benefits are marginal. Add to that the fact that it really pollutes your interfaces, I would argue that it is almost always better to just pass a null and force people to check it, or read the documented guarantees.
Basically, just assume every Java type is wrapped in Optional implicitly. But by adding Optional (and using it in standard libraries) it's often a matter dealing with code that interfaces between the two styles -- another problem that didn't exist without Optional.
And even if you think Optional is worth the small gains, it's still a sad mimic of the Option types implemented in other languages.
Adding my voice to the din of people noting how far afield of the point the author is:
You should almost never call .get(), except in cases where the code path does not allow an empty optional.
Even so, calling .get() on an empty optional is better than handing nulls around. A null may -- by chance, really -- make it several lines down the code, so that the stack trace points you much later in the code than where the null originated. Calling .get() on an empty optional immediately throws a NoSuchElementException, pointing you directly to the line where the optional was empty where it shouldn't have been.
Consider this code:
User user = getUser();
// other stuff here
// ...
// ...
String userSummary = formatUserSummary(user);
// ...
// ...
An NPE occurs inside of formatUserSummary. Where did the null come from? Is the user's Franchise null? Maybe the Franchise's Location? Or is it the User that's null?
If you instead took an Optional<User>, even if you just immediately unwrapped it with .get(), you'd at least get an error that unambiguously told you that it was the user that was missing.
A bit offtopic but SAP JVM puts that info in NPE error message[1], I don't know why others implementations don't, maybe it is incurring too large overhead?
[1] it is something like "Attempted to call getLocation() on Franchise object returned from getFranchise() but it was null"
Nulls can be particularly pernicious in Java because of auto-unboxing.
E.g., imagine you got a NPE from this:
String summary = getUserSummary( user.getId() );
...you'd think "user" must be null, right? But it's quite possible that user.id is a Long, the current value is null, and getUserSummary() requires a long primitive argument.
This compiles just fine, and at runtime the Long object will be auto-unboxed to a primitive long... unless it's null.
Surprise!
But things like Optional do help; doing a code review for the things that may be null will fix these kinds of mismatches (and you'll avoid relying on auto-unboxing). The above code is broken -- either user.getId() can never be null (in which case it should be a primitive already) or it might sometime be null, in which case this code must not allow auto-unboxing.
I had a fun one with this once. It looked something like this, using your `user.getId` method:
int userId = p ? 0 : user.getId();
We spent a few hours staring at this trying to figure out why there was a `NullPointerException` on a line dealing with `int` values. Eventually realized that `thing` was really an `Integer`, causing the entire trinary to be an `Integer`, which was then being unboxed to an `int`.
Brian Goetz's response: "They didn't solve the problem I thought they should have solved, so their solution is worthless." Thank you, internetz.[0]
In any case, that's not what Java's optional is there for[1]. Solving the safety problem is the job of Java 8's pluggable type systems[2] that are both more powerful than the static-analysis tools listed in the article and had Oracle's official support (in the form of JSR-308).
But is it really? Maybe I'm missing the point but it is a nice way to express that a function may or may not calculate something, no? At least it seems way better than returning bare pointers or pasing an argument by reference+returning bool, or returning a unique_ptr, or ...
If you come from ML or Haskell and start using boost or Java's option, you actually feel ripped off.
Some people claim it's a sort of documentation denoting 'intent', but I don't want documentation. I don't want options to infect the whole codebase, I just want to pattern match once and extract the value, and never have to check again for None or null.
The problem in Java is the type system has references.
X static use_it(X x){ ....}
really has a signature something like this:
use_it : X ref -> X ref
std::string always is a value, boost::optional<std::string> isn't (similar to std::string, just a lot safer).
The problem is that Java only has boost::optional<std::string>, which can be null, empty or a value.
Implementing a monad like structure without a monadic for loop syntax sugar tends to confuse people. It's hard to picture writing most operations as map or flatmap.
We've been using Optional<T> a lot in recent years (via Guava). It's useful, and has certainly reduced the number of NPE we've had slip through as it forces the developer to consider nullability up front. Looking forward to migrating to the Java 8 version when we upgrade to that JDK.
One addition that would be useful would be some syntactical sugar to reduce the verbosity of using Optional<T>. But otherwise I'm happy and if you combine with @Nonnull and @Nullable annotations you get the best of both worlds.
I don't program much Java, but how expensive are exceptions? Wouldn't it make more sense to expect functions that could return null to instead throw a checked exception if something goes wrong?
eg
public dbResult doDatabaseThing(stuff) throws DatabaseException
So if you do
result = doDatabaseThing(stuff)
the compiler will require you to do something about the DatabaseException.
It seems to me that the point of, for example, Option and Result in Rust is to avoid the whole checked exception mess while still passing useful information about things that go wrong. But, given the problems pointed out in this article (lack of pattern matching and algebraic data types mostly), checked exceptions might be a more correct way to do it in Java.
Yes. In this case the No-result means that the database query was successful, but no record was found. A connection error etc. should almost certainly still be thrown as an exception in Java.
Without exceptions you would have to return a value that could discriminate between all 3 possible outcomes: nothing, error, result. In Rust this is the `Result<T, E>` struct.
It that case, wouldn't it be better to return something other that `null` for no-result (an empty iterator? an empty list?)? That might be where Optional comes in, but I'm still not sure why a new language feature is needed for that.
A list has zero or more elements where as Optional is zero or one. The later case seems common enough in my experience to warrant an additional language feature
Generally I would think optional values are not indicating errors but to indicate an absence of something. A null database result would be the item was not found rather than an issue communicating with the database. Moving that to try / catch error would increase the possibility that another unexpected operation is inadvertantly caught. Somewhat similar examples can be found for Result but I do conced you have a better case.
Also, despite it not being "standard" @NotNull, and @Nullable have worked really well for me so far. And I don't think Optional was meant to replace them.
I've only used the Guava version of Optional, which appears to have a slightly different API than Java 8's. (You can tell there's a Java 8 committee, that's for sure.)
The biggest problem I found in applying Optional is that Optional is very much a Haskell-y concept to me, and Java programmers and Haskell programmers don't really overlap much. Therefore, to me, Optional is just a functor like a list or a set. So doing Optional.asSet() and iterating over the result is, to me, the correct way way of handling an optional value. If there's something in there, the loop runs once. If there's not anything in there, the loop runs zero times. This is exactly like "fmap" in Haskell:
Prelude Data.Functor> fmap (+1) $ Just 42
Just 43
Prelude Data.Functor> fmap (+1) $ Nothing
Nothing
(Except that Haskell automatically preserves the type safety, whereas you have to re-wrap the value if you're using Java. Functional programming in Java never works, and this where the Optional idea starts to go wrong.)
But I got "I don't really understand this" in code reviews every time I did that; the Java way is to check if it's set, then do a type-unsafe unboxing. Thus, like the article said, completely defeating the purpose of the Optional type.
That said, it's pretty rare to need exactly 0 or 1 of something. If you're doing a database query, you can probably make the API return a set of rows. Then it's natural to loop over the results and nobody raises their eyebrows.
> That said, it's pretty rare to need exactly 0 or 1 of something.
That's interesting and sort of hard to agree with. I think uniqueness is very often desirable and very often paired with a lack of certainty about existence.
"Does a user with this identity exist?" certain is handled properly as a 0-or-1 question, e.g..
It's pretty common to need to do something to your user object if it does exist. As a toy example, it's far better to have something like (using Scala syntax here):
def getUserGreeting(userDao: UserDAO)(username: String): String =
userDao.getOptionalUser(username) match {
case Some(user) => "Hello, " + user.getFullName
case None => "Not logged in"
}
For one thing, the first function probably only makes one database call, while the second (absent caching) makes two. Secondly, the second carries with it the (unlikely) possibility that the user is deleted between your DAO calls. Thirdly, I would argue that the first function is easier to read.
> Except that Haskell automatically preserves the type safety, whereas you have to re-wrap the value if you're using Java.
Optional has map() in Java.
Optional<Integer> result = Optional.empty().map(i -> i * i);
assert(!result.isPresent());
Optional<Integer> result = Optional.of(9).map(i -> i * i);
assert(result.get() == 81);
It always bothered me that Java8 and Guava optionals don't just implement Iterable. There must be a good reason for this, and I'm guessing they're worried about confusion when you have an `Iterable<List<T>>` but still, it'd be really nice to be able to just write
for (T result : potentialResult ) {
//...
}
I agree with you, that just seems so much more idiomatic than
if (potentialResult.isPresent()) {
T result = potentialResult.get();
}
But maybe it comes from my mental model of Optional being a 0- or 1-length collection. It's at least definitely treated as a functor in that for pure functional operations, you have .map (or .transform for Guava).
All of these problems seem trivial to me. The problem with Optional is that it isn't baked into the API, nor can it ever be, so any use of Optional is after the fact, applied only by the application, which means you need to wrap api calls in Optionals manually, which is something that gets real old, real quick.
We tried to use Optional<> instead of returning null. Accidentally returning null from methods returning Optional<> is something that actually happens, especially when migrating existing code. Of course you can solve that with @Nonnull annotations, but then why use Optional in the first place?
The other problem is that the Optional<> interface is clunky. There is ifPresent(), but no ifAbsent(). The stream syntax is incompatible with the rest of the Java world, in the same way that List<T>#toArray has always been ridiculous. And if you use ifPresent(), you have to live with the fact that you cannot return from Java lambdas, and you cannot change any variables or even use non-final variables. I want to scratch my eyes out when I see people allocate one-element arrays on the heap just to work around Java's lambdas.
I don't see the value of Optional<T> if you already have @Nullable.
I really wished C# would support Nullable for reference types too. This + the new traversal operator (?. ) would be a great combo to get rid of NullReferenceExceptions.
To me Optional<T> without pattern matching seems crap.
I think in this thread when we talk about "pattern matching" (which _is_ awesome) were caught up with two notions: pattern based destructuring of input and type enforced exhaustiveness checks. Java 8's Optional does give the latter:
> maybeT.map(t -> ...).orElse(...)
The author does make a good point that 'get' is unsafe. I think it was a necessary concession to team members who are learning how to program in this manner.
196 comments
[ 3.9 ms ] story [ 248 ms ] threadThe author apparently concludes that since it does not solve ALL error cases it does not solve ANY error cases...
The only value Optional has over regular values is that the developer has to assume the value can be missing - but the same was true for nullable properties.
A programmer looking at the code for the first time will know, just by looking at the return type, "Hey, this method may not return a record! I'll have to handle that scenario."
Making this clear to users of an API is something that nullable properties can not do. The user has to deliberately ignore the fact that the value might be missing by calling `.get()`.
This looks like a advantage to me, so to say that Option does not solve ANYTHING seems incorrect. It certainly does not solve EVERYTHING.
Well, that's basically the same thing with any nullable return then. Wether I return T or Optional<T>, there might be no record.
Even worse, when I return T, you only have 2 cases : I return something, or I return null.
With optional, you have 3 cases to check : I returned Something, I returned Optional.of(null), or I returned null.
I guess you are either saying (A) to treat every return type as possibly null or (B) to use your judgment when doing null checks of return types.
In case of (A): Do you want to write null-checks for methods like String.toUpperCase() or for user.getLastName()? That would lead to a lot of bloated, dead code. Also do you want to write error handling for all these cases you know can never happen?
In case of (B): NullPointerExceptions are most likely when the user of an API did not expect he might get a null value. For instance user.getDepartment -> null. "What? An employee does not have to have a department?!" In this case you force users of an API to guess if they might reive a null return value. And they might guess wrong, which either leads to dead code or NPEs. Using Optional as a return type makes the difference explicit.
To me, this is much clearer than having to guess on every call and I would prefer this use of Optional or not in an API to an API without Optional and having to check everything.
That hasn't been a problem for me in practice. It's pretty clear that a reference to Optional should never be null. It can be statically or dynamically checked with @Nonnull or a precondition.
> but the same was true for nullable properties.
It's inconvenient to have to treat everything as possibly null. It's a lot nicer to create the convention, within one's codebase, that references are always nonnull and only Optional are nullable.
It has for me. Functions have multiple return points, some of them returning "null". Lead developer insists we use Optional<> to clarify that these functions can return null. People go over the function, replacing each instance of "return null" with "return Optional.empty()", but forget some instances. Now we have Optionals that are sometimes null.
Optional allows the code to say "I never intend to return null, only an Optional that may be nothing". In theory, a library/module/whatever that's been converted to use Optional should not have any calls that do return null.
In theory, its no different than a C API that can return a size or -1 for failure. Sure, it could return -2 because the language allows it, but you shouldn't need to check for that in your code that uses it.
By applying these techniques you can quickly create a "demilitarized zone" for null, allowing you to use Optional confidently on the other side. Errors are caught quickly and locally by analysis and assertions. In a large codebase it's not an all at once upgrade, unless you refactor checked by static analysis, but you can be confident in the parts you've upgraded.
Sure, this doesn't live up to the actual guarantees provided by other languages, but that's not the same as being useless.
Optional is planned to become a value type.
The fact that you can get a null instance of Optional is not Java specific - Scala has the same issue, for example. Saying that Optional does not sort any problem because there is no way to enforce it cannot be null is just wrong - it sorts plenty of problems, just not all of them (you didn't mention the case of an optional that contains null, which is unpleasant as well).
You can indeed call .get - so can you in Scala (.get) or Haskell (fromJust) with exactly the same semantics. What do you propose for the case where you know that there is a value in there but can't make the type system aware of that fact (you've called isPresent beforehand)? pattern matching and fail in the empty case? How is that different from calling get? Is it because of the fact that the exception is unchecked? In that case, why not throw a more specific, checked exception through orElseGet? And if you know that there is data to be had, isn't that exactly why we have unchecked exceptions in the first place?
You absolutely are required to handle the not-present case - you can't treat an Optional<T> an a T, and someone will need to decide what to do at some point. Either the caller (ie you return an Optional yourself after mapping into it) or you. There's a wide range of ways you can do that, one of which being "I want to fail if there is no data" - that's either .get if you don't need a checked exception or .orElseGet if you need something more constrained. You can also decide to use a default value, or o different code path entirely...
Optional has issues - it can contain reference to a mutable variable, for exemple, or you might find yourself with a null optional or an optional that contains null. It's not perfect. It's also a lot better than the alternative, null, as it lets you communicate some of your constraints to the compiler and allow it to catch certain kinds of errors at compile-time rather than at runtime. Rather like the rest of the Java type system, in fact - it's far from perfect, it doesn't catch all type errors at compile time, but it's (arguably) better than a fully dynamic one.
That's not possible in Java. `Optional.ofNullable(null)` returns `Optional.empty()`, and `Optional.of(null)` throws a NullPointerException.
Doing this leads to checking if (obj == null) for dealing with the semantic case that the object is not present, but other kinds of failure may cause obj to be null, but should be treated differently (for being a lower-level issue, for example).
So, Optional<T> is basically a way to differentiate between different types of nullity.
In my (admittedly limited) experience, people omit them, forget to update them, and then rely on the analyser to magically catch everything.
Sure, library solutions like Optional can always be misused, but there's no silver bullet to negligence.
Yes, you can write get() and risk an exception. So don't do that. I'm still glad get() is present because there are times when the human knows the value will always be present, but sometimes in a way that can't be encoded in the type system. Alternatively, sometimes an absent value is an error and get() is an acceptable way to check-and-throw.
I agree with the suggestion that static analysis can be useful, though many users had that already with FindBugs or Fortify. If we're talking about annotations based analysis, then it only goes so far. Optional and streams provide useful new tools for documenting intentions from my perspective. Another reason I enjoy the type is because it makes it a little more reasonable to say that, at least in new code, all references are nonnull references by default, since they'd be Optional otherwise.
It's a curious feature, the way you've described it. I have virtually no experience with Java so I'm looking at this from a totally foreign perspective. The authors considers it an ineffective feature because you still have to think about null checks, whereas you consider it an effective feature because it forces you to think about the problem instead of ignoring it.
@jontro is saying that proper Optionals make it impossible (or at least, more difficult) to forget about the problem at compile time.
Option has other advantages, like its compositional qualities (thanks to Option being a monad). So imagine the following code
def foo(x:Option[String]) = x.filter(_.length<10).map(_ + "More data")
If the function returns None if it receives either None or a string longer than 10 characters, and if we got Some("short "), foo will return Some("short More data"). The alternatives without an option type, involve at least a couple of branching statements, even if you have a non nullable type, to convert from nullable to not nullable, while Option does it all for us.
This sounds odd. Does your code never interface with Java libraries and frameworks? I envy you! It's all too easy to forget to wrap a suspect value from Java-land with an Option(..). And cumbersome even if you do remember.
And then, there's the code written by Scala novices who just love using nulls. Yes, yes, "code reviews", we all know they are always practiced all the time :/
As for "Scala novices who just love using nulls", if you're seeing that, then the coder in question missed day 1 of Scala training, which is always "if you are writing the word null, you are doing something wrong" (right up there with "if you are writing the word var, you are probably doing something wrong").
As for novices: yes, novices tend to do that. They tend to miss, misremember or fail to put into practice what they should have learned on day 1. I've also seen senior devs who were not exactly newbies (in general) still using nulls because they don't grok Scala; the famous "you can program Fortran in any language". The reality of software development is that you have to deal with all these people.
However the functionality of a null reference type is useful, because it allows a function to return 'no value'. So the goal of an Option type should be to expose the contract for T when there's a value and remove the contract when there's no-value.
The programmer is then explicitly forced to acknowledge the two states that the value could be.
Shameless plug - My C# implementation of Option which is possibly the most complete example out there (at least in C# world):
https://github.com/louthy/language-ext/blob/master/LanguageE...
It's the Java language that forces you to think about nulls, not Optional.
Yes, you can dereference a null pointer and risk an exception. So don't do that.
See where this is going?
The problem with Java's optional is that it is by all appearances (in name and high-level description), a general purpose optional, albeit with a ton of gotchas that will result in people saying "Just don't do that."
Java's optional is not a general purpose option type. In fact, its intended use-case is very specific, according to a rather obscure SO answer by Brian Goetz: http://stackoverflow.com/a/26328555/547365
This is, of course, just tribal knowledge, meaning people are going to be doing shit with optional that they shouldn't, and it's going to be messy.
>Yes, you can dereference a null pointer and risk an exception. So don't do that.
Except these aren't at all the same. With a reference, you have no good way of telling whether your reference IS nullable. With Optional, it's ALWAYS optional.
Therefore it is trivial to a) train developer habits to always check before fetch, and b) enforce an isPresent check with static checks.
Sure pattern decomposition is nicer, but that doesn't mean that Java's implementation "defeats the purpose entirely" like the author suggests. I'd love everything to be checked statically, but if that can't be done, I still like strong conventions which make the code clearer and safer.
As the article mentions, we have annotations to do just that.
>Therefore it is trivial to a) train developer habits to always check before fetch, and b) enforce an isPresent check with static checks.
Again, if we're using static analysis to verify the correct use of Optional, why are we using optional at all? Why wouldn't we use static checks to verify null references can't be dereferenced, instead of hypothetical yet-to-be-implemented static checks that may not be implemented anyway? See https://github.com/findbugsproject/findbugs/issues/56
https://kotlinlang.org/docs/reference/null-safety.html
It's simple:
val x: Foo = maybeGetSomeFoo() // Error: method returns Foo?
val x: Foo? = maybeGetSomeFoo()
x.sayHello() // Error: x may be null
x!!.sayHello() // OK, the !! method throws an exception if it's null
if (x != null) x.sayHello() // OK, flow sensitive typing means the test narrows the type
val h = x?.sayHello() // OK, ?. yields null if left hand side is null, otherwise evaluates right
val h = x?.sayHello() ?: return // OK, ?: runs right hand side only if left hand side is null
I am not sure there is a distinction. Though it is true that some analyses may be run by a broader swath of users.
>As the article mentions, we have annotations to do just that.
Your mileage may vary, but my experience with nullable/nonnull annotations is that because they are not used across the board, you are either left with a ton of false positives or a ton of false negatives. The fact that there hasn't really been a great standardization in the core language is probably evidence of that (yes, JSR305 exists, but the JDK doesn't enforce its constraints by default).
With Optional, it's a clean slate. You can simply say "a reference to an Optional shall never be null" and you could probably enable such an enforcement for an existing project. Of course it's not going to be 100% while it's still a normal reference type, but catching 99% of the cases is a really big improvement!
1) A reference is @NonNull in my code, and I pass it to a component that accepts a regular reference as input. The annotation semantic is simply lost, and I may not even realize it as I'm writing that code. By comparison, if I need to pass an Optional<T> reference where a T reference is needed, then the type system reminds me to consider the present and absent cases. Optional does not allow the programmer to forget.
2) The fact that a component or library was written with annotations does not mean that all usage of the library will properly check the annotations. Sometimes people forget, or sometimes they interact with a component in a dynamic way like through Spring and the semantics get lost. An Optional makes this impossible to forget. You can confuse a @Nullable T with a T, or forget to check upon conversion, but there is no failure case that causes an Optional<T> to turn into a T without handling the present and absent cases.
More choice is better than less choice. You could design an Optional type to omit the get() method. That's less useful than what we have now. Like goto, it has rare but legitimate uses, and it's easy to avoid for the "right way" in the average case (don't use it, flag it during code review, and document the reason why when it's being used).
You do, a reference is ALWAYS nullable/optional. Just because people might expect a certain reference never to be null does not change that. You could equally as well as semantically expect an Optional never to be "optional" (it is just a matter of perspective). People who are too lazy to check for null might be too lazy to check isPresent() and both will give you, unsurprisingly, an exception.
java.util.Optional is really just yet another layer added to support laziness in software developers. To put it bluntly, write better manuals/JavaDocs if necessary and, most importantly, read them.
Good libraries allow their developers to be lazy.
If only FindBugs would warn about calling get() without checking isPresent() first. There's an open FindBugs ticket for this: http://sourceforge.net/p/findbugs/feature-requests/302/
If every method call either returns an actual object then you can get rid of null checks, and not bother doing any checks on method returns that don't return an Optional.
"The last flaw with Java's implementation is a bit ironic — It's possible for the optional reference itself to be null."
ouch.
sorry to rub it in, but Java needs value types. this whole issue, however maybe irrelevant, does not exist in C#.
Yes, it's not ideal, it's still a flaw that should be caught at compile-time, but it's not a big real-world problem, as returning null by accident instead of an option is a pretty rare bug.
The much more common bug (you didn't realize no return value was a case), the Option is a good counter to.
No, it's not a perfect solution, but it's not useless either.
Previous discussion on reddit: https://www.reddit.com/r/programming/comments/3pl7o0/java_8s...
tl;dr: use map, orElseGet, orElse
Checking presence of value of an optional is an anti-pattern, the same as doing != null checks.
You should treat Optional the same as a List type with size 1. You never check if a list is size 1 before doing map or filter.
"If it can't happen, it will".
If a function says the result is optional, then it's optional. It's not not optional because you decided it can't ever fail.
If you don't have a default value to provide, nor an alternate code path other than "sod this, I'm bailing", get is ok. Not quite as good as using a more specific exception through orElseGet, but not bad, exactly.
Also, if you have just called isPresent (and assume your data to be immutable), then get is ok - the alternative being to manually throw something like new IllegalStateException("the impossible has happened!"), which is not that much more useful, and is a pain to have to write all the time when you know it will not get called.
I agree that you should never return null, but if get() is bad style I think you should send out that memo because I haven't yet seen codebases that agreed.
"Comparing Optionals and Null in Swift, Scala, Ceylon and Kotlin"
http://codemonkeyism.com/comparing-optionals-and-null-in-swi...
http://nice.sourceforge.net/
I've included nice as it had a different syntax for optionals (?Type instead of Type?) which I found interesting. I was interested in Nice back in the day.
Tsssk, tssk, you wrote that on the 12th anniversary of the Groovy creator's announcement of the language at http://radio-weblogs.com/0112098/2003/08/29.html
May be name "get()" is too short and innocent-looking.
https://mail.haskell.org/pipermail/libraries/2015-February/0...
You shouldn't replace:
with You should replace it with:Is he? The point is that in Java you are still able to treat x unsafely, while languages with stronger typing do not. E.g. in Haskell, if a function returns a Maybe a, it will always be a Just a or Nothing value. Moreover, such languages allow you to make non-exhaustive matching against all constructors a compiler error.
tl;dr: Haskell, Rust, et al. put the burden on the compiler. Java puts the burden to ensure safety on the programmer. (As can be witnessed in your snippet.)
[1]: http://types.cs.washington.edu/checker-framework/current/che...
And that's OK too, if less than ideal. The Optional type serves as a reminder to use it differently.
Actually, (much to my disappointment as I'm just learning rust) you can just take an Option or Result and .unwrap() and the compiler won't complain at you for not checking it. For such a "safe" strongly-typed language, I'm surprised that so much new rust code does this when:
* the developer believes the Option/Result can never be None/Err (oh come on, just wait until you refactor your code a bit one day and miss a spot)
* the code is example code (I've had little luck finding "good" examples sprinkled about github)
* the developer is lazy (see above)
Allowing unsafe unwraps defeats a core purpose of rust, and the unwrap method shouldn't even exist in the API.
Definitely, but you can also still do this in Haskell (fromJust). But it's better than nullable types since you explicitly have call an unsafe method. (Assuming that you have set non-exhaustive pattern matching to be a warning/error.)
And I meant some notion of 'safety', not fromMaybe/fromJust.
Overall unwrap() isn't good. Really it should be replaced by TODO WRITE ERROR HANDLING in 90% of its cases. That being said there are still good uses for it.
The main _safe_ use for unwrap() I find is when dealing with an iterator of Results/Options (which seems to happen a lot). The pattern:
Is safe, and I really don't know a more eloquent method of handling these operations.> Really it should be replaced by TODO WRITE ERROR HANDLING in 90% of its cases.
That's what the Rust community reads it as, mostly.
I've seen it occur very infrequently in libraries (aside from example/benchmark/test/mocking code, of course). When it does its usually for really out of whack errors like poisoned mutexes, crashed threads, etc (mind you, these are for unwrapping error values, not options, but it's almost the same thing). Or for cases where it's known to be valid due to invariants that can't be expressed in the normal way.
I did a quick grep of hyper, and the unwraps are almost all in "dev" code (tests/benchmarks/etc). The cases where they aren't are where there was an `is_some()` earlier (and due to some reasons it couldn't be restructured as a match, perhaps just because of rightward drift), or the "invariants" thing -- i.e. it often calls `<some url>.serialize_path().unwrap()`, which is OK, because hyper only ever deals with relative schemes (e.g. `http://` and `https://`, not `data:` or `mailto:` -- the latter two don't have paths), which is checked in the constructors, so this should NEVER panic. I didn't go through all the code, but in the main HTTP and HTTP2 code serialize_path was the only offender, which was one that was justified.
I see it more often in applications. In the following cases:
- Example applications in blog posts (because you want to explain something else, not spend too much time on error handling)
- Slightly less out of whack errors which are weird enough that they should close the app
- Normal errors which should close the app (e.g. "insufficient permissions" for a command line util, or a port-opening error on a network app). unwrap() prints out the error message so this is a very rudimentary form of error handling anyway.
- Laziness, or often with a TODO. Not too common.
So, it's mostly used in justifiable ways, and that's why it exists in the stdlib. It doesn't "wipe out" anything that the borrow checker provides (borrowchk and exhaustive match are two different things), and while it's a kludge if used often there are legitimate reasons to use it; enough reasons to have it in the stdlib.
> I really don't know a more eloquent method of handling these operations.
:)(but yes, in general there are situations in which you know an Option is unwrappable, e.g. the serialize example above)
Thanks :D
I've been really struggling to get iterators in rust. I don't have a very strong functional background :|
So `filter_map(|x| foo(x))` can be written more simply as just `filter_map(foo)`.
https://github.com/Manishearth/rust-clippy/issues/116
This presupposes that the pattern that "unwrap" wraps is always a bad thing anyway, which it isn't. Sometimes the right thing to do is to panic your program if None is present. And sometimes the type system isn't sophisticated enough to express an invariant, and you need to use "unwrap" to work around it.
By the way, the borrow checker has nothing to do with unwrap().
(Not to mention that `unwrap` in no way subverts memory safety, which makes mentioning it in the same breath as the borrow checker puzzling.)
That said, I think you're too dismissive of "can never fail" assumptions. There's tons of places where something is optional in general, but based on various invariants you know it won't be. For instance, if you successfully acquire the first element in an array, you know you can get the last.
If you legitimately believe that Option is always Some, I don't understand what you think you would otherwise put there? Gonna have your function return an Err if an internal invariant is broken? Gonna have callers actually check for that?
Absolutely you should return an error. Whether the caller wants to panic or handle it or print unicorns should be left up to the caller, not your function. Functions should not be expected to tear down the thread in case of an error. Nothing that panics should belong anywhere in exported code
Most errors are recoverable. panic! is an antipattern in a library. Or at least, provide both a panic-ing and a non-panic-ing variant.
We're actually in the midst of arguing whether to violate this convention and do it for RefCell: https://github.com/rust-lang/rust/issues/27733
Using assert! to verify that your state is consistent is useful, in case you have a bug. But since it's all internal, it's not something that the caller can really recover from, either. It's not their fault, it's yours.
Worst of all, it leads to untestable code, because the calling test case can't properly check the error against some known result.
* Index out of bounds on every array op
* Integer overflow on every arithmetic op
* OOM on every allocating op
Maybe if you're writing an ironclad RTOS for a critical system without any good redundancy? Otherwise these are such pervasive operations that most have accepted that they're not worth handling everywhere they happen. Requiring that really dilutes the value/meaning of errors.
Given this signature:
fn do_thing() -> Result<T, E>
There's a clear signal that there's legitimate error conditions that you probably want to think about today. If every function that accessed arrays, worked with integers, or allocated memory returned a Result, it would border on meaningless. It would be like if there was no Throwable/Exception distinction in Java. Everything would `throw Exception`, eliminating the value of even noting that something can fail.
I think the only way such a system could be tolerable is if the language in question had really good dependent type support (but that wouldn't handle the OOM issue -- which you can't even reliably handle on some systems).
Unwinding/crashing is valuable in a truly robust system, because it needs to handle crashes anyway. Might as well punt obscure problems to that level of the reliability system. (This is basically the basis of Erlang's task system, AFAIK)
No. You have already diluted the meaning of errors, and you want them elevated to _your_ standard.
>Index out of bounds on every array op
These are removed if you build a rust program with --release.
>Integer overflow on every arithmetic op
Add the Wrapping class if you expect overflow. Overflow _shouldnt_ normally happen on an Integer operation. It is a hardware error when it happens, and can cause massive pain-in-ass bugs when it happens unexpectedly.
I'd rather get errors when it does happen, rather then find out 6 months into a production run.
>OOM on every allocating op
C does this also.
Rust doesn't remove bounds checks in --release. It's wrapping that gets turned on in release. I'm not sure why you're distinguishing overflow as truly exceptional, as opposed to any other "this should never happen" error?
Also I don't think many C libraries that allocate expose that as a failure condition (I've certainly seen some which don't even check!)
This is a very common mistake in most C code. Malloc can fail, and it return NULL when it does.
Explicit vs implicit. In Rust/Haskell you are forced to do _something_ about the nullability. If you use unwrap/expect/fromJust you are explicitly acknowledging that you want it to panic if it fails.
On the other hand, in Java, you can get an NPE where you didn't expect it because nulls move around easily and there's nothing in the type system to prevent that. That's the difference.
> I'm surprised that so much new rust code does this
Also, in my experience, unwrap() in Rust is mostly confined to irrecoverable errors in applications, and example blog posts (where the focus is on getting something done and not worrying about good error handling).
> defeats a core purpose of rust
What purpose? Whatever unwrap provides is still possible via a match+panic. Which is _more_ explicit, but they're both still explicit.
If you use Optional.get() without Optional.isPresent(), how is that any different? It's not as nice as pattern decomposition, but it's still fundamentally the same, and on top of that, transform functions are provided so that most of the time you can do null-safe operations.
In Rust you'd rarely write a library that panics on unexpected nulls.
(Also, panic != abort, yada yada)
unwrap() is a convenience method that still uses exhaustive pattern matching and will panic if no value is present. The developer is making a conscious decision to panic in a not-present scenario, and on a case-by-case basis. The compiler is still doing it's job. This is different than java, where a get() on a non-present value is a runtime violation.
Both languages discourage the use of these constructs, but they exist for good reasons.
Curious, what are your good reasons for fromJust in Haskell?
head []
In my code, I'm only dealing with null if a API gives me one. At that point, I take some sort of action to make absolutely sure I'm not passing it along to anyone else. This means using exceptions, the null object pattern, or Optional without a need to dereference. The goal is to prevent a proliferation of references that could be null and therefore have to be checked.
(Also I assume the first is intended to be x != null)
You also get map, filter, flatMap, orElse, orElseGet, ifPresent. These are all great ways of making your code more functional, and do more with less.
I don't think annotations are the answer either, as you can't always rely on static analysis to prevent errors, a lot will still appear at runtime.
Basically, just assume every Java type is wrapped in Optional implicitly. But by adding Optional (and using it in standard libraries) it's often a matter dealing with code that interfaces between the two styles -- another problem that didn't exist without Optional.
And even if you think Optional is worth the small gains, it's still a sad mimic of the Option types implemented in other languages.
You should almost never call .get(), except in cases where the code path does not allow an empty optional.
Even so, calling .get() on an empty optional is better than handing nulls around. A null may -- by chance, really -- make it several lines down the code, so that the stack trace points you much later in the code than where the null originated. Calling .get() on an empty optional immediately throws a NoSuchElementException, pointing you directly to the line where the optional was empty where it shouldn't have been.
Consider this code:
where An NPE occurs inside of formatUserSummary. Where did the null come from? Is the user's Franchise null? Maybe the Franchise's Location? Or is it the User that's null?If you instead took an Optional<User>, even if you just immediately unwrapped it with .get(), you'd at least get an error that unambiguously told you that it was the user that was missing.
A bit offtopic but SAP JVM puts that info in NPE error message[1], I don't know why others implementations don't, maybe it is incurring too large overhead?
[1] it is something like "Attempted to call getLocation() on Franchise object returned from getFranchise() but it was null"
https://scn.sap.com/people/desiree.matas/blog/2011/12/07/sap...
Here is a presentation with some interesting points, wonder if anyone has any real-life experiences?
https://www.youtube.com/watch?v=arryAs9lqfE
E.g., imagine you got a NPE from this:
...you'd think "user" must be null, right? But it's quite possible that user.id is a Long, the current value is null, and getUserSummary() requires a long primitive argument.This compiles just fine, and at runtime the Long object will be auto-unboxed to a primitive long... unless it's null.
Surprise!
But things like Optional do help; doing a code review for the things that may be null will fix these kinds of mismatches (and you'll avoid relying on auto-unboxing). The above code is broken -- either user.getId() can never be null (in which case it should be a primitive already) or it might sometime be null, in which case this code must not allow auto-unboxing.
In any case, that's not what Java's optional is there for[1]. Solving the safety problem is the job of Java 8's pluggable type systems[2] that are both more powerful than the static-analysis tools listed in the article and had Oracle's official support (in the form of JSR-308).
[0]: https://twitter.com/BrianGoetz/status/656860771196887040
[1]: http://stackoverflow.com/a/26328555/750563
[2]: http://types.cs.washington.edu/checker-framework/current/che...
Some people claim it's a sort of documentation denoting 'intent', but I don't want documentation. I don't want options to infect the whole codebase, I just want to pattern match once and extract the value, and never have to check again for None or null.
The problem in Java is the type system has references. X static use_it(X x){ ....}
really has a signature something like this: use_it : X ref -> X ref
no little library is going to fix that.
One addition that would be useful would be some syntactical sugar to reduce the verbosity of using Optional<T>. But otherwise I'm happy and if you combine with @Nonnull and @Nullable annotations you get the best of both worlds.
eg
So if you do the compiler will require you to do something about the DatabaseException.It seems to me that the point of, for example, Option and Result in Rust is to avoid the whole checked exception mess while still passing useful information about things that go wrong. But, given the problems pointed out in this article (lack of pattern matching and algebraic data types mostly), checked exceptions might be a more correct way to do it in Java.
Without exceptions you would have to return a value that could discriminate between all 3 possible outcomes: nothing, error, result. In Rust this is the `Result<T, E>` struct.
The biggest problem I found in applying Optional is that Optional is very much a Haskell-y concept to me, and Java programmers and Haskell programmers don't really overlap much. Therefore, to me, Optional is just a functor like a list or a set. So doing Optional.asSet() and iterating over the result is, to me, the correct way way of handling an optional value. If there's something in there, the loop runs once. If there's not anything in there, the loop runs zero times. This is exactly like "fmap" in Haskell:
(Except that Haskell automatically preserves the type safety, whereas you have to re-wrap the value if you're using Java. Functional programming in Java never works, and this where the Optional idea starts to go wrong.)But I got "I don't really understand this" in code reviews every time I did that; the Java way is to check if it's set, then do a type-unsafe unboxing. Thus, like the article said, completely defeating the purpose of the Optional type.
That said, it's pretty rare to need exactly 0 or 1 of something. If you're doing a database query, you can probably make the API return a set of rows. Then it's natural to loop over the results and nobody raises their eyebrows.
That's interesting and sort of hard to agree with. I think uniqueness is very often desirable and very often paired with a lack of certainty about existence.
"Does a user with this identity exist?" certain is handled properly as a 0-or-1 question, e.g..
probably more idiomatic for the 1st version would be:
i.e. string interpolate and map instead of match over Option.Another word I use a lot is "lookup".
Optional has map() in Java.
The other problem is that the Optional<> interface is clunky. There is ifPresent(), but no ifAbsent(). The stream syntax is incompatible with the rest of the Java world, in the same way that List<T>#toArray has always been ridiculous. And if you use ifPresent(), you have to live with the fact that you cannot return from Java lambdas, and you cannot change any variables or even use non-final variables. I want to scratch my eyes out when I see people allocate one-element arrays on the heap just to work around Java's lambdas.
I don't see the value of Optional<T> if you already have @Nullable.
That's what map/flatMap is for, ifPresent is really only useful for side-effects.
To me Optional<T> without pattern matching seems crap.
"What is worse than not given anything to somebody that need it? Give to him something broken".
> maybeT.map(t -> ...).orElse(...)
The author does make a good point that 'get' is unsafe. I think it was a necessary concession to team members who are learning how to program in this manner.