171 comments

[ 2.5 ms ] story [ 232 ms ] thread
Edge case removed:

   for i=0 to 10 {print(a[i])} vs for x in a {print(x)}
With functional programming it gets even better:

a.filter {}.map{print}

Pseudo code, of course. I list several “functional“ examples in Swift here:

https://github.com/melling/SwiftCookBook/blob/master/functio...

Functional programming is also a higher level language. map, reduce,filter, flatten, flapMap, drop, take, zip, ...

Learn the concepts in one language and they can be used in another functional language.

Last I checked functional programming doesn't have a monopoly on "map, reduce,filter, flatten, flapMap, drop, take, zip", these are functions and can be implemented in any language, object oriented or otherwise.
The problem in my domain, games, is the frequent boxing and unboxing of values when using functional iterator interfaces in languages common to the industry, particulalry C#. It just thrashes the heap and forces often too-frequent gc.
I was under the impression that the way C# did generics meant that there was no need to box values. (Unlike Java, which treats all values as objects, even value types.)
C# has value types and many things are value types but certain things are not. Core library interfaces are implemented as with Object enumerator types for example. This causes IList.GetEnumerator() to return a boxed type. Its smart enough to use the value type when it knows it can though, such as List.GetEnumerator().

Lambdas have some allocations. I like LINQ but it can have some allocation as well.

These are all tiny allocations but in a game you'd like to strive for 0 allocations.

Two ways to interpret boxing here:

1 Storing a primitive as a first class object

2 packaging and repackaging data

There may be situations where the former is not an issue, but the latter often is. If you don’t have a way to treat filter() as a generator/coroutine it means your traversing one collection and inserting those values into another, only to pass it to map() which iterates again and creates a third. If you’re doing that once a frame, maybe not a big deal. Three times per object per frame? That’s a lot of temporary data structures.

If the other responder is right, I speculate that the borrow checker forced someone’s hand in Rust and they found that a generator keeps the borrowed objects on the call stack where they already had tools to track ownership.

Correct usage of IEnumerable helps a lot, as well as the magic that Linq can do with expressions.
In my experience, the trick is to remove `using Linq` from the entire project.
As someone who has had a mild case of Linq jealousy for some time, I’m curious what makes you averse.
It's too easy to fall into the temporary collection trap you described; on a large project with many developers, allowing Linq is a bit like leaving a toaster plugged in beside the pool.
While it may or may not be optimal for you/your field for other reasons, I believe Rust does iterators in a 0-cost, no allocation way and generally generates code equivalent to or better than a hand-written for loop.
There are studios moving to rust, and there definitely is a great amount of interest in rust among my peers because of the 0-cost iterator features. The borrow-checking isn't even a big draw, it's the other features!
While it does remove edge cases, the code also becomes more opaque. At least to me, functional style programming seems much harder to read and then reason about.
It takes some time to get used to and change mental model from "program describes step by step what to do" to "program describes logical flow of information".

I moved from first camp to the second one somewhere during my adult life. And IMO: Both are valid, sometimes first one is necessary (because that's how the machines actually work), but second one is clearly better most of the time for humans, especially in business-logic heavy cases.

There's an analogous switch in algebra from notation like f(g(x)) to something more like X_{GF} (where the _{} indicates subscripting, not written symbols). Note the better left-to-right ordering in the terser system.

It's the same dichotomy; are you thinking of f(x) as an instruction to do f to x, or are you thinking of X_F as what happens when X goes through F?

This is a learning thing, honestly. Once you start thinking in types, it clicks: you're thinking at a higher level of abstraction, so the code is less important than understanding how the programmer put the types together in their head to solve a problem.
But I am already thinking about what the code should do. The problem is that if I read functional code I get bogged down in the details of what limits are set by the operations, because they're usually not as explicit. It's like reading a chain of functions on some element (actually, it quite literally is that).
It's subjective. My experience is exactly the opposite. A pipeline of sequence operations is something I can skim at a glance, but a for-loop is a pile of code that I need to stare at before I understand what it represents.

This does not invalidate your experience. It's common, and with good reason. This sort of dissimilar experience with the same code is why coding standards need to be views as social norms for the team using them, and not just technical things.

The problem with loops is that they are too powerful. They can do almost anything.

The functions map and filter are nice and restricted. If you want more power, you use eg a fold. And if you need even more power, you write a new recursive function.

So a quick look at the code will tell you what to expect.

Personally I've found that although this argument sounds good in theory, in practice I don't find loops harder to read than chains of combinators.
Which languages are you reading them in?

For me the argument holds in practice as well. I'm mostly reading my combinators in Haskell, and the loops in Python or some curly brace language.

If you use a general purpose combinator like mapAccumL that has almost as much power as a loop, you don't get that much extra readability compared to the loop.

It's still a bit better, because that combinator still has to process one element of your list for each run through it's "body"; and it clearly warns you by it's very name that it's not just a filter or map, so there's less possibility of confusing it for something simpler.

And, of course, not all loops are created equal. When applicable, for-each loops are much cleaner than classic C-style for-loops for example.

Depends. Though some black-boxing is intended: thanks to enforced information hiding the compiler (and you) can make additional assumptions about what your code can not do.
A lot of those concepts were already in Smalltalk.

    a do: [:x|  Transcript print: x ]
    [] = closure 
That is, because Smaltalk was based on Lisp. But the other OO languages removed this concept, but it came back in Scala.
I program mainly in C++, and I often start off writing things as:

    for (auto x : someContainer) { //do something with x }
But inevitably, I end up needing either an iterator pointing to a particular item at the end, or I need an index to pass to some other function, and this style makes that impossible. Perhaps it's just the domain I work in, but even after years of trying to make this work, it still only works about half the time for what I'm doing.
In Haskell

   map someFunction listOfValue
if you need an index:

   map someOtherFunction (zip [0..] listOfValues)
Python uses 'enumerate' to similar effect. Not sure if C++ has an equivalent?
You can also use:

  zipWith someOtherFunction [0..] listOfValues
  # , since someOtherFunction is likely to be
  # :: (Int -> Value -> Stuff) rather than
  # :: ((Int,Value) -> Stuff) .
  
  zipIndex f = zipWith f [0..]
  # is also often useful.
Yes. I wanted to keep things simpler for people who don't speak Haskell; and show off composition of simpler combinators to build more exciting things at the same time.

In eg OCaml you have to use something like zipIndex (but implemented differently) or mapWithIndex, because they don't do lazy data structures by default. (And in eg Python their lazy sequences come with lots of caveats because they are not referentially transparent.)

Why not just pass the reference instead of an iterator or index? That’s what I do the best majority of the time. It’s rare that I actually need a specific iterator or pointer.

Just change your loop to auto& x.

The goal of software development should be about reducing the number of edge cases to the absolute minimum. If a tool makes it easier to keep track of all edge cases, it is bound to encourage developers to write code which contains more edge cases. But fundamentally, it still reduces code quality. Code with more edge cases is less flexible and not good at handling changing requirements.

I've seen this over and over in my career; tools can make people lazy in certain areas, it can put them on auto-pilot. Often, this can be a bad thing.

I like coding with dynamic languages because they get out of my way and shift the full responsibility for correctness on me. This creates a certain mental tension which helps me to perform. Keeps me alert and mindful.

I particularly try to avoid languages that make me wait for code to compile (which is more common with statically typed languages); this is for the same reason why don't like it when someone distracts me while I'm "in the zone" while coding.

Isn't this akin to saying I don't like wearing seatbelts because it makes me more complacent in my driving?
It could actually be the case that seat-belts make drivers more complacent but the key difference in the case of coding is that you won't die if you make a typo in your code. If designed correctly, your tests will catch it anyway.
Your tests? You should despise tests if you want to be consistent with the attitude described in your first post.

In the mean time I will just keep using my strongly typed language and, of course, also automated tests. Thank you very much.

The selling point of types (and type inference) is that they save you from having to write half your tests.

And of course, lines of code that you didn't have to write are less likely to have bugs in them.

Someone once describing the forces acting on a race car pointed out that the horizontal forced in a curve were high enough to throw you from the car if not for the five point harness (F1 probably doesn’t have this behavior due to the cockpit design).

Sometimes safety equipment lets you go faster.

> Sometimes safety equipment lets you go faster.

Even simpler: brakes let you go faster.

When I was cycling a while ago, my brakes broke in the middle of a cross-country ride. I didn't want to push the bike back all the way, but you can bet your hat that I rode extremely slowly and cautious.

Both brakes broke, or the front brake did? You can get by with just the front brake. Question is if you had enough tools to do that surgery. Would depend on the brakes.

I don't think anybody remembers how shit bike brakes were before dual pivot came along. God bless Shimano and keep it forever. When I got my first set, I started emergency-braking with 2 fingers because I was worried I'd pole vault myself if I actually squeezed like I meant it. (I have high grip strength).

Was a 'flevo racer' recumbent bike that I was still repairing, and had only one brake (front) at the time.

I was only a few kilometres from home.

This is the opposite of my experience. In practice, being forced to deal with edge cases makes you find implementations that eliminate them because dealing with each is painful.
This is a wrong and surface-level reading of the original post. If there are "cases" in a system, then there are going to be "edge-cases" as well. The problem is in classifying certain cases as "edge" and others as regular. We do this because the regular cases are very obvious and are either the perfect happy-path, or a failure condition that we would immediately consider.

But between these two exist a permutation of conditions that exists in the domain and often manifest in production. Either we can actively manage them thanks to types, or we can let it end up in production with hard-to-track bugs.

Or perhaps it forces you to be alert and mindful of things that add no value, such as manually remembering to handle every edge case, and takes away focus from the problem in question?
(comment deleted)
This doesn't really make any sense. The idea that forcing you to handle all cases will encourage creating more edge cases is nonsensical.
I've been translating a personal project (game engine) from a dynamically typed language to a statically typed one. Having to pin down what the types of function parameters are and spell out interfaces for objects has been eye opening. Using a dynamically typed language had made it way too easy to stuff things that didn't really go together down "the same pipes" of the existing object collaborations. I thought I was writing simple code, but it only appeared simple because I wasn't fully aware of the implicit (defined by behavior) types of my dynamic code. Granted the pace re-writing this with a statically typed language has been slow enough that I can't say whether it's a win overall; my point is just that it has revealed edge cases I never considered in the dynamic code. The argument you made that tools can make people lazy - doesn't that apply as well to dynamic typing?
Forget the performance or the fancy memory management stuff, this is one of the best things about Rust: it makes you deal with edge cases by default (while allowing you to write code in a largely imperative or functional-lite style).

It's why I think it competes for market share with some of what is currently Java/C# and similar languages. A lot of people who use those languages care about correctness, and Rust is big step up in this regard.

You can use a functional style in those languages as well. Or do you mean with Rust's borrow checker you have less mutability to worry about?
If you have a Rust enum with Color{Red,Green,Blue), if for some reason you update to Color{Red,Green,Blue,Yellow}, the rustc compiler will force you to manage the new Yellow case wherever you want to "match" a pattern on Color, so overall you have less chance to forget to manage the new case everywhere. This kind of error is not managed by a C or C++ compiler (as far as I know).

But from my understanding, this is not directly related to the borrow checker, which tracks who owns a reference and is allowed to modify it or not.

(That being said if you had a "default" case earlier, the compiler won't complain)

If you compile with `-Wall` GCC will emit a warning for failing to exhaustively check enum values inside a switch statement

(I don't remember the specific underlying feature flag name, sorry).

EDIT: It's called `-Wswitch`

Yes, apparently, there is a -Wswitch statement in gcc, which looks quite similar, maybe I have used too old compilers...

> Warn whenever a switch statement has an index of enumerated type and lacks a case for one or more of the named codes of that enumeration. (The presence of a default label prevents this warning.) case labels outside the enumeration range also provoke warnings when this option is used (even if there is a default label). This warning is enabled by -Wall.

The issue with -Wswitch (and thus -Wall) is that the presence of a default case suppresses the warning. Which you may not want, if you need non-default behavior because you're adding a new value! There's -Wswitch-enum (not turned on by -Wall or -Wextra) that stops this being suppressed. This is good, though the (possible) list of cases that are well-handled by the default at the end of switches that just fall-through can be ugly.
A default case supreses the warning in Haskell too, there is no other option.

The real problems with C enums aren't about matching completion, they are that:

- There is no guarantee that a variable value is in the enum range;

- A C (or Java, or C#) enum is a very poor type and can not represent all the data that a Rust enum carries. Developers usually use them with union types to solve that problem, but then there aren't any warnings for most problems anymore.

The big difference is that C++ emums can't contain data. C++ has checking on enums, but enums are a relatively niche data structure for specific use cases.

In Rust, enums are a core data structure with completely equal status to structs, and are used pervasively throughout the language including the standard library.

Notably, both Option (which is used instead of null) and Result (which is used for error handling) are enums, which means that you thse correctness checks for every null and every error condition, on by default. Thats a huge deal as these are often the source of unexpected runtime errors.

This isn't particular of Rust. Nim also forces you to deal with all possible branches of a case statement. I'm pretty sure other languages do so as well.

This is just basic type safety stuff.

plenty of other languages do so, but it's both unfairly diminishing to the parent & questionably correct to refer to it as "basic type safety stuff"
JS has this too in TS, so I’m pretty sure the perception among users of typed languages is that this is a basic part of what type systems do. What does this have to do with FP?
TS actually has quite an advanced type system compared to languages like Java.
On the other hand, it's not a very advanced type level feature either to insist on total functions. I'm sure Rust's type system goes a lot further in enforcing a good level of type soundness.

Edit: typo

No, it is basic. If you have a sum type, and you have no defined behaviour for one of the terms of the sum type, your code is not type-safe and a language that aspires to "basic type safety stuff" should at least issue some kind of error about it. In a putatively statically-typed language (ie one that does type checking at compile time), you'd expect that error to happen at compile time.
> This kind of error is not managed by a C or C++ compiler (as far as I know).

C++ had boost.variant which did exhaustiveness checking since ... 2002 (and an std version since C++17).

Many people have their compilers set to enforce these things, and many of us go a step further and make our compiler warnings into errors so it’s impossible to build a program with these errors. That’s very commonplace, and C++ does it just fine. Missing cases in enums and at least 100 other very common matters can be checked at compile time in C++, the only difference is that it’s not the default as it is in some other languages.
Enums are no substitution for sum types, though.
The comment I was responding to was not about the functionality of a sum type, but rather the compilation errors from leaving out a case. Those are two different matters.
Kotlin also enforce exhaustivity in when(), for enums and ADTs (sealed class's)
The borrow checker takes care of a lot of the things that many functional patterns are supposed to prevent, such as referencing values in two locations.
This is also one of the problems with programming with exceptions. You lean on the fact that an exception will be thrown and just propogate up until it finds something, so the normal mindset of an exception-based programmer is to program for the happy-case and only vaguely think about what happens if it goes wrong. I know, I've written a lot of code in this style for many years myself.

In some cases, that's not a problem or it's even the best thing to do. However, you end up writing this way all the time in an exception-based language, which is where the problem arises.

Non-exception based languages tend to throw the problems in your face and make them something you can't ignore. Which is not fun. But it is often what you need.

There's also the OTP model, where you have exceptions as well as a robust "something" up there. You don't have to deal with the minutiae, and the pattern (processes (aka, isolated processes) and restarts) lets you deal with known or unknown errors.
Right, but the OTP doesn't make you deal with the exceptions upfront, only to architect your app in the way that they don't crash or infect other parts of the system.
That doesn't help you detect logic bugs, only bad input.
This is especially evident in e.g. third-party Python libraries.

For example, say you have a pandas dataframe with a column "foo" and you accidentally try to read column "fooo". Instead of getting a useful exception like "PandasException: Attempted to read nonexistent column 'fooo'", you get a traceback 10 layers deep, terminating in something like "pandas.hashtable.PyObjectHashTable.get_item" which ought to be totally invisible to you, the library user.

This is how it should be done, I'm such a fan of this page: https://www.psycopg.org/docs/errors.html

Edit: by "this" I meant to agree with parent: Getting a useful, well defined exception is how it should be done. Psycopg2 does this very well.

> the normal mindset of an exception-based programmer is to program for the happy-case and only vaguely think about what happens if it goes wrong.

I'm not sure that's such a bad thing. Exception handling doesn't mean you should be handling all exceptions, just that you can recover if one does occur. If an exception is 100% relevant to what you're doing then yes absolutely handle it, or even clean up your own state before propagating the exception if you can't. But please don't catch any and all exception to suppress or obfuscate them by wrapping it in another domain specific exception.

> Exception handling doesn't mean you should be handling all exceptions, just that you can recover if one does occur.

You can't because you don't know where it came from so you don't know what happened, why, or if it was recoverable. You can take a stab at it, but that's it.

> You can't because you don't know where it came from

Maybe that's a particular language constraint? In general exceptions have a stack trace and it's evident where an exception came from.

At least in my experience exceptions don't have to be handled unless or until they're disruptive. If you have a scheduler or task processor code that needs to run continuously then you're going to have robust exception handling, but not necessarily comprehensive or appropriate. For other areas of code however the exception handling isn't necessarily requisite.

My problem with exceptions, is it's generally not possible to tell which exceptions a given function might throw (especially problematic with nested dependencies). Thus even if there was a condition hat you could handle you might not know it was possible (until you hit the bug in production!)

Result/Maybe types are awesome, because you just don't get unexpected runtime exceptions due to programming error. You only get them when a genuine problem occurs.

> it's generally not possible to tell which exceptions a given function might throw

That's a valid concern. I've seen attempts to document the custom exception types that classes can throw but it's never discoverable. It's compounded by the fact that uncaught exceptions bubble up.

I don't know that most people would sift through all the possible exceptions and write selective catch blocks to handle them. I think in general they will either catch all exceptions or none, if or until they need to handle a specific scenario that presents itself in production.

My perspective though is primarily of LOB Apps or Integrations and not widely consumed public code or apps.

Exceptions are the worst because there's a hidden nested world of control flow you can't barely begin to reason about. Exceptions are a "comes-from" statement, like a "goto" but much worse.
This just gave me the thought... if exception stack traces came with the argument values of each method call, they would be 1000x more helpful. You wouldn't even have to attach a debugger and step through in many cases.
I think they do in python, or at least can be retrieved. I know they show up in the stack trace on Sentry when an unhandled exception occurs.
Interesting, I guess they don't get spit out to the console by default.

Overall Python has really good stack traces, but the hyper-dynamic nature of so many common libraries does make it tough to grok sometimes.

I usually end up reaching for the PyCharm debugger which is fantastic anyway.

(comment deleted)
Exceptions and Maybe types aren't mutually exclusive. They should be used together. Maybe types handle variability that is expected and predictable, like a dictionary not having a certain key; exceptions handle the unexpected situations, like a file handle being closed too early.

Maybes explicitly express what the programmer has foreseen, exceptions form a safety net over it. You need both in a wholesome language. That's wht even Haskell has exceptions.

Having used Rust, which almost exclusively uses Maybe types, I have to disagree. The "safety net" introduced by exceptions just hides possible errors, and causes reliability problems that would be entirely avoidable if Maybe types were used. The `?` operator makes propogating errors painless if that is what you want, but at least they're all documented so you have to make a concious chocie to pass the buck on error handling.
I don't know: if we really wanted to be 'strict' every integer operation should return a Maybe due to possible overflow, every IO operation also.

That's a lot of possible error to handle! I don't like exception's invisible flow but I think that they are good for 'should never happen' errors.

Rust forces you to deal with edge cases using very low-order, often Procrustean bed reasoning. This does not mean there are much more powerful higher level ways of doing this. (Pattern matching I got in exchange have made me cry)
That is simply not true.
If it forces you to check every single return code, this does not mean you can't have equivalently safe implementation without that level of detail.
Most languages rely on coding patterns for code safety. This is merely a best practice, usually involves writing multiple lines of code, and is merely a convention.

Functional languages have many of these patterns already encoded into common functions, like map and fold. As functions their correctness can be considered mathematically proven. Would you rather rely on convention or the compiler?

Consider adding on top of functional safety/correctness type safety with a functional language implementing a strong type system, like F#. Sure, F# is only functional first, not purely functional, and probably all type system have some sort of implementation edge issues. For most practical purposes you can consider the added level of type safety imposed by the compiler also to be mathematically provable.

What kind of mathematical correctness are you getting by using FP? The same kinds that the Rust compiler offers?
Different, but related. If you call 'map' then you know you're not mutating the values, and you get an output the same size as the input without dropping values, and...

It means you operate on a higher level of abstraction, and that inherently makes reading code faster. Not per line of code, but certainly per functional unit.

Idiomatic Rust tends to use a lot of functional code, and provides all the tools to make that safe. Its map function could mutate the input -- it would be unidiomatic, but certainly possible -- except that, if you don't pass a mutable reference, then you're safe to assume it won't.

I worked with Paul at a previous job and really enjoyed it. Such a great mind and passion for functional programming. Would love to work with him again (maybe in a different context).
That's unbelievably kind of you. Hopefully our paths will cross again!
Paul Snively (the author of the linked reddit post) also gave a great talk entitled 'Typed FP on the Job - Why Bother?' at LambdaConf a couple years back, that spoke strongly to me:

https://www.youtube.com/watch?v=8_HsFrXhZlA

> And now you can do the most important thing you can do with any piece of code... stop thinking about it! Go home! Pet the cat! Watch House with the wife! ... I love Friday deployments ... you know why? Because I know what my code is going to do before it runs!

(comment deleted)
Interesting talk, but does that retry logic really provide me with more confidence about its correctness than equivalent impure code?
Thanks, and good question.

I deliberately didn't go into the specifics about the various typeclasses at play and their laws, and how those are tested. Part of that is just for reasons of time, but the more salient reason is that I wanted to show how those of us who do purely functional programming do it in practice. So if you watched the presentation, you saw that most of my thinking was about "OK, how do I elaborate from the most trivial transformation (do nothing at all) to the one I really want?" And that proceeds compositionally: I transform this value to that value. OK, does that value have the right type? No? What do I need to do to ensure that it does? And the transformation steps have some important properties, like their scope being entirely local. I really tried to emphasize this at the end with `attemptRepeatedly`: my description of each line of the whopping three lines is exhaustive. When I say there's no point in writing a test for it, I mean that literally. There certainly are typeclasses at play, and I briefly talk about the `Catchable` typeclass, and show the ScalaDoc for it, which documents an important law: the relationship between catching ambient exceptions and the algebra provided by `Catchable`. I rely on the other typeclasses and other laws in a similar fashion.

The other thing I think is pretty important is the part where I say "Let's look at cases," because the point there is that I can reason about the code by reasoning about the shape of the data it's manipulating. So if a `step` is an `attempt` of `p` that, if successful, `kill`s the retry `schedule` or, if unsuccessful, logs the exception, I'm still dealing with a `Process` of one element (by assumption that `p` will emit one element). Then `retries` will be a `Process` of 0 to infinity elements, because we put no constraints on `schedule`. So `(step ++ retries)` will be one or more elements, and because `step` `kill`s `schedule` on success, because `retries` is derived from `schedule`, `retries` is also `kill`ed. So `(step ++ retries)` is a `Process` that will emit one or more elements, with the last element being the first successful one, or the last failed element if none of the `retries` succeeds, so we take `last`. Then we just `fold` the failure or value back into a single effect, and we're done.

As I discuss in the presentation, there certainly are questions. What happens if the `schedule` is empty? What happens if the `schedule` is infinite? An attendee in Q & A asked a really good question: was I sure `retries` would wait before the first retry, or was the semantics "try, then wait?" (It really is the former, but that wasn't clear from my recorded REPL session.)

Of course, this isn't very impressive for a three-line example, although I think it's pretty striking that it only takes three lines, each of which can be completely reasoned about independently, to achieve a pretty significant operational goal. The point, though, is we can build entire systems this way, and in fact `attemptRepeatedly` is part of a distributed monitoring system I worked on at Intel Media/Verizon Labs, which is written entirely in this way apart from the monitoring types themselves, which present an imperative API for familiarity's sake (and which we later came to regret).

I hope this helps!

I'm so glad that struck you! I think a lot of organizations talk about "work/life balance" without really doing anything concrete to make that possible, and I certainly appreciate the irony that this purportedly very abstract, theoretical approach to writing software is, head and shoulders, the most impactful thing to my quality of professional life.
I am not sure this is the only value, but it is generally true. And this is one of the great advantages of haskell.

By the way, now that I know some haskell, I see this edge case sloppiness all the time and it is really annoying. For example, here is something that annoyed me just recently. You go to yahoo finance and there they will show you the revenues of a company as well as the revenue growth from past year. But sometimes this revenue growth number is not available. For example, the company may not have existed past year, or perhaps it was not public past year and thus it did not publicly disclose its revenues. When they cannot compute revenue growth for these reasons, yahoo finance will helpfully show revenue growth as "N/A", i.e., not available.

That seems all nice and logical, but I noticed that yahoo finance tended to show that revenue growth was not available for many established public companies for which they should have the data. I looked at it more closely and noticed with shock that if the revenue growth of a company was about 0%, yahoo finance would still show it as "N/A"! So some programmer just decided to use 0 as not available and just erased a lot of useful information from their system. Now when I see a company that has N/A as revenue growth on yahoo finance I have to go and check whether the revenue growth was zero, or it was truly unavailable. As I said -- really annoying.

This is of course a textbook case for Haskells Maybe concept. You can have a type that is a Maybe Int, and that means it have a value or it can be Nothing. Thus, you encompass the edge case of not having the value available, without using an embarrassing hack that deletes real data. But of course Haskell gives you more power than just using Maybe. You can create a type that can encompass all kinds of edge cases and information about them. For example, you can embed a reason why the revenue growth data was not available, if that is the case.

Swift has this as well, but calls them “Optional”. The concept is definitely useful!
Also Java has Optional<T> and C# has Nullable<T>.
That's not important in practice. What's important is that null infects all reference types in Java and C#, whether you asked for it or not. Just today I've spent a half an hour looking for a null dereference error - which the type system should've warned me of but didn't. The sad fact is, neither Java nor C# are fixable, because no new feature will fix the broken, null-unsafe code already written. All because of this billion dollar mistake that should've had no place in a GCed language but was blindly copied over from C++. The sooner Java gives way to Kotlin and completely fades into obscurity, the better.
(comment deleted)
> was blindly copied over from C++.

From C! AFAIK C++'s references can not be null!

The big difference is that nullability is the standard in Java and C# while in Swift it's the other way round, it's non-nullable unless you clearly indicate it's not.
Ive noticed other languages with this feature, they call them optional types, you can pass in a null or a type value.

/s slightly, maybe more snarky, but seriously, optional types are way overhyped. Null objects are far more useful than optionals. Optionals tend to leak all over the show.

I don't understand. In this situation an optional type is a reasonable solution to the problem: None => N/A, Some(0%) => 0%.
Nulls leak exactly as badly as optionals, except it's additionally impossible to tell where nulls have leaked to.
Actually, nulls leak worse than optionals, because it's impossible to tell where nulls have leaked to (so you end up forgetting to filter out a null at a interface where you would have decided to filter out a maybe).
I upvoted you because I think it's unfair that you are being downvoted while contributing to the discussion, that said you are completely wrong. Without Optional/Maybe nulls leak everywhere except you have no idea where they are. This leads programmers to engage in defensive programming for all references since there is nothing to help them understand what they should expect to be null in some cases. Optional/Maybe encourages good data modelling by making values that might be missing very clear. It encourages a style of programming that eliminates these issue via better modelling or by rejecting bad data earlier.

For example consider a network response that in a JS app might be passed around as an opaque object. In a language like Swift this is infeasible, you have to transition from the network serialized version to concrete models at the network layer. If you encounter mismatches in the expectation of what is optional you have to handle it here at the edge of the program. This enables massively safer code and makes reasoning about your program much simpler if anything.

As a concerete example say an API response tells you if a user is verified(for some domain specific definition of "verified"), additionally you want to know when they were verified maybe to give them a hint when they need to do it again. You coudl model this as:

    {
      "verified": false,
      "verified_at": null
    }
before they have verified and

    {
      "verified": true,
      "verified_at": "2020-02-24T15:31:12Z"
    }
after they have verified.

In Swift this would then be modelled as

    struct Users {
        let verified: Bool
        let verifiedAt: Date?
    }
However this introduces an Optional which complicates code that needs to deal with users who are verified or not. Instead you could model the same thing and avoid this issue as:

    {
      "verification_status": {
        "status": "unverified"
      }
    }
for the unverified case and

    {
      "verification_status": {
        "status": "verified",
        "at": "2020-02-24T15:31:12Z"
      }
    }
for the verified case. In Swift this can be nicely modelled without any Optionals using an enum

    enum VerificationStatus {
        case unverified
        case verified(Date)
    }

    struct User {
        let verificationStatus: VerificationStatus
    }
Better optional value visibility leads to better data modelling.
The point of optionals is that they leak, and leak visibly. You cannot use an optional where a non-optional is expected, whereas in C nothing will stop you from writing `x->y` when x is, or might be, NULL. Thus the optional (and non-optional) types turn null pointer errors into compiler errors.

If you wish to reduce "leakiness", you can dispatch on the null-ness of the variable; Swift lets you write `if let x = x { /* x is non-optional in here * / }`, which binds a non-optional value inside the braces, and `guard let x = x else { return } /* x is non-optional now * /` which binds a non-optional value after the braces. So as soon as you consume an optional value, you are free to "un-leak" it and extract its value (if it has one) immediately.

> You can create a type that can encompass all kinds of edge cases and information about them.

This is not unique to functional programming languages.

Haskell was definitely a trailblazer for this, but plenty of conventional languages support this now with relative ease.

Any language with a type system at all can create a Maybe type. The difference is that failing to handle the None case is a runtime exception in most languages where in Haskel/Rust it won’t compile.

I don’t know of any mainstream language that has bolted on that level of verification.

Scala3 and Kotlin.
You mean Scala 2. And likely even Scala 1 had this. (Scala 3 is still in development).

But Kotlin? Since when does Kotlin have pattern matching with exhaustive checks? Also it does not have an `Maybe a` / `Option[A]` type out-of-the box.

Kotlin distinguishes between nullable and not-null references, which is an alternative to the maybe type. It serves roughly the same purpose, to replace runtime errors with compile-time verification that you haven't missed a corner case.
that prevents you from assigning null to a non-nullable reference. does it also error if you dereference a nullable reference without first checking if it's null?
It's not an alternative for a Maybe Monad. It's an orthogonal concept.

Whether a type is nullable or not is a property of the type. This creates more or less two mirror universes of types, and you can't mix even "the same" types from that incompatible universes.

An Option type is on the other hand side a distinguished parametric type on its own (I wrote `Maybe a` / `Option[A]` for a reason). It can "wrap" arbitrary other types without "infecting" those with the nullability property. Being a proper parametric type makes `Maybe` / `Option` also "just a type constructor". Therefore it can be composed with other types and type constructors according to common rules. `Maybe` / `Option` doesn't require any special rules in the language (which would make the language more difficult to learn and use)!

Also `null` is very different form `Empty` / `None`: A null value can still "sneak in" through arbitrary references and cause run-time errors if one has to interoperate with a "null unsafe" language or system (and that's more or less "the whole universe" out there). A `None` can not "sneak in" as it can be only assigned to something of `Option` type. Sure this doesn't solve the "null problem" as such but one can at least distinguish the cases where the model value is really optional, and the cases where one needs to deal with `null`s coming (potentially) form the outside world. In the one case one will use the Option type throughout the rest of the program, in the other case one will preform some null-check as early as possible and "unwrap" the underlying none-null value to work with it directly thereafter.

Kotlin will do exhaustive checks in a `when` block when it is an expression. You can add a `.let { }` at the end of the when statement if you want to enforce this and not really return anything.

    sealed class Communication
    data class Email(val emailAddress: String) : Communication()
    data class Shouting(val preferredName: String) : Communication()
    object HiveMind : Communication()

    fun exhaustiveWhen(comm: Communication) {
        // not exhaustively checked (will compile)
        when (comm) {
            is Email -> println("sending email to ${comm.emailAddress}")
        }

        // *is* exhaustively checked (error at compile time)
        //  must add HiveMind or 'else' branch
        val result = when (comm) {
            is Email -> println("sending email to ${comm.emailAddress}")
            is Shouting -> println("HELLO, ${comm.preferredName}!")
        }

        // *is* exhaustively checked
        when (comm) {
            is Email -> println("sending email to ${comm.emailAddress}")
            is Shouting -> println("HELLO, ${comm.preferredName}!")
        }.let {  }
    }
C.

  typedef struct{int filled; union{void* data;} val;} maybe;
How does that fail at compile time if you try to read a value when there is none?
Some of Dawson Engler’s work helps solve this problem, starting around [1]. In that, he shows us how to write custom compiler rules in a high-level language. These rules are sufficient to enforce that maybe type. In another paper, Engler (or one of his several talented students, can’t remember now) shows us how to automatically infer many of these rules.

[1] https://web.stanford.edu/~engler/mc-osdi.pdf

If you try to use it val directly, it fails. You must unpack it first.

To do something like x.get() in Scala when x is empty is a runtime error, not a compile-time error.

(comment deleted)
> This is of course a textbook case for Haskells Maybe concept. You can have a type that is a Maybe Int, and that means it have a value or it can be Nothing. Thus, you encompass the edge case of not having the value available

If you encompass it this way. Literally nothing stops you from encoding this behavior in Haskell. Because the behavior you described is *if revenue is equal to zero, display it as N/A".

No magic monad will prevent this.

No, but it's the difference between implicit and explicit stupidity. All of us do implicit stupidity all of the time - it's human nature (or more accurately a side effect of working in a difficult field). When transformed into a choice between explicit stupidity, or doing things the right way, we instead usually choose to to do things the right way -- or at minimum do it the wrong way to test something out then change it to the write way. Being explicit is what makes that change likely to happen, rather than just be ignored.
No. No matter how much you dress it with words, technology is not magic. Monads are not magic. Haskell is not magic.

If your data and/or problem is modeled as "if 0 then N/A" no amount of "human nature implicitly explicitly transformed" will help you.

Dealing with non-existent data as opposed to "value = 0" is just as natural in Java, C++ or Javascript as it is in Haskell. And yet, that didn't help.

Conversely, it's also one of the biggest pain-points when throwing together a quick prototype.
Then here's where typescript shines. To be able to move fluidly between no static checks of JavaScript and strong-ish type safety of typescript without too much work.

Then again, it's not always without hiccups.

That is where I think Flow really shines. Seems to be less hiccups, but that is just my anecdotal experience.
Nah, you can totally use things like holes or undefined (in haskell), unimplemented!() in rust, etc.

EDIT: and, as neighbour says, any in typescript ! But typescript, with its progressive typing, is in a league of its own.

They say that, and then turn around and show an example using IEEE-754 Doubles. You've got two infinities and two "not a number" values. What your type system thinks is a number isn't even necessarily a number. Maybe.

Better be sure to call the "isNaN()" and "isInfinite()" methods all over the place, because those are the worst edge cases of all and your language provides no help in detecting or avoiding them. It's like having 3 or 4 other None values, where they don't even use the same system as your standard Option type.

If I had a nickel for every time I saw "NaN" appear on a webpage or a web browser console log ... We throw up our hands and say, yeah, well, JS is terrible -- but no static functional language I know of is any better.

Haskell is so much better, and you would learn that with about 5 minutes learning about the number types
Haskell still doesn't track NaN in the type system.
You're right, it does, and it exists at the value level if you are using floating point. But in Haskell you have, for example, Rational numbers to avoid many such problems.

I was, however, surprised to not see any kind of a (significant) `safeDiv` function on hackage, and no NonZero newtype (outside of quickcheck). But, for this kind of thing its so easy to roll yourself, I am guessing this is what people do (and I have seen).

> But in Haskell you have, for example, Rational numbers to avoid many such problems.

Hey, it's for sure nice to have a rich numeric stack (see also some lisps), but in numeric work "use something else" is very rarely a practical answer to issues with IEEE-754.

Sure, it is, but the thing is that a hugely significant amount of the time I've seen those NaNs happen is because of a lack of understanding/care on the part of the devs, and not because they are doing scientific computing.

There seem to be a large disconnect here. A big chunk of people don't need the speed that FP math brings, what they need is exactness. And, for most languages with a C-based lineage, actually correct floating point math support seems to fit in the "someday/maybe" categories.

Yes, rational numbers are the better default choice. You should get floats only when you ask for them.
Unfortunately, for many languages, its the only real choice (without introducing some new significant dependency!)
Haskell's intro page starts with the infamous minimalist quicksort, which mishandles NaN!

https://wiki.haskell.org/Introduction#Quicksort_in_Haskell

How should it handle NaN? It's not clear where an incomparable value should go when sorting.
Why is NaN even a concept in modern languages? We don’t have a special “not a string” that all string functions return when they have no useful value to return.
Because it's part of IEEE 754.
So? There’s lots of old standards that don’t fit well with modern programming languages, so we abandoned them, or only use the parts that still make sense.
IEEE 754 is one of the standards like two's complement arithmetic which are widely used and have a number of clear advantages, so we continue to use them.
Because what is the CPU supposed to do if you divide zero by zero or take the square root of -1?

It's actually not exactly a decision being made by the language designers. The IEEE standard governs the floating point logic embedded in ALU hardware. Programming languages tend to expose floating point math in its rawest form for performance reasons.

I mean, this is a whole different thing, and I legitimately hate that "algorithm".

But you have to understand that the whole situation is a bit complicated. The thing though is that Haskell gives you possible ways to handle those complications.

I tried it and indeed, the results involving NaN are... interesting

     quicksort :: Ord a => [a] -> [a] 
  .. quicksort []     = [] 
  .. quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater) 
  ..     where 
  ..         lesser  = filter (< p) xs 
  ..         greater = filter (>= p) xs
     quicksort([sqrt(i)|i<-[0 .. 3]])
  => [0.0,1.0,1.4142135623730951,1.7320508075688772]
     quicksort([sqrt(i)|i<-[-3 .. 3]])
  => [NaN]
     quicksort([sqrt(i)|i<-[3 .. -3]])
  => []
Now, just for fun, let's reverse the condition...

     quicksort :: Ord a => [a] -> [a] 
  .. quicksort []     = [] 
  .. quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater) 
  ..     where 
  ..         lesser  = filter (\x -> not(x < p)) xs 
  ..         greater = filter (\x -> not(x >= p)) xs
     quicksort([sqrt(i)|i<-[0 .. 3]])
  => [1.7320508075688772,1.4142135623730951,1.0,0.0]
     quicksort([sqrt(i)|i<-[-3 .. 3]])
  => [1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0,NaN,1.7320508075688772,1.4142135623730951,1.0,0.0]
  
The last example will explode with larger values, like [-100..100].

Note, it is actually the first time I tried Haskell. And no, I don't think the language is broken because of that. In fact, it is exactly the behavior I expected.

You can always use a fixed-precision numeric type.

You might be thinking "what about libraries that require floating-point", and that's a good concern. Luckily in Haskell there's the Num typeclass, so your libraries can let users use whatever fractional number type they'd like.

Fun fact, Num basically requires inf and nan unless you're willing to tolerate either terrible overflow (non-)handling (eg Int,SatInt) or memory blowup (eg Integer,BaireReal).

  # where R is some r such that Num r
  let huge = <the largest (finite) value in R>
  let inf = huge * huge  # can't be finite unless you
  # truncate or saturate or something
  let nan = inf - inf  # can't be either of
  # infinite (infinitely < +inf and > -inf), or
  # finite (catastrophic rounding errors)
Num doesn't allow inf or nan to be Maybe R, so you're stuck filtering out in-band errors from a bare R.
There are some research languages that deal with this.
For me the value of typed functional programming has been it's ability to encode business logic in to the type system and thus validate some of my logic at compile time.
What's specifically functional about this advantage. Can't you encode the buisness logic into types in non-functional languages as well?
The type systems that are common in functional languages are usually more flexible and useful than what you find in most other common languages (in particular, usually more useful than the type systems of common object oriented languages).
The word “type“ has a fairly different meaning in functional programming compared to object oriented programming. At a very shallow level, they have some similarities. But the way they are used is vastly different between the two paradigms.
You can encode it, but it is not enforced. For instance, I can refer to every instance/argument when I use a Person object as `person` or whatever name of the class is, but when you have many developers working on a large system, these things tend to slowly spiral unless you establish strict guidelines early on and enforce through code reviews.

Or you can use types where it's enforced by the compiler.

On practice, no you can't. At least today.

But it may be a very interesting question for a researcher.

Something about putting business logic in a type system irks me more than someone building another compiler or database in Haskell.

Business logic relies so much on run-time values that (IMO) using data and predicates is a better fit.

Isn't that what also makes it hard to use without relying on the mathematical laws? While the math part is hard, it is what provides the strong guarantees in the form of laws.
I came to a similar insight by the corollary: a sound type system makes the edge cases obvious. When we make the type of a partial function total by introducing an optional value as a result there's an opportunity to develop an intuition that this is going to make my code harder to reason about. Assuming I fix all the warnings about incomplete pattern matches I know that users of this code will have to handle all of the edge cases I'm introducing. Therefore I might take another approach and introduce a type as my pre-condition: the type of non-empty lists. You can't even call my function unless I can be assured there's at least one element in the input. I haven't even had to look at the implementation to reason about my code yet.

    head :: [a] -> a
    -- versus
    saferHead :: [a] -> Maybe a
    -- versus
    safeHead :: NonEmpty a -> a

The first function is partial because we'll get an exception if we give it the empty list.

The second function is total but annoying to use because we're telling all the code that uses the result to check for two cases (it's an error not to).

The last one forces the responsibility on the caller to provide a non-empty input.

That was probably one of the more frustrating aspects about learning to program Haskell as someone who has been programming for more than fifteen years when I started. It revealed to me in stunning detail all of the edge cases that almost every other language I've used actively hides from me.

It doesn't absolve you of having to think about edge cases: even Haskell throws run-time exceptions. However it does give you tools to think about many of those edge cases up-front and in a direct way.

Update Added a trivial example to demonstrate "partial," etc.

100% that's a big part of it (although there are others).

Although if someone doesn't know haskell, then the above notation for functions getting the first element of a list is probably not of much value to them.

Clearer would be:

  char  head(List<char> x);
  char? saferHead(List<char> x);
  char  safeHead(NonEmptyList<char> x);
There's also another option in the middle which is a version that takes a value to return when the list is empty

    headOr :: a -> [a] -> a
which is similar to the Maybe solution but forces the caller to discharge the Maybe immediately rather than letting them potentially clutter up the rest of their code by proliferating the Maybe.
I wonder, aren't there problematic edge cases that depend on the data? For example, if you are computing a plane from a given set of points, there may be an edge case where the points are collinear. Assuming you always need to produce a valid plane, this would require some numerical analysis and regularization. Or is the answer to define some new type system that captures the non-collinearity? If so, it seems like the costs of enforcing that could overwhelm things...
The answer doesn’t necessarily have to be statically known, just ensure that the edge case is handled. computePlane() can return Maybe<Plane>, where Nothing is returned in the colinear case.

This seems not that different from throwing an exception, except the caller can’t accidentally forget to deal with the colinear case, there must be code to handle the Nothing case.

But perhaps the question you’re asking is “but how can the writer of toPlane() know THEY did the right thing”. There’s of course no solution to logic errors (function subtract(a,b) { return a + b } will get by most type systems, short of having the type system re-encode the function itself, at which point it’s just correctness through redundancy — unless both the type AND function are wrong!).

However, you COULD protect against future people breaking your implementation by doing something analogous time weak_ptr’s implementation and flipping the Maybe to the input vs the output). So for example, you could make a type NonColinearSet, and have the “constructor” take a PointSet, but return a Maybe<NonColinearSet>, so asNonColinearSet takes a PointSet and returns Nothing if the PointSet is colinear, or NonColinearSet if it isn’t. Now you computePlane() function can take a NonColinearSet and return Plane (not Maybe<Plane>) since it “knows” the input must be valid, so it can ignore edge cases internally, at the cost of the caller having to deal with toNonColinearSet returning Nothing before calling it, since you won’t be allowed to transparent pass the result of asNonColinearSet(pointSet) to computePlane since the types Wong match (Maybe<NonColinearSet> vs. NonColinearSet).

    maybeNonColinear = asNonColinearSet(pointSet);

    case Nothing: alertUserBadDataOrWhatever();
    case Just<NonColinearSet>: return toPlane(maybeNonColinear.value)
Notice in both cases, ultimately the caller must do something in the edge case, which is what you want.
OTOH it's nice to be able to do:

    try {
        // 500 lines
    catch (e) {
        return 4;
    }
Often, even functional languages have something similar, whether it's call/cc or something more cutting-edge like algebraic effects. All of these essentially augment the execution environment in various interesting ways, and are enhanced by other features of functional languages like immutability.

(More pithily: try/catch is not inherently non-functional.)

As a mere mortal reading the code, I would rather not have to reason about try blocks longer than a couple of lines.