2017-09-25

Golang’s panic unreachable is unnecessary

The only reason for multiple return parameters is error handling

I remember when I was learning Go and typing panic("unreachable") for the first time and not understanding why the type system didn’t know what I knew, that I have covered all the cases and unreachable is unnecessary.

Years later I learned that my instinct was right, lots of type systems are able to know that panic unreachable is unnecessary. To do this we need to add one feature called Sum types or as Rust calls them Enums.

What are sum types

A sum type has many names including: tagged union, oneof, sealed trait or enum. It is a way to represent a disjoint union of types in a single type. Say for example we have the sum type (int | bool). This sum type will be able to represent all possible int values PLUS all possible bool values. In contrast a product type, for example (int, bool) or struct {i int; b bool} will allow you to represent 2 times all the possible int values. All possible int values if the bool is true and all possible int values when the bool is false.

The advantage of sum types are that the compiler is able to tell whether you handle all the disjoint cases. This means that a type switch can be less error prone and allow the compiler to make sure that you handle all cases.

Sum types is how Elm can eliminate all possible runtime errors:

Richard Feldman explaining how sum types help Elm avoid runtime exceptions

I had the same experience with Elm, except my only two run time exceptions ever were both caused by division by zero.

But wait, I thought the most common runtime exception was a null pointer exception. Yes, technically a pointer is also a sum type. It can have all the values of the pointer type PLUS one for null.

Elm, Haskell, etc. has a Maybe type, which is used to represent these types of values. The same as a pointer, it can represent Just the value or Nothing for nil:

data Maybe a = Just a | Nothing

This enables the compiler to enforce you to do a "nil" check and avoid nil pointer exceptions. These days even Java has something similar called Optional.

Some History

Sum types is not a new language feature, but a very old one. Algol 68 first introduced united modes (sum types) in the 1960s. This has been adopted by Pascal, Ada, Modula-2 as variant records. Later Haskell, ML and now Scala, Elm, Rust, Swift, F#, Protobufs and even C++ have adopted sum types. Even Java has announced plans to also add sum types. I hope that one day Go will also adopt sum types.

Parody of Stan by Eminem complaining about sum types not being available in most programming languages

Some use cases for sum types

Firstly, we need sum types to return a value OR an error, instead of a value AND an error from functions, but we also need sum types to make panic unreachable unnecessary.

In Katydid’s Abstract Syntax Tree (a validation language I was working on) each Pattern can be one of several Patterns:

type Pattern struct {
	Empty      *Empty
	TreeNode   *TreeNode
	LeafNode   *LeafNode
	Concat     *Concat
	Or         *Or
	And        *And
  ...
}

Here I simulate a sum type with a struct which has several fields (a product type) where only one field should be non nil. This is not ideal in terms of type safety. For instance when writing a function that processes a pattern, I have to do a nil check for each field:

func Nullable(refs ast.RefLookup, p *ast.Pattern) bool {
    if p.Empty != nil {
        return true
    } else if p.TreeNode != nil {
        return false
    } else if p.LeafNode != nil {
        return false
    } else if p.Concat != nil {
        return Nullable(refs, p.Concat.GetLeftPattern()) &&
            Nullable(refs, p.Concat.GetRightPattern())
    } else if p.Or != nil {
        return Nullable(refs, p.Or.GetLeftPattern()) ||
            Nullable(refs, p.Or.GetRightPattern())
    } else if p.And != nil {
        return Nullable(refs, p.And.GetLeftPattern()) &&
            Nullable(refs, p.And.GetRightPattern())
    ...

This is fine, but when I add a new pattern the compiler is not going to tell me that I forgot to update one of these functions, like a sum type would have. This means that not only does a sum type make panic unreachable unnecessary, but it also helps you to remember to handle all cases. I opted for another implementation, using a type switch, but it only works because each field is of a unique type and I still have the runtime type check problem.

The use of an interface as a way to simulate a sum type is very popular and can also be found in a Protocol buffers library for Go where they have to implement oneof.

Given this proto message:

message MyMessage {
  oneof BoolOrInt {
    bool BoolValue = 1;
    int32 Int32Value = 2;
  }
}

The following go code is generated:

type MyMessage struct {
	BoolOrInt isMyMessage_BoolOrInt
}
type isMyMessage_BoolOrInt interface {
	isMyMessage_BoolOrInt()
}
type MyMessage_BoolValue struct {
	BoolValue bool
}
type MyMessage_Int32Value struct {
	Int32Value int32
}
func (*MyMessage_BoolValue) isMyMessage_BoolOrInt()        {}
func (*MyMessage_Int32Value) isMyMessage_BoolOrInt()       {}

This implementation tries really hard to work around the fact that Go does not have sum types. In an alternative Protocol buffer library for Go, developers cannot even agree on what they would prefer to have as a oneof implementation in Go, because Go does not lend itself to sum types.

A need for sum types can also be found in the go/ast library, where ast.Spec is documented to be one of the following types:

This is something that could have been enforced by the compiler, instead of relying on documentation.

Another example is the ast.Walk function, which ends with a classic runtime error, where a compile error would have been more appropriate:

default:
  panic(fmt.Sprintf("ast.Walk: unexpected node type %T", n))
}

The only reason for multiple return parameters is error handling

If you want to add a feature to a language and keep the language small you also want to remove a feature, so I will also explain why I think multiple return parameters are probably a feature we could have gotten away without.

I wrote a little tool which does some analysis of Go source code. Thank you to the go/types library. The tool simply counts the number of times multiple return parameters are used. I ran this tool over the standard library (back in 2017) and these were the results:

$ goanalysis std
functions with N return arguments:
-----------------------------
returns | number of functions
      0 | 8693
      1 | 7743
      2 | 1975
      3 | 205
      4 | 25
      5 | 4
      6 | 1
      7 | 1
-----------------------------
total number of functions: 18647
total number of functions with multiple return parameters: 2211
number of functions with 2 return arguments, where the second argument is an error: 1562
percentage of functions where multiple return parameters are really what we want: 3.480453

These results show us two things:

Only 3.5% of functions actually use multiple return parameters:

bar graph

The tool does not count functions that return a value AND an error as a proper use of multiple return parameters. This is because I think we should rather have sum types for this use case. I believe that most of the time we return:

This means that we would rather return a value OR an error, than a value AND an error. A sum type instead of a product type. This could be achieved with | character instead of a , for example:

func Atoi(s string) (int | error)

Tuples are not first class citizens

In the go/types library I found that multiple return parameters are actually called Tuples, but these Tuples are not first class citizens. For example nested tuples are currently not supported:

func() ((string, error), error)

I ran into this while building monadic error handling in goderive. I worked around this by using a function:

func() (func() (string, error), error)

This is not ideal, since a function implies some computation. We currently also cannot directly pass multiple return parameters to a function:

func f() (int, error) {
    return 1, nil
}

func g(i int, err error, j int) int {
    if err != nil {
        return 0
    }
    return i + j
}

func main() {
    i := g(f(), 1)
    println(i)
}

This gives us the following error:

prog.go:15:11: not enough arguments in call to g
prog.go:15:13: multiple-value f() in single-value context

Play with it on the Go Playground

Conclusion

I have shown that multiple return parameters are a feature Go could have lived without, since:

I have also demonstrated several use cases for sum types:

There are MANY more use cases, including avoiding null pointer exceptions.

I know Go will not get rid of multiple return parameters, because this will break backwards compatibility, but I do hope there is room to add sum types in the future, especially now that Go has added generics.

Thank you

Referenced