![](/style/images/good.png)
![](/style/images/bad.png)
Down the Golang nil Rabbit Hole
source link: https://blog.urth.org/2021/03/27/down-the-golang-nil-rabbit-hole/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Down the Golang nil Rabbit Hole
I’m not sure I have another Rust & Postgres blog post in me right now, so let’s learn something about Go instead.
Recently I decided I wanted to add a --unique
flag to
omegasort. Wait, what’s
omegasort?
It’s a text file sorting tool that supports lots of different sorting methods. For example, in addition a standard text sort, it can sort numbered lines, date-prefixed lines, paths (including Windows paths with and without drive letters), IP addresses, and IP networks. It also supports Unicode locales, reverse sorting, and locale-aware case insensitive sorting.
I use it together with precious
to sort things like .gitignore
files, spellchecker allowlists, and things of
that nature.
I realized that I really wanted a --unique
flag for all of this. While I
could just pipe its output to uniq
on a *nix system, this doesn’t work so
well on Windows. Plus with tools like precious it’s easier if I can use one
binary for a given task. If I want to pipe things I have to put that in a
shell script that precious calls.
But my rabbit hole experience didn’t happen with omegasort directly. Instead, it happened when I tried to add some integration tests.
While writing those integration tests, I was using
github.com/houseabsolute/detest
. This
is a Golang package I created that offers a test assertion interface inspired
by Test2-Suite
in Perl.
For reference, here’s a Test2-Suite
example:
use Test2::Suite;
object Subtest => sub {
call name => 'TestsFor::Basic';
call pass => T();
call subevents => array {
object Plan => sub {
call max => 4;
call trace => object {
call package => 'Test::Class::Moose::Role::Executor';
call subname => 'Test::Class::Moose::Util::context_do';
};
};
...
}
I think this is pretty self-explanatory, except for T()
, which means “true”.
And here’s something like that in Go with detest
:
import (
"testing"
"github.com/houseabsolute/detest/pkg/detest"
)
func TestSomething(t *testing.T) {
d := detest.New(t)
d.Is(
someStruct,
d.Struct(func(st *detest.StructTester) {
st.Field("size", 43)
st.Field("Name", "Douglas")
d.Map(func(mt *detest.MapTester) {
mt.Key("foo", d.Slice(func(st *detest.SliceTester) {
st.Idx(0, d.Map(func(mt *detest.MapTester) {
mt.Key("bar", d.Slice(func(st *detest.SliceTester) {
st.Idx(1, "buz")
st.Idx(2, "not quux")
}))
}))
st.Idx(1, d.Map(func(mt *detest.MapTester) {
mt.Key("nosuchkey", d.Slice(func(st *detest.SliceTester) {
st.Idx(1, "buz")
st.Idx(2, "not quux")
}))
}))
}))
})
}),
)
}
It’s not as nice as the Perl version because it gets quite verbose, but this was the closest I could come. Go’s type system, combined with a lack of syntactic flexibility, means a whole lot of func calls, braces, and parens.
Under the hood, this is implemented with a metric fork ton of runtime
reflection using the stdlib’s reflect
package. I don’t love this, but absent generics,
there’s no other way to implement this sort of API except with code
generation. And that codegen would have to be fed by a sort-of-Go language
that was translated to real Go, which seems like a terrible idea.
Getting to the Darn Point
So while I was writing those omegasort integration tests using detest, I managed to find a whole lot of bugs in detest.
But the title says nil
and I haven’t mentioned those yet.
So here’s a fun fact, Go has multiple “types” of nil
. Specifically, there
are both typed and untyped nil
variables. This surprised me at first, but it
makes sense when you think about it.
Let’s take this code1
:package main
import (
"fmt"
"reflect"
)
func main() {
v1 := reflect.ValueOf(nil)
var uninit []int
v2 := reflect.ValueOf(uninit)
logValue("nil", v1)
logValue("[]int", v2)
fmt.Printf("[]int == nil? %v\n", uninit == nil)
}
func logValue(what string, v reflect.Value) {
fmt.Printf("%s is valid? %v\n", what, v.IsValid())
if v.IsValid() {
fmt.Printf("%s is nil? %v\n", what, v.IsNil())
fmt.Printf("%s type = %v\n", what, v.Type())
}
}
This prints out the following:
nil is valid? false
[]int is valid? true
[]int is nil? true
[]int type = []int
[]int == nil? true
So a bare nil
and a variable that has a type but no value are equal, but if
you try to get a reflect.Value
for nil
, it’s not valid. If you try to call
other methods like v.IsNil()
or v.Type()
on an invalid2
reflect.Value
,
you will get a panic.I encountered this when trying to test that an error
returned by a func call
was nil
.
This led to a flurry of detest
releases as I realized how
many parts of the detest
code this impacted. In most places where it uses
reflect
, I have to guard against a bare nil
being passed in.
But wait, it gets even more confusing. Sometimes the Go compiler will turn an
untyped nil
into a typed nil
. Here’s an example3
package main
import (
"fmt"
"reflect"
)
func main() {
takesSlice("nil", nil)
var uninit []int
takesSlice("[]int", uninit)
}
func takesSlice(what string, s []int) {
logValue(what, reflect.ValueOf(s))
}
func logValue(what string, v reflect.Value) {
fmt.Printf("%s is valid? %v\n", what, v.IsValid())
if v.IsValid() {
fmt.Printf("%s is nil? %v\n", what, v.IsNil())
fmt.Printf("%s type = %v\n", what, v.Type())
}
}
And when we run it we get this:
nil is valid? true
nil is nil? true
nil type = []int
[]int is valid? true
[]int is nil? true
[]int type = []int
So when I pass a bare nil
to takesSlice
, it gets typed as whatever type
the function’s signature says it should be.
But wait, it gets even more confusing yet again! Sometimes the Go compiler
won’t turn an untyped nil
into a typed nil
. Here’s an example4
package main
import (
"fmt"
"reflect"
)
func main() {
takesError("nil", nil)
var uninit error
takesError("error", uninit)
}
func takesError(what string, e error) {
logValue(what, reflect.ValueOf(e))
}
func logValue(what string, v reflect.Value) {
fmt.Printf("%s is valid? %v\n", what, v.IsValid())
if v.IsValid() {
fmt.Printf("%s is nil? %v\n", what, v.IsNil())
fmt.Printf("%s type = %v\n", what, v.Type())
}
}
If the type of the argument in the function signature is any type of
interface, including interface{}
, then the underlying value is still untyped
and not valid. This … sort of makes sense? I think the way this works is
that anything typed as an interface also has a real underlying type. So an
error
can be an errors.errorString
or an exec.ExitError
or a
mypackage.DogError
. But if we pass a bare nil
or an uninitialized
variable, there’s no underlying type.
This came up with detest when I wanted to test that I didn’t get an error from a call.
err := doThing()
d.Is(err, nil, "no error from doing a thing")
Under the hood, the signature for d.Is()
uses interface{}
for the two
arguments being compared. So bare nil
as the second argument will never be
valid. And the first argument might be valid or it might not be. If
doThing()
’s return type is just error
and it returns a nil
, then the
value in err
has no type.
All of this led to a fair bit more code in the detest
guts to handle
this. For example, just because two variables don’t have the same type doesn’t
mean they’re not equal (from Go’s perspective). A bare nil
and an
uninitialized slice are equal when compared with ==
, which is what d.Is()
emulates using reflect
.
So there’s quite a few cases around one or both arguments being invalid that
need handling. And there are MANY other methods with the same issues to
consider, including things like d.Map()
and d.Struct
, all of which should
handle an invalid value properly.
What Does This Look Like in Other Languages?
Well, I don’t know that many other languages. In Perl this isn’t really a
thing, because it has a pretty minimal type system. Perl’s undef
can be
coerced to lots of things, although under
strict trying to use an undef
in certain
ways is an error, like writing this:
my $x;
say @{$x};
This will blow up with Can't use an undefined value as an ARRAY reference ...
at line 2.
Rust (at least safe Rust5
I know very little about unsafe Rust which is why I’m hedging.
) doesn’t have any notion ofnil
or undefined
values. Instead, you have the
Option<T>
type,
which always has a type. For example6:pub fn main() {
let a: Option<String> = None;
let b: Option<i32> = None;
println!("a == b? {}", a == b);
}
This just won’t compile. While both a
and b
are None
, they’re not the
same type of None
so you can’t just compare them with ==
. The compiler
says:
error[E0308]: mismatched types
--> src/main.rs:4:33
|
4 | println!("a == b? {}", a == b);
| ^ expected struct `String`, found `i32`
|
= note: expected enum `Option<String>`
found enum `Option<i32>`
By the way, aren’t these Rust compiler errors nice? The only other language I’ve seen with this type of extremely detailed compiler errors is Raku.
In Summary
It’s tempting to pick on Go and complain about it. I certainly do that a lot
at work. But to be fair, this really isn’t an issue for most Go code. It’s
only because I’m trying to do weird stuff with reflect
that I’m learning
about this internal weirdness. In day to day Go code, the compiler’s handling
of various types of nil
“just works” the way you’d expect it to. And being
able to use a bare nil
is quite handy.
But I still prefer how Rust does it, using a parameterized Option<T>
type. That way I can easily check if something is None
without any special
cases. Everything is using the same type system, though that type system is
much more complex than Go’s.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK