3D Printed Gopher

Before we discuss less important issues, such as software engineering, it’s important to note that I now own a custom 3D printed Gopher. The good people at the Golang Manchester meetup made it for me, based on my hobby (rock climbing). It was a gift for speaking at their meetup.

Boasting a pink climbing rope and three-pronged grappling hook, the Gopher is fully equipped to come climbing with me in the Peak District.

Now that I’ve unveiled the best piece of conference swag ever produced, on to more mundane matters: memory management in Go.

Slides

Here is a link to the slides of The Strength of the Weak Package. The talk was given at Golang Manchester, and at the Oxford and Birmingham Gophers meetups in April ‘25.

Go Memory Management

Ask a Java, PHP or C# developer to tell you about their language, and they will probably say 5,000 words before they mention the garbage collector. The same cannot be said of Gophers.

But while Go’s memory management is a distinguishing feature within its space (i.e. compared to C++ and Rust), not much has changed in it for a while. This was the case until Go 1.24 introduced the new weak package.

What is a Weak Pointer

‘Strong’ (normal) pointers stop the garbage collector from being able to reclaim memory. Weak pointers do not. Whether you’re building a cache with reclaimable memory, or a pub/sub design pattern where subscribers don’t prevent cleanup of publishers, weak pointers allow you to have variables in the system which could be garbage collected if necessary.

Nothing to Say

If a weak pointer is created, and then garbage collection of the object occurs, you don’t get a strong pointer when you ask for one. You get nil. Checking for nil should always be done when resolving a weak pointer to a strong pointer:

objInMemory := 42
ptr := &objInMemory
wp := weak.Make(ptr)

ptr := nil
runtime.GC() // force garbage collection for demonstration purposes

if val := wp.Value(); val != nil {
    fmt.Println("Thing still exists in memory")
} else {
    fmt.Println("Weak pointer is now nil") // This line is printed here
}

Features of Go’s Garbage Collector

Stack and Heap

Local variables in Go functions and goroutines are stored in an in-memory stack. It is fast, and grows/shrinks as needed.

The heap contains global (or package-level) variables, or anything that outlives the function that created it, e.g. returned pointers, variables output to a channel etc.

Broadly, if something in the heap is referred to somewhere in the stack, then it is still needed. If not, the variable is unreachable in user-land code, and the memory should be deallocated.

Mark and Sweep

Go uses a ‘mark and sweep’ garbage collector. This means that objects in memory are marked as still in use, or not, and then unneeded objects are cleared out of memory in the ‘sweep’ phase.

Tricolour Marking

Go is using three ‘colours’ to mark its objects: black, white and grey.

Tricolour mark-and-sweep diagram

Variables marked in black are definitely needed. Those still white at the end of the ‘mark’ phase of the mark-and-sweep garbage collection should be cleaned up. Grey objects in the heap are still being used, but may have ‘children’ which have not yet been scanned. A global struct, for example, should be grey if the individual properties are still waiting to be marked.

Non-Compacting Garbage Collection

In Go’s garbage collection, sweeping is the last phase. After the sweep, memory still allocated will be discontinuous. There will be gaps of unallocated memory. In some languages, a garbage compacting phase would then defrag memory allocation so there are no gaps. This is slow, but reduces memory usage. Go doesn’t do compacting garbage collection, so pointers you have in your code are still valid and do not need to be changed: the memory they refer to is still at the same address.

A gopher driving a bin lorry

Go’s garbage collector distinguishes it from other languages such as C++

Non-Generational Garbage Collection

Some garbage collectors have a ‘generational’ property, where the lindy effect is applied: the longer something has been in memory, the less likely it is that it is suddenly no longer needed. This means that older objects aren’t subject to mark-and-sweep. Go’s garbage collector is simpler here: everything gets assessed every time, regardless of age.

Weak Pointer Example

Caching with Weak Pointers

If building a cache in Go using the new weak pointers feature, there are three cases to consider:

Cache hit: we should return the strong pointer, or perhaps the object it points to

Cache miss: return nil or error, as we cannot provide a cached value

Key exists, value expired: we may have the requested key in our map, but get back nil from weak.Value(). The memory has been cleaned up, and we should remove the key from the map.

Code Example

func cacheDemo() {
    cache := make(map[string]weak.Pointer[int])
    
    // Insertion
    someData := 42
    someKey := "cache-key"
    cache[key] = weak.Make(&someData)

    // Lookup
    if wp, ok := cache[key]; ok {
        if objPtr := wp.Value(); objPtr != nil {
            // cache hit
        } else {
            // weak pointer returned nil, clean up cache key
            delete(cache, key)
        }
    }
    // handle cache miss here
}

Using weak pointers in Go 1.24+ to create an in-memory cache while allowing memory to be reclaimed if needed

Summary

Weak pointers are an interesting niche feature in Go 1.24. You might not see them in every Go service you encounter, but they do provide a convenient way of implementing caches with reclaimable memory, non-locking pub/sub patterns, or canonicalisation maps. See the slides for my The Strength of the Weak Package talk for more information.