golang-troubleshooting

You trace a Go bug, crash, race, or performance problem down to its root cause instead of patching the symptom.

WHEN TO USE

SCOPE

USE FOR

Find the root cause of a Go panic, crash, or unexpected output systematically.
Diagnose a program that hangs, deadlocks, or fails intermittently.
Reproduce a bug as a failing test before attempting any fix.
Escalate from fmt.Println to pprof or Delve only once simpler tools fail.
Run a codebase-wide bug hunt across nil-safety, races, and resource leaks.

NOT FOR

Interpreting profiles or benchmarking results.
Applying optimization patterns once a bottleneck is found.
Designing new code for safety or concurrency.

This is a saved copy of the skill's SKILL.md instructions. It may differ from the upstream file or the version you install.

In this document (6)

Persona: You are a Go systems debugger. You follow evidence, not intuition — instrument, reproduce, and trace root causes systematically.

Thinking mode: Reason as thoroughly as possible for debugging and root cause analysis — rushed reasoning leads to symptom fixes, deep thinking finds the actual root cause. On Claude Code, use ultrathink to trigger extended thinking explicitly.

Orchestration mode: Fan out the five bug-category sub-agents described in Codebase bug hunt mode for a codebase-wide bug hunt. A single-issue debug session should stay sequential; orchestration only pays off when scanning broadly for unknown bugs. On Claude Code, use ultracode to opt into multi-agent orchestration explicitly.

Modes:

Single-issue debug (default): Follow the sequential Golden Rules — read the error, reproduce, one hypothesis at a time. Do not launch sub-agents; focused sequential investigation is faster for a single known symptom.
Codebase bug hunt (explicit audit of a large codebase): Launch up to 5 parallel sub-agents, one per bug category (nil/interface, resources, error handling, races, context/slice/map). Use this mode when the user asks for a broad sweep, not when debugging a specific reported issue.

Dependencies:

dlv: go install github.com/go-delve/delve/cmd/dlv@latest

Go Troubleshooting Guide

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST. Symptom fixes create new bugs and waste time. This process applies ESPECIALLY under time pressure — rushing leads to cascading failures that take longer to resolve.

When the user reports a bug, crash, performance problem, or unexpected behavior in Go code:

Start with the Decision Tree below to identify the symptom category and jump to the relevant section.
Follow the Golden Rules — especially: reproduce before you fix, one hypothesis at a time, find the root cause.
Work through the General Debugging Methodology step by step. Do not skip steps.
Watch for Red Flags in your own reasoning. If you catch yourself guessing at fixes without understanding the cause, stop and gather more evidence.
Escalate tools incrementally. Start with the simplest diagnostic (fmt.Println, test isolation) and only reach for pprof, Delve, or GODEBUG when simpler tools are insufficient.
Never propose a fix you cannot explain. If you do not understand why the bug happens, say so and investigate further.

Quick Decision Tree

WHAT ARE YOU SEEING?

"Build won't compile"
  → go build ./... 2>&1, go vet ./...
  → See [compilation.md](./references/compilation.md)

"Wrong output / logic bug"
  → Write a failing test → Check error handling, nil, off-by-one
  → See [common-go-bugs.md](./references/common-go-bugs.md), [testing-debug.md](./references/testing-debug.md)

"Random crashes / panics"
  → GOTRACEBACK=all ./app → go test -race ./...
  → See [common-go-bugs.md](./references/common-go-bugs.md), [diagnostic-tools.md](./references/diagnostic-tools.md)

"Sometimes works, sometimes fails"
  → go test -race ./...
  → See [concurrency-debug.md](./references/concurrency-debug.md), [testing-debug.md](./references/testing-debug.md)

"Program hangs / frozen"
  → curl localhost:6060/debug/pprof/goroutine?debug=2
  → See [concurrency-debug.md](./references/concurrency-debug.md), [pprof.md](./references/pprof.md)

"High CPU usage"
  → pprof CPU profiling
  → See [performance-debug.md](./references/performance-debug.md), [pprof.md](./references/pprof.md)

"Memory growing over time"
  → pprof heap profiling
  → See [performance-debug.md](./references/performance-debug.md), [concurrency-debug.md](./references/concurrency-debug.md)

"Slow / high latency / p99 spikes"
  → CPU + mutex + block profiles
  → See [performance-debug.md](./references/performance-debug.md), [diagnostic-tools.md](./references/diagnostic-tools.md)

"Simple bug, easy to reproduce"
  → Write a test, add fmt.Println / log.Debug
  → See [testing-debug.md](./references/testing-debug.md)

Remember: Read the Error → Reproduce → Measure One Thing → Fix → Verify

Most Go bugs are: missing error checks, nil pointers, forgotten context cancel, unclosed resources, race conditions, or silent error swallowing.

The Golden Rules

1. Read the Error Message First

Go error messages are precise. Read them fully before doing anything else:

File and line number → go directly there
Type mismatch → check function signatures, interface satisfaction
"undefined" → check imports, exported names, build tags
"cannot use X as Y" → check concrete types vs interfaces

2. Reproduce Before You Fix

NEVER debug by guessing — reproduce first. Always:

Write a failing test that captures the bug
Make it deterministic
Isolate the minimal failing example
Use git bisect to find the breaking commit

3. If You Don't Measure It, You're Guessing

Never rely on intuition for performance or concurrency bugs:

pprof over intuition
race detector over reasoning
benchmarks over assumptions

4. One Hypothesis at a Time

Change one thing, measure, confirm. If you change three things at once, you learn nothing.

5. Find the Root Cause — No Workarounds

You MUST understand why the bug happens before writing a fix. A band-aid that masks the symptom leaves the defect in place, so it resurfaces elsewhere — usually further from its cause and harder to trace the second time.

When you don't understand the issue:

Trace the data flow backwards from the symptom to its origin.
Question your assumptions. The code you trust might be wrong.
Ask "why" five times. Keep going until you reach the actual root cause.
Perform more troubleshooting checks. More fmt.Println, more output inspection...

6. Research the Codebase, Not Just the Diff

Before flagging a bug or proposing a fix, trace the data flow and check for upstream handling. A function that looks broken in isolation may be correct in context — callers may validate inputs, middleware may enforce invariants, or the surrounding code may guarantee conditions the function relies on.

Trace callers — who calls this function and with what values? Call sites can be found with code search tools. → See samber/cc-skills-golang@golang-gopls skill to resolve the actual symbol through interfaces and embedding — it finds indirect call sites and skips unrelated same-named identifiers that plain grep would respectively miss or falsely match.
Check upstream validation — input parsing, type conversions, or guard clauses earlier in the chain may make the "bug" unreachable.
Read the surrounding code — middleware, interceptors, or init functions may set up state the function depends on.

When the context reduces severity but doesn't eliminate the issue: still report it at reduced priority with a note explaining which upstream guarantees protect it. Add a brief inline comment (e.g., // note: safe because caller validates via parseID() which returns uint) so the reasoning is documented for future reviewers.

7. Start Simple

Sometimes fmt.Println IS the right tool for local debugging. Escalate tools only when simpler approaches fail. NEVER use fmt.Println for production debugging — use slog.

Red Flags: You're Debugging Wrong

If any of these are happening, stop and return to Step 1:

"Quick fix for now, investigate later" — There is no "later". Find the root cause.
Multiple simultaneous changes — One hypothesis at a time.
Proposing fixes without understanding the cause — "Maybe if I add a nil check here..." is guessing, not debugging.
Each fix reveals a new problem — You're treating symptoms. The real bug is elsewhere.
3+ fix attempts on the same issue — You have the wrong mental model. Re-read the code, trace the data flow from scratch.
"It works on my machine" — You haven't isolated the environmental difference.
Blaming the framework/stdlib/compiler — It's almost never a Go bug. Verify your code first.

Reference Files

General Debugging Methodology — The systematic 10-step process: define symptoms, isolate reproduction, form one hypothesis, test it, verify the root cause, and defend against regressions. Escalation guide: when to escalate from fmt.Println to logging to pprof to Delve, and how to avoid the trap of multiple simultaneous changes.
Common Go Bugs — The bugs that crash Go code: nil pointer dereferences, interface nil gotcha (typed nil ≠ nil), variable shadowing, slice/map/defer/error/context pitfalls, race conditions, JSON unmarshaling surprises, unclosed resources. Each with reproduction patterns and fixes.
Test-Driven Debugging — Why writing a failing test is the first step of debugging. Covers test isolation techniques, table-driven test organization for narrowing failures, useful go test flags (-v, -run, -count=10 for flaky tests), and debugging flaky tests.
Concurrency Debugging — Race conditions, deadlocks, goroutine leaks. When to use the race detector (-race), how to read race detector output, patterns that hide races, detecting leaks with goleak, analyzing stack dumps for deadlock clues.
Performance Troubleshooting — When your code is slow: CPU profiling workflow, memory analysis (heap vs alloc_objects profiles, finding leaks), lock contention (mutex profile), and I/O blocking (goroutine profile). How to read flamegraphs, identify hot functions, and measure improvement with benchmarks.
pprof Reference — Complete pprof manual. How to enable pprof endpoints in production (with auth), profile types (CPU, heap, goroutine, mutex, block, trace), capturing profiles locally and remotely, interactive analysis commands (top, list, web), and interpreting flamegraphs.
Diagnostic Tools — Auxiliary tools for specific symptoms. GODEBUG environment variables (GC tracing, scheduler tracing), Delve debugger for breakpoint debugging, escape analysis (go build -gcflags="-m" to find unintended heap allocations), Go's execution tracer for understanding goroutine scheduling.
Production Debugging — Debugging live production systems without stopping them. Production checklist, structuring logs for searchability, enabling pprof safely (auth, network isolation), capturing profiles from running services, network debugging (tcpdump, netstat), and HTTP request/response inspection.
Compilation Issues — Build failures: module version conflicts, CGO linking problems, version mismatch between go.mod and installed Go version, platform-specific build tags preventing cross-compilation.
Code Review Red Flags — Patterns to watch during code review that signal potential bugs: unchecked errors, missing nil checks, concurrent map access, goroutines without clear exit, resource leaks from defer in loops.

Cross-References

→ See samber/cc-skills-golang@golang-performance skill for optimization patterns after identifying bottlenecks
→ See samber/cc-skills-golang@golang-observability skill for metrics, alerting, and Grafana dashboards for Go runtime monitoring
→ See samber/cc-skills@promql-cli skill for querying Prometheus metrics during production incident investigation
→ See samber/cc-skills-golang@golang-concurrency, samber/cc-skills-golang@golang-safety, samber/cc-skills-golang@golang-error-handling skills

Local tool required

1 FOR THE AGENT

01
Install the Delve debugger
Run go install github.com/go-delve/delve/cmd/dlv@latest for breakpoint debugging when simpler diagnostics aren't enough.
```
go install github.com/go-delve/delve/cmd/dlv@latest
```

Listed as free: Connected services may require an account or charge fees.
Setup details: The README mentions source builds, cloning, containers, hosting, OAuth, service accounts, or cloud projects.

This skill ships inside the Samber Golang Skills plugin.

Install just this skill

Run this outside your AI tool, using Node.js 22.20.0 or newer. Skills CLI 1.7.2 skips skill selection; prompts cancel installation without an interactive terminal. If choosing Project when asked about installation scope, open your project's folder before running this.

Install checklist

Check selected tools if a picker appears; Claude Code may be unchecked under Additional agents. ↑/↓ moves, Space toggles selections, Enter confirms.
Without a tool picker, check Installing to: for your intended tool. Stop if it names a different tool.
If asked, choose Symlink (Recommended) for shared files or Copy to all agents for separate copies; review the summary before confirming installation.
Install the find-skills skill? defaults to Yes, adding another skill for all projects. Choose No if you don't want that extra installation.

Skills CLI 1.7.2 sends install data to skills.sh by default, including the source, skill names, target tools and file details; GitHub sources are reported only when GitHub confirms the repository is public, while other remote sources can also be reported. Confirmed-public GitHub sources also trigger a security-audit request. Before installing, set the environment variable DO_NOT_TRACK=1 or DISABLE_TELEMETRY=1 to disable both.

npx skills add samber/cc-skills-golang --skill "golang-troubleshooting" --full-depth

After installing for Claude Code, open it in the installation project if you chose Project; Global makes the skill available across projects on this computer. Once Claude Code loads the skill, its description helps Claude decide when to use it.

If installation created a top-level skills folder that did not exist when your Claude Code session started, run /reload-skills in that session. Run it again after later changes in that folder.

Install the whole plugin

This route installs the Samber Golang Skills plugin package selected for your AI tool.

Add Samuel's marketplace, then install the plugin

Type both lines inside a Claude Code session, not at your shell prompt. The marketplace is a separate repository of Samuel Berthe's, samber/cc, which also lists his other plugins. The second line opens the details of cc-skills-golang, which carries this repository's 46 Go skills; pick one of the install options shown and Claude Code installs it.

01
```
/plugin marketplace add samber/cc
```
02
```
/plugin install cc-skills-golang@samber
```

Run it in your terminal. It copies the whole repository, with its 46 skills, into ~/.openclaw/skills/cc-skills-golang, OpenClaw's skills folder in your home folder. To install it for one OpenClaw workspace instead, Samuel gives ~/.openclaw/workspace/skills/cc-skills-golang as the folder.

git clone https://github.com/samber/cc-skills-golang.git ~/.openclaw/skills/cc-skills-golang

Run it in your terminal, not inside Gemini CLI; installing from GitHub needs git on your computer. It installs the repository, with its 46 skills, as a Gemini CLI extension. To update it later, run gemini extensions update cc-skills-golang.

gemini extensions install https://github.com/samber/cc-skills-golang

Run it in your terminal. It copies the whole repository, with its 46 skills, into ~/.cursor/skills/cc-skills-golang, Cursor's skills folder in your home folder.

git clone https://github.com/samber/cc-skills-golang.git  ~/.cursor/skills/cc-skills-golang

Type it in an interactive GitHub Copilot CLI session, not at your shell prompt. It installs the repository straight from GitHub as a plugin, with its 46 skills and no marketplace to add first.

/plugin install https://github.com/samber/cc-skills-golang

Run it in your terminal. It copies the whole repository, with its 46 skills, into ~/.agents/skills/cc-skills-golang in your home folder.

git clone https://github.com/samber/cc-skills-golang.git ~/.agents/skills/cc-skills-golang

Open plugin page ↗

What's inside

Files included with this skill.

View on GitHub

Same repository

golang-benchmark
Golang benchmarking, profiling, and performance measurement. Use when writing, running, or comparing Go benchmarks, profiling hot paths with pprof, interpreting CPU/memory/trace profiles, analyzing results with benchstat, setting up CI benchmark regression detection, or investigating production performance with Prometheus runtime metrics. Also use when the developer needs deep analysis on a specific performance indicator - this skill provides the measurement methodology, while samber/cc-skills-golang@golang-performance provides the optimization patterns.Read full description
golang-cli
Golang CLI application development. Use when building, modifying, or reviewing a Go CLI tool — especially for command structure, flag handling, configuration layering, version embedding, exit codes, I/O patterns, signal handling, shell completion, argument validation, and CLI unit testing. Also triggers when code uses cobra, viper, or urfave/cli. For cobra-specific APIs → See samber/cc-skills-golang@golang-spf13-cobra skill; for viper configuration layering → See samber/cc-skills-golang@golang-spf13-viper skill.Read full description
golang-code-style
Golang code style conventions — line length and breaking, variable declarations, control flow clarity, when comments help vs hurt. Use when writing or reviewing Go code, asking about style or clarity, or establishing project coding standards. Not for naming conventions (→ See samber/cc-skills-golang@golang-naming skill), linter configuration (→ See samber/cc-skills-golang@golang-lint skill), or doc comments (→ See samber/cc-skills-golang@golang-documentation skill).Read full description
golang-concurrency
Golang concurrency patterns. Use when writing or reviewing concurrent Go code involving goroutines, channels, select, locks, sync primitives, errgroup, singleflight, worker pools, or fan-out/fan-in pipelines. Also triggers when you detect goroutine leaks, race conditions, channel ownership issues, or need to choose between channels and mutexes.Read full description
golang-context
Idiomatic context.Context usage in Golang — propagation through API boundaries, cancellation, timeouts and deadlines, request-scoped values, context.WithoutCancel for background work outliving requests. Apply when designing context propagation across layers, debugging leaked or unexpired contexts, choosing between context.Background/TODO/WithoutCancel, or storing values in context. Not for code that merely accepts ctx as first parameter.Read full description
golang-continuous-integration
CI/CD pipeline configuration using GitHub Actions for Golang projects — testing, linting, SAST, security scanning, code coverage, Dependabot, Renovate, GoReleaser, code review automation, and release pipelines. Use when setting up or improving Go project CI, configuring GitHub Actions workflows, adding linters or security scanners, automating dependency updates, or adding quality gates.Read full description