Go Goroutines — go Keyword WaitGroups and Concurrent Execution Explained
In this tutorial, you will learn about Go Goroutines. We cover key concepts, practical examples, and best practices to help you master this topic.
Go goroutines are lightweight threads managed by the Go runtime, started with the go keyword, with WaitGroups for coordinating completion and channels for safe communication between concurrent operations.
What You'll Learn
- Starting goroutines with the go keyword
- Synchronizing with sync.WaitGroup
- Goroutine lifecycle and cleanup
- Concurrency patterns
Why It Matters
Goroutines make concurrency accessible. Docker uses goroutines for container management. Kubernetes uses them for controller loops. DodaZIP uses goroutines for parallel compression. Goroutines are cheap — you can start thousands in a single Process.
Real-World Use
HTTP servers handle each request in a goroutine. Background workers process jobs concurrently. File processors scan multiple files simultaneously. Web scrapers fetch pages in parallel.
flowchart LR
A["Goroutines"] --> B["go keyword"]
B --> C["WaitGroup"]
C --> D["Closures"]
D --> E["Lifecycle"]
A:::current --> B
style A fill:#2563eb,stroke:#2563eb,color:#fff
style B fill:#dbeafe,stroke:#2563eb,color:#1e40af
style C fill:#dbeafe,stroke:#2563eb,color:#1e40af
style D fill:#dbeafe,stroke:#2563eb,color:#1e40af
style E fill:#f1f5f9,stroke:#94a3b8,color:#64748b
Basic Goroutines
func sayHello() {
fmt.Println("Hello from goroutine!")
}
func main() {
// Start a goroutine
go sayHello()
// Start an anonymous goroutine
go func() {
fmt.Println("Hello from anonymous!")
}()
// Give goroutines time to run
time.Sleep(100 * time.Millisecond)
fmt.Println("Main function done")
}
WaitGroup
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done() // Signal completion when done
fmt.Printf("Worker %d starting\n", id)
time.Sleep(time.Second)
fmt.Printf("Worker %d done\n", id)
}
func main() {
var wg sync.WaitGroup
for i := 1; i <= 5; i++ {
wg.Add(1) // Increment counter
go worker(i, &wg) // Start goroutine
}
wg.Wait() // Wait for all goroutines to finish
fmt.Println("All workers completed")
}
Goroutines with Closures
func main() {
var wg sync.WaitGroup
// Bad — captures loop variable
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println(i) // May print 5 for all!
}()
}
wg.Wait()
// Good — pass as parameter
for i := 0; i < 5; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
fmt.Println(id)
}(i)
}
wg.Wait()
// Good — capture local copy
for i := 0; i < 5; i++ {
wg.Add(1)
id := i // Local copy
go func() {
defer wg.Done()
fmt.Println(id)
}()
}
wg.Wait()
}
Goroutine Lifecycle
func main() {
var wg sync.WaitGroup
// Start a goroutine that might panic
wg.Add(1)
go func() {
defer wg.Done()
defer func() {
if r := recover(); r != nil {
fmt.Println("Recovered:", r)
}
}()
// Risky operation
panic("something went wrong")
}()
wg.Wait()
fmt.Println("Program continues...")
}
Working with Channels
func worker(id int, jobs <-chan int, results chan<- int) {
for j := range jobs {
fmt.Printf("Worker %d processing job %d\n", id, j)
time.Sleep(time.Second)
results <- j * 2
}
}
func main() {
const numJobs = 5
jobs := make(chan int, numJobs)
results := make(chan int, numJobs)
// Start 3 workers
var wg sync.WaitGroup
for w := 1; w <= 3; w++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
worker(id, jobs, results)
}(w)
}
// Send jobs
for j := 1; j <= numJobs; j++ {
jobs <- j
}
close(jobs)
// Wait for workers and close results
go func() {
wg.Wait()
close(results)
}()
// Collect results
for result := range results {
fmt.Println("Result:", result)
}
}
Goroutine Pool Pattern
type Pool struct {
workers int
jobs chan func()
wg sync.WaitGroup
}
func NewPool(workers int) *Pool {
p := &Pool{
workers: workers,
jobs: make(chan func(), 100),
}
for i := 0; i < workers; i++ {
p.wg.Add(1)
go p.worker()
}
return p
}
func (p *Pool) worker() {
defer p.wg.Done()
for job := range p.jobs {
job()
}
}
func (p *Pool) Submit(job func()) {
p.jobs <- job
}
func (p *Pool) Shutdown() {
close(p.jobs)
p.wg.Wait()
}
Atomic Operations
var counter int64
func main() {
var wg sync.WaitGroup
for i := 0; i < 1000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
atomic.AddInt64(&counter, 1)
}()
}
wg.Wait()
fmt.Println("Counter:", counter) // 1000
}
Common Mistakes
1. Not Waiting for Goroutines
// Bad — goroutine may not execute before program exits
go func() {
fmt.Println("Never printed if main exits first")
}()
// Good — use WaitGroup
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println("Always printed")
}()
wg.Wait()
2. Loop Variable Capture
// Bad — all goroutines see last i value
for i := 0; i < 5; i++ {
go func() {
fmt.Println(i) // May print 5, 5, 5, 5, 5
}()
}
// Good — pass copy
for i := 0; i < 5; i++ {
go func(id int) {
fmt.Println(id)
}(i)
}
3. Not Recovering from Panics
// Bad — panic crashes program
go func() {
doRiskyWork()
}()
// Good — recover in goroutine
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("goroutine panic: %v", r)
}
}()
doRiskyWork()
}()
4. Forgetting to Pass WaitGroup as Pointer
// Bad — WaitGroup copied, Add/Done on copy
func worker(wg sync.WaitGroup) {
wg.Done() // Modifies copy, not original
}
// Good — pass pointer
func worker(wg *sync.WaitGroup) {
wg.Done()
}
5. Deadlock from Improper Channel Use
// Deadlock — no one reads from channel, writer blocks
ch := make(chan int)
ch <- 42 // Blocks forever — no reader
// Fix — buffer or reader
ch := make(chan int, 1)
ch <- 42 // Buffered, doesn't block
Practice Questions
1. What is a goroutine?
A lightweight thread managed by the Go runtime. Started with the go keyword. Goroutines are multiplexed onto OS threads and are much cheaper than OS threads (2KB stack vs 1MB+).
2. What is sync.WaitGroup used for?
A counter that waits for a collection of goroutines to finish. Add increments, Done decrements, Wait blocks until the counter is zero.
3. How do goroutines differ from OS threads?
Goroutines start with ~2KB stack (vs ~1MB for threads), have fast creation/context switching, and are multiplexed onto fewer OS threads by the Go runtime.
4. What happens if a goroutine panics without recover?
The panic crashes the entire program, not just the goroutine. Always use defer/recover in goroutines to prevent crashes.
Challenge: Create a concurrent web page fetcher that fetches multiple URLs in parallel, limits concurrency to 5 workers, and reports results including HTTP status and response size.
Solution
package main
import (
"fmt"
"io"
"net/http"
"sync"
"time"
)
type FetchResult struct {
URL string
StatusCode int
Size int64
Duration time.Duration
Error error
}
func fetch(url string) FetchResult {
start := time.Now()
resp, err := http.Get(url)
if err != nil {
return FetchResult{URL: url, Error: err}
}
defer resp.Body.Close()
size, err := io.Copy(io.Discard, resp.Body)
if err != nil {
return FetchResult{URL: url, Error: err}
}
return FetchResult{
URL: url,
StatusCode: resp.StatusCode,
Size: size,
Duration: time.Since(start),
}
}
func main() {
urls := []string{
"https://example.com",
"https://golang.org",
"https://github.com",
"https://google.com",
"https://stackoverflow.com",
}
const concurrency = 3
jobs := make(chan string, len(urls))
results := make(chan FetchResult, len(urls))
var wg sync.WaitGroup
// Start workers
for w := 0; w < concurrency; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for url := range jobs {
results <- fetch(url)
}
}()
}
// Send jobs
for _, url := range urls {
jobs <- url
}
close(jobs)
// Wait and close results
go func() {
wg.Wait()
close(results)
}()
// Collect results
for result := range results {
if result.Error != nil {
fmt.Printf("FAIL: %s - %v\n", result.URL, result.Error)
} else {
fmt.Printf("OK: %s [%d] %d bytes in %v\n",
result.URL, result.StatusCode, result.Size, result.Duration)
}
}
}
FAQ
{{< faq question="How many goroutines can I start?" >}} Thousands to millions. Each goroutine starts with ~2KB stack. Practical limit depends on memory. 100,000 goroutines use ~200MB stack space. Benchmark your specific use case. {{< /faq >}}
{{< faq question="Do goroutines run in parallel?" >}}
By default, goroutines run concurrently (interleaved) on GOMAXPROCS threads. Set GOMAXPROCS to the number of CPU cores for parallel execution. runtime.GOMAXPROCS(runtime.NumCPU()).
{{< /faq >}}
{{< faq question="What is the difference between concurrency and parallelism?" >}} Concurrency is dealing with multiple tasks at once (structuring programs). Parallelism is doing multiple tasks at once (execution). Go supports both, but concurrency is the primary design goal. {{< /faq >}}
{{< faq question="How do I stop a goroutine?" >} Use a done channel or context. The goroutine should periodically check the channel and return early when signaled. There's no way to kill a goroutine externally. {{< /faq >}}
{{< faq question="Should I use goroutines for everything?" >}} No. Goroutines add complexity and potential for bugs (races, deadlocks). Use them when you have independent work that can proceed concurrently — I/O, CPU-bound tasks, waiting operations. {{< /faq >}}
Try It Yourself
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
// Start 3 goroutines
for i := 1; i <= 3; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
fmt.Printf("Goroutine %d: starting\n", id)
time.Sleep(time.Duration(id) * 500 * time.Millisecond)
fmt.Printf("Goroutine %d: finished\n", id)
}(i)
}
fmt.Println("Waiting for goroutines...")
wg.Wait()
fmt.Println("All done!")
}
Expected output:
Goroutine 1: starting
Goroutine 2: starting
Goroutine 3: starting
Goroutine 1: finished
Goroutine 2: finished
Goroutine 3: finished
Waiting for goroutines...
All done!
What's Next
Now that you understand goroutines, learn about channels for communication between concurrent goroutines.
| Topic | Description | Link |
|---|---|---|
| Go Channels | Channel types, buffering, range | {{< ref "18-channels" >}} |
| Go WaitGroups | Synchronization patterns | {{< ref "20-waitgroups" >}} |
| Rust Threads | Compare Rust's threading model | Rust |