Background Processing

HTTP request–response is not the only kind of work a system does.

As systems mature, they inevitably need to:

• Send emails
• Publish events
• Generate reports
• Process uploads
• Sync external systems
• Run scheduled or long-running tasks

The challenge is not how to do background work, but:

How to do it without corrupting request semantics or hiding system behavior.

Plumego provides no built-in background processing framework — deliberately.

This document explains how to add background processing explicitly and safely.

First Principle: Requests Are Finite

A Plumego request has a strict lifecycle:

• It starts
• It runs middleware and a handler
• It ends
• Its Context is discarded

This implies a non-negotiable rule:

Background work must not depend on request-scoped Context.

If a background task needs request data, that data must be copied explicitly.

When Background Processing Is Appropriate

Background processing is justified when:

• Work is slow or unpredictable
• Results are not needed synchronously
• Failure can be retried
• Throughput matters more than latency
• The task outlives the HTTP request

Examples:

• Email notifications
• Analytics events
• Image processing
• Cache warming
• External system syncs

If the client must wait for the result, it is not background work.

What Background Processing Is Not

Background processing should not be used to:

• Hide slow synchronous dependencies
• Avoid proper timeouts
• Mask architectural problems
• Perform critical transactional logic
• Circumvent error handling

If correctness depends on the task completing immediately,
it should not be asynchronous.

Pattern 1: Fire-and-Forget Goroutines (Use with Care)

The simplest form is spawning a goroutine:

go sendEmail(payload)

This is acceptable only if all of the following are true:

• The task is short-lived
• Failure is acceptable or logged
• No retries are required
• The process lifecycle is managed externally

Important constraints:

• Never capture *plumego.Context
• Never assume completion
• Never leak goroutines

This pattern is intentionally limited.

Pattern 2: In-Process Job Queue

For more control, introduce an explicit job queue:

type Job interface {
	Run(ctx context.Context) error
}

Characteristics:

• Bounded concurrency
• Controlled retries
• Explicit lifecycle
• Shared process memory

This works well for:

• Moderate workloads
• Single-service deployments
• Tasks tightly coupled to the service

The queue should be initialized at startup,
not inside handlers.

Pattern 3: External Job Workers

For durable or heavy workloads, external workers are preferred.

Common choices:

• Message queues
• Task brokers
• Event streams
• Dedicated worker services

Flow:

HTTP Handler
  → validate input
  → enqueue job
  → return response

Worker
  → consume job
  → perform work
  → record result

Plumego remains focused on HTTP boundaries.
Workers are separate concerns.

Passing Data to Background Jobs

Only pass explicit, minimal data:

• Identifiers (IDs)
• Immutable snapshots
• Serialized payloads

Never pass:

• Database connections
• In-memory pointers
• Context objects
• Closures over request state

Background jobs should be replayable and idempotent.

Error Handling in Background Tasks

Background tasks fail differently than requests.

Guidelines:

• Failures must be observable
• Retries must be bounded
• Permanent failures must be recorded
• Silent failure is unacceptable

Logging alone is insufficient for critical tasks.

Background Processing and Transactions

A common pitfall:

“I’ll start a background job inside a transaction.”

This is dangerous.

Correct pattern:

1. Commit the transaction
2. Emit an event or enqueue a job
3. Process asynchronously

Never rely on background work to complete
before a transaction commits.

Scheduling Background Work

Scheduled tasks (cron-like behavior) should:

• Live outside request handlers
• Be initialized at process startup
• Use explicit scheduling mechanisms
• Respect process shutdown signals

Do not simulate scheduling with HTTP requests.

Lifecycle and Shutdown Coordination

Background processing must integrate with process lifecycle.

Rules:

• Workers must stop on shutdown
• In-flight jobs must be handled deliberately
• Shutdown behavior must be explicit

Plumego does not manage this for you — by design.

Observability for Background Work

Background tasks need the same observability discipline:

• Structured logs
• Trace or correlation IDs
• Metrics (success/failure, duration)
• Alerting for stuck or failing jobs

If background work is invisible, it is unmaintainable.

Background Work and Multi-Service Systems

In multi-service setups:

• Prefer asynchronous communication
• Use events rather than synchronous chains
• Accept eventual consistency where appropriate

Background processing is often the key
to avoiding distributed monoliths.

Common Anti-Patterns

Capturing Request Context in Goroutines

This leads to race conditions and undefined behavior.

Unbounded Goroutine Creation

This causes memory and scheduling collapse under load.

Hiding Critical Logic in Background Jobs

Critical business decisions must be explicit and observable.

Treating Background Failures as “Best Effort”

If failure matters, design for it explicitly.

Summary

In Plumego:

• Requests are finite and explicit
• Background work must be decoupled
• Context does not escape request scope
• Async behavior is intentional, not implicit
• Lifecycle and observability matter

Background processing is powerful —
but only when its boundaries are respected.

Next

The final major advanced topic is:

→ Advanced / API Versioning

This explains how to evolve HTTP interfaces over time
without breaking clients or losing clarity.