Structured Logging

The dcc_backend_common.logger module provides structured, consistent logging across all services using structlog.

Logging Standards

For general logging best practices and standards, see the Python Coding Standards - Logging section.

Overview

The module provides:

• init_logger(): Initialize the logging system (call once at startup)
• get_logger(): Get a structured logger instance for any module
• get_usage_logger(): Get the pinned usage logger for usage/audit events
• USAGE_LOGGER_NAME: The name of that logger ("usage") — use it to filter in OpenSearch
• FocusedTracebackFormatter: Advanced traceback rendering for development that highlights user code
• DevTracebackStyle: Configuration options for traceback styles in development mode
• Automatic context: Timestamps, module names, function names, and line numbers (plus request_id when the logging middleware is installed)
• Environment-aware rendering: JSON in production (fluentbit compatible), colored console in development

One Pipeline for Everything

init_logger() installs a single handler on the root logger. structlog events and stdlib records (uvicorn, third-party libraries, Python warnings) are rendered by the same pipeline, so every line has the same shape:

• Handlers attached by uvicorn / fastapi-cli are cleared so records flow through the root handler.
• uvicorn.access is disabled entirely (propagate = False) — per-request 200 lines are noise for fluentbit/OpenSearch, and the logging middleware reports the failures that matter.
• Chatty libraries (httpx, httpcore, openai, urllib3, aiohttp) are capped at WARNING.
• logging.captureWarnings(True) routes DeprecationWarning & friends through the pipeline instead of raw stderr.

Serving

Run with plain uvicorn, not fastapi run. The fastapi run Rich banner and handlers bypass the JSON pipeline:

uvicorn my_app.app:app --host 0.0.0.0 --port 8090 --no-access-log

Installation

The logger module is part of the dcc-backend-common package:

uv add dcc-backend-common

Quick Start

Initialize the logger once at your application's entry point:

# app.py
from contextlib import asynccontextmanager

from dcc_backend_common.logger import init_logger, get_logger
from fastapi import FastAPI


@asynccontextmanager
async def lifespan(app: FastAPI):
    """Application lifespan handler."""
    init_logger()  # Initialize once at startup
    logger = get_logger(__name__)
    logger.info("Application started")
    yield
    logger.info("Application shutting down")


app = FastAPI(lifespan=lifespan)

Configuration

The logger is configured via environment variables:

Variable	Default	Description
LOG_LEVEL	INFO	Logging level for application diagnostics (DEBUG, INFO, WARNING, ERROR, CRITICAL). Does not affect usage events
IS_PROD	-	Set to true for JSON output (fluentbit compatible)
DEV_TRACEBACK_STYLE	focused	Traceback style in development: focused (user code locals) or rich (all locals)
LOGGER_USER_CODE_PATHS	dcc_backend_common, src/, app/, tests/	Comma-separated paths to identify user code for detailed tracebacks

Development vs Production

• Development (IS_PROD not true): Colored console output for readability.
  • Traceback Styles:
    • focused (default): Shows the full Rich traceback but only displays local variables for frames identified as "user code" (based on LOGGER_USER_CODE_PATHS). This reduces noise from library internals.
    • rich: Shows the full Rich traceback with local variables displayed for all frames, including third-party libraries. This is more verbose but useful for deep debugging.
• Production (IS_PROD=true): JSON output for log aggregation tools like fluentbit. Tracebacks are rendered into a string field by format_exc_info.

Recommended levels

LOG_LEVEL=debug on test stages, LOG_LEVEL=info in production.

Usage Events

Usage/audit events go through a separate, pinned logger named usage. init_logger() sets that logger to INFO explicitly, so its events survive any LOG_LEVEL — even LOG_LEVEL=WARNING.

from dcc_backend_common.logger import get_usage_logger

usage_logger = get_usage_logger()
usage_logger.info("app_event", action="translator.translate", pseudonym_id=..., chars=1200)

Two producers already use it:

Producer	Event	Emitted by
UsageTrackingService.log_event()	app_event	Usage Tracking
BaseAgent (every LLM call)	llm_call	LLM Agent

In OpenSearch, filter on logger: "usage" to separate business events from diagnostics. The constant USAGE_LOGGER_NAME holds that name if you need it in code.

Keep diagnostics out

Do not route ordinary application logging through the usage logger — it is never silenced by LOG_LEVEL.

Usage

Getting a Logger

Use get_logger() in any module to get a structured logger:

from dcc_backend_common.logger import get_logger

# Pass the module name for context
logger = get_logger(__name__)


def process_translation(request: TranslationRequest) -> TranslationResult:
    logger.info(
        "Processing translation request",
        source_lang=request.source_lang,
        target_lang=request.target_lang,
        text_length=len(request.text),
    )

    try:
        result = translate(request)
        logger.info("Translation completed", result_length=len(result.text))
        return result
    except TranslationError as e:
        logger.error("Translation failed", error=str(e), request_id=request.id)
        raise

Automatic Context

The logger automatically adds context to every log entry:

Field	Description
timestamp	ISO-8601 formatted timestamp (UTC)
logger	The logger name passed to get_logger()
module	The module name where the log was called
func_name	The function name
lineno	The line number
level	The log level
request_id	Unique UUID for request tracing — only present when the logging middleware is installed

Example JSON output in production:

{
  "timestamp": "2024-01-15T10:30:45.123456Z",
  "logger": "myapp.services.translator",
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "module": "translator",
  "func_name": "translate",
  "lineno": 42,
  "level": "info",
  "event": "Translation completed",
  "result_length": 256
}

Integration with FastAPI

Request Logging Middleware

Do not hand-roll request logging — the package ships a middleware that binds a request_id to every log line emitted while handling a request:

from dcc_backend_common.fastapi_logging_middleware import add_logging_middleware

add_logging_middleware(app)

See Logging Middleware for details.

API Reference

DevTracebackStyle

Enum defining the available traceback styles for development mode.

Value	Description
FOCUSED	Rich traceback + focused locals for user code only (default)
RICH	Default Rich traceback with full locals for all frames (verbose)

FocusedTracebackFormatter

A custom exception formatter used in development mode.

• Shows detailed tracebacks for user code (based on LOGGER_USER_CODE_PATHS).
• Keeps library code traces dense and minimal.
• Appends a specific "Local variables in your code" section for user frames.

init_logger()

Initialize the logger configuration. Must be called once at application startup.

from dcc_backend_common.logger import init_logger

init_logger()

Environment Variables:

• IS_PROD: "true" for production (JSON output), "false" for development (console output).
• LOG_LEVEL: Logging level (default: "INFO").
• DEV_TRACEBACK_STYLE: Traceback style in dev mode ("focused" or "rich").
• LOGGER_USER_CODE_PATHS: Comma-separated paths to consider as user code.

get_logger(name)

Get a structured logger instance.

from dcc_backend_common.logger import get_logger

logger = get_logger(__name__)  # Pass module name for context
logger = get_logger()  # Or get anonymous logger

Parameters:

• name (str | None): Optional name for the logger, typically __name__

Returns:

• BoundLogger: A structlog bound logger instance

get_usage_logger()

Get the logger for usage/audit events. Events logged here are always emitted (INFO and up), regardless of LOG_LEVEL.

from dcc_backend_common.logger import get_usage_logger

usage_logger = get_usage_logger()
usage_logger.info("llm_call", input_tokens=812, output_tokens=134)

Returns:

• BoundLogger: A structlog bound logger named usage

USAGE_LOGGER_NAME

Module-level constant holding the usage logger's name ("usage"). Use it when filtering log records or configuring downstream tooling.

Source Code

The full implementation is available on GitHub: dcc_backend_common/logger/logger.py