DataDog has published the source code for pg_tracing, a new PostgreSQL extension engineered to integrate distributed tracing directly at the database server level. This release aims to provide developers and operators with deeper visibility into query execution dynamics, aiding in performance tuning and latency identification within complex application stacks. The extension is currently available on GitHub and supports PostgreSQL versions 14, 15, and 16.
When active, pg_tracing samples queries and generates detailed performance spans covering crucial internal events such as the Planner, ExecutorRun, and Transaction Commit stages, including fsync times on the Write-Ahead Log (WAL). The extension tracks standard DML statements like SELECT and INSERT, as well as utility operations and the individual nodes within execution plans, such as SeqScan or HashJoin.
Trace context propagation, a necessary component for linking database activity to upstream microservices, is supported through two mechanisms. These include standard SQL comments utilizing SQLCommenter syntax or configuration via the GUC parameter pg_tracing.trace_context. Users can then access the generated spans either by querying the pg_tracing_consume_spans view or by using the pg_tracing_json_spans function for OTLP JSON output.
Installation requires compiling the extension against the PostgreSQL development packages and loading it via the shared_preload_libraries directive in postgresql.conf, necessitating a server restart. Configuration parameters allow users to define the maximum number of spans to retain and optionally set an OTLP endpoint for automatic asynchronous export to an OpenTelemetry collector.
DataDog notes that the pg_tracing extension remains in early development, suggesting potential instability for production environments at this initial stage. Operators can control sampling rates using the pg_tracing.sample_rate parameter, where setting the value to one traces all queries for comprehensive diagnostics.
This development moves observability deeper into the persistence layer, addressing a common blind spot where application-level tracing often terminates before capturing the full duration of database interaction. By exposing internal PostgreSQL mechanics as trace spans, pg_tracing offers a more holistic view of request latency.
Looking ahead, the successful adoption of pg_tracing will depend on its stability and community contributions, particularly regarding support for newer PostgreSQL features or versions. Its ability to integrate seamlessly with existing OpenTelemetry pipelines positions it as a significant tool for modern database performance engineering.
