Skip to content

Server function breaks tracing'span parenting (issue with axum-tracing-opentelemetry?) #4896

@davidB

Description

@davidB

Problem

I try to set up tracing + opentelemetry on the server (fullstack web), using axum-tracing-opentelemetry.

  • Instrumenting the server function generates 2 unrelated traces (no parenting between the span of the handler and the span of the function)
  • Instrumenting directly an axum handler generates 1 trace as expected

The "root"/parent span is created in the middleware.

Steps To Reproduce

Steps to reproduce the behavior:

Sorry for the long code listing

  • Create a project with OpenTelemetry enabled on the server
  • Add 2 endpoints
    • a server function get_200()
    • an axum route + handler for get_200_direct()
  • #[tracing::instrument] both function

The code below should reproduce the case:

[dependencies]
anyhow = { version = "*", optional = true } # same version as dioxus
axum-tracing-opentelemetry = { version = "0.32", optional = true }
dioxus = { version = "0.7", features = ["router", "fullstack"] }
init-tracing-opentelemetry = { version = "0.33", features = [
  "tracing_subscriber_ext",
], optional = true }
serde_json = "*"
tokio = { version = "*", features = [
  "full",
], optional = true } # same version as dioxus
tower = { version = "0.5", features = ["util"], optional = true }
tower-http = { version = "0.6", features = [
  "compression-full",
  "cors",
  "decompression-full",
  "sensitive-headers",
  "timeout",
  "trace",
  "validate-request",
], optional = true }
tracing = { version = "*", optional = true } # same version as dioxus
tracing-opentelemetry-instrumentation-sdk = { version = "0.32", optional = true }

[dev-dependencies]
pretty_assertions = "1"
rstest = "0.26"

[features]
default = ["web", "server"]
# The feature that are only required for the web = ["dioxus/web"] build target should be optional and only enabled in the web = ["dioxus/web"] feature
web = ["dioxus/web"]
# # The feature that are only required for the desktop = ["dioxus/desktop"] build target should be optional and only enabled in the desktop = ["dioxus/desktop"] feature
# desktop = ["dioxus/desktop"]
# # The feature that are only required for the mobile = ["dioxus/mobile"] build target should be optional and only enabled in the mobile = ["dioxus/mobile"] feature
# mobile = ["dioxus/mobile"]
# The feature that are only required for the server = ["dioxus/server"] build target should be optional and only enabled in the server = ["dioxus/server"] feature
server = [
  "dioxus/server",
  "dep:anyhow",
  "dep:axum-tracing-opentelemetry",
  "dep:init-tracing-opentelemetry",
  "dep:tokio",
  "dep:tower",
  "dep:tower-http",
  "dep:tracing",
  "dep:tracing-opentelemetry-instrumentation-sdk",
]
use dioxus::prelude::*;

#[derive(Debug, Clone, Routable, PartialEq)]
#[rustfmt::skip]
enum Route {
    #[route("/")]
    Home,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Run `serve()` on the server only
    #[cfg(feature = "server")]
    main_server::launch(App)?;

    // When not on the server, just run `launch()` like normal
    #[cfg(not(feature = "server"))]
    dioxus::launch(App);

    Ok(())
}

#[component]
pub(crate) fn App() -> Element {
    rsx! {
        Router::<Route> {}
    }
}

#[component]
pub(crate) fn Home() -> Element {
    rsx! {
        "hello"
    }
}

#[cfg(feature = "server")]
mod main_server {
    use anyhow::Result;
    use axum_tracing_opentelemetry::middleware::{OtelAxumLayer, OtelInResponseLayer};
    use dioxus::fullstack::response::IntoResponse;
    use dioxus::fullstack::Json;
    use dioxus::prelude::*;
    use dioxus::server::axum::{routing::get, Router};
    use tokio::runtime;

    //TODO initialize tracing and other logging here
    //TODO move "/healthz" "/readyz" "/metrics" into an other port
    pub(crate) fn launch(app: fn() -> Element) -> Result<()> {
        // duplicate code of dioxus::serve because exporter of opentelemetry need a reactor
        // else "there is no reactor running, must be called from the context of a Tokio 1.x runtime"
        // a mutli-thread scheduler
        let rt = runtime::Runtime::new()?;

        // tracing's subscriber (logging) should be set before dioxus try to initialiaze logging
        let _guard = rt.block_on(async {
            init_tracing_opentelemetry::TracingConfig::production()
                .with_stderr() // to avoid mixing with dioxus accesslog like (on stdout)
                .init_subscriber()
        })?;
        // `dioxus::serve` can not be called insode a Runtime::block_on
        // "Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks"
        dioxus::serve(|| async move { new_router(app) })
        //unreachable!("Serving a fullstack app should never return")
    }

    fn new_router(app: fn() -> Element) -> Result<Router> {
        let router = dioxus::server::router(app)
            .route("/api/200-direct", get(get_200_direct))
            .layer((
                OtelAxumLayer::default().filter(|path| path.starts_with("/api")),
                OtelInResponseLayer,
            ));
        Ok(router)
    }

    #[get("/api/200")]
    #[tracing::instrument]
    pub(crate) async fn get_200() -> Result<Option<String>, HttpError> {
        use tracing_opentelemetry_instrumentation_sdk::find_current_trace_id;
        let trace_id = find_current_trace_id();
        sleep_10ms().await;
        Ok(trace_id)
    }

    #[tracing::instrument]
    pub(crate) async fn get_200_direct() -> impl IntoResponse {
        use tracing_opentelemetry_instrumentation_sdk::find_current_trace_id;
        let trace_id = find_current_trace_id();
        sleep_10ms().await;
        Json(serde_json::json!(trace_id))
    }

    #[tracing::instrument]
    async fn sleep_10ms() {
        tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    }

    #[cfg(test)]
    mod tests {
        use super::*;
        use dioxus::server::axum::{
            body::{to_bytes, Body},
            http::{header::*, Request, StatusCode},
        };
        use pretty_assertions::assert_eq;
        use rstest::*;
        use tower::ServiceExt; // for `oneshot`

        #[rstest]
        #[case("/api/200-direct")]
        #[case("/api/200")]
        #[tokio::test(flavor = "multi_thread")]
        async fn test_header_tracing(#[case] uri: &str) {
            let _guard = init_tracing_opentelemetry::TracingConfig::development()
                .with_global_subscriber(true)
                .with_compact_format()
                .with_log_directives("warn")
                .init_subscriber()
                .unwrap();
            let app = new_router(crate::App).unwrap();
            let response = app
                .oneshot(Request::builder().uri(uri).body(Body::empty()).unwrap())
                .await
                .unwrap();
            let headers = response.headers().clone();
            dbg!(&headers);
            assert_eq!(response.status(), StatusCode::OK);
            assert_eq!(headers.get(CONTENT_TYPE).unwrap(), "application/json");
            //FIXME
            let body = to_bytes(response.into_body(), 64).await.unwrap();
            let body = String::from_utf8(body.to_vec()).unwrap();
            assert_eq!(
                &headers.get("traceparent").unwrap().to_str().unwrap()[3..35],
                &body[1..33]
            );
        }
    }
}
  • launch test
❯ mise run //server:test
[//server:test] $ mkdir -p target/debug/deps/public
[//server:test] $ cargo nextest run --all-features --workspace
   Compiling cdviz-saas v0.2.1 (/mnt/nvme0n1p1/david/src/github.com/cdviz-dev/cdviz-saas/server)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.71s
────────────
 Nextest run ID 68c51663-62f8-478c-85d8-4c99417c8ca6 with nextest profile: default
    Starting 2 tests across 1 binary
        PASS [   0.031s] cdviz-saas::bin/cdviz-saas main_server::tests::test_header_tracing::case_1
        FAIL [   0.033s] cdviz-saas::bin/cdviz-saas main_server::tests::test_header_tracing::case_2
  stdout ───

    running 1 test
    test main_server::tests::test_header_tracing::case_2 ... FAILED

    failures:

    failures:
        main_server::tests::test_header_tracing::case_2

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.03s

  stderr ───
       0.000146885s  WARN main_server::tests::test_header_tracing::case_2 dioxus_server::config:66: No index.html found in public directory, using default index.html
       0.002113288s TRACE main_server::tests::test_header_tracing::case_2 HTTP request: otel::tracing:15: new http.request.method=GET network.protocol.version=1.1 server.address="" user_agent.original="" url.path="/api/200" url.scheme="" otel.name=GET otel.kind=Server span.type=web
       0.015495455s TRACE main_server::tests::test_header_tracing::case_2 HTTP request: otel::tracing:15: close time.busy=319µs time.idle=13.1ms http.request.method=GET network.protocol.version=1.1 server.address="" user_agent.original="" url.path="/api/200" url.scheme="" otel.name=GET otel.kind=Server span.type=web http.route="/api/200" otel.name="GET /api/200" http.response.status_code=200
    [src/main.rs:125:13] &headers = {
        "content-type": "application/json",
        "traceparent": "00-c8e5cdbd12fefc7f65cb47bc7f933a16-bb7ff2d11fea3446-01",
        "tracestate": "",
        "content-length": "4",
    }

    thread 'main_server::tests::test_header_tracing::case_2' (159922) panicked at src/main.rs:133:22:
    byte index 33 is out of bounds of `null`
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

  Cancelling due to test failure:
────────────
     Summary [   0.035s] 2 tests run: 1 passed, 1 failed, 0 skipped
        FAIL [   0.033s] cdviz-saas::bin/cdviz-saas main_server::tests::test_header_tracing::case_2
error: test run failed
[//server:test] ERROR task failed

Or Launch the server dx serve

Call the server function (Failed mismatch between trace_id (in header, second part of traceparent, and body)

❯ curl -i http://127.0.0.1:8080/api/200
HTTP/1.1 200 OK
content-type: application/json
traceparent: 00-716593b9cca26cba621caca4fd0c9b27-8ef08e999afbab13-01
tracestate:
content-length: 34
date: Mon, 03 Nov 2025 15:11:17 GMT
vary: origin, access-control-request-method, access-control-request-headers
access-control-allow-origin: *

"4b39265aa38511b6aa22b5e67cd68b80"

Call the direct axum handler (match of trace_id)

❯ curl -i http://127.0.0.1:8080/api/200-direct
HTTP/1.1 200 OK
content-type: application/json
traceparent: 00-ae6c2cf564ca9695dc11754e58f74ac5-ab343a0cc008cfc7-01
tracestate:
content-length: 34
date: Mon, 03 Nov 2025 15:11:49 GMT
vary: origin, access-control-request-method, access-control-request-headers
access-control-allow-origin: *

"ae6c2cf564ca9695dc11754e58f74ac5"
Image

Expected behavior

Same behavior (tracing span, parenting) between a server function and axum direct handler:

  • no break in the parenting,
  • only one trace_id

Environment:

  • Dioxus version: 0.7.0
  • Rust version: 1.91.0
  • OS info: Linux 6.12.48-1-MANJARO
  • App platform: web

Questionnaire

I'm interested in fixing this myself but don't know where to start.

I'm the maintainer of axum-tracing-opentelemetry, so I can update it if requested.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions