[bug] Traces appear to be broken when used with agent.ainvoke(..) async function in langgraph

[MD-AZMAL]

Describe the bug
Traces appear to be broken when used with agent.ainvoke(..) async function in langgraph

To Reproduce
Run a multistep agent using ainvoke

result = await chat_agent.ainvoke(
                {
                    "question": message.query,
                    "messages": HumanMessage(content=message.query),
                    "observations": [],
                },
                config=config,
            )

Expected behavior
Traces should appear in correct order with proper nesting and aggregation

Screenshots
Without async (using invoke(..))

With async (using ainvoke(..))

Desktop (please complete the following information):

• OS: Windows 11

Additional context
Here is my code for running the agent

async run():
# ....
    with (
        using_attributes(session_id=str(thread_id), user_id=uid)
        if settings.ENABLE_TRACING
        else nullcontext()
    ):

        chat_agent_workflow = AgfiChatV2Agent(memory=checkpointer)

        chat_agent = chat_agent_workflow.agent

        callbacks = []
        if settings.ENABLE_LANGGRAPH_DEBUG:
            callbacks.append(ConsoleCallbackHandler())

        config = {
            "configurable": {
                "thread_id": thread_id,
            },
            "metadata": {"uid": uid, "chat_type": "global"},
            "callbacks": callbacks,
        }

        result = await chat_agent.ainvoke(
                {
                    "question": message.query,
                    "messages": HumanMessage(content=message.query),
                    "observations": [],
                },
                config=config,
            )

        result["final_answer"].sources = []
# ....

Here is the setup for tracing which is run on startup

from openinference.instrumentation.langchain import LangChainInstrumentor
from opentelemetry.instrumentation.asyncio import AsyncioInstrumentor
from phoenix.otel import BatchSpanProcessor, register

from app.config import settings


def setup_tracing_pheonix():

    if not settings.ENABLE_TRACING:
        print("Tracing is disabled")
        return

    # AsyncioInstrumentor().instrument()

    tracer_provider = register(
        project_name="mlb", auto_instrument=True, batch=True, protocol="http/protobuf"
    )

    batch_processor = BatchSpanProcessor(protocol="http/protobuf")
    tracer_provider.add_span_processor(batch_processor)

    LangChainInstrumentor(tracer_provider=tracer_provider).instrument(
        skip_dep_check=True,
    )

    print("Tracing enabled for langgraph with arize phoenix")

Other Information

• I am using fastapi as backend server
• If i simply use the blocking invoke it works perfectly, but on using ainvoke it creates traces that are not grouped and not in order

[dosubot]

Your async traces are not grouped or nested correctly because AsyncioInstrumentor is required for context propagation in async Python code. Without it, OpenTelemetry (and thus OpenInference/Arize Phoenix) can't maintain parent-child relationships between spans in async workflows, which breaks trace grouping for ainvoke and similar async methods. This is a well-documented requirement in both OpenTelemetry and OpenInference code and docs—there is no custom async context patching; instrumenting asyncio is the prescribed approach.

To fix your issue, uncomment and enable AsyncioInstrumentor().instrument() in your tracing setup before any async agent calls. This should restore correct trace order and nesting for async agent runs, matching what you see with the sync invoke method. See the OpenInference code and docstring guidance for more details on why this is necessary.

Also, make sure you are using the latest versions of openinference-instrumentation-agno and related packages, as recent updates (post-2025-08-19) include explicit async context management and fixes for parent-child span relationships in async agent runs (OpenInference PR #2090).

If you are passing config to ainvoke, ensure it is the same config object as for invoke, as missing or mismatched config can also affect trace hierarchy (example).

Let me know if enabling AsyncioInstrumentor resolves your trace grouping!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MD-AZMAL]

The issue persists even after adding AsyncioInstrumentor().instrument() in the tracing setup, and i have updated the installed library but the issue is still persisting

[dosubot]

If traces are still not grouped or nested correctly after enabling AsyncioInstrumentor and updating your libraries, the most common root causes are:

• Not passing the RunnableConfig to ainvoke. Make sure you always call await agent.ainvoke(..., config=config) and that config is the same object as for sync invoke. This is required for correct trace hierarchy in async LangGraph workflows (example).
• Not starting a parent span before invoking the agent. For proper context propagation, explicitly create a parent span using tracer.start_as_current_span before calling ainvoke. This ensures the span context is available and helps maintain trace nesting (details).
• Mixing sync and async nodes in your LangGraph can cause race conditions and break trace grouping. Try making all nodes async if possible (discussion).
• If you use FastAPI, check for any custom middleware, contextvars, or other async frameworks that might interfere with OpenTelemetry context propagation. OpenInference relies on OpenTelemetry's async context propagation and does not add FastAPI-specific logic (reference).

If you want to isolate the issue, try running the minimal FastAPI+LangGraph+Phoenix async tracing example at phoenix-tracing-fastapi-async to see if the problem is environmental or code-specific.

If none of these resolve the issue, enable OpenTelemetry debug logging to check for context propagation failures at runtime. Let me know if you want a code snippet for the parent span setup or if you see any specific errors in your logs.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MD-AZMAL]

the config is passed, and there are no sync operation so far. can you explain how to use tracer.start_as_current_span in a sample code?

[dosubot]

To create a parent span for proper trace grouping in async workflows, use tracer.start_as_current_span as a context manager around your async agent call. Here’s a minimal example:

from opentelemetry.trace import get_tracer

tracer = get_tracer(__name__)

async def run_agent(chat_agent, message, config):
    # Create a parent span for this agent run
    with tracer.start_as_current_span("agent_run"):
        result = await chat_agent.ainvoke(
            {
                "question": message.query,
                "messages": HumanMessage(content=message.query),
                "observations": [],
            },
            config=config,
        )
    return result

This ensures the parent span context is active during the async agent execution, so all child spans (from LangGraph, LLM calls, etc.) are properly nested and grouped in your traces. This pattern works the same way inside FastAPI endpoints or any async function. If you see harmless "Failed to detach context" errors with async generators, you can safely ignore them—they don’t affect tracing functionality (example).

Make sure you use the tracer instance from your configured tracer provider, and that AsyncioInstrumentor is enabled for context propagation. This approach is recommended for both sync and async tracing scenarios (reference).

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MD-AZMAL]

using tacer.start_as_current_span does prevent creating multiple traces, but the span is empty

[dosubot]

The empty parent span you’re seeing after using tracer.start_as_current_span is a known limitation when combining manual span creation with LangChain’s auto-instrumentation: child spans from LangChain (like those created by ainvoke) don’t always nest under manually created parent spans, especially in async workflows. This happens because LangChain’s instrumentation manages its own context and doesn’t reliably attach its spans to a manually created parent, even if OpenTelemetry context propagation is set up correctly reference.

If you’re passing the config to ainvoke and have AsyncioInstrumentor enabled, but still see only an empty parent span, it means the child spans are being created as separate traces or not at all. This is a limitation of how LangChain and OpenInference interact in async contexts reference.

For now, the best approach is to rely on auto-instrumentation (LangChainInstrumentor) and avoid manual parent spans if you want correct nesting. If you need to annotate or group spans, use the config and callback mechanisms provided by LangChain, and ensure all nodes and calls are async. You can also review the minimal FastAPI + LangGraph + Phoenix async tracing example for a working setup reference.

If you need to capture additional metadata, consider setting attributes on the spans created by LangChain itself, rather than relying on manual parent spans. This will ensure your trace hierarchy is preserved and visible in Phoenix.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MD-AZMAL]

@dosu the solutions didnt work, i tried what you mentioned, any specific workaround or guides you could suggest? or anybody in the team who can help?

[dosubot]

Async trace grouping in LangGraph/OpenInference is tricky—manual parent spans (with tracer.start_as_current_span) often show up empty because LangChain’s auto-instrumentation doesn’t reliably nest its child spans under them in async workflows. The best-supported pattern is to rely on auto-instrumentation (just LangChainInstrumentor().instrument(...)) and pass the config object to every ainvoke call; avoid manual parent spans if you want correct nesting reference.

If you’re already doing this and traces are still broken, compare your setup to the minimal working example at phoenix-tracing-fastapi-async—it demonstrates correct async trace grouping with FastAPI, LangGraph, and Phoenix. Sometimes, subtle differences in instrumentation order, config structure, or dependency versions can break context propagation reference.

If you need to annotate or group spans, use LangChain’s config and callback mechanisms, and ensure all nodes and calls are async. For complex graphs, mixing sync and async nodes can cause race conditions and annotation loss—making all nodes async helps, but some edge cases remain reference.

For advanced troubleshooting, contributors like BenMcH, Arize-Labs, jacobmarks, and axiomofjoy are active around async tracing issues. If you can isolate the problem with a minimal reproducible example, sharing it in the repo or discussion threads often gets targeted help.

Let me know if you want help comparing your setup to the example repo or have specific error logs—sometimes the fix is a small config or version tweak.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}