[BUG]: Disabling OpenAIAgent Tracing changes Phoenix Tracing

[brandonbiggs]

Where do you use Phoenix

Local (self-hosted)

What version of Phoenix are you using?

10.0.4

What operating system are you seeing the problem on?

MacOS

What version of Python are you running Phoenix with?

3.12.7

What version of Python or Node are you using instrumentation with?

3.12.7

What instrumentation are you using?

Python

openai                                      1.82.0
openai-agents                               0.0.16
openinference-instrumentation               0.1.31
openinference-instrumentation-openai        0.1.28
openinference-instrumentation-openai-agents 0.1.12
openinference-semantic-conventions          0.1.17
opentelemetry-api                           1.33.1
opentelemetry-exporter-otlp                 1.33.1
opentelemetry-exporter-otlp-proto-common    1.33.1
opentelemetry-exporter-otlp-proto-grpc      1.33.1
opentelemetry-exporter-otlp-proto-http      1.33.1
opentelemetry-instrumentation               0.54b1
opentelemetry-proto                         1.33.1
opentelemetry-sdk                           1.33.1
opentelemetry-semantic-conventions          0.54b1

What happened?

I am trying to uninstrument OpenAIAgents so my traces do not get sent to them. Their documentation indicates that this is enabled by default. I can disable it, but if I do, my traces get sent to Phoenix as ChatCompletions rather than Agent Workflows.

I disabled it by setting the env variable OPENAI_AGENTS_DISABLE_TRACING=1 but this changes how my traces get sent to Phoenix.

What did you expect to happen?

I would have expected my Agent Workflows to get sent to Phoenix the same way regardless of OpenAI's agent tracing being enabled.

How can we reproduce the bug?

I'm not sure how exactly you can reproduce this. Here is my code:

import os
import asyncio

from dotenv import load_dotenv
from pydantic import BaseModel
from openai import AsyncOpenAI
from agents import (
    Agent,
    Model,
    ModelProvider,
    OpenAIChatCompletionsModel,
    RunConfig,
    Runner,
    InputGuardrail,
    GuardrailFunctionOutput,
    exceptions as agent_exceptions
)

load_dotenv(".env")

# Get API Settings from .env
API_KEY = os.getenv("OPENAI_API_KEY")
BASE_URL = os.getenv("BASE_URL")
MODEL_NAME = os.getenv("MODEL")

client = AsyncOpenAI(
    base_url=BASE_URL, 
    api_key=API_KEY,
)

# configure the Phoenix tracer
from phoenix.otel import register
tracer_provider = register(
    project_name="agents",
    endpoint="https://my_url.com/v1/traces",
    auto_instrument=True,
    batch=True
)

class HomeworkOutput(BaseModel):
    is_homework: bool
    reasoning: str

class CustomModelProvider(ModelProvider):
    def get_model(self, model_name: str | None) -> Model:
        return OpenAIChatCompletionsModel(model=model_name or MODEL_NAME, openai_client=client)

async def homework_guardrail(ctx, agent, input_data):
    result = await Runner.run(
        starting_agent=guardrail_agent,
        input=input_data,
        context=ctx.context,
        run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
    )
    final_output = result.final_output_as(HomeworkOutput)
    return GuardrailFunctionOutput(
        output_info=final_output,
        tripwire_triggered=final_output.is_homework,
    )

CUSTOM_MODEL_PROVIDER = CustomModelProvider()

# Define your agents
guardrail_agent = Agent(
    name="Guardrail check",
    instructions="Check if the user is asking about homework.",
    output_type=HomeworkOutput,
)

math_tutor_agent = Agent(
    name="Math Tutor",
    handoff_description="Specialist agent for math questions",
    instructions="You provide help with math problems. Explain your reasoning at each step and include examples",
)

history_tutor_agent = Agent(
    name="History Tutor",
    handoff_description="Specialist agent for historical questions",
    instructions="You provide assistance with historical queries. Explain important events and context clearly.",
)

triage_agent = Agent(
    name="Triage Agent",
    instructions="You determine which agent to use based on the user's homework question",
    handoffs=[history_tutor_agent, math_tutor_agent],
    # input_guardrails=[
    #     InputGuardrail(guardrail_function=homework_guardrail),
    # ],
)

async def main():
    result = await Runner.run(
        starting_agent=triage_agent,
        input="who was the first president of the united states?",
        run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
    )
    print(result.final_output)

if __name__ == "__main__":
    try:
        asyncio.run(main())
    except agent_exceptions.InputGuardrailTripwireTriggered:
        print("Guardrail InputGuardrail triggered tripwire")

This same code produces different results on the Phoenix dashboard if I either set or unset OPENAI_AGENTS_DISABLE_TRACING=1

Additional information

No response

[brandonbiggs]

I believe #1586 is closely related.

[mikeldking]

Thanks for the report @brandonbiggs ! I'm gonna transfer this issue to OpenInference as it seems related to instrumentation. cc @Jgilhuly