Skip to content

feat(semcov): Add Image Token Count and Cost Semantic Conventions #1787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(semcov): Add Image Token Count and Cost Semantic Conventions #1787

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1d05ce4
add image semantic conventions
sallyannarize Jun 16, 2025
43ff2d8
updates
sallyannarize Jun 18, 2025
256d982
fix sem conv
sallyannarize Jun 19, 2025
08f6583
Update js/packages/openinference-semantic-conventions/src/trace/Seman…
sallyannarize Jun 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions js/.changeset/long-terms-knock.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@arizeai/openinference-semantic-conventions": major
---

Add image semantic conventions
30 changes: 26 additions & 4 deletions js/packages/openinference-semantic-conventions/src/trace/SemanticConventions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,10 @@ export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING =
export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.completion_details.audio` as const;

/** Token count for image output generated by the model (in tokens) */
export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.completion_details.image` as const;

/** Token count for the prompt to the llm (in tokens) */
export const LLM_TOKEN_COUNT_PROMPT =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt` as const;
Expand All @@ -229,21 +233,25 @@ export const LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ =
export const LLM_TOKEN_COUNT_PROMPT_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt_details.audio` as const;

/** Token count for image input presented in the prompt (in tokens) */
export const LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt_details.image` as const;

/** Token count for the entire transaction with the llm (in tokens) */
export const LLM_TOKEN_COUNT_TOTAL =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.total` as const;

/**
* Key prefix for additional prompt token count details. Each detail should be a separate attribute
* with this prefix, e.g. llm.token_count.prompt_details.reasoning, llm.token_count.prompt_details.audio.
* with this prefix, e.g. llm.token_count.prompt_details.reasoning, llm.token_count.prompt_details.audio, llm.token_count.prompt_details.image.
* All values should be in tokens (integer count of tokens).
*/
export const LLM_TOKEN_COUNT_PROMPT_DETAILS =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt_details` as const;

/**
* Key prefix for additional completion token count details. Each detail should be a separate attribute
* with this prefix, e.g. llm.token_count.completion_details.reasoning, llm.token_count.completion_details.audio.
* with this prefix, e.g. llm.token_count.completion_details.reasoning, llm.token_count.completion_details.audio, llm.token_count.completion_details.image.
* All values should be in tokens (integer count of tokens).
*/
export const LLM_TOKEN_COUNT_COMPLETION_DETAILS =
Expand All @@ -258,14 +266,16 @@ export const LLM_TOKEN_COUNT_COMPLETION_DETAILS =
* "completion_details": {
* "output": 0.0009, # Cost in USD
* "reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
* "audio": 0.0012 # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
* "audio": 0.0012, # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
* "image": 0.0018 # Cost in USD (e.g., 60 tokens * $0.03/1K tokens)
* },
* "prompt_details": {
* "input": 0.0003, # Cost in USD
* "cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
* "cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
* "audio": 0.0003 # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "audio": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "image": 0.0009 # Cost in USD (e.g., 30 tokens * $0.03/1K tokens)
* }
* }
* Note: This is a key prefix - individual attributes are stored as separate span attributes with this prefix,
Expand Down Expand Up @@ -306,6 +316,10 @@ export const LLM_COST_COMPLETION_DETAILS_REASONING =
export const LLM_COST_COMPLETION_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.completion_details.audio` as const;

/** Cost of image tokens in the completion in USD */
export const LLM_COST_COMPLETION_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.completion_details.image` as const;

/** Cost of prompt tokens written to cache in USD */
export const LLM_COST_PROMPT_DETAILS_CACHE_WRITE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.cache_write` as const;
Expand All @@ -322,6 +336,10 @@ export const LLM_COST_PROMPT_DETAILS_CACHE_INPUT =
export const LLM_COST_PROMPT_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.audio` as const;

/** Cost of image tokens in the prompt in USD */
export const LLM_COST_PROMPT_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.image` as const;

/**
* The role that the LLM assumes the message is from
* during the LLM invocation
Expand Down Expand Up @@ -638,11 +656,13 @@ export const SemanticConventions = {
LLM_TOKEN_COUNT_COMPLETION_DETAILS,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE,
LLM_TOKEN_COUNT_PROMPT,
LLM_TOKEN_COUNT_PROMPT_DETAILS,
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_WRITE,
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ,
LLM_TOKEN_COUNT_PROMPT_DETAILS_AUDIO,
LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE,
LLM_TOKEN_COUNT_TOTAL,
LLM_SYSTEM,
LLM_PROVIDER,
Expand All @@ -655,10 +675,12 @@ export const SemanticConventions = {
LLM_COST_OUTPUT,
LLM_COST_COMPLETION_DETAILS_REASONING,
LLM_COST_COMPLETION_DETAILS_AUDIO,
LLM_COST_COMPLETION_DETAILS_IMAGE,
LLM_COST_PROMPT_DETAILS_CACHE_WRITE,
LLM_COST_PROMPT_DETAILS_CACHE_READ,
LLM_COST_PROMPT_DETAILS_CACHE_INPUT,
LLM_COST_PROMPT_DETAILS_AUDIO,
LLM_COST_PROMPT_DETAILS_IMAGE,
MESSAGE_ROLE,
MESSAGE_NAME,
MESSAGE_TOOL_CALLS,
Expand Down
141 changes: 92 additions & 49 deletions python/openinference-semantic-conventions/src/openinference/semconv/trace/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,65 +70,88 @@ class SpanAttributes:
"""
The version of the prompt template being used.
"""
LLM_TOKEN_COUNT_COMPLETION = "llm.token_count.completion"
"""
Number of tokens in the completion (in tokens).
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO = "llm.token_count.completion_details.audio"
LLM_TOKEN_COUNT_PROMPT = "llm.token_count.prompt"
"""
The number of audio tokens in the completion (in tokens).
Number of tokens in the prompt.
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING = "llm.token_count.completion_details.reasoning"
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_WRITE = "llm.token_count.prompt_details.cache_write"
"""
Number of tokens used for reasoning steps in the completion (in tokens).
Number of tokens in the prompt that were written to cache (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT = "llm.token_count.prompt"
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ = "llm.token_count.prompt_details.cache_read"
"""
Number of tokens in the prompt.
Number of tokens in the prompt that were read from cache (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS = "llm.token_count.prompt_details"
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_INPUT = "llm.token_count.prompt_details.cache_input"
"""
Key prefix for additional prompt token count details. Each detail should be a separate attribute
with this prefix, e.g. llm.token_count.prompt_details.reasoning,
llm.token_count.prompt_details.audio. All values should be in tokens.
Number of input tokens in the prompt that were cached (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS_AUDIO = "llm.token_count.prompt_details.audio"
"""
The number of audio tokens in the prompt (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_INPUT = "llm.token_count.prompt_details.cache_input"
LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE = "llm.token_count.prompt_details.image"
"""
Number of input tokens in the prompt that were cached (in tokens).
The number of image tokens in the prompt (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ = "llm.token_count.prompt_details.cache_read"

LLM_TOKEN_COUNT_COMPLETION = "llm.token_count.completion"
"""
Number of tokens in the prompt that were read from cache (in tokens).
Number of tokens in the completion (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_WRITE = "llm.token_count.prompt_details.cache_write"
LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING = "llm.token_count.completion_details.reasoning"
"""
Number of tokens in the prompt that were written to cache (in tokens).
Number of tokens used for reasoning steps in the completion (in tokens).
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO = "llm.token_count.completion_details.audio"
"""
The number of audio tokens in the completion (in tokens).
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE = "llm.token_count.completion_details.image"
"""
The number of image tokens in the completion (in tokens).
"""

LLM_TOKEN_COUNT_TOTAL = "llm.token_count.total"
"""
Total number of tokens, including both prompt and completion (in tokens).
"""

LLM_TOKEN_COUNT_PROMPT_DETAILS = "llm.token_count.prompt_details"
"""
Key prefix for additional prompt token count details. Each detail should be a separate attribute
with this prefix, e.g. llm.token_count.prompt_details.reasoning,
llm.token_count.prompt_details.audio. All values should be in tokens.
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS = "llm.token_count.completion_details"
"""
Key prefix for additional completion token count details. Each detail should be a separate
attribute with this prefix, e.g. llm.token_count.completion_details.reasoning,
llm.token_count.completion_details.audio. All values should be in tokens
(integer count of tokens).
"""

LLM_COST_PROMPT = "llm.cost.prompt"
"""
Total cost of all input tokens sent to the LLM in USD. This includes all tokens that were
processed as part of the prompt, including system messages, user messages, and any other input.
"""
LLM_COST_COMPLETION = "llm.cost.completion"
"""
Total cost of all output tokens generated by the LLM in USD. This includes all tokens that were
generated in response to the prompt, including the main response and any additional output.
"""
LLM_COST_COMPLETION_DETAILS = "llm.cost.completion_details"
Copy link
Collaborator

@caroger caroger Jun 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is removing or renaming LLM_COST_COMPLETION_DETAILS intentional?

if so, tests in test_attributes.py should be updated to reflect this change otherwise CI wouldn't pass

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must have accidentally deleted these, I have added them back

LLM_COST_TOTAL = "llm.cost.total"
"""
Key prefix for additional completion cost details. Each detail should be a separate attribute
with this prefix, e.g. llm.cost.completion_details.reasoning,
llm.cost.completion_details.audio. All values should be in USD.
Total cost of the LLM call in USD (prompt + completion).
"""
LLM_COST_COMPLETION_DETAILS_AUDIO = "llm.cost.completion_details.audio"
LLM_COST_INPUT = "llm.cost.input"
"""
Cost of audio tokens in the completion in USD.
Total cost of input tokens in USD. This represents the cost of tokens that were used as
input to the model, which may be different from the prompt cost if there are additional
processing steps.
"""
LLM_COST_COMPLETION_DETAILS_OUTPUT = "llm.cost.completion_details.output"
LLM_COST_OUTPUT = "llm.cost.output"
"""
Total cost of output tokens in USD. This represents the cost of tokens that were generated
as output by the model, which may be different from the completion cost if there are
Expand All @@ -138,42 +161,62 @@ class SpanAttributes:
"""
Cost of reasoning steps in the completion in USD.
"""
LLM_COST_PROMPT = "llm.cost.prompt"
"""
Total cost of all input tokens sent to the LLM in USD. This includes all tokens that were
processed as part of the prompt, including system messages, user messages, and any other input.
"""
LLM_COST_PROMPT_DETAILS = "llm.cost.prompt_details"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comments here about LLM_COST_PROMPT_DETAILS

LLM_COST_COMPLETION_DETAILS_AUDIO = "llm.cost.completion_details.audio"
"""
Key prefix for additional prompt cost details. Each detail should be a separate attribute
with this prefix, e.g. llm.cost.prompt_details.reasoning,
llm.cost.prompt_details.audio. All values should be in USD.
Cost of audio tokens in the completion in USD.
"""
LLM_COST_PROMPT_DETAILS_AUDIO = "llm.cost.prompt_details.audio"
LLM_COST_COMPLETION_DETAILS_IMAGE = "llm.cost.completion_details.image"
"""
Cost of audio tokens in the prompt in USD.
Cost of image tokens in the completion in USD.
"""
LLM_COST_PROMPT_DETAILS_CACHE_INPUT = "llm.cost.prompt_details.cache_input"
LLM_COST_PROMPT_DETAILS_CACHE_WRITE = "llm.cost.prompt_details.cache_write"
"""
Cost of input tokens in the prompt that were cached in USD.
Cost of prompt tokens written to cache in USD.
"""
LLM_COST_PROMPT_DETAILS_CACHE_READ = "llm.cost.prompt_details.cache_read"
"""
Cost of prompt tokens read from cache in USD.
"""
LLM_COST_PROMPT_DETAILS_CACHE_WRITE = "llm.cost.prompt_details.cache_write"
"""
Cost of prompt tokens written to cache in USD.
LLM_COST_PROMPT_DETAILS_CACHE_INPUT = "llm.cost.prompt_details.cache_input"
"""
LLM_COST_PROMPT_DETAILS_INPUT = "llm.cost.prompt_details.input"
Cost of input tokens in the prompt that were cached in USD.
"""
Total cost of input tokens in USD. This represents the cost of tokens that were used as
input to the model, which may be different from the prompt cost if there are additional
processing steps.
LLM_COST_PROMPT_DETAILS_AUDIO = "llm.cost.prompt_details.audio"
"""
LLM_COST_TOTAL = "llm.cost.total"
Cost of audio tokens in the prompt in USD.
"""
Total cost of the LLM call in USD (prompt + completion).
LLM_COST_PROMPT_DETAILS_IMAGE = "llm.cost.prompt_details.image"
"""
Cost of image tokens in the prompt in USD.
"""

LLM_COST = "llm.cost"
"""
Key prefix for cost information. When these keys are transformed into a JSON-like structure,
it would look like:
{
"prompt": 0.0021, # Cost in USD
"completion": 0.0045, # Cost in USD
"total": 0.0066, # Cost in USD
"input": 0.0003, # Cost in USD
"output": 0.0009, # Cost in USD
"completion_details": {
"reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
"audio": 0.0012, # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
"image": 0.0018 # Cost in USD (e.g., 60 tokens * $0.03/1K tokens)
},
"prompt_details": {
"cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"audio": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"image": 0.0009 # Cost in USD (e.g., 30 tokens * $0.03/1K tokens)
}
}
Note: This is a key prefix - individual attributes are stored as separate span attributes
with this prefix, e.g. llm.cost.prompt, llm.cost.completion_details.reasoning, etc.
The JSON structure shown above represents how these separate attributes can be conceptually
organized. All monetary values are in USD with floating point precision.
"""

LLM_TOOLS = "llm.tools"
Expand Down
Loading
Loading