FIX: langfuse_v3 pipeline

[rgaricano]

Revision of the pipeline, clean and fixed some issues.

[YetheSamartaka]

Hi @rgaricano . I tried applying your changes to my variant of the function filter (open-webui/functions#11) but it performed worse as some traces were not traced at all or token counts were wrong. Where did you find that it has to be named "usage_details" instead of just "usage"?

[rgaricano]

@YetheSamartaka,
#580
https://python.reference.langfuse.com/langfuse/api#Usage
https://python.reference.langfuse.com/langfuse/api#Observation.usage_details

Btw, In my PR I removed the "total": input_tokens + output_tokens, because it is done automaticatly by langfuse, (same for "unit": "TOKENS",)

[rgaricano]

@YetheSamartaka,
qq,
do you know if we can have the situation where only one input_tokens|output_tokens are None?

pipelines/examples/filters/langfuse_v3_filter_pipeline.py

Line 312 in 14c2803

if input_tokens is not None and output_tokens is not None:

Maybe when streaming or when some error occurs during generation?
(In those cases the tokens update it's not going to be done)

I updated the PR for handle these cases, also I restored the total field (just in case).

[YetheSamartaka]

Thank you

[YetheSamartaka]

Hi. Perhaps you would want to take a look at this PR of mine as I have the Langfuse as a function. The pipeline itself is not always working correctly, mainly the issue is when user closes the chat window while the chat is being generated, the Langfuse will never get the data properly.
https://github.com/open-webui/functions/pull/11/files#diff-81f33f1799933618031da975bc0b32306c7736ec23b78dc8ca3ca29a6698a1b3