Skip to content

[BUG]: Karafka's "worker.process" trace is lost after iterating the messages (when distributed tracing is enabled) #4873

@Drowze

Description

@Drowze

Tracer Version(s)

2.19.0

Ruby Version(s)

ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin23]

Relevent Library and Version(s)

karafka 2.5.0

Bug Report

When using the Karafka integration with distributed_tracing, the original trace is lost once we start iterating through the messages.

Reproduction Code

# karafka.rb
class KarafkaApp < Karafka::App
  setup do |config|
    config.kafka = { 'bootstrap.servers': '127.0.0.1:9092' }
  end

  routes.draw do
    topic :example do
      consumer ExampleConsumer
    end
  end
end

# app/consumers/example_consumer.rb
class ExampleConsumer < ApplicationConsumer
  def consume
    log("before #each")

    messages.each do |message|
      log("inside #each")
      puts "doing some hard work!"
    end

    log("after #each")
  end

  def log(description)
    trace = Datadog::Tracing.active_trace
    span = Datadog::Tracing.active_span

    data = {id: trace&.id, name: trace&.name, resource: trace&.resource, parent_span_id: trace&.parent_span_id}
    puts "[consumer - #{description}] trace: #{data}"

    data = {id: span&.id, name: span&.name, resource: span&.resource, parent_id: span&.parent_id}
    puts "[consumer - #{description}] span: #{data}"
  end
end

# producing messages in a terminal
["msg1", "msg2"].each do |msg|
  Datadog::Tracing.trace("producer.#{msg}") do
    digest = Datadog::Tracing.active_trace.to_digest
    headers = {}
    Datadog::Tracing::Contrib::Karafka.inject(digest, headers)
    Karafka.producer.produce_async({
      topic: "example", payload: { foo: msg }.to_json, headers: headers
    })
  end
end

Result of the above code (with some colors to help visualizing). Note that the consumer - after #each lines don't have an active trace/span.
Image

Configuration Block

Datadog.configure do |c|
  c.tracing.instrument :karafka, distributed_tracing: true, enabled: true
end

Error Logs

No response

Operating System

No response

How does Datadog help you?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugInvolves a bugcommunityWas opened by a community member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions