Skip to content

[WIP] OTel Bridge API support #1886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 33 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b108750
Add OTel span support
sdaubin May 3, 2024
53a0a96
Documentation
sdaubin May 3, 2024
50ada7a
Add static constructor
sdaubin May 3, 2024
fe873d7
Add test showing the SpanBuilder.setParent doesn't link async work
sdaubin May 3, 2024
8faee60
Implement more of the OTel API for spans
sdaubin May 6, 2024
583a36a
Add a switch to disable custom otel span builder
sdaubin May 7, 2024
bcb9146
Merge remote-tracking branch 'origin' into saxon/replace-otel-spans2
sdaubin May 7, 2024
c7bdc0a
Fix test
sdaubin May 8, 2024
4563453
Log if `otel.exporter.otlp.endpoint` is set
sdaubin May 8, 2024
224947c
Update DefaultTracerTest.java
sdaubin May 8, 2024
6041d13
Allow individual opentelemetry instrumentation scopes to be disabled
sdaubin May 10, 2024
a9029cc
Try to fix test
sdaubin May 10, 2024
67efb93
Use OpenTelemetry no op Span instance
sdaubin May 16, 2024
509ce8a
Fix sql obfuscation of QueryConverter used by otel spans
sdaubin May 21, 2024
4911fc1
Add another external span test
sdaubin May 21, 2024
ba4835a
Merge branch 'main' into saxon/replace-otel-spans2
jasonjkeller Jun 28, 2024
972a16c
Add copyright header and formatting
jasonjkeller Jul 1, 2024
12e2ef4
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Sep 23, 2024
f99b9ed
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Oct 7, 2024
35c91f8
Add some comments
jasonjkeller Oct 29, 2024
0750c0b
Merge main
jasonjkeller Nov 14, 2024
c308a60
Add readme for OTel functionality
jasonjkeller Dec 2, 2024
61bf958
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Jan 6, 2025
34ac22a
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Mar 10, 2025
df75b02
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Mar 13, 2025
36a4c8f
Additional unit tests
jtduffy Mar 28, 2025
9a1c6a4
Fix broken trace propagation with w3c headers
jasonjkeller Apr 4, 2025
a7bd252
Cleanup
jasonjkeller Apr 4, 2025
6199e9a
Merge pull request #2299 from newrelic/fix-broken-trace-propagation
jasonjkeller Apr 7, 2025
1eafb84
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Apr 7, 2025
d7ab6b6
Add assertions on OTel span data
jasonjkeller Apr 28, 2025
f15b8ee
Merge pull request #2338 from newrelic/otel-cross-agent-tests
jasonjkeller Apr 28, 2025
01ccb3e
Merge branch 'main' of github.com:newrelic/newrelic-java-agent into s…
jasonjkeller Jul 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

package com.newrelic.agent.bridge;

import com.newrelic.api.agent.Token;

import java.lang.reflect.InvocationHandler;


Expand All @@ -19,10 +21,19 @@ public interface ExitTracer extends InvocationHandler, TracedMethod {
*/
void finish(int opcode, Object returnValue);

default void finish() {
// 177 is Opcodes.RETURN
finish(177, null);
}

/**
* Called when a method invocation throws an exception.
*
* @param throwable
*/
void finish(Throwable throwable);

default Token getToken() {
return NoOpToken.INSTANCE;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@

import com.newrelic.api.agent.NewRelic;
import com.newrelic.api.agent.Token;
import com.newrelic.api.agent.Trace;

public interface Instrumentation {

Expand Down Expand Up @@ -59,6 +58,10 @@ ExitTracer createTracer(Object invocationTarget, int signatureId, boolean dispat

ExitTracer createScalaTxnTracer();

default ExitTracer createTracer(String metricName, int flags) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The createSegment API still generates an asynchronous tracer, which is not what we want. And from the bridge there's no easy way to register a signature to get a signatureId. This was the easiest way to get a normal, synchronous tracer.

return null;
}

/**
* Returns the current transaction. This should not be called directly - instead use {@link Agent#getTransaction()}.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@
*/
public interface TracedMethod extends com.newrelic.api.agent.TracedMethod {

default String getTraceId() {
return "0000000000000000";
}
default String getSpanId() {
return "0000000000000000";
}
/**
* Returns the parent of this traced method, or null if this is the root tracer.
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/*
*
* * Copyright 2024 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/

package com.newrelic.agent.bridge.datastore;

import com.newrelic.api.agent.QueryConverter;

public final class SqlQueryConverter implements QueryConverter<String> {
public static final QueryConverter<String> INSTANCE = new SqlQueryConverter();

private SqlQueryConverter() {
}

@Override
public String toRawQueryString(String rawQuery) {
return rawQuery;
}

@Override
public String toObfuscatedQueryString(String rawQuery) {
return null;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
import java.util.Map;

public interface SpanEvent {
String getGuid();

String getName();

float duration();
Expand All @@ -33,4 +35,6 @@ public interface SpanEvent {
String getStatusText();

Map<String, Object> getAgentAttributes();

Map<String, Object> getUserAttributes();
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ public SpanEventImpl(com.newrelic.agent.model.SpanEvent spanEvent) {
this.spanEvent = spanEvent;
}

/**
* This method is just to facilitate testing
*
* @return String representing Span ID
*/
@Override
public String getGuid() {
return spanEvent.getGuid();
}

@Override
public String getName() {
return spanEvent.getName();
Expand Down Expand Up @@ -74,9 +84,13 @@ public String getStatusText() {
return (String) spanEvent.getAgentAttributes().get("http.statusText");
}


@Override
public Map<String, Object> getAgentAttributes() {
return spanEvent.getAgentAttributes();
}

@Override
public Map<String, Object> getUserAttributes() {
return spanEvent.getUserAttributesCopy();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# OpenTelemetry Instrumentation

This instrumentation module weaves parts of the OpenTelemetry SDK to incorporate bits of OpenTelemetry functionality into the New Relic Java agent.

Specifically it can:
* Detect OpenTelemetry Spans and include them in New Relic Java agent traces.
* Detect OpenTelemetry dimensional metrics and report them to the APM entity being monitored by the Java agent.
* Autoconfigure the OpenTelemetry SDK so that OpenTelemetry data is sent to New Relic and properly associated with an APM entity guid.

## New Relic Java Agent Configuration

To use the OpenTelemetry Span and dimensional metric functionality incorporated into the New Relic Java agent you must enable the following config options:

Configuration via yaml:
```
opentelemetry:
sdk:
autoconfigure:
enabled: true
spans:
enabled: true
```

Configuration via system property:
```
-Dopentelemetry.sdk.autoconfigure.enabled=true
-Dopentelemetry.sdk.spans.enabled=true
```

Configuration via environment variable:
```
NEW_RELIC_OPENTELEMETRY_SDK_AUTOCONFIGURE_ENABLED=true
NEW_RELIC_OPENTELEMETRY_SDK_SPANS_ENABLED=true
```

## OpenTelemetry Dimensional Metrics

OpenTelemetry APIs can be used to create dimensional metrics which will be detected by the New Relic Java agent and reported to the APM entity being monitored by the New Relic Java agent.

To use this functionality, enable the feature as documented above, add the required `opentelemetry` dependencies to your application:
```groovy
implementation(platform("io.opentelemetry:opentelemetry-bom:1.44.1"))
implementation("io.opentelemetry:opentelemetry-sdk-extension-autoconfigure")
implementation("io.opentelemetry:opentelemetry-exporter-otlp")
```

Then utilize the OpenTelemetry APIs to record dimensional metrics:
```java
LongCounter longCounter = GlobalOpenTelemetry.get().getMeterProvider().get("my-application").counterBuilder("my.application.counter").build();
longCounter.add(1, Attributes.of(AttributeKey.stringKey("foo"), "bar"));
```

Any recorded dimensional metrics can be found in the Metrics Explorer for the associated APM entity and can be used to build custom dashboards.

## OpenTelemetry Spans

Documented below are several approaches for incorporating OpenTelemetry Spans into New Relic Java agent traces.

### `@WithSpan` Annotation

The New Relic Java agent will detect usage of the OpenTelemetry [@WithSpan](https://opentelemetry.io/docs/zero-code/java/agent/annotations/) annotation. The `@WithSpan` annotation can be used as an alternative to the `@Trace` annotation.

This does not currently support the following config options:
* [Suppressing @WithSpan instrumentation](https://opentelemetry.io/docs/zero-code/java/agent/annotations/#suppressing-withspan-instrumentation)
* [Creating spans around methods with otel.instrumentation.methods.include](https://opentelemetry.io/docs/zero-code/java/agent/annotations/#creating-spans-around-methods-with-otelinstrumentationmethodsinclude)

Note that OpenTelemetry config properties can be set through environment or system properties, like our agent, and eventually through a config file. We can use our existing OpenTelemetry instrumentation model to get access to the normalized version of the instrumentation settings to include and exclude methods and pass those to the core agent through the bridge.

See `ClassTransformerConfigImpl.java` for implementation details of the `@WithSpan` annotation.

### Spans Emitted From OpenTelemetry Instrumentation

The New Relic Java agent will detect Spans emitted by [OpenTelemetry instrumentation](https://opentelemetry.io/docs/languages/java/instrumentation/). It does this by weaving the `io.opentelemetry.sdk.trace.SdkTracerProvider` so that it will create a New Relic Tracer each time an OpenTelemetry Span is started and weaving the `io.opentelemetry.context.Context` to propagate context between New Relic and OpenTelemetry Spans.

Currently, the New Relic Java agent does not load any OpenTelemetry instrumentation it simply detects Spans emitted by OpenTelemetry manual instrumentation, native instrumentation, library instrumentation, or zero code instrumentation (i.e. bytecode instrumentation that would also require running the OpenTelemetry Java agent).

Depending on the OpenTelemetry Span `SpanKind`, it may result in the New Relic Java agent starting a transaction (when one doesn't already exist).

* `SpanKind.INTERNAL`
* Creating a span with no `SpanKind`, which defaults to `SpanKind.INTERNAL`, will not start a transaction
* If `SpanKind.INTERNAL` spans occur within an already existing New Relic transaction they will be included in the trace
* `SpanKind.CLIENT`
* Creating a span with `SpanKind.CLIENT` will not start a transaction. If a `CLIENT` span has certain db attributes it will be treated as a DB span, and other specific attributes will cause it to be treated as an external span
* If `SpanKind.CLIENT` spans occur within an already existing New Relic transaction they will be included in the trace
* `SpanKind.SERVER`
* Creating a span with `SpanKind.SERVER` will start a `WebTransaction/Uri/*` transaction.
* If `SpanKind.SERVER` spans occur within an already existing New Relic transaction they will be included in the trace
* `SpanKind.CONSUMER`
* Creating a span with `SpanKind.CONSUMER` will start a `OtherTransaction/*` transaction.
* If `SpanKind.CONSUMER` spans occur within an already existing New Relic transaction they will be included in the trace
* `SpanKind.PRODUCER`
* Creating a span with `SpanKind.PRODUCER` will not start a transaction. There is no explicit processing for `PRODUCER` spans currently.
* If `SpanKind.PRODUCER` spans occur within an already existing New Relic transaction they will be included in the trace (though it's effectively no different from a `SpanKind.INTERNAL` span)
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ dependencies {
implementation(project(":newrelic-weaver-api"))
implementation("io.opentelemetry:opentelemetry-sdk-extension-autoconfigure:1.28.0")
testImplementation("junit:junit:4.12")
testImplementation("io.opentelemetry:opentelemetry-exporter-otlp:1.28.0")
testImplementation("io.opentelemetry.instrumentation:opentelemetry-instrumentation-annotations:1.28.0")
testImplementation("com.google.guava:guava:30.1.1-jre")
}

jar {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
/*
*
* * Copyright 2025 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/

package com.nr.agent.instrumentation.header.utils;

public class HeaderType {
public static final String NEWRELIC = "newrelic";
public static final String W3C_TRACEPARENT = "traceparent";
public static final String W3C_TRACESTATE = "tracestate";
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
*
* * Copyright 2025 New Relic Corporation. All rights reserved.
* * SPDX-License-Identifier: Apache-2.0
*
*/

package com.nr.agent.instrumentation.header.utils;

import io.opentelemetry.api.trace.SpanContext;

public class W3CTraceParentHeader {

static final String W3C_VERSION = "00";
static final String W3C_TRACE_PARENT_DELIMITER = "-";

public static String create(SpanContext parentSpanContext) {
final String traceId = parentSpanContext.getTraceId();
final String spanId = parentSpanContext.getSpanId();
final boolean sampled = parentSpanContext.isSampled();

String traceParentHeader =
W3C_VERSION + W3C_TRACE_PARENT_DELIMITER + traceId + W3C_TRACE_PARENT_DELIMITER + spanId + W3C_TRACE_PARENT_DELIMITER + sampledToFlags(sampled);

boolean valid = W3CTraceParentValidator.forHeader(traceParentHeader)
.version(W3C_VERSION)
.traceId(traceId)
.parentId(spanId)
.flags(parentSpanContext.getTraceFlags().asHex())
.isValid();

return valid ? traceParentHeader : "";
}

private static String sampledToFlags(boolean sampled) {
return sampled ? "01" : "00";
}
}
Loading
Loading