[Entities] Add entity events specification

[dmitryax]

Add a new specification to send entity information as structured log events for communicating richer entity information based on https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/entities/0256-entities-data-model.md

[lmolkova]

I believe these events should be formally defined in semconv where we can validate, document, enforce back compat, etc.

It's still make sense to have some context in the spec, but just link to semconv similarly to what we do for exceptions: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/exceptions.md

[jsuereth]

Entity SIG discussion - We still need to decide if these will be a separate signal or actual events. We're going to hold of on that part of this until that decision is made.

For now this will describe the entity model and usage of events to report entity state.

[tigrannajaryan]

Do we want this to be the only way to communicate entity events in Otel, or this is a way to communicate it using an existing signal (logs), while we continue working on a new native signal to carry entity events?

[tigrannajaryan]

Do we want this to be the only way to communicate entity events in Otel, or this is a way to communicate it using an existing signal (logs), while we continue working on a new native signal to carry entity events?

We discussed in spec call today. @jsuereth's position was that it is possible that in the future we may have entity events represented in a different way (e.g. as a new signal in OTLP).

I think it would be useful to restructure this document slightly to have 2 sections:

1. Entity event logical data model definition. This should be logical definition that is not tied to a particular representation.
2. Representation of entity events as log records. This maps logical definition into a log records as a representation medium.

I would advise to make these 2 sections 2 separate documents, so that in the future if we add a new representation (e.g. OTLP signal) that new representation can refer to the logical data model document only.

As an example see how this docs presents it: https://docs.google.com/document/d/1Tg18sIck3Nakxtd3TFFcIjrmRO_0GLMdHXylVqBQmJA/edit?tab=t.0

[thompson-tomo]

For me, I think this runs the risk of overloading the term state and could lead to confusion especially when you consider status metrics.

Would it be better to describe this as Entity Update Event.

[dmitryax]

The state here doesn't necessarily imply an update. It can also be a heartbeat-like event. See the OTEP section.

That said, it might be helpful to distinguish between events generated by a change vs heartbeat events. @tigrannajaryan @jsuereth WDYT?

[tigrannajaryan]

Update is more vague to me. Does update reflect the full state or the change (delta)? "State" to me says it is the full state, not the delta.

[thompson-tomo]

For me when I hear state I think of things like created, running, paused etc. Which is Effectively the state attribute.

Howabout splitting heartbeat event out & renaming state event to definition as that would be clear that it is the full definition. Alternatively we call it description but suspect links/relationship would need to be an additional event to avoid confusion.

[jsuereth]

I'm more concerned we iron out the meaning of events FIRST then we can find a nomenclature. Once we understand the meaning of signals, we can find terms we're comfortable with.

[thompson-tomo]

If I think about usage of this for something like process.

The expectation would be we send an update/state event to set the process.state attribute to terminated and then send a deleted event. Could this not be consolidated into the update event with a boolean attribute entity.final_state to indicate if has ended.

[dmitryax]

Why not? A delete event can go along with any other state events.

[thompson-tomo]

Understand, hence the idea to simply have a property on the update event to indicate that this is the final state of the entity and all future updates should be ignored. This would reduce the qty of events and it avoids using the term delete to describe that an entity should no longer be tracked

[dmitryax]

To address the use case of the process.state attribute, we can include entity.description in the payload of entity.delete events as well. However, entity.interval and entity.relationships still remain useless for the delete or final state. So I’d rather keep the existing format and add only entity.description to the delete event for backends that want to capture the latest state. Curious what @tigrannajaryan and @jsuereth think about it.

[tigrannajaryan]

Understand, hence the idea to simply have a property on the update event to indicate that this is the final state of the entity and all future updates should be ignored.

Whoever is reporting that the entity is gone via Entity Delete Event may no longer have full state information about the entity that is gone. Which mean it is impossible to correctly report the Entity State Event, which requires full state to be communicated.

[thompson-tomo]

What about if we did the following:

• Add an additional optional property to what is currently the update event. This property would be finalized which is a boolean (default false) or status which is an enum - unset, finalised (default unset) to mirror span status.
• rename delete event to match property name

This way we cater for both use cases.

[jsuereth]

@thompson-tomo Can you outline the use case you want to support here?

I'm with @dmitryax and @tigrannajaryan - I'm not sure we can require description here, and including it may be problematic to implement.

As always, would be nice to see a prototype or use-case where this is a must-have, so we can evaluate design decisions.

[thompson-tomo]

@jsuereth

I'm not sure we can require description here, and including it may be problematic to implement.

As per my previous comment, I am not seeking that change but rather adding an optional property ie finalised (boolean) or status (enum), then renaming this event.

The use case I am thinking of is that we want to update the process entity with the exit time ie process.exit.time. With my suggestion the update event which sets the exit time also marks it as finalized/deleted, without it you would need to send 2 events and hope they are processed in the right order.

[dmitryax]

With my suggestion the update event which sets the exit time also marks it as finalized/deleted, without it you would need to send 2 events and hope they are processed in the right order

I don't see a big problem in sending 2 events. This use case seems to be pretty rare, and it's not a lot of duplication, just the entity ID to be sent twice. And even if they are handled out of order, it must not be an issue for backends:

1. A delete event marks an entity as removed.
2. An update event sets its latest state.

I'd rather keep the events separate for clarity than optimizing for this use case.

[tigrannajaryan]

@dmitryax thanks for the updates. I would also like @jsuereth to reply before we move forward. I think he and I have somewhat different processing models in our mind that aren't necessarily compatible. I would like to converge on a common thinking here.

[jsuereth]

This looks good for us to start building against.