Skip to content
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Normalize and enforce LF line endings for all text files
* text=auto eol=lf
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## vNext
- Updates golang to 1.25.1
- Added receiver `swok8sdiscovery` to publish known database entities and relationships

## v0.131.8
- `solarwindsprocessor` Refactored collector decoration configuration: introduced `collector_attributes_decoration` section with `enabled` and `extension` fields, deprecated root-level `extension` field for better configuration organization
Expand Down
1 change: 1 addition & 0 deletions receiver/swok8sdiscovery/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
98 changes: 98 additions & 0 deletions receiver/swok8sdiscovery/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
## swok8sdiscovery Receiver

| Status | |
| ------------- |-----------|
| Stability | [beta]: logs |
| Distributions | [k8s] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/solarwinds/solarwinds-otel-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fswok8sdiscovery%20&label=open&color=orange&logo=opentelemetry)](https://github.com/solarwinds/solarwinds-otel-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fswok8sdiscovery) [![Closed issues](https://img.shields.io/github/issues-search/solarwinds/solarwinds-otel-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fswok8sdiscovery%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/solarwinds/solarwinds-otel-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fswok8sdiscovery) |

[beta]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#beta
[k8s]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-k8s



The `swok8sdiscovery` receiver performs periodic discovery of databases used in a Kubernetes cluster and emits **entity events as OpenTelemetry log records** describing discovered database instances and their relationships to owning Kubernetes workloads (Deployment / StatefulSet / DaemonSet / Job / CronJob).

Discovery currently supports two complementary strategies (both optional – enable either or both):

1. Image-based discovery (`database.image_rules`):
- Matches container image names against user-provided regular expressions.
- Optionally constrains to a single default port when specified.
- Resolves a stable endpoint using the best matching Service (selector overlaps chosen ports) or falls back to the Pod name.
2. Domain-based discovery (`database.domain_rules`):
- Matches `ExternalName` Services whose external DNS name matches configured patterns.
- When multiple rules match, the one whose `database_type` or any of its `domain_hints` appears in either the service name or external domain is preferred.

Each discovered database produces:
* An entity state log (type = `entity_state`) with attributes under `otel.entity.id` identifying the database (`sw.discovery.dbo.address`, `sw.discovery.dbo.type`, `sw.discovery.id`).
* (If workload ownership resolved) A relationship log (type = `entity_relationship_state`) linking the database entity to a Kubernetes workload (relation type `DiscoveredBy`).

### Emitted Attributes (selection)
| Attribute | Description |
|-----------|-------------|
| `otel.entity.event.type` | `entity_state` or `entity_relationship_state` |
| `otel.entity.type` | Always `DiscoveredDatabaseInstance` for entity events |
| `sw.discovery.dbo.address` | Endpoint + (resolved) port list, e.g. `mongo-svc:27017` |
| `sw.discovery.dbo.type` | Logical database type (e.g. `mongo`, `postgres`, `redis`) |
| `sw.discovery.dbo.name` | Endpoint plus workload name (`<endpoint>#<workload>`) when workload present |
| `sw.discovery.source` | Value of configured `reporter` (for provenance) |
| `k8s.<workload kind>.name` | Name of owning workload (when resolved) |
| `k8s.namespace.name` | Namespace of the workload/pod/service |
| `sw.k8s.cluster.uid` | Cluster UID (from environment `CLUSTER_UID`) |

### Configuration

Top-level settings:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `interval` | duration | `5m` | Time between discovery cycles. Shorten in tests (e.g. `15s`). |
| `reporter` | string | empty | Optional source label recorded as `sw.discovery.source`. |
| `k8s` auth fields | (inlined via `APIConfig`) | | Standard Kubernetes client auth (service account, kubeconfig, etc.). |
| `database` | object | nil | Enables database discovery if provided. |

`database.image_rules` entries:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `database_type` | string | yes | Logical database type label. |
| `patterns` | []string (regex) | yes | Regex patterns matched against full container image (e.g. `docker.io/library/mongo:.*`). |
| `default_port` | int | no | If present and exists among container ports, only that port will be emitted (deduping multi-port images). |

`database.domain_rules` entries:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `database_type` | string | yes | Logical database type label. |
| `patterns` | []string (regex) | yes | Patterns matched against `ExternalName` value. |
| `domain_hints` | []string | no | Tie-break hints (substring matches in service name or external domain). |

### Example Configuration

```yaml
receivers:
swok8sdiscovery:
interval: 30s
reporter: "agent"
# Kubernetes auth (service account in-cluster example)
auth_type: serviceAccount
database:
image_rules:
- database_type: mongo
patterns: [".*/mongo:.*"]
default_port: 27017
- database_type: postgres
patterns: [".*/postgres:.*"]
default_port: 5432
domain_rules:
- database_type: redis
patterns: [".*redis.example.com"]

exporters:
debug:
verbosity: detailed

service:
pipelines:
logs:
receivers: [swok8sdiscovery]
exporters: [debug]
```
162 changes: 162 additions & 0 deletions receiver/swok8sdiscovery/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
// Copyright 2025 SolarWinds Worldwide, LLC. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package swok8sdiscovery

import (
"context"
"errors"
"regexp"
"strings"
"time"

"github.com/solarwinds/solarwinds-otel-collector-contrib/internal/k8sconfig"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
k8s "k8s.io/client-go/kubernetes"
)

const (
defaultInterval time.Duration = time.Minute * 5
)

type Config struct {
k8sconfig.APIConfig `mapstructure:",squash"`

Interval time.Duration `mapstructure:"interval"`

Reporter string `mapstructure:"reporter"`

Database *DatabaseDiscoveryConfig `mapstructure:"database"`

// For mocking purposes only.
makeClient func() (k8s.Interface, error)
}

type DatabaseDiscoveryConfig struct {
ImageRules []*ImageRule `mapstructure:"image_rules"`
DomainRules []*DomainRule `mapstructure:"domain_rules"`
}

type ImageRule struct {
DatabaseType string `mapstructure:"database_type"`
// regular expressions patterns to match against container images
Patterns []string `mapstructure:"patterns"`
PatternsCompiled []*regexp.Regexp `mapstructure:"-"` // compiled from Patterns during validation

// default port for database communitation if not specified elsewhere
DefaultPort int32 `mapstructure:"default_port"`
}

type DomainRule struct {
DatabaseType string `mapstructure:"database_type"`
// communication endpoint must match at least one of these patterns
Patterns []string `mapstructure:"patterns"`
PatternsCompiled []*regexp.Regexp `mapstructure:"-"` // compiled from Patterns during validation

// in case more DomainRules match, this one will be preferred to be found in service name or endpoint self
DomainHints []string `mapstructure:"domain_hints"`
}

func (c *Config) Validate() error {
if err := c.APIConfig.Validate(); err != nil {
return err
}

if c.Interval == 0 {
c.Interval = defaultInterval
}

// validate that rules doesn't have databaseType empty
if c.Database != nil {
if err := ValidateDatabaseDiscovery(c.Database); err != nil {
return err
}
}

return nil
}

func ValidateDatabaseDiscovery(databaseDiscovery *DatabaseDiscoveryConfig) error {
for i := range databaseDiscovery.ImageRules {
r := databaseDiscovery.ImageRules[i]
if r.DatabaseType == "" {
return errors.New("database_type must be specified for all image_rules")
}
r.DatabaseType = strings.ToLower(r.DatabaseType)

if len(r.Patterns) == 0 {
return errors.New("at least one match pattern must be specified for all image_rules")
}

r.PatternsCompiled = make([]*regexp.Regexp, len(r.Patterns))
for j, pattern := range r.Patterns {
compiled, err := regexp.Compile(pattern)
if err != nil {
return err
}
r.PatternsCompiled[j] = compiled
}
}

for i := range databaseDiscovery.DomainRules {
r := databaseDiscovery.DomainRules[i]
if r.DatabaseType == "" {
return errors.New("database_type must be specified for all domain_rules")
}
r.DatabaseType = strings.ToLower(r.DatabaseType)
if len(r.Patterns) == 0 {
return errors.New("at least one match pattern must be specified for all domain_rules")
}

r.PatternsCompiled = make([]*regexp.Regexp, len(r.Patterns))
for j, pattern := range r.Patterns {
compiled, err := regexp.Compile(pattern)
if err != nil {
return err
}
r.PatternsCompiled[j] = compiled
}

for j, hint := range r.DomainHints {
r.DomainHints[j] = strings.ToLower(hint)
}
}
return nil
}

func (c *Config) getClient() (k8s.Interface, error) {
if c.makeClient != nil {
return c.makeClient()
}
return k8sconfig.MakeClient(c.APIConfig)
}

// listPods lists all pods across all namespaces using the typed client.
func (c *Config) listPods(ctx context.Context, client k8s.Interface) ([]corev1.Pod, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are listPods and listServices in this file and bound to *Config instead of being in in receiver.go?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just as Client is a part of config, so I placed it here.

pl, err := client.CoreV1().Pods(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}
return pl.Items, nil
}

// listServices lists all services across all namespaces using the typed client.
func (c *Config) listServices(ctx context.Context, client k8s.Interface) ([]corev1.Service, error) {
sl, err := client.CoreV1().Services(metav1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
return nil, err
}
return sl.Items, nil
}
Loading
Loading