-
Notifications
You must be signed in to change notification settings - Fork 788
feat: adding driver to export to disk #3832
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: adding driver to export to disk #3832
Conversation
Working on docs still. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3832 +/- ##
==========================================
- Coverage 54.49% 48.04% -6.46%
==========================================
Files 134 236 +102
Lines 12329 20124 +7795
==========================================
+ Hits 6719 9669 +2950
- Misses 5116 9544 +4428
- Partials 494 911 +417
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
pkg/export/disk/diskwriter.go
Outdated
if path, ok := cfg["path"].(string); ok { | ||
if conn.Path != path { | ||
if err := os.RemoveAll(conn.Path); err != nil { | ||
cleanUpErr = fmt.Errorf("connection updated but failed to remove content form old path: %w", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be an error or log?
pkg/audit/manager.go
Outdated
@@ -280,17 +259,28 @@ func (am *Manager) audit(ctx context.Context) error { | |||
timestamp := startTime.UTC().Format(time.RFC3339) | |||
am.log = log.WithValues(logging.AuditID, timestamp) | |||
logStart(am.log) | |||
exportErrorMap := make(map[string]error) | |||
if err := am.exportSystem.Publish(context.Background(), *auditConnection, *auditChannel, exportutil.ExportMsg{Message: "audit is started", ID: timestamp}); err != nil { | |||
exportErrorMap[err.Error()] = err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can lead to memory consumption if by any chance driver returns unique error for all violations instead of returning one. Instead, we can modify returned error type to ExportError and use error type as keys to limit the number of errors we are storing in map. Do folks think this is a way to go about limiting the number of errors stored? I am open to other suggestions.
I will make the change if we agree to use ExportError
because that may need updating interface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed, your suggestion make sense. if we want to avoid interface changes, another idea might be to extract error category by checking with regex or before :
for example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Parsing the error message and using the string before :
as key to store errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Overview
This PR adds a new disk export driver for persisting audit violations to the file system and updates related configuration, tests, and documentation.
- Introduces a new disk driver under pkg/export/disk.
- Adds GitHub workflow and test configuration to exercise the disk export.
- Updates documentation to guide users through configuring and using the disk export driver.
Reviewed Changes
File | Description |
---|---|
.github/workflows/disk-export.yaml | Workflow definition to run disk export tests |
test/export/fake-reader/.yaml,.go | Test configuration and a fake-reader to simulate exports |
website/docs/export.md | Documentation updates for the disk export driver |
pkg/export/disk/disk.go | New disk export driver implementation |
pkg/export/util/util.go | Utility types updated for export messages |
pkg/audit/manager.go | Audit manager updates to integrate disk export; error map |
pkg/export/system.go | Updated export system to include the disk driver |
Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (2)
pkg/export/disk/disk.go:69
- The variable 'err' is never updated in CreateConnection; consider returning 'nil' explicitly to avoid confusion.
return err
pkg/audit/manager.go:263
- [nitpick] Consider defining and using constants for the audit trigger messages ("audit is started" and "audit is completed") to avoid typos and ensure consistency across the codebase.
am.exportSystem.Publish(context.Background(), *auditConnection, *auditChannel, exportutil.ExportMsg{Message: "audit is started", ID: timestamp})
pkg/export/disk/disk.go
Outdated
return fmt.Errorf("missing or invalid values in config for connection: %s", connectionName) | ||
} | ||
var err error | ||
maxResults, maxResultsOk := cfg["maxAuditResults"].(float64) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why float?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because the inherent value is stored in float64
when the object is getting unpacked, so checking and converting to int64
does not work here.
.github/workflows/disk-export.yaml
Outdated
egress-policy: audit | ||
|
||
- name: Check out code into the Go module directory | ||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 | |
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 |
pkg/export/disk/disk.go
Outdated
maxAllowedAuditRuns = 5 | ||
) | ||
|
||
const ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: merge these
test/export/fake-reader/Dockerfile
Outdated
ARG BUILDERIMAGE="golang:1.22-bookworm" | ||
ARG BASEIMAGE="gcr.io/distroless/static-debian12:nonroot" | ||
|
||
FROM --platform=$BUILDPLATFORM $BUILDERIMAGE as builder |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's add it here instead of using an arg, and dependabot can bump it. please make sure its covered in dependabot config
test/export/fake-reader/Dockerfile
Outdated
|
||
RUN go build -o main | ||
|
||
FROM $BASEIMAGE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
pkg/export/disk/disk.go
Outdated
return fmt.Errorf("invalid data type, cannot convert data to exportMsg") | ||
} | ||
|
||
if violation.Message == "audit is started" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we use a const for these to avoid issues?
pkg/export/disk/disk.go
Outdated
return fmt.Errorf("error writing message to disk: %w", err) | ||
} | ||
|
||
if violation.Message == "audit is completed" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
pkg/export/disk/disk.go
Outdated
return fmt.Errorf("missing or invalid 'path' for connection: %s", connectionName) | ||
} | ||
|
||
if maxResults, ok := cfg["maxAuditResults"].(float64); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const these?
pkg/export/disk/disk.go
Outdated
} | ||
|
||
var cleanUpErr error | ||
if path, ok := cfg["path"].(string); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here
website/docs/export.md
Outdated
2. Update `gatekeeper-audit` deployment to add `emptyDir` volume. | ||
|
||
```yaml | ||
volumes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we want to add this to helm chart with conditional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found it to be more complecated to add a whole sidecar container and its config because unlike adding dapr annotation which adds a container with specification required for dapr sidecar to run. A sidecar reader for purposes beyond just reading and logging would require specific customizations. We can add it but the chart will get more complicated and I do not see the value since we might not be able to make it completly customizable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to add helm variables for volume
, volumeMount
and sidecar
as well.
website/docs/export.md
Outdated
|
||
```yaml | ||
... | ||
- --enable-violation-export=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same for these?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as above comment.
c99e491
to
68a0110
Compare
website/docs/export.md
Outdated
You can use below command that uses a rule defined in [Makefile](https://github.com/open-policy-agent/gatekeeper/blob/master/Makefile) to deploy gatekeeper that mounts emptyDir with sidecar reader container. | ||
|
||
|
||
make deploy IMG=gatekeeper-e2e:latest IMG=<gatekeeper_image> EXPORT_BACKEND=disk FAKE_READER_IMAGE=<your_reader_image> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i am confused, isn't this going to add the sidecar to audit automatically? instructions in the prereqs are saying to do this manually
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Made it more clear a bit.
website/docs/export.md
Outdated
3. Create connection config to establish a connection. | ||
|
||
```shell | ||
kubectl apply -f - <<EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this might be okay on the website, but copy pasting this have some formatting issues
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah this is happening due to indentation. I tested copying from website and I am not seeing any formatting issues.
|
||
```log | ||
kubectl logs -l gatekeeper.sh/operation=audit -c go-sub -n gatekeeper-system | ||
2025/03/05 00:37:16 {"id":"2025-03-05T00:37:13Z","details":{"missing_labels":["test"]},"eventType":"violation_audited","group":"constraints.gatekeeper.sh","version":"v1beta1","kind":"K8sRequiredLabels","name":"pod-must-have-test","message":"you must provide labels: {\"test\"}","enforcementAction":"deny","resourceAPIVersion":"v1","resourceKind":"Pod","resourceNamespace":"nginx","resourceName":"nginx-deployment-2-79479fc6db-7qbnm","resourceLabels":{"app":"nginx-ingress","app.kubernetes.io/component":"controller","pod-template-hash":"79479fc6db"}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am seeing this \u003
, not sure if this is something on my side
2025/03/24 04:25:27 {"id":"2025-03-24T04:24:27Z","details":{},"eventType":"violation_audited","group":"constraints.gatekeeper.sh","version":"v1beta1","kind":"K8sAllowedRepos","name":"repo-is-mcr","message":"container \u003ccoredns\u003e has an invalid image repo \u003cregistry.k8s.io/coredns/coredns:v1.11.3\u003e, allowed repos are [\"quay.io/\"]","enforcementAction":"dryrun","resourceAPIVersion":"v1","resourceKind":"Pod","resourceNamespace":"kube-system","resourceName":"coredns-668d6bf9bc-57xm9","resourceLabels":{"k8s-app":"kube-dns","pod-template-hash":"668d6bf9bc"}}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I have not run into similar issues. How are you testing this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using the instructions here on a kind cluster. might be on my side, unless @ritazh can repro too
pkg/audit/manager.go
Outdated
@@ -280,17 +259,28 @@ func (am *Manager) audit(ctx context.Context) error { | |||
timestamp := startTime.UTC().Format(time.RFC3339) | |||
am.log = log.WithValues(logging.AuditID, timestamp) | |||
logStart(am.log) | |||
exportErrorMap := make(map[string]error) | |||
if err := am.exportSystem.Publish(context.Background(), *auditConnection, *auditChannel, exportutil.ExportMsg{Message: "audit is started", ID: timestamp}); err != nil { | |||
exportErrorMap[err.Error()] = err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed, your suggestion make sense. if we want to avoid interface changes, another idea might be to extract error category by checking with regex or before :
for example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with a few more suggestions
website/docs/export.md
Outdated
|
||
```yaml | ||
audit: | ||
exportVolume: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indent issue i think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, fixed it.
|
||
```log | ||
kubectl logs -l gatekeeper.sh/operation=audit -c go-sub -n gatekeeper-system | ||
2025/03/05 00:37:16 {"id":"2025-03-05T00:37:13Z","details":{"missing_labels":["test"]},"eventType":"violation_audited","group":"constraints.gatekeeper.sh","version":"v1beta1","kind":"K8sRequiredLabels","name":"pod-must-have-test","message":"you must provide labels: {\"test\"}","enforcementAction":"deny","resourceAPIVersion":"v1","resourceKind":"Pod","resourceNamespace":"nginx","resourceName":"nginx-deployment-2-79479fc6db-7qbnm","resourceLabels":{"app":"nginx-ingress","app.kubernetes.io/component":"controller","pod-template-hash":"79479fc6db"}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using the instructions here on a kind cluster. might be on my side, unless @ritazh can repro too
pkg/export/disk/disk.go
Outdated
if err != nil { | ||
return fmt.Errorf("failed to acquire lock: %w", err) | ||
} | ||
log.Info("Writing latest violations at ") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this missing something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed the log statement,
pkg/export/disk/disk.go
Outdated
dir := path.Join(conn.Path, topic) | ||
if err := os.MkdirAll(dir, 0o755); err != nil { | ||
return fmt.Errorf("failed to create directories: %w", err) | ||
} | ||
|
||
file, err := os.OpenFile(path.Join(dir, appendExtension(conn.currentAuditRun, "txt")), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) | ||
if err != nil { | ||
return fmt.Errorf("failed to open file: %w", err) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Readers might want to truncate and remove files after processing, but this seems to only grant write if the sidecar container runs as the same user id of the main container. But sidecars might need to run as root or other users. How can we support if sidecar needs to run as root, should we handle this in the permissions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed the permissions issue by setting file and dir permissions to allow reader to modify the same if needed.
website/docs/export.md
Outdated
|
||
```shell | ||
helm upgrade --install gatekeeper gatekeeper/gatekeeper --namespace gatekeeper-system \ | ||
... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed it
website/docs/export.md
Outdated
|
||
#### Configure Gatekeeper with Export enabled to Disk | ||
|
||
1. Build `fake-reader` image from [gatekeeper/test/export/fake-reader](https://github.com/open-policy-agent/gatekeeper/tree/master/test/export/fake-reader) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of having users do this, can this image be published as one of the Gatekeeper images? built/released/tagged like other GK images.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify, I'm suggesting to add this as part of the makefile and publish it to the registry so we can use it as a default value in the values.yaml. this is to ensure users have an easy way to test things. they can always replace with their own image.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Published fake-reader image on ghcr - https://github.com/open-policy-agent/gatekeeper/pkgs/container/fake-reader. This should improve experience trying out this feature with defaults for the first time.
website/docs/export.md
Outdated
1. Build `fake-reader` image from [gatekeeper/test/export/fake-reader](https://github.com/open-policy-agent/gatekeeper/tree/master/test/export/fake-reader) | ||
|
||
```bash | ||
docker buildx build -t <your_img_name:tag> --load -f test/export/fake-reader/Dockerfile test/export/fake-reader |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be part of the makefile like docker-buildx-crds
website/docs/export.md
Outdated
<your-side-car> | ||
``` | ||
|
||
Here is the default `values.yaml` that you can use. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can this be deployed as part of the chart but conditional on a field in values.yaml? e.g. audit.enableExportToDisk
website/docs/export.md
Outdated
--values /path/to/values.yaml | ||
``` | ||
|
||
4. Create connection config to establish a connection. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same, can this be part of the chart but conditionally install only if the feature is enabled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added this to helm charts.
@@ -905,7 +902,7 @@ func (am *Manager) addAuditResponsesToUpdateLists( | |||
if *exportController.ExportEnabled { | |||
err := am.exportSystem.Publish(context.Background(), *auditConnection, *auditChannel, violationMsg(constraint, ea, r.ScopedEnforcementActions, gvk, namespace, name, msg, details, labels, timestamp)) | |||
if err != nil { | |||
am.log.Error(err, "error exporting audit violation") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should still be logged in the audit pod log
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was removed to prevent flood of logs with the same error for each violation in favor of only logging unique errors - https://github.com/open-policy-agent/gatekeeper/pull/3832/files#diff-2a233457fc5c89cf0dd91c044d7582aa1cc0c835feabf4638882ac54ba3ca63aR286.
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
.github/workflows/disk-export.yaml
Outdated
timeout-minutes: 15 | ||
steps: | ||
- name: Harden Runner | ||
uses: step-security/harden-runner@cb605e52c26070c328afc4562f0b4ada7618a84e # v2.10.4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please double check these GitHub action versions to ensure they are the latest
| audit.connection | (alpha) Connection name for exporting audit violation messages | `audit-connection` | | ||
| audit.channel | (alpha) Channel name for exporting audit violation messages | `audit-channel` | | ||
| audit.exportVolume | (alpha) Volume for audit pod to export violations. | `{"name":"tmp-violations","emptyDir":{}}` | | ||
| audit.exportVolumeMount.path | (alpha) VolumeMount for audit pod manager container to export violations and sidecar container to read from. | `/tmp/violations` | | ||
| audit.exportSidecar | (alpha) Sidecar container to read violations from disk. | `{"name":"reader","image":"ghcr.io/open-policy-agent/fake-reader:latest","imagePullPolicy":"Always","securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsGroup":999,"runAsNonRoot":true,"runAsUser":1000,"seccompProfile":{"type":"RuntimeDefault"}},"volumeMounts":[{"mountPath":"/tmp/violations","name":"tmp-violations"}]}` | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where is the code that pushes ghcr.io/open-policy-agent/fake-reader
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now I have pushed them manually before building a CI for that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a follow up, please add this to CI. I can see maybe it could have CVEs in the future that we may need to patch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 This should be added for all sample images as follow up.
test/export/fake-reader/Dockerfile
Outdated
@@ -0,0 +1,31 @@ | |||
FROM --platform=$BUILDPLATFORM golang:1.23-bookworm@sha256:462f68e1109cc0415f58ba591f11e650b38e193fddc4a683a3b77d29be8bfb2c AS builder |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please check this is the latest
test/export/fake-reader/Dockerfile
Outdated
|
||
RUN go build -o main | ||
|
||
FROM gcr.io/distroless/static-debian12@sha256:8dd8d3ca2cf283383304fd45a5c9c74d5f2cd9da8d3b077d720e264880077c65 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same
1. Deploy Gatekeeper with disk export configurations. | ||
|
||
```shell | ||
helm upgrade --install gatekeeper gatekeeper/gatekeeper --namespace gatekeeper-system \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we move this step after user has updated their values yaml?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure, in quickstart steps user doesnt need to create values.yaml to try it out. So the yaml of variables in values
is just a guide and not a requirement for users to try this. By that flow I think this is fine 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldnt encourage users to use the fake-reader container image in production. IMO we should move this up and then add a warning to replace fake-reader for any production use since its just for demo purpose. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yeah from that point of view moving it down makes sense. I am updating docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated docs to move up default values before installing Gk with added warning about production use.
website/docs/export.md
Outdated
--set audit.channel=audit-channel \ | ||
--set audit.exportConfig.maxAuditResults=3 \ | ||
--set exportBackend=disk \ | ||
--set audit.exportSidecar.image=ghcr.io/open-policy-agent/fake-reader:latest \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since this is the default value, should this be removed assuming users have updated their values.yaml or add a comment here to --set audit.exportSidecar.image
to override the default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, removed these values.
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces a new disk driver for exporting audit violations to disk and integrates it into the existing export system. Key changes include:
- A new disk driver implementation (pkg/export/disk/disk.go) with connection, publishing, and file handling logic.
- Integration of the disk driver into the supported drivers map and updates to export configuration and audit manager.
- Updates to Helm charts and documentation to support configuration of the disk export backend.
Reviewed Changes
Copilot reviewed 20 out of 26 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
pkg/export/util/util.go | Added export message and error types used by both drivers and audit manager. |
pkg/export/system.go | Included the disk driver into the SupportedDrivers map. |
pkg/export/disk/disk.go | New implementation of the disk driver with file and connection management. |
pkg/controller/export/export_config_controller.go | Updated to lower-case the driver value for consistency. |
pkg/audit/manager.go | Propagated export error mapping and adjusted audit response handling. |
manifest_staging/charts/gatekeeper/*.yaml and README.md | Updated configuration values and documentation for the disk export backend. |
cmd/build/helmify/* | Adjusted templating to support the export disk configuration. |
.github/workflows/*.yaml | Added CI jobs to run tests for the disk export functionality. |
.github/dependabot.yml | Added updates for docker package ecosystems related to export testing. |
Files not reviewed (6)
- Makefile: Language not supported
- cmd/build/helmify/static/templates/gatekeeper-audit-violation-export-config.yaml: Language not supported
- cmd/build/helmify/static/values.yaml: Language not supported
- test/bats/helpers.bash: Language not supported
- test/bats/test.bats: Language not supported
- test/export/fake-reader/Dockerfile: Language not supported
Comments suppressed due to low confidence (1)
pkg/export/disk/disk.go:160
- There is a stray backtick in the error message format string; remove it to correctly format the error message.
if err := os.MkdirAll(dir, 0o777); err != nil { return fmt.Errorf("failed to create directories:` %w", err) }
pkg/export/disk/disk.go
Outdated
|
||
dir := path.Join(conn.Path, topic) | ||
if err := os.MkdirAll(dir, 0o777); err != nil { | ||
return fmt.Errorf("failed to create directories:` %w", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove ` before %w
@@ -1,4 +1,4 @@ | |||
FROM --platform=$BUILDPLATFORM golang:1.24-bookworm@sha256:fa1a01d362a7b9df68b021d59a124d28cae6d99ebd1a876e3557c4dd092f1b1d AS builder |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will prob get bumped by dependabot to use the latest and to avoid CVEs. hence my question that maybe we would need to publish these in the future like all other GK images.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1. I will raise a PR as follow up.
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
What this PR does / why we need it: Adding driver to export violation on disk.
Which issue(s) this PR fixes (optional, using
fixes #<issue number>(, fixes #<issue_number>, ...)
format, will close the issue(s) when the PR gets merged):Fixes ##1037
Special notes for your reviewer: