-
Notifications
You must be signed in to change notification settings - Fork 38
cli: Initial version of Solar tool #305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: msft-main
Are you sure you want to change the base?
Conversation
This allows passing config maps and secrets (as well as any other resource kinds relevant in the future) using the -c flag. Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
This processes all samples in parallel rather than sequentially and speeds up the whole process from ~10 mins to ~90 secs. $ python3 update_policy_samples.py Compiling genpolicy v0.1.0 (/home/abombo/tmp/kata-containers/src/tools/genpolicy) Finished dev [unoptimized + debuginfo] target(s) in 5.57s COMMAND: cargo build COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/limits.yaml Time taken: 0.05 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/namespace.yaml Time taken: 0.05 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/quota.yaml Time taken: 0.05 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod8.yaml Time taken: 6.48 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod11.yaml Time taken: 7.12 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/deployment/deployment-docker-busybox.yaml Time taken: 9.15 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/replica-set/replica2.yaml Time taken: 9.22 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures2/valid-pod.yaml Time taken: 9.64 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/job/test-job2.yaml Time taken: 9.9 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/deployment/deployment-busybox.yaml Time taken: 10.06 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/configmap/pod-cm1.yaml Time taken: 10.31 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/csi-hostpath-testing.yaml Time taken: 10.46 seconds [2024-04-15T21:01:52Z WARN genpolicy::pod] Can't find the value of annotation batch.kubernetes.io/job-completion-index. Allowing any value. COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/job/test-job.yaml Time taken: 10.61 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-lifecycle.yaml Time taken: 10.71 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod10.yaml Time taken: 10.71 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod9.yaml Time taken: 14.71 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod13.yaml Time taken: 14.81 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/hello-populator-deploy.yaml Time taken: 14.94 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/conformance-e2e.yaml Time taken: 14.99 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod4.yaml Time taken: 15.06 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod5.yaml Time taken: 15.08 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/replica-set/replica-busy.yaml Time taken: 15.18 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook2/webhook-pod12.yaml Time taken: 15.24 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-one-container.yaml Time taken: 15.38 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/secrets/azure-file-secrets.yaml Time taken: 15.49 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/daemon.yaml Time taken: 15.55 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/configmap/pod-cm3.yaml Time taken: 15.65 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/rc-noexist.yaml Time taken: 15.79 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod1.yaml Time taken: 15.87 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod2.yaml Time taken: 15.93 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-persistent-volumes.yaml Time taken: 16.02 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-exec.yaml Time taken: 16.13 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/deployment/deployment-azure-vote-back.yaml Time taken: 16.21 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance2/ingress-static-ip-rc.yaml Time taken: 16.23 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance2/ingress-http2-rc.yaml Time taken: 16.27 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/netexecrc.yaml Time taken: 16.38 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/rc-lastapplied.yaml Time taken: 16.49 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance2/ingress-http-rc.yaml Time taken: 16.52 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance2/ingress-multiple-certs-rc.yaml Time taken: 16.63 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/stateful-set/web.yaml Time taken: 16.68 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/appsv1deployment.yaml Time taken: 16.78 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/configmap/pod-cm2.yaml Time taken: 16.92 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod3.yaml Time taken: 16.97 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/replication.yaml Time taken: 17.01 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/multi-resource-yaml.yaml Time taken: 17.07 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-ubuntu.yaml Time taken: 17.17 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod6.yaml Time taken: 17.23 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/deploy-clientside.yaml Time taken: 17.31 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/stateful-set/web2.yaml Time taken: 17.34 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures2/rc-service.yaml Time taken: 17.42 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-same-containers.yaml Time taken: 17.45 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook/webhook-pod7.yaml Time taken: 17.47 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/etcd-statefulset.yaml Time taken: 17.63 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/incomplete-init/controller.yaml Time taken: 17.71 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/incomplete-init/cassandra-statefulset.yaml Time taken: 19.18 seconds Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs Skipping link to non-file: etc/rc0.d/.wh.K01sendsigs [2024-04-15T21:02:06Z WARN genpolicy::registry] Failed to parse www-data as u32, using uid = 0 - error invalid digit found in string COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance2/ingress-nginx-rc.yaml Time taken: 23.91 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/rc-nginxhttps.yaml Time taken: 27.22 seconds [2024-04-15T21:02:09Z WARN genpolicy::mount_and_storage] Unable to determine backing storage of persistent volume claim 'datadir'. Pass `-a <pvc.yaml>` to get rid of this warning. [2024-04-15T21:02:09Z WARN genpolicy::mount_and_storage] Unable to determine backing storage of persistent volume claim 'datadir'. Pass `-a <pvc.yaml>` to get rid of this warning. COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/incomplete-init/cockroachdb-statefulset.yaml Time taken: 27.3 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/conformance/csi-hostpath-plugin.yaml Time taken: 29.12 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook3/dns-test.yaml Time taken: 55.68 seconds COMMAND: sudo target/debug/genpolicy -d -s -y ../../agent/samples/policy/yaml/webhook3/many-layers.yaml Time taken: 55.66 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-spark.yaml Time taken: 59.81 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/kubernetes/fixtures/job.yaml Time taken: 60.45 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/deployment/deployment-azure-vote-front.yaml Time taken: 61.6 seconds COMMAND: sudo target/debug/genpolicy -d -y ../../agent/samples/policy/yaml/pod/pod-three-containers.yaml Time taken: 96.99 seconds Total time taken: 97.0137689113617 seconds
genpolicy: add support for cc-managed-csi
Signed-off-by: Aurélien Bombo <abombo@microsoft.com>
- Add --version flag to the genpolicy tool that prints the current version - Add version.rs.in template to store the version information - Update makefile to autogenerate version.rs from version.rs.in - Add license to Cargo.toml Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
genpolicy: add --version flag
Linux kernel generates a panic when the init process exits. The kernel is booted with panic=1, hence this leads to a vm reboot. When used as a service the kata-agent service has an ExecStop option which does a full sync and shuts down the vm. This patch mimicks this behavior when kata-agent is used as the init process. Fixes: kata-containers#9429 Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
genpolicy: add support for cc-local-csi
agent: shutdown vm on exit when agent is used as init process
Add missing cache improvements specifically missing in containerd pull Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
…improvements genpolicy: add missing cache improvements
This patch adds support for the cc-azurefile-csi driver to the genpolicy. Signed-off-by: Archana Choudhary <archana1@microsoft.com>
This patch updates policy samples, required after adding support for cc-azurefile-csi driver in genpolicy. Signed-off-by: Archana Choudhary <archana1@microsoft.com>
genpolicy: add support for cc-azurefile-csi driver
This reverts commit 627be9b, that was insufficient. Waiting for blk devices used just the PCI device/slot index, but not the PCI segment/domain index. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Initialize the CLH Platform a single time. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Hotplug block devices on PCI segments >= 1. PCI segment 0 is used for the network interface, any disks present at Guest boot time, etc. Just bus 0 of each segment is used, and up to 31 devices can be hotplugged to each bus. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This pod starts successfully when using default AKS-CC settings, and a permissive policy. When the Kata debug options are enabled, this pod fails to start while trying to hotplug image layer index 41. This bug is being investigated. The genpolicy tool should also try to create a smaller policy for this pod, because otherwise "kubectl apply" rejects the policy annotation as being too large. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Lock anyhow version to 1.0.58 because: - Versions between 1.0.59 - 1.0.76 have not been tested yet using Kata CI. However, those versions pass "make test" for the Kata Agent. - Versions 1.0.77 or newer fail during "make test" - see kata-containers#9538. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Implement Agent Policy using the regorus crate instead of the OPA daemon. The OPA daemon will be removed from the Guest rootfs in a future PR. Fixes: kata-containers#9388 Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Bump release version to 3.2.0-azl1.genpolicy0 Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
genpolicy: bump release version
Move pod-many-layers.yaml to needs_containerd_pull category Signed-off-by: Saul Paredes <saulparedes@microsoft.com>
runtime: agent: use PCI segments 1+ for blk devices
agent: use regorus instead of opa
Since OPA binary was replaced by the regorus crate, we can finally stop building and shipping the binary. Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Delete: - systemd-journald-audit.socket - systemd-journal-catalog-update.service - systemd-journal-flush.service - systemd-journald@.service - systemd-journald@.socket - journalctl Other journal files were already deleted. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Delete blk-availability.service and the blkdeactivate binary. They are not needed on the Guest VMs. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Not needed on a Kata Guest VM. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deleted files: - systemd-logind.service - systemd-logind Not needed on the Guest VM. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Not needed on Guest VMs. Deleting this service disables modprobe@efi_pstore.service too. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deleted files: - systemd-sysupdate.service - systemd-sysupdate.timer - systemd-sysupdate-reboot.timer - systemd-sysupdate-reboot.service - systemd-sysupdate Guest VMs are not being updated this way. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Delete: - systemd-network-generator.service - systemd-network-generator Generating network configuration based on kernel command line arguments is currently not supported for Guest VMs. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deleted files: - systemd-tpm2-setup.service - systemd-tpm2-setup-early.service - systemd-tpm2-setup TPM is not used on Guest VMs. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Delete *.mount files in addition to other systemd files. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
There are no userspace filesystems on the Guest VMs, so no need to mount the FUSE control filesystem. Note that the FUSE protocol used by virtio-fs uses a server on the Host, and doesn't depend on the FUSE control filesystem on the Guest. Without sys-fs-fuse-connections.mount, modprobe@fuse.service gets disabled too. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
This mount was not functional due to missing CONFIG_CONFIGFS_FS in the Guest kernel. Deleting sys-kernel-config.mount also disables modprobe@configfs.service. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Delete: - systemd-update-utmp-runlevel.service - systemd-update-utmp in addition to systemd-update-utmp.service that was already deleted before this commit. systemd-update-utmp-runlevel.service depends on /var/log/wtmp. However, systemd-tmpfiles-setup.service was already deleted, so /var/log/wtmp was not present. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Deleted files: - systemd-vconsole-setup.service - systemd-vconsole-setup The Guest VM doesn't use virtual consoles. Signed-off-by: Dan Mihai <dmihai@microsoft.com>
Implement HypervisorLoglevel config option for clh. Signed-off-by: Cameron Baird <cameronbaird@microsoft.com>
Cameronbaird/hyp loglevel
rootfs: delete more systemd files
Solar tool is used for calculating root hashes of container image layers, signing them and producing a json manifest. Signed-off-by: Jiri Appl <jiria@microsoft.com>
Notes for reviewers:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Overview
This PR introduces the initial version of the Solar tool, an extension of the genpolicy tool, that calculates root hashes of container image layers, signs them, and produces a JSON manifest consumable by the tardev-snapshotter.
- Introduces new CLI functionality for signing OCI layer root hashes.
- Adds a Cargo.toml with updated dependencies and new modules (main, version, utils, and verity).
- Implements the core logic for computing and signing root hashes and for processing image lists.
Reviewed Changes
File | Description |
---|---|
src/tools/sign-oci-layer-root-hashes/Cargo.toml | Adds package definition and dependency configuration for the new tool. |
src/tools/sign-oci-layer-root-hashes/src/version.rs.in | Provides auto-generated version information. |
src/tools/sign-oci-layer-root-hashes/src/main.rs | Contains the CLI entry point and core logic for processing images and signing hashes. |
src/tools/sign-oci-layer-root-hashes/src/utils.rs | Implements CLI argument parsing and configuration management using clap. |
src/tools/sign-oci-layer-root-hashes/src/verity.rs | Implements the verity hashing logic used for generating and finalizing layer hashes. |
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
src/tools/sign-oci-layer-root-hashes/src/main.rs:53
- [nitpick] The printed tool name "SOLaR" differs in capitalization from the package name "solar". Consider using consistent capitalization for clarity.
println!("SOLaR tool: id: {}, version: {}, commit: {}", env!("CARGO_PKG_NAME"), env!("CARGO_PKG_VERSION"), version::COMMIT_INFO);
src/tools/sign-oci-layer-root-hashes/src/main.rs:100
- Using .as_mut() on the temporary Vec from the collect operation with Vec::append can lead to issues. Consider replacing this with image_tags.extend( ... ) to append the values directly.
.collect::<Vec<String>>().as_mut(),
.as_mut(), | ||
); | ||
} else if let Some(images) = &config.image { | ||
image_tags.append(images.clone().as_mut()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Appending a cloned vector using .as_mut() is problematic because Vec::append expects a mutable vector reference. Use image_tags.extend(images.clone()) instead to correctly merge the vectors.
image_tags.append(images.clone().as_mut()); | |
image_tags.extend(images.clone()); |
Copilot uses AI. Check for mistakes.
Solar tool is used for calculating root hashes of container image layers, signing them and producing a json manifest.
Merge Checklist
upstream/missing
label (orupstream/not-needed
) has been set on the PR.Summary
The Solar tool is an extension of genpolicy tool, that creates JSON manifests that hold layer digests and root hash signatures, that can be consumed by the tardev-snapshotter.
Test Methodology
The tool output hash been validated against the tardev-snapshotter.