Skip to content

feat(l2): implement TDX attestation infrastructure #2646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "crates/l2/tee/verifier/lib/forge-std"]
path = crates/l2/tee/verifier/lib/forge-std
url = https://github.com/foundry-rs/forge-std
6 changes: 6 additions & 0 deletions crates/l2/tee/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
image
image.*
mkosi.crt
mkosi.key
mkosi.tools.manifest
mkosi.tools
85 changes: 85 additions & 0 deletions crates/l2/tee/DOCS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# TDX execution module

## Usage

On a machine with TDX support [with the required setup](https://github.com/canonical/tdx) run
```
mkosi build
mkosi vm
```

## What is TDX?

TDX is an Intel technology implementing a Trusted Execution Environment.
Such an environment allows verifying certain code was executed without being tampered with or observed.

These verifications (attestations) are known as "quotes" and contain signatures verifying the attestation was generated by a genuine processor, the measurements at the time, and a user-provided piece of data binding the proof.

The measurements happen into four Run Time Measurement Registers (RTMR), with each RTMR respresenting a boot stage.
This is analogous to [how PCRs work](https://uapi-group.org/specifications/specs/linux_tpm_pcr_registry/).

## Usage considerations

Do not hardcode quote verification parameters as [they might change](https://cc-enabling.trustedservices.intel.com/intel-tdx-enabling-guide/02/infrastructure_setup/#tcb-recovery-tcb-r).

It's easy to silently overlook non-verified areas such as accidentally leaving login enabled, not verifying the integrity of the state.

## Boot sequence

- Firmware (OVMF here) is loaded (and hashed into RTMR[0])
- [UKI](https://uapi-group.org/specifications/specs/unified_kernel_image/) is loaded (and hashed into a RTMR)
- kernel and initrd are extracted from the UKI and executed
- root partition is verified using the `roothash=` value provided on the kernel cmdline and the `hash` partition with the dm-verity merkle tree
- root partition is mounted read-only
- (WIP) systemd executes the payload

## Image build components

To build images we use [mkosi](https://github.com/systemd/mkosi)

### Tooling image

`mkosi.tools.conf` defines the tool configuration, and `mkosi.tools.skeleton` imports the kobuk-team PPA (used by [canonical/tdx](https://github.com/canonical/tdx)) with the modified qemu build

This allows the build process to not depend on the host's tooling

### Image preparation

Runs `mkosi.prepare.chroot`, which has network access, to download crate dependencies.

### Image building

Runs `mkosi.build.chroot` to produce the output

## Debug suggestions

- Adding `bash` to mkosi scripts to drop an interactive shell that lets you explore the build process
- Adding a root password in `mkosi.conf` to allow logging in to the container


## Chain verification


```
# NOTE: initialize&update submodules on all repos
(ethrex) make dev # start L1
(ethrex crates/l2) make deploy-l1 # deploy create2 deployer
(daimo-eth/p256-verifier) scripts/deploy.sh # deploy P256 verifier
(automata-network/automata-on-chain-pccs) edit script/utils/P256Configuration.sol with daimo addr and chainid
(automata-network/automata-on-chain-pccs) forge install && make deploy-all
(automata-network/automata-on-chain-pccs) # look at deployment/CHAIN_ID.json
(automata-network/automata-dcap-attestation evm/deployment/) add CHAINID/onchain_pccs.json
(automata-network/automata-dcap-attestation evm/) edit script/utils/P256Configuration.sol with daimo addr and chainid
(automata-network/automata-dcap-attestation evm/) forge install && make deploy-all
(automata-network/automata-dcap-qpl automata-dcap-qpl-tool/) # edit and source env/automata_testnet
Replay [1], [2] and [3] against AutomataPcsDao, with `rex send CONTRACT_AutomataPcsDao 0 PRIVATE_KEY --calldata CALLDATA`
Run AutomataDcapAttestationFee.setQuoteVerifier(CONTRACT_V4QuoteVerifier)

# To validate a quote
(automata-network/automata-dcap-qpl automata-dcap-qpl-tool/) cargo run -- -p PRIVATE_KEY --chain_id=$CHAIN_ID --rpc_url=$RPC_URL --quote_hex QUOTE # Inserts missing pieces
Now AutomataDcapAttestationFee.verifyAndAttestOnChain(QUOTE) can validate the quote.
```

[1] CA ROOT https://explorer.ata.network/tx/0x3c44e3eb9c866052ff84f1b11d91440b36cfc28ea5aeed78264d17411089de3d
[2] PLATFORM https://explorer.ata.network/tx/0x8b615efe4016176183145e34e1d659acb2788ff6763e6645c6ab1a77b1a3a828
[3] ROOT CRL https://explorer.ata.network/tx/0xf6e5cffa806261b7fdff6c0623ed1361d6e12073f4af026200735d1d87b1dafe
5 changes: 5 additions & 0 deletions crates/l2/tee/mkosi.build.chroot
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
source /.cargo/env
cd /work/src/proj
cargo build --release
cp target/release/quote-gen /work/dest/payload
42 changes: 42 additions & 0 deletions crates/l2/tee/mkosi.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
[Distribution]
Distribution=arch

[Output]
Format=disk
SplitArtifacts=roothash
Seed=00000000-0000-0000-0000-000000000000

[Content]
#TODO: when systemd v258 is released, add PCR generation
Packages=linux,systemd
BuildPackages=bash,curl,ca-certificates,base-devel
SourceDateEpoch=0
Bootable=true
Bootloader=uki
KernelModules=tdx_guest,tsm,vmw_vsock_virtio_transport,virtio_net,dm_mod,dm_verity,erofs
RootPassword=1234
ExtraTrees=./mkosi.extra
KernelCommandLine=console=hvc0 systemd.firstboot=no
ip=enc0:any ip=enp0s1:any ip=enp0s2:any ip=host0:any
systemd.mount-extra=TMPDIR=:/var/tmp:tmpfs
systemd.wants=payload.service

[Validation]
Verity=hash
Checksum=true

[Build]
Incremental=true
BuildSources=quote-gen:proj
CacheDirectory=/tmp/ethrex-mkosi.cache/
ToolsTree=default
ToolsTreeDistribution=ubuntu
ToolsTreeRepositories=main,universe

[Runtime]
TPM=false
QemuArgs=-bios /usr/share/ovmf/OVMF.fd
-machine smm=off,confidential-guest-support=tdx
-object '{"qom-type":"tdx-guest","id":"tdx","quote-generation-socket":{"type": "vsock", "cid":"2","port":"4050"}}'
Firmware=bios
Ephemeral=true
2 changes: 2 additions & 0 deletions crates/l2/tee/mkosi.extra/etc/resolv.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
nameserver 1.1.1.1
nameserver 8.8.8.8
10 changes: 10 additions & 0 deletions crates/l2/tee/mkosi.extra/etc/systemd/system/payload.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[Unit]
After=network-online.target

[Service]
Type=oneshot
ExecStart=/payload
StandardOutput=journal+console

[Install]
WantedBy=multi-user.target
9 changes: 9 additions & 0 deletions crates/l2/tee/mkosi.prepare.chroot
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
if [[ "$1" != "build" ]]; then
exit
fi
cd /work/src/proj
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source /.cargo/env
rustup toolchain install
cargo fetch
7 changes: 7 additions & 0 deletions crates/l2/tee/mkosi.repart/00-esp.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[Partition]
Type=esp
Format=vfat
CopyFiles=/efi:/
CopyFiles=/boot:/
SizeMinBytes=512M
SizeMaxBytes=512M
7 changes: 7 additions & 0 deletions crates/l2/tee/mkosi.repart/11-root-verity.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[Partition]
Type=root-verity
Label=%M_%A_verity
Verity=hash
VerityMatchKey=root
SplitName=%t.%U
Minimize=yes
9 changes: 9 additions & 0 deletions crates/l2/tee/mkosi.repart/12-root.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[Partition]
Type=root
Format=erofs
Label=%M_%A_root
Verity=data
VerityMatchKey=root
CopyFiles=/
Minimize=yes
SplitName=%t.%U
5 changes: 5 additions & 0 deletions crates/l2/tee/mkosi.tools.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[Content]
Packages=bash,git

[Build]
SandboxTrees=./mkosi.tools.skeleton
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
Types: deb
URIs: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu/
Suites: noble
Components: main
Signed-By: -----BEGIN PGP PUBLIC KEY BLOCK-----
.
mQINBGTSU84BEADVSs1CRYCbfu4xppSmtCntU5KeefhklqbmGRBLJzqHGLlX7snZ
vMyWdSitX/ZGH3KwZar58524URkm6+o6qWVjCZVdSoDWDYvY4vmQV7enFv34Whg2
iMDP92+O8WJchWwPrnVSiwffD/g5QvrXeR8foJGMmP0/u4+QHBDZNV2R82B129FX
eWBuMZ4skBOdYg/pE0axhfKMY7I1AQe9GM3/IuGKHwMBWBRfEDHz8C1DzvLeIJP6
OBIjBplpLiCODl3ljeCh/cjakdTnqvJrLOqtyeTRRZMFaDnWByHYPQ8pMzGzKj2I
0ij088LV4w8FwNq2jU9zXXxItgQg9YLgq/cbNE4CCLu7/ivvjObQfJIMGg0DZdV6
rauwXlLWC+qhSrc+/SAU0Of8EKvgTHHnibC/p+QoptM11pRD5IG11QOS4r/UXyzm
PuWIgw1d6YW4vimy0CD61AxoyvTbkvWa+t2pApTmpGMxZmgwzZ+GkCz21KIkjSEq
6aMwFDeiTiuDojBXPuzt95yC4Up4/PU//h/Bos9NQaXjJuBr+Z9SIqtF6WxqZBLp
eXd55Jl5FSYUvhP0lIwSbjf8SzxlJbvTUCeXpyWG+EWUnuYfeXBt6oe18kMmkDn7
fdnrspWf2V0OSxxopBJpDjYDpflHuVmvNXH8BPics8HVSzCZqQ0STIv5wwARAQAB
tCBMYXVuY2hwYWQgUFBBIGZvciBUaGUgS29idWsgdGVhbYkCTgQTAQoAOBYhBAwO
avlVzkY8A/xRV00JjXCvvl4fBQJk0lPOAhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4B
AheAAAoJEE0JjXCvvl4fzEkP/3GXwyiDmpg4zMfOLFJATf1AiULugIGqdwN4XQ+K
Eu1pi37Zg98rJW9WtulDMIvlcTyqyTEgWLNfswDmG4OlW5roWR58d3nYg0doF4Nk
tr+dkk3oTKBoW12/aHhkW9T5fPB0wzT1xK3OH/ksG1/KcV65X9ivKUL9KMYoTtYZ
iL3+bCdvH3Ad6l4WPzrT3HI8WWqKfR8jAWzWjUYLHq8eOaBTdiFCmTL77bPVofl+
PG2FoPIWAPzy1iepKnN9b66ayekPQyfXynxP6nJ6OIBR/HrjZDBT9JTulBvn3n/k
pnA4Fnrw1KBAEcnQdMXbsHkrJXvjnaZdtus8s/BSJPl5p4knGkSTIUdo9O2ChSjd
EFKcfioXmprC7yHOA6qbCr1V8+j5nIldxfKZ7+dplC+2/dbYX84zN4XfAWox5xYc
qxc0Q4SP4BGoUVUtupmadUVSRQFxYAhhDta/8GmPED++U3H70Ux8CkRDBgw83pk3
F44ekI3IJY9LRmGCxPQmGFLBpajarxIcKgf1P8z0KfLWYhp3wDtGun5TfxXsEy5s
DdqFNJo26Qnr7A4uav6NhPN59198wKR2YT7Flzb5x83a4EZAn2vvsp4GJpvO7EnX
P6mWFKjZUFCjqd6EywvWuHwDcn5TRJ4CqEBcRNR9Ye1YAVrpwlRrHznTlWn4DN1I
FOOn
=2YVf
-----END PGP PUBLIC KEY BLOCK-----
Loading
Loading