CI | Mint Integration with NooBaa #9123
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
|
|
||
| name: Mint NC Tests | ||
| on: [workflow_call] | ||
|
|
||
| jobs: | ||
| mint-nc-tests: | ||
| name: Mint NC Tests | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 90 | ||
| permissions: | ||
| actions: read # download-artifact | ||
| contents: read # required for actions/checkout | ||
| steps: | ||
| - name: Checkout noobaa-core | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: 'noobaa/noobaa-core' | ||
| path: 'noobaa-core' | ||
|
|
||
| - name: Download artifact | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: noobaa-tester | ||
| path: /tmp | ||
|
|
||
| - name: Load image | ||
| run: docker load --input /tmp/noobaa-tester.tar | ||
|
|
||
| - name: Create Mint logs directory | ||
| run: | | ||
| set -x | ||
| cd ./noobaa-core | ||
| mkdir -p logs/mint-nc-test-logs | ||
| chmod 777 logs/mint-nc-test-logs | ||
|
|
||
| - name: Run NC Mint tests | ||
| run: | | ||
| set -x | ||
| cd ./noobaa-core | ||
| make test-nc-mint -o tester | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| name: Mint Tests | ||
| on: [workflow_call] | ||
|
|
||
| jobs: | ||
| mint-tests: | ||
| name: Mint Tests | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 90 | ||
| permissions: | ||
| actions: read # download-artifact | ||
| contents: read # required for actions/checkout | ||
| steps: | ||
| - name: Checkout noobaa-core | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: 'noobaa/noobaa-core' | ||
| path: 'noobaa-core' | ||
|
|
||
| - name: Download artifact | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: noobaa-tester | ||
| path: /tmp | ||
|
|
||
| - name: Load image | ||
| run: docker load --input /tmp/noobaa-tester.tar | ||
|
There was a problem hiding this comment. 💡 Verification agent 🧩 Analysis chainVerify artifact path is correct. Based on past feedback, ensure the artifact path 🏁 Script executed: #!/bin/bash
# Description: Check how actions/download-artifact works with the specified path
# Search for other download-artifact usage patterns in the codebase
echo "=== Searching for download-artifact usage patterns ==="
rg -A 5 -B 2 "download-artifact" .github/workflows/
echo "=== Checking for docker load patterns ==="
rg -A 2 -B 2 "docker load" .github/workflows/Length of output: 14634 Update docker load path to include artifact directory The Please update the “Load image” step in
- run: docker load --input /tmp/noobaa-tester.tar
+ run: docker load --input /tmp/noobaa-tester/noobaa-tester.tar(You may also want to apply the same fix across the other workflows that use 🤖 Prompt for AI Agents |
||
|
|
||
| - name: Create Mint logs directory | ||
| run: | | ||
| set -x | ||
| cd ./noobaa-core | ||
| mkdir -p logs/mint-test-logs | ||
| chmod 777 logs/mint-test-logs | ||
|
|
||
| - name: Run Mint tests | ||
| run: | | ||
| set -x | ||
| cd ./noobaa-core | ||
| make test-mint -o tester | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # Mint Github Action, Tests and Tool | ||
|
|
||
| 1. [Introduction](#introduction) | ||
| 2. [Mint GitHub actions](#mint-github-actions) | ||
| 3. [Mint Makefile Targets](#mint-makefile-targets) | ||
| 4. [Manual Mint Installation](#manual-mint-installation) | ||
|
|
||
|
|
||
|
|
||
| ## Introduction | ||
|
|
||
| [Mint](https://github.com/minio/mint) is a testing framework for S3-compatible object storage systems, NooBaa CI runs Mint as correctness/benchmarking and stress tests for the NooBaa system on both containerized and Non Containerized flavors. | ||
| Following are the SDKs/tools used in correctness tests. | ||
|
|
||
| - awscli | ||
| - aws-sdk-go | ||
| - aws-sdk-java | ||
| - aws-sdk-java-v2 | ||
| - aws-sdk-php | ||
| - aws-sdk-ruby | ||
| - healthcheck | ||
| - mc | ||
| - minio-go | ||
| - minio-java | ||
| - minio-js | ||
| - minio-py | ||
| - s3cmd | ||
| - s3select | ||
| - versioning | ||
|
|
||
| ## Mint GitHub actions | ||
|
|
||
| NooBaa CI contains 2 Github actions that build, configure and run Mint. These Github actions run automatically on every PR and on every push, and can run by workflow dispatch manually. | ||
| * [Mint Tests](../../.github/workflows/mint-tests.yaml) - Based on NooBaa Tester image, runs Mint on standard NooBaa (db configuration). | ||
| * [Mint NC Tests](../../.github/workflows/mint-nc-tests.yaml) - Based on NooBaa Tester image, runs Mint on non-containerized NooBaa (ConfigFS configuration). | ||
|
|
||
| Our next goal is to add longer Mint runs as part of NooBaa's nightly CI process. | ||
|
|
||
| ## Mint Makefile Targets | ||
|
|
||
| One can run Mint tests on NooBaa using Mint Makefile targets - | ||
| * `make test-mint` - Based on NooBaa Tester image, runs Mint on standard NooBaa (db configuration). | ||
| * `make test-nc-mint` - Based on NooBaa Tester image, runs Mint on non-containerized NooBaa (ConfigFS configuration). | ||
|
|
||
| The above makefile targets, build NooBaa tester image, and later deploy NooBaa (DB/ConfigFS deployments), create default account and runs the supported sdks on Mint per the deployment type. | ||
|
|
||
| Currently, the supported mint test frameworks are: | ||
| 1. s3cmd | ||
| 2. minio-go | ||
|
|
||
| ## Manual Mint Installation | ||
|
|
||
| NC deployment - | ||
| 1. Tab 1 - Install NooBaa | ||
| 2. Tab 2 - Create a NooBaa account. | ||
| 2. Tab 2 - Run Mint pointing to NooBaa endpoint - | ||
| ``` | ||
| docker run -e SERVER_ENDPOINT=<noobaa-endpoint-address>:<noobaa-endpoint-http-port> -e ACCESS_KEY=<pre-existing-account-access-key> -e SECRET_KEY=<pre-existing-account-secret-key> -e ENABLE_HTTPS=0 minio/mint <sdk-or-tool-name> | ||
| ``` | ||
|
|
||
| Developer notes - | ||
|
|
||
|
|
||
| To manually run a MinIO Mint container that connects to a noobaa-tester container, ensure both containers are on the same Docker network (noobaa-net). | ||
|
|
||
|
|
||
| If the noobaa-tester is already connected to noobaa-net, your Mint run command should look like this: | ||
| ``` | ||
| docker run -e SERVER_ENDPOINT=<noobaa-tester-container-id-or-name>:<noobaa-http-endpoint-port> -e ACCESS_KEY=<pre-existing-account-access-key> -e SECRET_KEY=<pre-existing-account-secret-key> -e ENABLE_HTTPS=0 --network noobaa-net minio/mint <sdk-or-tool-name> | ||
| ``` | ||
|
|
||
| Replace <sdk-or-tool-name> with the specific SDK or tool you want to test (e.g., aws-sdk-java, minio-go, s3cmd, etc.). | ||
|
|
||
| ## Debugging Mint on NooBaa locally - | ||
| Running `make test-mint` will generate the following debug log files - | ||
|
|
||
| ```bash | ||
| noobaa-core % tree logs/mint-test-logs | ||
| logs/mint-test-logs | ||
| ├── log.json // contains the Mint run results | ||
| ├── minio-go | ||
| │ └── error.log // contains errors coming from minio-go run | ||
| ├── mint-test-logs | ||
| │ ├── backingstore1.log // contains noobaa backingstore logs | ||
| │ ├── bg.log // contains noobaa BG workers logs | ||
| │ ├── hosted_agents.log // contains noobaa hosted agents logs | ||
| │ ├── s3.log // contains noobaa endpoint logs | ||
| │ └── web.log // contains noobaa webserver logs | ||
| └── s3cmd | ||
| └── error.log // contains errors coming from s3-cmd run | ||
|
|
||
| ``` | ||
|
|
||
| Running `make test-nc-mint` will generate the following debug log files - | ||
|
|
||
| ```bash | ||
| noobaa-core % tree /mint-nc-test-logs | ||
| logs/mint-nc-test-logs | ||
| ├── log.json // contains the Mint run results | ||
| ├── minio-go | ||
| │ └── error.log // contains errors coming from minio-go run | ||
| ├── mint-nc-test-logs | ||
| │ └── nsfs.log // contains NC noobaa endpoint logs | ||
| └── s3cmd | ||
| └── error.log // contains errors coming from s3-cmd run | ||
|
|
||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.