test: Implement file based configuration for test environments#2512
Conversation
Added a yaml configuration mechanism for the testing framework to map provider settings directly to environment variables. Excluded the new execution entrypoint from naming validation checks since it functioned as a harness. Using a structured file eliminated the overhead of managing numerous separate variables across different git providers. Retained environment variable precedence to allow selective overrides. Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
Summary of ChangesHello @chmouel, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the end-to-end testing framework by implementing a file-based YAML configuration mechanism. This change streamlines the management of test environment settings for various providers, reducing the need to juggle numerous individual environment variables. The new system prioritizes existing environment variables, allowing for seamless integration and selective overrides, while also refining test naming conventions to properly account for test harness functions. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Pull request overview
Adds a YAML-based configuration mechanism for E2E tests, allowing provider settings to be loaded from a single file and mapped into the existing TEST_* environment variables while keeping env-var precedence for overrides.
Changes:
- Introduce
test/pkg/configfilewith YAML parsing and env-var export logic, plus unit tests. - Add an E2E-only
TestMainentrypoint to optionally load config fromPAC_E2E_CONFIG. - Document the new config flow and provide an example
e2e-config.yaml.example; update naming-check script to ignoreTestMain.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
test/pkg/configfile/config.go |
Implements YAML load + reflection-based env-var population for E2E provider settings. |
test/pkg/configfile/config_test.go |
Adds unit tests for valid config, env override, missing file, partial config, invalid YAML, and all-providers coverage. |
test/main_test.go |
Adds TestMain (build-tagged e2e) to load config from PAC_E2E_CONFIG before running tests. |
test/e2e-config.yaml.example |
Provides a template YAML config file documenting all supported provider sections/fields. |
test/README.md |
Documents using PAC_E2E_CONFIG and the YAML config file approach. |
hack/check-e2e-test-naming.sh |
Excludes TestMain from E2E test naming validation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a convenient file-based configuration system for the e2e tests, which is a great improvement for managing test environments. The implementation uses a TestMain entrypoint to load a YAML file and populate environment variables, with existing environment variables taking precedence. The code is well-structured and includes comprehensive unit tests.
I have a couple of suggestions to improve the robustness and clarity of the new configuration loading logic. My main points are about making the reflection-based logic safer for future changes and improving the readability and reliability of the test setup code, especially concerning environment variable management to prevent flaky tests. Overall, this is a solid contribution.
📝 Description of the Change
This PR implements a YAML-based configuration mechanism for the E2E testing framework. The changes enable test engineers to define provider-specific settings in a single
e2e-config.yamlfile instead of managing numerous individual environment variables across different git providers (GitHub, GitLab, Bitbucket, Gitea, Forgejo).Key Improvements
e2e-config.yaml) replaces scattered environment variablesPAC_E2E_CONFIGenv var is set👨🏻 Linked Jira
🔗 Linked GitHub Issue
Fixes #1519
🚀 Type of Change
feat:)fix:)feat!:,fix!:)docs:)chore:)refactor:)enhance:)deps:)🧪 Testing Strategy
🤖 AI Assistance
If you have used AI assistance, please provide the following details:
Which LLM was used?
Extent of AI Assistance:
Important
If the majority of the code in this PR was generated by an AI, please add a
Co-authored-bytrailer to your commit message.For example:
Co-authored-by: Gemini gemini@google.com
Co-authored-by: ChatGPT noreply@chatgpt.com
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Cursor noreply@cursor.com
Co-authored-by: Copilot Copilot@users.noreply.github.com
**💡You can use the script
./hack/add-llm-coauthor.shto automatically addthese co-author trailers to your commits.
✅ Submitter Checklist
fix:,feat:) matches the "Type of Change" I selected above.make testandmake lintlocally to check for and fix any issues. For an efficient workflow, I have considered installing pre-commit and runningpre-commit installto automate these checks.