-
Notifications
You must be signed in to change notification settings - Fork 0
Create main-ci-pipeline.yml #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 15 commits
a549684
dd38398
7ed2e6e
6e8834a
330ecf5
08d3383
6cfa374
7d9098d
5f11da5
8b2a7ec
c3b33df
9f42da3
2af0eae
4b42ba6
1801aaf
2240e29
e2ce52c
d7811aa
9f7ef38
c62ffe7
3da4874
8a25609
16b568a
761f87e
551ec19
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,185 @@ | ||
name: CI/CD Pipeline | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
types: | ||
- opened | ||
- reopened | ||
- synchronize | ||
|
||
jobs: | ||
|
||
build-job: | ||
name: Build | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up CMake | ||
uses: lukka/get-cmake@v3.31.0 | ||
|
||
# Run the setup script to install dependencies | ||
- name: Run setup script | ||
run: | | ||
chmod +x build-utils/pipe_setup.sh | ||
./build-utils/pipe_setup.sh | ||
|
||
- name: Build project | ||
run: | | ||
cmake -S . -B cmake-build-debug -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DENABLE_COVERAGE=ON -DCOVERAGE_DIR_NAME="coverage" | ||
cmake --build cmake-build-debug | ||
|
||
- name: Upload build artifacts | ||
uses: actions/upload-artifact@v3 | ||
bjoheimer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
with: | ||
name: build-artifacts | ||
path: | | ||
cmake-build-debug | ||
retention-days: 1 | ||
bjoheimer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
test-job: | ||
name: Test | ||
runs-on: ubuntu-latest | ||
needs: build-job | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up CMake | ||
uses: lukka/get-cmake@v3.31.0 | ||
|
||
# Run the setup script to install dependencies | ||
- name: Run setup script | ||
run: | | ||
chmod +x build-utils/pipe_setup.sh | ||
./build-utils/pipe_setup.sh | ||
|
||
- name: Download build artifacts | ||
uses: actions/download-artifact@v3 | ||
with: | ||
name: build-artifacts | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems to be unused at the moment. Are you trying to share something between jobs, or is it just a leftover? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this is one of the main issues I am having right now. I am trying to share the cmake-build-debug dir created in the build job so we do not have to build the project again in every job. The dir is needed in the clang-tidy and the test job. However, this does not seem to work the way I want it to as the dir apparently cannot be found in the test job for example:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Independent of the direction you take next, I think that uploading/downloading artifacts is not intended for this use case. I think this could be much easier if you indeed ran everything inside a container, because then you could write into a persistent directory (depends a bit on how you run the container). Another alternative would be to not split multiple jobs, but just have one job with multiple steps. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Indeed. The persistent directory is why we will try with docker container next (see comment below) 👍 |
||
|
||
- name: Run tests | ||
run: | | ||
ctest --test-dir cmake-build-debug/test --rerun-failed --output-on-failure | ||
cmake --build cmake-build-debug --target coverage | ||
COVERAGE=$(lcov --summary cmake-build-debug/../coverage/coverage.info | awk '/lines/ {print $2}') | ||
echo "Line coverage $COVERAGE" | ||
|
||
# Remove percentage symbol and compare numeric value | ||
COVERAGE_VALUE=$(echo "$COVERAGE" | sed 's/%//') | ||
if [ "$COVERAGE_VALUE" -lt 80 ]; then | ||
echo "Coverage is below 80%! Failing pipeline." | ||
# TODO once the test coverage is high enough, we should fail the pipeline here | ||
# exit 1 | ||
else | ||
echo "Coverage is sufficient: $COVERAGE_VALUE%" | ||
fi | ||
Comment on lines
+71
to
+79
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could also store the coverage value in a file and check if the PR increases or decreases the coverage. This would be easier to track than eventually (and cumulatively) dropping below a threshold. Or run There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is a great idea! I will tackle this once the pipeline is up and running. |
||
|
||
- name: Upload coverage report | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: coverage-report | ||
path: coverage/report | ||
retention-days: 1 | ||
|
||
clang-format: | ||
name: Clang Format | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up CMake | ||
uses: lukka/get-cmake@v3.31.0 | ||
|
||
# Run the setup script to install dependencies | ||
- name: Run setup script | ||
run: | | ||
chmod +x build-utils/pipe_setup.sh | ||
./build-utils/pipe_setup.sh | ||
|
||
- name: Debug LLVM Binaries | ||
run: | | ||
sudo find /usr /lib /opt -name clang-format-18 -o -name clang-tidy-18 | ||
dpkg-query -L clang-format-18 || echo "clang-format-18 not found in package files" | ||
llvm-config --bindir || echo "llvm-config not installed or misconfigured" | ||
echo $PATH | ||
Comment on lines
+103
to
+108
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All the clang-format setup looks too complicated. Do you know that you can get any clang-format version from pip? https://pypi.org/project/clang-format/ You could also use a predefined clang-format action or, even better, use a predefined pre-commit hook. See an example in https://github.com/precice/precice/blob/develop/.pre-commit-config.yaml There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The setup was originally not that complicated. Those are just remnants of my efforts to debug the pipeline and will be removed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could still run pre-commit on all files in the CI ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is smart. Originally, we had a very weak runner which is why we split up the pre-commit linting into clang-format and clang-tidy. clang-tidy needs a build which took forever on our old runner, so we performed clang-format first so that the pipe "fails fast". However, now performance is no longer an issue, so it is sensible to reuse the pre-commit checks for the reasons you mentioned. |
||
|
||
|
||
- name: Run clang-format | ||
run: | | ||
clang-format -style=file $(find src/ test/ -type f \( -name "*.cpp" -o -name "*.hpp" -o -name "*.h" \)) --Werror --dry-run | ||
|
||
clang-tidy: | ||
name: Clang Tidy | ||
runs-on: ubuntu-latest | ||
needs: [build-job, test-job] | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up CMake | ||
uses: lukka/get-cmake@v3.31.0 | ||
|
||
# Run the setup script to install dependencies | ||
- name: Run setup script | ||
run: | | ||
chmod +x build-utils/pipe_setup.sh | ||
./build-utils/pipe_setup.sh | ||
|
||
- name: Check clang-tidy version | ||
run: | | ||
# Check and install clang-tidy version 18 if needed | ||
if command -v clang-tidy &> /dev/null; then | ||
INSTALLED_VERSION=$(clang-tidy --version | grep -oP '(?<=version )\d+' | head -n 1) # Extracts major version after "version" | ||
if [ "$INSTALLED_VERSION" -lt 18 ]; then | ||
echo "Clang-tidy version $INSTALLED_VERSION is less than 18. Installing clang-tidy version 18..." | ||
sudo apt-get install -y clang-tidy-18 | ||
sudo update-alternatives --install /usr/bin/clang-tidy clang-tidy /usr/bin/clang-tidy-18 100 | ||
else | ||
echo "Clang-tidy version $INSTALLED_VERSION is sufficient." | ||
fi | ||
else | ||
echo "Clang-tidy is not installed. Installing clang-tidy version 18..." | ||
sudo apt-get install -y clang-tidy-18 | ||
sudo update-alternatives --install /usr/bin/clang-tidy clang-tidy /usr/bin/clang-tidy-18 100 | ||
fi | ||
|
||
- name: Download build artifacts | ||
uses: actions/download-artifact@v3 | ||
with: | ||
name: build-artifacts | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same comment as above: Do you need this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, if it does what I assume it does, I need the compile_commands.json file which is outputted to cmake-build-debug during the build. |
||
|
||
- name: Run clang-tidy | ||
run: | | ||
clang-tidy -p=cmake-build-debug/ $(find src/ test/ -type f \( -name "*.cpp" -o -name "*.hpp" -o -name "*.h" \)) --warnings-as-errors=* --header-filter="^((src|test)/.*)$" --use-color | ||
|
||
run-exe: | ||
name: Run Executable | ||
runs-on: ubuntu-latest | ||
needs: build-job | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up CMake | ||
uses: lukka/get-cmake@v3.31.0 | ||
|
||
# Run the setup script to install dependencies | ||
- name: Run setup script | ||
run: | | ||
chmod +x build-utils/pipe_setup.sh | ||
./build-utils/pipe_setup.sh | ||
|
||
- name: Download build artifacts | ||
uses: actions/download-artifact@v3 | ||
with: | ||
name: build-artifacts | ||
|
||
- name: Run Executable | ||
run: ./build-utils/run_check.sh |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
#!/bin/bash | ||
|
||
# Update the package list | ||
sudo apt-get update | ||
|
||
# Install required dependencies | ||
sudo apt-get install -y \ | ||
build-essential \ | ||
cmake \ | ||
xorg-dev \ | ||
libsdl2-dev \ | ||
clang-tidy \ | ||
clang-format \ | ||
git \ | ||
pre-commit \ | ||
python3 \ | ||
lcov | ||
|
||
sudo apt-get install -y software-properties-common wget build-essential cmake xorg-dev libsdl2-dev git pre-commit python3 lcov | ||
|
||
# Add the official LLVM APT repository for the latest Clang versions | ||
wget https://apt.llvm.org/llvm.sh | ||
chmod +x llvm.sh | ||
sudo ./llvm.sh 18 # Install LLVM version 18 | ||
|
||
# Verify installation paths | ||
CLANG_FORMAT_PATH=$(find /usr /lib /opt -name clang-format-18 | head -n 1) | ||
CLANG_TIDY_PATH=$(find /usr /lib /opt -name clang-tidy-18 | head -n 1) | ||
|
||
# Update alternatives to point to LLVM tools | ||
if [ -f "$CLANG_FORMAT_PATH" ]; then | ||
echo "Setting up clang-format version 18..." | ||
sudo update-alternatives --install /usr/bin/clang-format clang-format "$CLANG_FORMAT_PATH" 100 | ||
else | ||
echo "Error: clang-format-18 binary not found at $CLANG_FORMAT_PATH" | ||
fi | ||
|
||
if [ -f "$CLANG_TIDY_PATH" ]; then | ||
echo "Setting up clang-tidy version 18..." | ||
sudo update-alternatives --install /usr/bin/clang-tidy clang-tidy "$CLANG_TIDY_PATH" 100 | ||
else | ||
echo "Error: clang-tidy-18 binary not found at $CLANG_TIDY_PATH" | ||
fi | ||
|
||
# Echo installed software versions | ||
echo "Installed software versions:" | ||
echo "--------------------------------" | ||
echo "gcc version: $(gcc --version | head -n 1)" | ||
echo "cmake version: $(cmake --version | head -n 1)" | ||
echo "clang-tidy version: $(clang-tidy --version | head -n 1)" | ||
echo "clang-format version: $(clang-format --version | head -n 1)" | ||
echo "git version: $(git --version)" | ||
echo "pre-commit version: $(pre-commit --version)" | ||
echo "python3 version: $(python3 --version)" | ||
echo "lcov version: $(lcov --version | head -n 1)" | ||
echo "--------------------------------" |
Uh oh!
There was an error while loading. Please reload this page.