Skip to content

Commit 9da357e

Browse files
committed
feat: add Dockerfile analysis for build command detection
Changes: -Function find_dockerfile_from_job: handles finding Dockerfile inside workflow in 2 cases of workflow jobs: -run and -uses. -Simple DockerNode class, so far it stores mainly the dockerfile path retrieved from workflow -Parsing Dockerfile using dockerfile-parse and RUN instruction commands using bashparser.py -Parsing and storing build commands found in Dockerfiles Signed-off-by: Achraf Maghous <achraf.maghous@oracle.com>
2 parents 7013775 + 77eac50 commit 9da357e

File tree

190 files changed

+13161
-2512
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

190 files changed

+13161
-2512
lines changed

.github/pull_request_template.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,18 @@
1-
## Checklist
2-
<!-- Go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
1+
## Summary
2+
<!-- Briefly summarize the purpose and scope of this PR. -->
3+
4+
## Description of changes
5+
<!-- Provide a detailed explanation of the changes made in this PR, why they were needed, and how they address the issue(s). -->
36

4-
-->
7+
## Related issues
8+
<!-- List any related issue(s) this PR addresses, e.g., `Closes #123`, `Fixes #456`. -->
9+
10+
## Checklist
11+
<!-- Go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) -->
512

613
- [ ] I have reviewed the [contribution guide](../CONTRIBUTING.md).
714
- [ ] My PR title and commits follow the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) convention.
815
- [ ] My commits include the "Signed-off-by" line.
916
- [ ] I have signed my commits following the instructions provided by [GitHub](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits). Note that we run [GitHub's commit verification](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) tool to check the commit signatures. A green `verified` label should appear next to **all** of your commits on GitHub.
1017
- [ ] I have updated the relevant documentation, if applicable.
1118
- [ ] I have tested my changes and verified they work as expected.
12-
- [ ] I have referenced the issue(s) this pull request solves.

.github/workflows/_build.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
1+
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
22
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
33

44
# This is a trusted builder implemented as a reusable workflow that can be called by other
@@ -129,7 +129,7 @@ jobs:
129129
# Currently reusable workflows do not support setting strategy property from the caller workflow.
130130
- name: Upload the package artifact for debugging and release
131131
if: matrix.os == env.ARTIFACT_OS && matrix.python == env.ARTIFACT_PYTHON
132-
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
132+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
133133
with:
134134
name: artifact-${{ matrix.os }}-python-${{ matrix.python }}
135135
path: dist

.github/workflows/_build_docker.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2023 - 2024, Oracle and/or its affiliates. All rights reserved.
1+
# Copyright (c) 2023 - 2025, Oracle and/or its affiliates. All rights reserved.
22
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
33

44
# This is a reuseable workflow to build and test the Docker image. Note that this workflow does not
@@ -53,6 +53,10 @@ jobs:
5353
echo "Hash of package should be $ARTIFACT_HASH."
5454
echo "$ARTIFACT_HASH" | base64 --decode | sha256sum --strict --check --status || exit 1
5555
56+
# Login so the docker build has access to the internal dependencies image
57+
- name: Log in to GitHub Container Registry
58+
run: docker login ghcr.io --username ${{ github.actor }} --password ${{ secrets.GITHUB_TOKEN }}
59+
5660
# Build the Docker image without pushing it.
5761
- name: Build the Docker image
5862
env:
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Copyright (c) 2025 - 2025, Oracle and/or its affiliates. All rights reserved.
2+
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
3+
4+
# This is a manually-triggered workflow to build the minimal macaron dependencies image that stores the built-from-source
5+
# Semgrep wheel file. Note that this workflow DOES push the built image.
6+
7+
name: Build Semgrep Wheel Artifact
8+
9+
on: workflow_dispatch
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
build-semgrep-wheel:
16+
name: Build Semgrep wheel
17+
runs-on: ubuntu-latest
18+
permissions:
19+
packages: write # to push the docker image
20+
defaults:
21+
run:
22+
shell: bash
23+
24+
steps:
25+
# To update the semgrep version, please apply the following changes:
26+
# change the version tag in the 'name' description
27+
# change the 'ref' field to use the commit hash of that tag
28+
- name: Check out Semgrep v1.113.0 repository
29+
uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0
30+
with:
31+
repository: semgrep/semgrep.git
32+
ref: 4729a05d24bf9cee8face447e8a6d418037d61d8 # v1.113.0
33+
fetch-depth: 1 # only need most recent commits to this tag
34+
submodules: recursive # semgrep uses many of their own ocaml submodules, which are required to build
35+
36+
- name: Build wheel through docker
37+
# we build to the 'semgrep-wheel' target as we don't need the performance testing, and want to extract the wheel
38+
run: |
39+
docker build --target semgrep-wheel -t semgrep .
40+
docker create --name temp semgrep
41+
mkdir -p wheels/
42+
docker cp temp:/semgrep/cli/dist/. wheels/
43+
docker container rm temp
44+
45+
- name: Log in to GitHub Container Registry
46+
run: docker login ghcr.io --username ${{ github.actor }} --password ${{ secrets.GITHUB_TOKEN }}
47+
48+
# The manylinux image will be a static binary built using musl, suitable for Oracle linux
49+
- name: Build and push semgrep wheel image
50+
run: |
51+
cd wheels
52+
WHEEL=$(find . -type f -name 'semgrep-*manylinux*.whl')
53+
echo "FROM scratch
54+
COPY ${WHEEL} /" >> Dockerfile.semgrep
55+
docker build -t ghcr.io/oracle/macaron-deps:latest -f Dockerfile.semgrep .
56+
docker push ghcr.io/oracle/macaron-deps:latest

.github/workflows/codeql-analysis.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ jobs:
5252

5353
# Initializes the CodeQL tools for scanning.
5454
- name: Initialize CodeQL
55-
uses: github/codeql-action/init@b56ba49b26e50535fa1e7f7db0f4f7b4bf65d80d # v3.28.10
55+
uses: github/codeql-action/init@45775bd8235c68ba998cffa5171334d58593da47 # v3.28.15
5656
with:
5757
languages: ${{ matrix.language }}
5858
config-file: .github/codeql/codeql-config.yaml
@@ -65,4 +65,4 @@ jobs:
6565
# queries: ./path/to/local/query, your-org/your-repo/queries@main
6666

6767
- name: Perform CodeQL Analysis
68-
uses: github/codeql-action/analyze@b56ba49b26e50535fa1e7f7db0f4f7b4bf65d80d # v3.28.10
68+
uses: github/codeql-action/analyze@45775bd8235c68ba998cffa5171334d58593da47 # v3.28.15

.github/workflows/scorecards-analysis.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
1+
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
22
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
33

44
# Run Scorecard for this repository to further check and harden software and process.
@@ -49,13 +49,13 @@ jobs:
4949

5050
# Upload the results as artifacts (optional).
5151
- name: Upload artifact
52-
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
52+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
5353
with:
5454
name: SARIF file
5555
path: results.sarif
5656

5757
# Upload the results to GitHub's code scanning dashboard.
5858
- name: Upload to code-scanning
59-
uses: github/codeql-action/upload-sarif@b56ba49b26e50535fa1e7f7db0f4f7b4bf65d80d # v3.28.10
59+
uses: github/codeql-action/upload-sarif@45775bd8235c68ba998cffa5171334d58593da47 # v3.28.15
6060
with:
6161
sarif_file: results.sarif

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,3 +181,4 @@ docs/_build
181181
bin/
182182
requirements.txt
183183
.macaron_env_file
184+
**/.DS_Store

.pre-commit-config.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ repos:
3030
- id: isort
3131
name: Sort import statements
3232
args: [--settings-path, pyproject.toml]
33+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
3334

3435
# Add Black code formatters.
3536
- repo: https://github.com/ambv/black
@@ -38,6 +39,7 @@ repos:
3839
- id: black
3940
name: Format code
4041
args: [--config, pyproject.toml]
42+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
4143
- repo: https://github.com/asottile/blacken-docs
4244
rev: 1.19.1
4345
hooks:
@@ -65,6 +67,7 @@ repos:
6567
files: ^src/macaron/|^tests/
6668
types: [text, python]
6769
additional_dependencies: [flake8-bugbear==22.10.27, flake8-builtins==2.0.1, flake8-comprehensions==3.10.1, flake8-docstrings==1.6.0, flake8-mutable==1.2.0, flake8-noqa==1.4.0, flake8-pytest-style==1.6.0, flake8-rst-docstrings==0.3.0, pep8-naming==0.13.2]
70+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
6871
args: [--config, .flake8]
6972

7073
# Check GitHub Actions workflow files.
@@ -82,6 +85,7 @@ repos:
8285
entry: pylint
8386
language: python
8487
files: ^src/macaron/|^tests/
88+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
8589
types: [text, python]
8690
args: [--rcfile, pyproject.toml]
8791

@@ -94,6 +98,7 @@ repos:
9498
language: python
9599
files: ^src/macaron/|^tests/
96100
types: [text, python]
101+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
97102
args: [--show-traceback, --config-file, pyproject.toml]
98103

99104
# Check for potential security issues.
@@ -106,6 +111,7 @@ repos:
106111
files: ^src/macaron/|^tests/
107112
types: [text, python]
108113
additional_dependencies: ['bandit[toml]']
114+
exclude: ^tests/malware_analyzer/pypi/resources/sourcecode_samples.*
109115

110116
# Enable a whole bunch of useful helper hooks, too.
111117
# See https://pre-commit.com/hooks.html for more hooks.
@@ -197,6 +203,18 @@ repos:
197203
always_run: true
198204
pass_filenames: false
199205

206+
# Checks that tests/malware_analyzer/pypi/resources/sourcecode_samples files do not have executable permissions
207+
# This is another measure to make sure the files can't be accidentally executed
208+
- repo: local
209+
hooks:
210+
- id: sourcecode-sample-permissions
211+
name: Sourcecode sample executable permissions checker
212+
entry: scripts/dev_scripts/samples_permissions_checker.sh
213+
language: system
214+
always_run: true
215+
pass_filenames: false
216+
217+
200218
# A linter for Golang
201219
- repo: https://github.com/golangci/golangci-lint
202220
rev: v1.64.6

.semgrepignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Items added to this file will be ignored by Semgrep.

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,22 @@
1+
## v0.16.0 (2025-04-24)
2+
3+
### Feat
4+
5+
- detect vulnerable GitHub Actions (#1021)
6+
- check PyPI registry when deps.dev fails to find a source repository (#982)
7+
- add callgraph and build cmd detection for Jenkins (#977)
8+
9+
### Fix
10+
11+
- fix incorrect skip result evaluation causing false positives in PyPI malware reporting (#1031)
12+
- use 'isDefault' version from deps dev api (#1019)
13+
14+
### Refactor
15+
16+
- log the SLSA summary in verbose mode only (#1063)
17+
- log relative paths for file (#1032)
18+
- use problog for suspicious combinations (#997)
19+
120
## v0.15.0 (2025-03-10)
221

322
### Feat

0 commit comments

Comments
 (0)