Skip to content

Commit fcee655

Browse files
authored
Merge pull request #272 from rajuljha/chore/report/week_9_10
chore(report): Week 9 and 10 project report CI Scanner
2 parents 52fbe40 + 6f72d54 commit fcee655

File tree

3 files changed

+90
-0
lines changed

3 files changed

+90
-0
lines changed
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: Week 9
3+
author: Rajul Jha
4+
tags: [gsoc24, CI]
5+
---
6+
<!--
7+
SPDX-License-Identifier: CC-BY-SA-4.0
8+
9+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
10+
-->
11+
12+
# Week 9
13+
*(July, 26 2024 - Aug 1, 2024)*
14+
15+
## Meeting 1
16+
*(July 31, 2024)*
17+
18+
## Attendees
19+
20+
* [Rajul Jha](https://github.com/rajuljha)
21+
* [Gaurav](https://github.com/GMishx)
22+
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
23+
* [Kaushlendra](https://github.com/Kaushl2208)
24+
* [Avinal Kumar](https://github.com/avinal)
25+
26+
## Discussions
27+
28+
* Discussed potential issues that might arise with the approach we had in mind:
29+
* Current package-url python tool doesn’t support download urls for PyPi, PHP, and cocoapods package managers. They are working on providing the support in this [PR](https://github.com/nexB/fetchcode/pull/119) and this [issue](https://github.com/nexB/fetchcode/issues/116)
30+
* Similarly it also does not support golang packages download urls, however the repository urls are available. Since golang packaging is quite complicated. [Here](https://github.com/package-url/packageurl-python/pull/113) is where they are tracking it.
31+
* The GH Actions for some of the cyclone dx-tools uses outdated and unmaintained versions of their binaries.
32+
* All the GH Actions just use their respective cli tools to generate the SBOM’s ultimately. We can remove dependency from it completely by using cli tools directly inside our environment. Since these dependencies might be unnecessary.
33+
* [Gaurav](https://github.com/GMishx) suggested we create our own Github Actions for generating the Software BOMs in a language dependant manner for the packages using cyclonedx tools under the hood.
34+
35+
## Work Done
36+
37+
* Worked out a plan for how the dependencies scanning would be done inside the CI workflow.
38+
39+
![Screenshot](/img/ci/Download-deps-fossology.png)
40+
* Started working on the new Github Action for scanning Python Dependencies using [CycloneDX Python BOM tool](https://cyclonedx-bom-tool.readthedocs.io/en/latest/)
41+
42+
## Planning for next week
43+
* Complete testing the Github Action and successfully generate SBOMs for python projects.
44+
* Need to figure out how to extract the download-urls from the generated BOMs.
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: Week 10
3+
author: Rajul Jha
4+
tags: [gsoc24, CI]
5+
---
6+
<!--
7+
SPDX-License-Identifier: CC-BY-SA-4.0
8+
9+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
10+
-->
11+
12+
# Week 10
13+
*(Aug, 02 2024 - Aug 08, 2024)*
14+
15+
## Meeting 1
16+
*(Aug 07, 2024)*
17+
18+
## Attendees
19+
20+
* [Rajul Jha](https://github.com/rajuljha)
21+
* [Gaurav](https://github.com/GMishx)
22+
* Katharina
23+
24+
## Discussions
25+
* To tackle the issue with the python-packageurl tool not supporting `PyPI` and other registries, [Gaurav](https://github.com/GMishx) suggested to try out new tools like [ORT Downloader](https://oss-review-toolkit.org/ort/docs/tools/downloader), [pypi-simple](https://pypi.org/project/pypi-simple/) and REST API for [warehouse](https://warehouse.pypa.io/api-reference/json.html)
26+
* Talked about tagging and releasing the [fossology-action](https://github.com/fossology/fossology-action) after renaming it to something like **FOSSOps**
27+
28+
## Work Done
29+
30+
* Completed the Github Action for generating SBOMs for python packages. Their are four ways in which we can create an SBOM:
31+
* From Python Virtual Environment: `cyclonedx-py environment`
32+
* From Pipfile and Pipfile.lock: `cyclonedx-py pipenv`
33+
* From pyproject.toml and poetry.lock: `cyclonedx-py poetry`
34+
* From requirements.txt file: `cyclonedx-py requirements`
35+
* Currently the actions looks for these files in the root folder of the repo from where the action is triggered. In future releases, new features like custom path for the file, creating BOMs in xml foramt etc. can be provided.
36+
* It creates Software Bill of Materials in [cyclonedx](https://cyclonedx.org/specification/overview/) format.
37+
* BOMs are saves in a directory called `sbom` and for each language, currently python, the bom file is appended with the language specific identifier.
38+
For example, for python files, the bom file will `sbom_py.json`.
39+
* After we have the bom file, we can provide a new cli flag to `fossologyscanner` that will take these sbom files, download the project dependecies listed in them, scan them using fossology scanners (nomos, ojo etc) and spit out the results.
40+
* Started working on the new feature in `fossologyscanner`
41+
42+
## Planning for next week
43+
44+
* Figure out the method for getting the package metadata after trying out different approaches.
45+
* Work on successfully downloading and scanning the packages once finalized.
46+
* Keep an eye on making the code modular in order for easy integration of other language specific actions.
786 KB
Loading

0 commit comments

Comments
 (0)