Merge pull request #272 from rajuljha/chore/report/week_9_10

chore(report): Week 9 and 10 project report CI Scanner

Author: GMishx

docs/2024/ci-scanner/updates/2024-08-01.md

@@ -0,0 +1,44 @@
+---
+title: Week 9
+author: Rajul Jha
+tags: [gsoc24, CI]
+---
+<!--
+SPDX-License-Identifier: CC-BY-SA-4.0
+
+SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
+-->
+
+# Week 9
+*(July, 26 2024 - Aug 1, 2024)*
+
+## Meeting 1
+*(July 31, 2024)*
+
+## Attendees
+
+* [Rajul Jha](https://github.com/rajuljha)
+* [Gaurav](https://github.com/GMishx)
+* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
+* [Kaushlendra](https://github.com/Kaushl2208)
+* [Avinal Kumar](https://github.com/avinal)
+
+## Discussions
+
+* Discussed potential issues that might arise with the approach we had in mind:
+  * Current package-url python tool doesn’t support download urls for PyPi, PHP, and cocoapods package managers. They are working on providing the support in this [PR](https://github.com/nexB/fetchcode/pull/119) and this [issue](https://github.com/nexB/fetchcode/issues/116)
+  * Similarly it also does not support golang packages download urls, however the repository urls are available. Since golang packaging is quite complicated. [Here](https://github.com/package-url/packageurl-python/pull/113) is where they are tracking it.
+  * The GH Actions for some of the cyclone dx-tools uses outdated and unmaintained versions of their binaries.
+* All the GH Actions just use their respective cli tools to generate the SBOM’s ultimately. We can remove dependency from it completely by using cli tools directly inside our environment. Since these dependencies might be unnecessary.
+* [Gaurav](https://github.com/GMishx) suggested we create our own Github Actions for generating the Software BOMs in a language dependant manner for the packages using cyclonedx tools under the hood.
+
+## Work Done
+
+* Worked out a plan for how the dependencies scanning would be done inside the CI workflow.
+
+![Screenshot](/img/ci/Download-deps-fossology.png)
+* Started working on the new Github Action for scanning Python Dependencies using [CycloneDX Python BOM tool](https://cyclonedx-bom-tool.readthedocs.io/en/latest/)
+
+## Planning for next week
+* Complete testing the Github Action and successfully generate SBOMs for python projects.
+* Need to figure out how to extract the download-urls from the generated BOMs.

docs/2024/ci-scanner/updates/2024-08-08.md

@@ -0,0 +1,46 @@
+---
+title: Week 10
+author: Rajul Jha
+tags: [gsoc24, CI]
+---
+<!--
+SPDX-License-Identifier: CC-BY-SA-4.0
+
+SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
+-->
+
+# Week 10
+*(Aug, 02 2024 - Aug 08, 2024)*
+
+## Meeting 1
+*(Aug 07, 2024)*
+
+## Attendees
+
+* [Rajul Jha](https://github.com/rajuljha)
+* [Gaurav](https://github.com/GMishx)
+* Katharina
+
+## Discussions
+* To tackle the issue with the python-packageurl tool not supporting `PyPI` and other registries, [Gaurav](https://github.com/GMishx) suggested to try out new tools like [ORT Downloader](https://oss-review-toolkit.org/ort/docs/tools/downloader), [pypi-simple](https://pypi.org/project/pypi-simple/) and REST API for [warehouse](https://warehouse.pypa.io/api-reference/json.html)
+* Talked about tagging and releasing the [fossology-action](https://github.com/fossology/fossology-action) after renaming it to something like **FOSSOps**
+
+## Work Done
+
+* Completed the Github Action for generating SBOMs for python packages. Their are four ways in which we can create an SBOM:
+  * From Python Virtual Environment: `cyclonedx-py environment`
+  * From Pipfile and Pipfile.lock: `cyclonedx-py pipenv`
+  * From pyproject.toml and poetry.lock: `cyclonedx-py poetry`
+  * From requirements.txt file: `cyclonedx-py requirements`
+* Currently the actions looks for these files in the root folder of the repo from where the action is triggered. In future releases, new features like custom path for the file, creating BOMs in xml foramt etc. can be provided.
+* It creates Software Bill of Materials in [cyclonedx](https://cyclonedx.org/specification/overview/) format.
+* BOMs are saves in a directory called `sbom` and for each language, currently python, the bom file is appended with the language specific identifier. 
+For example, for python files, the bom file will `sbom_py.json`.
+* After we have the bom file, we can provide a new cli flag to `fossologyscanner` that will take these sbom files, download the project dependecies listed in them, scan them using fossology scanners (nomos, ojo etc) and spit out the results.
+* Started working on the new feature in `fossologyscanner`
+
+## Planning for next week
+
+* Figure out the method for getting the package metadata after trying out different approaches.
+* Work on successfully downloading and scanning the packages once finalized.
+* Keep an eye on making the code modular in order for easy integration of other language specific actions.