diff --git a/docs/2024/ci-scanner/index.md b/docs/2024/ci-scanner/index.md index d0acb50500..4cd845e015 100644 --- a/docs/2024/ci-scanner/index.md +++ b/docs/2024/ci-scanner/index.md @@ -6,7 +6,7 @@ slug: /2024/ci-scanner/ ## Author @@ -15,8 +15,8 @@ SPDX-FileCopyrightText: 2024 Rajul Jha ## Contact info -- [Email](mailto:email.here) -- [LinkedIn](https://linkedin.com/in/my-user) +- [Email](mailto:rajuljha49@gmail.com) +- [LinkedIn](https://linkedin.com/in/rajuljha) ## Project title @@ -24,8 +24,37 @@ CI Scanner Improvements ## What's the project about? -Insert Text Here +To be able to easily and continuously scan packages with +fossology checks in CI pipelines, a docker image [fossology/fossology:scanner](https://hub.docker.com/layers/fossology/fossology/scanner/images/sha256-a625b1b10832b98d47429387c18b4fb042f7b09f912b50da14da61fddb11a2ff?context=explore) capable of running license checks (using nomos or ojo) and keyword and copyright scans is available. + +The main aims of this projects is to improve the CI pipeline with various quality of life improvements like: +- Highlight the exact location of violations in the results. +- Enable customization of keywords used by the scanner. +- Allow whitelisting from a custom location. +- Provide the ability to download and scan dependencies. + ## What should be done? -What are the plans for the project? +### Reporting line numbers for violations +- For calculating the previous and new line number from the diff scan output, an algorithm has to be made. +- The line number start byte and end byte information is spit out by all scanners except nomos in json output. That has to be fixed. +- Add the line number calculated to the finding log information as well as write it in results file. + +### Keyword scanning using custom keyword.conf +- Currently, the keyword scanner uses a predefined set of keywords stored at `/usr/local/share/fossology/keyword/agent/keyword.conf.` +- To support this, we also need to document the regex-like format used for specifying these keywords. +- Decision to be made : Should custom `keyword.conf` overwrite the previous one? + +### Providing allowlist.json from a different path +- Currently, the `allowlist.json` is located at the root of the project. +- We want to allow users to optionally specify a different path, using a CLI argument, like --allowlist + +### Allow users to download and scan dependencies +- Currently, the project only scans the source code of the project either in repo/diff manner. +- We additionally want to allow the functionality to scan and dependencies of the project. + +#### Steps to achieve this: + - With the [CycloneDX](https://cyclonedx.org/tool-center/) tool center, we can generate SBOM which contains the dependency download url. + - The SBOM format specifies the package URL (purl) for each dependency. + - Using the [python-packageurl](https://github.com/package-url/packageurl-python#purl-to-url) tool, we can extract the download url from the purl for this purpose. \ No newline at end of file diff --git a/docs/2024/ci-scanner/updates/2023-05-30.md b/docs/2024/ci-scanner/updates/2023-05-30.md deleted file mode 100644 index 4af3bece4a..0000000000 --- a/docs/2024/ci-scanner/updates/2023-05-30.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Community bonding -author: Rajul Jha ---- - - -# Meeting 1 - -*(May 30,2024)* - -## Attendees: - -## Discussion: - -# Meeting 2 - -*(May 18,2023)* - -## Attendees: - -## Discussion: diff --git a/docs/2024/ci-scanner/updates/2024-05-07.md b/docs/2024/ci-scanner/updates/2024-05-07.md new file mode 100644 index 0000000000..58537ac32d --- /dev/null +++ b/docs/2024/ci-scanner/updates/2024-05-07.md @@ -0,0 +1,80 @@ +--- +title: Community bonding +author: Rajul Jha +--- + + +### Meeting 1 + +*(May 7, 2024)* + +### Discussion: +- Could not attend due to family emergency. + + +# Community Bonding Week 1 + +*(May 9, 2024 - May 15, 2024)* + +### Meeting 2 + +*(May 9, 2024)* + +### Discussion: +- I gave my introduction in the meeting. +- Got to know my mentors and colleagues. + +### Work Done: +- Tried to setup the coding environment. +- Set up a Virtual Machine because fossology does not work on Mac.:pensive: +- Faced challenges installing some python packages which were not available for aarch64 architecture. +Worked around the issue by commenting [this out](https://github.com/fossology/fossology/blob/6e6b00c2ded6a1db7647d0da9e97c78ed9ffddf8/install/fo-postinstall.in#L261-L263). + +# Community Bonding Week 2 + +*(May 16, 2024 - May 23, 2024)* + +### Meeting 3 + +*(May 16, 2024)* + +### Discussion: +- Contributors shared their weekly updates. +- Discussed and decided time for weekly project specific meetings. +- Mentors talked about the importance of open communication in open source. + +### Work Done: +- Played around with fossology and scanned a few repositories. +- Tried to understand how scanners work internally. +- Talked with other contributors about the project. + +# Community Bonding Week 3 + +*(May 23, 2024 - May 30, 2024)* + +### Meeting 4 + +*(May 23, 2024)* + +### Discussion: +- Had final discussions on projects. +- Finalized any changes to the project milestones. + +### Work Done: +- Tried building the fossology scanner image locally. +- Played around a while with the image, trying to understand what it does. +- Faced an issue of UI freezing in the VM. Solved it by using [SSH Remote Tunneling](https://code.visualstudio.com/docs/remote/ssh) +- Started to theorize how the line number algorithm would work. + + +### Meeting 5 +*(May 30, 2024)* +- Discussed the project updates from the mentors and contributors. +- Got clear understanding of how to document our progress during the whole program. +- Had discussions with the mentors about how to approach the algorithm for line numbers. + +***This summarizes my community bonding period at Fossology*** \ No newline at end of file