Skip to content

Commit d83bee5

Browse files
GMishxvalens200
GMishx
authored and
valens200
committed
chore(2024): add meeting link for REST API
Signed-off-by: Gaurav Mishra <mishra.gaurav@siemens.com> chore(report): Community bonding report Signed-off-by: Divij Sharma <divijs75@gmail.com> chore(report) : Add community bonding report chore(report) : Update report date chore(report) : Add detailed report for community bonding and elaborated project info chore(report): REST APIs & Text phrases scanning week 3 report chore(report): removed typos fix build issues chore(report): REST APIs week 1 report Signed-off-by: Divij Sharma <divijs75@gmail.com> chore(report): Added REST API Guidelines Signed-off-by: Divij Sharma <divijs75@gmail.com> chore(report): REST APIs week 2 report Signed-off-by: Divij Sharma <divijs75@gmail.com> chore(report): Add coding period week 1 and week 2 project reports chore(deps): bump braces from 3.0.2 to 3.0.3 Bumps [braces](https://github.com/micromatch/braces) from 3.0.2 to 3.0.3. - [Changelog](https://github.com/micromatch/braces/blob/master/CHANGELOG.md) - [Commits](micromatch/braces@3.0.2...3.0.3) --- updated-dependencies: - dependency-name: braces dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> chore(deps): bump ws from 7.5.9 to 7.5.10 Bumps [ws](https://github.com/websockets/ws) from 7.5.9 to 7.5.10. - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@7.5.9...7.5.10) --- updated-dependencies: - dependency-name: ws dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Update docs/2024/text-phrases/index.md Co-authored-by: Gaurav Mishra <gmishx@gmail.com> Signed-off-by: valens niyonsenga <95166477+valens200@users.noreply.github.com> fix(build): re-organized the folder structure and removed build issues
1 parent 74073dc commit d83bee5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+1724
-114
lines changed

‎docs/2024/ci-scanner/index.md

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ slug: /2024/ci-scanner/
66
<!--
77
SPDX-License-Identifier: CC-BY-SA-4.0
88
9-
SPDX-FileCopyrightText: 2024 Rajul Jha <email.here>
9+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49@gmail.com>
1010
-->
1111

1212
## Author
@@ -15,17 +15,46 @@ SPDX-FileCopyrightText: 2024 Rajul Jha <email.here>
1515

1616
## Contact info
1717

18-
- [Email](mailto:email.here)
19-
- [LinkedIn](https://linkedin.com/in/my-user)
18+
- [Email](mailto:rajuljha49@gmail.com)
19+
- [LinkedIn](https://linkedin.com/in/rajuljha)
2020

2121
## Project title
2222

2323
CI Scanner Improvements
2424

2525
## What's the project about?
2626

27-
Insert Text Here
27+
To be able to easily and continuously scan packages with
28+
fossology checks in CI pipelines, a docker image [fossology/fossology:scanner](https://hub.docker.com/layers/fossology/fossology/scanner/images/sha256-a625b1b10832b98d47429387c18b4fb042f7b09f912b50da14da61fddb11a2ff?context=explore) capable of running license checks (using nomos or ojo) and keyword and copyright scans is available.
29+
30+
The main aims of this projects is to improve the CI pipeline with various quality of life improvements like:
31+
- Highlight the exact location of violations in the results.
32+
- Enable customization of keywords used by the scanner.
33+
- Allow whitelisting from a custom location.
34+
- Provide the ability to download and scan dependencies.
35+
2836

2937
## What should be done?
3038

31-
What are the plans for the project?
39+
### Reporting line numbers for violations
40+
- For calculating the previous and new line number from the diff scan output, an algorithm has to be made.
41+
- The line number start byte and end byte information is spit out by all scanners except nomos in json output. That has to be fixed.
42+
- Add the line number calculated to the finding log information as well as write it in results file.
43+
44+
### Keyword scanning using custom keyword.conf
45+
- Currently, the keyword scanner uses a predefined set of keywords stored at `/usr/local/share/fossology/keyword/agent/keyword.conf.`
46+
- To support this, we also need to document the regex-like format used for specifying these keywords.
47+
- Decision to be made : Should custom `keyword.conf` overwrite the previous one?
48+
49+
### Providing allowlist.json from a different path
50+
- Currently, the `allowlist.json` is located at the root of the project.
51+
- We want to allow users to optionally specify a different path, using a CLI argument, like --allowlist
52+
53+
### Allow users to download and scan dependencies
54+
- Currently, the project only scans the source code of the project either in repo/diff manner.
55+
- We additionally want to allow the functionality to scan and dependencies of the project.
56+
57+
#### Steps to achieve this:
58+
- With the [CycloneDX](https://cyclonedx.org/tool-center/) tool center, we can generate SBOM which contains the dependency download url.
59+
- The SBOM format specifies the package URL (purl) for each dependency.
60+
- Using the [python-packageurl](https://github.com/package-url/packageurl-python#purl-to-url) tool, we can extract the download url from the purl for this purpose.

‎docs/2024/ci-scanner/updates/2023-05-30.md

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎docs/2024/ci-scanner/updates/2024-05-07.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
title: Community bonding
3+
author: Rajul Jha
4+
---
5+
<!--
6+
SPDX-License-Identifier: CC-BY-SA-4.0
7+
8+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
9+
-->
10+
11+
### Meeting 1
12+
13+
*(May 7, 2024)*
14+
15+
### Discussion:
16+
- Could not attend due to family emergency.
17+
18+
19+
# Community Bonding Week 1
20+
21+
*(May 9, 2024 - May 15, 2024)*
22+
23+
### Meeting 2
24+
25+
*(May 9, 2024)*
26+
27+
### Discussion:
28+
- I gave my introduction in the meeting.
29+
- Got to know my mentors and colleagues.
30+
31+
### Work Done:
32+
- Tried to setup the coding environment.
33+
- Set up a Virtual Machine because fossology does not work on Mac.:pensive:
34+
- Faced challenges installing some python packages which were not available for aarch64 architecture.
35+
Worked around the issue by commenting [this out](https://github.com/fossology/fossology/blob/6e6b00c2ded6a1db7647d0da9e97c78ed9ffddf8/install/fo-postinstall.in#L261-L263).
36+
37+
# Community Bonding Week 2
38+
39+
*(May 16, 2024 - May 23, 2024)*
40+
41+
### Meeting 3
42+
43+
*(May 16, 2024)*
44+
45+
### Discussion:
46+
- Contributors shared their weekly updates.
47+
- Discussed and decided time for weekly project specific meetings.
48+
- Mentors talked about the importance of open communication in open source.
49+
50+
### Work Done:
51+
- Played around with fossology and scanned a few repositories.
52+
- Tried to understand how scanners work internally.
53+
- Talked with other contributors about the project.
54+
55+
# Community Bonding Week 3
56+
57+
*(May 23, 2024 - May 30, 2024)*
58+
59+
### Meeting 4
60+
61+
*(May 23, 2024)*
62+
63+
### Discussion:
64+
- Had final discussions on projects.
65+
- Finalized any changes to the project milestones.
66+
67+
### Work Done:
68+
- Tried building the fossology scanner image locally.
69+
- Played around a while with the image, trying to understand what it does.
70+
- Faced an issue of UI freezing in the VM. Solved it by using [SSH Remote Tunneling](https://code.visualstudio.com/docs/remote/ssh)
71+
- Started to theorize how the line number algorithm would work.
72+
73+
74+
### Meeting 5
75+
*(May 30, 2024)*
76+
- Discussed the project updates from the mentors and contributors.
77+
- Got clear understanding of how to document our progress during the whole program.
78+
- Had discussions with the mentors about how to approach the algorithm for line numbers.
79+
80+
***This summarizes my community bonding period at Fossology***

‎docs/2024/ci-scanner/updates/2024-06-06.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
title: Week 1
3+
author: Rajul Jha
4+
---
5+
<!--
6+
SPDX-License-Identifier: CC-BY-SA-4.0
7+
8+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
9+
-->
10+
11+
# Week 1
12+
*(May 31, 2024 - June 6, 2024)*
13+
14+
## Meeting 1
15+
*(June 5, 2024)*
16+
17+
## Attendees
18+
* [Rajul Jha](https://github.com/rajuljha)
19+
* [Gaurav](https://github.com/GMishx)
20+
* [Kaushlendra](https://github.com/Kaushl2208)
21+
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
22+
* [Avinal Kumar](https://github.com/avinal)
23+
* Katharina
24+
25+
## Discussions
26+
27+
* Discussed [unified diff](https://www.gnu.org/software/diffutils/manual/html_node/Example-Unified.html) format to populate the data fetched from the Github and Gitlab API's
28+
* We also discussed after extraction of the content in unified diff format, how will we extract the line number from it.
29+
* We discussed potential risks that we had to keep in mind before approaching this:
30+
* The scanner results should give required info for searching line number.
31+
* The scanner results should not be affected by this.
32+
33+
34+
## Updates
35+
* Came across [this thread](https://stackoverflow.com/questions/24455377/git-diff-with-line-numbers-git-log-with-line-numbers) on stackoverflow. Used this gawk command as a reference and wrote a python script to convert the api content into unified diff format.
36+
* Create a new class `FormatResult` to handle all the formatting of the results and diff content.
37+
* Also, created a function to extract the line number from the formatted diff content.
38+
* Tested both the scripts extensively and all cover potential edge cases.
39+
40+
## Planning for next week
41+
* Use the script on the diff content and try to find the line number for copyright and keyword scanners.
42+
* Add relevant byte info to the JSON output of nomos scanner.
43+
* Figure out what to do for repo scans.

‎docs/2024/ci-scanner/updates/2024-06-13.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: Week 2
3+
author: Rajul Jha
4+
---
5+
<!--
6+
SPDX-License-Identifier: CC-BY-SA-4.0
7+
8+
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
9+
-->
10+
11+
# Week 2
12+
*(June 6, 2024 - June 13, 2024)*
13+
14+
## Meeting 1
15+
*(June 12, 2024)*
16+
17+
## Attendees
18+
* [Rajul Jha](https://github.com/rajuljha)
19+
* [Gaurav](https://github.com/GMishx)
20+
* [Kaushlendra](https://github.com/Kaushl2208)
21+
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
22+
23+
## Discussions
24+
25+
* Mentors reviewed my PR's and gave me feedback like changes regarding squashing commits into one.
26+
* There was a discussion regarding the copyright and keyword scanners breaking because of new formatting since they are regex based. I tested the code and it was working without any descrepencies.
27+
* Mentors gave me pointers for adding byte info to nomos, which I was unable to figure out before.
28+
* I also found a bug in `SpdxReport` where it breaks when an unknown license reference is found by the scanner. I had a discussion with the mentors regarding solving it.
29+
30+
## Updates
31+
* Completed the Format Results functionality for the copyright and keyword scanners. :grin:
32+
* Add the line numbers to the STDOUT, text_report and bom report function.
33+
* Sent out a [PR(#2754)](https://github.com/fossology/fossology/pull/2754) with the given changes.
34+
* Add the line numbers for repo scans also in a seperate [PR(#2756)](https://github.com/fossology/fossology/pull/2756)
35+
36+
## Planning for next week
37+
* Add line numbers for the remaining nomos and ojo scanners which requires:
38+
* Adding relevant byte info to the JSON output of nomos scanner.
39+
* Solve the bug found in `SpdxReport`.
40+
* Upgrade the spdx_tools library to latest version.
41+
* Try to figure out a method of providing custom keyword scanning by providing a path to a custom `keyword.conf` file by the user.

‎docs/2024/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ More info to come here.
5454
| General Meeting | *Thursday* 13:30 - 14:30 UTC | [Jitsi](https://meet.jit.si/moderated/5a655b3b6f3b4f83cddb13b93ac5408d6de48bf4ce1049f4128aa1c885478d48) | [.ics](/ics/gsoc_2024_weekly.ics) |
5555
| - | - | - | - |
5656
| SPDX Related | *Tuesday* 10:30 - 11:20 UTC | [Jitsi](https://moderated.jitsi.net/d623bb1284a54c83958eff31d2ecce9ed6b894312eda4ed9b400d5963f4e18b6) | [.ics](/ics/gsoc_2024_spdx.ics) |
57+
| REST API | *Tuesday* 11:00 - 11:50 UTC | [Jitsi](https://moderated.jitsi.net/d623bb1284a54c83958eff31d2ecce9ed6b894312eda4ed9b400d5963f4e18b6) | [.ics](/ics/gsoc_2024_rest.ics) |
5758
| CI Scanner | *Wednesday* 10:30 - 11:00 UTC | [Jitsi](https://moderated.jitsi.net/39896aad61bc4a27b9418ee6b78689348c65790e889046069dbe9c8c34110c9a) | [.ics](/ics/gsoc_2024_ci.ics) |
5859
| Scheduler Overhaul | *Friday* 10:30 - 11:00 UTC | [Jitsi](https://moderated.jitsi.net/5444f675f5ce47c788fa4238a6a958c53d3e62804e9243d5b807fbaa81f3120f) | [.ics](/ics/gsoc_2024_scheduler.ics) |
5960
| AI Powered Scanners | *Thursday* 10:30 - 11:20 UTC | [Jitsi](https://moderated.jitsi.net/15ee0bf46cb345e4accc817ed2967b55db216bf57c894c30bd1550ecf3ec3ace) | [.ics](/ics/gsoc_2024_ai.ics) |
@@ -62,3 +63,4 @@ More info to come here.
6263
Photos to come soon.
6364

6465
Thanks for being part of the community. 💚
66+

0 commit comments

Comments
 (0)