Structured vulnerabilities data are no longer available?

[pombredanne]

Hi!
Until recently, OpenSSL was publishing structured data about known vulnerabilities at:
https://www.openssl.org/news/vulnerabilities.xml and later JSON at news/secjson

The last archived copy is at https://web.archive.org/web/20230601233405/https://www.openssl.org/news/vulnerabilities.xml

Also the page listing is not longer available at https://web.archive.org/web/20221117101928/https://www.openssl.org/news/secjson/ for the CVE in JSON format as in https://www.openssl.org/news/secjson/CVE-2002-0659.json

We are/were relying on this data https://github.com/nexB/vulnerablecode/blob/4a6734b1bbaa8df6fd816f3eb4fd843a88c1ecec/vulnerabilities/importers/openssl.py#L35

It would be useful to restore some of this data to avoid having to resort to screen scraping the web site and parsing the unstructured advisory text files.
Publishing the backing git repos that contain the data would be even better.

Do you need some help there?

[pombredanne]

Can I interest you in creating a parseable format for your textual advisories?
We recently worked with the GNU libc team for something similar. See aboutcode-org/vulnerablecode#1362

[pombredanne]

Also since we are discussing this, it would make sense to have an official openssl package type at https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst since PURLs are used everywhere in SBOMs and security databases.

This would make this easier for everyone and push this informal usage up to the spec https://github.com/nexB/vulnerablecode/blob/4a6734b1bbaa8df6fd816f3eb4fd843a88c1ecec/vulnerabilities/importers/openssl.py#L64

It would be better than the generic PURLs documented as examples here https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst#generic

[mattcaswell]

https://www.openssl.org/news/vulnerabilities.xml and later JSON at news/secjson

We no longer maintain vulernabilities.xml since it was superseded by the json format. All the CVEs should have an equivalent json file.

Also the page listing is not longer available at https://web.archive.org/web/20221117101928/https://www.openssl.org/news/secjson/

I suspect that was an accidental change as a result of recent technical changes on the website.
@vavroch2010 might know what happened here.

[pombredanne]

@mattcaswell Thanks for the explanation!

To recap, ideally I would like to have:

1. the published advisory text and JSON files kept in a public git repo
2. a structured index of https://www.openssl.org/news/secadv/ and https://www.openssl.org/news/secjson/ (or re-enable HTTP directory listing)
3. a structured format for the text advisories

Some details why:

The data is scattered in multiple places:

• The text advisory https://www.openssl.org/news/secadv/20240125.txt contains most of the data in the CVE JSON but also severity and some specific "non impacted" statements that do not fit in the CVE.
• The email https://mta.openssl.org/pipermail/openssl-announce/2024-January/000292.html often contains the same as text, unless the text is updated later with commits like here. Can it be safely ignored?
• The CVE5 JSON https://www.openssl.org/news/secjson/CVE-2024-0727.json is likely created by hand using the vulnogram app and is missing the text data severity info.
• The web page is derived from the CVE JSON using https://github.com/openssl/web/blob/master/bin/cvejsontohtml.py so I would skip it.
• The CVE.org record at https://www.cve.org/CVERecord?id=CVE-2024-0727 may contain extra data from other CNA (here netapp reference)
• And the NVD record at https://nvd.nist.gov/vuln/detail/CVE-2024-0727 typically contains all CVE data plus NVD or other CNAs CVSS (if worked on)

Therefore:

• It would make sense to keep (and share) the text advisories and the CVE JSON data in a git repo... like this repo. This would help a lot and avoid having to resort to screen scraping, and show a proper versioning of the data.

• Publishing the text advisories in a structured format would be great as there is data only available there. Parsing a custom text format is possible but not fun, nor reliable.

• And being able to list all advisories without having to scrape HTML would be much better!

[pombredanne]

@t8m anyway I can help here too?