-
Notifications
You must be signed in to change notification settings - Fork 203
MeetingMinutes2022
We meet online on Tuesday at 16:00 UTC as a reference. See https://www.timeanddate.com/worldclock/meeting.html to get the time in your timezone.
Join us at https://meet.jit.si/VulnerableCode
The current meeting notes is at:
Here are the running meeting notes:
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran) Agenda:
Ziad:
- question on cwe2 issue #4 wrt. cwe 399 which does not exists in the download from MITRE. The upstream page even states that this is deprecated and should not be used.
- importlib.resources usage for cwe2 data file loading
Hritik
- There is new PR to validate link typos in the CI that needs approval
Tushar
- licensing issues with importer for "suse scores", and "ubuntu: need to reach out to respective security teams
- suse license was changed for CVRF to CC-BY
- ubuntu license may be GPL which is not practical for data
Keshav:
- nothing special to bring up.
John:
- working on the Tomcat importer migration, wrestling with the HTML structure of Tomcat data
Philippe
- The "packaging" library issue (that led to packvers creation) may mean that we need to vendor some libraries to avoid these issues.
Agenda:
- Apache Tomcat and CVE entries
- CWE
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
John:
- Discussing the structure of apache tomcat data.
Ziad:
- Discussing his progress and follow-up question on CWE.
Agenda:
- Apache HTTPD and Kafka
- CWE
- Conflicting advisories
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
Philippe:
- We have two advisories from different sources that tell different ranges for a package, we should add advisory scoring.
Ziad:
- Discussing progress in CWE.
John:
- Follow up questions on apache kafka and httpd.
Agenda:
- Apache Kafka follow-up questions
- UI for CWE
- Ingesting CWEs
- Problem of conflicting advisories
- Ingesting advisories that don't have identifiable purls
Participants:
- Philippe (@pombredanne)
- Tushar (@tg1999)
- Ziad (@ziadhany)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
Philippe
- We found 2 datasources NVD and Github which were reporting different affected version ranges for same vulnerability
- Solution: store the source of advisory with the package-vulnerability relation, add some vulnerability clarity scoring.
John
- Discussing different type of version ranges found in Apache Kafka
Ziad
- Ingesting CWE data through NVD importer and modify improver to store CWEs.
Tushar
- CISA provides https://www.cisa.gov/known-exploited-vulnerabilities-catalog and these products can't be identified as a purl, how should we store them ?
- Solution: We should store them as a reference
Agenda:
- Follow up issues on Apache HTTPD importer
- Review on 597
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
Discussions:
John
- Discussed issues https://github.com/nexB/vulnerablecode/issues/1018, https://github.com/nexB/vulnerablecode/issues/1019 and https://github.com/nexB/vulnerablecode/issues/1020.
Philippe
- Status review on https://github.com/nexB/vulnerablecode/issues/597.
Agenda:
- Model changes to accomodate CWE
- Status on PRs
- Questions related to Apache HTTPD
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
Discussions:
John
- Suggested his approach to add fixed version from the Apache HTTPD advisory data and asked some clarifying questions regarding the comparators used.
Ziad
- Asked how can we add CWE field in the VCIO models.
Philippe
- Quick status on pending PRs.
Agenda:
- Apache Httpd importer
- Releasing CWE 2 library
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
Discussions:
John
- Continuing his work on migrating the Apache HTTPD importer.
Ziad
- Needs CWE2 library to be released so he can continue https://github.com/nexB/vulnerablecode/pull/782.
Agenda:
- Update on public vulnerablecode instance
- Forever vulnerable packages
- Ruby Importer
- Milestone 31 review
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
- Michael Herzog (@mjherzog)
Discussions:
Philippe
- Public instance may be released by today
- Manually enter forever vulnerable packages wherever we can collect it from.
- Collect advisories that have inaccurate data and store them manually.
Ziad
- Discussed Ruby Importer https://github.com/rubysec/ruby-advisory-db/blob/master/rubies/ruby/CVE-2022-28738.yml . We will store inverted of fixed version ranges.
Agenda:
- Namespace for postgres importer
- Improve swap function for CVSS
- A comment over current progress
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
- Michael Herzog (@mjherzog)
Discussions:
John
- Do we need namespace for postgres importer?
- Philippe replied: We don't need a namespace for postgres importer, but we need to specify a type for postgres https://github.com/nexB/vulnerablecode/issues/990
Dennis
- Needed a comment from Philippe over current status of VCIO.
- Comment from Philippe: We are very near to a public release.
Ziad
- Discussed CVSS swap function, will have a session with Philippe to discuss it further.
Agenda:
- Fireeye Importer
- Comment over performance of VCIO
- Support for generic versions univers
- Documentation issues
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis M. Clark (@DennisClark)
- Michael Herzog (@mjherzog)
Discussions:
Michael
- Discussed https://github.com/nexB/vulnerablecode/issues/977 and https://github.com/nexB/vulnerablecode/issues/976 and we also decide to get rid of wiki and use google docs to take meeting minutes in future.
Dennis
- Needed a comment from Philippe over current performance of VCIO.
- Comment from Philippe: Issues still persist on search, we take 10-20 seconds for a search, we don't use index right now, we should have a full text search index.
Philippe
- Went through how we can develop generic version ranges using libversion.
Ziad
- Discussed https://github.com/nexB/vulnerablecode/pull/795 .
Agenda:
- Discussed all PRs
- Fix vulnerability lookup
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Tushar
- Reviewed all open PRs and marked the status Draft wherever it was required and closed some redundant PRs and also merged some PRs.
Philippe
- Fixing vulnerability lookup and also make the vulnerability searching faster https://github.com/nexB/vulnerablecode/pull/955.
Agenda:
- Importers with no declared license
- Milestone v31
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
John
- Will be using "LicenseRef-scancode-unknown" as spdx license expression, when there is no declared license associated with any importer.
Philippe
- Discussed milestone v31.0 https://github.com/nexB/vulnerablecode/milestone/5
Agenda:
- Testing Git Importer
- CWE and CVSS
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Tushar
- We released v30.0.0, all issues are completed https://github.com/nexB/vulnerablecode/milestone/4. https://github.com/nexB/vulnerablecode/releases/tag/v30.0.0
John
- John made some progress in migrating ArchLinux Importer https://github.com/nexB/vulnerablecode/pull/935
Ziad
- Working on merging latest skeleton in cwe2 https://github.com/nexB/cwe2/pull/2.
- Writing migrations for calcualating cvss score from vector https://github.com/nexB/vulnerablecode/pull/747.
Agenda:
- v30.0.0
- ArchLinux Importer
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Tushar
- All pending issues for milestone v30.0.0 are completed https://github.com/nexB/vulnerablecode/milestone/4 and will be released soon.
John
- John picked up migration of ArchLinux Importer.
Agenda:
- Ziad pending PRs
- Python2JS
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Ziad
- Provided order for merging pending PRs https://github.com/nexB/vulnerablecode/issues/925
Keshav
- Discussed how we can add VulnTotal in browser using Python2JS.
Agenda:
- CWE and CVSS
- Re-discussed Vulnerability Severity
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Ziad
- Forked CWE repo from https://github.com/Julian-Nash/cwe here https://github.com/nexB/cwe2 since we didn't get any update here https://github.com/Julian-Nash/cwe/issues/4
Dennis
- Brought up https://github.com/nexB/vulnerablecode/issues/889 and Ziad is working on this.
Tushar
- Working on issues related to release https://github.com/nexB/vulnerablecode/pull/898, https://github.com/nexB/vulnerablecode/pull/899
Agenda:
- UI Review
- Release Status
- VCID
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Philippe
- Status on release https://github.com/nexB/vulnerablecode/milestone/4
- Discussed UI improvements done here in https://github.com/nexB/vulnerablecode/pull/894
Tushar
- Working on migrating from VULCOID to VCID https://github.com/nexB/vulnerablecode/pull/896
Agenda:
- Changes in VulnTotal CLI
- Threading in VulnTotal
- Next steps and priority for UI
- Release Status
- Better way to represent Vulnerability Severity
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
Discussions:
Philippe
- Status on release https://github.com/nexB/vulnerablecode/milestone/4
Keshav
- VulnTotal CLI PR discussion https://github.com/nexB/vulnerablecode/pull/801
- Discussed that we will not be using threading in VulnTotal for now.
Dennis
- Better way to represent Vulnerability Severity https://github.com/nexB/vulnerablecode/issues/889
Agenda:
- Alias URL
- Running profiling for all importers
- Rust Importer
- Review on status
- Issue on affected vs fixed package
- Staging Instance
- Status on release
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Michael Herzog (@mjherzog)
Discussions:
Philippe
- Quick status on release https://github.com/nexB/vulnerablecode/milestone/4
- A better way to represent affected vs fixed package https://github.com/nexB/vulnerablecode/issues/863
- Need a staging instance for reviewing UI PRs
Ziad
- Working on profiling all importers
- Started work on Rust Importer
Keshav
- VulnTotal planning https://github.com/nexB/vulnerablecode/projects/8
Agenda:
- Snyk.io and follow-up questions
- GitImporter and parallel run importer
- Status on release
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Michael Herzog (@mjherzog)
Discussions:
Philippe
- Quick status on release https://github.com/nexB/vulnerablecode/milestone/4
Ziad
- Discussed parallel run of importers https://github.com/nexB/vulnerablecode/pull/843
Keshav
- Discussed forever malicious packages found in snyk.io for example: https://security.snyk.io/vuln/SNYK-PYTHON-TESTPIPPER-2980411
Agenda:
- UI
- VulnTotal Planning
- Status on release
- PR for shared Importer
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Michael Herzog (@mjherzog)
Discussions:
Philippe
- Quick status on release https://github.com/nexB/vulnerablecode/milestone/4
Ziad
- Discussed shared importer PR https://github.com/nexB/vulnerablecode/pull/780
Keshav
- VulnTotal Planning https://github.com/nexB/vulnerablecode/projects/8
John
- Working on UI https://github.com/nexB/vulnerablecode/pull/813
Agenda:
- UI
- Git Importer
- Status on public release
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Michael Herzog (@mjherzog)
Discussions:
Philippe & Michael
- Identified issues that are needed to be completed for public release here https://github.com/nexB/vulnerablecode/milestone/4
Ziad
- Discussed refactoring of Git Importer
Keshav
- Discussed PR at univers for refactoring gem and making nuget hashable https://github.com/nexB/univers/pull/75
John
- Working on UI
- Questions on UI
- Review on UI
Agenda:
- Tag Release
- API Design
- Categorize Version
- VulnTotal
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Thomas Druez (@tdruez)
- Michael Herzog (@mjherzog)
Discussions:
Philippe
- Tagged a release
- https://github.com/nexB/vulnerablecode/issues/811
Ziad
- Categorize Versions in NPM Importer
Thomas
API Design Changes
- For a single PURL - Show full detail about that package (GET)
- For a list of PURLs - Details to be shown API URL, PURL, IS_VULNERABLE (POST)
- Version the API
- Add a search endpoint in API https://github.com/nexB/vulnerablecode/issues/810
- Add support for bulk search in CPEs https://github.com/nexB/vulnerablecode/issues/808
- API fixed_packages issues https://github.com/nexB/vulnerablecode/issues/809
Agenda:
- Release Candidate
- VulnTotal Status
- Ruby Importer PR
- New UI
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- John M. Horan (@johnmhoran)
- Dennis Clark (@DennisClark)
Discussions:
Philippe
- Discussed Release Candidate
- Put DB Dump and change documentation
- Philippe will refresh the DB and put a weekly scheduler for this
Ziad
- Discussed Ruby Importer PR https://github.com/nexB/vulnerablecode/pull/799
- Discussed Git Importer
- Will work on NPM importer
Hritik
- Status on VulnTotal project
Keshav
- Will work on GHSA
- OSV - problems in deps and osv to convert into purl, will push tests for deps
- CLI - Need a raw dump
- GitHub Validator - Need to query by version - Reuse the existing GraphQL query and try to modify the existing query
Dennis
- Scheduling refresh of VCDB
- Make API more efficient https://github.com/nexB/vulnerablecode/issues/771
John Horan
- Exploring VC and VCIO
- Worked on a seperate django project
- Creating a new branch and adapt to current UI, and made a new UI
Agenda:
- Release and Status
- CWE
- GSD Importer
- Change of timing for weekly sync
Participants:
- Tushar (@tg1999)
- Philippe Ombredanne (@pombredanne)
- Ziad (@ziadhany)
Discussions:
Philippe
- Discussed Release
- Updated status of importer-improver migration https://github.com/nexB/vulnerablecode/issues/597
Ziad
- Discussed GSD Importer https://github.com/nexB/vulnerablecode/pull/787
- Discussed CWE integration in Vulnerablecode https://github.com/nexB/vulnerablecode/pull/782
Tushar
- Change weekly sync time to 15:00 GMT every Tuesday
Agenda:
- Keshav's GSoC status
- Ziad's GSoC status
Participants:
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
Discussions:
Ziad
- Added GSD Importer
- Will be working on CERTCC importer
- GSD Importer the reference and severities are separated, I think We have the same issue like osv. https://github.com/cloudsecurityalliance/gsd-database
Keshav
- Added OSV Importer Test
- Added Deps dev validator
- Merged the Initial Vulntotal PR
- Add tests for deps and github validator
- Discussed parsing of Maven Purl
Agenda:
- Keshav's GSoC status
- Ziad's GSoC status
- Fixed Packages filter in API
- Adding URLs for CPEs
Participants:
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
Discussions:
Ziad
- Worked on Pypa Importer and Shared OSV and added support for CWE
- Will be working on UVI database Importer
Keshav
- Worked on CLI support for VulnTotal and adding tests for OSV
- Will be working on deps.dev
Tushar
- Added fixed Packages filter in API and URLs for CPEs
Agenda:
- Keshav's PR
- Vulnerablecode Release
- Importing Android Advisories
Participants:
- Tushar (@tg1999)
- Keshav (@keshav-space)
Discussions:
Keshav
- https://github.com/nexB/vulnerablecode/issues/778 reported this for importing data from Android advsiories.
- https://github.com/nexB/vulnerablecode/pull/777 Initial config for vulnTotal
- https://github.com/nexB/univers/pull/75 Refactor gem and make nuget hashable
Tushar
- Discussed the prep for release https://github.com/nexB/vulnerablecode/pull/776 for 30.0.0rc1
Agenda:
- Univers PR on semver (Keshav)
- Code orgaznition for osv importer (Ziad)
- Upcoming release and licensing (Philippe)
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Ziad (@ziadhany)
- Keshav (@keshav-space)
Discussions:
Keshav
- There is a PR pending in univers, not yet merged
- we discussed a possible bug in the current PR as the composer version should sort correctly and does not
- Ziad offered help for setting up a debugger
- we discussed possible forking of our own semantic version library and a possible generic version implementation
Ziad
- OSV advisory uses same OSV schema as used in Pysec
- we discussed how to possibly organize the code using a shared osv.py module and specific importers (like pysec) that use this code. Ziad plans to move existing pysec code there.
Philippe
- Working on release
- Discussing license of the data: CC-BY-SA option
Agenda:
- VulnerableCode release
- CSV Report
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- lf32 (@lf32)
Philippe is working on the 0.9.0 release.
Tushar prepared a CSV file to help improve our findings https://github.com/nexB/vulnerablecode/issues/755#issuecomment-1147901429
Agenda:
- VulnerableCode Release
- SyliusResourceBundle versioning issues
- scoring systems
- VulnTotal
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
Planning a release 0.9.0 by the end of this week.
Tracking at https://github.com/Sylius/SyliusResourceBundle/issues/455
We could adopt something like the following for our scoring systems::
scoring_elements = models.CharField(
max_length=100,
help_text="Supporting scoring elements as a string. "
"For example a CVSS vector or Important, High, Medium, Low. "
"Typically used to compute the value.
)
def save(self, *args, **kwargs):
if not self.value and self.scoring_elements:
self.value = SCORING_SYSTEMS[self.scoring_system].as_score(self.scoring_elements)
super().save(*args, **kwargs)
Need to schedule a VulnTotal specific meet sometime following week.
Agenda:
- Welcome GSoC selected students
- OSV Importer
- Strange GitLab version ranges
- VulnerableCode security issues
- Need for VulnTotal (rant)
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
We congratulate and welcome this year's Google Summer of Code students - Keshav (@keshav-space) and Ziad (@ziadhany). We will be working towards the respective proposals and set the next milestone for VulnerableCode.
Tushar is working on importing OSV for github (via https://github.com/github/advisory-database). Ziad will be working on PyPa advisory database (https://github.com/pypa/advisory-database/). It's is expected to be exactly the same db as OSV. We will find out more as we implement. We also need support for different ecosystems.
Keshav observed some test cases failing against this composer test data <https://github.com/nexB/univers/blob/main/tests/data/composer_gitlab.json#L11852-L11857>
_.
Gitlab is making up version ranges, the following range supplied by gitlab in the aforementioned data makes no sense::
"gitlab_native": "<1.3||>=1.3.0
Gitlab also converts these ranges to English text which is more painful than helpful. For the above range, we have the following advisory: https://advisories.gitlab.com/advisory/advpackagist_sylius_resource_bundle_CVE_2020_5220.html
Upon further investigation, we find that upstream itself is not making any sense with their version range and the representation used https://github.com/Sylius/SyliusResourceBundle/security/advisories/GHSA-8vp7-j5cj-vvm2
The NVD Entry (https://nvd.nist.gov/vuln/detail/CVE-2020-5220) points us to other sources:
- Upstream: https://github.com/Sylius/SyliusResourceBundle/security/advisories/GHSA-8vp7-j5cj-vvm2
- Friendsofphp: https://github.com/FriendsOfPHP/security-advisories/blob/master/sylius/resource-bundle/CVE-2020-5220.yaml
Friendsofphp <https://github.com/FriendsOfPHP/security-advisories/blob/master/sylius/resource-bundle/CVE-2020-5220.yaml>
_ is parsing the advisory in a different way that appears to be inconsistent with the upstream <https://github.com/Sylius/SyliusResourceBundle/security/advisories/GHSA-8vp7-j5cj-vvm2>
_ itself. This could make it a potential candidate for VulnTotal.
As the upstream data is not clear by itself, following interpretations were suggested by Philippe::
<=1.3.12||>=1.4.0,<=1.4.5||=1.5.0||>=1.6.0,<=1.6.2
vers:composer/<=1.3.12|>=1.4.0|<=1.4.5|1.5.0|>=1.6.0|<=1.6.2
They might want to say this:
<=1.3.12||>=1.4.0 <=1.4.5||=1.5.0||>=1.6.0 <=1.6.2
Though, we cannot be totally certain before confirming. Keshav will be creating a ticket about this on the upstream and will follow up regarding the same.
PS: The semver specs does not consider 1.3
as a valid semver. It needs to be coerced using Version.coerce
and then Version.coerce(1.3) == Version.coerce(1.3.0)
We need to plan for a private tracker for VulnerableCode security issues. In order to protect downstream from possible threats, security issues will be first taken care of in the private tracker and made public at a later date.
Of all the datasoures, there's one that's good: The question is which one is the good one ?
Everyone is making something up. Each of them makes something up slightly differently, you have as many ranges as there are downstream interpretations. None of them is faithful to the upstream. We have to ensure - at some point of time - not so much notion of confidence but notion of upsteram and downstream. If everyone interprets version ranges at their whim, there's this game in France - it's called The Arab Telephone aka Chinese Whispers <https://en.wikipedia.org/wiki/Chinese_whispers>
_ - it's a game for kids - you start in circle and you have to say somethitng to the person next to you and repeat what was said to you when it gets back. The end result is very different than the original sentence. Imagine the problem with large s/w team with 100s of false positives. You have to redo everything, thus we need VulnerableCode and VulnTotal.
Agenda:
- Severity score and CVSS vector
- Reference model
- OSS Summit
- Unstable sort
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
If a CVSS vector is available, we should calculate score from there and store it. If not, then store the score only. The following library could come handy while dealing with cvss vectors and scores https://github.com/skontar/cvss
Our reference model needs to be self standing. Also, the severity needs to be detached from reference and should be treated as its own entity. A structure like the following is possible:: vuln -- Refencce |----Severity --> reference
We need to schedule a meet for preparing VulnerableCode's presentation for the summit
unstable sort in case where versions only differ in their build was addressed. More details in the following tickets:
- https://github.com/rbarrois/python-semanticversion/issues/132
- https://github.com/nexB/univers/pull/69#issuecomment-1129756631
Agenda:
- Release design
- Gitlab version ranges
- Findings on CWE
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
We need better storage of version ranges. Confidence for github and osv are low, for now we may assign 9 (a discount, we'll need to come up with a proper method of assigning confidence). See ticket: https://github.com/nexB/vulnerablecode/issues/733
Deadline: End of may for release and make public.
Targets:
- Fix ziad's PR
- Run cron job / systemd timer to run importers and improvers.
- 632, 719, 723 are good to go for the release
https://github.com/nexB/univers/issues/67 - semver is the way to go, debain and redhat are good to go
We can use the list of all CWEs for populating our CWE database. See ticket comment: https://github.com/nexB/vulnerablecode/issues/651#issuecomment-1122211176
Agenda:
- UI Changes
- Openssl univers PR
- AffectedPackage as a model
- PyPI OSV Importer references
- Next release
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
Issue: https://github.com/nexB/vulnerablecode/issues/714 was discussed briefly. We used to have something simiar in the past. Need to look that up and see the improvements we can make.
PR: https://github.com/nexB/univers/pull/61 The __post_init__
usage for major, minor versions were discussed. (See PR for more)
We are missing crucial information contained inside AffectedPackage
in Advisory
model. We might need a model specifically for AffectedPackage
so that we may create relations to vulnerabilities and possibly packages based on version ranges. Being tracked in: https://github.com/nexB/vulnerablecode/issues/727
Meaningless severities in OSV PyPI references were discovered. We don't want to import anything that does not start with PYSEC in the PyPI OSV importer for now, we'll address rest in OSV and GitHub importer. Do not do random assignments for severites (as done by OSV). See: https://github.com/nexB/vulnerablecode/pull/632#discussion_r863331941
We need the following fixed before the next release:
- endpoints to consume our data
- sufficient data
- UI (Can be postponed until next release as well)
Agenda:
- UI Changes
- PR Reviews on univers: quick reminder
- Scoring system docs
- Reference urls
- CWE
- Improvers redesign
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
UI Changes to accomodate the new model were addressed and agreed upon. We want to get a release after the fresh UI, also merging in-flight PRs for importers and improvers in the release could be a plus if possible.
Quick reminder to reviem in flight PRs in univers.
What's the difference between cvss3.1 with vector or without. See https://www.first.org/cvss/calculator/3.0 and the ticket for more: https://github.com/nexB/vulnerablecode/issues/713
Should we add type for refernence urls? Should it be a list of choices or just a charField. It might help us better categorize references and not leave the upstream data about reference types.
Philippe suggests having a JSONField
with elements nvd:customer-entitlement, osv:fix, osv:web
might be helpful.
For more, see: https://github.com/nexB/vulnerablecode/issues/712
To implement CWE (categorization) support a system similar to currently implemented ScoringSystem could be used. We are considering CWE, OWASP top 10 and many more categorization/scoring techniques. See ticket comment for more: https://github.com/nexB/vulnerablecode/issues/651#issuecomment-1111549674
Discussed the current problems with the improver design and a possible solution. There exists TOCTOU problems and non-scalability. Possibility of should_run()
method was also discussed but was not preferred over the other solution. Refer the ticket for more details: https://github.com/nexB/vulnerablecode/issues/701#issuecomment-1111546652
Agenda:
- VULCOID
- Design for Improver
- LF Submission
- OpenSSL PR
- Commit Reviews
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
See: https://github.com/nexB/vulnerablecode/issues/695
Improvers are constraining. For eg: Improving reference id to reference URL, improving gity data (not an advisory).
The problem is with both interesting_advisories
and get_inferences
where both of them expect AdvisoryData
A condition which checks if this improver runs or not then a handling thing.
We should receive a QuerySet and do our stuff with this QuerySet.
TOCTOU conditions are also present, you check at interesting_advisories
and use the value at get_inferences
. Could do https://docs.djangoproject.com/en/4.0/ref/models/querysets/#select-for-update to avoid TOCTOU but not in interesting_advisories
but closer where things are going to change
Current implementation will become a subclass which is a advisory based improver.
See: https://github.com/nexB/vulnerablecode/issues/701
Need to review: https://github.com/nexB/univers/pull/61
https://github.com/nexB/vulnerablecode/pull/693/files#r852942085
Remove or []
from https://github.com/nexB/vulnerablecode/blob/47f6ae62d919fff5094b960c1f56bc0c6b8b4be3/vulnerabilities/improver.py#L69-L74
Agenda:
- Review of Nginx Test PR
- Default Improver
- Status on Migrations
- UI issues
- Public Deployment
- OpenSSL Issue
Participants:
- Philippe (@pombredanne)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Using Testing Env variables like this VULNERABLECODE_REGEN_TEST_FIXTURES=yes pytest -vvs
- Add License for Nginx Importer
TBD:
- Shifting from instance based method for Scoring System to Subclasses https://github.com/nexB/vulnerablecode/issues/694
- Move evolve_purl to PackageURL
- Creating a VulnerableCode score as an interpretation of the other scoring system
- Updated the status and order on https://github.com/nexB/vulnerablecode/issues/597
- https://github.com/nexB/vulnerablecode/issues/675
- https://github.com/nexB/vulnerablecode/issues/676
- https://github.com/nexB/vulnerablecode/issues/695
- Beta comes before Official Release but when sorting Beta is coming after Official Release ( need to add tests for this )
Agenda:
- Regen
- NVD Licence
- Map cpe to packageurl
- GSoC Proposal Review
Participants:
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
What is regen in test cases ? Where is the docs ? Need to talk to Philippe
We have vulnerablecode/vulnerabilities/management/commands/create_cpe_to_purl_map
as of now. Need to discover more.
Issue: https://github.com/nexB/vulnerablecode/issues/665 It was not present earlier as well: https://github.com/nexB/vulnerablecode/blob/ed2131656b1b3030c2f9eb28e25c10dbbedc8e1d/vulnerabilities/importer_yielder.py#L129-L133
Share your proposals at Gitter
Agenda:
- GitLab importing
- Nix tests failing
- Postgres index issue
- Server Deployment
- VulnTotal project idea
- Univers
- GSoC Video
- OSV ecosystem
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Ziad (@ziadhany)
There is need to have a special common way to generate a range from OSV. Normally, ecosystems in OSV should map 1-1 with versioning scheme in univers and package type in PackageURLs. The reason being, when OSV was being developed, PackageURL was a part of the discussion.
Philippe will follow up on that and will write a blog post. Also, YouTube channel and the logo.
Made changes by Keshav. There was a bug in black, which broke all CIs they have issued a fix <https://github.com/psf/black/issues/2964>
_. Thus, updated black.
A periodic CI run of all projects in AboutCode will be a good idea.
In Scancode Toolkit, we have gen_requirements.py
and scripts to download wheels of all requirements packages, these can be shipped along with our projects.
In Scancode Toolkit, we have open ranges on setup.cfg
direct dependencies and then in the requirements, everything is pinned to single version.
There is a configure
script that provides pip install --editable`` and use requirement files as a constraint. So, all dependencies are fetched from ``setup.cfg`` and constrained to ``requirements.txt`` See
here https://github.com/nexB/scancode-toolkit/blob/7bc0782fdfda9da5dba0500446ff3e8d58623e99/configure#L31`_
One at a time or as batch scanning: One at a time. A few providers are giving limits, so we will have restrictions. There are a few commercial projects that we could compare against eg: copilot.blackducksoftware.com, snyk, https://ossindex.sonatype.org/, gitlab, github, osv
Philippe is working on server deployment by 1st week of April for demonstration. We don't want to ever drop the database (again) and redo the whole thing.
Added failing test here <https://github.com/nexB/vulnerablecode/pull/659>
_ and added a workaround for postgres index issue <https://github.com/nexB/vulnerablecode/pull/653>
_
Have a fast test on nix. We want to have nix.
We're using FetchCode for git imports
Agenda:
- Postgres Issue: https://github.com/nexB/vulnerablecode/issues/650
- Trimming gourl path
- Example proposals
- Gitlab Importers
Participants:
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
- Naashit (@naashit10)
Just write importer, no improver for now because univers requires a lot of work. Slowly, improver will be written.
Old proposals after the GSoC Video. Meanwhile a project template can be found at: https://github.com/nexB/aboutcode/wiki/GSOC-2022#about-your-project-application
Needs to be tested and reviewed in the PR.
| https://github.com/nexB/vulnerablecode/issues/650 | We need to look into more alternatives
Agenda:
- Test cases
- Univers: Multiple unimplemented version ranges
- GSoC Event Status
- GSoC Questions
- Fixed version ranges
- Status updates of migrations
- LFX Event
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Keshav (@keshav-space)
- Chaitanya
| Presentation: Remote + In Person, Pre-recorded | Q&A With LFX chat | Time: June | Promotion: Will begin with time | Waiting for acceptance
- Npm importer: Working on mutliple fix versions like
>=3.5.1 <4.0.0 || >=4.1.3 <5.0.0 || >=5.6.1 <6.0.0 || >=6.1.2
- GitHub importer: Importer improver are done. In testing phase right now. Doctests for small tests.
- Issue: Severeal references could be treated as an aliases. There should be improvers which would take references, verify them as valid unique aliases and add them. Ticket: https://github.com/nexB/vulnerablecode/issues/646
- fix_version and fix_version_range both
- have one fix_version_range
- have advisory_raw_data and improve from there SELECTED
Can participants work on multiple projects within the same organization?
Yes, feel free to have the draft proposal so that we may provide a feedback
- 30 mins presentation
- slides on introduction
- record presentation
- decide dates
- Blog post with recorded session
- Different presenters for each project
- Create event for the event demo
Philippe will be providing the templates today.
Does anyone want to join the unimplemented version ranges ?
We're lacking a lot of test cases, trying to cover them up. We won't be using async in our codebase.
Agenda:
- Importers update
- Univers
- YouTube Channel
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
We want asserts in importers to make it helpful for mypi and log loudly. GitHub importer is on the way by Tushar and Npm Importer by Hritik.
| Openssl License: found Apache license, @keshav-space will create a ticket and add reference to the license
| PR for Openssl version is on the way at https://github.com/nexB/univers/pull/42
| Overriding __new__
method is generally not a good idea in univers (in context of the above PR).
Pinged @nspsjsu
Agenda:
- GSoC idea list sorting
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
Idea list sorted by priority has been made available at https://github.com/nexB/aboutcode/wiki/GSOC-2022#vulnerablecode-project-ideas-by-priority
Agenda:
- License for improvers
- PR Reviews
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@tg1999)
- Keshav (@keshav-space)
| Tushar: Review VulnerableCode PR | Keshav: Openssl PR, VersionConstraint Class bug, GSoC Proposal | Philippe - Principles for ignore and done data, hosting a public instance, sharing vulnerabilities, gsoc project | Hritik | - License improvers: Not required to have a license, as they augment data bits | - Migrate to attrs as discussed: Shouldn't be complex. We should wait until everything is back in shape ie vast majority of importers
We have been ignoring a lot of data. It is a catastrophe. Log all failing records. Provide verbose functionality.
Fail loudly and avoid being smart in importers.
We would want to keep the raw data for importers stored in our database as well. We need to figure out a proper way to store raw data without duplicates. Ticket:
Until then, we need to have log
model with key fields: importer, date, data, error message, type. The type could also be unprocessed_imports
, success_imports
etc. This could be used in a future-dashboard where we can show the errors, success logs etc. Ticket:
Also, clean and clean migrations from now on should be done.
After v30.0
All of them aggregated and will be published on AboutCode page by Philippe.
A version constraint with a star (*) range does not make sense. Other than that, we want to have no constraint (thus VersionConstraint) if all versions are accepted.
A list of GSoC proposal possibly in our AbouCode Wiki. To be created by Philippe.
Agenda:
- License in VulnerableCode
- Documentation
- Management commands
- GSoC Idea list
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
- Tushar (@TG1999)
- Keshav (@keshav-space)
- We should be adding license URL in importers
- Unknown license identifier:
LicenseRef-scancode-unknown
url: https://scancode-licensedb.aboutcode.org/unknown.html - License checks can be performed via https://github.com/nexB/license-expression/ Ticket: https://github.com/nexB/vulnerablecode/issues/629
- License expression or identifier: expression is better as it is capable of representing one identifier as well as the combinations
- Philippe will be sharing new license header for our source code
- Add tl;dr in project README
Hritik:
Having ./manage.py import --list does not make a lot of sense
Philippe:
Let's use ./manage.py import --list-importers to be more explicit
- Decentralized vulnerability upload and share
- vers implementation in many programming languages
- Build test suites and fix them for all the versions in univers and vers implementations
- Ideas from last meet and 2021 Idea list
Agenda:
- Clean slate for PRs, missing DCO
- VulnerableCode logo
- YouTube channel for AboutCode
- GSoC Idea list
- Meet timings on gitter
Participants:
- Philippe (@pombredanne)
- Hritik (@hritik14)
| Pinged @Pushpit07 for https://github.com/nexB/vulnerablecode/pull/464 | Others PRs are good to go.
Under development
Ping Nusrat and Adam
- Good coverage of vulnerabilites
- publishing purl database
- data synchronization
- sharding vulnerablecode data on serverless platforms(lambda, functions etc)
- return spdx or cyclonedx
- vex: vulnerability exploitability
- A decent UI
- previous year idea list
Meet timings on gitter: Updated to Tuesday 10 UTC / 11 CET