The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.
Difference between revisions of "FLOP:CVE Monitoring"
m |
m |
||
| Line 96: | Line 96: | ||
* the sync of <tt>cve-search</tt> and <tt>jira</tt> must be always provable | * the sync of <tt>cve-search</tt> and <tt>jira</tt> must be always provable | ||
However, we don't need to deal with <tt>cve-search</tt> directly: we can transform it into an intermediate state associated with <tt>cver</ | However, we don't need to deal with <tt>cve-search</tt> directly: we can transform it into an intermediate state associated with <tt>cver</tt> that has its own paths, and then make the equivalent <tt>jira</tt> path from those. We may just 'bulk transform' the <tt>cve-search</tt> to a (probably much simpler) schema more directly related to that of a <tt>jira</tt> issue. We just need a collection of what are essentially <tt>jira</tt> records, with meta data to control the sync. | ||
Revision as of 03:13, January 25, 2020
Funtoo Linux Optimization Proposal: CVE Monitoring
cver: A Tool for Monitoring CVEs
Summary
Ultimately, not all ebuilds are created equal. Hence they are updated at different rates according to their popularity in the tree of available packages and this is generally fine: packages with a lot of use get updated frequently, and vulnerabilities are generally dealt with. Unpopular ebuilds can languish, and no one really cares. However, unpopular ebuilds with a significant vulnerability should be updated, popular or not, as they represent a potential vector for attack, if they can be installed.
Identifying ebuilds with an associated CVE will bring them to 'head of the queue' for pull requests and updates, which should often be trivial, as the vulnerability is dealt with upstream and released as a new hotfix version. Or, we can fork and provide our own mitigation, merging with upstream again when a new release comes out (if at all).
The cver (pronounced ça-veer) tool is built around redis cached mongodb collections that are regularly updated with newly filed CVEs. The tool queries the collections to produce a set of text data appropriate to fill fields on a newly created security vulnerability issue on the Funtoo bug tracker. The data can be output in various formats (current just formatted text on stdout), and eventually input directly to the bug tracker via its REST api.
Architecture
The architecture is simple:
┌─────────┐
│redis │ ┌────┐
│┌───────┐│ │jira│──────────┐
││mongoDB││ └──┬─┘ │
│└───────┘│ │ │
└────┬────┘ │ │
│ │ *********
│ ┌───┐ │ *** ***
├─────┤dev│──────────* discussion *
│ └─┬─┘ │ *** ***
│ │ │ *********
│ │ │
│ ┌─┴─┐ │
├─────┤bot│────┘
│ └───┘
│
│
│ ┌───┐
└─────│usr│
└───┘- A dev uses the tool to query the redis cache of the CVE data held in the mongoDB, update and admin the mongoDB, create reports for discussion, and control a bot.
- The bot can query the redis cache and create issues to post via the REST api of jira.
- A user can query the redis cache and create reports.
- discussion produces issues to be posted at jira.
Algorithm
The cvedb.cves collection provided by cve-search has the following estimated schema (see variety, a schema estimator for mongodb):
+--------------------------------------------------------------------------------+
| key | types | occurrences | percents |
| -------------------------------- | -------- | ----------- | ------------------ |
| Modified | Date | 136539 | 100.00000000000000 |
| Published | Date | 136539 | 100.00000000000000 |
| _id | ObjectId | 136539 | 100.00000000000000 |
| access | Object | 136539 | 100.00000000000000 |
| assigner | String | 136539 | 100.00000000000000 |
| cvss | Number | 136539 | 100.00000000000000 |
| cwe | String | 136539 | 100.00000000000000 |
| id | String | 136539 | 100.00000000000000 |
| impact | Object | 136539 | 100.00000000000000 |
| references | Array | 136539 | 100.00000000000000 |
| summary | String | 136539 | 100.00000000000000 |
| vulnerable_configuration | Array | 136539 | 100.00000000000000 |
| vulnerable_configuration_cpe_2_2 | Array | 136539 | 100.00000000000000 |
| vulnerable_product | Array | 136539 | 100.00000000000000 |
| access.authentication | String | 128583 | 94.17309340188518 |
| access.complexity | String | 128583 | 94.17309340188518 |
| access.vector | String | 128583 | 94.17309340188518 |
| cvss-time | Date | 128583 | 94.17309340188518 |
| cvss-vector | String | 128583 | 94.17309340188518 |
| impact.availability | String | 128583 | 94.17309340188518 |
| impact.confidentiality | String | 128583 | 94.17309340188518 |
| impact.integrity | String | 128583 | 94.17309340188518 |
+--------------------------------------------------------------------------------+An important key in the collection is that of vulnerable_product. It contains the Common Platform Enumeration of the affected piece of software, and can potentially be matched (along with the affected product's version(s)) to packages in the Funtoo portage meta-repo.
This is the bird's eye view of what a CPE is:
CPE is a structured naming scheme for information technology systems, software, and packages. Based upon the generic syntax for Uniform Resource Identifiers (URI), CPE includes a formal name format, a method for checking names against a system, and a description format for binding text and tests to a name.
Thus, filtering packages by CVE requires a map between package names and CPE. The current algorithm is the simplest possible: if a CVE has a list of CPEs, each CPE is interpreted to yield a single token and an exact match with package name is attempted for the whole meta-repo using app-portage/eix. If there is a match, then a jira issue can be constructed and reported. Even this simple algorithm produces quite a few matches, but it also misses very significant issues if the CPEs are not added properly to the CVE database for the issue. FL-6938 is a case in point: it was not filed with a CPE for sys-apps/portage (does it exist?) so the algorithm skipped right over it. A more sophisticated algorithm would have done regular expression matching on the summary key of the issue, perhaps matching on the string 'Gentoo Portage,' and producing a report for discussion, and eventual posting to jira.
Once a match is made, the cve-search collection and the portage package database (via app-portage/eix) can be combined to produce the data appropriate for a report.
State
The cver tool is currently stateless: it takes some bytes and it makes some bytes. Of course, jira and mongoDB are not, and their states must be kept in sync. Does cver require its own set of mongoDB collections to maintain the sync? This probably the most challenging aspect of the proposal.
- every update of the cve-search database must trigger an update of jira
- every CRUD path of cve-search must have an equivalent CRUD path of jira
- the sync of cve-search and jira must be always provable
However, we don't need to deal with cve-search directly: we can transform it into an intermediate state associated with cver that has its own paths, and then make the equivalent jira path from those. We may just 'bulk transform' the cve-search to a (probably much simpler) schema more directly related to that of a jira issue. We just need a collection of what are essentially jira records, with meta data to control the sync.