Commit b4ad1b70 authored by Stein Welberg's avatar Stein Welberg Committed by Clement Bois
Browse files

feat: add VEX file uploading feature 



This feature adds VEX file uploading to the sbom scanner. This allows one to store Vulnerability information (e.g. suppressions) in a GIT repository and upload them when scanning an SBOM. This is especially useful when you want to keep track vulnerabilities in multiple versions of the same project in Dependency Track. By keeping the VEX file in git, you do not have to manually apply the same suppressions to all project versions.

Signed-off-by: default avatarStein Welberg <mail@steinwelberg.nl>
parent a4b94c5a

README.md

+17 −2
Original line number Diff line number Diff line
@@ -42,7 +42,8 @@ docker run \ 


```bash

usage: sbom-scanner [-h] [-u BASE_API_URL] [-k API_KEY] [-i] [-p PROJECT_PATH] [-s PATH_SEPARATOR] [-t TAGS] [--parent-collection-logic {NONE,ALL,TAG,LATEST}]

 [--parent-collection-logic-tag PARENT_COLLECTION_LOGIC_TAG] [-m] [-o MERGE_OUTPUT] [-l PURL_MAX_LEN] [-S] [-R RISK_SCORE_THRESHOLD] [sbom_patterns ...]

                    [--parent-collection-logic-tag PARENT_COLLECTION_LOGIC_TAG] [-m] [-o MERGE_OUTPUT] [-l PURL_MAX_LEN] [-U] [-V MERGED_VEX_FILE] [-S] [-R RISK_SCORE_THRESHOLD]

                    [sbom_patterns ...]



This tool scans for SBOM files and publishes them to a Dependency Track server.
@@ -77,6 +78,12 @@ SBOM management: 
  -l PURL_MAX_LEN, --purl-max-len PURL_MAX_LEN

                        PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)



VEX:

  -U, --upload-vex      Upload VEX file after SBOM analysis (requires VULNERABILITY_ANALYSIS permission). The VEX file(s) are resolved based on the sbom pattern(s). The first part of the SBOM file name is used

                        to match it with a VEX file (e.g. if there is an SBOM file 'example.cyclonedx.json', the corresponding VEX file name must be 'example.vex.json')

  -V, --merged-vex-file MERGED_VEX_FILE

                        The VEX file to upload if multiple SBOMS are merged (--merge). Can only be used with --upload-vex and --merge.



Miscellaneous:

  -S, --show-findings   Wait for analysis and display found vulnerabilities

  -R RISK_SCORE_THRESHOLD, --risk-score-threshold RISK_SCORE_THRESHOLD
@@ -118,6 +125,13 @@ If none is specified, the program will look for SBOM files matching `**/*.cyclon 
| `-o` / `--merge-output`         | `$DEPTRACK_MERGE_OUTPUT`                | Output merged SBOM file (only used with merge enabled) - for debugging purpose  |

| `-l` / `--purl-max-len`         | `$DEPTRACK_PURL_MAX_LEN`                | PURLs max length (`-1`: auto, `0`: no trim, `>0`: trim to size - default: `-1`) |



#### VEX



| CLI option                 | Env. Variable               | Description                                                                                                                                                                                                                                                                                                                                |

| -------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |

| `-U` / `--upload-vex`      | `$DEPTRACK_UPLOAD_VEX`      | Upload VEX file after SBOM analysis (requires VULNERABILITY_ANALYSIS permission). The VEX file(s) are resolved based on the sbom pattern(s). The first part of the SBOM file name is used to match it with a VEX file (e.g. if there is an SBOM file 'example.cyclonedx.json', the corresponding VEX file name must be 'example.vex.json') |

| `-V` / `--merged-vex-file` | `$DEPTRACK_MERGED_VEX_FILE` | The VEX file to upload if multiple SBOMS are merged (--merge). Can only be used with --upload-vex and --merge.                                                                                                                                                                                                                             |



#### Miscellaneous



| CLI option                      | Env. Variable                           | Description                                                                    |
@@ -133,6 +147,7 @@ If none is specified, the program will look for SBOM files matching `**/*.cyclon 
  Granting those permissions without enabling [Portfolio ACLs](https://github.com/DependencyTrack/dependency-track/issues/1127) is not recommended in the general case as give a read access to all projects.

- The extra `VIEW_PORTFOLIO` and `PORTFOLIO_MANAGEMENT` permissions are required if you want to automatically create one or several project ancestors prior to uploading the SBOM files.<br/>

  Granting those permissions is not recommended in the general case as they virtually give administration rights to the API Key owner.

- The extra `VULNERABILITY_ANALYSIS` permission is required if you want to upload VEX files after SBOM analysis.



## Project Path
@@ -193,7 +208,7 @@ A collection project won't be able to host a SBOM, however it will display all v 
| `LATEST` | `AGGREGATE_LATEST_VERSION_CHILDREN`  | Parent project will collect metrics from direct children flagged as latest |

| `NONE`   | `NONE`                               | Parent project will behave like a normal project |



:information_source: since `sbom-scanner` do not update existing projects, collection logic will only be configured on parent project created by `sbom-scanner`

:information_source: since `sbom-scanner` does not update existing projects, collection logic will only be configured on the parent project created by `sbom-scanner`



See [Dependency Track documentation about collection project](https://docs.dependencytrack.org/usage/collection-projects/) for more details.

sbom_scanner/scan.py

+118 −15
Original line number Diff line number Diff line
@@ -247,6 +247,8 @@ class Scanner: 
        tags: str = "",

        parent_collection_logic: str = CollectionLogic.ALL.name,

        parent_collection_logic_tag: str = "",

        upload_vex: bool = False,

        merged_vex_file = None,

        **_: None,

    ):

        self.base_api_url = base_api_url
@@ -264,6 +266,8 @@ class Scanner: 
        self.parent_collection_logic_tag = parent_collection_logic_tag

        self.sbom_count = 0

        self.sbom_scan_failed = 0

        self.upload_vex = upload_vex

        self.merged_vex_file = merged_vex_file



    @property

    @cache
@@ -482,7 +486,7 @@ class Scanner: 
            )

            raise



    def publish(self, sbom: Bom, file_prefix: str):

    def publish(self, sbom: Bom, file_prefix: str, vex_file_path: Path):

        sbom_type = None

        sbom_name = None

        sbom_version = None
@@ -513,11 +517,11 @@ class Scanner: 
            sbom_utils.trim_purls(sbom, self.purl_max_len)



        self.do_publish(

            sbom_utils.to_json(sbom, self.cdx_schema_version), project_path, sbom_type

            sbom_utils.to_json(sbom, self.cdx_schema_version), project_path, sbom_type, vex_file_path

        )



    def do_publish(

        self, sbom_json: str, project_path: str, sbom_type: str, allow_retry=True

        self, sbom_json: str, project_path: str, sbom_type: str, vex_file_path: Path, allow_retry=True

    ):

        project_path_parts = project_path.split(self.path_separator)

        # determine publish params
@@ -582,6 +586,7 @@ class Scanner: 
                    sbom_json,

                    self.path_separator.join(project_path_parts),

                    sbom_type,

                    vex_file_path,

                    allow_retry=False,

                )

                # to prevent do_scan one more time (must have been done in the retried do_publish())
@@ -589,10 +594,54 @@ class Scanner: 
            else:

                raise



        event_id = resp.json().get("token")



        # import VEX file

        if self.upload_vex:

            event_id = self.do_vex_publish(project_def, vex_file_path, event_id)



        if self.need_findings:

            event_id = resp.json()["token"]

            self.do_scan(project_def, event_id)



    def do_vex_publish(self, project_def: DtProjectDef, vex_file_path: Path,  event_id: str):

        self.wait_for_event_processing(event_id)



        params = {}

        if project_def.is_uuid:

            # target project definition is a UUID: nothing more is required

            params["project"] = project_def.uuid

        else:

            # target project definition is a project name: assume exists or set autoCreate with parent if permission PROJECT_CREATION_UPLOAD

            params["projectName"] = project_def.name

            params["projectVersion"] = project_def.version



        if not vex_file_path.exists():

            print(

                f"- VEX file {AnsiColors.YELLOW}not found, skipping upload{AnsiColors.RESET}: {AnsiColors.HGRAY}{vex_file_path}{AnsiColors.RESET}"

            )

            return event_id



        with open(vex_file_path, "r") as vex_file:

            resp = requests.post(

                f"{self.base_api_url}/v1/vex",

                headers={"X-API-Key": self.api_key, "accept": MIME_APPLICATION_JSON},

                files={"vex": vex_file},

                data=params,

                verify=self.verify_ssl,

            )

            try:

                resp.raise_for_status()

                print(

                    f"- VEX import {AnsiColors.HGREEN}succeeded{AnsiColors.RESET}: {AnsiColors.HGRAY}{resp.text}{AnsiColors.RESET}"

                )

            except requests.exceptions.HTTPError as he:

                print(

                    f"- VEX import {AnsiColors.HRED}failed{AnsiColors.RESET} (err {he.response.status_code}): {AnsiColors.HGRAY}{he.response.text}{AnsiColors.RESET}",

                )

                raise



            return resp.json().get("token")



    def do_scan(self, project_def: DtProjectDef, event_id: str):

        print(f"- scan: {AnsiColors.HGRAY}{event_id}{AnsiColors.RESET}...")

        if project_def.is_uuid:
@@ -612,10 +661,8 @@ class Scanner: 


        self.wait_for_event_processing(event_id)

        # MAYBE: get SBOM with VEX curl -sSf f"{self.base_api_url}/v1/bom/cyclonedx/project/{project_id}?variant=withVulnerabilities"

        resp = requests.get(

            f"{self.base_api_url}/v1/finding/project/{project_id}",

            headers={"X-API-Key": self.api_key, "accept": MIME_APPLICATION_JSON},

            verify=self.verify_ssl,

        resp = self.api_client.get(

            f"/v1/finding/project/{project_id}",

        )

        resp.raise_for_status()

        risk_score = 0
@@ -654,7 +701,7 @@ class Scanner: 
                headers={"X-API-Key": self.api_key, "accept": MIME_APPLICATION_JSON},

                verify=self.verify_ssl,

            )

            if resp.json().get("processing", False):

            if not resp.json().get("processing", False):

                break



    def scan(self, sbom_patterns: list[str]):
@@ -690,6 +737,10 @@ class Scanner: 
                fail(

                    "VIEW_PORTFOLIO permission is mandatory to show finding or compute risk score after SBOM analysis"

                )

        if self.upload_vex and not self.has_permission(DtPermission.VULNERABILITY_ANALYSIS):

            fail(

              "VULNERABILITY_ANALYSIS permission is mandatory to import VEX files"

            )



        # scan for SBOM files

        sboms = []
@@ -699,12 +750,20 @@ class Scanner: 
                    f"{AnsiColors.BOLD}📄 SBOM: {AnsiColors.BLUE}{file}{AnsiColors.RESET}"

                )

                # load the SBOM content

                file_path = Path(file)

                sbom = sbom_utils.load_bom(file_path)

                sbom_file_path = Path(file)

                sbom_file_prefix = sbom_file_path.name.split(".")[0]

                vex_file_path = sbom_file_path.with_name(f"{sbom_file_prefix}.vex.json")



                if self.upload_vex and not vex_file_path.exists():

                    fail(

                        f"- VEX file {AnsiColors.HRED}not found{AnsiColors.RESET}: {AnsiColors.HGRAY}{vex_file_path}{AnsiColors.RESET}"

                    )



                sbom = sbom_utils.load_bom(sbom_file_path)

                if self.merge:

                    sboms.append(sbom)

                else:

                    self.publish(sbom, file_path.name.split(".")[0])

                    self.publish(sbom, sbom_file_prefix, vex_file_path)



                print()

                self.sbom_count += 1
@@ -741,7 +800,13 @@ class Scanner: 
                sbom_utils.save_bom(

                    merged_sbom, Path(self.merge_output), self.cdx_schema_version

                )

            self.publish(merged_sbom, "merged")

            vex_file_path = Path(self.merged_vex_file) if self.merged_vex_file else None

            if self.upload_vex and not vex_file_path.exists():

                fail(

                    f"- VEX file {AnsiColors.HRED}not found{AnsiColors.RESET}: {AnsiColors.HGRAY}{vex_file_path}{AnsiColors.RESET}"

                )



            self.publish(merged_sbom, "merged", vex_file_path)





def fail(msg: str) -> None:
@@ -838,6 +903,22 @@ def run() -> None: 
        help="PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)",

    )



    vex_group = parser.add_argument_group("VEX")

    vex_group.add_argument(

        "-U",

        "--upload-vex",

        action="store_true",

        default=os.getenv("DEPTRACK_UPLOAD_VEX") in ["true", "yes", "1"],

        help="Upload VEX file after SBOM analysis (requires VULNERABILITY_ANALYSIS permission). The VEX file(s) are resolved based on the sbom pattern(s). The first part of the SBOM file name is used to match it with a VEX file (e.g. if there is an SBOM file 'example.cyclonedx.json', the corresponding VEX file name must be 'example.vex.json')",

    )

    vex_group.add_argument(

        "-V",

        "--merged-vex-file",

        type=str,

        default=os.getenv("DEPTRACK_MERGED_VEX_FILE"),

        help="The VEX file to upload if multiple SBOMS are merged (--merge). Can only be used with --upload-vex and --merge.",

    )



    misc_group = parser.add_argument_group("Miscellaneous")

    misc_group.add_argument(

        "-S",
@@ -886,6 +967,22 @@ def run() -> None: 
        fail(

            f"You need to specify a tag with --parent-collection-logic-tag (or DEPTRACK_PARENT_COLLECTION_LOGIC_TAG env var) if parent collection logic has been set to {CollectionLogic.TAG.name}"

        )

    if (

        args.merge

        and args.upload_vex

        and not args.merged_vex_file

    ):

        fail(

            "You need to specify a VEX file with --merged-vex-file (or DEPTRACK_MERGED_VEX_FILE env var) if you want to upload a VEX file and are merging SBOM files (--merge)"

        )

    if (

        not args.merge

        and args.upload_vex

        and args.merged_vex_file

    ):

        fail(

            "You cannot specify a VEX file with --merged-vex-file (or DEPTRACK_MERGED_VEX_FILE env var) if you are NOT merging SBOM files (--merge is not set)"

        )



    # print execution parameters

    print("Scanning SBOM files...")
@@ -927,6 +1024,12 @@ def run() -> None: 
    print(

        f"- insecure             (--insecure): {AnsiColors.CYAN}{args.insecure}{AnsiColors.RESET}"

    )

    print(

        f"- Upload VEX         (--upload-vex): {AnsiColors.CYAN}{args.upload_vex}{AnsiColors.RESET}"

    )

    print(

        f"- VEX file path for merged SBOM (--merged-vex-file): {AnsiColors.CYAN}{args.merged_vex_file}{AnsiColors.RESET}"

    )

    print(

        f"- SBOM file pattern                : {AnsiColors.CYAN}{', '.join(args.sbom_patterns)}{AnsiColors.RESET}"

    )