Merge branch '6-allow-creation-of-aggregate-parent' into 'main' (cd920fb8) · Commits · to be continuous... / tools / dt-sbom-scanner

.gitignore

+1 −0

Original line number	Diff line number	Diff line
		@@ -206,3 +206,4 @@ pip-selfcheck.json
		reports/
		.DS_Store
		.bin/
		.ruff-cache/
		No newline at end of file

README.md

+68 −18

Original line number	Diff line number	Diff line
		@@ -41,7 +41,8 @@ docker run \
		## Usage

		```bash
		usage: sbom-scanner [-h] [-u BASE_API_URL] [-k API_KEY] [-p PROJECT_PATH] [-s PATH_SEPARATOR] [-m] [-o MERGE_OUTPUT] [-S] [-R RISK_SCORE_THRESHOLD] [-i] [-l PURL_MAX_LEN] [-t TAGS] [sbom_patterns ...]
		usage: sbom-scanner [-h] [-u BASE_API_URL] [-k API_KEY] [-i] [-p PROJECT_PATH] [-s PATH_SEPARATOR] [-t TAGS] [--parent-collection-logic {NONE,ALL,TAG,LATEST}]
		[--parent-collection-logic-tag PARENT_COLLECTION_LOGIC_TAG] [-m] [-o MERGE_OUTPUT] [-l PURL_MAX_LEN] [-S] [-R RISK_SCORE_THRESHOLD] [sbom_patterns ...]

		This tool scans for SBOM files and publishes them to a Dependency Track server.

		@@ -50,24 +51,37 @@ positional arguments:

		options:
		-h, --help show this help message and exit

		Connection to Dependency Track platform:
		-u BASE_API_URL, --base-api-url BASE_API_URL
		Dependency Track server base API url (includes '/api')
		-k API_KEY, --api-key API_KEY
		Dependency Track API key
		-i, --insecure Skip SSL verification

		Project settings:
		-p PROJECT_PATH, --project-path PROJECT_PATH
		Dependency Track target project path to publish SBOM files to (see doc)
		-s PATH_SEPARATOR, --path-separator PATH_SEPARATOR
		Separator to use in project path (default: '/')
		-t TAGS, --tags TAGS Comma separated list of tags to attach to the project
		--parent-collection-logic {NONE,ALL,TAG,LATEST}
		Set up how the parent aggregates its direct children (ALL: all, TAG: with tag matching --parent-collection-logic-tag, LATEST: flagged as latest, NONE: disable), default is ALL (DT version >= 4.13.0)
		--parent-collection-logic-tag PARENT_COLLECTION_LOGIC_TAG
		Tag for aggregation if --parent-collection-logic is set to TAG

		SBOM management:
		-m, --merge Merge all SBOM files into one
		-o MERGE_OUTPUT, --merge-output MERGE_OUTPUT
		Output merged SBOM file (only used with merge enabled) - for debugging purpose
		-l PURL_MAX_LEN, --purl-max-len PURL_MAX_LEN
		PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)

		Miscellaneous:
		-S, --show-findings Wait for analysis and display found vulnerabilities
		-R RISK_SCORE_THRESHOLD, --risk-score-threshold RISK_SCORE_THRESHOLD
		Risk score threshold to fail the scan (<0: disabled - default: -1)
		-i, --insecure Skip SSL verification
		-l PURL_MAX_LEN, --purl-max-len PURL_MAX_LEN
		PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)
		-t TAGS, --tags TAGS Comma separated list of tags to attach to the project

		```

		### Arguments
		@@ -78,19 +92,38 @@ If none is specified, the program will look for SBOM files matching `*/.cyclon

		### Options

		#### Dependency Track connection

		\| CLI option \| Env. Variable \| Description \|
		\| ------------------------------- \| --------------------------------- \| ------------------------------------------------------------------------------- \|
		\| ------------------------------- \| --------------------------------------- \| ------------------------------------------------------------------------------ \|
		\| `-u` / `--base-api-url` \| `$DEPTRACK_BASE_API_URL` \| Dependency Track server base API url (includes `/api`) (mandatory) \|
		\| `-k` / `--api-key` \| `$DEPTRACK_API_KEY` \| Dependency Track API key (mandatory) \|
		\| `-i` / `--insecure` \| `$DEPTRACK_INSECURE` \| Skip SSL verification \|

		#### Project selection and settings

		\| CLI option \| Env. Variable \| Description \|
		\| ------------------------------- \| --------------------------------------- \| ------------------------------------------------------------------------------ \|
		\| `-p` / `--project-path` \| `$DEPTRACK_PROJECT_PATH` \| Dependency Track target project path to publish SBOM files to (mandatory) \|
		\| `-s` / `--path-separator` \| `$DEPTRACK_PATH_SEPARATOR` \| Separator to use in project path (default `/`) \|
		\| `-t` / `--tags` \| `$DEPTRACK_TAGS` \| Comma separated list of tags to put in the project in `autoCreate` mode \|
		\| `--parent-collection-logic` \| `$DEPTRACK_PARENT_COLLECTION_LOGIC` \| Set up how the parent aggregates its direct children (see doc), default is ALL \|
		\| `--parent-collection-logic-tag` \| `$DEPTRACK_PARENT_COLLECTION_LOGIC_TAG` \| Tag for aggregation if `--parent-collection-logic` is set to `TAG` \|

		#### SBOM options

		\| CLI option \| Env. Variable \| Description \|
		\| ------------------------------- \| --------------------------------------- \| ------------------------------------------------------------------------------- \|
		\| `-m` / `--merge` \| `$DEPTRACK_MERGE` \| Merge all SBOM files into one (default `false`) \|
		\| `-o` / `--merge-output` \| `$DEPTRACK_MERGE_OUTPUT` \| Output merged SBOM file (only used with merge enabled) - for debugging purpose \|
		\| `-l` / `--purl-max-len` \| `$DEPTRACK_PURL_MAX_LEN` \| PURLs max length (`-1`: auto, `0`: no trim, `>0`: trim to size - default: `-1`) \|
		\| `-t` / `--tags` \| `$DEPTRACK_TAGS` \| Comma seperated list of tags to put in the project in autoCreate mode \|

		#### Miscellaneous

		\| CLI option \| Env. Variable \| Description \|
		\| ------------------------------- \| --------------------------------------- \| ------------------------------------------------------------------------------ \|
		\| `-S` / `--show-findings` \| `$DEPTRACK_SHOW_FINDINGS` \| Wait for analysis and display found vulnerabilities \|
		\| `-R` / `--risk-score-threshold` \| `$DEPTRACK_RISK_SCORE_THRESHOLD` \| Risk score threshold to fail the scan (`<0`: disabled - default: `-1`) \|
		\| `-i` / `--insecure` \| `$DEPTRACK_INSECURE` \| Skip SSL verification \|

		## API Key permissions

		@@ -147,6 +180,23 @@ Project path examples:

		> :bulb: you may decide to overwrite the path separator (ex: double slash `//`) if you want your project names to contain slashes.

		### Project collection logic

		Since [4.13.0 version](https://docs.dependencytrack.org/changelog/#v4-13-0), Dependency Track supports the concept of [collection project](https://docs.dependencytrack.org/usage/collection-projects/).
		A collection project won't be able to host a SBOM, however it will display all vulnerabilities and policy violations of its children.
		`sbom-scanner` will set the collection logic with the `--parent-collection-logic` optional argument (default value is `ALL`).

		\| `--parent-collection-logic` value \| Dependency Track API value \| Explanation \|
		\| -------- \| ------------------------------------ \| --------------------- \|
		\| `ALL` \| `AGGREGATE_DIRECT_CHILDREN` \| Parent project will collect metrics from all its direct children \|
		\| `TAG` \| `AGGREGATE_DIRECT_CHILDREN_WITH_TAG` \| Parent project will collect metrics from direct children specified by a tag (which can be different from tags on the parent project itself) \|
		\| `LATEST` \| `AGGREGATE_LATEST_VERSION_CHILDREN` \| Parent project will collect metrics from direct children flagged as latest \|
		\| `NONE` \| `NONE` \| Parent project will behave like a normal project \|

		:information_source: since `sbom-scanner` do not update existing projects, collection logic will only be configured on parent project created by `sbom-scanner`

		See [Dependency Track documentation about collection project](https://docs.dependencytrack.org/usage/collection-projects/) for more details.

		## Developers

		`sbom-scanner` is implemented in Python and relies on [Poetry](https://python-poetry.org/) for its packaging and dependency management.

sbom_scanner/scan.py

+116 −37

Original line number	Diff line number	Diff line
		@@ -50,6 +50,25 @@ SEVERITY_RANKS = [
		]


		class CollectionLogic(str, Enum):
		"""Dependency Track collection logics.

		See: https://github.com/DependencyTrack/dependency-track/blob/master/src/main/java/org/dependencytrack/model/ProjectCollectionLogic.java#L30
		"""

		NONE = "NONE"
		"Project is not a collection project"
		ALL = "AGGREGATE_DIRECT_CHILDREN"
		"Project aggregate all direct children"
		TAG = "AGGREGATE_DIRECT_CHILDREN_WITH_TAG"
		"Project aggregate direct children with a specific tag"
		LATEST = "AGGREGATE_LATEST_VERSION_CHILDREN"
		"Project aggregate only direct children marked as latest"

		def __str__(self) -> str:
		return self.name


		class DtPermission(str, Enum):
		"""Dependency Track permissions.

		@@ -226,6 +245,8 @@ class Scanner:
		show_findings: bool = False,
		risk_score_threshold: int = -1,
		tags: str = "",
		parent_collection_logic: str = CollectionLogic.ALL.name,
		parent_collection_logic_tag: str = "",
		**_: None,
		):
		self.base_api_url = base_api_url
		@@ -239,6 +260,8 @@ class Scanner:
		self.show_findings = show_findings
		self.risk_score_threshold = risk_score_threshold
		self.tags = list(filter(None, map(str.strip, tags.split(",")))) if tags else []
		self.parent_collection_logic = parent_collection_logic
		self.parent_collection_logic_tag = parent_collection_logic_tag
		self.sbom_count = 0
		self.sbom_scan_failed = 0

		@@ -302,7 +325,9 @@ class Scanner:
		# rewinds the given project path and creates a DT project for each non-UUID defined project
		# returns the tail project UUID
		@cache
		def get_or_create_project(self, project_path: str, classifier="application") -> str:
		def get_or_create_project(
		self, project_path: str, classifier="application", is_parent: bool = False
		) -> str:
		project_path_parts = project_path.split(self.path_separator)
		project_def = DtProjectDef(project_path_parts[-1])
		if project_def.is_uuid:
		@@ -399,6 +424,17 @@ class Scanner:
		"classifier": classifier.upper(),
		"active": True,
		}

		# Set up collection logic if supported
		if is_parent and self.dt_version >= Version("4.13.0"):
		data["collectionLogic"] = CollectionLogic[
		self.parent_collection_logic
		].value
		if data["collectionLogic"] == CollectionLogic.TAG:
		data["collectionTag"] = {
		"name": self.parent_collection_logic_tag.strip()
		}

		# TODO: externalReferences
		# data["externalReferences"] = [{"type":"vcs","url":project_url}],
		if len(project_path_parts) > 1:
		@@ -407,7 +443,9 @@ class Scanner:
		if not parent_def.is_uuid:
		# create parent project
		parent_uuid = self.get_or_create_project(
		self.path_separator.join(project_path_parts[:-1])
		self.path_separator.join(project_path_parts[:-1]),
		classifier=classifier,
		is_parent=True,
		)
		# now parent def must be a UUID
		parent_def = DtProjectDef("#" + parent_uuid)
		@@ -536,7 +574,7 @@ class Scanner:
		# replace last path part with project UUID
		# TODO: retrieve classifier from SBOM
		project_path_parts[-1] = "#" + self.get_or_create_project(
		project_path, sbom_type
		project_path, sbom_type, is_parent=False
		)
		# then retry
		print("- retry publish...")
		@@ -717,79 +755,105 @@ def run() -> None:
		prog="sbom-scanner",
		description="This tool scans for SBOM files and publishes them to a Dependency Track server.",
		)
		parser.add_argument(
		dt_platform_group = parser.add_argument_group(
		"Dependency Track connection"
		)
		dt_platform_group.add_argument(
		"-u",
		"--base-api-url",
		default=os.getenv("DEPTRACK_BASE_API_URL"),
		help="Dependency Track server base API url (includes '/api')",
		)
		parser.add_argument(
		dt_platform_group.add_argument(
		"-k",
		"--api-key",
		default=os.getenv("DEPTRACK_API_KEY"),
		help="Dependency Track API key",
		)
		parser.add_argument(
		dt_platform_group.add_argument(
		"-i",
		"--insecure",
		action="store_true",
		default=os.getenv("DEPTRACK_INSECURE") in ["true", "yes", "1"],
		help="Skip SSL verification",
		)

		project_selection_group = parser.add_argument_group("Project settings")
		project_selection_group.add_argument(
		"-p",
		"--project-path",
		default=os.getenv("DEPTRACK_PROJECT_PATH"),
		help="Dependency Track target project path to publish SBOM files to (see doc)",
		)
		parser.add_argument(
		project_selection_group.add_argument(
		"-s",
		"--path-separator",
		default=os.getenv("DEPTRACK_PATH_SEPARATOR", "/"),
		help="Separator to use in project path (default: '/')",
		)
		parser.add_argument(
		project_selection_group.add_argument(
		"-t",
		"--tags",
		type=str,
		default=os.getenv("DEPTRACK_TAGS", ""),
		help="Comma separated list of tags to attach to the project",
		)
		project_selection_group.add_argument(
		"--parent-collection-logic",
		type=str,
		default=os.getenv(
		"DEPTRACK_PARENT_COLLECTION_LOGIC",
		CollectionLogic.ALL.name,
		),
		choices=list(map(lambda x: x.name, list(CollectionLogic))),
		help="Set up how the parent aggregates its direct children (ALL: all, TAG: with tag matching --parent-collection-logic-tag, LATEST: flagged as latest, NONE: disable), default is ALL (DT version >= 4.13.0)",
		)
		project_selection_group.add_argument(
		"--parent-collection-logic-tag",
		type=str,
		default=os.getenv("DEPTRACK_PARENT_COLLECTION_LOGIC_TAG", ""),
		help="Tag for aggregation if --parent-collection-logic is set to TAG",
		)

		sbom_management_group = parser.add_argument_group("SBOM management")
		sbom_management_group.add_argument(
		"-m",
		"--merge",
		action="store_true",
		default=os.getenv("DEPTRACK_MERGE") in ["true", "yes", "1"],
		help="Merge all SBOM files into one",
		)
		parser.add_argument(
		sbom_management_group.add_argument(
		"-o",
		"--merge-output",
		default=os.getenv("DEPTRACK_MERGE_OUTPUT"),
		help="Output merged SBOM file (only used with merge enabled) - for debugging purpose",
		)
		parser.add_argument(
		# <0: auto (from DT version) / 0: no trim / >0 max length
		sbom_management_group.add_argument(
		"-l",
		"--purl-max-len",
		type=int,
		default=int(os.getenv("DEPTRACK_PURL_MAX_LEN", "-1")),
		help="PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)",
		)

		misc_group = parser.add_argument_group("Miscellaneous")
		misc_group.add_argument(
		"-S",
		"--show-findings",
		action="store_true",
		default=os.getenv("DEPTRACK_SHOW_FINDINGS") in ["true", "yes", "1"],
		help="Wait for analysis and display found vulnerabilities",
		)
		parser.add_argument(
		misc_group.add_argument(
		"-R",
		"--risk-score-threshold",
		type=int,
		default=int(os.getenv("DEPTRACK_RISK_SCORE_THRESHOLD", "-1")),
		help="Risk score threshold to fail the scan (<0: disabled - default: -1)",
		)
		parser.add_argument(
		"-i",
		"--insecure",
		action="store_true",
		default=os.getenv("DEPTRACK_INSECURE") in ["true", "yes", "1"],
		help="Skip SSL verification",
		)
		# <0: auto (from DT version) / 0: no trim / >0 max length
		parser.add_argument(
		"-l",
		"--purl-max-len",
		type=int,
		default=int(os.getenv("DEPTRACK_PURL_MAX_LEN", "-1")),
		help="PURLs max length (-1: auto, 0: no trim, >0: trim to size - default: -1)",
		)
		parser.add_argument(
		"-t",
		"--tags",
		type=str,
		default=os.getenv("DEPTRACK_TAGS", ""),
		help="Comma separated list of tags to attach to the project",
		)

		parser.add_argument(
		"sbom_patterns",
		nargs="*",
		@@ -815,6 +879,13 @@ def run() -> None:
		fail(
		"Dependency Track target project path is required (use --project-path CLI option or DEPTRACK_PROJECT_PATH variable)"
		)
		if (
		not args.parent_collection_logic_tag
		and args.parent_collection_logic == CollectionLogic.TAG.name
		):
		fail(
		f"You need to specify a tag with --parent-collection-logic-tag (or DEPTRACK_PARENT_COLLECTION_LOGIC_TAG env var) if parent collection logic has been set to {CollectionLogic.TAG.name}"
		)

		# print execution parameters
		print("Scanning SBOM files...")
		@@ -827,6 +898,14 @@ def run() -> None:
		print(
		f"- project tags (--tags): {AnsiColors.CYAN}{args.tags}{AnsiColors.RESET}"
		)
		print(
		f"- parent collection logic (--parent-collection-logic): {AnsiColors.CYAN}{args.parent_collection_logic}{AnsiColors.RESET}"
		+ (
		f" matching {AnsiColors.CYAN}{args.parent_collection_logic_tag}{AnsiColors.RESET} (--parent-collection-logic-tag)"
		if args.parent_collection_logic == CollectionLogic.TAG.name
		else ""
		)
		)
		print(
		f"- path separator (--path-separator): {AnsiColors.CYAN}{args.path_separator}{AnsiColors.RESET}"
		)