Commit 56c72291 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feature/semgrep' into 'main' 

feat: implement semgrep SAST

Closes #14

See merge request to-be-continuous/dotnet!20
parents a7bb637c ccb1be2b

README.md

+29 −0
Original line number Diff line number Diff line
@@ -336,6 +336,35 @@ The following reports are generated: 
| `reports/dotnet-security-scan-*.gitlab-codequality.json` | [Code Climate](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types) | [GitLab integration](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportscodequality) |





### `dotnet-semgrep` job



This job performs a [Semgrep](https://semgrep.dev/docs/) analysis.



It is bound to the `test` stage, and uses the following variables:



| Input / Variable | Description | Default Value |

| ---------------- | ----------- | ------------- |

| `semgrep-enabled` / `DOTNET_SEMGREP_ENABLED` | Set to `true` to enable this job | `false` |

| `semgrep-image` / `DOTNET_SEMGREP_IMAGE` | The Docker image used to run [Semgrep](https://semgrep.dev/docs/) | `docker.io/semgrep/semgrep:latest` <br/>[![Trivy Badge](https://to-be-continuous.gitlab.io/doc/secu/trivy-badge-DOTNET_SEMGREP_IMAGE.svg)](https://to-be-continuous.gitlab.io/doc/secu/trivy-DOTNET_SEMGREP_IMAGE) |

| `semgrep-args` / `DOTNET_SEMGREP_ARGS` | Semgrep [scan options](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | `--metrics off --disable-version-check` |

| `semgrep-rules` / `DOTNET_SEMGREP_RULES` | Space-separated list of [Semgrep rules](https://semgrep.dev/docs/running-rules).<br/>Can be both local YAML files or remote rules from the [Segmrep Registry](https://semgrep.dev/explore) (denoted by the `p/` prefix). | `p/csharp` |

| `semgrep-download-rules-enabled` / `DOTNET_SEMGREP_DOWNLOAD_RULES_ENABLED` | Download Semgrep remote rules | `true` |



> :information_source: Semgrep may [collect some metrics](https://semgrep.dev/docs/metrics), especially when using rules from the Semgrep Registry.

> To protect your privacy and let you run Semgrep in air-gap environments, this template disables all Semgrep metrics by default:

>

> * rules from the Semgrep registry are pre-downloaded and passed to Semgrep as local rule files (can be disabled by setting `semgrep-download-rules-enabled` / `DOTNET_SEMGREP_DOWNLOAD_RULES_ENABLED` to `false`),

> * the `--metrics` option is set to `off`,

> * the `--disable-version-check` option is set.



In addition to a textual report in the console, this job produces the following reports, kept for one week:



| Report | Format | Usage |

| ------ | ------ | ----- |

| `$DOTNET_PROJECT_DIR/reports/dotnet-semgrep-<project>.gitlab-sast.json` | [GitLab's SAST format](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | [GitLab integration](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportssast) |

| `$DOTNET_PROJECT_DIR/reports/dotnet-semgrep-<project>.native.json` | [Semgrep's JSON format](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options) | [DefectDojo integration](https://docs.defectdojo.com/supported_tools/parsers/file/semgrep/)<br/>_This report is generated only if DefectDojo template is detected_ |

| `$DOTNET_PROJECT_DIR/reports/dotnet-semgrep-<project>.gitlab-codequality.json` | [Code Climate](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types) | [GitLab integration](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportscodequality) |

| `$DOTNET_PROJECT_DIR/reports/dotnet-semgrep-<project>.sarif.json` | [OASIS Open Static Analysis Results Interchange Format (SARIF)](https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif) | Converted to Gitlab reports |



### `dotnet-sbom` job

kicker.json

+29 −0
Original line number Diff line number Diff line
@@ -198,6 +198,35 @@ 
        }

      ]

    },

    {

      "id": "dotnet-semgrep",

      "name": "Semgrep",

      "description": "[Semgrep](https://semgrep.dev/docs/) analysis",

      "enable_with": "DOTNET_SEMGREP_ENABLED",

      "variables": [

        {

          "name": "DOTNET_SEMGREP_IMAGE",

          "description": "The Docker image used to run [Semgrep](https://semgrep.dev/docs/)",

          "default": "docker.io/semgrep/semgrep:latest"

        },

        {

          "name": "DOTNET_SEMGREP_ARGS",

          "description": "Semgrep [scan options](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options)",

          "default": "--metrics off --disable-version-check"

        },

        {

          "name": "DOTNET_SEMGREP_RULES",

          "description": "Space-separated list of [Semgrep rules](https://semgrep.dev/docs/running-rules).\n\nCan be both local YAML files or remote rules from the [Semgrep Registry](https://semgrep.dev/explore) (denoted by the `p/` prefix)",

          "default": "p/csharp"

        },

        {

          "name": "DOTNET_SEMGREP_DOWNLOAD_RULES_ENABLED",

          "description": "Download Semgrep remote rules",

          "type": "boolean",

          "default": "true"

        }

      ]

    },

    {

      "id": "publish",

      "name": "dotnet publish",

templates/gitlab-ci-dotnet.yml

+109 −0
Original line number Diff line number Diff line
@@ -131,6 +131,26 @@ spec: 
       

       _defaults to [GitLab project's packages repository](https://docs.gitlab.com/user/packages/nuget_repository/)_ 

      default: ${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/nuget/index.json

    semgrep-image:

      description: The Docker image used to run [Semgrep](https://semgrep.dev/docs/)

      default: docker.io/semgrep/semgrep:latest

    semgrep-enabled:

      description: Enable Semgrep

      type: boolean

      default: false

    semgrep-args:

      description: Semgrep [scan options](https://semgrep.dev/docs/cli-reference#semgrep-scan-command-options)

      default: --metrics off --disable-version-check

    semgrep-rules:

      description: |-

        Space-separeted list of [Semgrep rules](https://semgrep.dev/docs/running-rules).



        Can be both local YAML files or remote rules from the [Semgrep Registry](https://semgrep.dev/explore) (denoted by the `p/` prefix)

      default: p/csharp

    semgrep-download-rules-enabled:

      description: Download Semgrep remote rules

      type: boolean

      default: true

---

# default workflow rules: Merge Request pipelines

.tbc-workflow-rules:
@@ -257,6 +277,13 @@ variables: 
  DOTNET_NUGET_SYMBOL_REPO: $[[ inputs.nuget-symbol-repo ]]

  DOTNET_NUGET_SYMBOL_API_KEY: $CI_JOB_TOKEN



  # Semgrep

  DOTNET_SEMGREP_IMAGE: $[[ inputs.semgrep-image ]]

  DOTNET_SEMGREP_ENABLED: $[[ inputs.semgrep-enabled ]]

  DOTNET_SEMGREP_ARGS: $[[ inputs.semgrep-args ]]

  DOTNET_SEMGREP_RULES: $[[ inputs.semgrep-rules ]]

  DOTNET_SEMGREP_DOWNLOAD_RULES_ENABLED: $[[ inputs.semgrep-download-rules-enabled ]]



  # default production ref name (pattern)

  PROD_REF: '/^(master|main)$/'

  # default integration ref name (pattern)
@@ -1473,6 +1500,64 @@ stages: 
    log_info "Code formatting check completed"

  }



  function dotnet_setup_semgrep_rules() {

    if [[ "${DOTNET_SEMGREP_DOWNLOAD_RULES_ENABLED}" == "true" ]]

    then

      log_info "Download Semgrep rule files..."

      for rule in $DOTNET_SEMGREP_RULES

      do

        if [[ -r $rule ]]

        then

          log_info "... rule file $rule found: skip"

          SEMGREP_RULES="${SEMGREP_RULES} $rule"

        else

          log_info "... rule file $rule not found : download (https://semgrep.dev/c/$rule)"

          dest_file="semgrep-${rule/p\//}.yml"

          wget "https://semgrep.dev/c/$rule" -O "$dest_file"

          SEMGREP_RULES="${SEMGREP_RULES} $dest_file"

        fi

      done

      SEMGREP_RULES="${SEMGREP_RULES:1}"

      export SEMGREP_RULES

    else

      # download not enabled: simply use $DOTNET_SEMGREP_RULES

      export SEMGREP_RULES="${DOTNET_SEMGREP_RULES}"

    fi

  }



  function dotnet_run_semgrep_scan() {

    log_info "Running Semgrep scan on project: ${DOTNET_PROJECT_DIR}"

    # shellcheck disable=SC2174

    mkdir -p -m 777 "${DOTNET_PROJECT_DIR}/reports"

    dotnet_setup_semgrep_rules



    local semgrep_reports

    semgrep_reports="--gitlab-sast-output=${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.gitlab-sast.json --sarif-output=${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.sarif.json  ${DEFECTDOJO_SEMGREP_REPORTS:+--json-output=${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.native.json}"



    local semgrep_rc

    semgrep_rc=0

    # shellcheck disable=SC2086

    if [[  CI_COMMIT_REF_NAME =~ $RELEASE_REF || $CI_COMMIT_REF_NAME =~ $INTEG_REF || $CI_COMMIT_REF_NAME =~ $PROD_REF ]]; then

      semgrep "${DOTNET_PROJECT_DIR}" --baseline=origin/${CI_DEFAULT_BRANCH} ${TRACE+--verbose} ${DOTNET_SEMGREP_ARGS} ${semgrep_reports} || semgrep_rc=$?

    else

      semgrep ci --subdir "${DOTNET_PROJECT_DIR}" ${TRACE+--verbose} ${DOTNET_SEMGREP_ARGS} ${semgrep_reports} || semgrep_rc=$?

    fi



    sed -i "s|${CI_PROJECT_DIR}/||g" "${CI_PROJECT_DIR}/${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.gitlab-sast.json" || true

    sed -i "s|${CI_PROJECT_DIR}/||g" "${CI_PROJECT_DIR}/${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.sarif.json" || true



    if ! echo "$GITLAB_FEATURES" | grep -qE "sast_|security_advanced"; then

      log_warn "GitLab security dashboard feature is not enabled. Reporting to codequality."

      _dotnet_convert_sarif_reports codequality "${CI_PROJECT_DIR}/${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.sarif.json"

    fi



    if [[ "${semgrep_rc}" -ne 0 ]]; then

      fail "Semgrep scan detected issues in the codebase. Review the report at: ${DOTNET_PROJECT_DIR}/reports/${CI_JOB_NAME}-${DOTNET_PROJECT_NAME}.gitlab-sast.json"

    else

      log_info "Semgrep scan completed with no issues detected"

    fi

  }



  function _dotnet_generate_os_arch_template() {

    local template=${1}

    local separator=${2:-}
@@ -2046,6 +2131,30 @@ dotnet-format: 
      when: never

    - !reference [.test-policy, rules]



dotnet-semgrep:

  extends: .dotnet-env-base

  image: $DOTNET_SEMGREP_IMAGE

  cache : {}

  stage: test

  before_script:

    - !reference [.dotnet-scripts]

  script:

    - dotnet_run_semgrep_scan

  artifacts:

    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $ĈI_COMMIT_REF_SLUG"

    when: "always"

    expire_in: 1 week

    reports:

      sast: "${DOTNET_PROJECT_DIR}/reports/dotnet-semgrep-*.gitlab-sast.json"

      codequality: "${DOTNET_PROJECT_DIR}/reports/dotnet-semgrep-*.gitlab-codequality.json"

    paths:

      - ${DOTNET_PROJECT_DIR}/reports/dotnet-semgrep-*

  rules:

    # exclude if not enabled

    - if: '$DOTNET_SEMGREP_ENABLED != "true"'

      when: never

    - !reference [.test-policy, rules]



dotnet-sbom:

  extends: .dotnet-env-base

  stage: test