Commit a12a23c9 authored by Thibaud-Vdb's avatar Thibaud-Vdb
Browse files

Merge branch '674-mega_linter-fix-sarif-files-converting-to-xml' into 'latest' 

Resolve "[mega_linter] - Fix sarif files converting to xml"

Closes #674

See merge request r2devops/hub!426
parents 1aa69322 88c15106

.gitlab-ci.yml

+2 −2
Original line number Diff line number Diff line
@@ -137,9 +137,9 @@ job_image_scan: 
    -   IMAGE=$(pipenv run python3 tools/job_image/job_image.py ${JOB})

    -   if [ ! -z ${IMAGE} ]; then

    -     NAME=$(basename ${IMAGE})

    -     ./trivy image --template "@${TEMPLATE_NAME}" --security-checks vuln --vuln-type ${TRIVY_VULN_TYPE} --cache-dir ${TRIVY_CACHE_DIR} -o ${OUTPUT_DIR}/${NAME}.${TRIVY_OUTPUT} ${IMAGE}

    -     ./trivy image --template "@${TEMPLATE_NAME}" --security-checks vuln --vuln-type ${TRIVY_VULN_TYPE} --offline-scan --cache-dir ${TRIVY_CACHE_DIR} -o ${OUTPUT_DIR}/${NAME}.${TRIVY_OUTPUT} ${IMAGE}

    -     if [ ! -z ${TRIVY_EXIT_ON_SEVERITY} ]; then

    -       ./trivy image --template "@${TEMPLATE_NAME}" --security-checks vuln --vuln-type ${TRIVY_VULN_TYPE} --cache-dir ${TRIVY_CACHE_DIR} --exit-code 1 --severity ${TRIVY_SEVERITY} -o ${OUTPUT_DIR}/${NAME}-failed-${TRIVY_OUTPUT} ${IMAGE}

    -       ./trivy image --template "@${TEMPLATE_NAME}" --security-checks vuln --vuln-type ${TRIVY_VULN_TYPE} --offline-scan --cache-dir ${TRIVY_CACHE_DIR} --exit-code 1 --severity ${TRIVY_SEVERITY} -o ${OUTPUT_DIR}/${NAME}-failed-${TRIVY_OUTPUT} ${IMAGE}

    -     fi

    -   fi

    -   IMAGE=""

jobs/mega_linter/CHANGELOG.md

+4 −0
Original line number Diff line number Diff line 
# Changelog

All notable changes to this job will be documented in this file.



## [1.8.0] - 2023-01-18

* Bump megalinter image version to v6.18.0

* Remove Junit reports



## [1.7.0] - 2022-12-11

* Set `SARIF_REPORTER` to true to enable the SARIF reporter

* Use name in `artifact:paths` instead of variable because they are not expanded there

jobs/mega_linter/README.md

+5 −8
Original line number Diff line number Diff line
@@ -46,11 +46,12 @@ customize its behavior. 
| `MEGALINTER_CONFIG` | MegaLinter configuration file location | `.mega_linter.yml` |

| `VALIDATE_ALL_CODEBASE` | Whether linters should only go through **edited** or **new** files | `true` |

| `REPORT_OUTPUT_FOLDER` | Folder where are stored all the reports | `megalinter-reports` |

| `CONVERTED_OUTPUT_FOLDER` | Folder where are stored `JUnit` or `CodeClimate` reports | `converted-xml.report` |

| `CONVERTED_OUTPUT_FOLDER` | Folder where are stored `CodeClimate` reports | `converted.report` |

|`GITLAB_COMMENT_REPORTER` | Posts Mega-Linter results summary in the comments of the related merge request ([⚠️ GitLab API access require](https://oxsecurity.github.io/megalinter/latest/reporters/GitlabCommentReporter/#configuration){:target="_blank"}) | `true` |

| `REPORT_SUITE_TEST_NAME` | `JUnit` report suites name | `mega_linter` |

| `DISABLE_LINTERS` | Comma separated list of linters to be disabled | ` ` |

| `IMAGE_TAG` | The default tag for the docker image | `v6.13.0` |

| `IMAGE_TAG` | The default tag for the docker image | `v6.18.0` |

| `SARIF_CODECLIMATE_VERSION` | Version tag of the [sarif-codeclimate](https://www.npmjs.com/package/sarif-codeclimate) package to convert sarif reports | `1.0.2` |





## Optimize MegaLinter
@@ -71,14 +72,10 @@ mega_linter: 


## Artifacts



Two formats are available for the artifacts:

- [JUnit](https://junit.org/junit5/)'s XML report to display error report directly in pipeline `Test` tab and in

merge request widget.

- [CodeClimate](https://codeclimate.com/quality)'s JSON report to display error report directly in merge request widget.



## Dependencies

The job uses the following dependencies for converting the `output` to `JUnit` or `CodeClimate`:

- [sarif-junit](https://www.npmjs.com/package/sarif-junit) tool to convert `SARIF` to `JUnit`

The job uses the following dependencies for converting the `output` to `CodeClimate`:

- [sarif-codeclimate](https://www.npmjs.com/package/sarif-codeclimate) tool to convert `SARIF` to `CodeClimate`



## Author and contributors

jobs/mega_linter/mega_linter.yml

+8 −8
Original line number Diff line number Diff line
@@ -36,15 +36,17 @@ mega_linter: 
    SARIF_REPORTER: "true"

    ### Custom variables

    # End folder where reports are held

    CONVERTED_OUTPUT_FOLDER: "converted-xml.report"

    # Name of the test suite in JUnit

    REPORT_SUITE_TEST_NAME: "mega_linter"

    IMAGE_TAG: "v6.13.0"

    CONVERTED_OUTPUT_FOLDER: "converted.report"

    # Image tag of the MegaLinter image used in this job

    IMAGE_TAG: "v6.18.0"

    # Version tag of the sarif-codeclimate package to convert sarif reports

    # https://www.npmjs.com/package/sarif-codeclimate

    SARIF_CODECLIMATE_VERSION: "1.0.2"

  script:

    - FAILED=false

    - |

      if /entrypoint.sh true; then

        echo -e "\e[32mMegaLinter seemed not to have found any issue!"

        echo -e "\e[32mMegaLinter haven't found any issue!"

      else

        echo -e "\e[31mMegaLinter found some issues. Check this job log and the artifact reports."

        FAILED=true
@@ -53,8 +55,7 @@ mega_linter: 
    - mkdir ${CI_PROJECT_DIR}/${CONVERTED_OUTPUT_FOLDER}

    - cp ${CI_PROJECT_DIR}/${REPORT_OUTPUT_FOLDER}/megalinter-report.sarif ${CI_PROJECT_DIR}/${CONVERTED_OUTPUT_FOLDER}

    - cd ${CI_PROJECT_DIR}/${CONVERTED_OUTPUT_FOLDER}

    - npx -y sarif-codeclimate@1.0.2 -i megalinter-report.sarif -o code-climate.json

    - npx -y sarif-junit@1.1.3 -i megalinter-report.sarif -o junit.xml --test-suite ${REPORT_SUITE_TEST_NAME}

    - npx -y sarif-codeclimate@${SARIF_CODECLIMATE_VERSION} -i megalinter-report.sarif -o code-climate.json

    - if [ "$FAILED" = true ]; then

    -   exit 1

    - fi
@@ -64,6 +65,5 @@ mega_linter: 
      - megalinter-reports

    expire_in: 1 week

    reports:

      junit: "${CI_PROJECT_DIR}/${CONVERTED_OUTPUT_FOLDER}/*.xml"

      codequality: "${CI_PROJECT_DIR}/${CONVERTED_OUTPUT_FOLDER}/code-climate.json"

  allow_failure: true

tools/job_av/job_av.py

+1 −1
Original line number Diff line number Diff line
@@ -26,7 +26,7 @@ JOB_DIR = "job_av" 
SCANNED_IMAGES_FILE = os.getenv("SCANNED_IMAGES_FILE")

SCANNED_IMAGES = []



BLACKLIST = ["github/super-linter:v3.14.3", "shiftleft/sast-scan:v1.9.29", "oxsecurity/megalinter:v6.13.0", "github/super-linter:v4.9.0"]

BLACKLIST = ["github/super-linter:v3.14.3", "shiftleft/sast-scan:v1.9.29", "oxsecurity/megalinter:v6.18.0", "github/super-linter:v4.9.0"]





def argparse_setup():