Commit 8d851174 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feature/support-cobertura' into 'master'

feat(coverage): add Cobertura support

See merge request to-be-continuous/node!32
parents 87cc81f2 478dd580
Loading
Loading
Loading
Loading
+127 −123
Original line number Diff line number Diff line
@@ -60,143 +60,147 @@ This job is bound to the `build` stage, and uses the following variables:
| `NODE_BUILD_ARGS`             | npm [run script](https://docs.npmjs.com/cli/v8/commands/npm-run-script) arguments <br/> yarn [run script](https://classic.yarnpkg.com/en/docs/cli/run) arguments  | `run build --prod`    |
| `NODE_TEST_ARGS`              | npm [test](https://docs.npmjs.com/cli/v8/commands/npm-test) arguments <br/> yarn [test](https://classic.yarnpkg.com/en/docs/cli/test) arguments                   | `test -- --coverage`  |

The job generates a unit test report that you will find here: `NODE_PROJECT_DIR/reports/node-test.xunit.xml`.

#### Unit testing with Jest

If you're using [Jest](https://jestjs.io/) as unit testing framework, you'll have to make the following configuration in
order to [integrate your unit tests results to GitLab](https://docs.gitlab.com/ee/ci/unit_test_reports.html) (and optionally to SonarQube).

##### Coverage
#### Unit Tests and Code Coverage reports

For coverage, Jest comes built-in with **Istanbul** package. So no need for extra dependency.
This chapter details the required configuration (depending on the unit testing framework you're using) in
order to integrate your [unit tests reports](https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html) and [code coverage reports](https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html) to GitLab.

```js
"jest": {
...
    "coverageDirectory": './reports',
...
},
```
Additionally, if also using SonarQube, you'll have to enable some extra reporters.

##### JUnit Report
#### Unit testing with Jest

By default Jest doesn't generate any test report supported by GitLab. To do so, you need to use the [jest-junit](https://www.npmjs.com/package/jest-junit) package.
Here is the required configuration if you're using [Jest](https://jestjs.io/) as unit testing framework.

Add the package as a development dependency:
| Reporter         | Needs `npm install` | Expected report file  | Usage             |
| ---------------- | --------------------- | ----------------- | ----------------- |
| [jest-junit](https://github.com/jest-community/jest-junit) | Yes | `reports/node-test.xunit.xml` | [GitLab unit tests integration](https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html) _(JUnit format)_ |
| istanbul [text](https://istanbul.js.org/docs/advanced/alternative-reporters/#text) | No | N/A _(stdout)_ | [GitLab MR test coverage results](https://docs.gitlab.com/ee/ci/pipelines/settings.html#merge-request-test-coverage-results) _(GitLab grabs coverage from stdout)_ |
| istanbul [cobertura](https://istanbul.js.org/docs/advanced/alternative-reporters/#cobertura) | No | `reports/cobertura-coverage.xml` | [GitLab code coverage integration](https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html) _(Cobertura format)_ |
| [jest-sonar](https://github.com/sh33dafi/jest-sonar)       | Yes | `reports/node-test.sonar.xml` | [SonarQube unit tests integration](https://docs.sonarqube.org/latest/analysis/generic-test/) _(generic SonarQube format)_ |
| istanbul [lcovonly](https://istanbul.js.org/docs/advanced/alternative-reporters/#lcovonly) | No | `reports/lcov.info` | [SonarQube code coverage integration](https://docs.sonarqube.org/latest/analysis/test-coverage/javascript-typescript-test-coverage/) _(JS/TS LCOV format)_ |

```bash
npm install --save-dev jest-junit
```

Then update your `jest.config.js` or `package.json` as follows:
Here is an example of a `jest.config.js` configuration file with all the above reporters configured as expected:

```js
"jest": {
...
    "reporters": [
  reporters: [
    "default",
    // 'jest-junit' to enable GitLab unit test report integration
    [
      "jest-junit",
      {
                "outputDirectory": "reports",
                "outputName": "node-test.xunit.xml"
            }
        ]
    ]
...
          outputDirectory: "reports",
          outputName: "node-test.xunit.xml",
      },
```

##### SonarQube report

By default Jest doesn't generate any test report supported by SonarQube. To do so, you need to use the [jest-sonar](https://www.npmjs.com/package/jest-sonar) package.

Add the package as a development dependency:

```bash
npm install --save-dev jest-sonar
```

Then update your `jest.config.js` or `package.json` as follows:

```js
"jest": {
...
    "reporters": [
        "default",
        [
            "jest-junit",
            {
                "outputDirectory": "reports",
                "outputName": "node-test.xunit.xml"
            }
    ],
    // [OPTIONAL] only if using SonarQube
    // 'jest-sonar' to enable SonarQube unit test report integration
    [
      "jest-sonar",
      {
                "outputDirectory": "reports",
                "outputName": "node-test.sonar.xml"
            }
        ]
    ],
...
          outputDirectory: "reports",
          outputName: "node-test.sonar.xml",
      },
    ],
  ],
  coverageDirectory: "reports",
  coverageReporters: [
    // 'text' to let GitLab grab coverage from stdout
    "text",
    // 'cobertura' to enable GitLab test coverage visualization
    "cobertura",
    // [OPTIONAL] only if using SonarQube
    // 'lcovonly' to enable SonarQube test coverage reporting
    "lcovonly",
  ],
```

#### Unit testing with Mocha

If you're using [Mocha](https://mochajs.org/) as unit testing framework, you'll have to make the following configuration in
order to [integrate your unit tests results to GitLab](https://docs.gitlab.com/ee/ci/unit_test_reports.html) (and optionally to SonarQube).
Here is the required configuration if you're using [Mocha](https://mochajs.org/) as unit testing framework.

##### Coverage
| Reporter         | Needs `npm install` | Expected report file  | Usage             |
| ---------------- | --------------------- | ----------------- | ----------------- |
| [mocha-junit-reporter](https://github.com/michaelleeallen/mocha-junit-reporter) | Yes | `reports/node-test.xunit.xml` | [GitLab unit tests integration](https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html) _(JUnit format)_ |
| istanbul [text](https://istanbul.js.org/docs/advanced/alternative-reporters/#text) | Yes (in `nyc` package) | N/A _(stdout)_ | [GitLab MR test coverage results](https://docs.gitlab.com/ee/ci/pipelines/settings.html#merge-request-test-coverage-results) _(GitLab grabs coverage from stdout)_ |
| istanbul [cobertura](https://istanbul.js.org/docs/advanced/alternative-reporters/#cobertura) | Yes (in `nyc` package) | `reports/cobertura-coverage.xml` | [GitLab code coverage integration](https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html) _(Cobertura format)_ |
| [mocha-sonarqube-reporter](https://github.com/mmouterde/mocha-sonarqube-reporter)       | Yes | `reports/node-test.sonar.xml` | [SonarQube unit tests integration](https://docs.sonarqube.org/latest/analysis/generic-test/) _(generic SonarQube format)_ |
| istanbul [lcovonly](https://istanbul.js.org/docs/advanced/alternative-reporters/#lcovonly) | Yes (in `nyc` package) | `reports/lcov.info` | [SonarQube code coverage integration](https://docs.sonarqube.org/latest/analysis/test-coverage/javascript-typescript-test-coverage/) _(JS/TS LCOV format)_ |

By default Mocha doesn't provide coverage. To do so you need to use the [Istanbul](https://www.npmjs.com/package/nyc) package.

Add the package as a development dependency:
:warning: Remarks:

1. By default - unlike Jest - Mocha doesn't provide code coverage. To do so you need to install [Istanbul](https://www.npmjs.com/package/nyc) package (`nyc`):
    ```shell
    npm install --save-dev nyc
    ```
2. the default `xunit` Mocha reporter doesn't produce a JUnit format supported by GitLab, that's why we recommend you to use  `mocha-junit-reporter` instead.
3. Mocha doesn't support multiple unit tests reporters. 
So unfortunaltely, if you're using SonarQube, you'll have to choose which report you want to generate. 
Another option is to use [mocha-multi-reporters](https://github.com/stanleyhlng/mocha-multi-reporters) (see documentation)

Then update your `package.json` as follows:
Mocha may be either configured with CLI options of using separate Mocha and `nyc` config files.

```js
Here is the required configuration with CLI options directly in the `package.json` file:

```json
  "scripts": {
    "test": "npm run mocha",
  "mocha": "nyc --reporter=lcov --report-dir=./reports --reporter=text mocha app/tests/*.test.js",
}
```

##### JUnit report

By default Mocha doesn't generate any test report supported by GitLab. To do so you need to use the [mocha-junit-reporter](https://www.npmjs.com/package/mocha-junit-reporter) package.

Add the package as a development dependency:

```shell
npm install --save-dev mocha-junit-reporter
    "mocha": "nyc --report-dir=reports --reporter=text --reporter=lcovonly --reporter=cobertura mocha --reporter mocha-junit-reporter --reporter-option mochaFile=reports/node-test.xunit.xml test/*.js",
    ...
  },
  ...
```

Then update your `package.json` as follows:
Here is the equivalent using separate config files:

```js
* `package.json`:
    ```json
      "scripts": {
        "test": "npm run mocha",
  "mocha": "nyc mocha app/tests/*.test.js --reporter spec --reporter mocha-junit-reporter --reporter-options mochaFile=./reports/node-test.xunit.xml",
        "mocha": "nyc mocha test/*.js",
        ...
      },
      ...
    ```
* `.mocharc.json`:
    1. with `mocha-junit-reporter` (for GitLab):
        ```json
        {
          "reporter": "mocha-junit-reporter",
          "reporter-option": ["mochaFile=reports/node-test.xunit.xml"]
        }
        ```

##### SonarQube report

Mocha provides XUnit reporter which is compatible with SonarQube. So no need for extra dependency, just add this parameter `--reporter-option`

Then update your `package.json` as follows:

```js
"scripts": {
  "test": "npm run mocha",
  "mocha": "nyc mocha app/tests/*.test.js --reporter-option output=./reports/node-test.sonar.xml",
    2. with `mocha-sonarqube-reporter` (for SonarQube):
        ```json
        {
          "reporter": "mocha-sonarqube-reporter",
          "reporter-option": ["output=reports/node-test.sonar.xml"]
        }
        ```
    3. with both (using `mocha-multi-reporters`):
        ```json
        {
          "reporter": "mocha-multi-reporters",
          "reporter-option": ["configFile=.mmr.json"]
        }
        ```
        With `.mmr.json`:
        ```json
        {
            "reporterEnabled": "spec, mocha-junit-reporter, mocha-sonarqube-reporter",
            "mochaJunitReporterReporterOptions": {
              "mochaFile": "reports/node-test.xunit.xml"
            },
            "mochaSonarqubeReporterReporterOptions": {
              "output": "reports/node-test.sonar.xml"
            }
        }
        ```
* `.nycrc.json`:
    ```json
    {
      "reporter": ["text", "cobertura", "lcovonly"],
      "report-dir": "reports"
    }
    ```

+7 −1
Original line number Diff line number Diff line
@@ -319,14 +319,20 @@ node-build:
  coverage: '/^All files\s*\|\s*(\d+(?:\.\d+)?)/'
  artifacts:
    reports:
      junit: $NODE_PROJECT_DIR/reports/node-test.xunit.xml
      coverage_report:
        coverage_format: cobertura
        path: "$NODE_PROJECT_DIR/reports/cobertura-coverage.xml"
      junit: 
        - $NODE_PROJECT_DIR/reports/node-test.xunit.xml
    when: always
    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"
    paths:
      - $NODE_PROJECT_DIR/$NODE_BUILD_DIR
      - $NODE_PROJECT_DIR/reports/node-test.*
      - $NODE_PROJECT_DIR/reports/node-coverage.*
      # no way to override default coverage reports when using Mocha
      - $NODE_PROJECT_DIR/reports/lcov.info
      - $NODE_PROJECT_DIR/reports/cobertura-coverage.xml
    expire_in: 1 day
  rules:
    # always if $NODE_BUILD_DISABLED not set