Commit 6e30702f authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feat/normalize-reports' into 'master'

Normalize reports

See merge request to-be-continuous/postman!24
parents 164488c0 30115140
Loading
Loading
Loading
Loading
+4 −5
Original line number Diff line number Diff line
@@ -31,12 +31,11 @@ It uses the following variable:
| `POSTMAN_EXTRA_ARGS`  | Newman extra [run options](https://github.com/postmanlabs/newman#command-line-options) (to use global variables, an environment or a data source for e.g.) | _none_ |
| `REVIEW_ENABLED`      | Set to `true` to enable Postman tests on review environments (dynamic environments instantiated on development branches) | _none_ (disabled) |

### Unit tests report integration
In addition to a textual report in the console, this job produces the following reports, kept for one day:

Postman test reports are [integrated to GitLab by generating JUnit reports](https://docs.gitlab.com/ee/ci/junit_test_reports.html).

This is done using the `newman` [JUnit reporter](https://github.com/postmanlabs/newman#junitxml-reporter)
with the following CLI options: `--reporters cli,junit --reporter-junit-export postman-junit.xml`  
| Report         | Format                                                                       | Usage             |
| -------------- | ---------------------------------------------------------------------------- | ----------------- |
| `reports/postman-*.xunit.xml` | [JUnit](https://github.com/postmanlabs/newman#junitxml-reporter) test report(s) | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsjunit) |

### Postman `{{base_url}}` auto evaluation

+5 −5
Original line number Diff line number Diff line
@@ -225,22 +225,22 @@ postman:
    # retrieve server url to test from 'environment_url.txt' file
    - eval_base_url
  script:
    - mkdir -p reports/postman
    - mkdir -p -m 777 reports
    - |
      for collection in $POSTMAN_COLLECTIONS
      do
        log_info "Running collection \\e[33;1m$collection\\e[0m..."
        log_info "Running collection \\e[33;1m${collection}\\e[0m..."
        namenoext=$(basename $collection | sed 's/\..*//')
        newman run $base_url_opt $ca_certs_opt ${TRACE+--verbose} --reporters cli,junit --reporter-junit-export reports/postman/$namenoext.xml $POSTMAN_EXTRA_ARGS "$collection"
        newman run $base_url_opt $ca_certs_opt ${TRACE+--verbose} --reporters cli,junit --reporter-junit-export reports/postman-$namenoext.xunit.xml $POSTMAN_EXTRA_ARGS "$collection"
      done
  artifacts:
    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"
    when: always
    paths:
      - reports/postman/
      - "reports/postman-*"
    reports:
      junit:
        - reports/postman/*.xml
        - "reports/postman-*.xunit.xml"
    expire_in: 1 day
  rules:
    - !reference [.acceptance-policy, rules]