Commit 9271907b authored by Gaëtan Montury's avatar Gaëtan Montury
Browse files

Merge branch 'fix/dep-group' into 'main' 

fix: split extras dependency and implemented the new "dependency-groups" standard.

Closes #135

See merge request to-be-continuous/python!196
parents 2e59e86f 78798b1d

README.md

+3 −1
Original line number Diff line number Diff line
@@ -51,7 +51,9 @@ The Python template uses some global configuration used throughout all jobs. 
| `PIP_EXTRA_INDEX_URL` | Extra Python repository url                                                           | _none_             |

| `pip-opts` / `PIP_OPTS` | pip [extra options](https://pip.pypa.io/en/stable/cli/pip/#general-options)     | _none_             |

| `pip-install-deps-opts` / `PIP_INSTALL_DEPS_OPTS` | pip install dependencies extra [options](https://pip.pypa.io/en/stable/cli/pip/#general-options) used with `pip` build-system only | _none_             |

| `extra-deps` / `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install for:<br/>- [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies)<br/>- [Poetry](https://python-poetry.org/docs/pyproject/#extras)<br/>- [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups)<br/>- pip with pyproject.toml ([PEP517](https://peps.python.org/pep-0517/) [PEP735](https://peps.python.org/pep-0735/)) require pip >= 25.1 | _none_ |

| `install-deps-opts` / `PYTHON_INSTALL_DEPS_OPTS` | Python build-system install dependencies extra options:<br/>  - For Poetry: [CLI options](https://python-poetry.org/docs/cli/#install)<br/>  - For uv: [CLI options](https://docs.astral.sh/uv/concepts/projects/sync/#syncing-optional-dependencies)<br/>  - For hatch: [CLI options](https://hatch.pypa.io/1.13/cli/reference/#hatch-env-create)<br/>  - For PDM: [CLI options](https://pdm-project.org/latest/reference/cli/#sync)<br/>  - For pipenv: [CLI options](https://pipenv.pypa.io/en/latest/commands.html#sync) | _none_             |

| `extra-deps` / `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install for:<br/>- [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies)<br/>- [Poetry](https://python-poetry.org/docs/pyproject/#extras)<br/>- [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups) | _none_ |

| `dep-groups` / `PYTHON_DEP_GROUPS` | Python dependency groups to install for:<br/>- [Poetry](https://python-poetry.org/docs/managing-dependencies#installing-group-dependencies)<br/>- [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups)<br/>- pip with pyproject.toml ([PEP517](https://peps.python.org/pep-0517/) [PEP735](https://peps.python.org/pep-0735/)) require pip >= 25.1<br/>- [PDM](https://pdm-project.org/latest/usage/dependency/#select-a-subset-of-dependency-groups-to-install) | _none_ |

| `reqs-file` / `PYTHON_REQS_FILE` | Main requirements file _(relative to `$PYTHON_PROJECT_DIR`)_<br/>For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only | `requirements.txt` |

| `extra-reqs-files` / `PYTHON_EXTRA_REQS_FILES` | Extra dev requirements file(s) to install _(relative to `$PYTHON_PROJECT_DIR`)_ | `requirements-dev.txt` |

kicker.json

+11 −2
Original line number Diff line number Diff line
@@ -50,12 +50,21 @@ 
    },

    {

      "name": "PIP_INSTALL_DEPS_OPTS",

      "description": "pip install dependencies extra [options](https://pip.pypa.io/en/stable/cli/pip/#general-options) used with `pip` build-system only",

      "description": "pip install dependencies options [options](https://pip.pypa.io/en/stable/cli/pip/#general-options) used with `pip` build-system only",

      "advanced": true

    },

    {

      "name": "PYTHON_INSTALL_DEPS_OPTS",

      "description": "Python build-system install dependencies extra options:\n  - For Poetry: [CLI options](https://python-poetry.org/docs/cli/#install)\n  - For uv: [CLI options](https://docs.astral.sh/uv/concepts/projects/sync/#syncing-optional-dependencies)\n  - For hatch: [CLI options](https://hatch.pypa.io/1.13/cli/reference/#hatch-env-create)\n  - For PDM: [CLI options](https://pdm-project.org/latest/reference/cli/#sync)\n  - For pipenv: [CLI options](https://pipenv.pypa.io/en/latest/commands.html#sync)",

      "advanced": true

    },

    {

      "name": "PYTHON_EXTRA_DEPS",

      "description": "Python extra sets of dependencies to install for:\n- [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies)\n- [Poetry](https://python-poetry.org/docs/pyproject/#extras)\n- [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups)\n- pip with pyproject.toml ([PEP517](https://peps.python.org/pep-0517/) [PEP735](https://peps.python.org/pep-0735/)) require pip >= 25.1"

      "description": "Python extra sets of dependencies to install for:\n- [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies)\n- [Poetry](https://python-poetry.org/docs/pyproject/#extras)\n- [uv](https://docs.astral.sh/uv/concepts/projects/sync/#handling-of-extraneous-packages)"

    },

    {

      "name": "PYTHON_DEP_GROUPS",

      "description": "Python dependency groups to install for:\n- [Poetry](https://python-poetry.org/docs/managing-dependencies#installing-group-dependencies)\n- [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups)\n- pip with pyproject.toml ([PEP517](https://peps.python.org/pep-0517/) [PEP735](https://peps.python.org/pep-0735/)) require pip >= 25.1\n- [PDM](https://pdm-project.org/latest/usage/dependency/#select-a-subset-of-dependency-groups-to-install)"

    }

  ],

  "features": [

templates/gitlab-ci-python.yml

+45 −13
Original line number Diff line number Diff line
@@ -53,15 +53,31 @@ spec: 
      description: pip extra [options](https://pip.pypa.io/en/stable/cli/pip/#general-options)

      default: ''

    pip-install-deps-opts:

      description:  pip install dependencies extra [options](https://pip.pypa.io/en/stable/cli/pip/#general-options) used with `pip` build-system only

      description:  pip install dependencies options [options](https://pip.pypa.io/en/stable/cli/pip/#general-options) used with `pip` build-system only

      default: ''

    install-deps-opts:

      description: |- 

        Python build-system install dependencies extra options:

          - For Poetry: [CLI options](https://python-poetry.org/docs/cli/#install)

          - For uv: [CLI options](https://docs.astral.sh/uv/concepts/projects/sync/#syncing-optional-dependencies)

          - For hatch: [CLI options](https://hatch.pypa.io/1.13/cli/reference/#hatch-env-create)

          - For PDM: [CLI options](https://pdm-project.org/latest/reference/cli/#sync)

          - For pipenv: [CLI options](https://pipenv.pypa.io/en/latest/commands.html#sync)

      default: ''

    extra-deps:

      description: |-

        Python extra sets of dependencies to install for:

        - [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies)

        - [Poetry](https://python-poetry.org/docs/pyproject/#extras)

        - [uv](https://docs.astral.sh/uv/concepts/projects/sync/#handling-of-extraneous-packages)

      default: ''

    dep-groups:

      description: |-

        Python dependency groups to install for:

        - [Poetry](https://python-poetry.org/docs/managing-dependencies#installing-group-dependencies)

        - [uv](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups)

        - pip with pyproject.toml ([PEP517](https://peps.python.org/pep-0517/) [PEP735](https://peps.python.org/pep-0735/)) require pip >= 25.1

        - [PDM](https://pdm-project.org/latest/usage/dependency/#select-a-subset-of-dependency-groups-to-install)

      default: ''

    package-enabled:

      description: Enable package
@@ -367,7 +383,9 @@ variables: 
  PYTHON_BUILD_SYSTEM: $[[ inputs.build-system ]]

  PIP_OPTS: $[[ inputs.pip-opts ]]

  PIP_INSTALL_DEPS_OPTS: $[[ inputs.pip-install-deps-opts ]]

  PYTHON_INSTALL_DEPS_OPTS: $[[ inputs.install-deps-opts ]]

  PYTHON_EXTRA_DEPS: $[[ inputs.extra-deps ]]

  PYTHON_DEP_GROUPS: $[[ inputs.dep-groups ]]

  PYTHON_PACKAGE_ENABLED: $[[ inputs.package-enabled ]]

  PYLINT_ENABLED: $[[ inputs.pylint-enabled ]]

  PYLINT_ARGS: $[[ inputs.pylint-args ]]
@@ -964,20 +982,23 @@ variables: 
        log_warn "Using Poetry but \\e[33;1mpoetry.lock\\e[0m file not found: you shall commit it with your project files"

      fi

      maybe_install_build_system

      poetry install ${TRACE+--verbose} ${PYTHON_EXTRA_DEPS:+--extras "$PYTHON_EXTRA_DEPS"}

      # shellcheck disable=SC2086

      poetry install ${TRACE+--verbose} ${PYTHON_EXTRA_DEPS:+--extras "$PYTHON_EXTRA_DEPS"} ${PYTHON_DEP_GROUPS:+--only $PYTHON_DEP_GROUPS} ${PYTHON_INSTALL_DEPS_OPTS}

      ;;

    setuptools)

      maybe_install_build_system

      # shellcheck disable=SC2086

      pip install ${TRACE+--verbose} ${PIP_OPTS} ".${PYTHON_EXTRA_DEPS:+[$PYTHON_EXTRA_DEPS]}"

      pip install ${TRACE+--verbose} ${PIP_OPTS} ".${PYTHON_EXTRA_DEPS:+[$PYTHON_EXTRA_DEPS]}" ${PIP_INSTALL_DEPS_OPTS}

      ;;

    pipenv*)

      maybe_install_build_system

      if  [[ ! -f "Pipfile.lock" ]]; then

        log_warn "Using Pipenv but \\e[33;1mPipfile.lock\\e[0m file not found: you shall commit it with your project files"

        pipenv install ${TRACE+--verbose} --dev --system

        # shellcheck disable=SC2086

        pipenv install ${TRACE+--verbose} --dev --system ${PYTHON_INSTALL_DEPS_OPTS}

      else

        pipenv sync ${TRACE+--verbose} --dev --system

        # shellcheck disable=SC2086

        pipenv sync ${TRACE+--verbose} --dev --system ${PYTHON_INSTALL_DEPS_OPTS}

      fi

      ;;

    pip*)
@@ -988,12 +1009,12 @@ variables: 
      . .venv/bin/activate



      maybe_install_build_system

      extra_groups=""

      for group in ${PYTHON_EXTRA_DEPS}; do

        extra_groups="${extra_groups} --group $group"

      dep_groups=""

      for group in ${PYTHON_DEP_GROUPS}; do

        dep_groups="${dep_groups} --group $group"

      done

      # shellcheck disable=SC2086

      pip install . ${PIP_OPTS} ${extra_groups} ${PIP_INSTALL_DEPS_OPTS}

      pip install . ${PIP_OPTS} ${dep_groups} ${PIP_INSTALL_DEPS_OPTS}

      ;;

    reqfile)

      if [[ -f "${PYTHON_REQS_FILE}" ]]; then
@@ -1006,7 +1027,7 @@ variables: 
        for extrareqsfile in $(echo "$found_reqs_files"); do

          log_info "--- installing extra requirements from \\e[33;1m${extrareqsfile}\\e[0m"

          # shellcheck disable=SC2086

          pip install ${TRACE+--verbose} ${PIP_OPTS} -r "${extrareqsfile}"

          pip install ${TRACE+--verbose} ${PIP_OPTS} -r "${extrareqsfile}" ${PIP_INSTALL_DEPS_OPTS}

        done

      else

        log_warn "--- requirements build system defined, but no ${PYTHON_REQS_FILE} file found"
@@ -1017,15 +1038,26 @@ variables: 
        log_warn "Using uv but \\e[33;1muv.lock\\e[0m file not found: you shall commit it with your project files"

      fi

      maybe_install_build_system

      uv sync ${TRACE+--verbose} --frozen ${PYTHON_EXTRA_DEPS:+--extra "$PYTHON_EXTRA_DEPS"}

      extra_deps=""

      for group in ${PYTHON_EXTRA_DEPS}; do

        extra_deps="${extra_deps} --extra $group"

      done

      dep_groups=""

      for group in ${PYTHON_DEP_GROUPS}; do

        dep_groups="${dep_groups} --group $group"

      done

      # shellcheck disable=SC2086

      uv sync ${TRACE+--verbose} --frozen ${extra_deps} ${dep_groups} ${PYTHON_INSTALL_DEPS_OPTS}

      ;;

    hatch*)

      maybe_install_build_system

      hatch env create ${TRACE+--verbose}

      # shellcheck disable=SC2086

      hatch env create ${TRACE+--verbose} ${PYTHON_INSTALL_DEPS_OPTS}

      ;;

    pdm*)

      maybe_install_build_system

      pdm sync ${TRACE+--verbose} --dev ${PYTHON_EXTRA_DEPS:+--extra "$PYTHON_EXTRA_DEPS"}

      # shellcheck disable=SC2086

      pdm sync ${TRACE+--verbose} --dev ${PYTHON_DEP_GROUPS:+--group $PYTHON_DEP_GROUPS} ${PYTHON_INSTALL_DEPS_OPTS}

      ;;

    esac