Commit 5cf3a8cc authored by GridexX's avatar GridexX
Browse files

docs(aws s3): List README, text & new variable, ACL modified

parent 09ec0fdf

jobs/aws_s3_sync/README.md

+9 −4
Original line number Diff line number Diff line
@@ -9,15 +9,15 @@ This job uses the AWS CLI (v1 for now) in order to sync files between a director 
    ```yaml

      - remote: 'https://jobs.r2devops.io/latest/aws_s3_sync.yml'

    ```

2. Fill the variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` or `AWS_DEFAULT_REGION` with yours (see the [documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html){:target="_blank"} ).

1. Set your crendentials variables in the Gitlab CI/CD variables section of your project (see the [S3 documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html){:target="_blank"} about credentials) .



    ??? summary "Need a custom Endpoint ⚓ ?"

      Just type your custom url in the variable `AWS_ENDPOINT`

      For example, you can find the custom endpoint url for Scaleway [here](https://www.scaleway.com/en/docs/storage/object/api-cli/object-storage-aws-cli/){:target="_blank"}.



3. If you need to customize the job (stage, variables, ...) 👉 check the [jobs

1. If you need to customize the job (stage, variables, ...) 👉 check the [jobs

   customization](/use-the-hub/#jobs-customization)

4. Well done, your job is ready to work ! 😀

1. Well done, your job is ready to work ! 😀



## Job details

* Job name: `aws_s3_sync`
@@ -35,4 +35,9 @@ This job uses the AWS CLI (v1 for now) in order to sync files between a director 
| `AWS_ACL` | If you want to add an ACL (e.g. `public-read`) | ` ` |

| `SYNC_DIR` | Directory to sync | `build` |

| `BUCKET_NAME`| The name of the bucket | ` ` |

| `ADDITIONAL_OPTIONS` | Additional options to the CLI | ` ` |
 
 No newline at end of file
 
| `S3_SIGNATURE_VERSION` | S3 signature version | `s3v4` |

| `S3_MAX_CONCURRENT_REQUESTS` | The maximum concurrent requests' number | `100` |

| `S3_MAX_QUEUE_SIZE` | The maximum queue's size | `1000` |

| `S3_MULTIPART_THRESHOLD` | The multipart threshold (MB at end) | `50MB` |

| `S3_MULTIPART_CHUNCKSIZE` | The multipart chuncksize (MB at end) | `10MB` |

| `S3API_ENDPOINTURL` | The S3 endpoint URL | ` ` |
 
 No newline at end of file

jobs/aws_s3_sync/aws_s3_sync.yml

+20 −3
Original line number Diff line number Diff line
@@ -17,6 +17,12 @@ aws_s3_sync: 
    BUCKET_NAME: " "

    AWS_ACL: " "



    S3_SIGNATURE_VERSION: "s3v4"

    S3_MAX_CONCURRENT_REQUESTS: "100"

    S3_MAX_QUEUE_SIZE: "1000"

    S3_MULTIPART_THRESHOLD: "50MB"

    S3_MULTIPART_CHUNCKSIZE: "10MB"

    S3API_ENDPOINTURL: " "



  script:
@@ -33,9 +39,20 @@ aws_s3_sync: 
    -   aws configure set default.s3.endpoint_url ${AWS_ENDPOINT}

    - fi



    #update the ACL with the AWS json input

    - if [ ! -z ${AWS_ACL} ]; then

    -   aws s3api get-bucket-acl --bucket ${BUCKET_NAME} --cli-input-json ${AWS_ACL}

    - aws configure set default.s3.signature_version ${S3_SIGNATURE_VERSION}

    - aws configure set default.s3.max_concurrent_requests ${S3_MAX_CONCURRENT_REQUESTS}

    - aws configure set default.s3.max_queue_size ${S3_MAX_QUEUE_SIZE}

    - aws configure set default.s3.multipart_threshold ${S3_MULTIPART_THRESHOLD}

    - aws configure set default.s3.multipart_chunksize ${S3_MULTIPART_CHUNCKSIZE}



    - if [ ! -z ${S3API_ENDPOINTURL} ]; then

    -   aws configure set default.s3api.endpointurl ${S3API_ENDPOINTURL}

    - fi





    #update the ACL

    - if [ ! -z "$AWS_ACL" ]; then

    -   aws s3api put-bucket-acl $AWS_ACL

    - fi



    #sync to the specified bucket