csm-data s3-bucket-load

Description

Use csm-data to connect to an S3 bucket and download its content. Requires a valid S3 configuration, here using environment variables.

Command

csm-data s3-bucket-load --target-folder $CSM_DATASET_ABSOLUTE_PATH --bucket-name $CSM_DATA_BUCKET_NAME

Defined Environment

Variable Name	Description
CSM_DATASET_ABSOLUTE_PATH	Target folder in which to write the content of the S3 Bucket
CSM_DATA_BUCKET_NAME	Name of the bucket in the S3 instance to download
AWS_ENDPOINT_URL	URI required to connect to a given S3 instance
AWS_ACCESS_KEY_ID	User access key required to authenticate to the S3 instance
AWS_SECRET_ACCESS_KEY	User secret required to authenticate to the S3 instance

Will be passed all system Environment Variables as well as the defined Environment

csm-data s3-bucket-load json file

{
  "id":"csm-data s3-bucket-load",
  "command":"csm-data",
  "arguments":[
    "s3-bucket-load",
    "--target-folder",
    "$CSM_DATASET_ABSOLUTE_PATH",
    "--bucket-name",
    "$CSM_DATA_BUCKET_NAME"
  ],
  "environment":{
    "CSM_DATASET_ABSOLUTE_PATH":{
      "description":"Target folder in which to write the content of the S3 Bucket"
    },
    "CSM_DATA_BUCKET_NAME":{
      "description":"Name of the bucket in the S3 instance to download"
    },
    "AWS_ENDPOINT_URL":{
      "description":"URI required to connect to a given S3 instance"
    },
    "AWS_ACCESS_KEY_ID":{
      "description":"User access key required to authenticate to the S3 instance"
    },
    "AWS_SECRET_ACCESS_KEY":{
      "description":"User secret required to authenticate to the S3 instance"
    }
  },
  "description":"Use csm-data to connect to an S3 bucket and download its content. Requires a valid S3 configuration, here using environment variables.",
  "useSystemEnvironment":true
}