csm-data s3-bucket-load¶
Description
Use csm-data to connect to an S3 bucket and download its content. Requires a valid S3 configuration, here using environment variables.
Command
csm-data s3-bucket-load --target-folder $CSM_DATASET_ABSOLUTE_PATH --bucket-name $CSM_DATA_BUCKET_NAME
Defined Environment
Variable Name | Description |
---|---|
CSM_DATASET_ABSOLUTE_PATH | Target folder in which to write the content of the S3 Bucket |
CSM_DATA_BUCKET_NAME | Name of the bucket in the S3 instance to download |
AWS_ENDPOINT_URL | URI required to connect to a given S3 instance |
AWS_ACCESS_KEY_ID | User access key required to authenticate to the S3 instance |
AWS_SECRET_ACCESS_KEY | User secret required to authenticate to the S3 instance |
Will be passed all system Environment Variables
as well as the defined Environment
csm-data s3-bucket-load json file
{
"id":"csm-data s3-bucket-load",
"command":"csm-data",
"arguments":[
"s3-bucket-load",
"--target-folder",
"$CSM_DATASET_ABSOLUTE_PATH",
"--bucket-name",
"$CSM_DATA_BUCKET_NAME"
],
"environment":{
"CSM_DATASET_ABSOLUTE_PATH":{
"description":"Target folder in which to write the content of the S3 Bucket"
},
"CSM_DATA_BUCKET_NAME":{
"description":"Name of the bucket in the S3 instance to download"
},
"AWS_ENDPOINT_URL":{
"description":"URI required to connect to a given S3 instance"
},
"AWS_ACCESS_KEY_ID":{
"description":"User access key required to authenticate to the S3 instance"
},
"AWS_SECRET_ACCESS_KEY":{
"description":"User secret required to authenticate to the S3 instance"
}
},
"description":"Use csm-data to connect to an S3 bucket and download its content. Requires a valid S3 configuration, here using environment variables.",
"useSystemEnvironment":true
}