dump-to-s3¶
Help command
> csm-data store dump-to-s3 --help
Usage: csm-data store dump-to-s3 [OPTIONS]
Dump a datastore to a S3
Will upload everything from a given data store to a S3 bucket.
3 modes currently exists :
• sqlite : will dump the data store underlying database as is
• csv : will convert every table of the datastore to csv and send them as
separate files
• parquet : will convert every table of the datastore to parquet and send
them as separate files
Giving a prefix will add it to every upload (finishing the prefix with a "/"
will allow to upload in a folder inside the bucket)
Make use of the boto3 library to access the bucket
More information is available on this page:
d=831099;https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html\https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.ht]8;;\
d=831099;https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html\ml]8;;\
╭─ OPTIONS ────────────────────────────────────────────────────────────────────╮
│ * --store-folder PATH The folder containing the │
│ store files │
│ ENV: │
│ CSM_PARAMETERS_ABSOLUTE_PATH │
│ [required] │
│ --output-type [sqlite|csv|parquet] Choose the type of file │
│ output to use │
│ * --bucket-name BUCKET The bucket on S3 to upload to │
│ ENV: CSM_DATA_BUCKET_NAME │
│ [required] │
│ --prefix PREFIX A prefix by which all │
│ uploaded files should start │
│ with in the bucket │
│ ENV: CSM_DATA_BUCKET_PREFIX │
│ --use-ssl/--no-ssl Use SSL to secure connection │
│ to S3 │
│ * --s3-url URL URL to connect to the S3 │
│ system │
│ ENV: AWS_ENDPOINT_URL │
│ [required] │
│ * --access-id ID Identity used to connect to │
│ the S3 system │
│ ENV: AWS_ACCESS_KEY_ID │
│ [required] │
│ * --secret-key ID Secret tied to the ID used to │
│ connect to the S3 system │
│ ENV: AWS_SECRET_ACCESS_KEY │
│ [required] │
│ --ssl-cert-bundle PATH Path to an alternate CA │
│ Bundle to validate SSL │
│ connections │
│ ENV: CSM_S3_CA_BUNDLE │
│ --web-help Open the web documentation │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯