Skip to content

UNIL S3 Bucket

Data transfer from the University of Lausanne (UNIL) is now performed using Simple Storage Service (S3).

Before using the service, the principal investigator (PI) of the project must complete the account registration process. UNIL will provide the necessary information and instructions by email.

Once the account has been registered, data can be transferred using rclone, which is available on Merlin 7. To verify that rclone is installed, run:

Bash
rclone --version

UNIL will provide the specific rclone configuration and commands required to access the S3 storage. The Merlin 7 login nodes are already configured to support this service.

Example commands

After configuring rclone with the credentials provided by UNIL, the configuration file should be located at:

Text Only
~/.config/rclone/rclone.conf

You can then use the following commands.

List files in the bucket

To check the connection and list the directories available in the bucket:

Bash
rclone -vvP lsd S3-UNIL-RW:

Copy data from the S3 bucket

To copy data from the S3 bucket to a local directory:

Bash
rclone -vvP copy S3-UNIL-RW:Test/ IntoMyFolder/IntoMySubFolder/ --s3-nocheck-bucket

Command-line options

  • -vv enables verbose logging, which is useful for troubleshooting.
  • -P displays transfer progress.
  • --s3-nocheck-bucket disables the bucket existence check. This may be required for some S3 configurations.

Access credentials

Make sure that your rclone configuration contains the correct access credentials provided by UNIL. Do not share your rclone.conf file, as it may contain sensitive credentials.

Test the connection first

Before starting a large data transfer, use rclone lsd to verify that your configuration is working and that you can access the expected bucket.