Read and Write Data

Read and Write Data

Once you have an allocation and have created a bucket (or, someone sent you keys or an open access endpoint ), you’re ready to read or write data.

Dataset types

OSN buckets can be either open access or protected:

  • Open access buckets are readable by anyone without credentials (“anonymous access”). Writing new data still requires keys.
  • Protected buckets require keys to both read and write data.

Protected datasets have two sets of keys. One set allows reading and writing to the dataset while the second set of keys only provides read access. Anonymous access is not allowed on protected datasets. For information on sharing credentials, see the bucket documentation page.

Connecting to OSN buckets

OSN supports a RESTful API that is compatible with the basic data access model of the Amazon S3 API. Any software that complies with that API can access data stored on the OSN.

Here, we describe several popular tools in detail. However, there are many more commercial and open source software tools for moving files to and from S3 buckets. OSN buckets are typically compatible with any tool that supports AWS S3.

info

Do you have a favorite tool for using OSN that we don’t mention here? Let us know at help@osn.mghpcc.org!

Before you begin

All S3 access methods require

  1. The bucket endpoint
  2. The bucket name
  3. For protected datasets, either the RW Access Key/Secret pair or the RO Access Key/Secret pair

To find this info, see the bucket attribute descriptions.

Graphical applications

Command line applications