Refinitiv Data Platform - CFS - S3 native download

We are using CFS via the Refinitiv Data Platform to download a custom Tick History extraction. We can access the files through api.refinitiv.com and providing the token as a header, as in the [Python examples](https://github.com/Refinitiv-API-Samples/Article.RDP.Python.IntroToCFS/blob/main/CFSIntro.ipynb).

However we wish to download the files natively in AWS, so that we can transfer them directly to our own S3 bucket without going out to the web. E.g. `aws s3 cp s3://refinitiv-bucket/.../file.csv.gz s3://our-bucket/file.csv.gz`

The file API exposes the url for each file, such as

`https://s3-us-east-1.amazonaws.com/a.../..../normalised/..../data/merged/merged-Report.csv.gz`

Can we access this file directly in AWS?

Find more posts tagged with

aws

rdp

rdp-api

#technology

Accepted answers

Jirapongse

@Edward Richards

I got the response from the product team that the direct access to the file in the bucket is not allowed.

Howover, you can contact the helpdesk team directly via MyRefinitiv to confirm this.

All comments

wasin.w

Hello @Edward Richards

Unfortunately, the RDP Bulk file does not expose the S3 location. However, you can download the API from RDP API /file-store/v1/files/{id}/stream service directly by calling the API without "?doNotRedirect=true" parameter

Example:

https://api.refinitiv.com/file-store/v1/files/{fileId}/stream?doNotRedirect=true

Result: Return the expose public HTTP URL Like "https://s3-us-east-1.amazonaws.com/a.../..../normalised/..../data/merged/merged-Report.csv.gz"

https://api.refinitiv.com/file-store/v1/files/{fileId}/stream

Result: Download the file directly.

I hope this information helps.

Jirapongse

@Edward Richards

I think the Refinitiv Data Platform - CFS returns a presigned URL. You can use it to download the file.

However, according to this discussion, you can't copy it directly to another S3 bucket.

Edward Richards

Thank you both.

This is surprising, why does the RDP CFS files API (https://api.refinitiv.com/file-store/v1/files?filesetId=) give information about the S3 location, such as S3 URL and AWS ARN, if accessing it natively in AWS isn't supported?

Jirapongse

@Edward Richards

I got the response from the product team that the direct access to the file in the bucket is not allowed.

Howover, you can contact the helpdesk team directly via MyRefinitiv to confirm this.

EXPLORE OUR SITES