Get CFS File with ESG Data

Adrian Marti · November 2019

Hi,

Im on the API Playground where I can see all my API's. My goal is to download a file that contains all ESG data for all supported assets. I figured out that I can download a file from CFS containing the data.

On the API Playground I can only choose

/file-store/beta1/files/{id}
/file-store/beta1/files/{id}/stream

But where can I get the id for a particular file from? Should I have access to the bucket?

Best,

Adrian

Gurpreet · November 2019

Hi @Adrian Marti Please see this bulk ESG CFS tutorial using Postman. The setup instructions for Postman and the tutorial source code is also available in the downloads tab.

Adrian Marti · November 2019

Gurpreet,

thank you for your answer. That helped a lot. I tried to GET the file on the url

https://api.refinitiv.com/file-store/beta1/files/60da5c1c-8129-41e3-bdee-a7b706ebe2ca

but then I received an HTTP 404 error.

{  "error": {    "id": "64822c5c-fd20-4e9e-b56d-ef56804b66e5",    "code": "EDS.CFS.NoFileFound",    "status": 404,    "message": "No resource found for: ResourceType=File; ResourceId=60da5c1c-8129-41e3-bdee-a7b706ebe2ca"  } }

Gurpreet · November 2019

@Adrian Marti Seems like you are trying to use the id parameter in the JSON to request the file. The file ID is contained in the JSON array key name: files.

E.g:

https://api.refinitiv.com/file-store/beta1/files/400d-f258-5856e077-97f6-97e3f197c1ee

{
  "id": "400d-****-5856e077-97f6-97e3f197c1ee",
  "filename": "esg-esg-scores-init-109.json.gz",
  "filesetId": "4010-80e2-****-acb5-7c8d4cf6063d",
  "fileType": "init",
  "description": "File Generated from Bulk",
  "storageLocation": {
    "url": "https://205941-eds-bulk-ondemand-avro-production.s3.amazonaws.com/60e0b0a9-*******-62f9af1031c7_71da94c03a974820bde5e2c00a3c0162.json.gz",
    "@type&quot;: "S3"
  },
  "createDateTime": "2019-11-28T08:57:20Z",
  "updateDateTime": "2019-11-28T08:57:20Z",
  "href": "https://api.refinitiv.com/file-store/beta1/files/400d-****-5856e077-97f6-97e3f197c1ee/stream",
  "fileSizeInBytes": 0
}

Adrian Marti · December 2019

Gupreet,

but where can I get the information from? I cannot download any file that contains a JSON key "files" and in the file EDP.post_collection.json is no key named "files".

I can only access a file directly by {id} but as long as I don't know which id, I'm not able to download.

Is it correct, the JSON key "files" is in a bucket? But as I cannot GET a bucket, I cannot access the information files.

Best,

Adrian

Christiaan Meihsl · December 2019

Hi @Adrian Marti,

As described near the end of the bulk ESG CFS tutorial, there are basically 3 steps to follow to retrieve the bulk ESG files:

1. Retrieve the latest file-set(s) from the bulk ESG bucket:

GET https://api.refinitiv.com/file-store/beta1/file-sets?bucket=EDS-BULK-ESG-Production&availableFrom=2019-12-01T00:00:00Z

You can change the availableFrom date depending on when you run the call, and which file sets you want to retrieve.

2. For each new file-set, retrieve all the file identifiers and file names:

GET https://api.refinitiv.com/file-store/beta1/files?filesetId=<file-set id>

The <file-set-id> value is taken from the output of the 1st call.

For each record in the response, save the filename and its corresponding file id.

3. Download and save each file:

Use the file id as a parameter in the call:

GET https://api.refinitiv.com/file-store/beta1/files/<file id>/stream

Save the file using the filename received in the file metadata.

That is all there is to it, hope this helps :-)

Adrian Marti · December 2019

Christiaan,

thank you for your answer and that was exactly what I've expected that I haqve to do. Nevertheless, the problem still is, that I do not have the permission to perform Step 1 & Step 2.

Can you please provide me with the access to those methods?

Best,

Adrian

Christiaan Meihsl · December 2019

https://community.developers.refinitiv.com/discussion/comment/52521#Comment_52521

@Adrian Marti,

All these APIs are fee liable products, and access is delivered based on your subscription. For access to functionality and permissioning issues, you need to talk to your account manager.

Christiaan Meihsl · December 2019

@Adrian Marti,

After internal investigation I found out that you are permissioned for ESG data on demand, not in bulk. This is an important difference, the API calls are not the same. That is why you cannot run the API calls on CFS.

The on-demand ESG API calls you are permissioned for are documented in the ESG user guide, as well as the ESG FAQ. The calls are also described and illustrated in this ESG data Postman tutorial, and in this ESG data Python tutorial (the code for the tutorials is available in the downloads section of the portal).

I hope this helps solve the issues you have been facing. Please tell us if that is the case, or if any further help is required.

Kind regards,

Christiaan

Christiaan Meihsl · December 2019

https://community.developers.refinitiv.com/discussion/comment/52686#Comment_52686

As an additional comment, please note that ESG bulk data will be made available in Q2 2020.

Get CFS File with ESG Data

Best Answer

Answers

Categories

EXPLORE OUR SITES

Get CFS File with ESG Data

Best Answer

Answers

Categories