Download multiple extracted files simultaneously

WDS Team

Hello, I have established scheduled extractions via DSS GUI and would like to download multiple files simultaneously using the REST API. It appears that the URL below can recall data from one file. I would like to recall the data on around 100 files.

https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractedFiles('{{ExtractedFileId}}')/$value

eg, a monthly download of data comprising 4 data files per business day.

Can someone please provide some guidance on the best way to achieve this. FYI - I have about 4 hours of coding experience.

I am using Postman

zoya faberov

Hello @WDS Team,

Glad to suggest how to get up to speed on DSS REST API and to design a programmatic integration that is robust. Probably we can not encompass everything in one answer, so consider this a starting point.

DSS Quickstart Guide is built around DSS .NET SDK. If you are looking to integrate using .NET this is an excellent starting point.

If you are looking to integrate using another language, for example Python or Java, I suggest to get started with out targeted introductory tutorials:

Refinitiv DataScope Select - REST API -> Tutorials Introduction

and

Programming without SDK Tutorial

My understanding from your question is, that you are not looking to pursue a very popular On Demand integration approach that is described at high level in intros, that you are interested in GUI emulation, and specifically, in extracted file retrieval with this approach.

My understanding also is that you have downloaded, imported into your Postman and are looking at our DSS REST Tutorials Postman collection.

If this is the case, next suggest to review REST API Tutorial 10: GUI control calls: immediate extract, especially the second half with checking the status of the extractions and retrieving extracted files should be what you are looking to do, described in brief.

Going back to your specific question.

In a typical scenario, multiple separate scheduled extractions would complete at different times, asynchronously, we check the status shortly after the expected time, and once the specific extraction is complete, we retrieve the result.

If in your case all 100 results should be ready at the same time, the simplest approach is to sequentially go over the list of the extraction Ids, check the status, and if complete- get the data, the result files per extraction IDs.

To design the integration efficiently to your requirement, you may also wish to review REST API Reference Tree -> Report Extraction section of the spec. The methods described in this section, GetByScheduleID, GetCompleted and GetCompletedByDateRange can be of help with rounding up the specific extraction results that you require.

Let us know if this information is of help, or you are looking for something different?

Jirapongse

@WDS Team

You can also refer to the DSS REST API: Downloading Completed Extraction Files by a Schedule Name article which explains the steps to use the DSS REST API to download extraction files created by scheduled extractions. There are several samples in Postman, Python, C#, and Dotnet core. However, the examples download the latest file from the scheduled extraction.

I have created a new Python example that can load multiple files simultaneously by using the method mentioned on StackOverflow. The example is on GitHub. You can run it with these parameters

[-u dss_username] [-p dss_password] [-s schedule_name] [-n number of files (1)]

You can specify the number of downloaded files in the -n parameter. It will download files in the order that appears in the response of the Extractions/ReportExtractionGetCompletedByScheduleId(ScheduleId='<ScheduleId>') endpoint.