New posts are disabled while we improve the user experience.

You can browse the site, or for urgent issues, raise a query at MyAccount.

question

Upvotes
Accepted
43 3 4 5

DataScopeSelect: RawTickExtraction, MarketDepthExtraction not working

logResponseMarketDepthExtraction.txtlogResponseRawTickExtraction.txt


Hi there, Im trying to extract tick data with DataScopeSelect .net library.

Im using Rics I previously found in SearchChainResolution function and passed it to the extraction functions.


As response I get an ExtractionId. But when I try to read the stream, I wont receive any payload. It ends the extraction after some time.

I ve attached the logs


here is my code to review the streamresponse:


var response = await _extractionsContext.ExtractRawAsync(request).ConfigureAwait(false);

var result = new StringBuilder();

var streamResponse = await _extractionsContext.GetReadStreamAsync(response).ConfigureAwait(false);

using (var reader = new StreamReader(streamResponse.Stream))

{

var resultStr = await reader.ReadLineAsync().ConfigureAwait(false);

while (!string.IsNullOrEmpty(resultStr))

{

result.AppendLine(resultStr);

resultStr = await reader.ReadLineAsync().ConfigureAwait(false);

}

}

Any help is appreciated

Thanks a lot!

dss-rest-apidatascope-selectdsson-demand-extraction
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
11.3k 26 9 14

@marko.drews,

The query range can be adjusted using the QueryStartDate and QueryEndDate parameter. From the log, the QueryEndDate was set to "2022-12-29T00:00:00.0000000+00:00". Normally, the data is available up to the previous day's. Current day's data will be available 2 hours after market closed.

With regard to the quota, you can check your quota limit and usage on the Quota Usage page. The quota depends on data usage. I would suggest you contact the DSS support via MyRefinitiv for more information about your quota limit.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvote
11.3k 26 9 14

Hi @marko.drews,

I have verified the MarketDepth extraction log file and found the note information related to embargo. The extraction for the RICs; F1BYF1 - F1BYF6 was not generated because the query date range is on the embargo time of RICs. You may adjust the query range to not after 09:30:00 GMT.

(RIC,F1BYF1,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details. 
(RIC,F1BYF2,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details. 
(RIC,F1BYF3,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details. 
(RIC,F1BYF4,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details. 
(RIC,F1BYF5,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details. 
(RIC,F1BYF6,EEX) Some data suppressed for release cycle(s) U1, U2, U3, U4, PE 634.  Request occurred during embargo.  Data currently available through 05/15/2020 09:30:00.  See Release Cycle schedule in the user documentation for details.

For the Raw extraction, I have not seen any logs regarding extraction request and result. Could you reproduce the issue and provide the entire log file?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
43 3 4 5

Hi Veerapath,

thanks for your reply. I didn't know there is an embargo on data. How do I adjust the query range.

I presume its defined in the extraction condition:

Condition = new TickHistoryMarketDepthCondition
{
ReportDateRangeType = ReportDateRangeType.Range,
QueryStartDate = start,
QueryEndDate = end,
NumberOfLevels = 2,
ExtractBy = TickHistoryExtractByMode.Ric,
MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,
SortBy = TickHistorySort.SingleByRic,
View = TickHistoryMarketDepthViewOptions.NormalizedLL2,
DisplaySourceRIC = true
}

I tried another query on a different root Ric, which is DEBM.

the response gave me following response


Extraction ID: 2000000149837714
Schedule: 0x0719aa0cf9a19071 (ID = 0x0000000000000000)
Input List (47 items): (ID = 0x0719aa0cf9a19071) Created: 05/18/2020 13:54:51 Last Modified: 05/18/2020 13:54:51
Report Template (6 fields): _OnD_0x0719aa0cf9a19071 (ID = 0x0719aa0cf9d19071) Created: 05/18/2020 13:52:16 Last Modified: 05/18/2020 13:52:16
Schedule dispatched via message queue (0x0719aa0cf9a19071), Data source identifier (C8516953C2F3498A811D555078929FCC)
Schedule Time: 05/18/2020 13:52:17
Processing started at 05/18/2020 13:52:18
Processing completed successfully at 05/18/2020 13:54:51
Extraction finished at 05/18/2020 13:54:51 UTC, with servers: tm04n01, TRTH (126.547 secs)
Historical Instrument <RIC,DEBMF0> expanded to 1 RIC: DEBMF0.
Instrument <RIC,DEBMF1> expanded to 1 RIC: DEBMF1.
Historical Instrument <RIC,DEBMF8> expanded to 1 RIC: DEBMF8.
Historical Instrument <RIC,DEBMF9> expanded to 1 RIC: DEBMF9.
Historical Instrument <RIC,DEBMG0> expanded to 1 RIC: DEBMG0.
Instrument <RIC,DEBMG1> expanded to 1 RIC: DEBMG1.
Historical Instrument <RIC,DEBMG8> expanded to 1 RIC: DEBMG8.
Historical Instrument <RIC,DEBMG9> expanded to 1 RIC: DEBMG9.
Historical Instrument <RIC,DEBMH0> expanded to 1 RIC: DEBMH0.
Historical Instrument <RIC,DEBMH8> expanded to 1 RIC: DEBMH8.
Historical Instrument <RIC,DEBMH9> expanded to 1 RIC: DEBMH9.
Historical Instrument <RIC,DEBMJ0> expanded to 1 RIC: DEBMJ0.
Historical Instrument <RIC,DEBMJ7> expanded to 1 RIC: DEBMJ7.
Historical Instrument <RIC,DEBMJ8> expanded to 1 RIC: DEBMJ8.
Historical Instrument <RIC,DEBMJ9> expanded to 1 RIC: DEBMJ9.
Instrument <RIC,DEBMK0> expanded to 1 RIC: DEBMK0.
Historical Instrument <RIC,DEBMK7> expanded to 1 RIC: DEBMK7.
Historical Instrument <RIC,DEBMK8> expanded to 1 RIC: DEBMK8.
Historical Instrument <RIC,DEBMK9> expanded to 1 RIC: DEBMK9.
Instrument <RIC,DEBMM0> expanded to 1 RIC: DEBMM0.
Historical Instrument <RIC,DEBMM7> expanded to 1 RIC: DEBMM7.
Historical Instrument <RIC,DEBMM8> expanded to 1 RIC: DEBMM8.
Historical Instrument <RIC,DEBMM9> expanded to 1 RIC: DEBMM9.
Instrument <RIC,DEBMN0> expanded to 1 RIC: DEBMN0.
Historical Instrument <RIC,DEBMN7> expanded to 1 RIC: DEBMN7.
Historical Instrument <RIC,DEBMN8> expanded to 1 RIC: DEBMN8.
Historical Instrument <RIC,DEBMN9> expanded to 1 RIC: DEBMN9.
Instrument <RIC,DEBMQ0> expanded to 1 RIC: DEBMQ0.
Historical Instrument <RIC,DEBMQ7> expanded to 1 RIC: DEBMQ7.
Historical Instrument <RIC,DEBMQ8> expanded to 1 RIC: DEBMQ8.
Historical Instrument <RIC,DEBMQ9> expanded to 1 RIC: DEBMQ9.
Instrument <RIC,DEBMU0> expanded to 1 RIC: DEBMU0.
Historical Instrument <RIC,DEBMU7> expanded to 1 RIC: DEBMU7.
Historical Instrument <RIC,DEBMU8> expanded to 1 RIC: DEBMU8.
Historical Instrument <RIC,DEBMU9> expanded to 1 RIC: DEBMU9.
Instrument <RIC,DEBMV0> expanded to 1 RIC: DEBMV0.
Historical Instrument <RIC,DEBMV7> expanded to 1 RIC: DEBMV7.
Historical Instrument <RIC,DEBMV8> expanded to 1 RIC: DEBMV8.
Historical Instrument <RIC,DEBMV9> expanded to 1 RIC: DEBMV9.
Instrument <RIC,DEBMX0> expanded to 1 RIC: DEBMX0.
Historical Instrument <RIC,DEBMX7> expanded to 1 RIC: DEBMX7.
Historical Instrument <RIC,DEBMX8> expanded to 1 RIC: DEBMX8.
Historical Instrument <RIC,DEBMX9> expanded to 1 RIC: DEBMX9.
Instrument <RIC,DEBMZ0> expanded to 1 RIC: DEBMZ0.
Historical Instrument <RIC,DEBMZ7> expanded to 1 RIC: DEBMZ7.
Historical Instrument <RIC,DEBMZ8> expanded to 1 RIC: DEBMZ8.
Historical Instrument <RIC,DEBMZ9> expanded to 1 RIC: DEBMZ9.
Total instruments after instrument expansion = 47
Quota Message: INFO: Tick History Futures Quota Count Before Extraction: 0; Instruments Approved for Extraction: 0; Tick History Futures Quota Count After Extraction: 0, 100% of Limit; Tick History Futures Quota Limit: 0
Quota Message: ERROR: The RIC 'DEBM' in the request would exceed your quota limits. Adjust your input list to continue.
Quota Message: WARNING: Tick History Futures Quota has been reached or exceeded


It seems, my quota limit was exceeded. Where can I check my quote limits?

Can I request the tick history in batches then? How can I still fetch the required data?


Thanks a lot!



icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.