Case 14545631: Why are we missing trade data for <.OQ> RICs?

narya
narya Newcomer
edited April 4 in TRTH

Hi Team - I have download the Raw data for 28 Feb 2025 but in manifest file we can see the data is download. After download the I unzip the using command..
gunzip trth_20250228_eod.gz after unzip the command, I run the below command
onetick@tstcapp008:/apps/OMD/raw_data/equity_tr/eod_raw> cat trth_20250228_eod | grep -i ABCL.0Q | grep -i trade | we -l
0
onetick@tstcapp008:/apps/OMD/raw_data/equity_tr/eod_raw> cat trth_20250228_eod.old | grep -i ABCL.0Q | grep - i trade | we - l
0

I already raised the request on LSEG Support and whole email chain with case id is present.
Can you please answer it while we are able to load as well as see the data after downloading the data from datascope?

Answers

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @narya

    Thank you for reaching to us.

    The RIC name should be ABCL.OQ. It is the 'O' character. However, the item that you used in the grep command is ABCL.0Q. It is the zero (0) character, not the 'O' character.

    image.png

    You may use the Extraction ID to check the size of the downloaded file.

    Extraction ID: 2000000879815096
    

    The endpoint is:

    https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ReportExtractions('2000000879815096')/Files
    

    This endpoint will return the files' information including file sizes. For example:

            {
    "ExtractedFileId": "VjF8MHgwOTU1OTU2Y2RiYWFmNmQ5fA",
    "ReportExtractionId": "2000000891162848",
    "ScheduleId": "0x0956e7e72e8af95b",
    "FileType": "Full",
    "ExtractedFileName": "_OnD_0x0956e7e72e8af95b.csv",
    "LastWriteTimeUtc": "2025-04-04T05:28:00.546Z",
    "ContentsExists": true,
    "Size": 446,
    "ReceivedDateUtc": "2025-04-04T05:28:00.576Z"
    },
  • narya
    narya Newcomer

    My apology for spell mistake, please see the snippet below I used the correct character while checking it..

    image.png
  • Jirapongse
    Jirapongse ✭✭✭✭✭

    The output shows that some files have data for ABCL.OQ.

    image.png

    I don't have permission to access other extractions. You may need to record the Notes and downloaded file sizes.

    I can check the request message. Please share the request message you are using. I can test it and verify if the data contains ABCL.OQ.

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    Typically, you need to download gzip files and then uncompress them on the machine.

    If you download the files with the auto decompression feature, the content may be incomplete.

  • narya
    narya Newcomer

    We uncompress it on our machine we didn't use any auto decompression feature.

    when you say "Please share the request message you are using". what does it mean?
    you meant to say manifestation file?

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    It is a request body message sent to the Extractions/ExtractRaw endpoint. For example:

    image.png
  • narya
    narya Newcomer

    We don't use any template to download the data.. we download the whole raw data..

    Below is the extraction summary logs

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @narya

    You need to send an on-demand request to extract data. Please check the Tick History Tutorials.

    Did you compare the downloaded file size and the file size from this endpoint?

    https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ReportExtractions('2000000879815096')/Files
    
  • narya
    narya Newcomer

    Hi @Jirapongse

    We do send on-demand requests to extract data from Data Scope. Unfortunately, we don't have the file needed to compare the data for February 28th.

    Just curious to know - What we are trying to check here from file size?

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    I need to make sure the downloaded file size is equal to the file size shows in the endpoint.

    https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ReportExtractions('2000000879815096')/Files
    

    If they are equal, you need to contact the product team to verify it.

  • narya
    narya Newcomer

    Before created the ticket here, I contacted with the product team, please find the below…

    Hi Navneet, 
    With regards to your follow up questions, I would like to highlight that Job ID is basically extraction ID (alphanumeric code) which we get when the output data is ready. And using this Job ID, we can run a API request to download the output data. Whenever there are any issues with the Tick History extractions, we usually ask the notes file which includes the JOB ID. This Job id helps our development team to investigate at the backend if there were some issues with the extractions. However with regards to your issue, I dont think the Job id is an issue here as you were able to download the data correctly and even the notes file says that the request was completed successfully. And you are observing the issue only when you try to run command on the downloaded data, you are unable to get the trade updates on some of the RICs including RIC <ABCL.OQ>. I have further checked this internally with my team and would like to let you know that our team will be able to assist only if there are any issues with the extraction or the output data. But here since the extraction is completed successfully and output data is also available, we will not be able to assist further as the issue could be with your command on how you run a request on the downloaded data. To add further, when we do a extraction for RIC <ABCL.OQ> for 28th Feb 2025, I was able to get the same count of records (35465) in my notes file as seen in your notes file. And when I open the downloaded data, I could see trade updates
    for RIC <ABCL.OQ>. Below screenshot for your reference: Details from my notes file: 

    image.png

     Details from your notes file: 

    image.png

     This indicates that there is no issue in the data or the extraction. I would suggest you to post this query on our developer portal in Q&A section where one of our developers would review your command used to query the downloads and provide inputs on why trade updates were not available or would help to amend the details in your command. LSEG Tick History - REST API | Devportal Kindly let me know if you have any additional questions. 
    Regards,Shyamala KLead Specialist - Tick HistoryLSEG

    Thanks
    Navneet Arya

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    That is why I asked for the on-demand request that you are using and the file size's checking.