question

Upvotes
Accepted
22 3 3 10

Tick history raw data extract is not sort in timestamp

Hi,

I have a python code to use REST API to extract tick history raw from DSS. The extraction works but the file output is not sort by timestamp but instead sort by FIDs. How can I fix that?


Below is the example using my program to extract 0005.HK on Nov 1 from 0930 - 1000 local exchange time, for FIDs 22, 25 and 6. Here's a snapshot of the extraction using my program. Its sorted by BID (22), following by ASK (25) and then by TRDPRC_1 (6)


capture2.png


I compare the same extraction using TRTH with the following template which is same as my program

Below is the extract from TRTH. You can see the data is sorted by timestamp.


capture5.png


trth-rest-api
capture2.png (100.2 KiB)
capture5.png (228.8 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
80.3k 259 52 75

@KAKIT.LAI

The request contains only one FID.

1637642086562.png

Therefore, the result contains only values of the TRDPRC_1 field.

1637642161325.png


1637642086562.png (30.0 KiB)
1637642161325.png (30.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
80.3k 259 52 75

@KAKIT.LAI

Please share the request message that you are using.

I used the following request.

{
  "ExtractionRequest": {
    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
    "IdentifierList": {
      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [{
        "Identifier": "0005.HK",
        "IdentifierType": "Ric"
      }],"ValidationOptions": {"AllowHistoricalInstruments": true}
    },
    "Condition": {
      "MessageTimeStampIn": "LocalExchangeTime",
      "ReportDateRangeType": "Range",
      "QueryStartDate": "2021-11-01T01:30:00.000Z",
      "QueryEndDate": "2021-11-01T02:00:00.000Z",
      "ExtractBy": "Ric",
      "SortBy": "SingleByRic",
      "DomainCode": "MarketPrice",
      "Fids": "22,25,6",
      "DisplaySourceRIC": true,
      "FidListOperator": "OR"
    }
  }
}

The output is:

1636949150330.png


1636949150330.png (102.6 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
22 3 3 10

This is the code


    "ExtractionRequest": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest",
        "IdentifierList": {
        "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
        "InstrumentIdentifiers": list_of_ric2,
        "UseUserPreferencesForValidationOptions":"false"
        },    
        "Condition": {
        "DateRangeTimeZone": DateRangeTimeZone,
        "TimeRangeMode": TimeRangeMode,
        "MessageTimeStampIn": MessageTimeStampIn,
        "ReportDateRangeType": "Range",
        "QueryStartDate": start_datetime,
        "QueryEndDate": end_datetime,
        "ExtractBy": ExtractBy,
        "SortBy": SortBy,
        "DomainCode": DomainCode,
        "DisplaySourceRIC":"false",
        "Fids": ",".join(fids)
        }
    }
    }


And this is what I input into my program


capture.png



capture.png (25.3 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@KAKIT.LAI

The request message doesn't have the FidListOperator property. I checked and found that, on my account, the default value is "AND" so the output is:

1636962252327.png

Please check the request message with other tools, such as Postman or curl.

1636962252327.png (30.0 KiB)
Upvotes
22 3 3 10

Thanks. I tried but still giving me the same output...


"Condition": {
        "DateRangeTimeZone": DateRangeTimeZone,
        "TimeRangeMode": TimeRangeMode,
        "MessageTimeStampIn": MessageTimeStampIn,
        "ReportDateRangeType": "Range",
        "QueryStartDate": start_datetime,
        "QueryEndDate": end_datetime,
        "ExtractBy": ExtractBy,
        "SortBy": SortBy,
        "DomainCode": DomainCode,
        "DisplaySourceRIC":"false",
        "Fids": ",".join(fids),
        "FidListOperator": "AND"
        }
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@KAKIT.LAI

With the "AND" operator, the output should be like this:

1637028897894.png

Please check the request message with other tools, such as Postman or curl.

1637028897894.png (107.5 KiB)
Upvotes
22 3 3 10

Sorry for the delay in getting back. I tried again and output still sorted by the FIDs instead of time of the update. The following is the actual parameter passed to in ExtractionRequest.


{'ExtractionRequest': {'@odata.type': '#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest', 'IdentifierList': {'@odata.type': '#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList', 'InstrumentIdentifiers': [{'Identifier': '3333.HK', 'IdentifierType': 'Ric'}], 'UseUserPreferencesForValidationOptions': 'false'}, 'Condition': {'DateRangeTimeZone': 'Local Exchange Time Zone', 'TimeRangeMode': 'Inclusive', 'MessageTimeStampIn': 'LocalExchangeTime', 'ReportDateRangeType': 'Range', 'QueryStartDate': '2021-11-08T10:00:00.000Z', 'QueryEndDate': '2021-11-08T10:30:00.000Z', 'ExtractBy': 'Entity', 'SortBy': 'SingleByRic', 'DomainCode': 'MarketPrice', 'DisplaySourceRIC': 'false', 'Fids': '6', 'FidListOperator': 'AND'}}}

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
22 3 3 10

I have fixed the code and issue is fixed. You can close the ticket. Thanks for the help.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.