How to deal with TickHistoryRawExtractionRequest csv data

Hi,
I am having trouble visualising the way to deal with the TickHistoryRawExtractionRequest csv data.
I am fine with the key value nature of the raw data, but it is more the understanding of how to deal with a identifier line, with the related key value lines underneath, e.g.:

#RIC,Domain,Date-Time,GMT Offset,Type,MsgClass/FID number,UpdateType/Action,FID Name,FID Value,FID Enum String,PE Code,Template Number,Key/Msg Sequence Number,Number of FIDs
EUR=,Market Price,2017-09-29T12:00:00.262065741Z,+0,Raw,UPDATE,UNSPECIFIED,,,,213,,37182,4
,,,,FID,999,,GEN_VAL4,1.181,
,,,,FID,1061,,GV1_TIME,12:00:00.000000000,
,,,,FID,8107,,BID_HOURLY,1.181,
,,,,FID,14208,,BID_HR_MS,12:00:00.175000000,
EUR=,Market Price,2017-09-29T12:00:00.626077969Z,+0,Raw,UPDATE,UNSPECIFIED,,,,213,,37198,34
,,,,FID,22,,BID,1.1812,
,,,,FID,393,,PRIMACT_1,1.1812,
,,,,FID,25,,ASK,1.1813,
,,,,FID,275,,SEC_ACT_1,1.1813,
,,,,FID,875,,VALUE_DT1,2017-09-29,
,,,,FID,1010,,VALUE_TS1,12:00:00.000000000,
,,,,FID,5,,TIMACT,12:00:00.000000000,

Does anyone have any code of how they handled this? Or just advice?
Any language will be graciously received.

Find more posts tagged with

csv

tick-history-rest-api

Accepted answers

Jirapongse

To understand it, you need to know the Market Price domain model.

The Market Price domain provides access to Level I market information such as trades, indicative quotes, and top-of-book
quotes. All information is sent as field-value pairs (i.e., net change, bid, ask, volume, high, low, or last price).

An identifier line indicates a type of message and the related key value lines underneath are payload associated with that message. You can refer to the top row for the meaning of each column.

Typically, you will see two types of message in a CSV file.

1. REFRESH message

The payload of the refresh message contains all available fields and values for the associated RIC.

EUR=,Market Price,2017-07-22T00:05:06.885058466Z,+0,Raw,REFRESH,,,,,213,99,8975,317
,,,,FID,1,,PROD_PERM,213,
,,,,FID,2,,RDNDISPLAY,153,
,,,,FID,3,,DSPLY_NAME,SE BANKEN    NYC,
,,,,FID,5,,TIMACT,21:00:00.000000000,
,,,,FID,11,,NETCHNG_1,-0.0003,
,,,,FID,12,,HIGH_1,1.1658,
,,,,FID,13,,LOW_1,1.1664,
,,,,FID,15,,CURRENCY,840,USD
,,,,FID,17,,ACTIV_DATE,2017-07-21,
,,,,FID,19,,OPEN_PRC,1.1658,
,,,,FID,21,,HST_CLOSE,1.1661,
,,,,FID,22,,BID,1.1658,
,,,,FID,23,,BID_1,1.1661,
...
...

The above REFRESH message indicates that there are 317 fields for EUR=.

2. UPDATE message

The UPDATE message is used to convey any changes to an item's data. Its payload contains only updated fields.

EUR=,Market Price,2017-09-29T12:00:00.262065741Z,+0,Raw,UPDATE,UNSPECIFIED,,,,213,,37182,4
,,,,FID,999,,GEN_VAL4,1.181,
,,,,FID,1061,,GV1_TIME,12:00:00.000000000,
,,,,FID,8107,,BID_HOURLY,1.181,
,,,,FID,14208,,BID_HR_MS,12:00:00.175000000,

The above UPDATE message indicates the values of these four fields are updated. The values of other fields remain the same.

For more information regarding Market Price domain, please refer to RDM Usage Guide.

All comments

Jirapongse

To understand it, you need to know the Market Price domain model.

An identifier line indicates a type of message and the related key value lines underneath are payload associated with that message. You can refer to the top row for the meaning of each column.

Typically, you will see two types of message in a CSV file.

1. REFRESH message

The payload of the refresh message contains all available fields and values for the associated RIC.

EUR=,Market Price,2017-07-22T00:05:06.885058466Z,+0,Raw,REFRESH,,,,,213,99,8975,317
,,,,FID,1,,PROD_PERM,213,
,,,,FID,2,,RDNDISPLAY,153,
,,,,FID,3,,DSPLY_NAME,SE BANKEN    NYC,
,,,,FID,5,,TIMACT,21:00:00.000000000,
,,,,FID,11,,NETCHNG_1,-0.0003,
,,,,FID,12,,HIGH_1,1.1658,
,,,,FID,13,,LOW_1,1.1664,
,,,,FID,15,,CURRENCY,840,USD
,,,,FID,17,,ACTIV_DATE,2017-07-21,
,,,,FID,19,,OPEN_PRC,1.1658,
,,,,FID,21,,HST_CLOSE,1.1661,
,,,,FID,22,,BID,1.1658,
,,,,FID,23,,BID_1,1.1661,
...
...

The above REFRESH message indicates that there are 317 fields for EUR=.

2. UPDATE message

The UPDATE message is used to convey any changes to an item's data. Its payload contains only updated fields.

EUR=,Market Price,2017-09-29T12:00:00.262065741Z,+0,Raw,UPDATE,UNSPECIFIED,,,,213,,37182,4
,,,,FID,999,,GEN_VAL4,1.181,
,,,,FID,1061,,GV1_TIME,12:00:00.000000000,
,,,,FID,8107,,BID_HOURLY,1.181,
,,,,FID,14208,,BID_HR_MS,12:00:00.175000000,

The above UPDATE message indicates the values of these four fields are updated. The values of other fields remain the same.

For more information regarding Market Price domain, please refer to RDM Usage Guide.

Keech.Achara

Hey @JirapongseI

I was more mentioning the fact that the funky structure of the raw file means that only quite slow sequential processing can be performed.

I was hoping that someone would have a scalable library for processing the file into the Market Price Types, to get me started, and if not someone will have pointers on how to process this accordingly.

zoya faberov

Hello @Keech.Achara,

Thank you for your participation in the forum.

It appears to me that the question has been answered fully, and in detail, the way it has been first asked.

However, in terms of your clarification, regarding your intention to solicit feedback from the community members, it's a much harder task, as for the most part, unfortunately, developers are not verbally inclined, and only speak up (or write up ) when they are stuck and need help. Sharing within the community is a huge step forward, we are working in that direction.

Therefore, if you feel that this output format is not convenient to handle, or requires improvement, please help us by raising product enhancement request via My Account Raise A Case , otherwise, please click the 'Accept' text next to the reply. This will guide all community members who have a similar question.

Thanks,

-AHS

Keech.Achara

Hey @zoya faberov

I am not happy that the comment above has been marked as the the accepted answer.

I requested help with "understanding of how to deal with a identifier line,with the related key value lines underneath", I also requested "any code of how they handled this? Or just advice? Any language will be graciously received".

The comment marked as an answer does not in any way answer those questions, or provide the advice.

I can raise a case, but having answers like this, does not help anyone that has the same question in the future.

zoya faberov

Hi @Keech.Achara,

In my answers, I always aim to help.

The raw output, as provided is just a mirror image of the raw output as available from Elektron data feed. Each update consists of the header line, followed by line per fid. Each line has the fid number, fid type and value. This is what Jirapongse has explained much better and in much more detail.

Therefore, a way of dealing with this is reading the output line by line, and processing in a way that is consistent with your use case. The processing would differ depending on what the application is doing, what fids it's looking for, etc.

Consequently, we do not have examples of this handling. There was some chance of community sharing their experience on this matter, and sometimes it happens, when developers feel strongly, but that did not happen in this specific case.

However, if you feel the presentation could be enhanced, or should be different, or there should be options, I suggest, that raising a service case as a client will get you request directly routed to the product management team, who will consider, validate and prioritize your enhancement request.

Hope this explanation helps,

-Zoya

EXPLORE OUR SITES