Custom data types for a blob of 200,000 char

johan.lundquist

Hi,

We are currently investigating how to transport internal messages of tick data for large string objects. One idea is to publish the through ATS to TREP, and via an internal feed distribute the data internally. Do you have a supporting custom data type for string sizes of 200,000 characters?

Kind regards,

Johan

umer.nalla

Hi @johan.lundquist

You can write a Provider application (either Interactive or non-interactive depending our your requirement/preference) to publish your data.

Would you intend to publish the 200k characters in only a RefereshMsg or in an UpdateMsg as well? If only in RefreshMsg, would the UpdateMsg payload be less than 64k?

If only in a RefreshMsg (and sub 64K UpdateMsg) and if you are able to split the 200k then you could use the OMM support for multipart RefreshMsg to split the payload across the RefreshMsgs and then set the Completion flag on the final RefreshMsg. The sub 64k UpdateMsg could then be published as normal.

You can also use payloads larger than 64k but we do not recommend this as the TREP infrastructure is optimised for 64k message sizes.

You don't necessarily have to use a Custom Domain model - you could use MarketPrice and include a Buffer type field which allows upto 65535 bytes.

If you cannot split the 200k across fields or messages, then you could zip the 200k before populating the field and then the consumer has to unzip the field at their end. For this scenario it would be better to define a custom domain model - to differentiate it from the existing domains. Please see one of of our Machine Readable News tutorial for an example of using zip to transport larger payloads.

You may also find this post useful in terms of payloads > 64K and this post on large generic messages.