How to generate RWF test data from XML or other human-readable format?

Various Reuters tools, such as rmdstestclient and the ETA's rsslIoctl (RSSL_TRACE...) option, are able to format sent/received RWF messages as human-readable XML files (well, sort-of-readable, for RSSL_TRACE).

Does there exist a library or other sample code which can encode messages *from* human-readable XML (or JSON, or ...) back into raw RWF messages? The idea is that it should be possible to substitute a canned message for the RWF data buffer normally received from rsslRead(), something that'd be very helpful when writing unit tests to exercise ETA-based message decoding and processing software.

(Yes, it is of course possible to write bespoke code to generate the encoded RWF messages; but now you have *two* things to test...)

Find more posts tagged with

elektron

RWF

elektron-sdk

refinitiv-realtime

Accepted answers

All comments

zoya faberov

sink_driven_src tool, from the same Infra Tools package as rmdstestclient, can likely do what you are looking for, in terms of testing. This is not a library or a code sample, rather a built tool command-line driven testing tool.

Another option, more extensive in terms of features, to simulate/replay pre-recorded data, is product ReplayService for TREP. There are both GUI and shell script options to record, replay and analyze the recorded data.

Martin J

Thanks, Zoya. I'm aware of the various standalone tools such as sink_driven_src. These are very useful for integration testing. The TradeWeb ReplayService sounds nice, but is a) expensive, and b) massive overkill for what I am trying to do.

As mentioned above, I am looking for something which will help with *unit* testing, accepting a text-based description of a message and returning its RWF encoding, which can then be passed to the decoding and processing code under test.

This would make it possible to test smaller pieces of our application quickly and in isolation, without the overhead (in time but also complexity) of establishing and maintaining a channel to a real TREP server or even a separate simulation process such as sink_driven_src.

Clearly the thing I am looking for exists in the sink_driven_src code, because that utility is able to read and replay a captured XML created by rmdstestclient. It would be great if that converter were available as a standalone library or tool -- something like:

$ xml2rwf < msgs.xml > msgs.rwf

Or, as a C/C++ library:

std::string xmlMsg = "<requestMsg domainType="RSSL_DMT_ ... </requestMsg>";
std::vector<uint8_t> rwfMsg = xml2rwf(xmlMsg);

Perhaps it's possible to accomplish this by duct-taping together a combination of:

sink_driven_src -Q msgs.xml ...  &  rmdstestclient -obf msgs.rwf ...

But that's pretty complicated / fragile / ugly. There must be a better alternative.

zoya faberov

Hello @Martin J,

I do not believe that we provide a library that encodes arbitrary messages as RWF.

From my experience, many choose to test with sink_driven_src or RS. Especially as after unit testing there is usually performance testing and error-condition testing requirements.

You can check out examples of providers and perfProviders found in our Elektron toolkits, they will encode and publish realistic data, although not arbitrary messages, and are also used for testing.

If something like this exists, I would check with third parties. If not found, it would have to be custom-developed, which would be time and effort.

Martin J

The XML-to-RWF converter does exist inside sink_driven_src; it's a pity it's not usable on its own.

I know that ETA is being open-sourced soon; are there any plans to do the same for sink_driven_src or other tools?

zoya faberov

To my knowledge, there is no plans to open-source infra tools. They are not under API products family, rather they belong to infrastructure, propriatory.

However, if anything changes, we will inform the community.

Jirapongse

I have done this before. I used RSSL_TRACE_HEX in ETA to display hex values for all messages. The trace files will contain both XML and hex string of raw binary.

<refreshMsg domainType="RSSL_DMT_MARKET_PRICE" streamId="6" containerType="RSSL_DT_FIELD_LIST" flags="0x1FA (RSSL_RFMF_HAS_PERM_DATA|RSSL_RFMF_HAS_MSG_KEY|RSSL_RFMF_HAS_SEQ_NUM|RSSL_RFMF_SOLICITED|RSSL_RFMF_REFRESH_COMPLETE|RSSL_RFMF_HAS_QOS|RSSL_RFMF_CLEAR_CACHE)" groupId="23" seqNum="44478" permData="0308 4362 C0" qosDynamic="0" qosRate="1" qosTimeliness="1" dataState="RSSL_DATA_OK" streamState="RSSL_STREAM_OPEN" code="RSSL_SC_NONE" text="All is well"  dataSize="1594">
    <key  flags="0x7 (RSSL_MKF_HAS_SERVICE_ID|RSSL_MKF_HAS_NAME|RSSL_MKF_HAS_NAME_TYPE)"  serviceId="2115" name="IBM.N" nameType="1"/>
...
...

</refreshMsg>


0032 0206 0000 0006 81FA 0400 00AD BE09 000B 416C 6C20 6973 2077 656C 6C02 0017
0503 0843 62C0 2280 0B07 FE08 4305 4942 4D2E 4E01 0903 0100 4F01 2800 0101 3E00
0201 4000 0310 494E 544C 2042 5553 204D 4143 4849 4E45 0004 0102 0006 0508 089E
E0F0 0007 0508 089E E0F0 0008 0508 089E 92D0 0009 0508 089E 92D0 000A 0508 089E

Then, I read this hex string and convert it to binary. I found code in stack overflow.

Finally, I assigned the binary to the RsslBuffer to decode it.

Martin J

Thank you for the suggestion. This does give you both the XML and hex representations -- provided that you have already captured an encoded RWF message.

The initial question, though, was about how to go in the opposite direction. I'd asked whether there is "a library or other sample code which can encode messages from human-readable XML (or JSON, or ...) back into raw RWF messages". I'm still hoping that Reuters will one day offer a standalone version of the function, already present in sink_driven_src, which converts an XML description of a message into the corresponding RWF.

EXPLORE OUR SITES