question

Upvotes
Accepted
1 0 0 4

EMA Java connection with RSSL_WEBSOCKET channel type

Hello everyone,

We have a server that accepts web socket communication in order to provide some financial data to one of our consumer applications. The server is set up correctly and it could be proven by using one of the examples of Web Socket API (see this example here: https://github.com/Refinitiv/websocket-api/blob/master/Applications/Examples/java/MarketPrice.java). Basically after adjusting properties from the "MarketPrice" program and starting it, it is able to connect to our platform and to successfully log in.

After that, I am trying to do a similar thing but with the EMA Java. I created the following EMA config.xml file:

<EmaConfig>
    <ConsumerGroup>
        <DefaultConsumer value="CONSUMER"/>
        <ConsumerList>
            <Consumer>
                <Name value="CONSUMER"/>
                <ChannelSet value="CHANNEL_1, CHANNEL_2"/>
                <XmlTraceToStdout value="0"/>
            </Consumer>
        </ConsumerList>
    </ConsumerGroup>

    <ChannelGroup>
        <ChannelList>
            <Channel>
                <Name value="CHANNEL_1"/>
                <ChannelType value="ChannelType::RSSL_WEBSOCKET"/>
            <Host value="${host1}"/>
            <Port value="${port}"/>
            </Channel>
            <Channel>
                <Name value="CHANNEL_2"/>
                <ChannelType value="ChannelType::RSSL_WEBSOCKET"/>
                <Host value="${host2}"/>
                <Port value="${port}"/>
            </Channel>
        </ChannelList>
    </ChannelGroup>
</EmaConfig>


And in one of my classes I create the OmmConsumer as follows:

        Client client = new Client(); // where Client implements OmmConsumerClient

        OmmConsumer consumer = EmaFactory.createOmmConsumer(
                EmaFactory.createOmmConsumerConfig()
                        .username(${userName})
                        .applicationId(${defaultApplicationId})
                        .consumerName("CONSUMER")
        );

        ReqMsg reqMsg = EmaFactory.createReqMsg();
        consumer.registerClient(reqMsg.serviceName(${serviceName}).name("${itemName}"), client);


The server utilizes a WebSocket on the following URL: ws://${host}:${port}/WebSocket.

However, with the following configuration, I receive the following response from the platform:

13:55:58.922 [main] WARN com.refinitiv.ema.access.OmmConsumerImpl - loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text:    Received ChannelDownReconnecting event on channel CHANNEL_1
RsslReactor @2783717b
RsslChannel @76f7d241
Error Id 0
Internal sysError 0
Error Location null
Error text Invalid Message Size. Message size is: 3431677. Max Message size is: 734964.
loggerMsgEnd


13:55:58.925 [main] WARN com.refinitiv.ema.access.OmmConsumerImpl - loggerMsg
ClientName: LoginCallbackClient
Severity: Warning
Text:    RDMLogin stream state was changed to suspect with status message
username <not set>
usernameType <not set>

State: Open/Suspect/None - text: ""
loggerMsgEnd


Maybe there is something missing in the configuration or there is another issue why the attempt to connect to the Refinitiv data provider via EMA Java fails?

Thank you very much for your help in advance!

Best regards,
Anton

ema-apiwebsocketsconfiguration
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

13 Answers

Upvotes
Accepted
53k 137 44 63

@anton.skripin

It may relate to the dictionary download. By default, EMA will download the dictionary from the server while the WebSocket API Java example doesn't download it. You can configure EMA to use a local dictionary instead by using the following configurations.

<EmaConfig>
    <ConsumerGroup>
        <DefaultConsumer value="CONSUMER"/>
        <ConsumerList>
            <Consumer>
                <Name value="CONSUMER"/>
                <ChannelSet value="CHANNEL_1, CHANNEL_2"/>
                <Dictionary value="Dictionary_2"/>
                <XmlTraceToStdout value="0"/>
            </Consumer>
        </ConsumerList>
    </ConsumerGroup>
 
    <ChannelGroup>
        <ChannelList>
            <Channel>
                <Name value="CHANNEL_1"/>
                <ChannelType value="ChannelType::RSSL_WEBSOCKET"/>
            <Host value="${host1}"/>
            <Port value="${port}"/>
            </Channel>
            <Channel>
                <Name value="CHANNEL_2"/>
                <ChannelType value="ChannelType::RSSL_WEBSOCKET"/>
                <Host value="${host2}"/>
                <Port value="${port}"/>
            </Channel>
        </ChannelList>
    </ChannelGroup>
    
    <DictionaryGroup>
        <DictionaryList>
            <Dictionary>
                <Name value="Dictionary_1"/>                
                <DictionaryType value="DictionaryType::ChannelDictionary"/>
            </Dictionary>
            <Dictionary>
                <Name value="Dictionary_2"/>
                <DictionaryType value="DictionaryType::FileDictionary"/>                
                <RdmFieldDictionaryFileName value="./RDMFieldDictionary"/>
                <EnumTypeDefFileName value="./enumtype.def"/>
            </Dictionary>            
        </DictionaryList>
    </DictionaryGroup>
</EmaConfig>

The dictionary files (RDMFieldDictionary and enumtype.def) are available in the \Java\etc directory in the EMA package.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvote
16.7k 42 12 19

Hello @anton.skripin,

There is a EMA websocket example and the associated configuration which ships with the EMA SDK. Have you tried using this?

https://github.com/Refinitiv/Real-Time-SDK/tree/master/Java/Ema/Examples/src/main/java/com/refinitiv/ema/examples/training/consumer/series100/ex180_MP_Websocket

and the example websocket configuration which uses Consumer6 and Channel_6: https://github.com/Refinitiv/Real-Time-SDK/blob/master/Java/Ema/EmaConfig.xml


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi,


thank you for your reply. Yes, sure, I have seen this example. Moreover, this example was taken as the basis of the program I provided above but it is not working as expected (see my initial question).

Any help would be appreciated. I do not see any problems with the XML configuration. Maybe there is some problem with how the OmmConsumerConfig is being created?


OmmConsumer consumer = EmaFactory.createOmmConsumer(
                EmaFactory.createOmmConsumerConfig()
                        .username(${userName})
                        .applicationId(${defaultApplicationId})
                        .consumerName("CONSUMER")
        );


In addition, could you please provide me the information where the login request is constructed in EMA with the WebSocket connection type before sending a login request? I would like to compare it with the login request that I created manually using the WebSocket-API example. The reason for it is that I suspect that somehow with the current configuration the login request is being constructed incorrectly because I receive the following warning in the console:

13:55:58.925 [main] WARN com.refinitiv.ema.access.OmmConsumerImpl - loggerMsg
ClientName: LoginCallbackClient
Severity: Warning
Text:    RDMLogin stream state was changed to suspect with status message
username <not set>
usernameType <not set>


But at least "username" is provided in Java code while creating the OmmConsumerConfig...

Thank you!

Upvotes
16.7k 42 12 19

Hi @anton.skripin,

It looks like you are connecting to a local market data server (ADS); are you sure you want to use the websocket protocol with EMA SDK. The standard way would be to use RSSL (binary) data format for an EMA application.

Can you please try to follow this quickstart guide and see if you can get a sample working.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16.8k 31 9 12

Hello @anton.skripin

What is the version of EMA Java (or RTSDK) that encounters the issue?

I cannot replicate the same "RDMLogin stream state was changed to suspect with status message" error message with EMA Java version 3.6.6 (RTSDK Java 2.6.0.L1).

The ex180_MP_Websocket can connect to my local ADS WebSocket connection by setting the API to use local dictionaries as suggested by @Jirapongse .


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
1 0 0 4

Thank you to everyone,

Indeed, after I provided the local dictionaries the program managed to connect to the ADS.

However, is it possible to get those dictionaries from the ADS at runtime? I am asking because the Robust Foundation API (RFA) supported the DICTIONARY message type which allowed to receive the appropriate message with a dictionary and load it at runtime. Now I am looking for a similar mechanism in EMA.

Thank you one more time.

Best regards,
Anton

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16.7k 42 12 19

@anton.skripin,

I am repeating my previous message - using websockets with EMA API when connecting to a local ADS is a poor choice. Is there a specific reason you want to do this? If not, please consider using EMA in a traditional way. Your RFA application did.

Start from the quickstart or these tutorials.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
1 0 0 4

@Gurpreet

one of the client's requirements was to replace the legacy RFA library with a newer WebSocket API in the hope of improving performance and getting rid of some legacy-related issues etc. After doing some research I found out that WebSocket API is not really a library, so one would have to take care of session management, recovery, etc by him/herself. Later on, I stumbled upon EMA that in theory supports the WebSocket communication mechanism.

In what cases then EMA API using WebSockets would be beneficial over the standard RSSL? In the end, the configuration setup should be the same both for RSSL and WebSocket except for providing different channel types in the configuration, shouldn't it?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@anton.skripin

replace the legacy RFA library with a newer WebSocket API in the hope of improving performance

Websockets (being a JSON, text based protocol) will not improve performance compared to RFA. It is primarily intended for applications that use non-SDK supported languages, i.e. any language that is not Java or C++.

EMA supports websockets API, primarily for sourcing data from cloud, where binary RSSL traffic might not be allowed by the firewalls.

Upvotes
1 0 0 4

@Gurpreet


Thank you for clarifying it. The logic and configuration of the consumer can be the same independent of whether the server accepts RSSL traffic or works via WebSocket. The only difference is that one should change in the configuration the channel type from RSSL to WebSocket or vice versa. Do I understand it correctly? Thanks!

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24.9k 87 10 22

hi @anton.skripin

Is it possible you confuse the Websocket API (designed primarily as JSON-based API for developers not using Java or C++) with the RSSL_WEBSOCKET Channel type?

Programming with EMA Java or C++, you can use RSSL_SOCKET (the default) or the RSSL_WEBSOCKET (as well as some others)

And even when using RSSL_WEBSOCKET Channel type, you can then choose to use different protocols (tr_json2, rssl.rwf, rssl.json.v2)

However, unless there is a particular requirement, you should stick to the default of RSSL_SOCKET (which uses RWF binary data format) - for the best performance.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
1 0 0 4

Hi @umer.nalla

No, I believe I do understand the difference between the WebSocket API and the RSSL_WEBSOCKET Channel Type. I assume that if one chooses to connect to the server (that accepts web socket connections) using the RSSL_WEBSOCKET then somewhere under the hood, the EMA will convert an EMA programmatic syntax into a JSON payload, will connect to the socket via WebSocket, and will listen to any market data coming from the server.

Is this not the case?

For example, if I have a web socket server listening to the following address: ws://localhost/WebSocket Then, I have two options on how to reach the market data from it:

1) Use EMA with the RSSL_WEBSOCKET Channel Type, configure host, port, and other parameters to connect to the ADS.

2) Setup the web socket and connect to the socket server by myself. Construct a request with a JSON body following the documentation of WebSocket API and so on.

Both options would work as far as I understand. It would be just more convenient and easier to use EMA with java than to use WebSocket API with Java just because EMA will handle many things for you as a library.

As before, I sincerely thank everyone for your help!

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
16.8k 31 9 12

Hello @anton.skripin

The data between the RFA API and the server (Legacy MarketFeed: encrypted text-based message, RSSL: encrypted binary) is smaller than the WebSocket JSON string message. The WebSocket connection uses JSON string format because it focuses on easy-to-read and easy to parse by other services.

The EMA/ETA API does not support dictionaries downloaded from ADS at runtime for the WebSocket connection. This feature is available for the RSSL connection only.

Note: The EMA API RSSL connection encoded data in an encrypted binary format, which is highly optimized for data distribution.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
53k 137 44 63

@anton.skripin

ADS 3.4.0 introduced the enableMultipartDictionary configuration to send dictionary with multiple part refresh messages.


1658461690004.png

*ads*enableMultipartDictionary : True

This configuration works fine with EMA C++ (WebSocket connection) but EMA Java (WebSocket connection) reports this error.

Failed to convert JSON message: JSON Converter parse error: Invalid UTF-8 middle byte 0x22
 at [Source: (com.refinitiv.eta.json.converter.ByteBufferInputStream); line: 1, column: 3187]

I submitted this issue to the product team via GitHub.


1658461690004.png (44.3 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
24.9k 87 10 22

Hi @anton.skripin

Reading your most recent response, am I right in thinking that you want to connect to your own Websocket Server using EMA - rather than an ADS providing a Websocket Service?


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.