EMA Java: "channel out of buffers errorId=-4" error with over 3,000 RICs

skajiwara

Hi,

I'm using EMA Java from Scala 2.11.

I `registerClient()` 3,558 RICs at once by the following code:

val client = new MyClient()  // sub-class of OmmConsumerClient

val rics = List("/1418.T", ...)  // Ordinary stocks in Tokyo Stock Exchange, whose length is 3558

rics.map { ric =>
  consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name(ric), client)}

Although `MyClient#on*Msg()` have almost empty implementation, my program uses a lot of memory (memory usage of the process is around 6GB on 16GB-memory OSX).

It leads to the following runtime error:

19:17:15.024 [pool-2-thread-1] ERROR c.t.e.a.OmmConsumerImpl - loggerMsg
    ClientName: EmaConsumer_1
    Severity: Error
    Text:    Call to rsslReactorDispatchLoop() failed. Internal sysError='0' Error text='channel out of buffers errorId=-4 errorText=channel out of buffers'. 
loggerMsgEnd

My questions are:

• Is it usual to consume such a large memory with 3,558 RICs registration?
  • If it's unusual, I will use some heap profiler (no idea currently) to detect potential memory leak.
  • If it's usual, could you tell me about how much memory would be required to get 3,558 RICs' feed? I appreciate if I can hear some reasonable tuning points.

Thank you.

Jirapongse

@skajiwara

It could be the size of GuaranteedOutputBuffers used by the underlying API (ETA).

You can try to increase its value via the configuration. For example:

<ChannelGroup>
	<ChannelList>
		<Channel>
			<Name value="Channel_1"/>	
			...
			<GuaranteedOutputBuffers value="5000"/>
			<Host value="hostname"/>
			<Port value="14002"/>
				...
		</Channel>

Nipat Kunvutipongsak

@skajiwara It seems like the problem didn't occur when the application tried to send a batch request with a lot of items, but the problem happened after that 'request' state (when the application received data), am I correct?

I have replicated this problem by putting 4000 items into a request message with EMA Java version emaj3.0.3.L1.all (from Elektron-SDK1.0.7java). However, it encountered the message below only.

"Failed to rsslElementEntry.encode() while encoding rssl elementlist. Reason='INVALID_DATA'"

Could you please confirm whether my understanding is correct or not?

Nipat Kunvutipongsak

https://community.developers.refinitiv.com/discussion/comment/9818#Comment_9818

Also, can I know the version of EMA/Elektron-SDK the application is using? Perhaps, I can't reproduce the problem because of different API version. Thanks.

jerome.raghunandan

Hi, I have the same error with the API version below. I don't have a EmaConfig.xml file. How can I increase the GuaranteedOutputBuffers in this version without using an EmaConfig.xml file?

<groupId>com.thomsonreuters.ema</groupId>
<artifactId>ema</artifactId>
<version>3.2.0.2</version>

This is the error I have:

ClientName: SingleItem
Severity: Error
Text: Internal error: rsslChannel.submit() failed in SingleItem.submit(PostMsg)RsslChannel 0
Error Id -3
Internal sysError 0
Error Location Watchlist.encodeIntoBufferAndQueue
Error Text channel out of buffers errorId=-4 errorText=channel out of buffers

jerome.raghunandan

https://community.developers.refinitiv.com/discussion/comment/100021#Comment_100021

EmaConfig.zip

I managed to resolve the issue by creating an EmaConfig.xml file and placing it in the root folder of my application. I found that is the default location the API looks in for the file. I set the GuaranteedOutputBuffers in the file and that fixed the issue. I have attached an example of the EmaConfig.xml file I created.