question

Upvotes
Accepted
skajiwara avatar image
3 ●0 ●2 ●3

EMA Java: "channel out of buffers errorId=-4" error with over 3,000 RICs

Hi,

I'm using EMA Java from Scala 2.11.

I `registerClient()` 3,558 RICs at once by the following code:

val client = new MyClient()  // sub-class of OmmConsumerClient

val rics = List("/1418.T", ...)  // Ordinary stocks in Tokyo Stock Exchange, whose length is 3558

rics.map { ric =>
  consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name(ric), client)}

Although `MyClient#on*Msg()` have almost empty implementation, my program uses a lot of memory (memory usage of the process is around 6GB on 16GB-memory OSX).

It leads to the following runtime error:

19:17:15.024 [pool-2-thread-1] ERROR c.t.e.a.OmmConsumerImpl - loggerMsg
    ClientName: EmaConsumer_1
    Severity: Error
    Text:    Call to rsslReactorDispatchLoop() failed. Internal sysError='0' Error text='channel out of buffers errorId=-4 errorText=channel out of buffers'. 
loggerMsgEnd

My questions are:

  • Is it usual to consume such a large memory with 3,558 RICs registration?
    • If it's unusual, I will use some heap profiler (no idea currently) to detect potential memory leak.
    • If it's usual, could you tell me about how much memory would be required to get 3,558 RICs' feed? I appreciate if I can hear some reasonable tuning points.

Thank you.

elektronrefinitiv-realtimeelektron-sdkema-apirrtelektron-message-apimemory
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

pimchaya.wongrukun01 avatar image   pimchaya.wongrukun01

Monitored by @Nipat Kunvutipongsak

AHS

Nipat Kunvutipongsak avatar image Nipat Kunvutipongsak

@skajiwara It seems like the problem didn't occur when the application tried to send a batch request with a lot of items, but the problem happened after that 'request' state (when the application received data), am I correct?

I have replicated this problem by putting 4000 items into a request message with EMA Java version emaj3.0.3.L1.all (from Elektron-SDK1.0.7java). However, it encountered the message below only.

"Failed to rsslElementEntry.encode() while encoding rssl elementlist. Reason='INVALID_DATA'"

Could you please confirm whether my understanding is correct or not?

Nipat Kunvutipongsak avatar image Nipat Kunvutipongsak Nipat Kunvutipongsak

Also, can I know the version of EMA/Elektron-SDK the application is using? Perhaps, I can't reproduce the problem because of different API version. Thanks.

pimchaya.wongrukun01 avatar image   pimchaya.wongrukun01

Hello @skajiwara

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query? If yes please click the 'Accept' text next to the reply. This will guide all community members who have a similar question. Otherwise please post again offering further insight into your question.

Thanks,

AHS

pimchaya.wongrukun01 avatar image   pimchaya.wongrukun01

Hello @skajiwara

Please be informed that a reply has been verified as correct in answering the question, and has been marked as such.

Thanks,

AHS

2 Answers

· Write an Answer
Upvote
Accepted
Jirapongse avatar image
78.8k ●249 ●52 ●74
@skajiwara

It could be the size of GuaranteedOutputBuffers used by the underlying API (ETA).

You can try to increase its value via the configuration. For example:

<ChannelGroup>
	<ChannelList>
		<Channel>
			<Name value="Channel_1"/>	
			...
			<GuaranteedOutputBuffers value="5000"/>
			<Host value="hostname"/>
			<Port value="14002"/>
				...
		</Channel>

config.png (26.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
jerome.raghunandan avatar image
1 ●0 ●0 ●1

Hi, I have the same error with the API version below. I don't have a EmaConfig.xml file. How can I increase the GuaranteedOutputBuffers in this version without using an EmaConfig.xml file?

<groupId>com.thomsonreuters.ema</groupId>
<artifactId>ema</artifactId>
<version>3.2.0.2</version>


This is the error I have:

ClientName: SingleItem
Severity: Error
Text: Internal error: rsslChannel.submit() failed in SingleItem.submit(PostMsg)RsslChannel 0
Error Id -3
Internal sysError 0
Error Location Watchlist.encodeIntoBufferAndQueue
Error Text channel out of buffers errorId=-4 errorText=channel out of buffers
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

jerome.raghunandan avatar image jerome.raghunandan

EmaConfig.zip

I managed to resolve the issue by creating an EmaConfig.xml file and placing it in the root folder of my application. I found that is the default location the API looks in for the file. I set the GuaranteedOutputBuffers in the file and that fixed the issue. I have attached an example of the EmaConfig.xml file I created.

emaconfig.zip (1.0 KiB)

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.