question

Upvotes
Accepted
26 2 5 6

EMA Java parameter GuardanteedOutputBuffers change not displayed in log or ADHMON

Hi I am running a RMDS publisher using Java EMA 3.0.45. We are seeing publisher disconnects with error:

 error:Failed to submit UpdateMsg. Reason: ReactorReturnCodes.NO_BUFFERS. Error text: channel out of buffers 


We have tried to fix this issue by increasing the setting "GuaranteedOutputBuffers" as recommended in the EMAJ_ConfigGuide. However when I do this the parameter change doesn't appear in adhmon or the application log. In fact the adhmon for my publisher shows GuaranteedOutputBuffers=200 which is neither the default nor the value set by my app. I've attached screenshots of adhmon of 1) RSSL Source IPC Statistics->IPC Statistics->page 3 its not clear what this value represents - is it the adh default? 2) RSSL Source IPC Statistics->My Session->IPC Session this is my publisher. I've highlighted the values I think are important on both screenshots and removed ipaddresses. Can you comment please?

ss-1.jpg


ss-2.jpg

Thanks

ema-apiADHpublishing
ss-1.jpg (127.4 KiB)
ss-2.jpg (212.0 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
49.4k 123 44 61

@duncan_kerr

I assumed that you are using non-interactive providers.

Those values are from the guaranteedOutbuffers and maxOutputBuffers in ADH configurations.

replablinux4*adh*rmds_rssl_source*guaranteedOutputBuffers : 2000
replablinux4*adh*rmds_rssl_source*maxOutputBuffers : 4000

1660105139957.png

1660105165958.png

Therefore, these values are the size of buffers on ADH.


1660105139957.png (12.7 KiB)
1660105165958.png (46.8 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
26 2 5 6

How do I set guaranteedOutputBuffers for just one publisher in the ADH config file?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
26 2 5 6

I am trying to set GuaranteedOutputBuffers programatically using a hardcoded EMAConfig.xml file. When I set the parameter I can never see the per-publisher GuaranteedOutputBuffers value change in adhmon, or the application log. Is this expected, or would you expect to see the correctly configured application's entry in adhmon, and display the new value of GuaranteedOutputBuffers ?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvote
49.4k 123 44 61

@duncan_kerr

I don't think that the API advertises the number of GuaranteedOutputBuffers to another end (ADH). The number of GuaranteedOutputBuffers is used internally by the application when sending the data. Each channel has its own GuaranteedOutputBuffers according to its setting.

1660183088224.png

You can verify the settings by enabling logging in EMA Java.

1. Create a logging.properties file with the following content

.level=FINEST
 
#handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=FINEST
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
 
#java.util.logging.FileHandler.level=FINEST
#java.util.logging.FileHandler.pattern=./emaj.log
 
# Write 100000 bytes before rotating this file
#java.util.logging.FileHandler.limit=50000000

# Number of rotating files to be used
#java.util.logging.FileHandler.count=20
#java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
# Format timestamp as date/time with millisecond
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL %4$-7s %2$s %n%5$s

2. Run an application with the following option: -Djava.util.logging.config.file=logging.properties

You will see the following log.

2022-08-11 08:41:29.544 FINEST  com.refinitiv.ema.access.ChannelCallbackClient initializeReactor 
loggerMsg
    ClientName: ChannelCallbackClient
    Severity: Trace
    Text:    Attempt to connect using 
    1] SOCKET
    Channel name Channel_7
    Instance Name Provider_1_1
    RsslReactor @600b9d27
    InterfaceName 
    hostName 127.0.0.1
    port 14003
    CompressionType None
    tcpNodelay true
    reconnectAttemptLimit -1
    reconnectMinDelay 1000 msec
    reconnectMaxDelay 5000 msec
    guaranteedOutputBuffers 5000
    numInputBuffers 10
    sysRecvBufSize 65535
    sysSendBufSize 65535
    connectionPingTimeout 30000 msec
    initializationTimeout 5 sec
    
loggerMsgEnd

1660183088224.png (19.4 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
26 2 5 6

I see that makes sense. A couple more questions:

  • We have also been using DirectWrite=1. Am I right in thinking that DirectWrite concerns buffering on the NIC, and that even with DirectWrite=1, the buffers configured with GuaranteedOutputBuffers are still used? Or do DirectWrite and GuaranteedOutputBuffers refer to the same buffers?
  • Is there are maximum value for GuaranteedOutputBuffers? We are running a high volume publisher which is configured as follows, and disconnects every 5 minutes with an out of buffers error (see above). Are these setting ok?
                <ChannelType value="ChannelType::RSSL_SOCKET"/>
                <GuaranteedOutputBuffers value="125000"/>
                <NumInputBuffers value="2048"/>
                <ConnectionPingTimeout value="30000"/>
                <TcpNodelay value="0"/>
                <DirectWrite value="1"/>
                <HighWaterMark value="6144"/>
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
49.4k 123 44 61

@duncan_kerr

Typically, when sending data, EMA gets an output buffer from the transport layer, decodes data into that buffer, and submits it to a queue in the transport layer. The buffer will be released back to the transport layer after the content in the buffer was sent to the network. The DirectWrite will attempt to pass the contents of the buffer directly to the socket write operation, bypassing any transport queuing. Yes, DirectWrite and GuaranteedOutputBuffers refer to the same buffers.

As I know, there is no maximum value for GuaranteedOutputBuffers. However, the size of each buffer is 6144 bytes (maxFragmentSize). Therefore, if you set the value of GuaranteedOutputBuffers to 125000, it will reserve around 768 MB for output buffers (125000 x 6144).


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.