Value of configuration parameter "SysSendBufSize" in EmaConfig.xml differs from ChannelInformation

Hello Michael @miczuc,

I have tested this behavior on Windows. The library code base is shared between the supported OSs, excluding few OS-specific aspects, where there may possibly be a difference.

I have re-tested on Linux, and the behavior that I observe is not clear to me either.

I will verify the behavior with Elektron SDK development, look out for the updates here.

Please be assured, that our goal is to help you, and next developers with the same or similar questions.

All comments

Hello @miczuc,

I am trying to better understand the question in order that we try to answer it.

On Server parameter you are requesting to set send buffer size to your infra server (ADH) by your provider to 1000. The channel is established between the provider and the infra component being accessed.

Are you looking to set SysReceiveBufferSize for the Consumer Channel? This setting may be useful, if you are looking to send from your consumer very large numbers of requests, including posts?

Or you looking to perchance increase SysRecvBufSize on your Consumer Channel?

Does this help, or are you trying to do something different, then please explain more.

I have a direct connection between my interactive provider and the consumer applications (like in the training programs). There is no ADH involved.

The "ChannelInformation" is a C++ API class in the EMA library to obtain information about a connected consumer.

Is has a field "_sysSendBufSize" and getter method getSysSendBufSize().

Now, in the provider's EmaConfig.xml, I set the configuration parameter "<SysSendBufSize>" in the <Server> element, which is assigned to my interactive provider within the <IProvider> element, to different values and look, what values for "_sysSendBufSize" are reported by the "ChannelInformation", when a consumer connects.

These are the results:

Default (<SysSendBufSize> not set): 131070

<SysSendBufSize> set to 1000 : 4608

So the actual question is: what does the "_sysSendBufSize" in the ChannelInformation mean and how is it related to the configuration parameter "<SysSendBufSize>".

I would expect, they should be identical. But that's obviously not the case ...

Maybe better go to GitHub site ...

Hello @miczuc,

Your question makes much better sense to me now.

Could you please provide the complete details, allowing us to try to reproduce what you are observing:

What version of SDK
On what platform
Which examples are you running
Are you running our examples as is, or modifications has been made,
Any other relevant details, that will help us to reproduce what you observe quickly and efficiently will be be tremendously helpful.

@miczuc,

While awaiting your information,

Let me try to provide some additional information:

Provider send buffer size (server) is different from consumer send buffer size (Channel). The channel from the provider side is different from the channel on the consumer side, even if the two are hard-wired to talk to each other directly, they are two separate channels, each having it's own send and receive buffer.

In some SDK versions' examples getChannelInfo methods, you may find two SendBuffSizes printed one after the other. If this is the case for you, the second one, if you examine the code, is RecvBuffSize and not Send

Of course there is a receive buffer size as well.

This is e.g. the output of ChannelInformation toString():

hostname: c7home
IP address: 127.0.0.1
connected component info: eta3.4.0.L1.rrg|ema3.4.0.L1.linux 64-bit Static(Debug)
channel state: active
connection type: socket
protocol type: Reuters wire format
major version: 14
minor version: 1
ping timeout: 255
max fragmentation size: 6142
max output buffers: 5000
guaranteed output buffers: 5000
number input buffers: 10
system send buffer size: 200000
system receive buffer size: 131070
compression type: none
compression threshold: 0
encryption protocol: 0

You mean, the the send buffer size is that of the client, the "remote send buffer size".

Could be. However, the value of "system send buffer size" depends also on the setting of <SysSendBufSize> on the provider side. In the example above I set
<SysSendBufSize value="100000"/> on the provider side. If you leave this parameter at it default on the provider side, you get 131070 for "system send buffer size".

I just want to know, what the value "system send buffer size" and
"system receive buffer size" exactly mean and what the units for these values are (without unit the numbers are useless), and how they are related to the configuration parameters.

Sorry, but this seems to be waste of time ...

Hello.

I found some useful information at question "Recv buffer size and system limits in ETA".

The system call setsockopt(SO_SNDBUF, ...) doubles the value for internal reasons, and also uses an internal minimum value.

So the "system send buffer size" in the channel information is indeed that of the provider's end (modified by the setsockopt()). However the question for the unit remains. In your Configuration Guide you can read for parameter "SysSendBufSize":

"Specifies the size (in KB) of the system's send buffer for this channel."

Does that mean, these values (131070 is shown for default configuration)

system send buffer size: 200000
system receive buffer size: 131070

are in KB?

Hello @miczuc,

Is my understanding correct, that you are running ESDK 1.4 on Linux

PerfTools:

ETA->ProvPerf

EMA->EmmCppConsPerf

Hello @miczuc,

The buffer sizes are in KB.

From the description of configuration in use, and from the output observed, the channel on which you set the SysSendbuffer size is not the same one on which you check/sample it.

If both your components are EMA components, and both read and their config info from EMA config file, no hard-coding:

Please try setting desired size of SysSendBufSize on <Channel> not on <Server>.

Perhaps it is worth taking a look at GuaranteedOutputBuffers parameter, which is within the app's control, for high message volume applications this is the parameter that is most commonly increased. The description of this parameter can also be found in EMA C++ Config Guide.

Let us know is this is what you are looking to configure?

Hello Zoya.

What you wrote is wrong. It is the same channel (socket) for which the send buf size is set and get.
As explained in my previous note, the phenomenon results from the behaviour of the setsockopt() call
for SO_SNDBUF (on linux platform CentOS 7). The value obtained by getsockopt() differs from the value
set with setsockopt(). I made a little C test program to verify that and indeed get exactly the same
numbers reported in your ChannelInformation:
set 1000 -> get 4608
set 100000 -> get 200000

So that is the explanation (for me, maybe not for you) how the numbers in the ChannelInformation
depend on the configuration parameter "SysSendBufSize" in the <Server>-Element used by the interactive
provider.

However, these number are measured in bytes, not kilo bytes.
So the statement for parameter "SysSendBufSize" in your ELEKTRON MESSAGE API CONFIGURATION GUIDE -
"Specifies the size (in KB) of the system's send buffer for this channel." -
is wrong, respectively the implementation in the EMA C++ library used the value as number of bytes,
not kilo bytes.

(By the way: It is now the third time, that the submit of my note was answered by your server
with an error page "Unable to execute your request". As a result, the text I entered was lost,
and I had to write my text again. That is simply annoying. One must learn to write notes
in a separate text editor before posting them on your web site ...)

I don't want to waste any more on time on this subject, do what ever you like with this question.

Regards, Michael

Hello Michael @miczuc,

Sorry you feel this way.

Firstly, about the AnswerHub bug "unable to execute your request". It has been reported to our provider AnswerHub a while back and is still under works by them, unfortunately. I totally agree it is very annoying, as after spending a couple of hours on a detailed answer to somebody, having it disappear is totally unacceptable. The real solution is still in the works, according to AnswerHub.

For now, we, moderators:

Try to refresh the page before submitting, to make sure we are logged in, the lgin expires leading to multiple data-loss issues on AnswerHub.
Always copy the answer text to clipboard prior to hitting Submit.
(Wow, usually this type of issues is only happening to moderators, who work on answers long, "the primary recipients", so to speak, as usages by other members are shorter)

Hello @miczuc,

Following your last comment I have reviewed our discussion and have agreed with you, that something does not add up, and the conclusion does not follow.

I have suspected that was because I have tested with ESDK 1.3.

I have re-tested with ESDK 1.4 as follows:

1. Run IProvider 100 example. My modifications to it were:

1a.Point to a specific emaConfig.xml, so I know I am reading the exact configuration that I am modifying

    OmmProvider provider( OmmIProviderConfig("C:\\esdk14\\Cpp-C\\Ema\\Examples\\").port( "14002" ), appClient );

1b. I have modified the prints in callbacks to reflect buffer info

void AppClient::processMarketPriceRequest( const ReqMsg& reqMsg, const OmmProviderEvent& event )
{
    if ( itemHandle != 0 )
    {
        processInvalidItemRequest(reqMsg, event);
        return;
    }
    cout << "*** consumer refresh event ***" << event.getChannelInformation() << endl;
...

2. Run Consumer MarketPrice 100 example. My modifications to it were in callbacks, for visibility:

void AppClient::onRefreshMsg( const RefreshMsg& refreshMsg, const OmmConsumerEvent& event) 
{
    cout << refreshMsg << endl;        // defaults to refreshMsg.toString()
    cout << "*** consumer refresh event ***" << event.getChannelInformation() << endl;
}

void AppClient::onUpdateMsg( const UpdateMsg& updateMsg, const OmmConsumerEvent& ) 
{
//    cout << updateMsg << endl;        // defaults to updateMsg.toString()
}

Upon run without any modifications to configs, I get default values of buffers printed:

Prov
max output buffers: 5000
guaranteed output buffers: 5000
system send buffer size: 65535
system receive buffer size: 65535

Cons
max output buffers: 100
guaranteed output buffers: 100
system send buffer size: 65535
system receive buffer size: 65535

Consequently I modified emaConfig.xml as you did

Server -> <SysSendBufSize value="1000"/>

And did a test run with the same code, resulting in output:

Prov
 max output buffers: 5000
 guaranteed output buffers: 5000
 system send buffer size: 1000
 system receive buffer size: 65535

Cons
max output buffers: 100
guaranteed output buffers: 100        
system send buffer size: 65535
system receive buffer size: 65535

As the configuration modification was reflected correctly in EMA sys send buffer size,

I think the best approach would be to start with same simple examples, see if on examples, using ESDK 1.4, you get the buffer size set correctly.

If you do, look into your code and compare for differences. If you don't, make sure you are on the latest SDK and the matching libraries (perhaps like me you have many different ones installed in your development environment) and verify the configuration you use.

Let me know if this helps?

Hello.

I use ESDK 1.4.

Compile, link and run your provider program on Linux. Guessing from your output, you run it on Windows (there the default TCP send buffer size is 65535).
On Windows getsockopt(.. SO_SNDBUF ..) returns exactly the value set before with setsockopt(.. SO_SNDBUF ..), so you cant observe the different values there.

However, the number is in bytes, not KB. So the statement in the configuration guide is wrong, or the implementation uses bytes and not KB. This is the problem and it is still not answered.

The difference between the configured send buf size and the value reported in the ChannelInformation on Linux results from the bevaviour of setsockopt() for SO_SNDBUF on Linux, I say this for the third time now ...

By the way: Why can't I see my questions in "My Q&A" ?

Regards, M. Zuck

@miczuc,

Unfortunately, questions will not show in "My Q&A" due to a known bug from AnswerHub.

Use "My Profile" to see your questions, answers and comments

Hello Michael @miczuc,

I have tested this behavior on Windows. The library code base is shared between the supported OSs, excluding few OS-specific aspects, where there may possibly be a difference.

I have re-tested on Linux, and the behavior that I observe is not clear to me either.

I will verify the behavior with Elektron SDK development, look out for the updates here.

Please be assured, that our goal is to help you, and next developers with the same or similar questions.

mitchell.kato

According to the socket(7) man page, Linux will double the value set with SO_SNDBUF and SO_RCVBUF on setsockopt(). This is an OS behavior. ESDK is just setting the value here as is from the user, and it is up to the various OS' implementations for their exact behavior.

http://man7.org/linux/man-pages/man7/socket.7.html

Hello Mitchell.

You are repeating what I already said. This is clear already. The question about the unit of parameter "<SysSendBufSize>" is still open. Obviously the value is applied as number of bytes, not kilo bytes.

Regards, Michael Zuck

Hello @mitchell.katoTR

Could you answer the question about the unit of parameter "<SysSendBufSize>" ?

Hello @miczuc
Please be informed that you can also submit the ESDK question in GitHub issues page directly.

Hello @miczuc

I saw that you have submit the question in GitHub issue #137, the issue has been fixed. Did you have a chance to check the latest version of the SDK?