Select calls and FD_SET grreater than or equal to 1024

sgarland

Hi,

We would like to know if there is an alternative and to
not do select calls?

We would like to know if Reuters is adding a socket to
select list for sockets descriptors greater than or equal to 1024?

As shown below in the threading and strace output we
believe this is causing us issues:

Thread output:

#20x00007f5ab6c4c71e in onload_select (nfds=1024, rds=0x7f59621fafe0,
wrs=0x7f59621faf60, exs=0x7f59621faee0, timeout=0x7f59621fb060) at
../../../../../src/lib/transport/unix/sockcall_intercept.c:1160

lib_context
= {thread = 0x7f59621fbb20, saved_errno = 115}

timeout_ms
= 1

used_ms = 0

rc =

#30x00007f5aa8385e17 in RTRSelectNotifier::enable() () from
/opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libRFA_SessionLayer.so

No symbol table info available.

#40x00007f5aa836d09f in rfa: (worry) essionLayer: (worry)
essionImpl::runThread() () from
/opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libRFA_SessionLayer.so

No symbol table info available.

#50x00007f5aa8d55f27 in rfa: (worry) upport::Thread: (worry)
tartWrapper(void*) () from
/opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libDacs.so.7

No symbol table info available.

#60x0000003bb36079d1 in start_thread () from /lib64/libpthread.so.0 No
symbol table info available.

#70x0000003bb32e88fd in clone () from /lib64/libc.so.6 No symbol table
info available.

strace output:

No Title

[a.bhise@dnjsvdev03v
0630_mergedProfile_TCP_Not_Acclerated_Breaks]$ grep "10.175.181.40"
strace.out

23879 14:13:16.527338 connect(1030, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:13:31.553478 connect(1131, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:13:46.565700 connect(1083, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:01.582052 connect(1162, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:16.599886 connect(1175, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:31.612650 connect(1191, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:46.625023 connect(1203, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:15:01.642350 connect(1213, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:15:16.667858 connect(1219, {sa_family=AF_INET,
sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

Also Just to add some context here, we are seeing reuters
making select calls and adding socket to FD_SET higher than 1024. If the
library attempts to add an fd that is greater than the limit to the fdset then
the behaviour is undefined, and won't necessarily result in what's being passed
to the select call itself being wrong, but could, for example, result in other
memory in the application being corrupted. Unfortunately, because this happens
at fdset creation, so before the call to select, it's not visible at the point
at which onload sees the select call, so we can't add instrumentation that
would detect this.

That means that we need confirmation from the library
whether it may be violating the restrictions on the use of fdset. So it would
be helpful if you could confirm with the providers of the library whether or not
they may be passing an invalid value to FD_SET.

warat.boonyanit

Hi @sgarland

I believe this is a known issue and has already been fixed in RFA C++ 7.6.1.E1

Here is the detail from README

-----------------------------
rfa7.6.1.E1.win-shared
-----------------------------
cxxut0458; RFA fails to connect when more than 1024 FDs are used
RFA application cannot connect to any host when file descriptor number exceed 1024.

RFA now use use fd_set sized to match the maximum number of file descriptors allowed by the operating system.

Are you using the latest version?

cmason

https://community.developers.refinitiv.com/discussion/comment/15468#Comment_15468

Thanks @Warat B

This confirms that RFA was not checking prior to 7.6.1.E1
however the question now is, is there an alternative? Now that RFA checks it'll
fail when file descriptor number exceeds 1024 (which is better than hanging in
connect) however on a large application there
is no guarantee that RFA will be able to connect if >1024 descriptors are already in use. Really an alternative to select such as poll is required.

warat.boonyanit

https://community.developers.refinitiv.com/discussion/comment/15493#Comment_15493

That's correct @cmason, which is why Elektron API use poll.

If you suspect that your application will use more that 1024 descriptors, I would suggest using Elektron API over RFA.