question

Upvote
Accepted
sgarland avatar image
16 ●0 ●1 ●4

Select calls and FD_SET grreater than or equal to 1024

Hi,

We would like to know if there is an alternative and to not do select calls?

We would like to know if Reuters is adding a socket to select list for sockets descriptors greater than or equal to 1024?

As shown below in the threading and strace output we believe this is causing us issues:

Thread output:

#20x00007f5ab6c4c71e in onload_select (nfds=1024, rds=0x7f59621fafe0, wrs=0x7f59621faf60, exs=0x7f59621faee0, timeout=0x7f59621fb060) at ../../../../../src/lib/transport/unix/sockcall_intercept.c:1160

lib_context = {thread = 0x7f59621fbb20, saved_errno = 115}

timeout_ms = 1

used_ms = 0

rc =

#30x00007f5aa8385e17 in RTRSelectNotifier::enable() () from /opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libRFA_SessionLayer.so

No symbol table info available.

#40x00007f5aa836d09f in rfa: (worry) essionLayer: (worry) essionImpl::runThread() () from /opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libRFA_SessionLayer.so

No symbol table info available.

#50x00007f5aa8d55f27 in rfa: (worry) upport::Thread: (worry) tartWrapper(void*) () from /opt/srlabs/smds/builds/ms-eur-smds-4.0.8.700//md-3rd/reuters_mapi/libs751/RHEL5_64_GCC412/libDacs.so.7

No symbol table info available.

#60x0000003bb36079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available.

#70x0000003bb32e88fd in clone () from /lib64/libc.so.6 No symbol table info available.

strace output:

No Title

[a.bhise@dnjsvdev03v 0630_mergedProfile_TCP_Not_Acclerated_Breaks]$ grep "10.175.181.40" strace.out

23879 14:13:16.527338 connect(1030, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:13:31.553478 connect(1131, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:13:46.565700 connect(1083, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:01.582052 connect(1162, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:16.599886 connect(1175, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:31.612650 connect(1191, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:14:46.625023 connect(1203, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:15:01.642350 connect(1213, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

23879 14:15:16.667858 connect(1219, {sa_family=AF_INET, sin_port=htons(14002), sin_addr=inet_addr("10.175.181.40")}, 16

Also Just to add some context here, we are seeing reuters making select calls and adding socket to FD_SET higher than 1024. If the library attempts to add an fd that is greater than the limit to the fdset then the behaviour is undefined, and won't necessarily result in what's being passed to the select call itself being wrong, but could, for example, result in other memory in the application being corrupted. Unfortunately, because this happens at fdset creation, so before the call to select, it's not visible at the point at which onload sees the select call, so we can't add instrumentation that would detect this.

That means that we need confirmation from the library whether it may be violating the restrictions on the use of fdset. So it would be helpful if you could confirm with the providers of the library whether or not they may be passing an invalid value to FD_SET.

treprfarfa-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Nipat Kunvutipongsak avatar image   Nipat Kunvutipongsak

Monitored by @veerapath.rungruengrayubkul

chavalit-jintamalit avatar image   chavalit-jintamalit

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query? If yes please click the 'Accept' text next to the reply. This will guide all community members who have a similar question. Otherwise please post again offering further insight into your question. Thanks, AHS

wasin.w avatar image   wasin.w

Hi @sgarland.

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query? If so please can you click the 'Accept' text next to the appropriate reply.

This will guide all community members who have a similar question.

Thanks AHS

Christiaan Meihsl avatar image   Christiaan Meihsl

@sgarland, please be informed that a reply has been verified as correct in answering the question, and has been marked as such. Thanks, AHS

1 Answer

· Write an Answer
Upvotes
Accepted
warat.boonyanit avatar image
4.4k ●10 ●6 ●9

Hi @sgarland

I believe this is a known issue and has already been fixed in RFA C++ 7.6.1.E1

Here is the detail from README

-----------------------------
rfa7.6.1.E1.win-shared
-----------------------------
cxxut0458; RFA fails to connect when more than 1024 FDs are used
RFA application cannot connect to any host when file descriptor number exceed 1024.

RFA now use use fd_set sized to match the maximum number of file descriptors allowed by the operating system.

Are you using the latest version?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

cmason avatar image cmason

Thanks @Warat B

This confirms that RFA was not checking prior to 7.6.1.E1 however the question now is, is there an alternative? Now that RFA checks it'll fail when file descriptor number exceeds 1024 (which is better than hanging in connect) however on a large application there is no guarantee that RFA will be able to connect if >1024 descriptors are already in use. Really an alternative to select such as poll is required.

warat.boonyanit avatar image warat.boonyanit cmason

That's correct @cmason, which is why Elektron API use poll.

If you suspect that your application will use more that 1024 descriptors, I would suggest using Elektron API over RFA.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.