RFA connection call

sgarland
sgarland Newcomer

– in one such case we
see that we retry several times when a connection is not available (retying in
1500ms is printed in the logs)



However in our netstat output – we see we do establish a connection – and it
subsequently goes into a close_wait status – ie:



tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002 ESTABLISHED
13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 58 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 61 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 64 0 138.20.27.91:51960
10.175.181.40:14002
ESTABLISHED 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002 CLOSE_WAIT
13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002 CLOSE_WAIT
13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc

tcp 68 0 138.20.27.91:51960
10.175.181.40:14002
CLOSE_WAIT 13608/mdrc



Where it remains there until the process is killed.



In particular we’d like to know what call the RFA API makes when trying to
establish a connection to the EED – is this possible?image

Tagged:

Answers

  • Hi @sgarland

    Could you please show us the logs from rfa.log?

    It should have additional info like this one.

    [Mon Oct 03 11:14:37 2016]: (ComponentName) RSSL_Adapter: (Severity) Error: RSSL Channel read failed on connection "Default::Connection_RSSL 192.168.27.46:14002".  Channel will be closed.  (Internal debug info: "<..\..\..\Ripc\Impl\ripcsrvr.c:6662> Error:1002 ripcRead() failure. System errno: (10053)
    ", errno: RSSL_RET_FAILURE [10053] - )

    The system errno should give you more information.

    On windows, the number is Windows Sockets Error codes

    On Linux, it is errno.h.

  • sgarland
    sgarland Newcomer

    logging we've seen in some cases is like this

    [Wed Apr 19 02:04:40 2017]: (ComponentName)
    RSSL_Cons_Adapter: (Severity) Information: Connection Status Changed,
    RSSL_Cons_Connection "Default::Connection_RSSL 192.168.220.55:14002"
    State: "Down" StatusCode: "None" StatusText: Connection
    down

    [Wed Apr 19 02:04:40 2017]: (ComponentName)
    RSSL_Cons_Adapter: (Severity) Information: RSSL_Cons_Connection
    "Default::Connection_RSSL 192.168.220.55:14002" will be tried in
    15000 milliseconds

    [Wed Apr 19 02:05:25 2017]: (ComponentName)
    RSSL_Adapter: (Severity) Error: Connection "Connection_RSSL" down or
    unavailable

    Possibly due to the provider not accepting client
    session requests.

    [Wed Apr 19 02:05:25 2017]: (ComponentName)
    RSSL_Adapter: (Severity) Warning: RSSL Channel closing on connection
    "Default::Connection_RSSL 192.168.220.55:14002" due to
    "connection channel init failed"


    Other times we get nothing and the connection seems to hang.


    We're using solarflares onload, so we're interesting in the actual call we make to connect

  • For RSSL connection, RFA use socket api, e.g. connect() select() etc..

    I'm not sure how solarflares' onload work under the hood, but does the application work on a machine that doesn't have onload?

  • sgarland
    sgarland Newcomer

    @Warat B - yes, disabling onload and running without this allows RFA to initialize correctly.

    In RFA which function call is reporting that no data is ready; when
    there actually is data ready?

    do you using level, or edge triggers?

    Is the socket directly in the set; or is it in some other
    waitable object first?

  • Hi @sgarland

    Are you on Linux? You can get the function call by using strace.

    Unlike Elektron API, RFA is a proprietary software.

    However, if you have specific questions from strace, I can pass the questions to development.