question

Upvotes
Accepted
217 7 19 35

RFA API behavior when application thread is blocked during market Data processing

Hello, we are using the RFA 8.2 C++ API to consume market data. In our application, the Driver Thread utilizes the RFA libraries to request data and receive callbacks from TREP through a TCP connection. If the Driver Thread becomes blocked and is unable to process incoming data, will the RFA libraries continue to receive and buffer data until the Driver Thread can resume processing, or will they stop receiving data from TREP if the Driver Thread is blocked? What is the expected behavior in this situation?

#technologyrfa-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
24.8k 64 15 21

Hello @mktdata,

The application behavior will depend on the data load on the application. There are internal buffers within the SDK and RTMDS which will start filling up, if the app thread does not empty the received messages fast enough. If this continues for too long, these buffers will eventually overflow, and the infrastructure will disconnect the offending consumer app.

This scenario is called a slow consumer.

If however, the data update rate is low, and the driver thread is blocked rarely, the inbuilt buffers should be able to handle this temporary disruption.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hello @Gurpreet , how big these buffers can grow in size at max before consumer app is disconnected? I am not looking for exact number but just indicative figures. Will that grow in GB's?
Upvotes
24.8k 64 15 21

The buffers in the RFA and the RTMDS are configurable. You can read about these parameters in the RFA configuration and logging guide, packaged with the SDK. Your market data admin should be able to configure the buffers on the infrastructure side.

Important thing to note is that - if the consumer is really slow and unable to process messages fast enough, then no amount of buffers can fix this situation. Having very large buffers will increase the latency in the application update rate, and the consumer will disconnect after a bit of time anyways.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
86.3k 292 53 79

@mktdata

Thank you for reaching out to us.

It depends on the thread model that you are using. Please check the section 17.4 Configuring the RFA Consumer for Performance in the RFA developer guide.

If the application uses Single Threaded With an Event Queue (Client Model), a RFA thread will push events in to an event queue while an application thread is busy or blocked.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.