Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • TREP APIs /
avatar image
Question by arvnar2407 · May 22, 2020 at 01:27 PM · javaupatransportbuffers

Java TREP UPA API - Choosing an Optimal Buffer SIze

Hi, my application is using the Java UPA API and requesting data from RWF. I have a large subscription list (~400k) and request data in batches of 2048. I have noticed that with such a large list of subscription, the subscription requests drop some symbols in random every time and end up subscribing to a lesser number of symbols. (~370k)

On Tweaking and increasing the buffer sizes, such as below :

guaranteedOutputBuffers(512);

sysSendBufSize(1024*1024*32);

sysRecvBufSize(1024*1024*32);

numInputBuffers(256);

the application reached a stable state where it does not drop requests.

Is there an optimal way to choose the buffer size so that it can handle future increase in subscription lists?

People who like this

0 Show 1
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
pratik.p.mehta · May 22, 2020 at 02:30 PM 0
Share

@Refinitiv - could you please help understand what these 4 settings mean?guaranteedOutputBuffers, sysSendBufSize , sysRecvBufSize and numInputBuffers.

Which one applies to consumers and servers? So our application connects with TREP using UPA 8.0.0.L1.all.rrg version and as our subscription increases, we notice that the refresh messages get dropped OR we don't even subscribe.

As a consumer, what should be the ideal settings? or how can we dynamically set them?

Our applications run on Unix

1 Reply

  • Sort: 
avatar image
REFINITIV
Best Answer
Answer by jirapongse.phuriphanvichai · May 26, 2020 at 04:12 AM

@arvnar2407

@pratik.p.mehta

SysRecvBufSize and SysSendBufSize control the underlying TCP send and receive buffers of system. They are related to TCP sliding Window which could affect maximum Bandwidth. These settings are for the underlying socket layer. The guideline is to set the receive buffer to match or exceed the send buffer of the source application.

GuaranteedOutputBuffers and NumInputBuffers are used in the UPA (ETA) layer.

GuaranteedOutputBuffers - Specifies the number of available buffers per channel. It is pre-allocated up front regardless of the client or server side. Trade-offs are memory footprint, especially on a server where there are many connections and each has all of this configured and allocated upfront.

NumInputBuffers - Specifies the number of sequential input buffers to allocate for reading data into. This controls the maximum number of bytes that can be handled with a single network read operation. This will more or less correlate to how fast we can pull in data from a full SysRecvBuf from the OS.

The GuarunteedOutputBuffer, SysRecvBufSize, and SysSendBufSize can be changed via the Channel.ioctl method.


I think there is no ideal setting for all consumers. It depends on the behavior of the application.

In this case, I assume that the application may send a lot of subscriptions that the API or TREP is unable to handle requests on time. Therefore, the approach is implementing subscription throttle in the application. For example, the application will not send the next batch request until all responses of the previous batch request are received, or the application can use a timer to send batch requests.



1590466008326.png (80.9 KiB)
Comment

People who like this

0 Show 3 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
pratik.p.mehta · May 26, 2020 at 01:36 PM 0
Share

Thank you for the update, when you say "The guideline is to set the receive buffer to match or exceed the send buffer of the source application." In our use case, it will be the ADS host right? so if Application A is consuming data from ADS, so you suggest to ensure the receive buffer on Application A to match or exceed that of ADS?

avatar image
pratik.p.mehta pratik.p.mehta · May 26, 2020 at 01:40 PM 0
Share

Our ADS buffer sizes are around 64k

*ads*tcpRecvBufSize : 64240

*ads*tcpSendBufSize : 64240

I assume these numbers are in bytes

avatar image
REFINITIV
jirapongse.phuriphanvichai ♦♦ · Jun 01, 2020 at 03:38 AM 0
Share

Yes, it is the ADS host.

tcpRecvBufSize and tcpSendBufSize are in bytes.

Watch this question

Add to watch list
Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
11 People are following this question.

Related Questions

Publishing my application version to TREP

Encode RsslSeries into RsslElementList

Dictionary Id 0 (zero) vs 1 (one)

Error decoding a FieldList's FieldEntry

Proper behavior for an interactive provider upon loss of upstream communication

  • Feedback
  • Copyright
  • Cookie Policy
  • Privacy Statement
  • Terms of Use
  • Careers
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Alpha
  • App Studio
  • Block Chain
  • Bot Platform
  • Calais
  • Connected Risk APIs
  • DSS
  • Data Fusion
  • Data Model Discovery
  • Datastream
  • Eikon COM
  • Eikon Data APIs
  • Electronic Trading
    • Generic FIX
    • Local Bank Node API
    • Trading API
  • Elektron
    • EMA
    • ETA
    • WebSocket API
  • Legal One
  • Messenger Bot
  • Messenger Side by Side
  • ONESOURCE
    • Indirect Tax
  • Open PermID
    • Entity Search
  • Org ID
  • PAM
    • PAM - Logging
  • ProView
  • ProView Internal
  • Product Insight
  • Project Tracking
  • Refinitiv Data Platform
    • Refinitiv Data Platform Libraries
  • Rose's Space
  • Screening
    • Qual-ID API
    • Screening Deployed
    • Screening Online
    • World-Check One
    • World-Check One Zero Footprint
  • Side by Side Integration API
  • TR Knowledge Graph
  • TREP APIs
    • CAT
    • DACS Station
    • Open DACS
    • RFA
    • UPA
  • TREP Infrastructure
  • TRIT
  • TRKD
  • TRTH
  • Thomson One Smart
  • Transactions
    • REDI API
  • Velocity Analytics
  • Wealth Management Web Services
  • World-Check Data File
  • Explore
  • Tags
  • Questions
  • Badges