question

Upvote
Accepted
MW5 avatar image
16 ●0 ●1 ●3

Memory leak using Reuters RFA7.4.1.L1.win

What appears to be a memory leak using Reuters RFA7.4.1.L1.win.

The issue can be duplicated using the StarterConsumer Example program. We used
the VS100_Shared project. We are using RSSL. We set the \itemList configuration parameter
in the StarterConsumer.cfg file to a list of actively traded instruments:

\itemList = "GE,DIS,AAPL,HD,BAC,T,FCX"

We then start the Release_WIN32_VS100_Shared\StarterConsumer.exe application from the
command prompt and bring up task manager. The memory used by StarterConsumer continues
to grow and never stablizes or shrinks.

We also tried changing: void StarterConsumer::processEvent(const Event& event) so that
it does not call processOMMItemEvent(...) on the theory that this routine (or perhaps
one it calls, like the Decoder) wasleaking the memory.

#ifdef notdef
case OMMItemEventEnum:
  processOMMItemEvent(static_cast<const OMMItemEvent&>(event));
  break;
#endif

The process still leaked memory at the same rate. Note that the amount of memory leaked, and the speed at
which it is leaked is directly related to market activity. The more active the market (and
the instruments being watched), the faster the amount of memory used increases.

We also tried to change the message pool memory parameters from:

\Control\MessagePools\OMMItemEventPool\InitialSize  = 10000
\Control\MessagePools\OMMItemEventPool\Increment  = 10000
\Control\MessagePools\OMMItemEventPool\MaximumSize  = 0

to:

\Control\MessagePools\OMMItemEventPool\InitialSize  = 100
\Control\MessagePools\OMMItemEventPool\Increment  = 100
\Control\MessagePools\OMMItemEventPool\MaximumSize  = 100

This did not change the behavior..

Any help is appreciated.

treprfarfa-apimemory
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Answers

Upvote
Accepted
Jirapongse avatar image
61.7k ●173 ●45 ●65

I got the different result when running StarterConsumer in RFA C++ 7.6.2.L1 package. I modified the code a bit. Instead of decoding the data, I modified it to print the update rate that StarterConsumer could handle and retrieve per second.

...
	int		_updateCount;
	unsigned long long endTime;
...
 bool StarterConsumer::init()
{
	...
	_updateCount = 0;
	endTime = AppUtil::getCurrentTime();
	endTime += 1000;
...
}

void StarterConsumer::processMarketPriceResponse(const rfa::sessionLayer::OMMItemEvent& event, const RespMsg& respMsg)
{
	++_updateCount;
	unsigned long long currentTime = AppUtil::getCurrentTime();
	
	if (endTime <= currentTime)
	{
		RFA_String text("Response Rate: ", 0, false);
		text.append(_updateCount);
		text.append("\n");
		AppUtil::log(__LINE__, AppUtil::TRACE, text.c_str());
		endTime = currentTime + 1000;
		_updateCount = 0;
	}
	return;
…
}

I used the following configurations.

\Control\MessagePools\OMMItemEventPool\InitialSize     = 10000
\Control\MessagePools\OMMItemEventPool\Increment       = 10000
\Control\MessagePools\OMMItemEventPool\MaximumSize     = 0

I connected StarterConsumer to my provider so I could control the update rate sent to the consumer. I have tested it with several updates per second (100, 1000, 10000, and 100000). The result is shown below.

From the result, the memory usage is stable. StarterConsumer without decoding can handle 100000 updates per second without memory growth.

Normally, the memory growth in RFA happens because the application is a slow consumer. For example, if the application is able to handle around hundreds of updates per second but the real update rate of all subscribed items is more than thousands, the memory will definitely grow due to the increasing numbers of unprocessed events in the event queue.

There are two ways to verify if the application is a slow consumer.

  1. Verify the number of updates per second that the application can handle and compare to the real update rate for those items
  2. Or, monitor the return value of EventQueue::dispatch() method. A non-negative return value from dispatch() indicates an estimate of the number of remaining events in the event queue or the event queue group. However, applications should not rely on the exact value as it is only an estimate

memory.png (50.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvote
zoya faberov avatar image
32.2k ●40 ●11 ●19

Would suggest using a newer version of RFA; going with the latest, RFA 8, if possible. Many important fixes have been introduced to RFA in version 7.6 (please check with version READMEs for the specifics).

RFA 8 offers major new functionality, while incorporating fixes from the previous versions.

RFA before 7.6.2 will be obsolete this year (2016), please refer to:

https://community.developers.refinitiv.com/questions/6299/is-there-a-planed-obsolescence-for-the-rfa-c-74-an.html

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sammy.li avatar image sammy.li

I encounter this problem too. I try using 7.6.2 shared or static version. I still get this problem. I download the version 8, but this is only 64 bit version which I need to change a lot for my code. Please help if any work around solution for this. I am working on this problem for nearly 2 months and finally find out this is a bug from your api. Please help.