Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in C++/EMA (in OmmConsumer/OmmLoggerClient) #118

Closed
pabk-ovka opened this issue Sep 18, 2019 · 8 comments
Closed

Memory leak in C++/EMA (in OmmConsumer/OmmLoggerClient) #118

pabk-ovka opened this issue Sep 18, 2019 · 8 comments

Comments

@pabk-ovka
Copy link

It looks like there is a significant memory leak in the C++ version of EMA.
SDK version: 1.3.1.L1 (Latest)

Please consider the following code:

#include "Consumer.h"

using namespace thomsonreuters::ema::access;
using namespace std;

void main( int argc, char* argv[] )
{ 
	OmmConsumer* myConsumer = NULL;

	while(true) {
		myConsumer = new OmmConsumer(
			OmmConsumerConfig()
				.host("123.456.123.456:123")
				.username("USERNAME")
                );

		delete myConsumer;
		myConsumer = NULL;
	}
}

It takes only 27 iterations (create consumer, destroy consumer) to reach roughly 16 GB of memory usage. According to the attached snapshot below it looks like OmmLoggerClient is flawed.

Snapshot

Progress

It should really be possible to open a connection to the data feed and later close it without leaving a memory trace. Especially if the app needs to reopen the connection several times during the app's lifecycle.

@geoffmallard
Copy link
Contributor

@pabk-ovka Thank you for submitting your issue with the detailed information. This will be tracked and once it is addressed, it will be included in a future release.

@soranat
Copy link
Contributor

soranat commented Sep 27, 2019

@pabk-ovka I have ran yours test code for 20 iterations in the while loop to create and destroy OmmConsumer using the LoggerType::File option with Valgrind(Memcheck) tool and let the application exits normally without any interruptions with Ctrl+C. Also, the Consumer application doesn't connect to a provider application and receives the OmmInvalidUsageException for the login timeout.

I haven't found any memory leak related to the OmmLoggerClient as the allocated memory in OmmLoggerClient::openLogFile() is free in OmmLoggerClient::closeLogFile() when the OmmConsumer is being destroyed.

However, I found the following memory leak from the Valgrind which will be fixed in next release.

==2059==
==2059== HEAP SUMMARY:
==2059== in use at exit: 13,600 bytes in 60 blocks
==2059== total heap usage: 6,167,712 allocs, 6,167,652 frees, 2,799,114,950 bytes allocated
==2059==
==2059== 160 bytes in 20 blocks are definitely lost in loss record 1 of 3
==2059== at 0x4C29E63: malloc (vg_replace_malloc.c:309)
==2059== by 0x6215A8: _rsslChannelCopyConnectionList (rsslReactorImpl.h:218)
==2059== by 0x624801: rsslReactorConnect (rsslReactor.c:1113)
==2059== by 0x46C972: thomsonreuters::ema::access::ChannelCallbackClient::initialize(RsslRDMLoginRequest*, RsslRDMDirectoryRequest*, RsslReactorOAuthCredential*) (ChannelCallbackClient.cpp:654)
==2059== by 0x52F354: thomsonreuters::ema::access::OmmBaseImpl::initialize(thomsonreuters::ema::access::EmaConfigImpl*) (OmmBaseImpl.cpp:1285)
==2059== by 0x4159D8: thomsonreuters::ema::access::OmmConsumerImpl::OmmConsumerImpl(thomsonreuters::ema::access::OmmConsumerConfig const&) (OmmConsumerImpl.cpp:25)
==2059== by 0x40CD58: thomsonreuters::ema::access::OmmConsumer::OmmConsumer(thomsonreuters::ema::access::OmmConsumerConfig const&) (OmmConsumer.cpp:23)
==2059== by 0x4084AE: main (Consumer.cpp:47)
==2059==
==2059== 13,440 (5,120 direct, 8,320 indirect) bytes in 20 blocks are definitely lost in loss record 3 of 3
==2059== at 0x4C29E63: malloc (vg_replace_malloc.c:309)
==2059== by 0x668AA4: tunnelManagerOpen (tunnelManager.c:35)
==2059== by 0x624CBC: rsslReactorConnect (rsslReactor.c:1219)
==2059== by 0x46C972: thomsonreuters::ema::access::ChannelCallbackClient::initialize(RsslRDMLoginRequest*, RsslRDMDirectoryRequest*, RsslReactorOAuthCredential*) (ChannelCallbackClient.cpp:654)
==2059== by 0x52F354: thomsonreuters::ema::access::OmmBaseImpl::initialize(thomsonreuters::ema::access::EmaConfigImpl*) (OmmBaseImpl.cpp:1285)
==2059== by 0x4159D8: thomsonreuters::ema::access::OmmConsumerImpl::OmmConsumerImpl(thomsonreuters::ema::access::OmmConsumerConfig const&) (OmmConsumerImpl.cpp:25)
==2059== by 0x40CD58: thomsonreuters::ema::access::OmmConsumer::OmmConsumer(thomsonreuters::ema::access::OmmConsumerConfig const&) (OmmConsumer.cpp:23)
==2059== by 0x4084AE: main (Consumer.cpp:47)
==2059==
==2059== LEAK SUMMARY:
==2059== definitely lost: 5,280 bytes in 40 blocks
==2059== indirectly lost: 8,320 bytes in 20 blocks
==2059== possibly lost: 0 bytes in 0 blocks
==2059== still reachable: 0 bytes in 0 blocks
==2059== suppressed: 0 bytes in 0 blocks

@pabk-ovka
Copy link
Author

@soranat , thank you for the investigation. I am not sure if I understand correctly your statement:

"Also, the Consumer application doesn't connect to a provider application and receives the OmmInvalidUsageException for the login timeout"

In my case, the consumer application connects just fine, does not receive any timeout (Of course the host and password values in the attached code snippet are just an example as I did not want to reveal the real values.). So I am just wondering if we are on the page - running the same scenario.

@soranat
Copy link
Contributor

soranat commented Sep 30, 2019

@pabk-ovka I did test yours scenario as well when the Consumer application can connect with a provider without any OmmInvalidUsageException regarding the login timeout to verify both scenarios.

I notice the memory growth with the consumer application only on Windows platform and the OS releases the memory before the application exits. I will investigate more and get back to you.

@soranat
Copy link
Contributor

soranat commented Sep 30, 2019

@pabk-ovka

I found the root case that the Consumer application consumes memory exponentially because there is a defect in the OmmLoggerClient::openLogFile() method causing the LoggerClientFiles.openFilesSize variable is multiplied by 2 for every additional OmmConsumer instances after destroying the previous one from the following condition and its memory allocation.

if ( clientFiles.openFilesSize )
clientFiles.openFilesSize *= 2;
else
clientFiles.openFilesSize = 4;

clientFiles.openFiles = static_cast<struct LoggerFile *>( realloc(static_cast<void *>( clientFiles.openFiles ), sizeof(struct LoggerFile) * clientFiles.openFilesSize ));

This memory consumption issue will be fixed in the next release as well. Please let me know if you have any questions.

@pabk-ovka
Copy link
Author

@soranat , that's great news. The bug you described exactly matches the symptoms. We are looking forward to the next release. Thank you.

@vlevendel
Copy link
Contributor

@pabk-ovka ESDK1.4.0.L1 has been released and include a fix to address this issue. Please re-open if ou find otherwise. Thank you.

@pabk-ovka
Copy link
Author

@vlevendel , thank you, we'll upgrade to 1.4 and let you know if issue persists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants