OmmConsumer registerClient single-threaded?
Does anyone know if the OmmConsumer.registerClient() method is single threaded? The reason for the question is during load testing of our application (EMAJ connecting to EZD), when running a single user (single client), the application runs fine for 1000 iterations at 5 second intervals.
But when we start introducing additional users at random intervals, it looks like requests are fighting for the registerClient resource, causing timeouts and reactor being shut down randomly.
We've been facing this issue in our Production environment and cannot pinpoint the issues we're experiencing. In our Production environment, we have many concurrent requests from all different sources and issues start occurring under heavy load.
Any assistance will much appreciated.
Hi @joe.hong,
unfortunately the EMA OMMConsumer public methods are using very primitive synchronization of a monitor/lock around nearly the complete method body. It is actually very close to have all methods simply synchronized. There is no level of concurrent processing if you call registerClient() from several threads. It could be still somehow acceptable as a simplification but it is not acceptable that the same monitor/lock is used around reading messages from the Reactor channel. More you register, more messages you are receiving, more contention happens between register/unregister/submit methods and the message reading. This design is way away from the RFA API which was/is providing several concurrency models where you could have a dedicated request queue for the register/unregister/submit methods which guaranteed instant processing as there was no contention with the message processing part. This design is actually returning us back to 90´s where the SSL Classic C API was using similar threading model implemented by a global mutex around all C API methods including the dispatch call.
It could be just a bug as there is actually not much need to use the _userLock around the dispatchAll call as the reactor is using its global _reactorLock anyway. But removing the _userLock would not help much as the register/unregister/submit methods are still speaking to the Reactor methods where the _reactorLock is requested anyway. The locking model between the Reactor and OMMConsumer is then very questionable, using basic lock/unlock to protect every method looks old school in the time where other API’s are using techniques like out-of-thin-air/relax reads, spinning, CAS, optimistic reads, memory fences, etc.
Dispatch
{
_userLock.lock();
ret = _rsslReactor.reactorChannel()!= null? _rsslReactor.dispatchAll(null, rsslDispatchOptions, _rsslErrorInfo) : ReactorReturnCodes.SUCCESS;
_userLock.unlock();
}
Register
try {
_userLock.lock();
long handle = _itemCallbackClient != null ? _itemCallbackClient.registerClient(reqMsg, client, null, 0) : 0;
return handle;
} finally {
_userLock.unlock();
}
Best regards,
Michal Frajt
Hi @joe.hong
Based on the API reference guide, the OmmConsumer.registerClient() method is Object Level Safe.
Object Level Thread Safe Method Multiple application threads of control may concurrently call this method on the same instance. Applications will, however, incur the cost of synchronization overhead (e.g., blocking) while calling this method. By synchronizing concurrent access to this method on per instance basis, EMA preserves the state of individual instances and resolves any race condition that may arise while multiple application threads of control concurrently call this method on the same instance.
Regarding the problem, does the application create multiple OmmConsumer instances for each request? What is a version of EMA Java that encounters the problem? Can you replicate the issue in the non-production environment?
Hi @joe.hong
You can submit this problem to developer directly via Github at https://github.com/thomsonreuters/Elektron-SDK/issues link.