Migrating from RFA to EMA - Dictionary management and access

To address both your recent posts.

As you may have realised EMA loads (or can download for Consumer) a dictionary behind the scenes for internal usage.

For the IProvider implementation, currently it can only use a local file based dictionary for internal usage. You cannot configure EMA IProvider programmatically or via the .xml file to download a dictionary for internal usage..

The purpose of example 350 is for when the developer needs access to the dictionary within his application code. But since the EMA internal dictionary , is not accessible to the developer - they can download it from the ADH as demonstrated in the example. It makes more sense to use the downloaded ADH dictionary within your application code - as you will most likely be using it to encode fields which you will then publish to the ADH.

All comments

moragodkrit.chumsri_1

@Paolo Parlapiano

Basically, you can configure EMA Consumer app to either use a local data dictionary or download it from Interactive Provider or ADS.

But for Interactive Provider, it does not use a dictionary to parse the data. And we do not have EMA Config to set dictionary in the Interactive Provider section. When the provider receives a dictionary request from the consumer side, it has to manually load a data dictionary from files using a DataDictionary class and then use it to publish the dictionary response.

I'm a bit confused about your requirement.

-Do you want to get the dictionary location defined in the EMA config so you can use it to load dictionary files in the Interactive Provider app?

Hi @moragodkrit,
thank you for the update.
Most probably the confusion is actually on my side, so let me try to recap my requirements and my findings on the EMA material.

My application is a generic provider that needs the dictionary to inspect the FIDs information given their names.
The provider must be configurable in terms of the dictionary to be used (file, network, anything else that makes sense for a dictionary).

In both SFC C++ and RFA C++, the data dictionary could be loaded from files or downloaded from the ADH supporting Provider Dictionary Download feature.

Also, the application configuration could be accessed from the application code, therefore I was able to take decisions according to it to handle the dictionary (even using custom configuration entries).

In the TransportAPI_C_DevGuide, I read the following (page 40):

"Chapter 7 Building an OMM Interactive Provider

Provide or download necessary dictionaries
If connected to a supporting ADH, a provider application may also download the RWFFld and RWFEnum dictionaries to retrieve the appropriate dictionary information for providing field list content.
A provider can use this feature to ensure it has the appropriate version of the dictionary or to encode data."

So I understand that still, using ETA the feature of downloading the data dictionary from network is available, also in case of a provider.

In addition, my understanding is that EMA as well would allow to do the same:

in case of a consumer application, the dictionary is automatically loaded by EMA according to the consumer configuration
in case of a provider instead, if the dictionary must be accessed by the application, then the application must take care of loading\downloading the dictionary on its own

Is this correct?
In that case, supposing that my provider must be configurable by the end-user to load the dictionary from file or network, how can I achieve this?
Can the EmaConfig.xml be extended with custom elements?

By the way, why there's an example in EMACPP_ConfigGuide.pdf (page 59) to programmatically configure the Dictionary for a Provider?
Who will use such dictionary configuration?

I'm referring to the excerpt below:

4.5.5 Example: Programmatic Configuration of a Provider

...

innerMap.addKeyAscii("Dictionary_1", ...
...

OmmProvider provider(OmmIProviderConfig().config(outermostMap).operationModel(OmmIProviderConfig::UserDispatchEnum ), appClient );

Thanks a lot

Best Regards,

Paolo

You cannot extend the EmaConfig.xml with custom elements at this point in time.

You could, however:

use your own config file / database / other source
read & parse your own config
populate the Config Programmatically using the values you read from your config (instead of EmaConfig.xml file)

Hi @Umer Nalla,

from what I understand, the EmaConfig.xml is the standard configuration entry point for end users using EMA based applications, defining a well known, documented syntax.
Like for the rfa config or the sslapi.cnf, I expect our users to start using the EmaConfig.xml model.

Having EMA that automatically handles the configuration and loading of the dictionary for a consumer is an extremely powerful feature respecting to legacy Apis like RFA, and the fact that instead a provider, needing the dictionary, must load it on its own is not an issue: as far as I understood we have everything we need to do it programmatically.

But forcing a provider application to move the whole EMA configuration to a custom file simply because it's not possible to configure the source of its data dictionary in the EmaConfig.xml, does not make sense to me. Please feel free to comment.

Is this limit on the configuration extensibility by design, or there are plans to improve it?

Thanks

Regards,
Paolo

With the current EMA Interactive Provider implementation, it only supports local dictionary files - as my colleague has already moragodkrit pointed out in his reply.

I was not suggesting that using Programmatic config would allow you to download dictionaries for IProvider - I was answering your question about custom elements - apologies if I was not clear on that point.
I misunderstood your question to be referring to the ability in RFA that allows you to read your own application level config files / entries - as opposed to the RFA specific config entries.
Certainly for the RSSL / RSSL_PROV type connection, there is no RFA config entry that allows you to specify download or local dictionary files. Any config entry you may have related to this would be an application level config entry, which is then acted upon within the code to decide if you want to load a local dictionary or request a download from the server.

I have checked with the Product Manager and there is nothing currently on the roadmap in terms of allowing download of Data Dictionary from the ADH.

If the flexibility of choosing the source of the dictionary for the Provider is critical to your requirement, then I recommend you consider using ETA.

Certainly, I have not come across any Provider / publisher implementation where the end user determines where the source of the dictionary files. In my experience, it is the developer / owner of the Provider who decides this.

veerapath.rungruengrayubkul

Thanks @umer.nalla for the quick update.
Just a last question that would help me clarify better things: regarding "With the current EMA Interactive Provider implementation, it only supports local dictionary files", what's then the purpose of the example 350__Dictionary__Streaming in the EMA package?

Isn't it an interactive provider downloading the dictionary from the network?
If not, please help me clarify the scenario.

Thank you

Best Regards,

Paolo

@Paolo Parlapiano

From my testing, the example 350_Dictionary_Streaming will download dictionary from supporting ADH via network and then store in DataDictionary class. Application can also use the DataDictinary class to load dictionary from local files.

EMA IProvider by default will load dictionary from files to handle dictionary request from Consumer. Application can control this behavior via OmmIProviderConfig().adminControlDictionary( OmmIProviderConfig::UserControlEnum ) demonstrated in the example 332__Dictionary_UserControl.

Thank you @veerapath.rungruengrayubkul, @umer.nalla, @moragodkrit.chumsri_1,

to summarize: I can download the dictionary from ADH in my provider application, or load it from files. To download it from the network, 350_Dictionary_Streaming example in the SDK is a good starting point.

If I want to make this behavior configurable, I must use a different source respecting to the EmaConfig.xml, as the EmaConfig.xml is meant to host only configuration that is known to EMA iteself.

Please feel free to comment on this if anything is incorrect.

Best Regards,
Paolo

moragodkrit.chumsri_1

@Paolo Parlapiano

Yes, your understanding regarding dictionary management is correct. So if you want to make this behaviour configurable, I would suggest you create your own configuration file such as using JSON file and then your application can read the parameter from JSON data and pass it to your app.

Regarding EMAConfig.xml, as far as I understand, if we add unknown config in the file, EMA will detect it and print some logger message to EMA log file/Console.