question

Upvotes
Accepted
3 1 0 2

How to decode enums starting with # using the Java SDK ?

Hi,

I am using the Java version of the Elektron SDK (1.2.2) and am having trouble dealing with these enums.

I am mostly using the EMA library with the FieldEntry:enumDisplay() function to get a string version of the enum. I have also tried to run the outputted string through the RmtesDecoder, and convert to a UCS2 string using the UPA library.

Neither seem to convert the string to something meaningful ( eg I can't get 0x e2 87 a7 from #DE# ) and I seem to get the same string for an uptick or a downtick for fid ACT_TP_1 even though the enumtype.def shows different values.

Any idea how I can get the correct enum display value, even if it's the hexadecimal string that represents it ? I've seen a few questions related to the C++ library but none that show the Java alternative.

Thanks.

elektronrefinitiv-realtimeelektron-sdkema-apirrtelektron-message-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

This questions has been monitored by Veerapath

Upvote
Accepted
11.3k 25 9 14

Hi @vinod.hirani

I am able to replicate the issue in Oracle Linux 6. The issue is that the enumDisplay() returns incorrect value because of default character encoding (in my testing, it is "UTF8"). Previously, I test the issue on Windows with Eclipse which sets "ISO-8859-1" by default, so it returns data correctly.

You can set the Character encoding by providing the file.encoding system property when JVM starts e.g. java -Dfile.encoding="ISO-8859-1"...

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
11.3k 25 9 14

Hi @vinod.hirani

The enums starting with # means that the hexadecimal value is in RMTES encoding. In this case, the FieldEntry:enumDisplay() returns string contains the hexadecimal value, so you need to convert it to UCS2 using ETA RmtesDecoder. Please note that the #DE# will be converted to "0x21 e7" in UCS2 which means ⇧ according to this table.

Below is the sample code.

/* create cache buffer for storing RMTES and applying partial updates */
RmtesCacheBuffer rmtesCache = CodecFactory.createRmtesCacheBuffer(100);
/* create RmtesBuffer to convert into */
com.thomsonreuters.upa.codec.RmtesBuffer rmtesBuffer =CodecFactory.createRmtesBuffer(100);
/* create RmtesDecoder used for the decoding process */
RmtesDecoder decoder = CodecFactory.createRmtesDecoder();
/*Our Buffer of data we are converting */
Buffer data = CodecFactory.createBuffer();
String enmStr = fieldEntry.enumDisplay();
data.data(java.nio.ByteBuffer.wrap(enmStr.getBytes(java.nio.charset.StandardCharsets.ISO_8859_1)));
// apply RMTES content to cache, if successful convert to UCS-2 
if ((retVal = decoder.RMTESApplyToCache(data, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error while applying to cache 
    System.out.println("Error encountered while applying buffer to RMTES cache. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else if ((retVal = decoder.RMTESToUCS2(rmtesBuffer, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error when converting 
    System.out.println("Error encountered while converting from RMTES to UCS-2. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else
{
 //SUCCESS: Conversion was successful – application can now use converted content stored in rmtesBuffer 
    System.out.println(rmtesBuffer.toString());
}

de.png (2.6 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi Veerapath,

Thanks for that. I used that approach as described in the upa manual.

//SUCCESS: ....

String rmStr = rmtesBuffer.toString();

rmStr.chars().forEach( x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.println(rmStr);

results in

\u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000 ?

So the first two characters are space and a question mark U+0020 and U+003F.

I am being cheeky and running this on linux with OpenJDK and Ubuntu which I know are not officially supported. Do you get the correct U+21E7 character if you run it on a linux system with Oracle JDK and RHEL ?

Thanks,

Vinod

Upvotes
11.3k 25 9 14

Hi @vinod.hirani

I have just tried your code and can get result properly. Could you add code to log the enumeration string returned from the enumDisplay() function, and then share both output and your code?

What if, you can do more test on the supported platform?

Example code:

String enm = fieldEntry.enumDisplay();
System.out.print("Enum String: ");
enm.chars().forEach( x -> {
	System.out.format("\\u%04x ",(int) x);
});

Output sample:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value:
Enum String: \u0042 \u00de 
\u0042 \u21e7 \u0000 \u0000 \u0000 \u0000 \u0000
Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 
Enum String: \u0042 \u00fe 
\u0042 \u21e9 \u0000 \u0000 \u0000 \u0000 \u0000
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
3 1 0 2

Thanks Veerapath,

I've run the code and it does look like the problem is with my version Java and OS. I will see if I can run this on a supported platform. What platform did you successfully run it on btw ? Code and result is below.

Code:

System.out.print("Fid: " + entry.fieldId() + " Name = " + entry.name() + " DataType: " + DataType.asString(entry.load().dataType()) + " Value: " + entry.enumValue() + "\n");

String enm = entry.enumDisplay();

System.out.println(enm);

enm.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

}); System.out.print("\n");

// SUCCESS: Conversion was succesful - application can now use converted content stored in rmtesBuffer

String rmStr = rmtesBuffer.toString();

System.out.println(rmStr);

rmStr.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.print("\n");

Result:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 2

\u0020 \ufffd

? \u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 3

� \u0020 \ufffd

? \u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@vinod.hirani

I'm sorry that my reply is not clear. I forgot to mention that I didn't try the code with OpenJDK. I just verify your code on my environment.

I'm looking into your code and will update you again.

vinod.hirani avatar image vinod.hirani veerapath.rungruengrayubkul

Hi Veerapath,

I understand you didn't try it with OpenJDK. What platform and JVM did you use for your testing ?

Thanks,

Vinod

Upvotes
3 1 0 2

Hi Veerapath,

Excellent. Using the new default character encoding I see the right unicode \u21e7 and \u21e9 ( after using the RMTESDecoder). So the result now ( using the same code as before with the -Dfile.encoding="ISO-8859-1" ) gives the result :

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 2

\u0020 \u00de

?

\u0020 \u21e7 \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 3

\u0020 \u00fe

?

\u0020 \u21e9 \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Thanks very much that's been a great help !!

Vinod

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@vinod.hirani

Thank you for confirmation. The setting character encoding could be just a workaround for this issue, since this method affects whole character encoding of the application. I will pass this query to EMA development and will keep you posted via email.

vinod.hirani avatar image   vinod.hirani veerapath.rungruengrayubkul

Thanks Veerapath, please keep me in the loop of any EMA changes via email.

Upvotes
1 0 0 0
Hi,@veerapath.rungruengrayubkul:
Because there are other services running on the system, we cannot modify the jvm encoding (the default is UTF-8), use: new String(fieldEntry.enumDisplay().getBytes("UTF-8"), "ISO-8859-1" ) Still garbled, what should I do?
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
9.6k 10 7 7

Hello @vinod.hirani

This EMA issue has been fixed in EMA 3.6.3.0 shipped with Real-Time-SDK-2.0.3.L1.java.zip as mentioned in CHANGELOG.md shown below:

Both ETA Java and EMA Java 3.6.3.L1 Issues Resolved

---------------------------------------------------

- [RTSDK-2661, RTSDK-5320] - Enhance ETA to not depend on the OS Character encoding for displaying non-ascii enum values [Case Number: 07311614, 10089325]

- [RTSDK-3966] - Support SO_REUSEADDR to permit server side socket to be reused for loadbalancing

...

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.