How to decode enums starting with # using the Java SDK ?

I am able to replicate the issue in Oracle Linux 6. The issue is that the enumDisplay() returns incorrect value because of default character encoding (in my testing, it is "UTF8"). Previously, I test the issue on Windows with Eclipse which sets "ISO-8859-1" by default, so it returns data correctly.

You can set the Character encoding by providing the file.encoding system property when JVM starts e.g. java -Dfile.encoding="ISO-8859-1"...

All comments

The enums starting with # means that the hexadecimal value is in RMTES encoding. In this case, the FieldEntry:enumDisplay() returns string contains the hexadecimal value, so you need to convert it to UCS2 using ETA RmtesDecoder. Please note that the #DE# will be converted to "0x21 e7" in UCS2 which means ⇧ according to this table.

Below is the sample code.

/* create cache buffer for storing RMTES and applying partial updates */
RmtesCacheBuffer rmtesCache = CodecFactory.createRmtesCacheBuffer(100);
/* create RmtesBuffer to convert into */
com.thomsonreuters.upa.codec.RmtesBuffer rmtesBuffer =CodecFactory.createRmtesBuffer(100);
/* create RmtesDecoder used for the decoding process */
RmtesDecoder decoder = CodecFactory.createRmtesDecoder();
/*Our Buffer of data we are converting */
Buffer data = CodecFactory.createBuffer();
String enmStr = fieldEntry.enumDisplay();
data.data(java.nio.ByteBuffer.wrap(enmStr.getBytes(java.nio.charset.StandardCharsets.ISO_8859_1)));
// apply RMTES content to cache, if successful convert to UCS-2 
if ((retVal = decoder.RMTESApplyToCache(data, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error while applying to cache 
    System.out.println("Error encountered while applying buffer to RMTES cache. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else if ((retVal = decoder.RMTESToUCS2(rmtesBuffer, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error when converting 
    System.out.println("Error encountered while converting from RMTES to UCS-2. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else
{
 //SUCCESS: Conversion was successful – application can now use converted content stored in rmtesBuffer 
    System.out.println(rmtesBuffer.toString());
}

Hi Veerapath,

Thanks for that. I used that approach as described in the upa manual.

//SUCCESS: ....

String rmStr = rmtesBuffer.toString();

rmStr.chars().forEach( x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.println(rmStr);

results in

\u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000 ?

So the first two characters are space and a question mark U+0020 and U+003F.

I am being cheeky and running this on linux with OpenJDK and Ubuntu which I know are not officially supported. Do you get the correct U+21E7 character if you run it on a linux system with Oracle JDK and RHEL ?

Thanks,

Vinod

I have just tried your code and can get result properly. Could you add code to log the enumeration string returned from the enumDisplay() function, and then share both output and your code?

What if, you can do more test on the supported platform?

Example code:

String enm = fieldEntry.enumDisplay();
System.out.print("Enum String: ");
enm.chars().forEach( x -> {
	System.out.format("\\u%04x ",(int) x);
});

Output sample:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value:
Enum String: \u0042 \u00de 
\u0042 \u21e7 \u0000 \u0000 \u0000 \u0000 \u0000
Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 
Enum String: \u0042 \u00fe 
\u0042 \u21e9 \u0000 \u0000 \u0000 \u0000 \u0000

Thanks Veerapath,

I've run the code and it does look like the problem is with my version Java and OS. I will see if I can run this on a supported platform. What platform did you successfully run it on btw ? Code and result is below.

Code:

System.out.print("Fid: " + entry.fieldId() + " Name = " + entry.name() + " DataType: " +
DataType.asString(entry.load().dataType()) + " Value: " + entry.enumValue() + "\n");

String enm = entry.enumDisplay();

System.out.println(enm);

enm.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

});
System.out.print("\n");

// SUCCESS: Conversion was succesful - application can now use converted content stored in rmtesBuffer

String rmStr = rmtesBuffer.toString();

System.out.println(rmStr);

rmStr.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.print("\n");

Result:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 2

�

\u0020 \ufffd

?
\u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 3

�
\u0020 \ufffd

?
\u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

@vinod.hirani

I'm sorry that my reply is not clear. I forgot to mention that I didn't try the code with OpenJDK. I just verify your code on my environment.

I'm looking into your code and will update you again.

Hi Veerapath,

I understand you didn't try it with OpenJDK. What platform and JVM did you use for your testing ?

Thanks,

Vinod

You can set the Character encoding by providing the file.encoding system property when JVM starts e.g. java -Dfile.encoding="ISO-8859-1"...