Hello Maury,
Per my colleague, there are UTF-8 issues when using websockets to retrieve data. I presume this applies to all data, not just TS1. Unfortunately, I don't have any ticket numbers available describing the issue of websockets handling UTF-8.
LPC comes with the testclient application for sample connections. It has the ability to output the data to a file or to STDOUT. Here is an example command line to use:
#> cd /opt/refinitiv/SOFTWARE/lpc1.2.0/linux7_x86_64/demo
#> export LD_LIBRARY_PATH=`pwd`
#> ./testclient -S ELEKTRON_DD -ct rssl -p 14002 -il dIBMd -o 512 -I 1 -O -u yourAppUserName -l stdout -dfile ../../etc/RDMFieldDictionary
The "stdout" option is using a lower case 'L'. The dictionary file is needed to decode the FIDs.
Retrieving TS1 data encoded in FIDs ROW64_1-14 from Elektron (RTO)
There are two problems with the way that RTO is encoding TS1 data in UTF-8
- One character is mapped to a different character in the same range
- Several characters are expanded out into two characters and given the way it's done, I'm not sure it's possible to reverse the expansion 100% correctly in all cases
I've also captured the websocket traffic in Fiddler and it looks like this is coming from upstream and we're not distorting it any way in our "app". I've attached this rto-202105181200-ts1.zip showing this binary data in FIDs ROW64_1-14 before it reaches our app. Rename the filename from .zip -> .saz to open in Fiddler.
Also attached is "ts1-rto character mappings.txt" which shows the IDN characters and how they are being mapped to RTO. We arrived at this by making the same request through the LPC (Legacy Protocol Converter) 1.2 which connects to Elektron (RTO) and then comparing the binary data in the FIDs.
Can anyone shed an light on this decoding process? Or whom to have look at this?
Maybe our request in the Fiddler SAZ is just wrong? Its pretty basic.
{
"ID":2,"Streaming":false
"Key":{
"Name":["dBOc1d"],
"Service":"ELEKTRON_DD"
}
}
Either the LPC is dealing with these different encodings or it is making a slightly different request that causes the data to come back encoded differently.
Best Answer
-
Hello Maury,
Per my colleague, there are UTF-8 issues when using websockets to retrieve data. I presume this applies to all data, not just TS1. Unfortunately, I don't have any ticket numbers available describing the issue of websockets handling UTF-8.
LPC comes with the testclient application for sample connections. It has the ability to output the data to a file or to STDOUT. Here is an example command line to use:
#> cd /opt/refinitiv/SOFTWARE/lpc1.2.0/linux7_x86_64/demo
#> export LD_LIBRARY_PATH=`pwd`
#> ./testclient -S ELEKTRON_DD -ct rssl -p 14002 -il dIBMd -o 512 -I 1 -O -u yourAppUserName -l stdout -dfile ../../etc/RDMFieldDictionary
The "stdout" option is using a lower case 'L'. The dictionary file is needed to decode the FIDs.
0
Answers
-
I created a MyRefinitiv ticket and was given the following response.
"We have received your query regarding problems with the way that RTO is encoding ts1realtime.txt in UTF-8 you saw.
We have tested on our side with ts1LPC.txt as well as with the real-time service but both cannot reproduce your mentioned issue. Please find our testing result in the attachment."We are now comparing the above two file against our raw request with out .NET app. It is a bit more work to get what the LPC is doing since it communicates with https. We are investigating the usage of a proxy like Fiddler to capture the data for review.
0 -
Thank you. I did also make a My Refinitiv ticket and received two files that we are in the process of comparing. But we still don't know if we are doing something wrong.
0 -
Is there somewhere I can raise a ticket?
0
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 684 Datastream
- 1.4K DSS
- 614 Eikon COM
- 5.2K Eikon Data APIs
- 10 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 248 ETA
- 554 WebSocket API
- 37 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 275 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 641 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 26 DACS Station
- 121 Open DACS
- 1.1K RFA
- 104 UPA
- 192 TREP Infrastructure
- 228 TRKD
- 915 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 89 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 46 中文论坛