WebSocket API - "NUL" in the response for the rate request

pradeep.badoni · March 2021

Hi Team,

I am getting "null" in the response message.

It only happened at the initial stage of the WebSocket API communication (only once).

I am requesting the RIC code in batches and I have total 500+ RIC codes and I am calling them in 200 batches.

Could you please help me to figure out what is root cause of the issue.

Let me know if you need more detail here.

Below is the error message (blank space in the below message is having the NUL character):

Exception - byte count[40960]. Value - [[{"ID":9,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"CHF="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUmw=","SeqNumber":41582,"Fields":{"DSPLY_NAME":"ARAB BANK MNA","HIGH_1":0.9304,"LOW_1":0.9274,"OPEN_PRC":0.9281,"HST_CLOSE":0.9275,"BID":0.9301,"ASK":0.9302,"BID_HIGH_1":0.9304,"BID_LOW_1":0.9273}},{"ID":71,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"AUD2M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":28158,"Fields":{"DSPLY_NAME":"BROKER GFX","HIGH_1":2.54,"LOW_1":2.45,"OPEN_PRC":2.45,"HST_CLOSE":2.47,"BID":2.39,"ASK":2.54,"BID_HIGH_1":2.54,"BID_LOW_1":2.15}},{"ID":77,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"AUD1Y="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":59502,"Fields":{"DSPLY_NAME":"BROKER GFX","HIGH_1":6.41,"LOW_1":6.20,"OPEN_PRC":4.69,"HST_CLOSE":5.05,"BID":5.15,"ASK":6.65,"BID_HIGH_1":6.41,"BID_LOW_1":3.92}},{"ID":86,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"CADTN="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":26478,"Fields":{"DSPLY_NAME":"Commerzbank LON","HIGH_1":0.0200,"LOW_1":-0.0040,"OPEN_PRC":-0.0530,"HST_CLOSE":-0.0390,"BID":-0.0 320,"ASK":0.0180,"BID_HIGH_1":0.0200,"BID_LOW_1":-0.1130}},{"ID":139,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"JPY2Y="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUkHA","SeqNumber":7166,"Fields":{"DSPLY_NAME":"STANCHART SIN","HIGH_1":-129.70,"LOW_1":-128.37,"OPEN_PRC":-131.25,"HST_CLOSE":-131.27,"BID":-132.53,"ASK":-128.04,"BID_HIGH_1":-129.70,"BID_LOW_1":-140.42}},{"ID":142,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"JPY5Y="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUkHA","SeqNumber":2702,"Fields":{"DSPLY_NAME":"STANDCHART NYC","HIGH_1":-672.49,"LOW_1":-678.83,"OPEN_PRC":-686.75,"HST_CLOSE":-717.50,"BID":-693.07,"ASK":-675.20,"BID_HIGH_1":-672.49,"BID_LOW_1":-728.50}},{"ID":131,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"JPY1M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":37086,"Fields":{"DSPLY_NAME":"COMMERZBANK FFT","HIGH_1":-3.58,"LOW_1":-3.74,"OPEN_PRC":-4.39,"HST_CLOSE":-3.97,"BID":-3.78,"ASK":-3.50,"BID_HIGH_1":-3.58,"BID_LOW_1":-4.39}},{"ID":136,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"JPY6M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":25886,"Fields":{"DSPLY_NAME":"BOA SIN","HIGH_1":-21.21,"LOW_1":-21.11,"OPEN_PRC":-22.95,"HST_CLOSE":-21.87,"BID":-21.42,"ASK":-20.92,"BID_HIGH_1":-21.21,"BID_LOW_1":-23.10}},{"ID":149,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"HKDTN="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBVFTA","SeqNumber":43758,"Fields":{"DSPLY_NAME":"BOA SIN","HIGH_1":0.50,"LOW_1":-0.27,"OPEN_PRC":-2.70,"HST_CLOSE":-1.20,"BID":-0.45,"ASK":0.05,"BID_HIGH_1":0.50,"BID_LOW_1":-2.80}},{"ID":14,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"EUR2M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":17006,"Fields":{"DSPLY_NAME":"SOC GENERAL PAR","HIGH_1":16.12,"LOW_1":15.74,"OPEN_PRC":14.97,"HST_CLOSE":14.51,"BID":15.70,"ASK":16.16,"BID_HIGH_1":16.12,"BID_LOW_1":14.43}},{"ID":41,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"GBP1Y="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":23294,"Fields":{"DSPLY_NAME":"SOC GENERAL PAR","HIGH_1":17.64,"LOW_1":18.06,"OPEN_PRC":15.82,"HST_CLOSE":16.14,"BID":14.80,"ASK":19.75,"BID_HIGH_1":17.64,"BID_LOW_1":12.44}},{"ID":95,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"CAD9M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":56062,"Fields":{"DSPLY_NAME":"NAB SYD","HIGH_1":1.00,"LOW_1":1.65,"OPEN_PRC":-0.30,"HST_CLOSE":-0.65,"BID":-0.10,"ASK":2.30,"BID_HIGH_1":1.00,"BID_LOW_1":-1.65}},{"ID":110,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"CHF1M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":36366,"Fields":{"DSPLY_NAME":"BOA SIN","HIGH_1":-7.89,"LOW_1":-8.56,"OPEN_PRC":-8.64,"HST_CLOSE":-8.88,"BID":-8.58,"ASK":-8.18,"BID_HIGH_1":-7.89,"BID_LOW_1":-11.00}},{"ID":115,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"CHF6M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":32158,"Fields":{"DSPLY_NAME":"STANCHART SIN","HIGH_1":-47.63,"LOW_1":-47.64,"OPEN_PRC":-49.70,"HST_CLOSE":-48.41,"BID":-50.07,"ASK":-44.64,"BID_HIGH_1":-47.63,"BID_LOW_1":-53.72}},{"ID":158,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"HKD9M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBVFTA","SeqNumber":45134,"Fields":{"DSPLY_NAME":"NAB SYD","HIGH_1":6.00,"LOW_1":-0.35,"OPEN_PRC":-4.00,"HST_CLOSE":-1.00,"BID":0.00,"ASK":10.00,"BID_HIGH_1":6.00,"BID_LOW_1":-10.00}},{"ID":17,"Type":"Refresh","Key":{"Service":"ELEKTRON_DD","Name":"EUR5M="},"State":{"Stream":"Open","Data":"Ok"},"Qos":{"Timeliness":"Realtime","Rate":"JitConflated"},"PermData":"AwEBUUHA","SeqNumber":222,"Fields":{"DSPLY_NAME":"NAB SYD","HIGH_1":39.69,"LOW_1":39.61,"OPEN_PRC":39.32,"HST_CLOSE":39.25,"BID":39.31,"ASK":39.71,"BID_HIGH_1":39.69,"BID_LOW_1":37.02}}] ]

EXCEPTION (type = JsonReaderException) : After parsing a value an unexpected character was encountered: 3. Path '[3].Fields.BID', line 1, position 8192.

at Newtonsoft.Json.JsonTextReader.ParsePostValue()

at Newtonsoft.Json.JsonTextReader.ReadInternal()

at Newtonsoft.Json.JsonTextReader.Read()

at Newtonsoft.Json.Linq.JContainer.ReadContentFrom(JsonReader r)

at Newtonsoft.Json.Linq.JContainer.ReadTokenFrom(JsonReader reader)

at Newtonsoft.Json.Linq.JArray.Load(JsonReader reader)

at Newtonsoft.Json.Linq.JArray.Parse(String json)

Thanks

Pradeep B

Jirapongse · March 2021

@pradeep.badoni

Please try to use result.Result.Count instead of readBuffer.Count when calling the memoryStream.Write method.

Does it help?

Jirapongse · March 2021

@pradeep.badoni

Could you please share the code that reads data into the byte count[40960]?

From the output, the size of the JSON message is less than 40960. I assume that it could be garbage characters in the count variable.

pradeep.badoni · March 2021

https://community.developers.refinitiv.com/discussion/comment/74776#Comment_74776

Hi @Jirapongse

Yes, agree that there are some garbage characters in the message (The same can we observed in the image attached in the initial message)

I am using Encoding.ASCII.GetBytes(jsonMsg) to get the count.

Thanks

Pradeep B

pradeep.badoni · March 2021

https://community.developers.refinitiv.com/discussion/comment/74781#Comment_74781

Hi Team,

Any comments here?

Thanks

Pradeep B

Jirapongse · March 2021

https://community.developers.refinitiv.com/discussion/comment/75034#Comment_75034

@pradeep.badoni

Please share the full source code so we can verify it and use it to replicate the issue. Moreover, please provide the version of .NET and the libraries used by the application.

Refer to the sample code, the size of the buffer is 8192.

 private static readonly int BUFFER_SIZE = 8192;

Can you replicate the issue with the sample code?

pradeep.badoni · March 2021

https://community.developers.refinitiv.com/discussion/comment/75148#Comment_75148

@Jirapongse

Sorry but I can provide the source as it has DB dependency and you will not bale to execute it.

My source code is build on top of the sample code provided in the WebSocket API example only and the buffer size if 8192.

I am using below .NET Library:

System.Net.WebSockets - 4.9

Newtonsoft.Json.dll - 6.0.0.0

Note: I am not getting this issue frequently, in 30 min time window I am getting it 2 or 3 time.

I think the problem is when I am getting the update for multiple RIC code in a single message.

I am getting some invalid char in the message which JSON is not able to deserialize.

Error message:

EXCEPTION (type = JsonReaderException) : After parsing a value an unexpected character was encountered: 2. Path '[7].Fields.ASK', line 1, position 8192.

at Newtonsoft.Json.JsonTextReader.ParsePostValue()

at Newtonsoft.Json.JsonTextReader.ReadInternal()

at Newtonsoft.Json.JsonTextReader.Read()

at Newtonsoft.Json.Linq.JContainer.ReadContentFrom(JsonReader r)

at Newtonsoft.Json.Linq.JContainer.ReadTokenFrom(JsonReader reader)

at Newtonsoft.Json.Linq.JArray.Load(JsonReader reader)

at Newtonsoft.Json.Linq.JArray.Parse(String json)

Please help me to resolve the issue.

Jirapongse · March 2021

https://community.developers.refinitiv.com/discussion/comment/75034#Comment_75034

@pradeep.badoni

It may relate to this code.

 if (!result.Result.EndOfMessage)
                        {
                            if (memoryStream == null) memoryStream = new MemoryStream(BUFFER_SIZE * 5);

                            memoryStream.Write(readBuffer.Array, readBuffer.Offset, readBuffer.Count);
                            readBuffer = new ArraySegment<byte>(new byte[BUFFER_SIZE]);
                        }

40960 = 8192 * 5

I will verify the problem. You may need to add additional logs in this section to verify its behavior.

pradeep.badoni · March 2021

https://community.developers.refinitiv.com/discussion/comment/75251#Comment_75251

Hi @Jirapongse

Thanks for the above detail.

Any suggestion on what king of more logging I can put?

Will be waiting for your response on your finding with the above code line.

Thanks

Pradeep B

Jirapongse · March 2021

@pradeep.badoni

I am still unable to replicate the issue. However, I have tested the JSON string posted in the question and found that the message is invalid. The value of BID fields of CADTN= contains a space.

I am not sure that it is copy/paste problem or the string really contains a space in the BID's value.

I assume that the application has logged this invalid message. Please save the invalid JSON string in the text file and then attach it in the answer.