No response when requesting empty string from Elektron Websocket
Hi, I encountered a problem when using the elektron real-time API described in this documentation https://github.com/Refinitiv/websocket-api/blob/master/WebsocketAPI_ProtocolSpecification.pdf
When I make a batch request via this API, I expect a batch response and then as many item responses as the number of strings I put in the "Key": "Name" array.
However when an item in the "Key": "Name" array is the empty string ("") the item's response never comes. I believe this is a bug.
Example of pathological request (this is Rust's representation of JSON):
Object({
"ID": Number(2),
"Key": Object({
"Name": Array([String("")]),
"Service": String("IDN_SELECTFEED"),
}),
"Streaming": Bool(false),
"View": Array([String("DSPLY_NAME")]),
})Then the batch response comes:
Object({
"ID": Number(2),
"State": Object({
"Data": String("Ok"),
"Stream": String("Closed"),
"Text": String("Processed 1 total items from Batch Request. 1 Ok."),
}),
"Type": String("Status"),
})But the item response for "" never comes, I believe I response containing an error message should be returned.
Find more posts tagged with
The expected behavior is what @Gurpreet provided. That being said, it is quite possible that the server you are connected to may be a different or older version. What are you connecting to?
Hi @nick.zincone, here are some elements of the login response
"ApplicationId": String(
"407",
),
"ApplicationName": String(
"ADS",
),
"Text": String(
"Login accepted by host ls9600ppc00471.",
),
@Gurpreet As a test I tried to only login and make a batch request, then wait for 2 responses. I receive the batch response but never the item response. The ping does come after about 20 seconds so everything seems fine on my side.
Looking at the above output, my suspicions are that the version of the ADS you are testing against may need to be updated. I can't confirm all the server details, but I would suggest you reach out to your market data team and explain and confirm the behavior you are seeing. When we test against up-to-date versions of the ADS, we see the expected results.
Hi Gurpreet,
I
am Julien from the marketdata team in charge on managing TREP components for Tristan. It is a bit weird, because we are able to reproduce the issue on our side with Refinitiv example code and the latest version of ADS (ads3.5.4.L1.linux.rrg 64-bit) on our side.
We opened the support ticket #11062603 on the Refinitiv ADS support page a few minutes ago to follow this up. What is weird is that we are seemly doing the exact same thing as you did (market price request with batch and view enabled). The request stays "pending" at the ADS level until a source error message comes :
SENT:
{
"ID":2,
"Key":{
"Name":[
""
],
"Service":"IDN_SELECTFEED"
},
"View":[
"BID",
"ASK",
"BIDSIZE"
]
}
RECEIVED:
[
{
"ID":2,
"State":{
"Data":"Ok",
"Stream":"Closed",
"Text":"Processed 1 total items from Batch Request. 1 Ok."
},
"Type":"Status"
}
]
(..ping/pongs)
RECEIVED:
[
{
"ID":3,
"Key":{
"Service":"IDN_SELECTFEED"
},
"State":{
"Data":"Suspect",
"Stream":"Open",
"Text":"A7: Source did not respond."
},
"Type":"Status"
}
]
Regards,
Julien
Hi Gurpreet / all,
We have just received the surprising answer below from the Enterprise products support :
"Hi Julien,
I raised your query with our specialist but they were able o replicate the issue on same ADS version with Phyton. They suggested you to continue reaching out to developers community as they are the right channel who can assist you with this.
Regards,
***** ********
Elektron Frontline Support
Refinitiv'
So they were able to replicate the issue but it seems they want us to rely on your help to understand / fix it. What do you think ?
Thanks
Julien
Hi @julien.dominici,
Can you please provide following details:
ADS version
ADS logs
Python version
Output from pip list showing websockets and requests version.
Hi Gurpreet, sure, here you go :
ADS version : ads3.5.4.L1.linux.rrg 64-bit
ADS logs : attached websocket_empty_string_issue_ads_log.txt
Python version : Python 3.6.8 [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Output from pip list :
requests (2.14.2)
websocket-client (0.47.0
Hi @Tristan6846132 , @julien.dominici
One thing I want to confirm is whether the application used is registering interest in "status" events. Looking at the original question, it states no response. However, I can see the server appears to be generating a response, i.e. "status" - "A7 - source did not respond". Applications should always register in "refresh" and "status" events.
That being said, this doesn't resolve why the server can't fulfill the request. Can you confirm exactly what the Enterprise team was able to replicate? As you stated, I'm also surprised why they indicated we can somehow resolve why this specific version of the ADS reacts like this - we can't - this forum is dedicated to APIs.
That being said, this is what happens when I connected into the ADS within Refinitiv Real-Time -- Optimized (within the cloud):
Request:
{
"ID": 2,
"Streaming": true,
"Key": {
"Name": [
""
]
},
"View": [
"BID",
"ASK",
"DSPLY_NAME"
]
}
Response:
{
"ID": 2,
"Type": "Status",
"State": {
"Stream": "Closed",
"Data": "Ok",
"Text": "Processed 1 total items from Batch Request. 1 Ok."
}
}
{
"ID": 3,
"Type": "Status",
"Key": {
"Service": "ELEKTRON_DD"
},
"State": {
"Stream": "Closed",
"Data": "Suspect",
"Code": "AlreadyOpen",
"Text": "*Request Rejected: Request key did not contain name."
}
}
Hi @nick.zincone , thanks for your help. I was personnaly able to replicate the issue by simply using this script against the ADS version mentioned earlier (but I have no details about the test done by the Elektron Front line support unfortunately) :
Here is the complete output if that helps (please note, there is >1mn between the first status reponse and the A7: Source did not respond message) :
[eurvlii******:/var/tmp/jd] python3 market_price_batch_view.py
Connecting to WebSocket ws://ads-uat-*******:15000/WebSocket ...
WebSocket successfully connected!
SENT:
{
"Domain":"Login",
"ID":1,
"Key":{
"Elements":{
"ApplicationId":"***",
"Position":"10.***.***.128"
},
"Name":"************"
}
}
RECEIVED:
[
{
"Domain":"Login",
"Elements":{
"MaxMsgSize":61516,
"PingTimeout":30
},
"ID":1,
"Key":{
"Elements":{
"AllowSuspectData":1,
"ApplicationId":"***",
"ApplicationName":"ADS",
"Position":"10.***.***.128",
"ProvidePermissionExpressions":1,
"ProvidePermissionProfile":0,
"SingleOpen":1,
"SupportBatchRequests":7,
"SupportEnhancedSymbolList":1,
"SupportOMMPost":1,
"SupportOptimizedPauseResume":0,
"SupportPauseResume":0,
"SupportStandby":0,
"SupportViewRequests":1
},
"Name":"**********"
},
"State":{
"Data":"Ok",
"Stream":"Open",
"Text":"Login accepted by host ls960********."
},
"Type":"Refresh"
}
]
SENT:
{
"ID":2,
"Key":{
"Name":[
""
],
"Service":"IDN_SELECTFEED"
},
"View":[
"BID",
"ASK",
"BIDSIZE"
]
}
RECEIVED:
[
{
"ID":2,
"State":{
"Data":"Ok",
"Stream":"Closed",
"Text":"Processed 1 total items from Batch Request. 1 Ok."
},
"Type":"Status"
}
]
RECEIVED:
[
{
"Type":"Ping"
}
]
SENT:
{
"Type":"Pong"
}
RECEIVED:
[
{
"Type":"Ping"
}
]
SENT:
{
"Type":"Pong"
}
RECEIVED:
[
{
"Type":"Ping"
}
]
SENT:
{
"Type":"Pong"
}
RECEIVED:
[
{
"ID":3,
"Key":{
"Service":"IDN_SELECTFEED"
},
"State":{
"Data":"Suspect",
"Stream":"Open",
"Text":"A7: Source did not respond."
},
"Type":"Status"
}
]
RECEIVED:
[
{
"Type":"Ping"
}
]
SENT:
{
"Type":"Pong"
Is there anything else I can do to help ?
I used the same python sample you used with ADS version 3.5.3.L1, and am able to get valid response.
Connecting to WebSocket ws://****:15000/WebSocket ...
WebSocket successfully connected!
SENT:
{
"Domain":"Login",
"ID":1,
"Key":{
"Elements":{
"ApplicationId":"256",
"Position":"****"
},
"Name":"root"
}
}
RECEIVED:
[
{
"Domain":"Login",
"Elements":{
"MaxMsgSize":61516,
"PingTimeout":30
},
"ID":1,
"Key":{
"Elements":{
"AllowSuspectData":1,
"ApplicationId":"256",
"ApplicationName":"RTC",
"Position":"****",
"ProvidePermissionExpressions":1,
"ProvidePermissionProfile":0,
"SingleOpen":1,
"SupportBatchRequests":7,
"SupportEnhancedSymbolList":1,
"SupportOMMPost":1,
"SupportOptimizedPauseResume":0,
"SupportPauseResume":0,
"SupportStandby":0,
"SupportViewRequests":1
},
"Name":"root"
},
"State":{
"Data":"Ok",
"Stream":"Open",
"Text":"Login accepted by host ****."
},
"Type":"Refresh"
}
]
SENT:
{
"ID":2,
"Key":{
"Name":[
""
],
"Service":"ELEKTRON_DD"
},
"View":[
"BID",
"ASK",
"BIDSIZE"
]
}
RECEIVED:
[
{
"ID":2,
"State":{
"Data":"Ok",
"Stream":"Closed",
"Text":"Processed 1 total items from Batch Request. 1 Ok."
},
"Type":"Status"
}
]
RECEIVED:
[
{
"ID":3,
"Key":{
"Service":"ELEKTRON_DD"
},
"State":{
"Code":"AlreadyOpen",
"Data":"Suspect",
"Stream":"Closed",
"Text":"*Request Rejected: Request key did not contain name."
},
"Type":"Status"
}
]
WebSocket Closed
I do notice that your requests and websocket-client modules are quite old. Can you please update these modules and try again. Here is what I have:
requests 2.26.0
websocket-client 1.2.1
Preliminarily, it seems that there is some configuration entry in your ADS which is making it treat the blank as a valid instrument. It sends the request to headend to get data for it, and after a while the request times out and you get the "source did not respond" message.
Some one from infrastructure might have to advise you on this. I will check and ask the guys to re-open your ticket; if upgrading the pip-modules don't fix this issue.
Thanks for the details. While @Gurpreet is looking into the ticket and confirming if the configuration within the ADS may need to change, I still want to focus on the main question by @Tristan6846132 - specifically "No response when requesting empty string...". As you demonstrated, there is a response, i.e a "status" response "A7 - Source did not respond". It's unclear what application is being used, but did you get confirmation that this application is registering interest in "status" events? The bottom line is that the request for an empty string will return some sort of response - whether what you are seeing or what we are seeing. The only issue that I see is that the ADS you are connected to is forwarding the request to the backend and eventually times out >1min. From an application / API perspective, do you see any incorrect behavior?
That being said, I'm in conversations with the server team to provide a detailed update. I just want to confirm if the preferred behavior you expect to see is the server responds in the same manner as what we observed within RTO? A status message that returns right away as opposed to the invalid request going to the backend and timing out?
thanks.
Hi @nick.zincone,
I re-ran my tests and I do see the "status" response "A7 - Source did not respond", so I will use that response to timeout and return user requests.
As for the behavior I want to see, the sooner the requests return the better, so a status message returning right away would be good.
Hi @nick.zincone , @Gurpreet thank you for your help again.
I think we need to resituate the issue a bit. I understand your points and reasoning but I think that we are moving from the real issue a bit. Let me summarize :
For these examples :
- we are not considering any internal development, only the Github examples from Refinitiv.
- The ADS reply is correct when we are using a standard websocket request (without batch & view) with this code example : https://github.com/Refinitiv/websocket-api/blob/master/Applications/Examples/python/market_price.py :
Connecting to WebSocket ws://ads-*********:15000/WebSocket ...
ERROR RECEIVED:
{
"ID":2,
"Key":{
"Service":"IDN_SELECTFEED"
},
"State":{
"Code":"AlreadyOpen",
"Data":"Suspect",
"Stream":"Closed",
"Text":"Request Rejected: Request key did not contain name."
},
"Type":"Status"
}
For that reason I do not think that the ADS configuration has to be changed.
- If we are doing the same test with the "batch + view" code example at https://github.com/Refinitiv/websocket-api/blob/master/Applications/Examples/python/market_price_batch_view.py then we have the issue mentioned earlier.
(cf. previous examples)
----
On side of that, if we look at the ADS at the same time, we can see that :
- The request is "pending" at the datastream level (note, it is pending, not pendingOut) :
Data Streams
opens : 1 opensFromOverflowQueue: 0
normalOpens : 1 thresholdOverflowTime : 0
snapshotOpens : 0 streamIdMismatch : 0
recoverableOpens : 0 appendixAReq : 0
nonRecoverableOpens : 0 total : 1
permissionDenials : 0 cached : 0
updatePermDenials : 0 active : 0
mapEntryPermDenials : 0 stale : 0
recoverableCloses : 0 paused : 0
nonRecoverableCloses : 0 private : 0
preemptionCloses : 0 privateStale : 0
updatesDelayed : 0 pending : 1
updatesMerged : 0 pendingOut : 0
updatesExtracted : 0 recovery : 0
updatesFiltered : 0 preemption : 0
And one minute after, once the "A7" message has been sent to the application, the counters are changing to this :
Data Streams
opens : 1 opensFromOverflowQueue: 0
normalOpens : 1 thresholdOverflowTime : 0
snapshotOpens : 0 streamIdMismatch : 0
recoverableOpens : 1 appendixAReq : 0
nonRecoverableOpens : 0 total : 1
permissionDenials : 0 cached : 0
updatePermDenials : 0 active : 0
mapEntryPermDenials : 0 stale : 0
recoverableCloses : 1 paused : 0
nonRecoverableCloses : 0 private : 0
preemptionCloses : 0 privateStale : 0
updatesDelayed : 0 pending : 0
updatesMerged : 0 pendingOut : 0
updatesExtracted : 0 recovery : 1
updatesFiltered : 0 preemption : 0
So if I resume my perspective, the issue happens only when we are using the batch+view features of the API. I am not sure why at this point but this was the whole point of the conversation, and why I opened the ticket on the product portal.
Hope that clarifies things a bit. Thanks again in advance for your help.
PS: I have upgraded my testing python modules to requests (2.27.1) and websocket-client (1.3.1) and the behavior is the same.
Julien
Hello @Tristan6846132 @julien.dominici
Could you please let me know why the application sends the batch request message with the empty array to the ADS?
The "A7: Source did not respond" is generated by the ADS server. This status message is sent when a requested service is known to be available but ADS did not receive a response for an item and the item retry limit has been reached. Therefore, this would indicate that the ADH is not sending the response to the ADS or ADS is not receiving the response from the ADH for some reason.
I and @Jirapongse can replicate the issue with the following RTDS deployment scenario:
WebSocket Client --> ADS --> ADH --> [Real-Time Feed]
If I send a batch request message with an empty array, I got the following result:
SENT:
{
"ID":2,
"Key":{
"Name":[
""
],
"Service":"ELEKTRON_DD"
}
}
RECEIVED:
[
{
"ID":2,
"State":{
"Data":"Ok",
"Stream":"Closed",
"Text":"Processed 1 total items from Batch Request. 1 Ok."
},
"Type":"Status"
}
]
... Ping and Pong
RECEIVED:
[
{
"ID":3,
"Key":{
"Service":"ELEKTRON_DD"
},
"State":{
"Data":"Suspect",
"Stream":"Open",
"Text":"A7: Source did not respond."
},
"Type":"Status"
}
]
If I send a sing request message with an empty string, I got the following result:
SENT:
{
"ID":2,
"Key":{
"Name":"",
"Service":"ELEKTRON_DD"
}
}
RECEIVED:
[
{
"ID":2,
"Key":{
"Service":"ELEKTRON_DD"
},
"State":{
"Code":"AlreadyOpen",
"Data":"Suspect",
"Stream":"Closed",
"Text":"Request Rejected: Request key did not contain name."
},
"Type":"Status"
}
]
@wasin.w ,
Why did you get:
A7: Source did not respond
for an empty batch request, where as I get:
Request Rejected: Request key did not contain name.
for the same empty batch request. I am using ADS version 3.5.3.L1.
Is there a configuration entry that is causing this difference?
Hi @Gurpreet
I am using the ADS -- ADH -- Refinitiv Real-Time Feed deployment scenario.
If I use ADS Pop -- RTO deployment, the ADS always returns "Request Rejected: Request key did not contain name." too.
Hi @wasin.w,
We intend to allow users to send arbitrary requests to our application so they can access real-time data. If there is an issue with their request we want to return as soon as possible.
Although we could filter out empty RICs on our side it would be cleaner if the issue was fixed upstream. In addition there might be other special cases where no Item response is returned (or an error is returned after 1 minute) but since we don't know of those cases yet we cannot filter them out, causing some delays in the response. So overall I would be happier with an upstream fix that makes requests return asap.
> "Text":"Request Rejected: Request key did not contain name."
Maybe you could clarify what is considered an invalid name? this will be useful until the issue is fixed.
Hello @Tristan6846132
The "Request Rejected: Request key did not contain name." status text message is generated from the RTDS/TREP. The message means the request message does not have the item or RIC name.
I highly recommend you filter out the request message with an empty RICs string/array at the application level while the RTDS support team is investigating the issue.
Hello @Tristan6846132
The "Request Rejected: Request key did not contain name." status text message is generated from the RTDS/TREP. The message means the request message does not have the item or RIC name.
I highly recommend you filter out the request message with an empty RICs string/array at the application level while the RTDS support team is investigating the issue.
Hi @Tristan6846132,
There must be some issue in your code, because I do get the correct response message about an invalid name.
Here are the raw json requests and responses: