Paging Quote Atom Feed using next link cause circular navigation

david.snowsill

First Request to quotes atom feed using:

https://permid.org/atom/quote?access-token=XXX

returns:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"> <title>Quote Feed</title> <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=25" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=300&length=25" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri&gt; </author> <subtitle>From date(UTC) : 2018-04-20T23:40:52.000Z, To date(UTC) : 2018-04-25T23:40:51.000Z, Window : Offset = 0, Length = 50, out of 325 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id&gt; <updated>2018-04-25T23:00:48Z</updated> <opensearch:totalResults>325</opensearch:totalResults> <opensearch:startIndex>1</opensearch:startIndex>

following the next link:

https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50

returns:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
<title>Quote Feed</title>
<link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" />
<link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" />
<link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=25" />
<link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=300&length=25" />
<author>
<name>Open PermID</name>
<uri>http://permid.org</uri&gt;
</author>
<subtitle>From date(UTC) : 2018-04-20T23:42:23.000Z, To date(UTC) : 2018-04-25T23:42:22.000Z, Window : Offset = 0, Length = 50, out of 325 results.</subtitle>
<id>https://api.thomsonreuters.com/permid/atom/quote</id&gt;
<updated>2018-04-25T23:00:48Z</updated>
<opensearch:totalResults>325</opensearch:totalResults>
<opensearch:startIndex>1</opensearch:startIndex>

which contains the next link of :

https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50

instead of:

https://api.thomsonreuters.com/permid/atom/quote&offset=100&length=50

so we are unable to use the Atom links to page through the Atom feed entities

david.snowsill

Thanks for the update. It sounds as though from your comments and my testing that the Quotes RSS feed is very broken and won't be available for use in production for the foreseeable future.

As per your question, the Quotes RSS Feed is currently producing the following results:

Request 1 : https://permid.org/atom/quote?access-token=XXX&format=ntriples
Response 1 :

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"> <title>Quote Feed</title> <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=16" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=400&length=16" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri&gt; </author> <subtitle>From date(UTC) : 2018-05-25T22:27:32.000Z, To date(UTC) : 2018-05-30T22:27:31.000Z, Window : Offset = 0, Length = 50, out of 416 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id&gt; <updated>2018-05-30T22:25:40Z</updated> <opensearch:totalResults>416</opensearch:totalResults>

Notes:
- "first" link is incorrect as it should contain length=50 not length = 16
- totalResults and last link are inconsistent with following response

Request 2 : https://permid.org/atom/quote?access-token=XXX&format=ntriples&offset=50&length=50
Response 2 :

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"> <title>Quote Feed</title> <link rel="prev" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=36" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=36" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=36" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri&gt; </author> <subtitle>From date(UTC) : 2018-05-25T22:32:12.000Z, To date(UTC) : 2018-05-30T22:32:11.000Z, Window : Offset = 50, Length = 50, out of 86 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id&gt; <updated>2018-05-30T16:22:05Z</updated> <opensearch:totalResults>86</opensearch:totalResults>

Notes:

- "first" link is incorrect as it should contain length=50 not length = 36

- "last" link is different to Response 1
- totalResults now 86 not 400 as per Response 2
- Due to the small amount of data, no "next" link is present so unable to verify circular reference at this moment

Jirapongse

I am unable to replicate this issue. My issue is different.

When I request https://permid.org/atom/quote?access-token=xxx, the next link returns correctly:

 <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" />

Then, when I request to https://api.thomsonreuters.com/permid/atom/quote?offset=50&length=50, the next link returns correctly:

<link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=100&length=50" />

However, when I request to https://api.thomsonreuters.com/permid/atom/quote?offset=100&length=50, the next link returns incorrectly:

<link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=17" />

I will contact the support team to verify it.

david.snowsill

Hi Jirapongse,
Thanks for your help with this matter. I have seen the scenario you have raised also.
Regards,
David

david.snowsill

Hi jirapongse.phuriphanvichai,

Has the support team verified this issue? If so, do you have a timeframe when it will be fixed?
Regards,
David

Jirapongse

https://community.developers.refinitiv.com/discussion/comment/26829#Comment_26829

@david.snowsill

Sorry for late response.

I have contacted the support team again to verify the problem.

Jirapongse

https://community.developers.refinitiv.com/discussion/comment/26829#Comment_26829

@david.snowsill

I got a response from the support team that they are investigating this issue.

I will keep you updated on its progress.

david.snowsill

https://community.developers.refinitiv.com/discussion/comment/27458#Comment_27458

Thanks for the update
David

Jirapongse

@david.snowsill

The development team is unable to reproduce the circular navigation with the next link. Are you still experiencing the circular navigation with the next link? If yes, please share all requests and responses.

For the issue about the incorrect next link, this issue has been identified with the underlying
queries that results in different total counts being provided when different
offsets are used, which then has a consequent impact on the pagination results. This will need to be
an enhancement and the team will need to consider approach, solution design and
implementation. At this point in time, we still don't have an estimated time when any
change will be implemented, as this will be dependent on approach/design.

I will keep you updated on its progress.

Jirapongse

https://community.developers.refinitiv.com/discussion/comment/28071#Comment_28071

@david.snowsill

From my test, the result looks correct.

If you still see the problem, please let me know.