question

Upvotes
Accepted
28 6 6 14

Paging Quote Atom Feed using next link cause circular navigation

First Request to quotes atom feed using:

https://permid.org/atom/quote?access-token=XXX

returns:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">; <title>Quote Feed</title> <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=25" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=300&length=25" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri> </author> <subtitle>From date(UTC) : 2018-04-20T23:40:52.000Z, To date(UTC) : 2018-04-25T23:40:51.000Z, Window : Offset = 0, Length = 50, out of 325 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id>; <updated>2018-04-25T23:00:48Z</updated> <opensearch:totalResults>325</opensearch:totalResults> <opensearch:startIndex>1</opensearch:startIndex>

following the next link:

https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50

returns:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">; <title>Quote Feed</title> <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=25" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=300&length=25" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri> </author> <subtitle>From date(UTC) : 2018-04-20T23:42:23.000Z, To date(UTC) : 2018-04-25T23:42:22.000Z, Window : Offset = 0, Length = 50, out of 325 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id>; <updated>2018-04-25T23:00:48Z</updated> <opensearch:totalResults>325</opensearch:totalResults> <opensearch:startIndex>1</opensearch:startIndex>

which contains the next link of :

https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50

instead of:

https://api.thomsonreuters.com/permid/atom/quote&offset=100&length=50

so we are unable to use the Atom links to page through the Atom feed entities

intelligent-tagging-apipermid-apiopen-permid-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
28 6 6 14

Thanks for the update. It sounds as though from your comments and my testing that the Quotes RSS feed is very broken and won't be available for use in production for the foreseeable future.

As per your question, the Quotes RSS Feed is currently producing the following results:

Request 1 : https://permid.org/atom/quote?access-token=XXX&format=ntriples
Response 1 :

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">; <title>Quote Feed</title> <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=16" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=400&length=16" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri> </author> <subtitle>From date(UTC) : 2018-05-25T22:27:32.000Z, To date(UTC) : 2018-05-30T22:27:31.000Z, Window : Offset = 0, Length = 50, out of 416 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id>; <updated>2018-05-30T22:25:40Z</updated> <opensearch:totalResults>416</opensearch:totalResults>

Notes:
- "first" link is incorrect as it should contain length=50 not length = 16
- totalResults and last link are inconsistent with following response

Request 2 : https://permid.org/atom/quote?access-token=XXX&format=ntriples&offset=50&length=50
Response 2 :

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">; <title>Quote Feed</title> <link rel="prev" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=50" /> <link rel="self" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=36" /> <link rel="first" href="https://api.thomsonreuters.com/permid/atom/quote&offset=0&length=36" /> <link rel="last" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=36" /> <author> <name>Open PermID</name> <uri>http://permid.org</uri> </author> <subtitle>From date(UTC) : 2018-05-25T22:32:12.000Z, To date(UTC) : 2018-05-30T22:32:11.000Z, Window : Offset = 50, Length = 50, out of 86 results.</subtitle> <id>https://api.thomsonreuters.com/permid/atom/quote</id>; <updated>2018-05-30T16:22:05Z</updated> <opensearch:totalResults>86</opensearch:totalResults>

Notes:

- "first" link is incorrect as it should contain length=50 not length = 36

- "last" link is different to Response 1
- totalResults now 86 not 400 as per Response 2
- Due to the small amount of data, no "next" link is present so unable to verify circular reference at this moment

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@david.snowsill

From my test, the result looks correct.

If you still see the problem, please let me know.

Upvotes
45.2k 103 43 60

I am unable to replicate this issue. My issue is different.

When I request https://permid.org/atom/quote?access-token=xxx, the next link returns correctly:

 <link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=50" />

Then, when I request to https://api.thomsonreuters.com/permid/atom/quote?offset=50&length=50, the next link returns correctly:

<link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=100&length=50" />

However, when I request to https://api.thomsonreuters.com/permid/atom/quote?offset=100&length=50, the next link returns incorrectly:

<link rel="next" href="https://api.thomsonreuters.com/permid/atom/quote&offset=50&length=17" />

I will contact the support team to verify it.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
28 6 6 14

Hi Jirapongse,
Thanks for your help with this matter. I have seen the scenario you have raised also.
Regards,
David

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
28 6 6 14

Hi jirapongse.phuriphanvichai,

Has the support team verified this issue? If so, do you have a timeframe when it will be fixed?
Regards,
David

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

@david.snowsill

Sorry for late response.

I have contacted the support team again to verify the problem.

@david.snowsill

I got a response from the support team that they are investigating this issue.

I will keep you updated on its progress.

Thanks for the update
David

Upvotes
45.2k 103 43 60

@david.snowsill

The development team is unable to reproduce the circular navigation with the next link. Are you still experiencing the circular navigation with the next link? If yes, please share all requests and responses.

For the issue about the incorrect next link, this issue has been identified with the underlying queries that results in different total counts being provided when different offsets are used, which then has a consequent impact on the pagination results. This will need to be an enhancement and the team will need to consider approach, solution design and implementation. At this point in time, we still don't have an estimated time when any change will be implemented, as this will be dependent on approach/design.

I will keep you updated on its progress.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.