question

Upvotes
Accepted
3 1 1 3

HttpRequestException. Error while copying content to a stream. The response ended prematurely.

Making 95 calls to https://api-eit.refinitiv.com/permid/search?q=ticker:{ticker} from a C# application in a loop we get a HttpRequestException, "Error while copying content to a stream" with inner exception "The response ended prematurely" when reading the response content. The resonse statuscode is 200 so it's not rate-limiting.

Here's some sample code to repeat, with a crude re-try that shows the issue is transient as we actually get data back for all tickers with the retry in place.

using System.Net.Http.Headers;

namespace MyApp
{
  internal class Program
  {
    static async Task Main(string[] args)
    {
      try
      {
        var tickerList = new List<string>()
        {
          "AAF", "AAL", "ABF", "ADM", "AHT", "ANTO", "AUTO", "AV/", "AVV", "AZN", "BA/",
          "BARC", "BATS", "BDEV", "BKG", "BLND", "BME", "BNZL", "BP/", "BRBY", "BT/A", 
          "CCH", "CNA", "CPG", "CRDA", "CRH", "CTEC", "DCC", "DGE", "DPH", "EDV", 
          "ENT", "EXPN", "FCIT", "FLTR", "FRAS", "FRES", "GLEN", "GSK", "HBR", "HL/", 
          "HLMA", "HLN", "HSBA", "HSV", "IAG", "ICP", "IHG", "III", "IMB", "INF", 
          "ITRK", "JD/", "KGF", "LAND", "LGEN", "LLOY", "LSEG", "MNDI", "MNG", "MRO", 
          "NG/", "NWG", "NXT", "OCDO", "PHNX", "PRU", "PSH", "PSN", "PSON", "REL", 
          "RIO", "RKT", "RMV", "RR/", "RS1", "RTO", "SBRY", "SDR", "SGE", "SGRO", 
          "SHEL", "SKG", "SMDS", "SMIN", "SMT", "SN/", "SPX", "SSE", "STAN", "STJ", 
          "SVT", "TSCO", "TW/", "ULVR", "UTG", "UU/", "VOD", "WPP", "WTB",
        };
        
        using var httpClient = new HttpClient();
        var accessToken = "PersonalAccessTokenDetails
        foreach (var ticker in tickerList)
        {
          var retriesAllowed=1;
          do
          {
            var request = new HttpRequestMessage(HttpMethod.Get, 
                          $"https://api-eit.refinitiv.com/permid/search?q=ticker:{ticker}");
            request.Headers.Add("x-ag-access-token", accessToken);
            request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")
            var response = await httpClient.SendAsync(request, 
                           HttpCompletionOption.ResponseHeadersRead);
                        
            Console.WriteLine($"[{response?.StatusCode}] ticker:{ticker}");
            if ((response is {IsSuccessStatusCode: false} or null))
              break;

            try
            {
              //Deserialize resonseBody removed for brevity
              var responseBody = await response.Content.ReadAsStringAsync(                            
              retriesAllowed = 0;
            }
            catch (HttpRequestException reqEx)
            {
              Console.WriteLine($"{ticker} failed: {reqEx.Message} [{reqEx.InnerException?.Message}] retrying...");
            }
          } while (retriesAllowed > 0);
        }
      }
      catch (Exception ex)
      {
        Console.WriteLine($"{ex.Message}:{ex.InnerException?.Message}");
      }
      finally
      {
        Console.WriteLine("press ENTER to exit");
        Console.ReadLine();
      }
    }
  }
}
#productpermid-apic#open-permid-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Contacted PGO Support Permid.

Hello @glyn.jagger

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?


If so please can you click the 'Accept' text next to the appropriate reply? This will guide all community members who have a similar question.

Thanks,


AHS

Upvote
Accepted
79.2k 251 52 74

@glyn.jagger

Thank you for sharing the code. I can replicate this issue.

From my test, the server is not quite stable when handling a lot of requests. It could be this error.

127.0.0.1:58294: GET https://api-eit.refinitiv.com/permid/search?q=ticker:VOD
              << 200 OK (content missing)
 << HTTP/1 protocol error: peer closed connection without sending complete message body (incomplete chunked read)

The server cut the connection while the application was reading the data. Therefore, it is a good idea to implement retrying in the code.

I will contact the product team to see what they can do regarding this issue.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
79.2k 251 52 74

@glyn.jagger

The product team mentioned that it could be a network issue.

Therefore, implementing the retry logic in the code is the best way to handle this kind of issue.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.