Eikon .Net API accepts 2-3 time series requests and returns data but then hangs without throwing ...

...any errors thrown.

I wrote a .Net C# wrapper to request and return historical time series data via Eikon .Net API. I can connect to Reuters servers, and can request 2, sometimes 3 time series but then the application hangs indefinitely (the problem 100% does not originate from the request content itself as I only request each time 1 day worth of 1 minute compressed fx data).

One complication is that I must access the Reuters API from within a console app thread and hence had to run a separate message pump. Please see below code :

Helper Methods:

private void PushDispatcherFrame(Action action)
        {
            _dispatcher.Invoke(() =>
            {
                //execute action
                action();


                //push frame
                _frame = new DispatcherFrame();
                Dispatcher.PushFrame(_frame);
            });
        }


        private void ReleaseFrame(bool isContinue)
        {
            _dispatcher.Invoke(() =>
            {
                _frame.Continue = isContinue;
            });
        }

Connection code:

PushDispatcherFrame(() =>
            {
                _dataServices = DataServices.Instance;
                _dataServices.StateChanged += DataServicesOnStateChanged;
                _dataServices.Initialize("ReutersMarketDataPlugin");
            });


            //wait until service state changes (with timeout)
            var startDt = DateTime.UtcNow;
            var timeoutDt = startDt + TimeSpan.FromSeconds(40);
            while (DateTime.UtcNow < timeoutDt && _serviceState != DataServicesState.Up)
            {
                Task.Delay(200).Wait();
            }


            if (_serviceState == DataServicesState.Up)
                return true;
            else
                return false;

        }


        private void DataServicesOnStateChanged(object sender, DataServicesStateChangedEventArgs args)
        {
            _serviceState = args.State;


            ReleaseFrame(false);
        }

Here is the data request (and after 2-3 successful requests the app hangs after executing "request.CreateAndSend();"

//structure request
            PushDispatcherFrame(() =>
            {
                var subscription = new TaggableTimeseriesSubscription(dto, OnSubscriptionDataReceived, OnSubscriptionDataUpdated);
                _historicalSubscriptions.Add(subscription.SubscriptionId, subscription);


                ITimeSeriesDataRequestSetup request = _dataServices.TimeSeries.SetupDataRequest();
                request.WithRic(GetRic(dto.SymbolId));
                request.From(dto.DateTimeFrom);
                request.To(dto.DateTimeTo);
                request.WithFields(GetHistoricalDataRequestFields(dto.QuoteType));
                request.WithTimeZone(TimezoneType.GMT);
                request.WithInterval(GetTimeSeriesInterval(dto.Compression));
                request.OnDataReceived(subscription.RegisterDataReceivedCallback);
                request.OnStatusUpdated(status =>
                {
                    //error captured
                    SendSystemMessage(new SystemMessage(SystemMessageType.Info, _properties.PluginName, dto.SubscriberId,
                        "Status: " + status.State + " - Message: " + status.Error));


                    //return request
                    _historicalDataCallback(subscription.HistoricalRequest);


                });


                //create and send
                subscription.ReutersHistoricalTimeSeriesRequest = request.CreateAndSend();
            });

And here the callback:

private void OnSubscriptionDataReceived(string subscriptionId, DataChunk chunk)
        {
            // chunk.Records   provides access to a dictionary of field -> value, so that
            // developpers can access values from field names and cast them in the proper type like in:
            // double open = (double) chunk.Records.First()["OPEN"];


            // In addition, for convenience Time Series API exposes shortcuts that allow developers 
            // to code against strongly typed accessors. This is done through conversion methods
            // such as ToBarRecords(), ToQuoteRecords(), ToTickRecords(), ToTradeRecords() and ToTradeAndQuoteRecords().
            // The full dictionary is still exposed in the resulting strongly-typed classes.
            // In this sample we know that records are typical "bar" records (open, close, high, low, volume) 
            // because we ask for a daily interval. So we can use ToBarRecords() helper method:


            if (_historicalSubscriptions.ContainsKey(subscriptionId))
            {
                //add quotes to request
                var request = _historicalSubscriptions[subscriptionId].HistoricalRequest;
                var quotes = RecordsToQuotes(request, chunk.Records);
                request.Quotes.AddRange(quotes);


                //return data if request is completed
                if (chunk.IsLast)
                {
                    //filter and sort quotes and update request
                    request.Quotes = request.Quotes.Where(x => x.TimeStamp >= request.DateTimeFrom && x.TimeStamp <= request.DateTimeTo).OrderBy(x => x.TimeStamp).Distinct().ToList();


                    //remove request
                    _historicalSubscriptions.Remove(subscriptionId);


                    //ReleaseFromMessagePump();


                    //return data
                    if (_historicalDataCallback != null)
                    {
                        _historicalDataCallback(request);
                    }


                    ReleaseFrame(false);
                }
                else
                {
                    ReleaseFrame(true);
                }
            }
        }

Find more posts tagged with

.net

eikon

eikon-com-api

eikon-data-api

Accepted answers

Alex Putkov.1

Hi @HFTVOL,

We strongly recommend single threaded architecture for the data retrieval part of the code using Eikon .NET SDK. The reason is the .NET assemblies in the SDK for historical reasons have a dependency on a single threaded COM library. This COM library is used by multiple components in Eikon, the .NET SDK is just one of them. The dependency on the COM library is precisely the reason why Windows message pump is required, as I explained to you previously on a different thread.

All comments

HFTVOL

Bump. Can a developer please chime in here or is there a way to communicate directly via phone or email? I have now been struggling with this topic for weeks and do not see any solution you previously provided worked. The main issue is the use Windows message pumps because the API becomes unstable when making requests or connecting from different threads, sometimes a UI thread, sometimes a non UI thread, sometimes from within a Console Application. I need a solution to this urgently. I am extremely frustrated and consider cancelling the entire Eikon subscription and moving to a competitor.

Jirapongse

@HFTVOL

Could you please share the cut-down version of your program with the source code? Therefore, I can use it to replicate and investigate the issue in my environment.

HFTVOL

@Jirapongse

ok, let me please ask you a question which may get to the ground of the whole issue: Is IDataServices and ITimeDataServices thread safe? After further debugging I believe it is not which would not make any sense given your request/callback methodology is async. Problem is that when a request for time series data is made that such request comes from different threads. However, I only instantiate IDataServices once and keep it connected and use it to structure new time series requests. Hence, IDataServices was created in a different thread...

HFTVOL

...than later time series requests. Is the only solution here to create an entirely new IDataServices instance each time a time series request is made GIVEN THE FACT that subsequent requests CAN potentially originate from different threads?

Jirapongse

From the snippet code, the application calls PushDispatcherFrame to create create a IDataServices and structure the request. It means that it uses the same _dispatcher thread to call the code.

private void PushDispatcherFrame(Action action)
        {
            _dispatcher.Invoke(() =>
            {
                //execute action
                action();
                //push frame
                _frame = new DispatcherFrame();
                Dispatcher.PushFrame(_frame);
            });
        }

Therefore, it should be thread safe because the code will be called in order by the dispatcher thread (_dispatcher).

HFTVOL

Mate, this is my code!!! I know what my code does. The problem here is that the application hangs on the second request at "Dispatcher.PushFrame(_frame)". I am almost sure that you have a huge bug in your API. Your API simply does not work when the same request is called from different threads. Why is it so hard to get to talk to a developer at Reuters? I have now asked many times to be put in touch with a developer. At Bloomberg it takes a phone call and 10 seconds before I am forwarded to a developer who can work with me directly. Can you PLEASE put me in touch with someone via phone?

zoya faberov

Hello @HFTVOL,

If you are a customer, please submit your query to My Account, https://my.thomsonreuters.comand click "RAISE A CASE". Then, an appropriate consultant will contact you and assist you on this issue.

If you are exploring the Eikon.NET API, we are glad to help, as Jirapongse has suggested, please provide a cut-down version of your code which we can use to reproduce the issue on our testbed, and any pertaining information that we would need to reproduce.

With respect,

-AHS

HFTVOL

@jirapongse.phuriphanvichai

@zoya faberov

I launched a query in my Account section. And I am not just "exploring" the API, I am a paying customer and would appreciate if support was a little more forthcoming.

Here is the reproducible code. Please note that the code works fine when running all requests from the same task/thread. As soon as multiple tasks are involved something internally in your API hangs. Goal is to connect/initialize ONCE only, but request many times from potentially different threads.

using System;
using System.Threading.Tasks;
using System.Windows.Threading;
using ThomsonReuters.Desktop.SDK.DataAccess;
using ThomsonReuters.Desktop.SDK.DataAccess.TimeSeries;


namespace ReutersTestCase
{
    class Program
    {
        static void Main(string[] args)
        {
            var testCode = new TestCode();


            //initialize
            testCode.Initialize();


            var task1 = Task.Run(() =>
            {
                testCode.SubmitRequest();
            }).ContinueWith(a =>
            {
                var task2 = Task.Run(() =>
                {
                    testCode.SubmitRequest();
                }).ContinueWith(b =>
                {
                    var task3 = Task.Run(() =>
                    {
                        testCode.SubmitRequest();
                    });
                });


            });



            Console.WriteLine("Press enter to exit");
            Console.ReadLine();
        }
    }


    public class TestCode
    {
        private int _iterationId;
        private IDataServices _currentDataService;
        private ITimeSeriesDataService _timeSeriesServices;
        private ITimeSeriesDataRequest _currentRequest;
        private DispatcherFrame _currentFrame;
        private Dispatcher _currentDispatcher;
        private bool _isInitCompleted;
        private bool _isRequestCompleted;

        public TestCode()
        {
            _isInitCompleted = false;
            _currentDispatcher = Dispatcher.CurrentDispatcher;
        }            


        public void Initialize()
        {
            _currentFrame = new DispatcherFrame();


            _currentDataService = DataServices.Instance;
            _currentDataService.StateChanged += CurrentDataServiceOnStateChanged;
            _currentDataService.Initialize("TestApp");
            _timeSeriesServices = DataServices.Instance.TimeSeries;
            _timeSeriesServices.ServiceInformationChanged += TimeSeriesServicesOnServiceInformationChanged;


            Dispatcher.PushFrame(_currentFrame);


            //wait hack
            while (_isInitCompleted == false)
            {
                Task.Delay(200).Wait();
            }


            Console.WriteLine("Init completed");
        }


        public void SubmitRequest()
        {
            PushDispatcherFrame(new Action(() =>
            {
                _isRequestCompleted = false;


                _currentFrame = new DispatcherFrame();


                _currentRequest = _timeSeriesServices.SetupDataRequest("EUR=")
                    .WithView("BID")
                    .WithAllFields()
                    .WithInterval(CommonInterval.Daily)
                    .WithNumberOfPoints(10)
                    .OnDataReceived(DataReceivedCallback)
                    .CreateAndSend();


                Dispatcher.PushFrame(_currentFrame);








                //wait hack
                while (_isRequestCompleted == false)
                {
                    Task.Delay(200).Wait();
                }
            }));
        }


        private void CurrentDataServiceOnStateChanged(object sender, DataServicesStateChangedEventArgs dataServicesStateChangedEventArgs)
        {
            ReleaseFrame(false);
            _isInitCompleted = true;
        }


        private void TimeSeriesServicesOnServiceInformationChanged(object sender, ServiceInformationChangedEventArgs serviceInformationChangedEventArgs)
        {
            //do nothing here
        }


        private void DataReceivedCallback(DataChunk chunk)
        {
            foreach (IBarData bar in chunk.Records.ToBarRecords())
            {
                if (bar.Open.HasValue && bar.High.HasValue && bar.Low.HasValue && bar.Close.HasValue && bar.Timestamp.HasValue)
                {
                    Console.WriteLine(
                        "{0}: EUR= OHLC {1} {2} {3} {4}",
                        bar.Timestamp.Value.ToShortDateString(),
                        bar.Open.Value.ToString("##.0000"),
                        bar.High.Value.ToString("##.0000"),
                        bar.Low.Value.ToString("##.0000"),
                        bar.Close.Value.ToString("##.0000")
                    );



                };
            }


            if (!chunk.IsLast)
                return;

            ReleaseFrame(false);


            Console.WriteLine();


            _currentRequest = null;
            _isRequestCompleted = true;
        }


        private void PushDispatcherFrame(Action action)
        {
            _currentDispatcher.Invoke(() =>
            {
                //create new frame
                _currentFrame = new DispatcherFrame();


                //execute action
                action();


                //push frame
                Dispatcher.PushFrame(_currentFrame);
            });
        }


        private void ReleaseFrame(bool isContinue)
        {
            _currentDispatcher.Invoke(() =>
            {
                _currentFrame.Continue = isContinue;
            });
        }
    }
}

Jonathan Haruni - Refinitiv

@HFTVOL we will look into thread-safety of the methods you're calling and provide an update here. To get 1:1 support for Eikon APIs we need to know your Eikon ID but you've not provided enough information in your Developer Community registration. Go to the Developer Community home page, click on your profile in the top-right of the screen and complete your profile with your real name and Company name or your business email address as used in your Eikon subscription. Or you can provide these or your business email address here in a response but that would be publicly visible.

Jirapongse

@HFTVOL

I have modified the code. Then, it runs fine in my environment.

output.txt program-cs.txt

HFTVOL

Thanks, it works but I hope it becomes now apparent that the API is anything but thread safe and should rather be considered an alpha version and "hack" than anything that can run in production...it requires the end user to manage threading him/herself and consumes an additional thread just in order to run the dispatcher on that thread.

Alex Putkov.1

Hi @HFTVOL,

Alex Putkov.1

You are of course perfectly entitled to your opinion about this SDK. I can only say that it is used in production by thousands of applications. And I should add that we consider this interface legacy. In two weeks we're Beta launching new Eikon Scripting API, which is based on Websockets for streaming data and on REST/POST for request/response, and which does not have any dependency on COM.

Quick Links

All Forums
Recent Questions

EXPLORE OUR SITES