The UPA tunneling solution uses Microsoft WinInet to establish its connections, and each RsslChannel you see in your application is really using two underlying physical connections in order to efficiently stream data.
There is a registry parameter that controls the number of simultaneous connections that WinInet is allowed to make and it sounds like you are bumping into that. If WinInet is unable to create a new connection, rsslInitChannel will continue attempting the handshake until a failure occurs or until WinInet is able to create a new connection/handle (usually because another one has gone away).
This link is from MSDN and describes the registry settings to control and increase this
https://support.microsoft.com/en-us/kb/183110/en-us
Additionally, this solution requires that the Certificate from the server is valid and recognized, so this could be resulting in some of your issues. Make sure you have the certificate from the host you are connecting to installed in your IE Certificate Store (under Tools->Internet Options->Content->Certificates).
Lastly, the setting MaxConnectionsPerProxy is also required.
