This may be a two-part challenge. The first challenge involves ensuring that your Web Resources are actually being loaded within USD, in Hosted Controls rather than external browser windows. The second challenge involves communicating between the Hosted Controls involved, once everything is happening within USD.
In USD, if window.open is launching external browser windows, then your Window Navigation Rules (or lack thereof) are allowing your Web Resource navigations to leave USD. To pass values as needed here, it's necessary to reign in such navigations using Window Navigation Rules. Define a Hosted Control that lives on FloatingPanel, and experiment with Rules until your Web Resources are loading as desired.
If the Web Resource is loading in a Hosted Control, then it just needs to speak USD. When it's time to return values, check if window.IsUSD, and raise a custom Event:
https://msdn.microsoft.com/en-us/library/dn864970.aspx#EventMoniker
This will allow you to write Action Calls that can consume your Event's key-value-pairs, which are accessible using [[replacement syntax]] during the Event. These Action Calls can RunXrmCommand on your "parent" control (which USD is simply treating as some other non-parent hosted browser), so your return values can ultimately find their way.