I am working on a website where right rail and menu components will be using an external data source exclusively. The external source is a Lucene based index which sits on a different server.
I want to implement Sitecore caching on these components but I want the cache to refresh when new data is available for the component in the index. New data will be available very frequently. I am talking in terms of seconds not minutes or hours in some cases. How can I achieve Sitecore caching in this instance?
I am using Sitecore 6.5 for this website.