Throttling at Office 365 Management Activity API

Question

Our multi-tenant system is using Office 365 management activity API to retrieve the data in the following approaches:

1. O365 activity data is stored as blocks identified by contentUri
2. For each tenant, retrieve contentUri list with start time and end time (we are breaking retrieval to 2 hours window), please see activity API reference
3. Then iterate through above contentUri list to retrieve the O365 activity data for current tenant.

We are facing throttling issue right now at above step #3 in some customers. For some very large customers in some regions, there could be a lot of contentUri. We could only retrieve activity data for average 1 contentUri or a little bit more per second and 60 or less per minute which is very well below the 60k per minute stated at activity API troubleshooting document.

We tried to improve the performance after trying all the following approaches without any success:

1. Decorate the http request with user agent as ISV/vendor name/product name
2. Retrieve in parallel with 2 – 5 threads: First request would be quick, then subsequent would be slow even with 2 requests in parallel
3. Office365 Management API is using PublishIdentifier to allocate the throttling quota about 60K request per minute per tenant as activity API troubleshooting document. We tried by having our company's tenant GUID (as ISV), customer tenant GUID or some random GUID as PublishIdentifier, and we didn’t see any performance improvement.

Any suggestions for what we should do to improve the performance?

Answer 1

As the doc tells: If you’re implementing a client for your company’s tenant, the PublisherIdentifier is the Tenant GUID. If you are creating an ISV application or add-in for multiple customers, the PublisherIdentifier should be the ISV’s Tenant GUID, and not the tenant GUID of end user’s company. And from my experience, the PublishIdentifier should be the tenant GUID where your APP is registered.