Understanding Twitter API rate limits using sandbox environment and searchtweets API

Question

I am trying to pull tweets over a 2 year period for a number of Twitter users. I am using the searchtweets library in Python and the sandbox version of the API.

I am using their Result Stream method:

import searchtweets
rule = gen_rule_payload("from:BBCNews", from_date="2017-01-01",
                            to_date="2017-01-01", results_per_call=100) 
rs = ResultStream(rule_payload=create_rule(i),
                                  max_results=18000,  
                                  max_pages=1,
                                  **premium_search_args) # these are credentials, ommitted for security 
                    tweets = list(rs.stream())

However, I get this returned:

HTTP Error code: 429: Request exceeds account’s current package request limits. Please upgrade your package and retry or contact Twitter about enterprise access.

I have reviewed the rate limits (https://developer.twitter.com/en/docs/basics/rate-limiting) and thought I had correctly deduced that the rate limit is 180 per 15 minutes for this sort of request, and therefore limiting max_results to 18,000 would prevent this one request for BBC News tweets exceeding the rate limit for a 15-minute window. There are 100 results returned in each request, so 18,000 results would be 100 results*180 requests.

This is not working, so I think I must have misunderstood. If anyone could provide some further insight that would be great.

Andy Piper Andy Piper · Accepted Answer · 2020-04-17T22:11:39

The web page you’ve cited states (first line at the top):

Standard API endpoints only, does not apply to premium APIs

The 180 requests per 15 minutes refers to the standard (legacy) 1.1 search API, limited to a 7 day non-complete index. In premium search with a sandbox, if you are searching the full archive product, you have 50 requests for the month, with up to 100 Tweets per request (see your app dashboard “subscriptions” section). In the paid tiers, you have up from 100 to 2500 requests per month each of which can cover up to 500 Tweets, with more complex search rules (1024 chars over the 128 chars in sandbox)

Check the apps dashboard Subscriptions panel which explains where you’re at in terms of requests for the month.

Understanding Twitter API rate limits using sandbox environment and searchtweets API

1 Answers