Twitter REST API - Number of Tweets

Question

I wrote a little Script using Python and Tweepy to save the tweets for a list of users and also to get some basic properties for those accounts.

Somehow the number of tweets stated in the user profile under statuses_count

(for an example of the json description of an account: https://api.twitter.com/1/users/show.json?screen_name=TwitterAPI&include_entities=true )

does not match the number of tweets i get when iterating through the tweets of the same users profile.

I am aware of the fact, that twitter limits the number of tweets per user available through the API to 3200 and even does not guarantee this number, but this behavior does even occur with users who have well less than 3200 tweets

My question is, whether this difference is common and why this happens? Is this just an issue of the twitter API, is it caused by deleted tweets (maybe they still count for statuses_count but can not be fetched anymore?), ...?

Thanks!

Thomas

Answer 1

I haven't messed with the Twitter API in several months, but I remember back when I was working with it I found inconsistencies due to retweets not showing up when iterating tweets, but getting counted in the number of Tweets. This seems to corroborate that, but its several months old and things may have changed since then.

Answer 2

Make sure include_rts is set to true, t, or 1 (in addition to specifying the same for include_entities, which you have done). When these aren't included by default (e.g. user lists) then you can get fewer tweets than what you specified with count.

The Twitter API documentation isn't clear on what the defaults are so it's safer to explicitly specify these optional parameters. And since you're specifically working with the user timeline you might also want exclude_replies turned off.