Downloading tweets with twitteR does not work as hoped

Question

I'm fairly new to R, I use it for a course on network analysis at my university. As part of a research project, I want to analyse tweets by Donald Trump and Hillary Clinton. I successfully managed to grant RStudio access to my twitter account, but every time I try to download tweets, I get a fairly meager selection ranging from 1,100 tweets at best to just 800-900 tweets at worst. I do not understand this as I do not get any error message, either. Am I missing something? I thought the limit on downloading tweets was at 3,200?

This is my code:

#load twittR package and necessary tool for login
library(twitteR)
library(ROAuth)
#load login data
api_key <- "blah"
api_secret <- "blah"
access_token <- "blah"
access_token_secret <- "blah"
#login
setup_twitter_oauth(api_key,api_secret,access_token,access_token_secret)
#retreive tweets by Donald Trump, maximum number is 3200
tweetsTrump <- userTimeline("realDonaldTrump", n=3200)
#convert those tweets to a dataframe
Trump.df <- twListToDF(tweetsTrump)

I am eternally grateful for every useful tip!

As mentioned in numerous similar questions: "The Twitter Search API searches against a sampling of recent Tweets published in the past 7 days.". It's not an archive for all tweets that everyone can query as he or she wishes... — lukeA
@lukeA, I'd add that as an answer. I can't seem to find any examples in R to mark this as a duplicate. — sebastian-c
@sebastian-c Hm I found e.g. stackoverflow.com/questions/22052811/… via google.com/…" — lukeA
searchTwitter functions differently than userTimeline, and going past 7 days is doable with userTimeline so both the question and answer are different in this case. — calder.ty

lukeA lukeA · Accepted Answer · 2016-10-05T08:47:01

Have a look at the Twitter API documentation where it says:

The Twitter Search API searches against a sampling of recent Tweets published in the past 7 days.

Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.

Thus, the results from the API are limited per se. If you want more, use the streaming api or services like Gnip.

Downloading tweets with twitteR does not work as hoped

1 Answers