Retrieving tweets from twitter using twitter4j

4
votes

I am developing an application to guess locations of tornadoes by analyzing twitter data. For this, I would first need to train a neural network on some manually annotated tweets. I am trying to get tweets from last year which have the word 'tornado' in them. This is my code below :-

Query query = new Query("tornado");
query.setRpp(100);
query.setSince("2010-11-01");
query.setUntil("2011-01-13");

QueryResult queryResult = instance.search(query);
tweetList = queryResult.getTweets();

I am able to retrieve tweets from periods closer to now such as last week and such, but am unable to get any results for periods such as the one listed above. Any clues, suggestions would be help. Thanks in advance.

1
twittertwitter4j

1 Answers

6
votes

I just found out the reason through a different medium, thought i'd share the answer in case there are other people with the same issue.

It turns out that the twitter search api does not return tweets older than around a week and also, depending on the server load, at times this could be as low as 24 hours ! Hence, any 3rd party libraries (such as twitter4j) which have a wrapper for the twitter search api will behave similarly.

The best way to go about this would be to use third party search and indexing sites such as snapbird, topsy, etc..