Rate Limit Prevention [Twitter4J]

Question

I am currently implementing a program which requires me to use Twitter4J to collect tweets and store them. However, I do realise that you can only make 180 requests every 15 minutes with Twitter's Developer API.

Due to this, I have created a method that stops the program after it gets 10 tweets, for 15 minutes while my consumer and access keys reset the rate limit. However, sometimes the Rate Limit still manages to run out in between getting those 10 tweets? So what I want to do is change the method so that it stops due to the rate limit about to run out.

For example...

if (rate limit = 0){
   stop program until rate limit resets
}

However, my method just now just implements a counter and when that counter reaches 10, it stops, which isn't very economical or effective. I thought 10 tweets would be an adequate amount but obviously not. Here is my method...

public void getRateLimit() throws TwitterException{
        if (count == 10){
            try {
                System.out.println("Rate Limit is about to be exhausted for resource...");
                System.out.println("Please wait for 15 minutes, while it resets...");
                TimeUnit.MINUTES.sleep(15);
                count = 0;
            } catch (InterruptedException e) {
                System.out.println(e);
            }
}

How could I alter this so that it runs out when the rate limit is about to run out and stops, and only starts when it is replenished. Thanks for any help.

kadir kadir · Accepted Answer · 2016-04-18T07:05:22

I have been faced same problem before and I try to calculate time spend for 1 request. If that request less than 5500 milisecond than program waits till it comes 5500 milisecond and it works perfectly for me,

You can ask why 5500 milisecond, it is because 180 requests 15 minutes makes it 5 second per request.

Here is code I use, hope it helps.

do {
    final long startTime = System.nanoTime();
    result = twitter.search(query);
    statuses = result.getTweets();
    for (Status status : statuses) {
        tweet = new Tweet(status);
        userProfile = new UserProfile(status.getUser());

        imageDownloader.getMedia(tweet.mediaEntities);
        imageDownloader.getProfilePhoto(userProfile.ProfileImageUrl);

        System.out.println(tweet);
        System.out.println(userProfile);
    }
    final long duration = System.nanoTime() - startTime;
    if ((5500 - duration / 1000000) > 0) {
        logger.info("Sleep for " + (6000 - duration / 1000000) + " miliseconds");
        Thread.sleep((5500 - duration / 1000000));
    }
} while ((query = result.nextQuery()) != null);

Rate Limit Prevention [Twitter4J]

1 Answers