Using Twitter API on shared server - Rate limit exceeded even though I am caching the response

Question

I have written a php script which gets the latest status update for 12 different twitter accounts by pulling an xml for each and caching it on my server. This currently runs every 30 minutes.

Unfortunately I keep getting the "Rate limit exceeded. Clients may not make more than 150 requests per hour." error event though i'm only making 24 requests from the 150 I should have.

I assume this is because my domain is on a shared server and twitter is counting other requests against me.

How can I authorise my requests so i'm not restriced by the standard IP limit?

I have no experience of OAuth so need step by step instructions if possible.

Thanks in advance!

jamesakers jamesakers · Accepted Answer · 2012-07-30T14:03:59

OK so I managed to get the most of this working with no previous experience of API's etc.

Here is my step by step guide:

Step 1. Create a Twitter list.

Go to: https://twitter.com/username/lists
Click "Create list"
Enter details and save.
Go to a twitter user you wish to add to the list and click the gear dropdown and select "Add or remove from lists". Tick the checkbox next to your list.

Step 2. Create a Twitter App via: https://dev.twitter.com/apps/new

Log in using your Twitter credentials.
Give your app a name, description etc.
Go to the Settings tab and change the Access type to Read and Write then click "Update this Twitter application's settings".
Click "Create my access token" at the bottom of the page.

You will now have a Consumer Key, Consumer secret, Access token and Access token secret. Make a note of these.

Step 3. Create API tokens.

Download and install onto your server the Abraham Twitter oAuth library from: https://github.com/abraham/twitteroauth (I'll use a folder called "twitter").

Create a new file, name it authorise.php in the oAuth folder and put the following code inside (with your generated keys in place of the named text). (Put the code between < ? PHP and ?> brackets).

// Create our twitter API object
require_once("twitteroauth/twitteroauth.php");
$oauth = new TwitterOAuth('Put-Consumer-Key-here', 'Put-Consumer-secret-here', 
'Put-Access-Token-here', 'Put-Access-token-secret-here');
// Send an API request to verify credentials
$credentials = $oauth->get("account/verify_credentials");
echo "Connected as @" . $credentials->screen_name;
// Post our new "hello world" status
$oauth->post('statuses/update', array('status' => "hello world"));

This has now authorised your twitter App for the API and posted a "hello world" status on your twitter account.

Note: The Read / Write access change we did earlier alowed the code to post the status update, it's not actually needed to pull the list from the API but I did it to make sure it was working OK. (You can turn this off again by going back to the Settings).

Step 4. Create PHP file to pull your list and cache the file.

Create an XML file (YOUR-FILE-NAME.xml) and save it in the oAuth folder.
Create a PHP file (YOUR-PHP-FILE.php) and save it in the oAuth folder

Edit the below code with your twitter API keys, file name and twitter list details and save it in your PHP file. (Put the code within < ? PHP and ?> brackets).

/* Twitter keys & secrets here */
$consumer_key = 'INSERT HERE';
$consumer_secret = 'INSERT HERE';
$access_token = 'INSERT HERE';
$access_token_secret = 'INSERT HERE';

// Create Twitter API object
require_once('twitteroauth/twitteroauth.php');
// get access token and secret from Twitter
$oauth = new TwitterOAuth($consumer_key, $consumer_secret, $access_token,                 $access_token_secret);
// fake a user agent to have higher rate limit
$oauth->useragent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9';

// Send an API request to verify credentials
$credentials = $oauth->get('account/verify_credentials');
echo 'Connected as @' . $credentials->screen_name . '\n';

// Show API hits remaining
$remaining = $oauth->get('account/rate_limit_status');
echo "Current API hits remaining: {$remaining->remaining_hits}.\n";

$ch = curl_init();
$file = fopen("YOUR-FILE-NAME.xml", "w+");
curl_setopt($ch, CURLOPT_URL,'https://api.twitter.com/1/lists/statuses.xml?slug=INSERT-LIST-NAME&owner_screen_name=INSERT-YOUR-TWITTER-USERNAME-HERE&include_entities=true');
curl_setopt($ch, CURLOPT_FILE, $file);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($file);?>

Copy the file path into your browser and test it. (e.g. http://www.yourwebsite.com/twitter/YOUR-PHP-FILE.php)

This should contact twitter, pull the list as an XMl file and save it into YOUR-FILE-NAME.xml. Test it by opening the XML file, it should have the latest statuses from the users in your twitter list.

Step 5. Automate the PHP script to run as often as you like (up to 350 times per hour) via a Cron job.

Open your Cpanel and click "Cron jobs" (usually under Advanced).
You can choose the regularity of your script using the common settings.
In the command field add the following code:

php /home/CPANEL-USERNAME/public_html/WEBSITE/twitter/YOUR-PHP-FILE.php >/dev/null 2>&1

Your script will now run as often as you have chosen, pull the list from twitter and save it into YOUR-FILE-NAME.xml.

Step 6. You can now pull statuses from the cached XML file meaning your visitors will not be making unnecessary calls to the API.

I've not worked out how to target a specific screen_name yet if anyone can help there?

Using Twitter API on shared server - Rate limit exceeded even though I am caching the response

2 Answers