PHP + Curl Post. It fills the form, but does not post

Question

My code

<?php

$url='Search.jsp';
// disguises the curl using fake headers and a fake user agent. 
function disguise_curl($url) 
{ 
  $curl = curl_init(); 

  // Setup headers - I used the same headers from Firefox version 2.0.0.6 
  // below was split up because php.net said the line was too long. :/ 
  $header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,"; 
  $header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"; 
  $header[] = "Cache-Control: max-age=0"; 
  $header[] = "Connection: keep-alive"; 
  $header[] = "Keep-Alive: 300"; 
  $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; 
  $header[] = "Accept-Language: en-us,en;q=0.5"; 
  $header[] = "Pragma: "; // browsers keep this blank. 


  curl_setopt($curl, CURLOPT_URL, $url); 
  curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)'); 
  curl_setopt($curl, CURLOPT_HTTPHEADER, $header); 
  curl_setopt($curl, CURLOPT_REFERER, 'https://lalpacweb.blackpool.gov.uk/protected/wca/publicRegisterVehicleSearch.jsp'); 
  curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate'); 
  curl_setopt($curl, CURLOPT_AUTOREFERER, 1); 
  curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
   curl_setopt($curl, CURLOPT_COOKIESESSION, false);

   curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);

   curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($curl, CURLOPT_COOKIEJAR, "cookies.txt");
    curl_setopt($curl, CURLOPT_COOKIEFILE, "cookies.txt");
   curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt( $curl, CURLOPT_POST, 1);
     curl_setopt ($curl, CURLOPT_POSTFIELDS, 'search.licenceTypeID=34&search.licenceLinkFileID=2&search.vehicleRegNumber=5&publicRegisterVehicle=Search');
  $html = curl_exec($curl); // execute the curl command 
  echo curl_getinfo($curl, CURLINFO_HTTP_CODE);
  curl_close($curl); // close the connection 
  return $html; // and finally, return $html 
} 

// uses the function and displays the text off the website 
$text = disguise_curl($url); 
echo $text; 
?>

It returns the page, with the form filled, but it does not post it. The curl_getinfo response I get is..

200HTTP/1.1 200 OK Pragma: no-cache Cache-Control: no-cache,no-store,must-revalidate Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: text/html;charset=ISO-8859-1 Content-Language: en-GB Content-Length: 5901 Date: Sun, 19 Feb 2012 12:24:08 GMT Server: Apache

Any ideas ?

Thanks for your help

It seems your code is correct, so you should make sure the fields you request are right, the respond with status 200 shows your request is right, but might your field is incorrect, you can write a test page on server and use above code to test. — Sean
Does the status 200 code show that the form was posted though? Or would it not respond with that information? The form has two submit buttons, and I specified one by saying 'publicRegisterVehicle=Search', is that correct? — Tom
There are 2 hidden form fields on the page that might need to be set. _sourcePage and __fp — ben

pjumble pjumble · Accepted Answer · 2012-02-24T20:25:40

There's a few things you'll probably want to do, firstly I believe it works better across different operating systems if you supply an absolute path to the cookiejar:

curl_setopt($curl, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
curl_setopt($curl, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");

Also, you can have the script visit the homepage first to grab the session cookie:

disguise_curl("https://lalpacweb.blackpool.gov.uk");

Then you can post the form to https://lalpacweb.blackpool.gov.uk/protected/actions/PublicRegister.action (make sure cookies.txt exists):

<?php

// disguises the curl using fake headers and a fake user agent. 
function disguise_curl($url, $post = false) 
{ 
  $curl = curl_init(); 

  // Setup headers - I used the same headers from Firefox version 2.0.0.6 
  // below was split up because php.net said the line was too long. :/ 
  $header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,"; 
  $header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"; 
  $header[] = "Cache-Control: max-age=0"; 
  $header[] = "Connection: keep-alive"; 
  $header[] = "Keep-Alive: 300"; 
  $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; 
  $header[] = "Accept-Language: en-us,en;q=0.5"; 
  $header[] = "Pragma: "; // browsers keep this blank. 


  curl_setopt($curl, CURLOPT_URL, $url); 
  curl_setopt($curl, CURLOPT_USERAGENT, 'Googlebot/2.1 (+http://www.google.com/bot.html)'); 
  curl_setopt($curl, CURLOPT_HTTPHEADER, $header); 
  curl_setopt($curl, CURLOPT_REFERER, 'https://lalpacweb.blackpool.gov.uk/protected/wca/publicRegisterVehicleSearch.jsp'); 
  curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate'); 
  curl_setopt($curl, CURLOPT_AUTOREFERER, 1); 
  curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($curl, CURLOPT_COOKIESESSION, false);

  curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);

  curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($curl, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
  curl_setopt($curl, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");
  curl_setopt($curl, CURLOPT_HEADER, 1);
  if ($post)
  {
    curl_setopt( $curl, CURLOPT_POST, 1);
    curl_setopt ($curl, CURLOPT_POSTFIELDS, 'search.licenceTypeID=34&search.licenceLinkFileID=2&search.vehicleRegNumber=5&publicRegisterVehicle=Search');
  }
  $html = curl_exec($curl); // execute the curl command 
  //echo curl_getinfo($curl, CURLINFO_HTTP_CODE);
  curl_close($curl); // close the connection 
  return $html; // and finally, return $html 
} 

// Visit the home-page first to get the session cookie
disguise_curl("https://lalpacweb.blackpool.gov.uk"); 

// uses the function and displays the text off the website 

$url = 'https://lalpacweb.blackpool.gov.uk/protected/actions/PublicRegister.action';

$text = disguise_curl($url, true); 
echo $text; 
?>

PHP + Curl Post. It fills the form, but does not post

3 Answers