perl cgi script can't post to site - works standalone but fails when running under apache

1
votes

I have written a Perl CGI script which fetches an html page from a site using POST, then does some processing on the html data and prints output html content.

My script url: http://mysite.com/cgi-bin/p.pl

Site URL: http://site.com/interesting.asp

#!/usr/bin/perl
# p.pl
...

sub post_url {
    my( $url, $formref ) = @_;
    my $ua = new LWP::UserAgent(timeout => 300);
    my $response = $ua->post($url, $formref );

if( $response->is_success ){
    return $response->content;
} else {
    return undef;
}
}

my $url = 'url: http://site.com/interesting.asp';
my %param = ('eno' => '1234', 'submit' => 'Submit');
my $htmlcontent = post_url( $url, \%param );    # <--- Not working

... do some processing on $htmlcontent ...

... print out html page ...

The script runs fine when run from the command line but fails to fetch the HTML page from the site when run from a Webserver. Is it because the script it trying to access a page from a different domain/IP address? Can anybody suggest a workaround?

3
javascriptjqueryperlcgi

3 Answers

2
votes

There could be many causes for this. Assuming that the cgi script is actually called (i.e., that it is installed in the proper location for your web server), I would ensure that LWP is actually installed on your server. Indeed, in various occasions I had to install it manually, YMMV.

It would be really helpful to output some diagnostics where the script fails... Does new LWP::UserAgent succeed? if so, what is $response->status_line after the post?

EDIT:

Since your error message is "500 permission denied", it is likely that your web server has blocked all outgoing connection. You should check whether your provider allows outgoing http connection through some proxy and configure that into LWP::UserAgent. If there is no proxy available, then possibly there is no way to go out. We cannot help here without knowing your hosting provider settings, though; maybe you can ask your hosting provider support...

0
votes

You should remove "url: " from $url:

my $url = 'http://site.com/interesting.asp';

0
votes

SELINUX enabled systems will not allow an outgoing connection from a web agent (httpd).

On CentOS 6.3 you can use this command to enable outbound web connections from Apache in a Perl script:

# setsebool -P httpd_can_network_connect on

This page can tell you more about SELinux and HTTPD settings: http://wiki.centos.org/TipsAndTricks/SelinuxBooleans