In Perl, how can I read an entire file into a string?

124
votes

I'm trying to open an .html file as one big long string. This is what I've got:

open(FILE, 'index.html') or die "Can't read file 'filename' [$!]\n";  
$document = <FILE>; 
close (FILE);  
print $document;

which results in: 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN

However, I want the result to look like: 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

This way I can search the entire document more easily.

15
stringperlslurp
Really should check what the definition of "Cant install" is, its a common problem and its commonly an argument that doesn't need to be made. stackoverflow.com/questions/755168/perl-myths/… – Kent Fredric
I'm actually unable to modify anything on the entire sever that this script is running on, apart from the script it self. – goddamnyouryan
So you aren't allowed to add any files, anywhere on the server? – Brad Gilbert
FatPack modules into your script? Also, it looks like you might be thinking of parsing HTML with regular expressions, don't. – MkV

15 Answers

100
votes

I would do it like this:

my $file = "index.html";
my $document = do {
    local $/ = undef;
    open my $fh, "<", $file
        or die "could not open $file: $!";
    <$fh>;
};

Note the use of the three-argument version of open. It is much safer than the old two- (or one-) argument versions. Also note the use of a lexical filehandle. Lexical filehandles are nicer than the old bareword variants, for many reasons. We are taking advantage of one of them here: they close when they go out of scope.

85
votes

Add:

 local $/;

before reading from the file handle. See How can I read in an entire file all at once?, or 

$ perldoc -q "entire file"

See Variables related to filehandles in perldoc perlvar and perldoc -f local.

Incidentally, if you can put your script on the server, you can have all the modules you want. See How do I keep my own module/library directory?.

In addition, Path::Class::File allows you to slurp and spew.

Path::Tiny gives even more convenience methods such as slurp, slurp_raw, slurp_utf8 as well as their spew counterparts.

80
votes

With File::Slurp:

use File::Slurp;
my $text = read_file('index.html');

Yes, even you can use CPAN.

56
votes

All the posts are slightly non-idiomatic. The idiom is:

open my $fh, '<', $filename or die "error opening $filename: $!";
my $data = do { local $/; <$fh> };

Mostly, there is no need to set $/ to undef.

19
votes

From perlfaq5: How can I read in an entire file all at once?:

You can use the File::Slurp module to do it in one step.

use File::Slurp;

$all_of_it = read_file($filename); # entire file in scalar
@all_lines = read_file($filename); # one line per element

The customary Perl approach for processing all the lines in a file is to do so one line at a time:

open (INPUT, $file)     || die "can't open $file: $!";
while (<INPUT>) {
    chomp;
    # do something with $_
    }
close(INPUT)            || die "can't close $file: $!";

This is tremendously more efficient than reading the entire file into memory as an array of lines and then processing it one element at a time, which is often--if not almost always--the wrong approach. Whenever you see someone do this:

@lines = <INPUT>;

you should think long and hard about why you need everything loaded at once. It's just not a scalable solution. You might also find it more fun to use the standard Tie::File module, or the DB_File module's $DB_RECNO bindings, which allow you to tie an array to a file so that accessing an element the array actually accesses the corresponding line in the file.

You can read the entire filehandle contents into a scalar.

{
local(*INPUT, $/);
open (INPUT, $file)     || die "can't open $file: $!";
$var = <INPUT>;
}

That temporarily undefs your record separator, and will automatically close the file at block exit. If the file is already open, just use this:

$var = do { local $/; <INPUT> };

For ordinary files you can also use the read function.

read( INPUT, $var, -s INPUT );

The third argument tests the byte size of the data on the INPUT filehandle and reads that many bytes into the buffer $var.

8
votes

A simple way is:

while (<FILE>) { $document .= $_ }

Another way is to change the input record separator "$/". You can do it locally in a bare block to avoid changing the global record separator.

{
    open(F, "filename");
    local $/ = undef;
    $d = <F>;
}
7
votes

Either set $/ to undef (see jrockway's answer) or just concatenate all the file's lines:

$content = join('', <$fh>);

It's recommended to use scalars for filehandles on any Perl version that supports it.

4
votes

Another possible way:

open my $fh, '<', "filename";
read $fh, my $string, -s $fh;
close $fh;
4
votes

Use

 $/ = undef;

before $document = <FILE>;. $/ is the input record separator, which is a newline by default. By redefining it to undef, you are saying there is no field separator. This is called "slurp" mode.

Other solutions like undef $/ and local $/ (but not my $/) redeclare $/ and thus produce the same effect.

3
votes

You're only getting the first line from the diamond operator <FILE> because you're evaluating it in scalar context:

$document = <FILE>;

In list/array context, the diamond operator will return all the lines of the file. 

@lines = <FILE>;
print @lines;
2
votes

I would do it in the simplest way, so anyone can understand what happens, even if there are smarter ways:

my $text = "";
while (my $line = <FILE>) {
    $text .= $line;
}
2
votes
open f, "test.txt"
$file = join '', <f>

<f> - returns an array of lines from our file (if $/ has the default value "\n") and then join '' will stick this array into.

2
votes

This is more of a suggestion on how NOT to do it. I've just had a bad time finding a bug in a rather big Perl application. Most of the modules had its own configuration files. To read the configuration files as-a-whole, I found this single line of Perl somewhere on the Internet:

# Bad! Don't do that!
my $content = do{local(@ARGV,$/)=$filename;<>};

It reassigns the line separator as explained before. But it also reassigns the STDIN.

This had at least one side effect that cost me hours to find: It does not close the implicit file handle properly (since it does not call closeat all).

For example, doing that:

use strict;
use warnings;

my $filename = 'some-file.txt';

my $content = do{local(@ARGV,$/)=$filename;<>};
my $content2 = do{local(@ARGV,$/)=$filename;<>};
my $content3 = do{local(@ARGV,$/)=$filename;<>};

print "After reading a file 3 times redirecting to STDIN: $.\n";

open (FILE, "<", $filename) or die $!;

print "After opening a file using dedicated file handle: $.\n";

while (<FILE>) {
    print "read line: $.\n";
}

print "before close: $.\n";
close FILE;
print "after close: $.\n";

results in:

After reading a file 3 times redirecting to STDIN: 3
After opening a file using dedicated file handle: 3
read line: 1
read line: 2
(...)
read line: 46
before close: 46
after close: 0

The strange thing is, that the line counter $. is increased for every file by one. It's not reset, and it does not contain the number of lines. And it is not reset to zero when opening another file until at least one line is read. In my case, I was doing something like this:

while($. < $skipLines) {<FILE>};

Because of this problem, the condition was false because the line counter was not reset properly. I don't know if this is a bug or simply wrong code... Also calling close; oder close STDIN; does not help.

I replaced this unreadable code by using open, string concatenation and close. However, the solution posted by Brad Gilbert also works since it uses an explicit file handle instead.

The three lines at the beginning can be replaced by:

my $content = do{local $/; open(my $f1, '<', $filename) or die $!; my $tmp1 = <$f1>; close $f1 or die $!; $tmp1};
my $content2 = do{local $/; open(my $f2, '<', $filename) or die $!; my $tmp2 = <$f2>; close $f2 or die $!; $tmp2};
my $content3 = do{local $/; open(my $f3, '<', $filename) or die $!; my $tmp3 = <$f3>; close $f3 or die $!; $tmp3};

which properly closes the file handle.

1
votes

I don't know if it's good practice, but I used to use this:

($a=<F>);
0
votes

You could simply create a sub-routine:

#Get File Contents
sub gfc
{
    open FC, @_[0];
    join '', <FC>;
}