Creating multiple hashes from multiple files in one go

Question

I want to perform a vlookup like process but with multiple files wherein the contents of the first column from all files (sorted n uniq-ed) is reference value. Now I would like to store these key-values pairs from each file in each hash and then print them together. Something like this:

file1: while(){$hash1{$key}=$val}...file2: while(){$hash2{$key}=$val}...file3: while(){$hash3{$key}=$val}...so on

Then print it: print "$ref_val $hash1{$ref_val} $hash3{$ref_val} $hash3{$ref_val}..."

$i=1;
@FILES = @ARGV;
foreach $file(@FILES)
{
open($fh,$file);
$hname="hash".$i; ##trying to create unique hash by attaching a running number to hash name
while(<$fh>){@d=split("\t");$hname{$d[0]}=$d[7];}$i++;
}
$set=$i-1;       ##store this number for recreating the hash names during printing

open(FH,"ref_list.txt");
while(<FH>)
{
chomp();print "$_\t";
## here i run the loop recreating the hash names and printing its corresponding value
for($i=1;$i<=$set;$i++){$hname="hash".$i; print "$hname{$_}\t";}
print "\n";
}

Now this where I am stuck perl takes $hname as hash name instead of $hash1, $hash2...

Thanks in advance for the helps and opinions

You are trying to use symbolic reference, to create variable names dynamically; it's only trouble and (almost) never necessary. Why not simply put references to those hashes in an array? — zdim
The hash is made with key of the first element ($d[0]) and value of ... the eight-th ($d[7]) -- is that intended? — zdim
@zdim...yes, correctly understood. Each line has 10 columns, but i need to form the hash out of column 1 (key) and column 7 (value)....yes that symbolic reference aspect did come, but i wasn't sure how to apply it here... :( — SangramJB
OK, thank you. The symbolic reference business -- you don't want it. Just don't do it. Now, your explanation says that "contents of the first column from all files (sorted n uniq-ed) is reference value" -- how is that? The "first column" from all files is a long list of values, how is that a reference? Can you show a sample? — zdim
OK...now i have files like : exp1.txt, exp2.txt, exp3.txt so on. To collect and create a list of all reference values I do this : cut -f 1 *.txt | sort | uniq > ref_list.txt — SangramJB

zdim zdim · Accepted Answer · 2018-11-10T09:33:35

The shown code attempts to use symbolic references to construct variable names at runtime. Those things can raise a lot of trouble and should not be used, except very occasionally in very specialized code.

Here is a way to read multiple files, each into a hash, and store them for later processing.

use warnings;
use strict;
use feature 'say';

use Data::Dump qw(dd);

my @files = @ARGV;

my @data;
for my $file (@files) {
    open my $fh, '<', $file  or do {
        warn "Skip $file, can't open it: $!";
        next;
    };
    push @data, { map { (split /\t/, $_)[0,6] } <$fh> };
}

dd \@data;

Each hash associates the first column with the seventh (index 6), as clarified, for each line. A reference to such a hash for each file, formed by { }, is added to the array.

Note that when you add a key-value pair to a hash which already has that key the new overwrites the old. So if a string repeats in the first column in a file, the hash for that file will end up with the value (column 7) for the last one. The OP doesn't discuss possible duplicates of this kind in data files (only for the reference file), please clarify if needed.

The Data::Dump is used only to print; if you don't wish to install it use core Data::Dumper.

I am not sure that I get the use of that "reference file", but you can now go through the array of hash references for each file and fetch values as needed. Perhaps like

open my $fh_ref, '<', $ref_file or die "Can't open $ref_file: $!";

while (my $line = <$fh_ref>) {

    my $key = ...   # retrieve the key from $line
    print "$key: ";

    foreach my $hr (@data) {
        print "$hr->{$key} ";
    }
    say '';
}

This will print key: followed by values for that string, one from each file.

Creating multiple hashes from multiple files in one go

1 Answers