Perl - sorting complex data structure

Question

i have an old perl project, a text parser of an eventlog, and received a request to sort the output by event id and to remove duplicate events. So the parser reads a text file and puts each event in an array. Each field on the array contains a hash with multiple key -> value pairs. one key is called sequence and it contains the number of the event. I now want to sort the array based on the sequence value of each array field. Second, I want to remove duplicate identical sequence numbers from the array.

Here some code how I create the array and hashes so you get an idea of the data structure:

open (my $mel, "<", $in_filename) or die "\nFile '$in_filename' does not exist or is not readable.\n";

my $i=0;
my $eventcount = 0;

while (<$mel>) {

        # Separate events by "Date/Time" :
        if (/^$/) {
            next;
        }
        if (/^Date\/Time:\s(.*)$/) {
            if ($eventcount >0) {
                $i++;
            }
            $eventcount++; # eventcount initialized with ‘0’
        }

        # Gathering information of the MEL event :
        if (/^Date\/Time:\s(.*)$/) {$MEL[$i]{date} = $1; next;}
        if (/^Sequence number:\s(\d+)$/) {$MEL[$i]{sequence} = $1; next;}
        if (/^Event type:\s([0-9|a-f|A-F]{1,6})$/) {$MEL[$i]{type} = lc $1; next;}
        if (/^Event category:\s(\w+)$/) {$MEL[$i]{category} = $1; next;}
        if (/^Priority:\s(\w+)/) {$MEL[$i]{priority} = $1; next;}
        if (/^Description:\s(.*)$/) {$MEL[$i]{description} = $1; next;}
        if (/^Event specific codes:\s(.*)$/) {$MEL[$i]{code} = $1; next;}
        if (/^Component location:\s(.*)$/) {$MEL[$i]{location} = $1; next;}
        if (/^Logged by:\s.*(.)$/) {$MEL[$i]{logged_by} = $1; next;}
        if (/^4[dD]\s45\s4[cC]\s48\s(\d\d)/) {$MEL[$i]{version} = hex $1;}

}

Example of an event in the text file:

Date/Time: 2/3/20, 12:18:20 PM
Sequence number: 200 <==============
Event type: 5023
Event category: Command
Priority: Informational
Event needs attention: false
Event send alert: false
Event visibility: true
Description: Controller return status/function call for requested operation
Event specific codes: b8/1/0
Component type: Controller
Component location: Shelf 99, Bay A
Logged by: Controller in bay A

So basically I want to sort an array containing references to hashes based on the value of a key within the hash.

Second I want to remove a field from the array, when the values of a key also exists in a different array field.

I hope someone understands what I need :-)

Is this possible?

choroba choroba · Accepted Answer · 2020-03-12T22:28:34

You can use a custom sort block to sort the array:

my @sorted = sort { $a->{sequence} <=> $b->{sequence} } @MEL;

But it's much easier to use hash of hashes rather than array of hashes.

#!/usr/bin/perl
use warnings;
use strict;
use feature qw{ say };

my $in_filename = ... ;
open my $mel, '<', $in_filename or die $!;

my %event;

my ($current, $id);
while (<$mel>) {

    next if /^$/;

    if (m{^Date/Time:\s(.*)$}) {
        if (defined $id) {
            $event{$id} = $current;
        }
        $current = { date => $1 };
    } elsif (/^Sequence number:\s(\d+)$/) {
        $id = $1;
    } elsif (/^Event type:\s([0-9|a-f|A-F]{1,6})$/) {
        $current->{type} = lc $1;
    } elsif (/^Event category:\s(\w+)$/) {
        $current->{category} = $1;
    } elsif (/^Priority:\s(\w+)/) {
        $current->{priority} = $1;
    } elsif (/^Description:\s(.*)$/) {
        $current->{description} = $1;
    } elsif (/^Event specific codes:\s(.*)$/) {
        $current->{code} = $1;
    } elsif (/^Component location:\s(.*)$/) {
        $current->{location} = $1;
    } elsif (/^Logged by:\s.*(.)$/) {
        $current->{logged_by} = $1;
    } elsif (/^4[dD]\s45\s4[cC]\s48\s(\d\d)/) {
        $current->{version} = hex $1;
    }
}

for my $e (sort { $a <=> $b } keys %event) {
    say 'Sequence number:', $e;
    for my $k (sort keys %{ $event{$e} }) {
        say "$k: $event{$e}{$k}";
    }
}

It can be further simplified by building one large regex to match most of the details:

my $regex = qr/
               Event\ type:\s(?<type>[0-9|a-f|A-F]{1,6})$
              |Event\ category:\s(?<category>\w+)$
              |Priority:\s(?<priority>\w+)
              |Description:\s(?<description>.*)$
              |Event\ specific\ codes:\s(?<code>.*)$
              |Component\ location:\s(?<location>.*)$
              |Logged\ by:\s.*(?<logged>.)$
              |4[dD]\s45\s4[cC]\s48\s(?<version>\d\d)
/x;

while (<$mel>) {
    next if /^$/;

    if (m{^Date/Time:\s(.*)$}) {
        if (defined $id) {
            $current->{type} = lc $current->{type}
                if exists $current->{type};
            $current->{version} = hex $current->{version}
                if exists $current->{version};
            $event{$id} = $current;
        }
        $current = { date => $1 };
    } elsif (/^Sequence number:\s(\d+)$/) {
        $id = $1;
    } elsif (/^$regex/) {
        $current->{ (keys %+)[0] } = (values %+)[0];
    } else {
        warn "Skipping: $_";
    }
}

Perl - sorting complex data structure

3 Answers