Day 2 – Rotation of Log files in a nutshell

Santa has a cloud-based application that helps him to deliver the gifts to the children. Once the gifts have been delivered Santa registers the delivery operation through the deliveries.log file. Just after the inspector elves review this log file comparing it with the list of children to ensure that all the children have received correctly their gifts.

The number of deliveries is very large and the price of the cloud-based storage too. To accomplish the cloud budget it’s neccesary to set a maximum log capacity such as:

• The log information will be distributed in 5 log files
• Each log file size should be about 20 MB

How will we do?

Santa needs a process that will run with regular basis. This process will rotate the log files if the size of the main log file is almost 20 megabytes. The maximum number of log files will be 5, that is:

• deliveries.log
• deliveries.log1
• deliveries.log2
• deliveries.log3
• deliveries.log4

When deliveries.log file size be almost 20 megabytes its name will changes to deliveries.log1.

Next time this happens, the name of the deliveries.log1 will changes to deliveries.log2 and the name of deliveries.log will changes to deliveries.log1.

And so on until reach to deliveries.log4. At this point, the deliveries.log4 file (the oldest) will be deleted because there are 5 log files, and the name of the deliveries.log3 file will changes to deliveries.log4.

Starting the Raku script to get it

First we need know the path of the log files, the name of the main log file and its absolute path:

my $path_logs     = '/var/log/gifts/';
my $main_log      = 'deliveries.log';
my $path_main_log = "$path_logs$main_log";

As we see in "$path_logs$main_log", using double quotes to concatenate strings is very visual, but a more elegant approach is achieved using ~ operator between the strings variables:

my $path_main_log = $path_logs ~ $main_log;

Let’s keep going setting the maximum size of the main log file (20 megabytes) in bytes:

my $max_size_log = 20000000;

Also, we need set the maximum number of log files that we will rotate:

my $max_logs = 5;

We have already the ingredients, now we are going to set the requirements.

Requirements

Now we need to know if the main log file deliveries.log exists and to check its size.

The Raku capacity to handle files with .IO is very robust, concise and complete. Also, we can use the method .e next to a absolute file path to check if it exists, and exit the script if the file doesn’t exist. All in one line:

exit unless $path_main_log.IO.e;

Similarly using the .s method we can get the size of a given file. We can exit if the main log size is lower than the specified in $max_size_log:

exit if $path_main_log.IO.s < $max_size_log;

As we see, this Raku code is very readable and concise, and makes it easier to read in case of debugging.

At this point, the deliveries.log file (the main log file) exists with a size of 20 megabytes or more. The next step is to find out how many log files exist in the logs folder.

How many log files we have?

We can populate an array with the absolute paths of the log files using chaining methods. Chaining methods are part of the Raku functional programming paradigm, a powerful feature that pass the result of a method to another as an argument, in similar way like pipes in bash shell:

my @log_files = $path_logs.IO.dir.grep(/$main_log$ | $main_log\d+$/).sort;

Let’s dissect the sneak:

• @log_files is an array that will populates with the absolute paths of the log files.

• First we use $path_logs whose value is /var/log/gifts/, that is the absolute path of the logs folder. The next methods will search the log files in this location.

• The .IO.dir method returns all the absolute paths of the files and folders located in the $path_logs folder. As curiosity, the dir command was already used by the RT-11 operating system 51 years ago. Later, this command was also adopted by CP/M and MS-DOS. There are things that will never change.

• The next method is .grep(/$main_log$ | $main_log\d+$/). The .grep method matches the regular expression pattern provided as parameter /$main_log$ | $main_log\d+$/ over each returned value of the previous method. This pattern admits two possibilities: matches the main log file $main_log$ that is deliveries.log or (|) matches the main log file $main_log next a number $main_log\d+$. This regex matches the absolute paths of deliveries.log or deliveries.log1 or deliveries.log4 log files but never matches strings like deliveries.log.old.

• The last method is .sort. This method sorts the elements returned by the previous method. This ordering is essential to continue with the next operations.

Many operations in one line, with chaining methods that’s be fine.

There are many log files, let’s fire the last

If the number of the log files has reached to the limit established in $max_logs we need to remove the last log file deliveries.log4 in our case, both in the array @log_files and in the filesystem. Your attention please:

@log_files.pop.unlink if @log_files.elems == $max_logs;

In this case we are also use chaining methods:

• @log_files is the array with the absolute path of each log file.

• The .pop method removes and returns the last element from @log_files array. This element is the absolute path of deliveries.log4 file.

• The .unlink method removes a file in the filesystem. In this case removes the returned element by the previous method .pop, that is the log file deliveries.log4.

Then comes the condition with if although it is evaluated before. This condition compares the number of elements of @log_files using the .elems method with $max_logs whose value is 5. If the condition returns true the deliveries.log4 element is fired, both from the @log_files array and from the filesystem.

The use of the methods .pop and .unlink seems amazing to me.

Moving log files to the right position

Now, we need to rename the log files as we seen before.

Here, the Raku magic help us with the for iterator:

for @log_files.kv.reverse -> $file, $idx { $file.rename($path_main_log ~ $idx + 1); }

The -> $file, idx is the signature of the block {} and is made up of the $file and $idx parameters that are populated by the .kv method. Each iteration provides the absolute path of the current log file in the $file variable and the current index iteration in the $idx variable. All this is done in reverse order using the .reverse method.

The content of the block {} simply use the .rename method with the current log file $file to change its name to the name of the next log file. The name of the next log file is the main log file $path_main_log plus the next index number ~ $idx + 1.

Beautiful one line code running many operations. A scripter dream.

Putting all together

my $path_logs     = '/var/log/gifts/';
my $main_log      = 'deliveries.log';
my $path_main_log = $path_logs ~ $main_log;
my $max_size_log  = 20000000;
my $max_logs      = 5;

exit unless $path_main_log.IO.e;
exit if $path_main_log.IO.s < $max_size_log;

my @log_files = $path_logs.IO.dir.grep(/$main_log$ | $main_log\d+$/).sort;

@log_files.pop.unlink if @log_files.elems == $max_logs;

for @log_files.kv.reverse -> $file, $idx { $file.rename($path_main_log ~ $idx + 1); }

Epilogue

The multiparadigm approach of Raku provides an astonishing capacity to perform complex operations concisely and pushes scripting to the next level, and Santa knows it.