Day 11 – Packaging with Libarchive

Distributing physical gifts involves wrapping them up into packages, but suppose you want to distribute digital gifts. How can you use Raku to help you wrap them up? Enter Libarchive!

Simple wrapping files into a package

Let’s wrap up just two files, myfile1 and myfile2 into a single package.zip file. (Libarchive just as easily creates tar files, cpio, rar, even iso9660 images for cds or dvds.)

use Libarchive::Simple;

given archive-write('package.zip') {
    .add: 'myfile1', 'myfile2';
    .close;
}

This very simple syntax looks a little weird for those unfamiliar… here is a more ‘traditional’ way of writing the same thing:

use Libarchive::Write;

my $handle = Libarchive::Write.new('package.zip');
$handle.add('myfile1', 'myfile2');
$handle.close;

What is the difference? Libarchive::Simple provides a few shorthand routines for accessing the various Libarchive functionalities. One of these is archive-write() which is identical to Libarchive::Write.new().

The second example takes the return from new() and stores it in the variable $handle. Then we call two methods on that variable to add the files, and close the file.

The given statement makes this even simpler by topicalizing that variable, that is, storing it in the topic variable $_. Since $_ can be used as the default object for method calls, we don’t need to explicitly refer to it when calling methods.

.add('myfile1') is equivalent to $_.add('myfile1')

But what happened to the parentheses? Another little shorthand when calling methods — rather than surrounding your arguments to a method with parentheses, you can just precede them with a colon:

.add: 'myfile1';

Nice! I love programming with Raku!

Package a bunch of files by smartmatching

A handy routine to help in your packaging is dir(). It will return a lazy list of IO::Path objects for a directory. By happy coincidence, Libarchive add can take IO::Path just as easily as a filename.

given archive-write('package.zip') {
    .add: 'mydir', dir('mydir');
    .close;
}

Note we’ve added the directory itself first, then used dir() to get a list of the files inside mydir, which also get added. If you don’t include the directory itself, it won’t be part of the package. That works fine most of the time, depending on your format and your unpackaging program, but it is good practice to include the directory to make sure it gets created the way you want it to.

dir has an extra feature — it can filter the directory by smartmatching the string with a :test argument. Lets include only jpeg files, allowing them to end in either .jpg or .jpeg:

given archive-write('package.zip') {
    .add: 'mydir', dir('mydir', test => /:i '.' jpe?g $/);
    .close;
}

Ecosystem modules like File::Find or Concurrent::File::Find can easily generate even more complicated lists of files for including by recursively adding an entire hierarchy to the package.

Create your files on the fly while packaging

You aren’t limited to adding existing files. You can use the write() method to generate a file for the package on the fly. You can specify content as a Str, a Blob, or even an IO::Handle or IO::Path to get the content from.

given archive-write('package.zip') {
    .write: 'myfile', q:to/EOF/;
        Myfile
        ------
        A special file for a special friend!
        EOF
    .close;
}

Here we’re using a special Raku quoting construct called the heredoc.

The q:to/EOF/ says to use the lines following up until the EOF marker and make them into the content of a file named ‘myfile’ included in the package file. As a friendly benefit, the amount of indentation of the terminator is automatically removed from each line to the quoted lines. How convenient!

Stream your package instead of making files

Making files with your packages is great and all, but I’ve got a web site I want to return my custom CD images from — why bother with a temporary file? Just stream the output on the fly!

For this example, we’re going to stream the package as an iso9660 file (the image used for CD writers) to STDOUT, but you can stream to other programs too.

given archive-write($*OUT, format => 'iso9660') {
    .add: 'myfile1', 'myfile2', 'mydir', dir('mydir');
    .close;
}

Usually the format can be inferred from the suffix on a specified filename, but since we are streaming there is no filename, so the format must be specified. $*OUT is a special filehandle that is automatically opened for you for writing to STDOUT.

Burn that image to a CD and mount it and you’ll see the specified files. So easy!

Libarchive has so many cool features it would take days to go over them all, but I hope this little intro has whet your appetite for packaging things up. Raku has fantastic syntax, features and expressivity that make it so easy to interface with libraries like this.

Have fun packaging your own things, be they physical or digital!