Sparrow is a Raku automation framework which could be easily integrated with many other programming languages. So if you come from no knowledge of the Raku language – you’re welcome.

In this post I’ll show you have one can effectively mix Bash scripts and Raku language using Sparrow.

The idea of Sparrow – to choose the language that fits best to your domain and let Raku orchestrate your code on a high level.

Let’s get started.

Installing Sparrow

Sparrow comes as a Raku module, so one has to use zef package manager to install it:

zef install --/test Sparrow6

Once Sparrow is installed you’ll get a s6 utility available in PATH so that you can execute Sparrow related tasks:

s6 --help

Create a Bash task

You start with a Bash script that does some useful job. The requirement is to name a script as a task.bash.

Let’s start with a simple example:

** task.bash **

echo "hello from Bash"

Now let’s run this script using Sparrow:

s6 --task-run .

[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: hello from Bash

So the script is executed and we could see an output.

It’d be silly to stop here, what is the reason to just run Bash script through Sparrow?

Here comes the most exiting part. Keep reading.

Handling input parameters

Say, we want to pass some input parameters to a Bash script. Handling input parameters with Bash could be a hassle, but not with Sparrow. Let’s update our script:

echo "hello from $(config language)"

Now we can run the script with parameters:

s6 --task-run .@language=Bash

[sparrowtask] :: run sparrow task .@language=Bash
[sparrowtask] :: run thing .
[.] :: hello from Bash

Easy? We don’t need to create args parser in Bash. It’s done by Sparrow by default!

We can even set default values for input parameters ( which are applied if one does not explicitly pass any parameters ).

Let’s just create a YAML format file named config.yaml which will contain all the default parameters:

** config.yaml **

language: Shell

And then run the script again without parameters:

s6 --task-run .

[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: hello from Shell

So default parameter Shell is applied from config.yaml file.

To pass multiple parameters through a command line, use comma delimiter:

s6 --task-run .@language=Raku,version=2020.11

Running Bash script as a Raku function

What makes a usage of Sparrow really interesting is that one can run the same Bash script as a Raku function.

Let’s create a Raku scenario named run.raku which is Raku equivalent to the command line above:

** run.raku **

use Sparrow6::DSL;

task-run ".", %(
  language => "Raku"
);

Let’s run run.raku script by using Raku:

raku run.raku

[.] :: hello from Raku

We have the same output.

Thus, Sparrow allows to run scripts as functions, which is pretty funny. Depending on a context, one can run the same code as a command line or as a function written on Raku.

Checking STDOUT

Sometimes when I write tests for command line tools, I’d like to verify that some script’s output contains certain lines.

The same way as Linux people would use | grep construction to check if a command produces given output.

Sparrow provides an equivalent for that operation called task checks.

Let’s create a file named task.check in the same directory with Bash script:

** task.check **

regexp: Bash || Shell || Perl || Python || Raku

The file contains rules defined in format of plain strings or Raku regular expressions. This syntax is a Sparrow Task Check DSL and explained in Sparrow documentation.

Let’s apply a task check to our task.bash script by running run.raku:

raku  run.raku 

[.] :: hello from Raku
[task check] stdout match  True

As one could see Sparrow has validated that script output for having at least one of the five words:

Bash, Shell, Perl, Python or Raku.

In case a task check fails, Sparrow will notify a user by throwing a proper exception.

Let’s see that by passing a Basic language parameter ( sorry Basic users 🙂 )

s6 --task-run .@language=Basic

[.] :: hello from Basic
[task check] stdout match  False
=================
TASK CHECK FAIL

echo $?

2

Thus, if a task check fails this results in none zero exec code, notifying that the whole script finished with a failure.

Script hooks

Those who are familiar with git source control system know that it allows to define scripts hooks – small tasks gets executed before a data gets send to a remote server. Those tasks could encompass, for example, unit tests or linters. This is handy if you want to ensure that code changes are valid and correct before you send actual data to a server.

The same way Sparrow allows to define hooks for users scripts. Those hooks – are also Sparrow tasks that get executed before a main script.

Let’s take a look at the example, where we have main script – task.bash and hook that triggers additional task named tasks/triggers/http/task.bash

To implement this Sparrow needs a dedicated dir tasks/ where all hook scripts exist:

mkdir tasks/triggers/http

** tasks/triggers/http/task.bash **

curl 127.0.0.1:3000 -o /dev/null -s && echo "I am triggered"

And a dedicated bash files named hook.bash that run a hook script, using run_task function:

** hook.bash ***

run_task triggers/http

Now let’s get it run:

s6 --task-run .

[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: I am triggered
[.] :: hello from Shell
[task check] stdout match  True

In this scenario we hit a URL (curl 127.0.0.1:3000) before we run a main script (that just prints out “hello from Shell”).

Overall, hooks provides one a way to split big Bash script into smaller independent ones.

It’s even possible to pass parameters between main script and a additional one. Let’s pass a parameter named param to tasks/triggers/http/task.bash script:

** nano hook.bash **

run_task triggers/http param value

A hook script reads a parameter passed to it as $param variable:

** tasks/triggers/http/task.bash **

curl 127.0.0.1:3000 -o /dev/null -s && echo "I am triggered. You passed me param: $param"

$ s6 --task-run .

[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: I am triggered. You passed me param: value
[.] :: hello from Shell
[task check] stdout match  True

Packaging things up

Finally, last but not the least Sparrow’s feature is packaging.

Packaging allows users distribute their scripts by using Sparrow plugins mechanism.

A script could be packaged and uploaded to a Sparrow repository – remote server which distributes scripts to users.

Let me show how can we do that, step by step.

Initialize Sparrow repository

First of all we need to create an internal file structure for our Sparrow repository where all the plugins are going to be stored.

It’s done by simple command of Sparrow cli. The command takes one argument – path to a repository file system root folder:

s6 --repo-init ~/repo

[repository] :: repo initialization
[repository] :: initialize Sparrow6 repository for /home/melezhik/repo

Create a plugin

Once a Sparrow repository is initialized, let convert our Bash script into a Sparrow plugin. To do that we need to create a plugin meta file named sparrow.json that contains all the package details.

The file should written in JSON format and placed on the same directory where we have the Bash script:

** sparrow.json **

{
  "name" : "hello-language",
  "version" : "0.0.1",
  "description" : "hello language plugin"
}

The meta file structure is pretty simple, and self-explanatory. A minimum parameters would be a plugin name, plugin version and short description.

Now let’s upload the plugin to a Sparrow repository by using s6 cli:

s6 --upload

[repository] :: upload plugin
[repository] :: upload hello-language@0.0.1

The command will archive and copy all plugin files into a repository file system.

Run a web server

To make a plugin available for end user, we need to spin up a web server that will serve a repository files. It could be any web server, the only requirement it should have a document root of a repository root folder. Here is an example for caddy http server:

caddy --root ~/repo --listen=192.168.0.1

Install a plugin

Now a user can install and run a plugin by using Sparrow command line.

First, let’s setup a remote Sparrow repository and fetch repository index file. The operation is similar to the one Linux user would execute when using standard Linux package managers (e.g for Debian: apt-get update ):

export SP6_REPO=http://192.168.0.1
s6 --index-update

[repository] :: update local index
[repository] :: index updated from file://home/melezhik/repo/api/v1/index

Once a repository is set up, use s6 to install and run a plugin. A user can choose between command line approach:

s6 --plg-run hello-language@language=Python

[repository] :: install plugin hello-language
[repository] :: installing hello-language, version 0.000001
[task] :: run plg hello-language@language=Python
[task] :: run thing hello-language
[hello-language] :: I am triggered. You passed me param: value
[hello-language] :: hello from Python
[task check] stdout match  True

Or Raku API:

** run.raku **

use Sparrow6::DSL;

task-run "hello language", "hello-language", %(
  language => "Raku"
)

raku run.raku

[hello language] :: I am triggered. You passed me param: value
[hello language] :: hello from Raku
[task check] stdout match  True

Different distribution protocols

Sparrow repositories support various protocols for scripts distribution, including http, https, rsync and ftp.

Public Sparrow repository – Sparrowhub.io

An official Sparrow repository is sparrowhub.io – contains a lot of examples of real plugins that could be handy to solve daily developers tasks. Check it out! It could be also a good way to get familiar with Sparrow.

Conclusion

As one can see Sparrow provides a lot of features for people using plain Bash scripting.

With reasonable amount of Raku code we can develop, manage and distribute Bash scripts efficiently.

As a result Sparrow allows to do things in Bash style where Bash is more appropriate while having Raku as as great glue and orchestration tool. If you find Sparrow interesting reach out Sparrow GH page as source for documentation, examples and links to dependent projects.

Merry Christmas!