Sparrow is a Raku automation framework which could be easily integrated with many other programming languages. So if you come from no knowledge of the Raku language – you’re welcome.
In this post I’ll show you have one can effectively mix Bash scripts and Raku language using Sparrow.
The idea of Sparrow – to choose the language that fits best to your domain and let Raku orchestrate your code on a high level.
Let’s get started.
Installing Sparrow
Sparrow comes as a Raku module, so one has to use zef package manager to install it:
zef install --/test Sparrow6
Once Sparrow is installed you’ll get a s6 utility available in PATH
so that you can execute Sparrow related tasks:
s6 --help
Create a Bash task
You start with a Bash script that does some useful job. The requirement is to name a script as a task.bash
.
Let’s start with a simple example:
** task.bash **
echo "hello from Bash"
Now let’s run this script using Sparrow:
s6 --task-run .
[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: hello from Bash
So the script is executed and we could see an output.
It’d be silly to stop here, what is the reason to just run Bash script through Sparrow?
Here comes the most exiting part. Keep reading.
Handling input parameters
Say, we want to pass some input parameters to a Bash script. Handling input parameters with Bash could be a hassle, but not with Sparrow. Let’s update our script:
echo "hello from $(config language)"
Now we can run the script with parameters:
s6 --task-run .@language=Bash
[sparrowtask] :: run sparrow task .@language=Bash
[sparrowtask] :: run thing .
[.] :: hello from Bash
Easy? We don’t need to create args parser in Bash. It’s done by Sparrow by default!
We can even set default values for input parameters ( which are applied if one does not explicitly pass any parameters ).
Let’s just create a YAML format file named config.yaml
which will contain all the default parameters:
** config.yaml **
language: Shell
And then run the script again without parameters:
s6 --task-run .
[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: hello from Shell
So default parameter Shell
is applied from config.yaml
file.
To pass multiple parameters through a command line, use comma
delimiter:
s6 --task-run .@language=Raku,version=2020.11
Running Bash script as a Raku function
What makes a usage of Sparrow really interesting is that one can run the same Bash script as a Raku function.
Let’s create a Raku scenario named run.raku
which is Raku equivalent to the command line above:
** run.raku **
use Sparrow6::DSL;
task-run ".", %(
language => "Raku"
);
Let’s run run.raku
script by using Raku:
raku run.raku
[.] :: hello from Raku
We have the same output.
Thus, Sparrow allows to run scripts as functions, which is pretty funny. Depending on a context, one can run the same code as a command line or as a function written on Raku.
Checking STDOUT
Sometimes when I write tests for command line tools, I’d like to verify that some script’s output contains certain lines.
The same way as Linux people would use | grep
construction to check if a command produces given output.
Sparrow provides an equivalent for that operation called task checks.
Let’s create a file named task.check
in the same directory with Bash script:
** task.check **
regexp: Bash || Shell || Perl || Python || Raku
The file contains rules defined in format of plain strings or Raku regular expressions. This syntax is a Sparrow Task Check DSL and explained in Sparrow documentation.
Let’s apply a task check to our task.bash
script by running run.raku
:
raku run.raku
[.] :: hello from Raku
[task check] stdout match True
As one could see Sparrow has validated that script output for having at least one of the five words:
Bash, Shell, Perl, Python or Raku.
In case a task check fails, Sparrow will notify a user by throwing a proper exception.
Let’s see that by passing a Basic language
parameter ( sorry Basic users 🙂 )
s6 --task-run .@language=Basic
[.] :: hello from Basic
[task check] stdout match False
=================
TASK CHECK FAIL
echo $?
2
Thus, if a task check fails this results in none zero exec code, notifying that the whole script finished with a failure.
Script hooks
Those who are familiar with git source control system know that it allows to define scripts hooks – small tasks gets executed before a data gets send to a remote server. Those tasks could encompass, for example, unit tests or linters. This is handy if you want to ensure that code changes are valid and correct before you send actual data to a server.
The same way Sparrow allows to define hooks for users scripts. Those hooks – are also Sparrow tasks that get executed before a main script.
Let’s take a look at the example, where we have main script – task.bash
and hook that triggers additional task named tasks/triggers/http/task.bash
To implement this Sparrow needs a dedicated dir tasks
/ where all hook scripts exist:
mkdir tasks/triggers/http
** tasks/triggers/http/task.bash **
curl 127.0.0.1:3000 -o /dev/null -s && echo "I am triggered"
And a dedicated bash files named hook.bash
that run a hook script, using run_task
function:
** hook.bash ***
run_task triggers/http
Now let’s get it run:
s6 --task-run .
[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: I am triggered
[.] :: hello from Shell
[task check] stdout match True
In this scenario we hit a URL (curl 127.0.0.1:3000
) before we run a main script (that just prints out “hello from Shell”).
Overall, hooks provides one a way to split big Bash script into smaller independent ones.
It’s even possible to pass parameters between main script and a additional one. Let’s pass a parameter named param
to tasks/triggers/http/task.bash
script:
** nano hook.bash **
run_task triggers/http param value
A hook script reads a parameter passed to it as $param
variable:
** tasks/triggers/http/task.bash **
curl 127.0.0.1:3000 -o /dev/null -s && echo "I am triggered. You passed me param: $param"
$ s6 --task-run .
[sparrowtask] :: run sparrow task .
[sparrowtask] :: run thing .
[.] :: I am triggered. You passed me param: value
[.] :: hello from Shell
[task check] stdout match True
Packaging things up
Finally, last but not the least Sparrow’s feature is packaging.
Packaging allows users distribute their scripts by using Sparrow plugins mechanism.
A script could be packaged and uploaded to a Sparrow repository – remote server which distributes scripts to users.
Let me show how can we do that, step by step.
Initialize Sparrow repository
First of all we need to create an internal file structure for our Sparrow repository where all the plugins are going to be stored.
It’s done by simple command of Sparrow cli. The command takes one argument – path to a repository file system root folder:
s6 --repo-init ~/repo
[repository] :: repo initialization
[repository] :: initialize Sparrow6 repository for /home/melezhik/repo
Create a plugin
Once a Sparrow repository is initialized, let convert our Bash script into a Sparrow plugin. To do that we need to create a plugin meta file named sparrow.json
that contains all the package details.
The file should written in JSON format and placed on the same directory where we have the Bash script:
** sparrow.json **
{
"name" : "hello-language",
"version" : "0.0.1",
"description" : "hello language plugin"
}
The meta file structure is pretty simple, and self-explanatory. A minimum parameters would be a plugin name, plugin version and short description.
Now let’s upload the plugin to a Sparrow repository by using s6
cli:
s6 --upload
[repository] :: upload plugin
[repository] :: upload hello-language@0.0.1
The command will archive and copy all plugin files into a repository file system.
Run a web server
To make a plugin available for end user, we need to spin up a web server that will serve a repository files. It could be any web server, the only requirement it should have a document root of a repository root folder. Here is an example for caddy http server:
caddy --root ~/repo --listen=192.168.0.1
Install a plugin
Now a user can install and run a plugin by using Sparrow command line.
First, let’s setup a remote Sparrow repository and fetch repository index file. The operation is similar to the one Linux user would execute when using standard Linux package managers (e.g for Debian: apt
-get update ):
export SP6_REPO=http://192.168.0.1 s6 --index-update
[repository] :: update local index
[repository] :: index updated from file://home/melezhik/repo/api/v1/index
Once a repository is set up, use s6
to install and run a plugin. A user can choose between command line approach:
s6 --plg-run hello-language@language=Python
[repository] :: install plugin hello-language
[repository] :: installing hello-language, version 0.000001
[task] :: run plg hello-language@language=Python
[task] :: run thing hello-language
[hello-language] :: I am triggered. You passed me param: value
[hello-language] :: hello from Python
[task check] stdout match True
Or Raku API:
** run.raku **
use Sparrow6::DSL;
task-run "hello language", "hello-language", %(
language => "Raku"
)
raku run.raku
[hello language] :: I am triggered. You passed me param: value
[hello language] :: hello from Raku
[task check] stdout match True
Different distribution protocols
Sparrow repositories support various protocols for scripts distribution, including http, https, rsync and ftp.
Public Sparrow repository – Sparrowhub.io
An official Sparrow repository is sparrowhub.io – contains a lot of examples of real plugins that could be handy to solve daily developers tasks. Check it out! It could be also a good way to get familiar with Sparrow.
Conclusion
As one can see Sparrow provides a lot of features for people using plain Bash scripting.
With reasonable amount of Raku code we can develop, manage and distribute Bash scripts efficiently.
As a result Sparrow allows to do things in Bash style where Bash is more appropriate while having Raku as as great glue and orchestration tool. If you find Sparrow interesting reach out Sparrow GH page as source for documentation, examples and links to dependent projects.
Merry Christmas!
One thought on “Day 7: Mixing Bash and Raku Using Sparrow”