by Geoffrey Broadwell

In parts 1 and 2 of these blog posts, I roughed out a simple ping chart program and then began to refactor and add features to improve the overall experience.

It’s functional, but there’s a lot to improve upon — it doesn’t use the screen real estate particularly well, there are some common network problems it can’t visualize, and frankly it just doesn’t look all that cool.

So let’s fix all that!

Another Dimension

A simple way to improve the chart’s overall information density is to encode more information into each rendered grid cell. Instead of always using the same glyph for every data point — providing no information other than its location — the shape, color, attributes, or pattern can be adjusted to show more useful information in each rendered cell of the screen.

The version of ping-chart in parts 1 and 2 only shows a relatively short history, since each grid column only represents at most one measurement (and possibly zero, if the “pong” reply packet was never received). Simply placing several measurements before moving on to the next column would improve that, but then overlaps become ambiguous. If ten measurements rendered as just three circles on the screen, how often did the measured ping times land on each of those circles? Were the measurements spread roughly evenly? Did most of them land on the highest or lowest circle?

The chart already looks a bit like a trail of bubbles or pebbles, so why not change the size of each pebble to indicate how often the measurement landed on a particular grid cell? There are many mappings usable for this, depending on which glyphs are available in the terminal font; here are a few obvious options:

ASCII:    . : o O 0
Latin-1:  · ° º o ö O Ö 0
WGL4:     · ◦ ∙ ●
Braille:  ⠄ ⠆ ⠦ ⠶ ⠷ ⠿ ⡿ ⣿

I’ll use ASCII for now, since every terminal font supports it. Making this work requires only a few changes to the update-chart sub. Instead of the original X coordinate calculation, I instead use:

    state    @counts;
    constant @glyphs = « ' ' . : o O 0 »;
    my $saturation   = @glyphs.end;
    my $x = ($id div $saturation) % $width + $x-offset;

This creates the @counts state variable to track how many measurements have landed on a particular Y coordinate and defines the glyphs to be used (including a leading space in the zero slot). The saturation point — the most measurements that can be recorded in a single column before moving forward — is calculated as the last glyph index (@glyphs.end), and finally the calculated X coordinate is (integer) divided by that saturation level to slow the horizontal movement appropriately.

Then I simply need to clear the @counts every time the chart moves to a new column:

        # Clear counts
        @counts = ();

And update the Y coordinate handling and glyph printing:

    # Calculate Y coord (from scaled ping time)
    # and cell content
    my $y =
      0 max $bottom - floor($time / $chart.ms-per-line);
    my $c = @glyphs[++@counts[$y]];

    # Draw glyph at (X, Y)
    $grid.print-cell($x, $y, $c);

The tiny change to the calculation of $y ensures that it can never be negative, and thus is always a valid array index. It’s then used to update the @counts, select the appropriate glyph based on the latest count, and print the chosen glyph in the right spot.

It looks like this now; instead of identical pebbles, the trail looks much more like various-sized pebbles on a scattering of sand:

ms│.    .              .   .             .         .
  │
  │
80│
  │
  │
  │
  │
60│
  │
  │
  │                                                                            .
  │
40│
  │                                .   .
  │     .        .   .                          .     .      ..
  │            .                           .  .                     .
  │  ... .           ..  .         :: .  .                .    ..       . ....
20│o:::.:.:.o ::: .. o.:: ...:..:.o:.:.O:.O:. ::::..oO. .:.....o.:..:oo:...o:.:.
  │.o:::.ooo:oo:oOOO. o.oOooOoOOoo: ::o o:.:O0.o:oOo:.o0OooOOoo.o:Oo:::ooOo.:ooo
  │    .   . :         .  .      .   .        .                  . .
  │
  │
 0│                 ^

It’s much easier to see where the most common measurements lie, and where the outliers are. As a bonus the change to force the Y coordinate onto the chart grid makes it now possible to see how often large outliers appear; they are no longer simply ignored when printing, but rather appear as a smattering of dots at the very top.

As there were five non-blank glyphs chosen, this version now shows five times as much history at once — a bit more than six minutes of history in a default width-80 terminal window.

A wider selection of @glyphs could further improve on that, but there are rapidly diminishing returns — too many different glyphs and the glanceability is lost because it becomes hard to tell the difference between them just by visual size or “density”. This is why I didn’t just choose the digits 1..9; there is very little density distinction between them all, and the overall effect is more confusing than enlightening.

Heating Up

Instead of changing the particular glyph drawn, we could also change its color and brightness; a bright line through a dark-background chart (or a dark line through a light-background chart) would then show where most ping times were clustered.

The original 16 color ANSI palette supported virtually everywhere is completely awful for this purpose, especially since every operating system and terminal program uses a different mapping for these colors. Thankfully there’s a better replacement: most modern terminal emulators support the xterm-256color extended colors and map them all equivalently.

These added colors are mapped in two blocks: a 6x6x6 RGB color cube and a 24-level gray scale. By choosing appropriate points on the color cube, I can create a decent heat map gradient:

# Calculate and convert the colormap once
constant @heatmap-colors =
   # Black to brick red
  (0,0,0), (1,0,0), (2,0,0), (3,0,0), (4,0,0),
  # Red to yellow-orange
  (5,0,0), (5,1,0), (5,2,0), (5,3,0), (5,4,0),
  # Bright to pale yellow
  (5,5,0), (5,5,1), (5,5,2), (5,5,3), (5,5,4),
  # White
  (5,5,5);

constant @heatmap-dark =
  @heatmap-colors.map: { ~(16 + 36 * .[0] + 6 * .[1] + .[2]) }

The formula used in the map converts a color cube RGB triple into a single color index in the range 16 to 231, which is what the terminal expects to see as a color specifier.

Another consideration is that subtly colored circles will probably be hard to distinguish; it would be clearer to just color the entire contents of each grid cell. The easiest way to do this is to set the cell background on a blank cell by using the on_prefix for the color and a blank space for the “glyph”.

Let’s look at the calculations of $saturation and $c again:

    my $saturation = @heatmap-dark.end;
    # ...
    my $c = 'on_' ~ @heatmap-dark[++@counts[$y]];

Modifying the call to print-cell allows setting the color:

    $grid.print-cell($x, $y, { char => ' ', color => $c });

Here’s what that looks like (as an image screenshot rather than a text capture now, in order to show the colors):

It’s beginning to look better, with a vague fiery look and a clear bright band where the ping times are concentrated. Furthermore with fifteen non-black colors in the map, this version of the program now has another three-fold history expansion over the five-glyph version in the previous section — almost 20 minutes of history across a default terminal window.

Precision Flames

While the heat map version has considerably improved information density horizontally, it’s done nothing to change the vertical density; the ping time resolution is just as bad now as it was in the very first version. And because terminal fonts usually make monospace character cells twice as tall as they are wide, the whole chart looks like it’s been smeared vertically. Time to fix that.

Around 2004 Microsoft and various type foundries standardized a list of standard glyphs that modern fonts should supply, called Windows Glyph List 4 or WGL4 for short. This standard was very well supported as a minimum subset for fonts (both free and proprietary) and its full character repertoire was later included in the first stable version of Unicode, cementing it as a solid compatibility baseline.

Among the many very useful glyphs in WGL4 (and thus Unicode 1.1) are the “half blocks”, which split each character cell in half either horizontally or vertically, displaying the foreground color on one half and the background color on the other half. Using the horizontal half blocks can effectively double the chart’s ping time resolution and simultaneously get rid of the vertical smearing effect.

This time all the changes occur in the last few lines of the update-chart sub, starting with a new Y calculation:

    # Calculate half-block resolution Y coord from
    # scaled ping time
    my $block-y = floor(2 * $time / $chart.ms-per-line);
    my $even-by = $block-y - $block-y % 2;
    my $y       = 0 max $bottom - $even-by div 2;
    @counts[$block-y]++;

    # Determine top and bottom counts for
    # half-block "pixels"
    my $c1 = @counts[$even-by + 1] // 0;
    my $c2 = @counts[$even-by]     // 0;

    # Create an appropriate colored cell, using half
    $ blocks if needed
    my $c = $c1 == $c2
      ?? $grid.cell(' ', 'on_'  ~ @heatmap-dark[$c1])
      !! $grid.cell('▀',          @heatmap-dark[$c1] ~
           ' on_' ~ @heatmap-dark[$c2]);

    # Draw colored cell at (X, Y)
    $grid.change-cell($x, $y, $c);
    $grid.print-cell($x, $y);

Since each grid cell now represents two half-block “pixels”, it’s necessary to keep track of both the per-half-block counts and the actual cell Y coordinate that a given block falls into. In addition since each cell could be generated as either one flat color or as two different colors, the code takes care to make an optimal custom grid cell, assign it with change-cell, and print it.

Here’s the result:

Much better — lots more detail and no distracting smearing effect.

Errors and Outages

While the half-block chart does a pretty good job showing network latency when the connection is relatively stable, it doesn’t do a good job of showing various errors: outages, individual lost packets, reordered packets, and so on. These can be detected by watching the sequence IDs carefully, and can be displayed on the top line of the chart to give a glanceable view of such problems.

First the error count for the current column must be kept as a new state variable and reset when the @counts are cleared for each new column:

    state ($errors, @counts);
    # ...

        # Clear counts and errors
        @counts = ();
        $errors = 0;

Then the previous sequence ID must be kept as well, and used to detect sequencing problems and gaps:

    # Determine if we've had any dropped packets or
    # sequence errors, while heuristically accounting
    # for sequence number wraparound
    state $prev-id = 0;
          $prev-id = -1 if $prev-id > $id + 0x7FFF;

    $errors += $id  >  $prev-id + 1
      ?? $id - ($prev-id + 1)
      !! $id  <= $prev-id
        ?? 1
        !! 0;
    $prev-id = $id max $prev-id;

It’s not perfect — it can certainly get confused by particularly horrid conditions — but the above algorithm for sequence tracking is similar to the one used by ping itself and should be resilient to many common problems.

If there are any errors, they should be marked after the normal ping time colors are drawn:

    # Mark errors if any
    if $errors {
        my $color = @heatmap-dark[$errors min $saturation];
        $grid.print-cell(
          $x, 0, {char => '╳', color => "black on_$color"}
        );
    }

This will indicate individual errors within a single column, but won’t show errors on skipped columns during an extended outage. To handle that, the first part of the code for moving to a new column needs some adjustment to update the error count and then mark the errors if any. Because the update-chart is now going to do the exact same error marking in two different places, it can be wrapped in a private helper sub called where needed:

    # Helper routine for marking errors for a
    # particular column
    my sub mark-errors($x, $errors) {
        if $errors {
            my $color =
              @heatmap-dark[$errors min $saturation];
            $grid.print-cell($x, 0, {
              char => '╳', color => "black on_$color"
            });
        }
    }

    # ...

        # If there was a *valid* previous column,
        # finish it off
        if $prev-x >= $x-offset {
            # Missing packets are counted as errors
            $errors += 0 max $saturation - @counts.sum;
            mark-errors($prev-x, $errors);

            # Remove the old "current column" marker
            # if it hasn't been overdrawn
            $grid.print-cell($prev-x, $bottom, ' ')
              if $grid.grid[$bottom][$prev-x] eq '^';
        }

    # ...

    # Mark errors if any
    mark-errors($x, $errors);

Here’s what an outage of a little less than a minute looks like now, showing a bright error bar on the top of the chart during the outage:

One Last Bug

There is a remaining subtle bug in the handling of long ping times. Counts are adjusted individually for each (quantized) ping time seen, but long times could map to a quantization bucket arbitrarily far off the top of the chart. Given only fifteen chances in each column, it’s unlikely that any two overlong times will map to the same (off screen) @counts bucket. So even though $y is forced onto the chart before printing, it will likely only ever show the darkest red color even if several very long pings were measured in that column.

To fix this, all of the overlong pings should be counted in a single bucket and be displayed appropriately in the top chart row. As with $errors, let’s track the number of overlong pings in a given column with a new state variable, and reset it when moving to a new column:

    state ($errors, $over, @counts);
    # ...

        # Clear counts and errors
        @counts = ();
        $errors = 0;
        $over   = 0;

Then it’s simply a matter of special-casing the top row when drawing the ping time results:

    my $c = $y  <= 0
      ?? $grid.cell('▲', @heatmap-dark[++$over])
      !! $c1 == $c2
        ?? $grid.cell(' ', 'on_'  ~ @heatmap-dark[$c1])
        !! $grid.cell('▀',          @heatmap-dark[$c1] ~
             ' on_' ~ @heatmap-dark[$c2]);

Since this happens before the call to mark-errors, an actual error in a given column will replace any overlong mark that was already there. This is intentional: The top row of the chart is used for “problems”, lost packets have a worse effect on user experience than slow replies, and there’s not enough value in using the top two rows of the screen to separate the two problem types visually.

Here’s the final result, my network roasting on an open fire when the ping time variance has got rather bad for a while:

A Final Present

If you’ve made it this far, I’ve got one last little trick for you. You can change the window title by printing a short escape sequence, so that it’s easier to identify in the giant mess of windows on the typical desktop. (What, you’re going to try to claim your desktop doesn’t have dozens of windows open? Mine certainly does!)

Just add this right after initializing Terminal::Print in MAIN:

    # Set window title
    my $title = "$target - ping-chart";
    print "\e]2;$title\e\\";

And that’s it! Happy holidays to all!

Appendix: (Possibly) Frequently Asked Question

But, but … color theory! The color map isn’t perceptually even!

Well yes, I did gloss over that a bit didn’t I?

In terms of perceptual distance between colors, it would seem much better to jump directly from bright yellow to white without the fine details of light yellows. The perceptual differences in light yellows are much less obvious than in the reds and oranges, so a color palette made from the heat map above appears to have 10 clearly different reds and oranges, and then a subtly-varying “smear” of light yellow. Jumping directly from bright yellow to white sets the two apart decently (though still not as much as the reds and oranges), so the color swatches would look more evenly spaced.

However in practice such a “corrected” map looks worse for the actual ping chart! Ping time jitter makes it unlikely that the top couple colors in the map will actually be shown, as that would require (nearly) every ping time to map to the very same pixel, and thus absolutely rock steady network performance. Aside from perhaps the loopback interface (the computer talking to itself), this is rather unlikely in actual practice. Thus to be able to produce the characteristic bright band of a mostly steady connection, the lightest colors need to be over-emphasized in the color map, which it happens the smooth walk through the light yellows achieves nicely.

by Geoffrey Broadwell

In part 1 of these blog posts, I roughed out a simple ping chart program that produced output like this as it ran:

                                                            o
                      o                                o  o     o
  o o      o o       o o o        o   oo                o  o o o  o o  o       o
 o o oooooo o ooooooo   o o oooooo ooo  ooooooooooooooo  o    o  o o oo ooooooo
o                          o

         ^

This is over-minimalist; a user couldn’t even tell the scale of the latency measurements. It’s not at all obvious whether this is a fast or slow connection, and whether the jitter in the results is something to worry about.

A Sense of Scale

One change that would help considerably is to show a Y axis with labels for various latency thresholds; as long as it is not overdrawn, it’s only necessary to draw this once.

Here’s a routine to do that:

#| Draw a Y axis with latency labels; returns width
#| of axis marks (to be used as a left offset for
#| the actual chart content)
sub draw-y-axis(Terminal::Print::Grid:D :$grid!,
                UInt:D :$ms-per-line!,
                UInt:D :$tick-every = 5
                --> UInt:D) {
    # Compute dimensions
    my $bottom      = $grid.h - 1;
    my $last-label  = $bottom - $bottom % $tick-every;
    my $label-width =
      2 max ($last-label * $ms-per-line).chars;
    my $x-offset    = $label-width + 1;

    # Add labels and axis line
    for ^$grid.h .reverse -> $y {
        my $is-tick = $y %% $tick-every;
        my $value   = $y * $ms-per-line;
        my $label   = $is-tick
          ?? $value.fmt("%{$label-width}d")
          !! ' ' x $label-width;
        $grid.print-string(0, $bottom - $y, "$label│");
    }

    # Show that the Y-axis scale is in milliseconds
    $grid.print-string(
      0, 0, 'ms'.fmt("%{$label-width}s") ~ '│'
    );

    # Return computed x-offset
    $x-offset
}

This draw-y-axis sub starts off by determining the required label size (which must be at least 2 because of the ms unit name printed at the top), and then prints the axis labels starting at the bottom (with a │ character on each line for the axis itself). Finally it prints the ms on the first line and returns the computed x-offset for the actual chart content.

With default settings on a default size terminal window, it would look like this:

ms│
  │
  │
80│
  │
  │
  │
  │
60│
  │
  │
  │
  │
40│
  │
  │
  │
  │
20│
  │
  │
  │
  │
 0│

DRY: Good for Kindling and for Code

The above version of draw-y-axis is certainly functional, and could be used as-is in a pinch. But it also has a rather verbose interface, demonstrating a problem that would quickly spread throughout the program:

sub draw-y-axis(Terminal::Print::Grid:D :$grid!,
                UInt:D :$ms-per-line!,
                UInt:D :$tick-every = 5
                --> UInt:D) {

Here draw-y-axis is taking separate arguments for several different values, as well as returning the x-offset to be used elsewhere in the program. Each of these values would need to be plumbed through the interfaces of MAIN, ping-chart, and update-chart too, just to make sure those values are available everywhere they are needed.

This violates the DRY principle: Don’t Repeat Yourself. It shouldn’t be necessary to copy all this information everywhere, but instead to just define it once. In fact I can head this problem off by defining a simple structure to hold any relevant configuration for the chart and passing that around rather than the individual values; then if future features require more config values, I can just add them to the structure.

If you’re thinking “Sounds like objects!” … well yes, but that’s actually a bigger hammer than I intend to use here. For this task I don’t need full object-oriented generality but rather a simple Data Record that can be treated as a unit (also known as a Passive Data Structure or Plain Old Data). Essentially, it’s a class without any methods:

#| All of the configuration and state for a ping chart
class Chart {
    has Terminal::Print::Grid:D $.grid is required;

    has UInt:D $.ms-per-line    = 4;
    has UInt:D $.tick-every     = 5;
    has UInt:D $.x-offset is rw = 0;
}

Packaging these details in a data record simplifies the interface to draw-y-axis a bit, and requires a couple changes in its code to reference those details via the record. Here’s a full rewrite for clarity:

sub draw-y-axis(Chart:D $chart) {
    my $grid        = $chart.grid;
    my $bottom      = $grid.h - 1;
    my $last-label  = $bottom - $bottom % $chart.tick-every;
    my $label-width =
      2 max ($last-label * $chart.ms-per-line).chars;
    $chart.x-offset = $label-width + 1;

    # Add labels and axis line
    for ^$grid.h .reverse -> $y {
        my $is-tick = $y %% $chart.tick-every;
        my $value   = $y * $chart.ms-per-line;
        my $label   = $is-tick
          ?? $value.fmt("%{$label-width}d")
          !! ' ' x $label-width;
        $grid.print-string(0, $bottom - $y, $label ~ '│');
    }

    # Show that the Y-axis scale is in milliseconds (ms)
    $grid.print-string(
      0, 0, 'ms'.fmt("%{$label-width}s") ~ '│'
    );
}

One-time interface changes to ping-chart and update-chart allow them to accept a Chart record rather than raw values:

sub ping-chart(Chart:D :$chart!, Str:D :$target!) {
    # unchanged until ...
              update-chart(:$chart, :$id, :$time);
    # likewise unchanged ...
}

sub update-chart(
  Chart:D :$chart!, UInt:D :$id!, Real:D :$time!
) {
    my $grid   = $chart.grid;
    # unchanged beyond this point ...
}

The Chart record of course has to be created in MAIN and passed in to draw-y-axis and ping-chart:

# Draw a ping chart on the current screen grid
my $grid  = T.current-grid;
my $chart = Chart.new(:$grid, :$ms-per-line);
draw-y-axis($chart);
ping-chart(:$chart, :$target);

Back On Track

With that refactoring done, a few simple changes in update-chart can now accommodate the x-offset computed by draw-y-axis so that the chart content doesn’t try to overwrite the Y axis and its labels:

    my $grid     = $chart.grid;
    my $x-offset = $chart.x-offset;
    my $bottom   = $grid.h - 1;
    my $right    = $grid.w - 1;
    my $width    = $grid.w - $x-offset;
    my $x        = $id % $width + $x-offset;

    # ...
    state $prev-x  = $x-offset - 1;
    # ...
        $grid.print-cell($prev-x, $bottom, ' ')
          if $prev-x >= $x-offset
          && $grid.grid[$bottom][$prev-x] eq '^';
    # ...
        $prev-x = $x-offset if ++$prev-x > $right;

Here’s what the full program’s output looks like now:

ms│
  │                               o
  │
80│
  │
  │
  │                         o
  │
60│
  │
  │
  │
  │
40│           o                                      o
  │
  │                                    o                    o         o
  │                                                                        o
  │               o                            o  o    ooo         o     o
20│o ooo         o   oooo oo        o   ooo ooo     o o   o  o   o             o
  │ o   oooo o oo  oo    o   o ooo   oo         o  o       o  ooo o oo oo o ooo
  │         o
  │
  │
 0│                      ^

Scale Adjustments

When I’m connected via direct cabling to my home ISP and it’s having a good day, most ping times are short enough to display reasonably in the chart with its default scaling. But when my ISP is having a bad day or I’m tethered via my cell phone, latency increases several times over and the chart marks no longer stay below the top of the terminal.

One quick fix for this is to add a command-line option to set the vertical scale in milliseconds per screen line, and respect that setting in update-chart. Here’s what the MAIN changes look like:

sub MAIN($target = '8.8.8.8',    #= Hostname/IP address to ping
         UInt :$ms-per-line = 4, #= Milliseconds per screen line
        ) {
    # ...
    my $chart = Chart.new(:$grid, :$ms-per-line);
    # ...
}

The change in update-chart is even simpler, just part of one line:

    my $y = $bottom - floor($time / $chart.ms-per-line);

Here’s the new USAGE:

$ ./ping-chart -?
Usage:
  ./ping-chart [--ms-per-line[=UInt]] [<target>] -- Slowly draw a chart of ping times

    [<target>]              Hostname/IP address to ping [default: '8.8.8.8']
    --ms-per-line[=UInt]    Milliseconds per screen line [default: 4]

And here it is in action:

$ ./ping-chart --ms-per-line=20


 ms│
   │
   │
400│
   │
   │
   │
   │
300│
   │
   │
   │
   │
200│
   │
   │
   │      o o
   │o                                                                          o
100│       o                             o   o                                o
   │                                       o   o o                           o
   │ o o o                                                                  o
   │  o o    o   o
   │          ooo                    o oo o         o  ooooooooooooooooooooo
  0│             ^ooooooooooooooooooo o     o o o oo oo

Note how the vertical scale has changed, and the Y axis labels and x-offset have automatically compensated. The low ping times are when connected via the local ISP (it’s doing OK tonight). The much higher and spread out values are when I switched to cell phone tethering; many would have been invisible without the rescaling.

For Next Time

The changes in this part have improved both functionality and maintainability, but haven’t addressed the chart’s low information density (or its overall lack of “cool”). I’ll pick that up in part 3.

Appendix: (Probably) Frequently Asked Question

Why not switch entirely to OO design, using proper methods?

I wanted to demonstrate how the (really old school) Data Record concept can be used to do just enough refactoring to permit some improved maintainability and DRY compliance without forcing a full code rewrite. This complexity threshold is very commonly reached when developing something that started as a small proof of concept, but need not cause a daunting — and thus probably never actually accomplished — full OO rewrite.

by Geoffrey Broadwell

My home Internet connection is less than ideal. On the good days it’s fine I suppose, but on the bad days — and there are a lot of them — well, my ISP seems to be doing its darnedest to be earning coal in its collective stockings. Meanwhile I hear shouts across the house of “DAAAD, the INTERNEEEET!” and have to diagnose yet again what’s causing the fuss.

Experience has shown that my ISP is easily overwhelmed by weekend and holiday traffic levels, but it degrades in all sorts of interesting ways:

• High latency
• High jitter
• Reduced bandwidth
• Reordered packets
• Lost packets
• Intermittent connectivity

Some of these are easy to work around: “Kids, stop streaming extra stuff you’re not actually paying attention to, and kindly save the giant game updates for after peak hours!” Other problems (the last two especially) are miserable for everyone no matter how you slice it, and much more difficult to work around.

This year I decided to whip up a little display to let me know at a glance exactly how our Internet connection was failing, without having to investigate from scratch each time. First I needed a tool to measure with, something that could produce a stream of data I could analyze to produce the glanceable display. Thankfully such a tool is quite easy to find.

Internet Ping Pong

Since the days of yore, just about every Internet-connected system has come with a simple utility called ping, which as its name implies acts a bit like a unidirectional sonar. It sends out a small “ping” packet and looks for a “pong” response; if there is one, it reports the time taken to send and receive. By default it sends a new ping once a second repeatedly, reporting on each pong received. The output format varies a bit depending on operating system, but here’s what it looks like on a modernish Linux system:

$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=19.8 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=14.3 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=55 time=21.0 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=55 time=17.7 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=55 time=14.3 ms
64 bytes from 1.1.1.1: icmp_seq=6 ttl=55 time=15.5 ms
^C
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5009ms
rtt min/avg/max/mdev = 14.292/17.099/21.016/2.621 ms

Much of what ping prints is not terribly useful for my immediate needs. The repeated info about target IP address and packet sizes doesn’t tell me much that I don’t already know, and the summary info is only printed when the pings stop; they’re not updated on the fly.

What I can easily use are the per-pong result lines, the ones that look like this:

64 bytes from 1.1.1.1: icmp_seq=4 ttl=55 time=17.7 ms

The time measurements will let me visualize latency and jitter problems, and the sequence numbers (icmp_seq) will let me track reordered or lost packets and periods of lost connectivity. Perfect; now I just need a UI.

Sketching Out the Basics

I normally keep quite a few terminal windows open on my desktop, so keeping one more open to continuously chart the pingresults seemed like a good place to start. Besides, Raku has long had a simple module for producing custom full-terminal displays, Terminal::Print, which treats the terminal window as a grid of single-character cells, each with optional color and style info.

Sketching out the top level of the program was thus pretty easy:

#!/usr/bin/env raku

use Terminal::Print <T>;

#| Slowly draw a chart of ping times
sub MAIN($target = '1.1.1.1', #= Hostname/IP address to ping
        ) {
    # Initialize Terminal::Print and show a blank screen
    T.initialize-screen;

    # Draw a ping chart on the current screen grid
    my $grid = T.current-grid;
    ping-chart(:$grid, :$target);

    # Shut down, restore the original screen, and exit
    T.shutdown-screen;
}

#| Set up a `ping` child process and convert the ping times
#| to a Terminal::Print::Grid chart asynchronously
sub ping-chart(Terminal::Print::Grid:D :$grid!,
               Str:D :$target!,
              ){
    # To be written ...
}

The MAIN sub sets our basic command line arguments and options, and the Terminal::Print module provides the T alias for terminal control. The simple logic in MAIN initializes full screen control, hands off drawing to the ping-chart sub, and shuts down and restores the original screen on exit.

As a side note, it’s not actually necessary to pass the grid explicitly to the drawing routine — ping-chart could just assume it should use the current screen grid always — but it’s a good habit to get into, as more advanced UIs will likely have multiple different visual grids and drawing routines will then need to know which one to draw on.

Saving this as ping-chart, here’s what I have so far:

$ ./ping-chart -?
Usage:
  ./ping-chart [<target>] -- Slowly draw a chart of ping times

    [<target>]    Hostname/IP address to ping [default: '1.1.1.1']

$ ./ping-chart
# Nothing appears to happen yet, but the program exits cleanly

Nothing appears to happen yet when the program is run without options, except maybe showing a quick flash of blank terminal. On my system the program runs fast enough that the terminal emulator just elides the flash completely.

Asynchronous Reactions

Next up, I filled in the ping-chart sub with some simple event reaction code:

# Prepare a `ping` child process that the reactor
# will listen to
my $ping = Proc::Async.new('ping', $target);

# Run main event reactor until interrupt signal
# or `ping` exits
react {
    # Parse `ping` results
    whenever $ping.stdout.lines {
        if $_ ~~ /'seq=' (\d+) .*? 'time=' (\d+ [\.\d+]?)/ {
            my ($id, $time) = +$0, +$1;
            update-chart(:$grid, :$id, :$time);
        }
    }

    # Quit on SIGINT (^C)
    whenever signal(SIGINT) { done }

    # Quit on child process exit
    whenever $ping.start    { done }
}

Starting an asynchronous child process is quite easy with Proc::Async, which takes the name of the program to run and its arguments, and produces a process object that is waiting to be started.

In order to process the ping output, I use the standard Raku react block to handle three different types of events:

• Whenever a new line arrives from ping‘s standard output, try to parse it and display the result in the chart.
• Whenever the user sends an interrupt signal (SIGINT, usually the result of pressing ^C), stop the reactor using done.
• Whenever the started child process exits on its own (i.e. $ping.start returns to the caller), likewise stop the reactor using done.

Of course stopping the reactor will cause execution to leave the ping-chart sub, and the last line of MAIN will then shut down and restore the normal terminal screen using T.shutdown-screen.

The parsing reaction code looks a bit hairy, but is conceptually simple:

1. Use a regular expression match on the current output line to try to capture the sequence number and recorded ping/pong time using capturing parentheses; if this fails, ignore the line and wait for another.
2. Convert the resulting Match objects to regular numbers (with prefix + ).
3. Call update-chart to actually do the charting work using those numbers.

Here’s a minimal implementation of update-chart:

#| Update the chart with a new ping result
sub update-chart(Terminal::Print::Grid:D :$grid!,
                 UInt:D :$id!,
                 Real:D :$time!) {
    my $x = $id % $grid.w;
    my $y = $grid.h - 1 - floor($time / 4);

    $grid.print-cell($x, $y, 'o') if $y >= 0;
}

The simple steps are as follows:

1. Select an X coordinate based on the sequence ID, wrapping around the grid width using % $grid.w.
2. Select a Y coordinate based on the recorded ping/pong time, accounting for terminal Y coordinates increasing from top to bottom instead of bottom to top as you might expect.
3. Draw a circle on the chart at that (X, Y) location.

Here’s what this minimal charting produces when the program is run without arguments:

o       o   o                   o
          o  o              o    oo    o
 ooooo o o o  oooooooooooooo ooo   oooo oooo
      o

Nothing all that fancy, but right off it’s obvious there’s some base latency plus some occasional timing jitter on top of that. Unfortunately, here’s what the screen looks like after the program is allowed to run for a few minutes:

                      o
                                          o
                                                                o
                            o                          o

                     o    o                          o
       o   o                                       o
                           o                o               o                  o
 o  o                  o     o       o              o   oo        o  o o  o  o
oooo  oooo oooo  o oooo oooo  o o  ooo o o ooo oo ooooo oo  oo  o  oo  ooo  oooo
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
ooo   oo o             o            o  o                o   o o o o

Since old cells are never cleared when the X coordinate wraps around, the most commonly detected ping times quickly fill up a solid band, though even this is somewhat useful as the scale of the outliers starts to be more obvious. Still, left long enough this would just fill a fair portion of the terminal solidly with circles with no indication of what the current connection status is — in fact the connection could completely go down at any point, and there’d be no obvious change!

Keeping the Screen Clean

There’s so little code in the minimalist update-chart above that it’s fairly easy to just rewrite it from scratch:

#| Update the chart with a new ping result
sub update-chart(Terminal::Print::Grid:D :$grid!,
                 UInt:D :$id!,
                 Real:D :$time!) {
    # Calculate chart edges (from grid size) and X coord
    # (from wrapped ID)
    my $bottom = $grid.h - 1;
    my $right  = $grid.w - 1;
    my $x      = $id % $grid.w;

    # Each time we start a new column, clear it and move
    # the "current column" marker; also make sure to
    # handle longer connection failures that might move
    # forward several columns at a time.
    state $prev-x  = -1;
    while $prev-x != $x {
        # Remove the old "current column" marker if it
        # hasn't been overdrawn
        $grid.print-cell($prev-x, $bottom, ' ')
          if $prev-x >= 0
          && $grid.grid[$bottom][$prev-x] eq '^';

        # Move prev-x forward, chasing current x; wrap
        # at right edge
        $prev-x = 0 if ++$prev-x > $right;

        # Clear the new column to blank
        $grid.print-cell($prev-x, $_, ' ') for ^($grid.h);

        # Move the "current column" marker to the newly
        # cleared column
        $grid.print-cell($prev-x, $bottom, '^');
    }

    # Calculate Y coord (from scaled ping time)
    my $y = $bottom - floor($time / 4);

    # Draw circle at (X, Y)
    $grid.print-cell($x, $y, 'o') if $y >= 0;
}

This does much better when left to run for several minutes:

                                                            o
                      o                                o  o     o
  o o      o o       o o o        o   oo                o  o o o  o o  o       o
 o o oooooo o ooooooo   o o oooooo ooo  ooooooooooooooo  o    o  o o oo ooooooo
o                          o

         ^

Rather than continuously building up into a solid bar, the chart now shows only the new measurements from the last horizontal pass (rather like a heart monitor does). Furthermore the ^ marker shows which measurement is changing and helps to show how large the “latency floor” is, since the marker is always printed on the bottom line of the screen grid.

More To Do

There’s still a lot more to be desired from the current output:

• There are no axis ticks to give a sense of scale and help the eye determine how bad the ping latency has actually been.
• It can be hard to see if there is a gap in the marks, and there’s no indication of other types of errors (such as reordered packets or ping times so long they are off the chart).
• The chart only shows the last screen-width seconds (80 for a default terminal); it would be nice to show additional history without having to open an extremely wide terminal window to do so.
• The vertical (ping time) detail is fairly poor as well; moving from pure ASCII glyphs to full Unicode provides a few ways around that problem.

I’ll take a look at each of these in the following parts. Until then, may your packets flow freely this holiday season!

Appendix: (Potentially) Frequently Asked Questions

Why use 1.1.1.1 as the default ping target?

That’s the primary public DNS server address for Cloudflare, a very large CDN (Content Delivery Network). It’s fairly responsive in most parts of the world, and if it’s down a fair portion of the Internet at large will be quite unhappy. There are quite a few other public DNS servers with similar properties, such as 8.8.8.8 for Google DNS; a community-curated list of commonly used public DNS servers can be found at Duck Duck Go using this query:

https://duckduckgo.com/?q=public+dns+servers&t=lm&ia=answer&iax=answer

Note: Some of those servers are highly likely to be untrustworthy or actively privacy invading. Do some research and caveat hacker.

State variables: Aren’t they concurrency bugs waiting to happen?

update-chart is only ever called from a whenever block within the main event reactor. Raku guarantees that only one thread at a time can ever be running a whenever block inside a particular react or supply. So update-chart can freely use state variables all it wants, and there can never be a problem caused by concurrent access to them.