Until a few days ago, I’d intended for this post to be an update on the Raku persistent data structures I’m developing. And I have included a (very brief) status update at the end of this post. But something more pressing has come to my attention: Someone on the Internet was wrong — and that someone was me.
Specifically, in my post about sigils the other day, I significantly misdescribed the semantics that Raku applies to sigiled-variables.
Considering that the post was about sigils, the final third focused on Raku’s sigils, and much of that section discussed the semantics of those sigils – being wrong about the semantics of Raku’s sigils isn’t exactly a trivial mistake. Oops!
In partial mitigation, I’ll mention one thing: no one pointed out my incorrect description of the relevant semantics, even though the post generated over two hundred comments of discussion, most of it thoughtful. Now, it could be no one read all the way to Part 3 of a 7,000 word post (an understandable choice!). But, considering the well-known popularity of correcting people on the Internet, I view the lack of any correction as some evidence that my misunderstanding wasn’t obvious to others either. In fact, I only discovered the issue when I decided, while replying to a comment on that post, to write an an oddly-designed Raku class to illustrate the semantics I’d described; much to my suprise, it showed that I’d gotten those semantics wrong.
Clearly, that calls for a followup post, which you’re now reading.
My goal for this post is, first of all, to explain what I got wrong about Raku’s semantics, how I made that error, and why neither I nor anyone else noticed. Then we’ll turn to some broader lessons about language design, both in Raku and in programming languages generally. Finally, with the benefit of correctly understanding of Raku’s semantics, we’ll reevaluate Raku’s sigils, and the expressive power they provide.
What I got wrong – and what I got right
In that post, I said that the @ sigil can only be used for types that implement the Positional (“array-like”) role; that the % sigil can only be used for types that implement the Associative (“hash-like”) role; and that the & sigil can only be used for types that implement the Callable (“function-like”) role. All of that is right (and pretty much straight from the language docs).
Where I went wrong was when I described the requirements that a type must satisfy in order to implement those roles. I described the Positional role as requiring an iterable, ordered collection that can be indexed positionally (e.g., with @foo[5] ); I described the Associative role as requiring an iterable, unordered collection of Pair s that can be indexed associatively (e.g., with %foo<key> ); and I described the Callable role as requiring a type to support being called as a function (e.g., with &foo() ).
That, however, was an overstatement. The requirements for implementing those three roles are actually: absolutely nothing. That’s right, they’re entirely “marker roles”, the Raku equivalent of Rust’s marker traits.
Oh sure, the Raku docs provide lists of methods that you should implement, but those are just suggestions. There’s absolutely nothing stopping us from writing classes that are Associative , Positional , or Callable , or – why not? – all three if we want to. Or, for that matter, since Raku supports runtime composition, the following is perfectly valid:
my @pos := 'foo' but Positional;
my %asc := 90000 but Associative;
my &cal := False but Callable;Yep, we can have a Positional string, an Associative number, and a Callable
How did we miss that?
So, here’s the thing: I’ve written quite a bit of Raku code while operating under the mistaken belief that those roles had the type constraints I described – which are quite a bit stricter than “none at all”. And I don’t think I’m alone in that; in fact, the most frequent comment I got on the previous post was surprise/confusion that @ and % weren’t constrained to concrete Array s and Hash es (a sentiment I’ve heard before). And I don’t think any of us were crazy to think those sorts of things – when you first start out in Raku, the vast majority (maybe all) of the @ – and % -sigiled things you see are Array s and Hash es. And I don’t believe I’ve ever seen an @ -sigiled variable in Raku that wasn’t an ordered collection of some sort. So maybe people thinking that the type constraints are stricter makes a certain amount of sense.
But that, in turn, just raises two more questions: First, given the unconstrained nature of those sigils, why haven’t I seen some Positional strings in the wild? After all, relying on programmer discipline instead of tool-enforcement is usually a recipe for quick and painful disaster. And, second, given that @ – and %
Good defaults > programmer discipline
Let’s address those questions in order: Why haven’t I seen @ -sigiled strings or % -sigiled numbers? Because Raku isn’t relying on programmer discipline to prevent those things; it’s relying on programmer laziness – a much stronger force. Writing my @pos := 'foo' but Positional seems very easy, but it has three different elements that would dissuade a new Rakoon from writing it: the := bind operator (most programmers are more familiar with assignment, and = is overwhelmingly more common in Raku code examples); the but operator (runtime composition is relatively uncommon in the wider programming world, and it’s not a tool Raku code turns to all that often) and Positional (roles in general aren’t really a Raku 101 topic, and Positional / Associative / Callable even less so – after all, all the built-in types that should implement those roles already do so).
Let’s contrast that line with the version that a new Rakoon would be more likely to write – indeed, the version that every Rakoon must have written over and over: my @pos = 'foo' . That removes all three of the syntactic stumbling blocks from the preceding code. More importantly, it works. Because the @ -sigil provides a default Array container, that line creates the Array ['foo'] – which is much more likely to be what the user wanted in the first place.
Of course, that’s just one example, but the general pattern holds: Raku very rarely prohibits users from doing something (even something as bone-headed as a Positional string) but it’s simultaneously good at making the default/easiest path one that avoids those issues. If there’s an easy-but-less-rigorous option available, then no amount of “programmer discipline” will prevent everyone from taking it. But when the safer/saner thing is also by far the easier thing, then we’re not relying on programmer discipline. We’re removing the temptation entirely.
And then by the time someone has written enough Raku that := , but , and Positional wouldn’t give them any pause, they probably have the “ @ means “array-like, but maybe not an Array ” concept so deeply ingrained that they wouldn’t consider creating a wacky Positional
Being stricter
What about the second question we posed earlier: Why doesn’t Raku enforce a tighter type constraint? It certainly could: Raku has the language machinery to really tighten down the requirements for a role. It would be straightforward to mandate that any type implementing the Positional role must also implement the methods for positional indexing. And, since Raku already has an Iterable role, requiring Positional types to be iterable would also be trivial. So why not?
Well, because – even if the vast majority of Positional types should allow indexing and should be iterable, there will be some that have good reasons not to be. And Raku could turn the “why not?” question around and ask “why?”
Providing guarantees versus communicating intent
All of this brings a question into focus – a question that goes right to the heart of Raku’s design philosophy and is an important one for any language designer to consider.
That question is: Is your language more interested in providing guarantees or in communicating intent
Guarantees are great
When I’m not writing Raku (or long blog posts), the programming language I spend the most time with is Rust. And Rust is very firmly on the providing guarantees side of that issue. And it’s genuinely great. There’s something just absolutely incredible and freeing about having the Rust compiler and a strong static type system at your back, of knowing that you just absolutely, 100% don’t need to worry about certain categories of bugs or errors. With that guarantee, you can drop those considerations from your mental cache altogether (you know, to free up space for the things that are cognitively complex in Rust – which isn’t a tiny list). So, yes, I saw the appeal when primarily writing Rust and I see it again every time I return to the language.
Indeed, I think Rust’s guarantees are 100% the right choice – for Rust. I believe that the strength of those guarantees was a great fit for Rust’s original use case (working on Firefox) and are a huge part of why Facebook, Microsoft, Amazon, and Google have all embraced Rust: when you’re collaborating on a team with the scope of a huge open-source project or a big tech company, guarantees become even more valuable. When some people leave, new ones join, and there’s no longer a way to get everyone on the same page, it’s great to have a language that says “you don’t have to trust their code, just trust me”.
But the thing about guarantees is that they have to be absolute. If something is “90% guaranteed”, then it’s not
Coding as a collaborative, asynchronous communication
Guarantees-versus-communication is one trade off where Raku makes the other choice, in a big way. Raku is vastly more interested in helping programmers to communicate their intent than in enforcing rules strictly enough to make guarantees. If Rust’s fundamental metaphor for code is the deductive proof – each step depends on the correctness of the previous ones, so we’d better be as sure as possible that they’re right – Raku’s fundamental metaphor is, unsurprisingly, more linguistic. Raku’s metaphor for coding is an asynchronous conversation between friends: an email exchange, maybe, or — better yet – a series of letters.
How is writing code like emailing a friend? Well, we talked last time about the three-way conversation between author, reader, and compiler, but that’s a bit of a simplification. Most of the time, we’re simultaneously reading previously-written code and writing additional code, which turns the three-way conversation into a four-way one. True, the “previous author”, “current reader/author”, and “future reader” might all be you, but the fact that you’re talking to yourself doesn’t make it any less of a conversation: either way, the goal is to understand the previous author’s meaning as well as possible, decide what you want to add to the conversation, and then express yourself as clearly as possible – subject to the constraint that the compiler also needs to understand your code.
A few words on that last point. From inside a code-as-proof metaphor, a strict compiler is a clear win. Being confident in the correctness of anything is hard enough, but it’s vastly harder as you increase the possibility space. But from a code-as-communication metaphor, there’s a real drawback to compilers (or formatters) that limit your ability to say the same thing in multiple ways. What shirt you wear stops being an expressive choice if you’re required to wear a uniform. In the same way, when there’s exactly one way to do something, then doing it that way doesn’t communicate anything. But when there’s more than one way to do it, then suddenly it makes sense to ask, “Okay, but why did they do it that way?”. This is deeply evident in Raku: there are multiple ways to write code that does the same thing, but those different ways don’t say the same thing – they allow you to place the emphasis in different points, depending on where you’d like to draw the reader’s attention. Raku’s large “vocabulary” plays the same role as increasing your vocabulary in a natural language: it makes it easier to pick just the right word.
When code is communication, rules become suggestions
When emailing a friend, neither of you can set “rules” that the other person must follow. You can make an argument for why they shouldn’t do something, you can express clearly and unequivocally that doing that would be a mistake, but you can’t stop them. You are friends – equals – and neither the email’s author nor its reader can overrule the other.
And the same is true of Raku: Raku makes it very difficult (frequently impossible) for the author of some code to 100% prevent someone from using their code in a particular way. Raku provides many ways to express – with all the intensity of an ALL CAPS EMAIL – that doing something is a really, really bad idea. But if you are determined to misuse code and knowledgeable enough, there’s pretty much no stopping you.
Coming from Rust, this took me a while to notice, because (at least in intro materials) Raku presents certain things as absolute rules (“private attributes cannot be accessed outside the class!”) when, in reality, they turn out to be strongly worded suggestions (”…unless you’re messing with the Meta Object Protocol in ways that you really shouldn’t”). From a Rust perspective, that just wouldn’t fly – private implementations should be private, But it fits perfectly with Raku’s overall design philosophy.
Communicating through sigils
Applying this design philosophy to sigils, I’ve come around to believing that making Possitional , Associative , and Callable marker roles was entirely the correct choice. After all, marker roles are entirely about communicating through code – even in Rust, the entire purpose of marker traits is to communicate some property that the Rust compiler can’t verify.
This is a perfect fit for sigils. What does @ mean? It means that the variable is Positional . Okay, what does Positional mean? It means “array-like”… Okay. What does “array-like” mean? Well, that’s up to you to decide, as part of the collaborative dialogue (trialogue?) with the past and future authors.
That doesn’t mean you’re on your own, crafting meaning from the void: Raku keeps us on the same general page by ensuring that every Rakoon has extensive experience with Array s, which creates a shared understanding for what “array-like” means. And the language documentation provides clear explanations of how to make your custom types behave like Raku’s Array . But – as I now realize – Raku isn’t going to stomp its foot and say that @ -sigiled variables must behave a particular way. If it makes sense – in your code base, in the context of your multilateral conversation – to have an @ -sigiled variable that is neither ordered nor iterable, then you can.
So, I’m disappointed that I was mistaken about Raku’s syntax when I wrote my previous post. And I’m especially sorry if anyone was confused by the uncorrected version of that post. But I’m really glad to realize Raku’s actual semantics for sigils, because it fits perfectly with Raku as a whole. Moreover, these semantics not only fit better with Raku’s design, they make Raku’s sigil’s even more better-suited for their primary purpose: helping someone writing code to clearly and concisely communicate their intent to someone reading that code
In keeping with my earlier post, I’ll include a table with the semantics of the three sigils we discussed:
| Sigil | Meaning |
|---|---|
@ |
Someone intentionally marked the variable Positional |
% |
Someone intentionally marked the variable Associative |
& |
Someone intentionally marked the variable Callable |
These semantics are perfect because, in the end, that’s what @ , % , & , and $ really are: signs of what someone else intended. Little, semantically dense, magic signs.
Raku Advent Calendar 
2 thoughts on “Day 23: Sigils followup: semantics and language design”