Day 2 – An update on raku.land

A couple of years ago we gave a talk at the The Raku Conference 2021 which served both as an announcement and an introduction of raku.land. It was an exciting time for us, since it was the first time we were officially addressing the community, and we were keen to see how our work would be received and what impact it would have.

Fast-forward to today, and some of you might have noticed that the venerable modules.raku.org domain now redirects to raku.land.

We thought this was a good chance to take a look at what lead us here, where the new site is, where it is going, and how you—yes, you!—can help get it there.

What happened to the old site?

The original modules.raku.org site was written to solve a very specific problem: the Raku community needed a way to browse the modules that had been released to its fledgling ecosystem. It also served as an early test-bed for the integration of some community features that grew around that ecosystem, like the citation index, for example.

This was in the early days of Raku, before there was a satisfactory way to build and serve a website with the features that were needed, so this initial version was built using Mojolicious, one of Perl’s shining web frameworks.

There was a lot to like in this website, but as time went on, and its original maintainers moved on to different projects, it was largely left unmaintained. This not only meant that it started to lag behind some of the new features that had been added to the language (most importantly what at the time was the new zef ecosystem), but it was also becoming harder for the infrastructure team to keep the site running.

This came to a head a couple of weeks ago, when the site went down and could not be brought back up. That precipitated a decision that had been in discussion for the better part of a year, and the domain was redirected to raku.land.

This is the end of an era. Raku would not be where it is today were it not for what the old site allowed us to do, and we celebrate it for what it achieved. But this is also the start of a new era.

In with the new

What began as an experiment about using Raku in a real website, has now started to become an essential part of the day-to-day activities of the Raku community. It was the first web-based front-end for the zef ecosystem, and for a little over a year it has powered the canonical URLs used by Rakudo Weekly News.

We take this responsibility very seriously, so we’ve tried to respond to the user feedback we have received since then. Some of the concrete changes to the site since our initial announcement include the ability to display multiple versions of a distribution, and to link to specific ones; an endpoint to render recent releases in JSON; the display of reverse dependencies, and dependencies by stage; and the addition of a dark theme.

We know that there are still plenty of things that the site could do better, but we also know that the best chance we have to accomplish them is with the help of the community at large. This is true specially for some of the trickiest problems we need to grapple with, where the difficulty lies not in solving the problem, but in deciding how to solve the problem.

Tough problems to solve

The main problem that raku.land aims to solve—namely, that of cataloguing and organising information about Raku distributions—is a difficult one. And although it is a well known problem (after all, a lot of languages have distribution ecosystems), some aspects of it are particularly difficult because we are dealing with Raku distributions (after all, not many languages support multiple ecosystems!).

Take the following example: if you use Digest::MD5, which module are you loading? The answer will actually depend on the distributions that have been installed in your system. You could be loading the Digest::MD5 module provided by Digest:auth<zef:grondilu>, or you could be loading the one provided by Digest::MD5:auth<github:cosimo>. These are incompatible distributions, written by different authors, released on different ecosystems… but both provide a module with the same name. We would all agree these are different, but it’s up to you to disambiguate.

That particular example happens across ecosystems, but it doesn’t have to. Take JSON::Class: there’s JSON::Class:auth<zef:vrurg> and JSON::Class:auth<zef:jonathanstowe>. These are both on the same ecosystem (zef) but other than a shared name, these are also incompatible. Chances are we would also agree these are different.

On the other hand, you have several distributions that have historical releases on one ecosystem, but then moved to a newer one when it became available. Take Cro:auth<cpan:JNTHN> and Cro:auth<zef:cro>, for example. These distributions share the same codebase, and have been released by the same author, but since they’ve been released on separate ecosystems they have different authorities. So, are these the same?

A lot of the hardest problems raku.land has to deal with stem from this disambiguation, and from the fact that sometimes we would expect these “ambiguous” distributions to be treated as logically different, and sometimes as logically the same. For Raku the language these are all unambiguously different, but for Raku programmers the answer will depend.

The ramifications are wide-ranging: they affect everything from our ability to have short canonical distribution URLs, to how distributions are listed on the site, to our ability to detect when a distribution depends on another: if it lists “Digest::MD5” as a dependency, how can we know which one that will be? Whatever solution we come to in raku.land for these issues must come as a decision from the community, because the site needs to reflect the ecosystems it serves, and not the other way around. So we need your help.

A community project

The term “community project” is a slippery one. We all think we understand the term when we read it, but when it comes to deciding whether a specific project does or does not belong to “the community”, opinions differ.

Regardless of these differences though, raku.land is a project that was always developed with the interests of the community as a top priority. And as such, it’s a project that relies on the participation of the community for its success.

We’ve tried to make it easier for anyone to get involved and make contributions: we’ve kept the source-code lightweight so that it’s (hopefully!) easier to jump in and understand; and we’ve tried to document how the code and its deployment works. We have a roadmap of things that we would like to see in the future, and the team is reachable through a project email address and our own IRC channel on libera.chat.

Although the goals in that roadmap could possibly be more fleshed out, there are a lot of ambitious ideas we have for the future:

• Source-code browsing
• Properly linked rendered documentation
• A unified API to expose some of the information we collect across ecosystems
• The ability to render diffs between versions
• Improvements to our search feature
• … and more!

What we lack the most right now is the help of people like you. Some of these are hard problems to solve, but some of them are entirely achievable goals that we could tackle right now.

And there are multiple ways to help. We want technical writers who can help us flesh out the documentation wherever it’s needed. We need help raising bug reports and triaging tickets. We need more voices contributing to the conversation and pointing out where we’ve grown complacent. And of course, we can always use help implementing new features and integrating with new and exciting parts of the ecosystem.

From our point of view, raku.land is undoubtedly a community project, and the best way to move it forward is to get involved. Come join us!