(This is a translated/improved version of the french article Raku en 2020 from the same author)
Thank you to Elizabeth Mattijsen who helped to proofread and with advices.
Table Of Contents
- About Raku
- Community
- Rakudo
- Features of the Raku language
- Implementations based on Rakudo
- Conclusion
In this first blog post in a serie of two, you will find a general introduction with some community insights and details about interpreter and VM (Rakudo and MoarVM).
In a second part you get a long description of the Raku features to discover why Raku is worth talking about and learning.
(Next blog post is about past implementations and history details)
This blog post is made with love ❤️
About Raku
Perl 6 Raku is a programming language powerful and mature that was difficult to implement (which will be described in the second part of this post).
Raku née "Perl 6" is no longer the next version of Perl 5 (will be Perl 5.36 or Perl 7). Raku has been renamed in 2019 to clarify this state. But Raku shares a lot with Perl 5:
- some technical characteristics (close to natural language, sigils...)
- the same foundation
- the same conferences (but Raku had its first dedicated conference recently)
- partly their communities (Damian Conway, Larry Wall...)
- some events (Perl Weekly Challenge...)
Lately, Raku has become too different from Perl 5 that we can simply say "Perl" to designate Perl 5 (or Perl 7) and we would prefer "Raku" in any circumstances to talk about Perl 6. There was a time where Perl 6 community was slightly against this renaming, but the overall opinion changed and the renaming was done smoothly between 2018 and 2019 (see next blog post of the série for more details).
You can follow Raku news by subscribing to the newsletter Rakudo Weekly News.
Raku has a small community but some "killer apps":
- its IDE "Comma"
- Cro, a toolbox for the web (applications, services)
- Sparrow an automation framework, and its modules
- a (small) CPAN
Big names from Raku world
Before going into details of renaming and Raku versions, let's hightlight some past and present big contributors to Raku:
- Jonathan Worthington: Current lead developer of Rakudo (interview)
- Elizabeth Mattijsen: very involved in Raku
- Patrick Michaud: former lead deveoper of Rakudo, NQP, etc....
- Audrey Tang: currently Taïwan minister (!), formerly main developer of Pugs (Perl 6 implementation in Haskell, see this old interview or this interview)
- Damian Conway: teacher, speaker, spokesperson and prolific developer
- Larry Wall: creator of Perl (BDFL) and creator of Raku. No need to present him! In 2021, no longer involved.
- Moritz Lenz: writer and big contributor
- Flavio S. Glock: lead developer of Perlito
Not all are still active and the list is not exhaustive!
The Raku community decided recently to setup a steering council.
Raku(do)
The most advanced implementation of Raku is Rakudo (interpreter) with NQP (builtins) on MoarVM (virtual machine).
When Rakudo is released, the bundle Rakudo + MoarVM is called Rakudo Star (or Rakudo *) ⭐
Since 2015, the implementation of Raku is "features-complete" and the effort on the compiler and VM are more about fixing issues and improving performances.
Raku had multiple official releases (the most important being in 2015) :
- first public version of Rakudo * in july 2010 (major version but not feature-complete, still based on Parrot → Rakudo + Parrot)
- 6.c "Christmas" in 2015 (announcement) (Rakudo + MoarVM)
- 6.d "Diwali" in november 2019 (details of the several changes) (Rakudo + MoarVM)
- 6.e will be the next major version (Rakudo + MoarVM).
From where comes this versioning scheme?
Simply from a recurring joke about the release date of Raku. It was common to answer "will be ready for christmas" 😀
In addition to major version, the team working on Raku produces monthly minor versions.
Some books
Before going into language details, here are some books to help you learn more about Raku:
- Perl 6 Essentials
- Learning Perl 6
- Think Perl 6
- Perl 6 quick syntax reference
- Perl 6 deep dive
- Raku fundamentals
Some links
The technical details starts here, I hope you're ready 😀
The power of Raku
Raku features are impressive, I wanted to start by this :)
Raku takes roots from the linguistic skills from Larry Wall, therefore Raku is very influenced by natural language. Raku is multi-paradigm (procedural, functional, concurrency programming and object oriented).
The values of Raku can be found in the Raku manifesto (1, 2, 3)
Like its cousin Perl:
- it's a rich language with a notion of context
- its regular expressions are very powerful and Raku even integrates grammar handling directly in the language (!)
- it has a CPAN.
Contrary to Perl 5:
- interfacing with native code is deadly simple
- oriented object programming is integrated in the language (framework + object types)
- the language can deal with concurrency and provides a full toolbox for it
- Unicode is handled "by default"
For more technical details, here is the Raku glossary and also a features comparison between the compilers (it gives a good overview of the language). In the coming paragraphs, the Raku features are explained more in detail.
Courses like courses.raku.org or raku.guide are probably good "showcases" of Raku features.
Typing
- Gradual typing (dynamic and static typing):
# Dynamic
my Cool $var = 42;
# Functions using $var will coerce to another type
# upon their needs
# Static
my Str $var = "Raku";
- Annotation in signatures:
sub ab(Int $a, Int $b) { return $a+$b; };
say ab(1,1);
- Type inference
- Allomorphs (and polymorphic evaluation):
IntStr $foo
- Junctions:
my $junction = 1|2|3;
$junction += 4; # junction now equals 5|6|7
$junction += (1&2); # junction now equals (6|7|8)&(7|8|9)
- Subsets: allow to extend isomorphic classes, can be used as constraints, while being resolved at runtime:
subset Even where * %% 2;
say 1 ~~ Even; # False
- Coercion:
sub i-cast(Int() $n) { say $n.^name };
i-cast "42";
Constraints + coercion ("I want a string object and I cast into integer object"):
sub f(Int(Str) $want-str-cast-to-int, Str() $cast-to-str) { } ;
f '10', 10;
Or conditional constraints ("my argument should be an integer object or a string object"):
sub xyz($n where $_ ~~ Int | Str ) { say +$n }
- Native types:
my int8 $verysmall = 42;
say $verysmall;
- Auto‑boxing : automatic conversion from native type to object
- Auto‑vivification in the hash/arrays:
my %a;
%a{'autovivification'}++;
say %a{'autovivification'};
- Twigils (!) for instance to define the variable scope
- A full collection of objects containers. See the list of composite types (SetHash, Bag, Map, list, Mix, Pair, Set...)
- Shaped array/hashes (a kind of matrix or multi dimension hash)
- Compact arrays:
my int @array
which is stored in a contiguous manner -
Lazy lists (see also
gather
):
my @list = (1, 1, * + * ... *).grep({ $_.is-prime });
say @list[4]; # This generates elements 0..4.
# Elements 5..∞ haven't been generated yet
Routines
- Argument:
sub plus2($foo) { return $foo+2; }
- Typed:
sub mul3(Int $foo) { return $foo*3; }
- With attribute rw or copy:
sub make42($foo is rw) { $foo = 42; }
- Multi‑dispatch:
multi sub sayit( Int $n ) { say "Number: $n"; }
multi sub sayit( Str $s ) { say "String: $s"; }
The dispatch is based on the number of arguments, their types, the constraints (we can even play with it with functions nextsame
, nextwith
, callsame
, callwith
).
See also "Proper narrowness analysis" for the best candidate resolution.
- Slurp of remaining arguments and flattening of arguments (like in Perl 5).
Operators
- Raku has a big number of operators, see the full list:
- Raku can be easily extended (see documentation of the operators)
- native operators (with inline or explicit optimizations)
- operators can even be in UTF-8
Async, parallelism and concurrency
A full toolbox to handle asynchronous, parallelism and concurrency :
- promises (or future) usable with Proc::Async for external calls
- streams handling with Channel and Supplies to implement event driven or patterns publish/subscribe
- an access to low level primitives like threads, scheduler and locks
Object oriented programming and Meta object programming
Ability to define our own operators (seen above) and features. The meta object programming is partially handled in the virtual machine:
- single or multiple inheritance
- introspection
- "edit" a class after instantiation (example)
- composition/encapsulation of methods
- trust relationship which allows access to private methods of another (parent) class without making them inherit
-
delegations
has $.base is rw handles <meth1 meth2>
- type whatever "*" (which gave its name to the cast "Rakudo star")
Grammars
Support for grammars (grammar
,token
, rule
,regex
) in the language!
grammar Calculator {
token TOP { <calc-op> }
proto rule calc-op {*}
rule calc-op:sym<add> { <num> '+' <num> }
rule calc-op:sym<sub> { <num> '-' <num> }
token num { \d+ }
}
class Calculations {
method TOP ($/) { make $<calc-op>.made; }
method calc-op:sym<add> ($/) { make [+] $<num>; }
method calc-op:sym<sub> ($/) { make [-] $<num>; }
}
say Calculator.parse('2 + 3', actions => Calculations).made;
Interfacing
Native language support is really very easy!
- Easy interfacing with native languages (Native Call):
use NativeCall;
# Interface a function from unistd.h
sub getpagesize() returns int32 is native {};
# Interface a function from a shared library (e.g. libcalculator.so)
sub add(int32, int32) returns int32 is native("calculator") {};
- Interfacing with Perl 5 with Inline::Perl5:
use Test::More:from<Perl5>;
plan tests => 1;
use Inline::Perl5;
my $p5 = Inline::Perl5.new;
$p5.call('print', 'Hello World');
Unicode processed at the grapheme level
Very high level of Unicode management, in particular thanks to primitives in the MoarVM virtual machine.
The language operators themselves can be UTF-8 (like the ones you create).
Other bulk features
- Pointy arrow block (anonymous closure):
my $cube = -> $x {$x ** 3};
say $cube(3); # 27
By Alphabetic order:
my $pow = {$^x ** $^y};
say $pow(3, 4); # 81
Or even:
for 0..9 {
say "$^n2, $^n1";
}
Or with naming :
my $pow = {$:base ** $:exp};
say $pow(:base(25), :exp(2)); # 625
Rakudo Star (from 2010)
The first major version of Rakudo Star (or "Rakudo *") was based on Parrot (PCT + ParrotVM), but we will ignore this version to focus on the current implementation (Rakudo + MoarVM).
Compilation (Rakudo)
Rakudo is a compiler for Raku that implements the entire language and can target several different VMs (the main one being MoarVM). Most of Rakudo is written in "simplified Raku" (NQP). Rakudo also implements precompilation to optimize its performance. In addition to that, Rakudo improves the syntax tree using several optimization methods, here are some examples:
- the converted range iterations are in native loops
- de-virtualization of private methods with compile-time resolution
- removal of (statically) dead code
- anonymize a variable
- reduce the scope of a variable
- unfolding of the junctions
NQP
NQP is a compiler like MiniPerl for Perl 5. It is a bootstrapping tool which helps to compile the Raku parts of Rakudo and compile the libraries before running have compiled the libraries. Unlike MiniPerl for Perl 5 (which is an interpreter that can interpret all Perl syntax but miss some batteries: means without modules mixing Perl code and native code), NQP can only compile a "simplified Raku". NQP refers to both the compiler and the source code contained in files with the ".nqp" extension.
Some NQP code looks like this:
method symtable() {
%!symbol := nqp::hash() if nqp::isnull(%!symbol);
%!symbol
}
method evaluate_unquotes(@unquotes) {
my $result := self.shallow_clone();
my $i := 0;
my $elems := nqp::elems(@(self));
while $i < $elems {
$result[$i] := self[$i].evaluate_unquotes(@unquotes);
$i := $i + 1;
}
$result
}
NQP is part of Rakudo and seems inspired from MiniPerl6/KindaPerl6.
Perlito (see below) also implements the same kind of "simplified Raku".
Execution (MoarVM)
MoarVM stands for Metamodel On A Runtime VM (official site). It's a registrers-based virtual machine like Parrot. Jonathan Worthington is the founder and architect of MoarVM.
Here are some features of MoarVM:
- internal representation closer to Raku (compared to Parrot) ("On the other hand, there are features that are hard to implement efficiently if the VM doesn't support it.", according to an interview with Flavio Glock );
- superb Unicode support, with strings represented at the grapheme level
- management of asynchronous and concurrent programming with control structures for concurrent programming, synchronous sockets, timers, processes, semaphores, blocking queues, etc.
- precise garbage collection, generational (young → semispace and old → big buckets) and managing the competition
- able to manage continuations
- bounded serialization, serialize the code of the loaded modules for later and save start-up time by avoiding reconstructing the intermediate representations of this code
- runtime loading of code
- optimizations: tail call eliminations (do not stack in the call stack if the function return is in the call), dispatch, elimination of dead code, on stack replacement (replacement on the routine stack not optimized by a routine optimized), escape analysis (transform heap allocations into allocation on the stack or even in a register) and scalar replacement (eliminate memory allocations)
- JIT compilation
- profiling of the call stack and allocations
- heap snapshotting: mainly for further scans (investigation and improvement of garbage collection, for example)
I also recommend reading this introduction to MoarVM.
MAST
MAST stands for MoarVM AST (Abstract Syntax Tree).
MBC
MBC stands for MoarVM Byte Code. Here what looks like MBC:
; Grab value out of Scalar
decont r4, r1
; Grab type ; de-Scalar if needed
wval r5, 1, 34
decont r3, r5
; Ensure arg is an Int
istype r6, r4, r3
assertparamcheck r6
Implementation based on Rakudo + MoarVM (since 2012)
MoarVM does not have an assembler.
Below is another architecture diagram taken from a presentation on Rakudo/MoarVM optimizations (2014):
Conclusion
This first blog post treated more about current information about Raku.
I hope it gave you the envy to try it 😘
Next blog post will dig into historical bits of Raku (and implementation archeology) with several technical sketches 💃
It took me a lot of time to gather these traces of a dead world, but believe me it's very interesting so just let yourself be guided!
Top comments (0)