112 comments

[ 23.5 ms ] story [ 4770 ms ] thread
Perl 6 name maybe was a bad choice back then, but we all got used to it - there was no need changing it nowadays.
I disagree. It's a completely different language.

Try reading some of this: https://github.com/Raku/problem-solving/issues/81

Yes, but I mean we all got used that "Perl 6" is a different language.
Indeed, everybody got used to the idea that "Perl 6" was a different language. But a vocal part of prominent Perl community members, preferred for "Perl 6" not to be named "Perl" anymore. As well as a part of the "Perl 6" community.

In the end it felt better to stop wasting all those emocycles and suggest a rename.

Had viewers of Police academy 6 been treated to a brutal crime drama, rather than a continuation of the farce found in the series to date, you'd expect them to just get used to it, rather than accept that naming as the next version of something vastly different is a dumb idea.

Edit* I make no criticism of the language, simply the argument that a different language should have been named as the incremental release of another.

Highlander 2: Scotts from Space would have been a better comparison than Police Academy 6.

They both got 0% on rotten tomatoes, but I don’t think Police Academy 6 tried to switch genres.

That is my point. "Perl" didn't try to change genres, something different came along and called itself "Perl 6". No disrespect to either language, but linguistically this is has a negative impact for both Perl 5 and "Perl 6". Maintaining that certain people got "used to" the idea proves that it was never a wise move.

Also, note what chromatic has to say:

https://perlmonks.com/?node_id=1220578

https://perlmonks.com/?node_id=11105180

And so on.

> something different came along and called itself "Perl 6"

You appear to forget that the "Perl 6" initiative originally came from the Perl 5 Porters. It did not just come along and called itself "Perl 6": the "Perl 6" name was used because originally it was intended as the successor of Perl 5.

That it didn't materialize in time, that it became something much different, is something that happened later. But even with Pugs around 2005, Perl 6 was still the successor to Perl 5. Only around 2010, when the sister language concept was introduced, was Perl 6 considered to be a different language.

The concept of perl 6, at some stage shifted from being the next incremental release of the Perl language, sure. For around 10 or so years the Perl 6 crew knew this was going to be a different language, yet held onto the name for this period of time and even made arguments in public that Perl 6 should remain the name, given all the facts, as recently as last year. This sort of thing can cause problems in communities, and nobody on the outside of the inner clan really cares, beyond the point that all the slight of hand and infighting pushes good folk away.

Bullying Isn't Effective Advocacy: https://perlmonks.com/?node_id=1221390

Also, when it was known Perl 6 was going to be a different language, they were several year away from a release

On the other hand Return of the Living Dead 3 did switch genres.
honestly to me it's like perl^perl
You'll get used to Raku too. When the rename went through I had similar thoughts but now I think it was a very good decision. Less "Perl is dead" BS at least. Now Perl 5.30+ can evolve and become Perl 7 and so on. I also thought Raku was a silly name, but it's short it's consistent with Rakudo.
Perl 7? There are tons of Perl6 features missing still, and I'm the only one working on that. There's zero interest to go to Perl 6, i.e. proper types, OO, FFI, macros, threading, parsing.
Everyone knows it's a completely different language, that's not in question.

What is in question was it worth making the already extremely messy name situation even more confusing?

Raku, Rakudo, Rakudo Star, it's like they went out of their way to make it confusing. Which, given the Perl culture of whimsy, isn't unlikely.

> Raku, Rakudo, Rakudo Star, it's like they went out of their way to make it confusing.

Not at all.

Haskell, GHC, hackage/cabal/whatever -- intentionally confusing? No, they are different things.

Same in the Raku case: Raku is the language, Rakudo a compiler, and Rakudo Star a distribution (with compiler, docs, module installer etc.)

Giving three different things the same name would be much more confusing.

> Giving three different things the same name would be much more confusing.

No different than "Perl 6" being a different thing from "Perl 5".

> Haskell, GHC, hackage/cabal/whatever -- intentionally confusing? No, they are different things.

Those are all completely different words though... that's my point - that's the sensible way to do it.

> Raku is the language, Rakudo a compiler, and Rakudo Star a distribution

These all start with Raku. Can you understand how they'd be easy to mix up, rather than using different words like in the previous example?

> Giving three different things the same name would be much more confusing.

You've got the wrong end of the stick. I think they should have different names. I think at the moment their names are too similar.

I do like the cleverness of it.

Raku is the language.

Rakudo = Raku “Do”

Rakudo Star = Rakudo* (i.e rakudo, rakudo-devel, rakudo-doc,…)

It has more linguistic cohesion that the tooling of a lot of languages.

Java, javac, java* seems totally normal. I feel like it’s because “Raku” sounds alien and so you don’t read Raku-do you read Rakudo because Raku “isn’t a word”.

> I do like the cleverness of it.

But that's the problem! I don't want clever, witty, whimsical. I want clear, simple, precise.

This is what went seriously wrong with Perl. In the 90s when Perl was doing well it was fashionable to be wacky like this. Then around 2000 the fashion suddenly changed and the industry grew up and people started to take things more seriously. Now Perl looks terribly out of date and silly.

> the industry grew up and people started to take things more seriously.

Really?

Goofy codenames for projects (Bionic Beaver, etc), the sign of a 'serious' project is having a cartoon mascot, 'open source' people relying on closed/proprietary things like slack, macOS, etc. instead of open/libre alternatives, 1 page github READMEs and API docs as 'documentation' but having a youtube channel full of 'howto' videos instead

there is probably an equal amount of goofyness/immaturity and seriousness then as now - it just manifests differently

> Really?

The programming language industry, yes. There was a marked shift towards simplicity away from silly whimsy. The extreme example is Go.

Then use a clear, simple, precise language; I think Java for instance, with the improvements in the last versions, could fit the bill.

Personally, I'm happy that we live in a time where we have many different languages featuring meny different paradigms; I find it refreshing and stimulating.

> Then use a clear, simple, precise language

I'm not in charge of what language the Perl projects choose to use.

> Personally, I'm happy that we live in a time where we have many different languages featuring meny different paradigms

You're not arguing against what anyone in this thread said.

Raku and Rakudo are different words.

“Raku” is the anglicized form of “楽”, which means “easy”. (It is also a form of pottery.)

“Rakuda do” is supposed to be “way of the camel”.

“Rakudo” is “Rakuda do” but shortened/portmanteau.

“Rakudo” is also supposed to mean “paradise”.

---

I want to know how “easy” and “paradise” are not considered different words.

And “Rakudo Star” is the “Rakudo” compiler combined with a bunch of useful modules. So it makes a lot of sense for it to contain “Rakudo”, because it does indeed include “Rakudo”.

---

It also makes a certain amount of sense that “Rakudo” is a thing which you use to “do” "Raku".

> Everyone knows it's a completely different language

Everyone who have been following the evolution of the language the last decade. But most people have not, and new people are born every day. So the name change will avoid confusion going forward and make it easier for both the Perl 5 and Raku communities.

Before the rename, any thread about any Perl 6 topic would harp about how it's a terrible name.

Now after the rename, I guess every HN discussion will have a subthread discussing how the rename was terrible, and it should've stayed Perl 6.

There's just no pleasing the Internet crowd.

This happens a lot. The thing is the “crowd” becomes a somewhat meaningless concept when you attempt to assign it opinions. There are two different subsets of the crowd and only one is complaining at any time.

This can also come up in political contexts when some of your adversaries say something, but you don’t track which of them said it, and instead assign the opinion to [democrats, republicans, libertarians] as a whole. Next thing you know you think they’re all dirty hypocrites arguing in bad faith, and it takes an effort to see otherwise.

> Now after the rename, I guess every HN discussion will have a subthread discussing how the rename was terrible, and it should've stayed Perl 6.

I don't see any evidence for this claim. Besides those on the inside, I have yet to see any support for naming a different language as logical next version of another.

So is Perl never going to go past version 5 now?
I thought the point of the rename was Perl 5 can now release Perl 6 or 7, with Raku being independent.
> I thought the point of the rename was Perl 5 can now release Perl 6

I think that would only add to the confusion, since Perl 6 was already a thing, albeit totally different from Perl 5

They should skip "Perl 6" and go directly to "Perl 7", like Windows that skip "Windows 9" to avoid confussion.
Unfortunately, "Perl 7" was a "thing" in the discussions before the name change. IMO, going to "Perl 32" makes more sense, as it actually reflects the current version numbering scheme. Or refer to the year: Perl 2020.
If Oracle had "bought" Perl they'd probably renamed it to Perl 32 by now and sued everyone else for using sygils.

The current numbering scheme uses odd minor version numbers for development versions and even for stable, just like Linux used to, until Linus got fed up with 2.6 after 8 years. Now it gets a bump whenever the minor version sticks too much.

https://askubuntu.com/a/843247

"Perl 32? But I have a 64 bit processor!"
I remember the "skipping version 9" in windows was due to many developers using "startsWith(os.version(), "9")" (pseudocode obviously) to detect windows 95/99 and therefore they avoid using version 9 in order to remain compatible, but I might remember wrong.
PHP did the same thing, skipping from 5 directly to 7 because PHP 6 ended up being a dead-end. It was pretty painless (at least as far as I could see).

“A foolish consistency is the hobgoblin of little minds.” — Ralph Waldo Emerson

That is not actually the same thing.

PHP 6 was thrown away never to be seen from again, because it was seen as bad.

Perl 6 was renamed because both Perl 5 and Perl 6 are significantly different active languages.

If Perl went to 7 it could indicate to some that 6 was bad in the same way. That is it would trade one confusion for another.

---

If Perl truly wants to go beyond 5, it should make a major step.

Either by going to the subversion (32), or the year (2020).

Well, the features in Raku ARE what would be (and was) Perl 6. Perl 5 most likely won’t inherit these features as the benefit of Perl 5 IMHO is that it is stable and ubiquitous.

Raku has some pretty amazing things going for it though. It is unique compared to any programming language with its grammars and such.

Good question. I has been suggested to use the major version number in the future. Which would mean in May we would see the release of Perl 32.
I would prefer if they created a year alias to the version.

So both of these lines would be equivalent.

    use v5.32;

    use Perl v2020;
There's no need to when there are an infinite amount of decimal places.
Your guess is as good as anyone else's at this point.
> by getting the whole parser onto a single screen, I find that I can get the whole problem into my brain’s working memory and avoid burning cycles scrolling up and down to pin down butterflies bugs.

This is an excellent point, and one that not enough programmers pay attention to (well, outside of the APL family of languages, anyway!)

Forth programmers often talk about keeping down the number of screens a lot too, for example page 23 of Thinking Forth: "A component can usually be written in one or two screens of Forth."

(Aside from the attraction in small and simple code, each screen in forth is traditionally allocated a designated spot on the hard-drive so it's partially also for space and HDD usage efficiency too)

As a C++ programmer, I've grown so sick of

                    }
                } // not
            } // a single
        } // fucking else
        // implicit else
        // return nullptr
        return result;
    }
A deeply-indented scope should mean "shit is complicated, think deeply about it," not "there were five times I tried something and elected to punt and return null if it failed." The LLVM style guide has the right idea; if you're gonna punt, early-return.

As a side-note, I really wish the C++ community would take a page out of Lisp's book and stop giving closing braces their own lines. But that'll never happen.

> giving closing braces their own lines

Oh for the love of God, please don't! That would make things even more unreadable.

Also, the closing braces are often accompanied by error handling and cleaning up code.

You might like the golang approach better, which is really the Linux kernel approach when I think about it....

I think that C-like languages wouldn’t be able to handle putting closing braces on the same line because of the lack of Paredit-style editor support. In Lisp, it works well.
What doesn't work well for C-like languages? I've been writing a lot of Rust lately with Smartparens (a paredit like package in Emacs) and haven't had any issues with its support for Rust.
We're talking about putting the closing brace on the same line of an expression, like this:

  int double(int x){
    return x * 2;}
instead of like this:

  int double(int x) {
    return x * 2;
  }
In Lisp, you would write that like this:

  (defun double (x)
    (* 2 x))
and not this:

  (defun double (x)
    (* 2 x)
  )
You can read either with syntax highlighting and the appropriate indentation, but it's much harder to get away with putting the closing brace on the same line in C-like languages because of how few people use something like Smartparens or Paredit and how having multiple kinds of brackets (parens, square brackets, curly braces, and angle brackets) introduces ambiguity.
Yeah, I understood what we were talking about – sorry for not being more clear in my reply earlier.

When you said that putting the closing bracket on the same line wouldn't work because of the "lack of Paredit-style editor support" for C-like languages, I thought you meant that the Paredit-style plugins wouldn't work well with/support C-like languages. That struck me as odd, because I use Smartparens when writing Rust, and it works just as well as it does when I write a lisp/scheme. In particular, it handles parens, square brackets, curly braces, and angle brackets just fine (and it even distinguishes between a single quote that's used to start a character literal (which should be paired) and a single quote that indicates a lifetime label (which should not))[0]. I could easily use it to write Rust with closing braces on the same line; the only reason I don't is because it's not the common style in that community.

Based on your last comment, I now understand that you're not saying Paredit-style plugins don't _work_ with C-like languages. You're saying that most programmers who write in those languages don't use those plugins. Is that right? If so, I agree that they aren't widely used and that it'd be hard for anyone to use the formatting style above without adopting a plugin.

[0]: The only thing Smartparens fails to handle is that it does not recognize that a pipe character is a pair when delimiting the arguments for a closure. I'm sure I could fix either Smartparens or rust-mode to fix that issue but so far I haven't been annoyed enough to bother.

[Edited to add a missing close-paren, fittingly enough.]

Paredit with Lisp is really slick for moving stuff (including yourself) around the AST. What you're describing, the auto-matching, is really just the tip of the iceberg.

With Paredit, there are key chords for moving up, down, and sideways (forwards and backwards) on the tree of Sexps, for transposing two sibling Sexps, for grabbing a sibling (before or after) of a Sexp and making it a child ("slurping"), for taking a child of Sexp and making at a sibling ("barfing").

I have not found a reliable way to slurp statements into the scope of an if statement (Paredit kind of works but tends to misplace semicolons and various other things).

My bad with the poor communication up above. Turns out that we were basically in agreement the whole time. I'm impressed that it can handle the single-quote label thing; that strikes me as the exact sort of thing that I would think a tool like that would have trouble with.
Lisp programmers lived without Paredit just fine for many decades. Even if we assume that developers aren't clever enough to write a Paredit extension for their editors/IDEs for curly-brace languages, I don't see how that's a problem if this style of coding were truly superior for those types of languages.
I experimented with such a C style once, years ago; it was perfectly workable in Vim.

The opening curly brace is better like this:

  int double(int x)
  { x += 3;
    return x * 2; }
I found it's good to have spaces within the braces, including when closing them:

  int double(int x, int b)
  { if (b)
    { x += 3;
      return x * 2; }
    else
    { x -= 3;
      return x * 4; } }
This naturally jives with a two space indentation, so everything is pleasingly copacetic. I feel I could easiy maintain a 200KLOC code base in this style.

Oh, incidentallly, this style is followed in the production rule action bodies of the TXR yacc file parser.y:

http://www.kylheku.com/cgit/txr/tree/parser.y

I wonder how well GCC's newer "misleading indentation warnings" handle this, though.

> C-like languages because of how few people use something like Smartparens or Paredit

However, everyone in C (or C-like) programming with two brain-cells to rub together uses at least auto-indention and brace/bracket/paren matching features in their editor. A stock installation of Vim turns some of it on by default if you open anything with a given suffix like .c or .js.

These editing features work equally well regardless of how you lay out the braces.

I call it telescope or pennant code, it's definitely a smell.
In Python, it's even worse, since there's no braces and not returning anything implicitly returns None.
Mypy fixes the second problem, at least.
Python at least has if ... elif ... else to deal with it.
I always take this as Python's way of strongly discouraging deeply nested and overly long functions... ( I realise this might be Stockholm syndrome...)
If a function returns early, its post-conditions cannot be checked.
This is not true in Raku: whichever way you leave a subroutine, the returned value will be checked against the return constraint. And throw if that fails.

More generally, you can specify KEEP and UNDO phasers in a block, that will fire whether the block is left successfully or not. So a typical usage would be `UNDO $dbh.rollback`.

could you give an example of this?

(all I can think of is something where you could handle it with a finally-type clause)

maybe a fn shouldn't have post conditions?

Do you mean checked by a human reader? If so, a linear sequence of if statements that return early is far easier to check than than deeply nested statements where the error handling for failures is effectively back-to-front at the end.

Or are you saying that there are some lines of code at the end checking the post conditions? I'm not exactly sure what that would look like, it sounds a bit insane, especially if you do it for every function. Could you give an example?

I mean programmatically with an assertion statement or similar. You don't want to add the same assertion before each return statement.
I’d love an example if you have one. I’ve used assertions lots as guards or pre-condition checks on entering the function but never right before returning.
A complicated sorting algorithm could assert that the result is sorted before returning.
Hopefully you saw Liz's comment.

Raku has a zero cost way to add arbitrary post processing that kicks in no matter where or how a block of code exits.

This provides a general mechanism accessible by arbitrary user defined control flow related constructs.

Raku also ships with a dozen or so built in constructs such as PRE and POST blocks that are the foundational elements for DbC, plus UNDO and KEEP blocks for transactional semantics, and so on.

In many situations you can fix this by adding a not to the if statement. eg,

Before:

    if (a.works()) {
      // some code
      if (b.works()) {
        // more code
      }
    }
After:

    if (!a.works()) {
      // error handling code, if necessary
      return;
    }
    
    // some code
    
    if (!b.works()) {
      // error handling code, if necessary
      return;
    }
    
    // more code
This is called "early return" and the parent already mentioned it:

> if you're gonna punt, early-return

here there is some significant restructuring of the flow control, early returns could have just meant return statements in the middle of the function.
I'm really quite sure they were referring to the type of code in your comment. Early return on error (as in, what you posted) vs single return with deeply nested if statements is an absolutely classic coding debate.
just as a quick clarification, I am not the OP
Oops, I did indeed miss that. I suppose that shows that it was helpful to spell it out.
I usually try to bail out of my code early on in Python so I have all my if bail condition statements before I bother trying to do any other logic. Actually I do this in all languages.

Bail early and you wont have spaghetti code, plus you wont even allocate things to memory you're not even going to use, cause you're going to bail.

>if you're gonna punt, early-return.

I don't know what it is but I once interviewed for an embedded C position. During the interview, I wrote out some code samples. The only issue the interviewer found was that I had "returned early". He said there could basically only be one return in a function and it had to be at the end. I told him I'd never heard of that. He only said it was a "standard" that had to be followed because of the crypo involved in the code base.

Sadly, even though I didn't get the job, that stupid statement of only one return call in a function has stuck with me and lead to many mental arguments.

Striving for a single return (without going out of your way to achieve it) is good practice.

There are two main advantages:

- it's obvious when the function ends, no hidden early exits in the middle of nowhere

- if you need to do something before returning, there is only one place to do it (e.g. trim the output, e.g. log performances), and it make it easier to move this kind of last mile logic to a different function (if it's of any interest)

About your interview, don't sweat it, is not for that you didn't pass (you could do a great job and yet lose to a more fit candidate)

The "standard" this might have been a reference to is MISRA C. It can be a useful rule to avoid leaking resources/finalising shit.
> by getting the whole parser onto a single screen, I find that I can get the whole problem into my brain’s working memory and avoid burning cycles scrolling up and down to pin down butterflies bugs.

I haven't seen many developers do this: get a computer screen that can stand in the portrait position.

My setup currently is: laptop monitor + 27" screen in portrait. I never have the trouble of scrolling back up and down again when it comes to reading code.

I use two large external monitors, one in portrait. It's changed my coding style.
Out of interest: anyone in here using Perl 6 / Raku for anything in production?
On the #raku channel on Freenode, several users are using Raku in production.
> Yes – a recursive descent parser written from scratch in perl5 – pay dirt!

What made the author write this in the first place? He could use one of the many many parsing libraries, and then at least the comparison with Raku grammars would be honest and fair.

I’d prefer to write a hundred or so lines of code to adding a dependency. I can usually get those 100 lines of code right in an afternoon, but keeping up with the dependency is ongoing maintenance.

The dependency could break compatibility or add a bug a year from now.

Parsers aren't particularly difficult to write, and even if a hand-coded parser might not be very efficient compared to a generated one, it's unlikely to be a significant factor in the overall performance of the application. Compared to the complexity overhead of adding another dependency, be it a library or an external parser generator, i feel it's a fair trade-off.
Not a Perl guy. What are the semicolons in that if statement all about?
Did you mean “for(;;)”? I didn’t notice semicolons in an if-statement. In the for-loop, it means “forever loop.” There is no declaration , check, iterate parts in the loop. You may be more familiar with “for(i=0; i<10; i++)”
It's a C idiom. Some people prefer `for(;;)` over `while(1)` to create infinite loops.
I see. Just a c-style loop absent conditions. I find it kind of ugly for some reason.
Weird, since Raku has `loop { }` that means exactly that. I guess the author was not aware of it (or just doesn't like it).

EDIT: Oh, that's in the Perl (5) code. Nevermind then.

In C-like it kind of makes sense – it's 1 character shorter, and on some ancient crappy compilers it may actually run faster than while(1) since it doesn't have to check the condition every time ;)

I thought it was just some old weird hate on the while() construct. I remember a long time ago people advocating for using for() always for looping because that way the language was more regular. I thought it was a weird argument.
I think it is usually written as while(1) { ... }
Can someone give a little more background why a unit conversion library needs a parser?

Like an ELI5 for someone not into neither Perl or Raku and haven’t met the units library before.

you would define values like

   weight = unit("10 kg")
   speed = unit("2.5 furlong per week")
My imaginary units library would have been like

    Unit(float, unit_as_string)
The free text version is pretty cool though
I would parse the free text version into something like your version. It could be used for user input from a text input field or the command line.
Even further, presumably things like kg·m·s⁻², which (perhaps) this library can equate to Newtons.
this works now x for input and output ... the challenge for this kind of thing in bare code is that you need a LOT of namespace ... .oO maybe override the dot?
But what does strings have to do with it? Why can't those expressions be plain Perl, maybe with some overloaded operators?

For example,

    import week, custom, si, type from stdlib.units
    weight = 10 * si.kg
    speed = 2.5 * (custom("furlong", type.LENGTH) / week)
this is also important - but imo, and in practice from perl5, the Units library experience is a lot richer if you can do both - especially for entry level programmers such as high school students - more on this to come in v2
You really wouldn't really need to in Raku.

Mainly because you can add things to the existing parser instead. (See postfix and infix below.)

    role Physics::Datum {
      has Real $.value is required;
    }
    role Physics::Distance {
      also does Physics::Datum;
    }
    role Physics::Volume {
      also does Physics::Datum;
    }

    class Physics::Volume::Gallon {
      also does Physics::Volume;
    }

    class Physics::Distance::Mile {
      also does Physics::Distance;
    }

    role Physics::Per[ ::N-Type, ::D-Type ] {
      has N-Type $.nu is required;
      has D-Type $.de is required;

      # could have a submethod TWEAK to reduce the fraction
    }

    constant gallon = Physics::Volume::Gallon.new( value => 1 );

    sub postfix:<miles> ( Real $value ){
      Physics::Distance::Meter.new( :$value )
    }

    sub infix:<per> ( ::NT Physics::Datum $nu, ::DT Physics::Datum $de ){
      Physics::Per[NT,DT].new( :$nu, :$de )
    }


    my $mpg = 5miles per gallon;
Which would result in the following data structure.

    Physics::Per[ Physics::Distance::Mile, Physics::Volume::Gallon ].new(
      nu => Physics::Distance::Mile.new( value => 5 ),
      de => Physics::Volume::Gallon.new( value => 1 ),
    )
I think the main reason it was written that way was it was copied from Perl. A lot of the above would be more difficult or slower in Perl.

(This was mostly a quick throwaway example of how it could be done.)

The main issue I have with swapping Perl with Raku is that a major strength of Perl is its ubiquitousness. Except on Windows, it's almost certainly installed on 90% of the systems I have ever used. Meanwhile, rakudo is not even in the repositories of most distributions. It's not very encouraging, if I have to be honest.
I'm the same way. I'd love to use Raku over Perl, because it looks like an improvement in many ways. Alas, the use case I have for Perl is whipping up a script on any server for some ad hoc analysis or data massaging or as a UI to something. If I first have to install Raku, I'm more likely to write a shell script or Python code -- both of which tend to execute out of the box.

This is also why I'm not as enthusiastic about CPAN as I could be. It's great that the modules are available for when you want to write real applications, but that's not what I do with Perl and one-off scripts really don't seem like it's worth pulling in external dependencies for.

ISTM that a grammar parser is a library feature, not a language feature, per se. Any reason you couldn’t port the parser back to Perl 5 and get a similar savings in line count?
In Raku, grammars are an integral part of the language. This is necessary, as the language itself uses a grammar to compile Raku code.
But the grammar parsing and processing is not something which is done with core-only features. Later hopefully, but currently it's very dumb, and can easily be done as library. Only longest token matching is needed.