Show HN: Eno – A lightning fast, user-friendly YAML/TOML alternative

Thanks for the kind words, much appreciated!

Have you seen strictyaml?

Thanks michael! <3

That's fantastic to hear! Do let me know via email or github etc. if you run into any issues, I'm eager to gather more insights from other people's usecases and work on improvements where needed!

Yay thanks for the detailed input!

If there is demand or initiative for a portable schema solution I'll gladly support it! The native architecture in the eno libraries is programmatic because that does have its own powerful merits which are employed to the fullest in the API design, like always there's not one best choice, and the 'validation creeping into code' can be turned around into 'external schema definition creeping out of line with code' just as well. ;) Do you have some concrete usecase in mind or planned where we could explore how a portable schema solution could look like for eno? Drafting things from various real life use cases has worked great for eno so far, so that's the route I would love to go here too if we follow that track!

Custom parser implementation is easier to answer: By now I've iterated through dozens of custom parser designs for eno in multiple languages, and I'm pretty much confident that generated parsers will not stand a chance of being faster, they after all do the same thing as I do, only I can't really hand-optimize what they produce afterwards. :) You can study the benchmarks I linked to under [3], there are some generated TOML parsers included with rather disappointing performance to put it mildly, and as it stands there's not much that's faster than the eno parsers in YAML/TOML land anyway, so I have low incentive to experiment in that domain currently. :) Long term goal is to (optionally) integrate (generated or custom) C (respectively Rust) parser cores through native bindings as well, so that will bring up this question again then for sure.

As someone who has worked has written parsers & worker with generated parsers, to add to what OP has said about performance, from my experience there is a lot of noise that comes from a generated parser, and interacting with the code/output is quite unpleasant.

Also you have to decide on checking in garbage generated code into your source or adding a build step so you don’t check the code in (which is less trivial for certain languages/stacks).

Unless I’m making an MVP or a prototype I would write the parser. It’s not as hard as it sounds

One of the design conderations was and is that the format is very strict (and that way predictable), but at the same time as helpful as possible in identifying, communicating and resolving issues.

To that end all error messages that can occur are handwritten, fully localized and shared across all eno libraries (see https://github.com/eno-lang/eno-locales/blob/master/specific...) and the API implicitly handles them for you when you write programs that consume eno.

So basically eno does no magic fallbacks of any sort when faults occur, but it is candid and friendly about it when it happens. :)

At first glance dhall looks a bit like some of the CSS preprocessors out there.

This! As the introduction doc coyly says,

> so as a user we usually just concern ourselves with editing the values and a friendly developer takes care of specifying the names for us. :)

The friendly developer gets the job of conveying -- somehow -- that values for landline: and mobile: have to be valid tel#s (but must they have the country prefix?) while "hire date:" must be in the form yyyy mm dd only... and also gets the friendly job of writing one-off, locally unique validation code for these fields and hacking them into the parser.

Regarding types check out https://eno-lang.org/javascript/#loaders - basically eno allows arbitrary types on the language level and provides loaders for all primitive types and currently also a small set of non-primitive types through the libraries. The extent of loaders provided out of the box might grow, likely also externalized into companion packages like https://github.com/eno-lang/enojs-exploaders/, which currently serves as experimentation ground for this.

The core might in fact be reimplemented in C or Rust and used across implementations through native bindings, although only a small portion of the actual parsing core can actually be outsourced like that, so it will depend what the actual benchmarks say then, there's also cost associated with passing around the data through bindings, the devil's in the details there unfortunately. :)

The editor in the introspection demo is atom, the introspection is based on the excellent automcomplete boilerplate at https://codersblock.com/blog/creating-an-autocomplete-plug-i... paired with a few lines that utilize https://eno-lang.org/javascript/#Section-lookup to determine the exact context for the autocomplete suggestions. Glad you like, thanks for your interest!

Well spotted, good question!

It's also been asked in another thread on HN, I'm quoting myself here: "eno has neither indentation nor closing tags of any sort, that means if you use a section to group some values, you need to start another section to end the previous one (no closing tags!), that's why there are fieldsets, which allow short groupings that automatically end with the next field/list/fieldset."

Hope this explains it :)

> b) do you have speed tests to show your libraries are better than existing ones?

The OP linked to their benchmarks in [3]: https://github.com/eno-lang/benchmarks/

Hey, first off thanks! :)

1a) because faster was only one aspect, it also needed to be easier, even more pressingly that in fact. 1b) see answer by other poster (thanks!) 2) Not whitespace sensitive, no way to enter wrong types through syntax mistakes, hardly any learning curve for users because there is so little syntax to memorize, fully localized, hand written parser and validation errors (provided on the library side) ... and so on, check out the website for more, it's all there! ;) Thanks!

1) Yes! (You can directly dump it to a language-native structure with the raw() method too, this is not 1:1 YAML/TOML style generic deserialization though as there are no fixed types in eno)

2) Some detail aspects of whitespace-parsing around the line continuation syntax will need to be specified by the language, the shared official API I am implementing for the different platforms is fully open to improvement and future reinvention though, I'd love to see a completely new take for a library API if it comes up in the future. :)

3) Definitely!

4) I try to keep things as consistent as possible across the platforms, but if there are important language specific paradigms I think these should be taken advantage of! I can't answer details regarding the PHP implementation yet but keep in touch, I'm happy about a dialogue here! (Also I can't be good at everything :)).

5) I want one! Obviously there can't be a stable generic "just dump it already" implementation, but a smart builder-type API is definitely on the list, I even started one for enojs but had to re-prioritize because there was so much else to do for the whole ecosystem. ;)

You're right, not yet! Jump-starting the whole ecosystem was a major time investment for me but now that there is public exposure providing a formal spec has a higher priority because someone might actually see it and do something with it ;) Keep an eye on https://github.com/eno-lang/eno, this is where I'm working on it, I'll also announce it on the newsletter (http://eepurl.com/dA9LcH) when it's there!

(You can with Rust, yes)

That road (C or Rust parsing core through bindings) will likely be taken, but for the initial development and jump-starting the ecosystem it was important for me to start with implementations that can be quickly experimented with and iterated and not spend a lot of extra time on dealing with segfaults, memory leaks, the different binding mechanisms on different platforms, etc. As things stand now, people are provided with multiple, fully functioning, pure implementations that already are faster than the majority of YAML/TOML parsers. In the coming months and years there will be plenty of time to make things even faster. :)

For me caring about performance also means caring about performance on all platforms, why not after all? You can take the tabular benchmark data I provide and paste it together, or use the raw data that is also available as eno files in the repository to compare language against language too (which I initially also did but later dropped because same-language comparison for libraries made more sense to me), if you want the quick run down as far as I remember it: mostly javascript parsers lead the ranking, ruby parsers are a bit behind and just slightly ahead of Python.

ruamel.yaml in python does that to a certain degree from what I've read, you might want to check it out if yaml is ok for your usecase! (https://yaml.readthedocs.io/en/latest/)

I've given this some thought as well, and given that the eno libraries hold their own representation of data in memory this might actually be plausible to implement in some way. Still I fear this will turn out to be a hard, hard problem (as eno is not even generically serializable by design), so that's why I haven't explored it further. So for the moment I can only say - Maybe in the near future sometime, check back every once in a while! :)

On npm that's in fact already covered, scrutinize the list https://www.npmjs.com/search?q=eno :)

Programming languages are more powerful than SQL, so it might make sense for "research" stuff.

Sure!

Cultural research = notoriously underfunded, so although they have and rely on a relational database that holds their data (previously Postgres) the cost and effort associated with maintaining and extending the system is pretty high.

With the new setup the thousands of eno files represent both the place of storage and the interface to edit the data, so by that we eliminated the development effort to provide and maintain a full web frontend to the database, and the effort to just maintain the actively deployed technology somewhere and keep it at least patched for security reasons.

All that remains now technology wise is an Atom plugin that is locally installed on each client at the institute and takes care of validating, provides relational autocomplete helpers as demonstrated in [4] and offers a few hooks to kick off local builds for multiple deployment targets and deploy them to live as well.

Hope this clarifies things! :)

That was true several years ago, much less so today

Python 2's end of life is in 1 year, 4 months. https://pythonclock.org/

If you already believe TOML is perfect, why do you reconsider configuration file markup languages every couple of years?

TOML/YAML:

You could read a truncated TOML file and not realize it.

Yes through sections! See https://eno-lang.org/introduction/.

You can nest as deeply as you want and multiple sections on the same level automatically turn into a list of sections. For just a list of flat dictionaries you can also use fieldsets, see https://eno-lang.org/advanced/. :)

From what I saw, I think for at least a few parsers this might be the case because they are built on generated parser code, and it's just easy to run into unfavorable bits and pieces in the output that way, which can drag down performance completely although 95% of the parser are just fine. Technically there's no reasons why toml parsers shouldn't be just as fast or faster as yaml or even eno. :) In any case I'd be happy if the benchmarks stir up some movement and maybe kick off some high performance toml parser intiative, toml is an awesome format and also 0.5.0 was just officially released so there's a good reason to update the parsers now anyway. :)