HN Explorer

Any resources for designing the syntax for a programming language?

32 points by _c83b ↗ HN
Hello,

I have found many articles and E-books on creating a compiler / interpreter for a programming language, but virtually none on how to design the language itself. Could anyone please recommend me to a few free E-books or articles on how to design the syntax, constructs and standard library of a custom programming language (from ground up)?

Thanks,

Ndrewxie

24 comments

[ 4.1 ms ] story [ 64.1 ms ] thread
yesenadam
Great question. On second thoughts, is there much to say except "Combine the parts you like from existing languages and the new things you want"? And, read stuff about how existing languages were designed.
ndrewxie
Yes, I've been combining features from different languages (Smalltalk and Haskell), but all I have now is this huge mess that I don't know how to fix.
ndrewxie
They're opposites, I know. I'm trying to put strong, static typing and laziness onto Smalltalk and clean up it's syntax a bit (it's already pretty clean, just a few things could be improved) - essentially a faster Smalltalk.
DonaldFisk
> I'm trying to put strong, static typing and laziness onto Smalltalk

This actually sounds quite interesting. You're starting from a good language (Smalltalk) and adding a feature absent from it but widely regarded as desirable (strong static typing).

> and clean up its syntax a bit (it's already pretty clean, just a few things could be improved)

If you go for type inference instead of having types explicitly declared, you might be able to avoid both run-time type checks and changes to Smalltalk's syntax.

> essentially a faster Smalltalk.

Bryce Kampjes (http://www.kampjes.demon.co.uk/index.html) was working on compiling Smalltalk byte code into machine code to speed it up, but that didn't involve changes to the language, so I think run-time type checks would still have been necessary. Making static type checks would result in a further speed-up.

Good luck.

woolybully
Have a look at StrongTalk.
ndrewxie
Yes, I saw that. I'm planning to make something similar, but still pretty different.
blt
how can laziness be reconciled with the statefulness of Smalltalk? wouldn't it require an incredibly smart compiler to reason about the order in which different parts of the code can modify state?
ndrewxie
Well, most variables are immutable in my language (took the lazy way out). The few mutable ones are "owned" by one scope, and must operate on the same thread.
ndrewxie
Not to bother you with this REALLY long mess, but I've been working on it for a while already, so could you please review it for me? Some flaws just won't work out correctly, and I need a pair of fresh eyes to look at it. Laziness isn't added yet, and there are a lot of flaws that make it not ready for any sort of use (recreational or otherwise).

Thanks!

Report can be found at https://gist.github.com/ndrewxie/54d3299c998569c27e281d2320b...

ndrewxie
I googled about the origins of many common languages, but most of them were just based off of another language. I tried finding out how those languages were made, but it just goes on...
PeterisP
Well, that's still the right direction that you should pursue.

The buck stops somewhere, the history is not that long - if you trace all the ancestors and key influencers of a single language, that's generally 10 at most; and if you trace another language, half of them will overlap.

You can't really skip that step anyway, it's not like you can make an informed decision about a particular syntax feature if you haven't taken a serious look on how that feature works or doesn't work in a dozen languages; and the more obscure languages tend to be very useful in this regard because they explore a larger variety of features and their combinations. Getting a short overview of syntax of at least some 25 languages would be a reasonable prerequisite before designing another one, and that overview would still be the easy and straightforward part.

ndrewxie
OK, I'll follow the chain to the end :)
rurban
The usually best introduction for this is “Compiler Construction using Flex and Bison” Anthony A. Aaby 2004 http://foja.dcs.fmph.uniba.sk/kompilatory/docs/compiler.pdf
yesenadam
Oh yeah. I read the original question as 'I know about lex/yacc' or their modern equivalents flex/bison.. but I guess not. You can do a lot just with lex/flex, e.g. it's fine for small languages. Probably play around with that before attempting something insanely ambitious.

Also, I'm not sure how getting to the first-ever-designed language will help with anything, as you (OP) seem to be trying to do. Probably it's not being aware of lexical analysis (lex family) + parsing (yacc family) . There are some great books on those.

derekp7
Take a look at the blog "lambda-the-ultimate.org", there is a thread on the sidebar called "getting started". There's links in there to several good resources (Types and Programming Languages; Structure and Interpretation of Computer Programs; Concepts, Types, and Models; Essentials of Programming Languages; etc).

Also: Start off by not thinking about syntax at all -- in fact, take all syntax out of your language. Break it down to the key concepts you want, such as how data is manipulated, what makes the program flow, etc. To do this, make your first stab either in a prefix-style language (Lisp, Scheme), or postfix-style (Postscript, Fourth, Factor). By minimizing the syntax you can figure out what makes your language different. Then start adding syntax back in.

If you take the approach of picking syntax from various languages and putting them together, it is kind of like taking pieces of several songs that you like, and trying to make a really good song out of those. Doesn't work as well as writing a new song from the bones up.

whoareu
hi @ndrewxie. who are u and what is ur underlying purpose ? what's ur background ?
ndrewxie
I'm a C/C++ developer for 15 years. I'm planning to do this mainly for learning purposes, and to create my newest project (actually an umbrella project containing 5 different projects).
enhray
When I started dabbling in programming languages design, Parsing Techniques - A Practical Guide was invaluable for me. I cannot recommend this book enough. First edition is available for free from one of co-authors site [0].

[0] https://dickgrune.com/Books/PTAPG_1st_Edition/

Edit: minor clarification.