The ‘normal’ way to redefine words in forth is a bit of cheating, though. Redefining a word merely introduces a word with a name that shadows that of the old, still existing word.
Words created before the definition still call the old implementation, and, depending on the implementation (a phrase that, I think applies to about any statement about forth) you can still find it when you walk the dictionary and call it.
Smalltalk is missing too, and that’s a bigger omission. When you redefine a function there, it starts getting used everywhere.
I wonder what a blog post written about this 50 years from now will look like. Would it still mention C and lisp? Would readers of said article still mention forth and smalltalk as missing? Also, what reasonably successful languages from the ‘60s/‘70s already are gone from the collective mind?
Being used today to write very popular OSes, I see C still being mentioned, but by then lisp might have fallen into the abyss of history (except for, probably, a few computer historians)
> by then lisp might have fallen into the abyss of history
I don't think so, because being a lisp is a trait of a language rather than a specific language. Scheme is a lisp. Racket is a lisp. Clojure is a lisp.
People keep making new ones because lisp continues to be a useful idea. It keeps not quite going mainstream, the reasons for which have been subject to much speculation. It's most likely that there will be some semi-popular lisp in 50 years.
There are languages which have Lisp in their name for a reason: Emacs Lisp, Visual Lisp, ISLisp, Common Lisp, ... Historically Interlisp (see https://interlisp.org ), Standard Lisp (see REDUCE), and many others.
A great language, I cut some of my programming teeth with GraFORTH on Apple II in 1982/1983 in high school. I wish I still had some of my old floppy disks. I did some floating-horizon for plotting f(x,y) and some basic animations. A friend and I did some demo programs for a couple of local entrepreneurs (Romel and Ludimar in case you're out there) selling Apple II clones, the MicroEngenho, in Brasília. Our work, displayed at a trade show, made it onto the nightly news.
I wish there was a Lisp-like language that was statically typed and compiled to optimized native code. The closest thing I know of is Nim, which has the programmability, but not the simple syntax.
Coalton* is a language that interoperates natively with Common Lisp, and has a type system that exceeds the expressiveness of Haskell 98 (but not modern GHC Haskell).
It compiles to native, highly optimized machine code.
The experience of writing nim is very different from the experience of writing common lisp. Nim may be more mutable than c, but that is not saying much.
> How do you optimize code without knowing the types of variables?
Guesswork, static analysis, experiment etc. JSC does all kinds of black magic to make JavaScript run fast.
For a large amount of programs (basically all "classical" programs like you'd read about in the past rather than modern apps) you have a fairly clear path through the code for their data, this means that you can optimize that path.
This doesn't mean the performance will be as good as a statically typed language but keep in mind 90% of compiler optimization is basically useless on many programs vs. the 10% that makes the important bits run fast.
The Mike Pall method uses a typed SSA intermediate language, and adds trace guards anywhere it can't guarantee a single type. LuaJIT optimizes fairly well.
CL is often seen as "dynamic" but it's better than that: we compile our code with a keystroke and we instantly get compile-time type warnings and errors. It is not fully statically typed, but it is still way better than say Python. Usually, type checks happen at the functions' boundaries: this expected a int but was given a string, this argument is declared but unused etc. Add in Coalton, and you're approaching a Haskell experience. And that happens in our tight read-eval-print-loop, the feedback is instant. We can also do many things at compile time (macros etc). It's better than one might think.
I do not have much personal experience, but SBCL's type inference is supposed to be decent/comparable to the Franz/Allegro/Lisp Works commercial options. I'm not sure if anyone has done "declare-density needed to achieve xyz perf" cross comparisons or the like.
There is also https://en.wikipedia.org/wiki/Stalin_(Scheme_implementation) which was doing lifetime analysis waaay back in the mid 90s, but I don't think Suskind had a declare system at all in Stalin. So, places where inference did not work were probably just "slow to be fixed by users tweaking code" to make inference better. I think he was mostly his own main user.
Both Coalton & Carp do look interesting. There can be complaints with how "prefix notation" scales to large expressions. Cython is a gradually typed Python, but late noughties ideas in metaprogramming/macros/etc. never quite took off.
There can be an undesirable effect of gradual/optional static types that code starts off "lazy" and then can require surprisingly large efforts to tighten down, especially in team settings. Even with profiling/90-10 heuristic rules, a later dev trying to optimize earlier lazy work can still be 10X, especially if type dynamism was really leaned upon. So, at least having an easy switch to force early/in-development static typing on original authors is likely a good thing (or at least strongly encourage with warnings...).
Probably doesn't cover the 'optimized native code' part but it obviously could, like many programming languages it is, practically speaking, a one-man show.
Not only is Shen inherently statically-typed (it's not annotations over a dynamic core language) but the type system is based on sequent calculus and is expressive and powerful. It would be nice to see what a Nim-sized community could do with it over five to ten years.
Shen looks interesting, but I can't figure out how to install it. I've entered a Common Lisp REPL in the downloaded directory and entered (load "install.lsp"), which printed out a lot of stuff. Now what?
Because Common Lisp is dynamic, you can redefine adder with more information for the compiler to optimize with (such as types or ability to freely inline). The compiler is part of the runtime (which really ups the programmability of the language too, you can teach SBCL's compiler new optimization rules to output even better assembly). So if you Know What You're Doing, you can get rid of the generic add by declaring the expected argument and return types (e.g. a fixnum) and redefining the function:
Note SBCL still has some safety and debugging bits in there. You can tell it you Really Know What You're Doing, and do so locally so you don't put the rest of your code at risk:
This is a fallacy* that is often repeated. It's like saying that nobody can work together on a Python project because everybody will create their own incompatible pet decorators.
Any reasonably run project will have standards and code review.
I have not, once, in my decade+ career writing Common Lisp professionally, run into a single instance where my peer programmers on a project have painted one another into silly corners due to unfettered use of macros of any ilk. It doesn't happen, because (1) code comes from some manner of requirements and (2) PRs are reviewed. Macros that don't make solutions easier to read and write for the team, not just a programmer are patently rejected.
* I feel this fallacy and others frequently come from people who hear about a relatively obscure feature or superpower of a language and—having never used it—then try to imagine pessimistic or pathological uses of said feature. Unfortunately, the speculation is usually flat-out wrong, and not even adjacent to some truth.
My guess would be that people that come from C++, where meta programming is insanely complex and hard to follow, would guess that a language that encourages meta programming would often lead to complex and hard to follow solutions. A DSL that looks simple on usage but then looking into how it works is 10 nested complex compile time macros.
I had a tiny habit of making cryptic names and structures that only I would perceive as good. But even then, with time I balanced things out.. I understand the value of other's perspective. And most books talk about it right away. People seeking advanced PLT features are not in it for foolish tricks.
A lot of us get burnt by inheriting older codebases where we have to work with code by a dev(s) who’s no longer there who got over enthusiastic with a powerful feature. What’s funny is you can usually trace that code back to a language adding some new improvement and it taking some time before people figured it out. Lisp seems to kind of avoid all that by just making everything available and not having a lot of language changes.
This was my experience with Ruby. The language makes it easy to create DSLs, so lots of libraries are implemented that way. To work on a large project you actually need to learn dozens of different languages. No thanks.
Over the last couple years a way I've come to view software is that a sufficiently complex library, class, api, or even just file IS a DSL. I don't see a useful clear line between "code" and "dsl" in the end, certainly not one based on whether you still use strictly the syntax of the base language.
I don't necessarily like learning a bunch of DSLs either, but I'm not sure the ruby situation is that different from what is always the case, just more honest about it. I've been trying to pay attention to the problems I solve and noticing early when they benefit from being explicitly defined as a little language and it's been quite successful in a few cases.
It reminds me of something a mentor pointed out early in my career that has remained a valuable insight: a whole lot of real world logic is implemented with state machines, but the only reliable and maintainable ones are where the author knew that from the beginning. Often we're using a known pattern to solve a problem without recognizing and naming it, so missing opportunities to apply hard-won knowledge and tools from elsewhere.
> Over the last couple years a way I've come to view software is that a sufficiently complex library, class, api, or even just file IS a DSL.
Yes. A macro gives a certain programming feature a name and a syntax.
But with a function we also need to give it a name and we need to define the arguments. In the case of function arguments, we also need to know what the input and output of those function needs to be.
For example in Lisp there is a macro WITH-OPEN-FILE, which opens a file, executes code using the input stream and then closes the file. It also ensures that the file is also closed in case of an error.
(with-open-file (stream filename)
(read stream))
We now may have two other ways to deal with that often needed functionality: write the code or use a function which does the same (here in this example this is possible, but often a macro does something which a function can't).
First we write the code. We may pull this from a snippet library. We have to remember what the primitives are and what they do.
Or we write a function. This gets a name CALL-WITH-OPEN-FILE, a list of arguments and in case of the function it needs to now what the input and outputs are: a stream is the input and the result is arbitrary, but will be returned by CALL-WITH-OPEN-FILE.
This used to be the case, but once Ruby stopped being the hot language, the examples of "just because you can doesn't mean you should" went to JS. Rake and RSpec format are the only two DSLs that get any major use anymore, and most other spec tools in other languages use similar describe/it DSLs to RSpec.
Lisp advocacy (for the last few decades) is always about how you can write anything in 10 lines of macros, but I never see anyone discuss security or whether the result actually does what you meant it to do.
What aspects of security are you seeking to understand vis-a-vis macros?
Macros are "just" functions whose inputs are S-expressions and outputs are S-expressions, and they are run at compile-time and not run-time. The semantics of the execution of a macro corresponds more or less to the semantics of Lisp itself, which, in a simplified view, can be completely understood in a first course in computer science (as in SICP).
Any statements about correctness of a macro will derive from statements about correctness of code more broadly, for which there's a lot of literature and is an ongoing active and popular field of research.
Any statements about the security implications of a macro will derive from (1) statements about the security of a compiler more generally and (2) statements about code more broadly, both of which have lots of literature, and both of which have active research communities.
I don't know if there are many useful things to frame around security and correctness about macros specifically (without resorting to the above derivations), which is probably why there's no literature that I know of about it.
I said security of Lisp, not of macros (or maybe I just meant to). Nb “my implementation is memory safe” as a response just means you don’t even know what the problems are - the world’s most popular Scheme implementation, JavaScript, has security issues all the time.
Any arbitrary implementation. Do specifications have security issues?
(The implementation is usually the one with the security issues, not the programs you run on it. Of course, either one’s possible if it crosses a trust boundary.)
Open-source projects like Steel Bank Common Lisp (SBCL) and commercial projects like Allegro Common Lisp alike respond to bugs reports of all manners, including security issues. I can't speak for Google, but looking at their commit history on SBCL, you can see various security-improving enhancements through the years, especially around the runtime written in C.
I don't think anybody has done a formal study or written some kind of report on the security of said implementations, however.
A gentleman named Paul Dietz implemented something called the "ANSI compliance test suite" [1], which is an implementation-agnostic battery of tests that examines how well an implementation adheres to the ANSI specification of Common Lisp. Dietz is also known for his prolific fuzz and randomized testing of Common Lisp implementations. He's been a source of obscure bug reports for years, and those bugs are always promptly fixed.
Thanks, that is informative, although now I’m confused why Google cares about SBCL. That seems like a Yegge-only thing.
I did think of a specification-level security issue - bignums reduce math errors but mean operations are O(N), which puts you at great risk of DoS issues and sidechannel leaks in security-sensitive math.
And of course Lisp’s features like macros tend to be powerful, and security has a “rule of least power” meaning those are harder to verify if you can’t statically analyze them.
Sure, the Common Lisp standard specifies a default syntax mode, where the s-expression reader (-> the function READ) can execute arbitrary code at read time.
I see this claim often. I'm not seeing how JavaScript is anything like Scheme. Scheme still ingests s-expressions and you can build compile time macros and manipulate those s-expressions at compile time before you get to runtime. What is the equivalent in JavaScript?
JavaScript is literally a Scheme “implementation”, as in it started as Scheme in a browser but they removed the s-exprs (the interface) and kept the implementation.
It does still have the other similarities; it has closures and a prototype based object system. In the 90s we only had boring Blub languages, so most of the influence that allowed such powerful concepts came from Lisp. (Though I think the prototypes are from Self.)
You say it yourself, js is part scheme part self, and the algol { clothing } shift it a large step away from scheme since the functional aspects became culturally secondary for more than a decade. The imperative aspects are the one causing bugs IMO.
Well remember “functional vs imperative” isn’t a real distinction - Lisp is imperative. That showed up elsewhere where the “pure functional” languages like Haskell are the not-imperative ones.
JS has other differences on the “correctness” spectrum, like all numbers are doubles instead of bignums. But even if that’s “incorrect”, just imagine how much more memory your Chrome tabs would use if you let people use bignums everywhere.
Scheme pushed for functional much more than early lisp (and Id love for early lispers to pitch in about the imperative traits of lisp, it's a bit fuzzy if it was a desired trait or a commercially necessary addition facing fortan).
Bye, why would a fully numeric tower js only use the least machine friendly format everywhere ? Did I miss something ?
> JavaScript is literally a Scheme “implementation”, as in it started as Scheme in a browser but they removed the s-exprs (the interface) and kept the implementation.
do you have any link to a source for this claim?
AFAIK JavaScript started as a language called Mocha, which was hacked together in a few days: Java syntax, Scheme functions + Self object system + plus a bunch of other things.
At that time: no symbols, no s-expressions, no cons cells, no lists, no macros, no tail call optimization, ...
Does Lisp do anything more than other similar languages to help you avoid mistakes in meaning? I guess not. But it doesn’t claim to, as you say, so what’s the problem?
Lisp advocates don’t talk about something Lisp doesn’t claim to do and most other languages don’t do. Why’s that surprising?
Well, the unique bit is images and homoiconic metaprogramming, not just metaprogramming. You can get that unsafely (code generation scripts) or safely (program extraction from a proof language) a lot of other places.
But the main argument is that Lisp is hyperproductive if you’re an individual brain genius who’s memorized every emacs key combo. And my point is that it doesn’t matter if you can produce something fast if you can’t know if that thing actually works.
This is the traditional “static vs dynamic typing” argument but you know, a lot of new languages have new kinds of safety features going beyond “static typing”.
And of course Lisp does make unusual decisions affecting safety. Making all numbers bignums means math is more likely to be correct since it won’t overflow, but untrusted user input can pretty easily DoS you.
Macros do reduce the surface area for mistakes, by removing redundant expressions that can be typoed. Say, you cannot forget to close a file if you use WITH-OPEN-FILE, but you can if you call OPEN and CLOSE yourself. There are other ways to not leak resources, but macros are just another way to not repeat oneself.
> But the main argument is that Lisp is hyperproductive if you’re an individual brain genius who’s memorized every emacs key combo.
I certainly haven't, though I suspect this is a bit of an exaggeration.
And in my little experience, homoiconicity also reduce the surface since you deal with sexp/ast in the same world as you code, there's no external dsl/lib to master.
>> it doesn’t matter if you can produce something fast if you can’t know if that thing actually works
In reality things that are hopelessly broken and released quickly win market share all the time which is in contradiction to your statement.
The old Tony Hoare quote seems apt: "There are two methods in software design. One is to make the program so simple, there are obviously no errors. The other is to make it so complicated, there are no obvious errors."
Above the 10loc macro it's also identifying larger scale structure / needs in your logic but make it as invisible as possible in your code, as opposed to verbose indirections. And I believe there are 40+ years of people ensuring macros do what they're supposed to, and discovering all the subtle failures, in many dimensions. You can have OnLisp semi hygienic raw macros, scheme hygienic macro systems, Matthew Flatt siloed macro systems.. same goes for other metaprogrammable languages. I guess same goes for security.
Smaller programs are easier to understand and often you can be sure that they are correct just by looking at them.
Macros allows you to decompose and simplify problems in a way that is impossible with functions. Macros are especially good at removing boilerplate and writing syntactic glue code.
Since macros are programs making sure that they do what you meant them to do you can use similar techniques as you use for other programs: making them small and obvious, testing.
The first question when building such a beast, or using one, is whether your mutations should be hygienic or not.
And thereafter you can consider if you're seeking a powerfully modifiable reader, whether through macros or other, or if you're looking to modify the executor.
The irony of the 2 + 3 = 5 example is that it shows how domain-specific languages work best with iconic forms. Paralinguistic symbols as constituency tends toward mismatched metaphors and visual sameness. However, crafted carefully, such tools represent the best available to tame complexity.
There is a level of metaprogramming even more fundamental than Lisp macros - the ancient and maligned fexpr [0]. They are almost as if you took lambda and just didn't evaluate the args when called by default. In fact, this is what Kernel [1] elegantly showed, bringing along with it the pure notion of a vau calculus, which can help restore optimization techniques who's absence resulted in the original defamation of the fexpr.
I don't know either, but in English one definition/sense of para is "beyond", so it is "beyond language symbols", which shapes up to be something in the sense of a sort of programming paradigm where everything is subject to redefinition, particularly on the fly.
I think the main difference is that macros statically evaluate their inputs ie. at compile-time. fexprs can selectively eval their inputs at runtime ie. dynamically.
This has given the impression of slowness and difficulty to grok with fexprs, as dynamic eval foregoes certain optimisations, and scoping the fexpr bindings is hard (variable capture is a problem).
$ /usr/local/bin/sbcl
This is SBCL 2.2.1, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* *evaluator-mode*
:COMPILE
* (setf *evaluator-mode* :interpret)
:INTERPRET
* (defmacro my-if (test a b)
(print "Hi, I'm the executing MY-IF macro")
`(if ,test ,a , b))
MY-IF
* (defun foo (a)
(my-if (> a 10) 'big 'small))
FOO
* (foo 11)
(foo 11)
"Hi, I'm the executing MY-IF macro"
BIG
Note that not the generated code ran twice (in LispWorks twice, in the specific SBCL interpreter once), just the code generation. In Common Lisp it's unspecified how often the code generation runs.
> This has given the impression of slowness and difficulty to grok with fexprs, as dynamic eval foregoes certain optimisations, and scoping the fexpr bindings is hard (variable capture is a problem).
And yet, Javascript, possibly the most widespread language on the planet, does exactly this and has been optimized for speed to hell and back several times.
The problem wasn't unsolvable. The problem was that there wasn't enough money behind solving it.
Consider this answer from lispm as to what fexprs can do that macros cannot, namely being first-class objects in the language that can be stored and passed as values, just as a number or a function can be.
They have the advantage of making macro writing a good deal simpler, by providing a higher-level control over metaprogramming. The effect is almost like CLOS-style classes in comparison to C++ classes - a more complete metaphor (metaprogramming/OOP) is more difficult to implement and optimize, but makes up for in improved programmability.
Of course this is all highly subjective. Perhaps fexpr was just a weird language construct that introduced more problems than it solved. I think it is an unexplored avenue that was brought to light again by Kernel. You really need fexprs to get a true programmable programming language in the sense their are no exceptions. Maybe the cost can become reasonable.
I mean I'm asking about concrete differences in how they fundamentally work, since I'm not familiar with fexprs in the first place, and (as someone who doesn't use Lisp) not very familiar with macros either. I'm not asking about larger consequences like the sort you describe in this answer.
I think you will be more successful communicating your ideas if you use less obscure language. Assuming of course that you are actually trying to communicate ideas and not just trying to virtue signal intelligence or education by being intentionally obfuscated.
Why not a single example of actual macro definitions?
The reader is only shown what they're replacing and how they might be used.
> When I was still researching this fabled obscure language called Lisp one thing people kept saying about it is that “Lisp is a programmable programming language”, but I could never figure out what they meant by that.
I think the usual reservation is that once you start showing how to write macros, it only follows with more questions. What's the syntax? What is quasiquoting? What is an unquote? Are arguments evaluated? Why not? What's code and what's data? How is it that a singly linked list data structure has anything to do with code that's executed?
...at which point it's almost more economical to just read a book about Lisp.
I think showing what instead of how can be instructive and motivating for some readers, but it definitely doesn't satisfy all of them.
I think this happens with a few interesting programming concepts. Haskell has its monads (person learns Haskell, gets excited when they start to grok monads, write a blog post explaining it in terms that make sense to them) and Lisp has macros (person learns Lisp, gets excited by macros, writes a blog post about it in a way that makes sense to them). It's sort of a rite of passage, only reason I don't have a monad blog post is because I made it too long so it sat for ages in my drafts and I lost the motivation to complete it :)
My Bad. I have special nlambda, which does not evaluate anything at any stage. It was originally for testing macros without macroexpand, which expanded too much.
I have also nslambda, which take only one argument. I think the macro in some ancient lisp was also like that. My macroes are called mlambda because of this confusion.
Sometimes you're writing code, and you want to be able to express an idea that your language does not elegantly support. But you know how to express the idea, but it takes a lot of boilerplate, or is just a massive nuisance to write.
Metaprogramming allows you to write a function that generates the boilerplate, and transparently use that function as if it's a brand new keyword in your programming language. That function in metaprogramming circles is called a "macro", not to be confused with C macros.
Stated another way: a macro = a function that takes as input the code you want to write, and provides as output code that your programming language can actually execute. (It's like a super miniature compiler.)
For example, let's pretend C is metaprogrammable.
Perhaps I want a new keyword "pfor" which is like "for" but executes the loop in parallel.
int x[20];
pfor(i = 0; i < 10; i++) {
x[2*i] = expensive(2*i);
x[2*i+1] = expensive(2*i+1);
}
Here, we assume "expensive" is some really expensive function that maybe might take several seconds to execute, which is why we want to parallelize it.
This "pfor" is just a sort is pseudo code. In reality, you know how we would actually have to write this: make a loop in which we spawn 10 threads, set i to be a thread local variable in each, call a callback function in each thread
and join the threads at the end. This would have the equivalent effect as our desired "pfor" keyword.
In a metaprogramming language, we can add this keyword by just telling the compiler what code to generate any time it sees "pfor". We could write something like
The triple ``` starts a template, and the $ escapes a template. We have to generate a fresh symbol for the callback name so that they don't collide when we use "pfor" multiple times.
Now every time we use "pfor", the compiler will look at our homemade code generator function and write out the code for us.
So in effect by telling the compiler how "pfor" can be implemented in more basic code, we've added "pfor" to the language.
This isn't a separate tool however! It's not a pre-processor or anything like that. It's built in to the language (hence why it's a programming language paradigm).
Yes. Macros are evaluated at compile-time. You can put arbitrary compile-time computation in a macro. For instance, you might want to initialize a look-up table at compile time. (Have you ever seen C code where it has a header file with a giant array of pre-computed constants, say, as a lookup table for values of sine? You can just generate that with a macro.)
Since macros in effect receive structured source code (basically an AST) as input, you can also do analysis of that code. For example, a macro to simplify algebraic expressions at compile-time.
Abstractions based on high-order functions also tend to allocate closures at run-time, so in practice you frequently get a performance penalty for their use.
High-order functions remove the need to use macros in some cases, but definitely don't replace all of them.
Not original questioner, but to rephrase the question slightly:
Is there anything you should do with macros that you can’t do with similar language features like higher order functions?
My concern with macros is that they are completely open-ended, so you can really do anything - and if you can do anything, then the language is not usefully defining a strict enough semantic structure. As an example, see the 'hygiene problem'.
Without rat-holing on "should," here's something I had to do the other day:
I am providing language-level APIs for a complex web service, one in Python and one in Common Lisp. The service has dozens of operations, all of which are wrapped around a REST API that takes query strings and headers in its inputs, and produces very different outputs. Model for this is something like S3.
In most languages, I have two choices: I can push the complexity to the users, making them plug the query string values into a big map or option object; or I can write dozens of functions, most of which are boilerplate.
In Python I wrote dozens of functions (I obviously reused a set of central ones, but there's lots of scaffolding to do, e.g., translation of arguments from user types to strings appropriate HTTP headers). In Lisp I wrote a macro that operates on a set of query string and header inputs, and then defines a function appropriate for the user-level API. When I fix bugs in the macro, I automatically fix all of the functions.
The trade-off is that the macro is itself a function that's operating at two different semantic levels (pre- and post-compilation), and it might be challenging to read for newcomers to Lisps. But this let me hide all the nasty API complexity from my users while writing substantially fewer lines of code. New additions are a single defmacro line.
A Lisp macro sees the actual source it encloses and this source does not need to adhere to any pre-defined syntax.
For example: we may want a macro which outputs the source it encloses, before execution:
CL-USER 12 > (defmacro my-loop (&body things)
`(progn
(pprint (list :doing-loop-with ',things))
(loop ,@things)))
MY-LOOP
CL-USER 13 > (my-loop for i from 1 upto 3 sum
(my-loop for j upto 3
sum (* i j)))
(:DOING-LOOP-WITH
(FOR I FROM 1 UPTO 3 SUM (MY-LOOP FOR J UPTO 3 SUM (* I J))))
(:DOING-LOOP-WITH (FOR J UPTO 3 SUM (* I J)))
(:DOING-LOOP-WITH (FOR J UPTO 3 SUM (* I J)))
(:DOING-LOOP-WITH (FOR J UPTO 3 SUM (* I J)))
36
A normal function can't do that, since it does not get source as an argument, but just sees the arguments values.
Since a macro sees source, it can not only print it, but also simplify, transform, mutate, etc. the source.
You might want to look at Template Haskell, which is used for macros in Haskell (a language which already makes heavy use of higher-order functions).
One nice thing I've used macros for is reading and parsing files at compile-time, to bake-in data into programs. That way, the compiled programs just process stdio, without having to care about file paths, permission errors, parse errors, etc. (those all become compile errors)
Now I feel a bit pressure parenting wise, to push my kids, so they will be able to understand all of this, by the age of 5.
So I am not sure, I will succeed at it, but I am saving your comment, to try when they are ready.
10 years might be doable. Probably depends how many more times they hit their head..
(And whether I can spark enthusiasm in them, but robots are fun..)
It can mean a few different things, in this case it's about programs that manipulate the program source and produce new programs (or program segments). Think of a code generator that can take a DSL and turn it into some valid (for execution) code:
for i in 1 to 10 do printf("%d\n", i)
Becomes (through a preprocessor in this hypothetical system) valid C as:
for(int i = 1; i <= 10; i++) { printf("%d\n", i); }
Now that's not too interesting, but it is a bit (subjective) nicer looping syntax than C's straight syntax. In the Lisp case, macros allow you to take an arbitrary set of values (the parameters to the macro) and produce some other value or values in their place. This could be just executing the code to get a kind of compile time evaluation:
Or it could be more transformative, like Lisp's loop macro that lets you write an Algol-ish loop instead of a lisp-y loop (or using recursion or other approaches):
(loop for i from 1 to 10 do (print i))
Which gets transformed into some other valid Lisp code (not on my laptop so not in a position to actually show the result).
David Moon's "Moon's Mark Down" is actually a pretty neat case of Lisp meta programming. He defines a macro defdot which will take a name for a particular directive for his markdown syntax and generate all the necessary functions so that things "just work". And when parsing, when a . is found at the start of a line, the correct handler is calculated and called. To add a new directive, you add a new defdot and that macro will create the relevant functions and register the new handler.
Metaprogramming is "programming programming". Consider "programming" as writing a series of steps for a computer to follow. Metaprogramming is writing a series of steps for a computer to follow to write a series of steps for a computer to follow.
When working with LISP, metaprogramming is just a normal way to program things that's built into the language. Rather than writing everything as a linear program as you would in conventional languages, you can write a program that "writes itself" (or modifies parts of itself).
While you could meta-program in many languages (using code generators and whatnot), in LISP this is easy because the programs themselves are written in a way that makes them easy to be manipulated by other parts of the program. This is why you'll see so many parentheses when looking at LISP code. The simplicity of the syntax is also what makes it so powerful.
Metaprogramming is a technique that often results in much shorter programs which can make single programmers very effective. The downside is that the higher level of abstraction (leaving ELI5 zone here, sorry) can make the workings of LISP programs quite opaque to newcomers as things seem a lot like magic. Therefore preserving the metaprogramming advantages in larger teams requires _especially_ good documentation and attention to design.
Short answer: Yes, but it's preferred to do something a little easier.
In Common Lisp, it's much easier if you prefix. It would totally be possible to do something like
[A x y]
to be an array accessor at indexes x and y by defining reader macros for the characters '[' and ']'. And then one could define the reader macro for the character 'ᵀ' so that
ᵀA
expands to the transpose of A. Then you could write
[ᵀA x y].
If you didn't like this prefix thing, you could instead be more elaborate with '[' and parse something more complicated with postfix notation.
If you're interested in something lile this, then check out the Smalltalk family of languages. They're similar to Lisps in the sense that the environment is completely programmable (metaprogramming is just regular programming); however, whereas Lisp uses source code to create new forms and perform metaprogramming, in Smalltalk there is no source code. In Smalltalk-based languages, you perform metaprogramming by acting on live objects (the "source code" is just an in-transit format for the true form of objects).
Please explain why you think that might be "a good thing". (Assuming you do.)
Please explain what you think are the differences between "source code" and "an in-transit format" that would allow us to distinguish one from the other.
It was not me who wrote that so I do not know his meaing.
> Are there method definitions outside the context of a class?
As I said above, I was talking about the basic compilation unit and it is simplified perspective.
> GNU Smalltalk?
Well, that's an exception :-) But I must note that not a nice one. About 15 years ago, I read an article about a novice evaluating Smalltalk based on experiences with GNU Smalltalk only, and he was complaining about how strange and crazy the grammar is with all that exclamation marks.
> So how can you say some other aspect is more important :-)
My opinions only :-)
> (Please provide some explanation of what you think is important about " method is the basic unit".)
I think it has many interesting consequences. The methods have only a few explicit dependencies required for compilation; they are not dependent on each other, they may be loaded and presented in any order, and the compilation is very fast. As I said, it may be comparable to having every small method in a single file. With such a setup, the code editor itself starts to lose its importance - everything on the level of notepad basically does its job. You need to focus more on the processing of the code as code objects graph. Not as a text file to edit, which may be an important shift in thinking.
Incremental compilation is not related to that directly. And yes, you still need a way how to define classes which Smalltalk file-out format does do as simple do-its, which is not a fortunate solution for statical analysis of the given file.
Do you mean to say — the more important aspect of the Smalltalk approach is incremental compilation ?
July 10, 1989 "ParcPlace Systems Inc, of Mountain View California, has launched its Objectworks software development system for AT&T’s C++ Release 2.0: the integrated set of object-oriented tools include an incremental compiler and linker, source level debugger and source code browsing, and will be available in late August for Sun Microsystems’s Sun-3 workstation, selling for $2,500."
> In your mind you can translate it back to English as “The numeric value of the sum of the numbers 2 and 3 is the same as the numeric value of the number 5”
You can, but there is a simpler and more natural one to one mapping: 2[two] +[plus] 3[three] =[equals] 5[five]. 'Two plus three equals five'.
DSL is how we're going to scale our B2B product, which relies on heavy customization per client. We have arrived at a very clear semantic model that both we and our customers can understand. So, we figure we should just get out of the way if possible.
Currently, looking at doing a fluent-style interface in C# (same as host language) which is inspected/compiled at runtime using Roslyn.
I use code generation all the time to automate programming in C++. I define what I want (the outcome) and the code generator generates the code needed to do it. It works in any language not just Lisp. It IMHO is better to use Lisp style macros. Because the generated code is straightforward C++, making it easy to read for other programmers, without them having to first learn all the macros.
132 comments
[ 2.7 ms ] story [ 204 ms ] threadThe ‘normal’ way to redefine words in forth is a bit of cheating, though. Redefining a word merely introduces a word with a name that shadows that of the old, still existing word.
Words created before the definition still call the old implementation, and, depending on the implementation (a phrase that, I think applies to about any statement about forth) you can still find it when you walk the dictionary and call it.
Smalltalk is missing too, and that’s a bigger omission. When you redefine a function there, it starts getting used everywhere.
I wonder what a blog post written about this 50 years from now will look like. Would it still mention C and lisp? Would readers of said article still mention forth and smalltalk as missing? Also, what reasonably successful languages from the ‘60s/‘70s already are gone from the collective mind?
Being used today to write very popular OSes, I see C still being mentioned, but by then lisp might have fallen into the abyss of history (except for, probably, a few computer historians)
I don't think so, because being a lisp is a trait of a language rather than a specific language. Scheme is a lisp. Racket is a lisp. Clojure is a lisp.
People keep making new ones because lisp continues to be a useful idea. It keeps not quite going mainstream, the reasons for which have been subject to much speculation. It's most likely that there will be some semi-popular lisp in 50 years.
MicroEngenho: https://www.youtube.com/watch?v=Mu9CsdGHybw
EDIT: YouTube link
It compiles to native, highly optimized machine code.
* https://coalton-lang.github.io/
Common lisp is compiled to optimised native code.
How do you optimize code without knowing the types of variables?
> How do you optimize code without knowing the types of variables?
Very carefully.
For a large amount of programs (basically all "classical" programs like you'd read about in the past rather than modern apps) you have a fairly clear path through the code for their data, this means that you can optimize that path.
This doesn't mean the performance will be as good as a statically typed language but keep in mind 90% of compiler optimization is basically useless on many programs vs. the 10% that makes the important bits run fast.
http://clhs.lisp.se/Body/d_type.htm
I bet Allegro or Lisp Works don't have any issues generating better code than Nim.
There is also https://en.wikipedia.org/wiki/Stalin_(Scheme_implementation) which was doing lifetime analysis waaay back in the mid 90s, but I don't think Suskind had a declare system at all in Stalin. So, places where inference did not work were probably just "slow to be fixed by users tweaking code" to make inference better. I think he was mostly his own main user.
Both Coalton & Carp do look interesting. There can be complaints with how "prefix notation" scales to large expressions. Cython is a gradually typed Python, but late noughties ideas in metaprogramming/macros/etc. never quite took off.
There can be an undesirable effect of gradual/optional static types that code starts off "lazy" and then can require surprisingly large efforts to tighten down, especially in team settings. Even with profiling/90-10 heuristic rules, a later dev trying to optimize earlier lazy work can still be 10X, especially if type dynamism was really leaned upon. So, at least having an easy switch to force early/in-development static typing on original authors is likely a good thing (or at least strongly encourage with warnings...).
Probably doesn't cover the 'optimized native code' part but it obviously could, like many programming languages it is, practically speaking, a one-man show.
Not only is Shen inherently statically-typed (it's not annotations over a dynamic core language) but the type system is based on sequent calculus and is expressive and powerful. It would be nice to see what a Nim-sized community could do with it over five to ten years.
I found the best way to learn shen was the book of shen: https://shenlanguage.org/TBOS/tbos_255.html (dynamic type checking example here)
Unfortunately, once you have more than 1 programmer involved in the project, you end up with slightly different versions the same concepts.
Any reasonably run project will have standards and code review.
I have not, once, in my decade+ career writing Common Lisp professionally, run into a single instance where my peer programmers on a project have painted one another into silly corners due to unfettered use of macros of any ilk. It doesn't happen, because (1) code comes from some manner of requirements and (2) PRs are reviewed. Macros that don't make solutions easier to read and write for the team, not just a programmer are patently rejected.
* I feel this fallacy and others frequently come from people who hear about a relatively obscure feature or superpower of a language and—having never used it—then try to imagine pessimistic or pathological uses of said feature. Unfortunately, the speculation is usually flat-out wrong, and not even adjacent to some truth.
My guess would be that people that come from C++, where meta programming is insanely complex and hard to follow, would guess that a language that encourages meta programming would often lead to complex and hard to follow solutions. A DSL that looks simple on usage but then looking into how it works is 10 nested complex compile time macros.
I don't necessarily like learning a bunch of DSLs either, but I'm not sure the ruby situation is that different from what is always the case, just more honest about it. I've been trying to pay attention to the problems I solve and noticing early when they benefit from being explicitly defined as a little language and it's been quite successful in a few cases.
It reminds me of something a mentor pointed out early in my career that has remained a valuable insight: a whole lot of real world logic is implemented with state machines, but the only reliable and maintainable ones are where the author knew that from the beginning. Often we're using a known pattern to solve a problem without recognizing and naming it, so missing opportunities to apply hard-won knowledge and tools from elsewhere.
Yes. A macro gives a certain programming feature a name and a syntax. But with a function we also need to give it a name and we need to define the arguments. In the case of function arguments, we also need to know what the input and output of those function needs to be.
For example in Lisp there is a macro WITH-OPEN-FILE, which opens a file, executes code using the input stream and then closes the file. It also ensures that the file is also closed in case of an error.
We now may have two other ways to deal with that often needed functionality: write the code or use a function which does the same (here in this example this is possible, but often a macro does something which a function can't).First we write the code. We may pull this from a snippet library. We have to remember what the primitives are and what they do.
Or we write a function. This gets a name CALL-WITH-OPEN-FILE, a list of arguments and in case of the function it needs to now what the input and outputs are: a stream is the input and the result is arbitrary, but will be returned by CALL-WITH-OPEN-FILE. For the macro we need to know:* the name of the macro
* the syntax of the macro:
* what the generated code is supposed to doFor the function we need to know:
* the name of the function
* the list of arguments and their order
* the types and semantics of the arguments: a filename and a function which takes a stream
* what the function is supposed to do
The effect is basically the same: both ways are essential leading to a DSL, where the language operators and their usage has to be learned.
Macros are "just" functions whose inputs are S-expressions and outputs are S-expressions, and they are run at compile-time and not run-time. The semantics of the execution of a macro corresponds more or less to the semantics of Lisp itself, which, in a simplified view, can be completely understood in a first course in computer science (as in SICP).
Any statements about correctness of a macro will derive from statements about correctness of code more broadly, for which there's a lot of literature and is an ongoing active and popular field of research.
Any statements about the security implications of a macro will derive from (1) statements about the security of a compiler more generally and (2) statements about code more broadly, both of which have lots of literature, and both of which have active research communities.
I don't know if there are many useful things to frame around security and correctness about macros specifically (without resorting to the above derivations), which is probably why there's no literature that I know of about it.
There is no the implementation of JavaScript. There are several, all with their own issues, security and otherwise.
(The implementation is usually the one with the security issues, not the programs you run on it. Of course, either one’s possible if it crosses a trust boundary.)
I don't think anybody has done a formal study or written some kind of report on the security of said implementations, however.
A gentleman named Paul Dietz implemented something called the "ANSI compliance test suite" [1], which is an implementation-agnostic battery of tests that examines how well an implementation adheres to the ANSI specification of Common Lisp. Dietz is also known for his prolific fuzz and randomized testing of Common Lisp implementations. He's been a source of obscure bug reports for years, and those bugs are always promptly fixed.
[1] https://ansi-test.common-lisp.dev/
I did think of a specification-level security issue - bignums reduce math errors but mean operations are O(N), which puts you at great risk of DoS issues and sidechannel leaks in security-sensitive math.
And of course Lisp’s features like macros tend to be powerful, and security has a “rule of least power” meaning those are harder to verify if you can’t statically analyze them.
Sure, the Common Lisp standard specifies a default syntax mode, where the s-expression reader (-> the function READ) can execute arbitrary code at read time.
It does still have the other similarities; it has closures and a prototype based object system. In the 90s we only had boring Blub languages, so most of the influence that allowed such powerful concepts came from Lisp. (Though I think the prototypes are from Self.)
JS has other differences on the “correctness” spectrum, like all numbers are doubles instead of bignums. But even if that’s “incorrect”, just imagine how much more memory your Chrome tabs would use if you let people use bignums everywhere.
Bye, why would a fully numeric tower js only use the least machine friendly format everywhere ? Did I miss something ?
do you have any link to a source for this claim?
AFAIK JavaScript started as a language called Mocha, which was hacked together in a few days: Java syntax, Scheme functions + Self object system + plus a bunch of other things.
At that time: no symbols, no s-expressions, no cons cells, no lists, no macros, no tail call optimization, ...
And what do you mean about doing what you mean it to? The semantics of macro expansion are well-defined.
Does Lisp do anything more than other similar languages to help you avoid mistakes in meaning? I guess not. But it doesn’t claim to, as you say, so what’s the problem?
Lisp advocates don’t talk about something Lisp doesn’t claim to do and most other languages don’t do. Why’s that surprising?
But the main argument is that Lisp is hyperproductive if you’re an individual brain genius who’s memorized every emacs key combo. And my point is that it doesn’t matter if you can produce something fast if you can’t know if that thing actually works.
This is the traditional “static vs dynamic typing” argument but you know, a lot of new languages have new kinds of safety features going beyond “static typing”.
And of course Lisp does make unusual decisions affecting safety. Making all numbers bignums means math is more likely to be correct since it won’t overflow, but untrusted user input can pretty easily DoS you.
> But the main argument is that Lisp is hyperproductive if you’re an individual brain genius who’s memorized every emacs key combo.
I certainly haven't, though I suspect this is a bit of an exaggeration.
In reality things that are hopelessly broken and released quickly win market share all the time which is in contradiction to your statement.
The old Tony Hoare quote seems apt: "There are two methods in software design. One is to make the program so simple, there are obviously no errors. The other is to make it so complicated, there are no obvious errors."
Macros allows you to decompose and simplify problems in a way that is impossible with functions. Macros are especially good at removing boilerplate and writing syntactic glue code.
Since macros are programs making sure that they do what you meant them to do you can use similar techniques as you use for other programs: making them small and obvious, testing.
And thereafter you can consider if you're seeking a powerfully modifiable reader, whether through macros or other, or if you're looking to modify the executor.
There is a level of metaprogramming even more fundamental than Lisp macros - the ancient and maligned fexpr [0]. They are almost as if you took lambda and just didn't evaluate the args when called by default. In fact, this is what Kernel [1] elegantly showed, bringing along with it the pure notion of a vau calculus, which can help restore optimization techniques who's absence resulted in the original defamation of the fexpr.
[0] https://fexpr.blogspot.com/2011/04/fexpr.html
[1] https://web.cs.wpi.edu/%7Ejshutt/kernel.html
What’s that? I googled and am still confuzled.
https://en.wikipedia.org/wiki/Metasyntactic_variable
This has given the impression of slowness and difficulty to grok with fexprs, as dynamic eval foregoes certain optimisations, and scoping the fexpr bindings is hard (variable capture is a problem).
https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unr... is a good paper on this stuff (see p41/42 for a discussion).
Another version of fexprs/vau is open composition https://www.piumarta.com/freeco11/freeco11-piumarta-oecm.pdf
I guess the main issue is handling dynamic binding cleanly. Koka https://koka-lang.github.io/koka/doc/book.html#why-handlers handles this well with static (HM) type inferrence.
Shen and Maude (https://shenlanguage.org/TBOS/tbos_255.html, https://maude.lcc.uma.es/maude321-manual-html/maude-manual.h...) are the only languages that I know that can do dynamic type checking, and thus a full handling of dynamic binding, the Shen calculus syntax is a bit clunky though.
Unless one uses an interpreter.
In this Lisp implementation, the macro even runs twice on each runtime invocations.And yet, Javascript, possibly the most widespread language on the planet, does exactly this and has been optimized for speed to hell and back several times.
The problem wasn't unsolvable. The problem was that there wasn't enough money behind solving it.
And, yet, somehow a lot of very clever people made Javascript one of the fastest languages around.
OTOH, typically Lisp code uses zillions of macros, incl. most of control flow constructs.
https://stackoverflow.com/questions/13393673/how-do-you-stor...
Of course this is all highly subjective. Perhaps fexpr was just a weird language construct that introduced more problems than it solved. I think it is an unexplored avenue that was brought to light again by Kernel. You really need fexprs to get a true programmable programming language in the sense their are no exceptions. Maybe the cost can become reasonable.
https://fexpr.blogspot.com/2011/04/fexpr.html
You mean constituents? Or operands?
"Paralinguistic"
I think you mean syncategoremata?
"Iconic forms"
?
"vau calculus"
Interesting. For the interested, see: https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unr...
> I think you mean syncategoremata?
I think this is the most HN thing I've ever read on this site. :D
The reader is only shown what they're replacing and how they might be used.
> When I was still researching this fabled obscure language called Lisp one thing people kept saying about it is that “Lisp is a programmable programming language”, but I could never figure out what they meant by that.
Somehow I feel they fell into the same trap.
...at which point it's almost more economical to just read a book about Lisp.
I think showing what instead of how can be instructive and motivating for some readers, but it definitely doesn't satisfy all of them.
In other words: if your function is able to produce (+ 1 2), same function produces 3, when used as macro.
a lambda evaluates all it's args, a macro not.
Metaprogramming allows you to write a function that generates the boilerplate, and transparently use that function as if it's a brand new keyword in your programming language. That function in metaprogramming circles is called a "macro", not to be confused with C macros.
Stated another way: a macro = a function that takes as input the code you want to write, and provides as output code that your programming language can actually execute. (It's like a super miniature compiler.)
For example, let's pretend C is metaprogrammable.
Perhaps I want a new keyword "pfor" which is like "for" but executes the loop in parallel.
Here, we assume "expensive" is some really expensive function that maybe might take several seconds to execute, which is why we want to parallelize it.This "pfor" is just a sort is pseudo code. In reality, you know how we would actually have to write this: make a loop in which we spawn 10 threads, set i to be a thread local variable in each, call a callback function in each thread
and join the threads at the end. This would have the equivalent effect as our desired "pfor" keyword.In a metaprogramming language, we can add this keyword by just telling the compiler what code to generate any time it sees "pfor". We could write something like
The triple ``` starts a template, and the $ escapes a template. We have to generate a fresh symbol for the callback name so that they don't collide when we use "pfor" multiple times.Now every time we use "pfor", the compiler will look at our homemade code generator function and write out the code for us.
So in effect by telling the compiler how "pfor" can be implemented in more basic code, we've added "pfor" to the language.
This isn't a separate tool however! It's not a pre-processor or anything like that. It's built in to the language (hence why it's a programming language paradigm).
Since macros in effect receive structured source code (basically an AST) as input, you can also do analysis of that code. For example, a macro to simplify algebraic expressions at compile-time.
Abstractions based on high-order functions also tend to allocate closures at run-time, so in practice you frequently get a performance penalty for their use.
High-order functions remove the need to use macros in some cases, but definitely don't replace all of them.
Is there anything you should do with macros that you can’t do with similar language features like higher order functions?
My concern with macros is that they are completely open-ended, so you can really do anything - and if you can do anything, then the language is not usefully defining a strict enough semantic structure. As an example, see the 'hygiene problem'.
I am providing language-level APIs for a complex web service, one in Python and one in Common Lisp. The service has dozens of operations, all of which are wrapped around a REST API that takes query strings and headers in its inputs, and produces very different outputs. Model for this is something like S3.
In most languages, I have two choices: I can push the complexity to the users, making them plug the query string values into a big map or option object; or I can write dozens of functions, most of which are boilerplate.
In Python I wrote dozens of functions (I obviously reused a set of central ones, but there's lots of scaffolding to do, e.g., translation of arguments from user types to strings appropriate HTTP headers). In Lisp I wrote a macro that operates on a set of query string and header inputs, and then defines a function appropriate for the user-level API. When I fix bugs in the macro, I automatically fix all of the functions.
The trade-off is that the macro is itself a function that's operating at two different semantic levels (pre- and post-compilation), and it might be challenging to read for newcomers to Lisps. But this let me hide all the nasty API complexity from my users while writing substantially fewer lines of code. New additions are a single defmacro line.
For example: we may want a macro which outputs the source it encloses, before execution:
A normal function can't do that, since it does not get source as an argument, but just sees the arguments values.Since a macro sees source, it can not only print it, but also simplify, transform, mutate, etc. the source.
One nice thing I've used macros for is reading and parsing files at compile-time, to bake-in data into programs. That way, the compiled programs just process stdio, without having to care about file paths, permission errors, parse errors, etc. (those all become compile errors)
So I am not sure, I will succeed at it, but I am saving your comment, to try when they are ready. 10 years might be doable. Probably depends how many more times they hit their head..
(And whether I can spark enthusiasm in them, but robots are fun..)
David Moon's "Moon's Mark Down" is actually a pretty neat case of Lisp meta programming. He defines a macro defdot which will take a name for a particular directive for his markdown syntax and generate all the necessary functions so that things "just work". And when parsing, when a . is found at the start of a line, the correct handler is calculated and called. To add a new directive, you add a new defdot and that macro will create the relevant functions and register the new handler.
http://users.rcn.com/david-moon/MMD/HTML/index.html
http://users.rcn.com/david-moon/MMD/MMD.lisp
Metaprogramming is "programming programming". Consider "programming" as writing a series of steps for a computer to follow. Metaprogramming is writing a series of steps for a computer to follow to write a series of steps for a computer to follow.
When working with LISP, metaprogramming is just a normal way to program things that's built into the language. Rather than writing everything as a linear program as you would in conventional languages, you can write a program that "writes itself" (or modifies parts of itself).
While you could meta-program in many languages (using code generators and whatnot), in LISP this is easy because the programs themselves are written in a way that makes them easy to be manipulated by other parts of the program. This is why you'll see so many parentheses when looking at LISP code. The simplicity of the syntax is also what makes it so powerful.
Metaprogramming is a technique that often results in much shorter programs which can make single programmers very effective. The downside is that the higher level of abstraction (leaving ELI5 zone here, sorry) can make the workings of LISP programs quite opaque to newcomers as things seem a lot like magic. Therefore preserving the metaprogramming advantages in larger teams requires _especially_ good documentation and attention to design.
example (A' 2 3) to access row 2 and col 3 of a transposed matrix A
In Common Lisp, it's much easier if you prefix. It would totally be possible to do something like
to be an array accessor at indexes x and y by defining reader macros for the characters '[' and ']'. And then one could define the reader macro for the character 'ᵀ' so that expands to the transpose of A. Then you could write If you didn't like this prefix thing, you could instead be more elaborate with '[' and parse something more complicated with postfix notation.https://en.wikipedia.org/wiki/The_Art_of_the_Metaobject_Prot...
Please explain why you think that might be "a good thing". (Assuming you do.)
Please explain what you think are the differences between "source code" and "an in-transit format" that would allow us to distinguish one from the other.
The difference is that the grammar of the transit file is not the language's grammar.
Good? Bad? Unimportant?
> … method is the basic unit…
Are there method definitions outside the context of a class?
> … not the language's grammar.
GNU Smalltalk?
It was not me who wrote that so I do not know his meaing.
> Are there method definitions outside the context of a class?
As I said above, I was talking about the basic compilation unit and it is simplified perspective.
> GNU Smalltalk?
Well, that's an exception :-) But I must note that not a nice one. About 15 years ago, I read an article about a novice evaluating Smalltalk based on experiences with GNU Smalltalk only, and he was complaining about how strange and crazy the grammar is with all that exclamation marks.
So how can you say some other aspect is more important :-)
(Please provide some explanation of what you think is important about " method is the basic unit".)
> … the basic compilation unit …
My opinions only :-)
> (Please provide some explanation of what you think is important about " method is the basic unit".)
I think it has many interesting consequences. The methods have only a few explicit dependencies required for compilation; they are not dependent on each other, they may be loaded and presented in any order, and the compilation is very fast. As I said, it may be comparable to having every small method in a single file. With such a setup, the code editor itself starts to lose its importance - everything on the level of notepad basically does its job. You need to focus more on the processing of the code as code objects graph. Not as a text file to edit, which may be an important shift in thinking.
Incremental compilation is not related to that directly. And yes, you still need a way how to define classes which Smalltalk file-out format does do as simple do-its, which is not a fortunate solution for statical analysis of the given file.
Unless they are — methods that change how methods are compiled.
> … loaded and presented in any order…
And which may give different results depending on order.
Isn't that what "First-Class Undefined Classes for Pharo" was about?
https://hal.archives-ouvertes.fr/hal-01585305/document
> … how to define classes which Smalltalk file-out format…
That problem isn't created by the file-out format. That problem is everything is a message send.
"Class definitions are [not] static, declarative syntactic structures."
1988 "An Overview of Modular Smalltalk"
https://dl.acm.org/doi/pdf/10.1145/62084.62095
Seems like there have been plenty of problems caused by "method is the basic unit" ;-)
July 10, 1989 "ParcPlace Systems Inc, of Mountain View California, has launched its Objectworks software development system for AT&T’s C++ Release 2.0: the integrated set of object-oriented tools include an incremental compiler and linker, source level debugger and source code browsing, and will be available in late August for Sun Microsystems’s Sun-3 workstation, selling for $2,500."
https://techmonitor.ai/technology/parcplace_launches_objectw...
Are you explaining what a Smalltalk browser is like?
https://files.mtstatic.com/site_7337/32007/0?Expires=1654646...
You can, but there is a simpler and more natural one to one mapping: 2[two] +[plus] 3[three] =[equals] 5[five]. 'Two plus three equals five'.
Currently, looking at doing a fluent-style interface in C# (same as host language) which is inspected/compiled at runtime using Roslyn.