Having played around with Clojure and Scheme for a while (but never got too serious), I always thought homoiconicity and macros were extremely cool concepts, but I never actually ran into a need for them in my everyday work.
>Now, if we agree that the ability to manipulate source code is important to us, what kind of languages are most conducive for supporting it?
This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
The remote code example given in the post is compelling, but again seems a bit niche. I don't doubt that it's sometimes useful but is it reason enough to choose the language? Are there examples of real-life (non-compilers-related) Lisp programs that show the power of homoiconicity?
Same goes with the concept of "being a guest" in the programming language. I have never wanted to change "constant" to "c". Probably I'm not imaginative enough, but this has never really been an issue for me. Perhaps it secretly has though, and some of my problems have been "being a guest" in disguise.
In my experience, macros in lisp codebases are mainly abstracting away common boilerplate patterns. Common Lisp has WITH-OPEN-FILE to open a file and make sure it closes if the stack unwinds. This is a macro based on the UNWIND-PROTECT primitive, which ensures execution of a form if the stack unwinds. Many projects will have various with-foo macros to express this pattern for arbitray things that are useful in the context of that project(though not all with- macros need UNWIND-PROTECT). Another example is custom loops that happen over and over.
Let's say I'm writing a chess engine. I regularly want to iterate over the legal moves in a position. So I might make a macro so I can write
(do-moves (mv pos) ...)
I find that because doing this is so simple, well written lisp codebases tend to be pretty easy to read. There's less of that learning curve in a new codebase of getting used to the codebase's particular boilerplate rituals, and learning to pick out the code that's interesting. In a good lisp codebase all the ritual is hidden away in self-documenting macros.
Of course this can get taken too far and then it becomes nightmarish to understand the codebase because so much stuff is in various complex macros that it's hard to tell where the shit goes down.
I'm not a JS programmer, but I didn't find that code terribly readable. And from a performance point of view, I would not want to copy the board state n times, though that's beside the point. It's a bit of a contrived example of course. But what this looks like in a real engine would be something like(no language in particular):
pseudo_legal = move_generator(pos, PSEUDOLEGAL);
while true {
mv = pseudo_legal.next();
if !mv {
break;
}
if mv == tt.excluded_move || !pos.legal(mv){
continue;
}
pos.make_move(mv);
...
pos.unmake_move();
}
The pseudolegal stuff and all that has to do with various performance considerations. This is roughly what Stockfish' move loop looks like. It looks overly verbose because abstracting away these things with functions adds runtime overhead.
With macros you can have your cake and eat it too. While it's technically true you're passing around unevaluated code, none of this is happening at runtime but at compile-time(technically macro-expansion time but that's beyond the scope of this comment). Think of it as the ability to inline pretty much anything you want.
In this particular example, there's little to none: truth be told, I don't like WITH-FOO as a good example of why macros are fun. That's because WITH-FOO macros are commonly implemented/implementable as simple wrappers around CALL-WITH-FOO functions.
For instance, if we had a CALL-WITH-OPEN-FILE function (it's not too hard to write one yourself!), the following two syntaxes - one macro, one functional - would be equivalent:
Notice that both of these accept a pathname, additional arguments passed to OPEN, and code to be called with the opened stream. The only differences are that the functional variant needs to have its code forms wrapped in an anonymous function and that the function object passed to it is replaceable (since it's just a value).
---------------
For a more serious example, try thinking of how to implement something like CL:LOOP (a complex iteration construct) without a macro system.
Sure, LOOP is a very complex macro. But my point was that most macros in real codebases are these simple boilerplate wrappers that help readability.
I don't like codebases too full of DSLs, necessarily.
A less trivial example is defining different types of subroutine. In StumpWM, a tiling WM written in common lisp, there is the concept of commands. They're functions, but they executed as a string. "command arg1 arg2". And these strings can be bound to keys. But args might be numbers, windows, frames, strings etc.
Commands are defined through a defcommand macro. It takes types! And there's a macro for defining arbitrary types and how to parse them from a string. A command is actually a function under the hood, with a bunch of boilerplate to: parse the arguments, stick the name in a hash table, call pre- and post-command hooks, set some dynamic bindings. and so on. Defcommand abstracts this away and you can just write it just like a normal Lisp function except for the types.
Well there's an old debate about macros and functions, especially when you have closures, since closures can also delay assembly of bits of logic (half the work macros do, the rest being actual syntactic processing when people do that).
You have to understand too that mainstream languages didn't have closure until very recently, so a lot of things look less obvious now.
I've been using Java since version 1.1 and I've seen features added that required a new version of the language spec to go through committee and get implemented before we got to use them where you could just add these features yourself at the top of any Lisp file and then immediately start using them. For example consider the try-with-resources [0] syntax sugar. So instead of thinking "I never actually ran into a need for them", think of new language features that you have started using: those are the kinds of things you could've added yourself.
Also look at any kind of code-gen tooling like parser generators or record specifications like Protocol Buffers as examples of what you could do within the language.
> This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
If you're talking about Haskell, you should be talking about folk who write template Haskell, which is the macro system for GHC. There are plenty of Haskell programmers who know how to write Template Haskell, and there are plenty of Haskell programmers who don't. By contrast, I don't think there's a single Lisp programmer who can't write Lisp macros.
That's homoiconicity. Once you learn Lisp, you automatically know how to write Lisp macros. Once you learn Haskell (or Ocaml or Rust), you don't automatically know how to write macros in that language (and the macro system may not even be portable across compilers).
Now, stuff that would have to be implemented in the compiler to update the language, can now be written just as a "normal" program, that adds whole new features to your programming language.
For example, the entire object system in Common Lisp was implemented as macros.
Yes, most programming tasks don't require this kind of power. But it does mean programming in Lisp it's very very rare you are going to be stuck because your programming language doesn't implement some feature you need for the task at hand.
> For example, the entire object system in Common Lisp was implemented as macros.
It wasn't. The original Object System implementation of Common Lisp is has tree layers: at the bottom layer it is object-oriented (especially the Meta-Object Protocol), then there is a functional layer and on top there are macros for the convenience of the user.
I actively want people to NOT change constant to c. I want the language to be a predictable shared base, not something I have to relearn and customize for every project.
I'm not a language designer. That's hard. That takes time and effort. And I don't do anything that can't be done in Python or Dart or whatever. Customizing the language is time not spent on the project, and batteries included stuff already has everything I need.
I think lisp is good for people who "think inside their heads", as in, they think "I want to do this, oh, I could do it this way, then I'd need this resource, let's build it".
If you think "Interactively" as in "I want to do this, what does Google tell me others are doing, oh yeah, this was made almost exactly the same use case, I'll start with this resource, now I'll adapt my design to fit it", you might not have any ideas for language tweaks to make in the first place.
I basically never code and think "I wish I could do that in this language" aside from minor syntactic sugar and library features. New abstractions and ideas don't just pop up in my mind, what the common mainstream languages have is the entirity of what programs are made of, as far as I'm concerned.
I don't entirely disagree that _changing_ the language is a bit of a no no, like changing the behavior of existing keywords and operators.
However, any program that declares a variable or new function could be said to extend the language, since, if you declare some function, well, that function is now, at least in any proper language, as much part of the language of that program, as any builtin function is.
Sure, if all you have is an empty .c file, you can say "this program is standard C", but as soon as you've declared a variable or funciton, your program becomes in a way a superset of C, it is all the standard C plus the functions, datastructures and variables that you've defined, and to extend that program, it is not enough to keep strictly to the standard language, you must also take into consideration the superset of functionality that is part of the program..
In this way, programming is much more about creating a language that speaks natively in the abstractions of the domain, and then using that language to solve specific tasks within the domain.
And so it becomes that, you're always tweaking the language, it's just the degree to which you can tweak it that is different..
> This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it?
It is also useful for anyone wanting to implement language-level features as simple libraries. Someone else brought up Nim here: it's a great example of what can be done with metaprogramming (and in a non-Lisp language) as it intentionally sticks to a small-but-extendable-core design.
There's macro-based libraries that implement the following, with all the elegance of a compiler feature: traits, interfaces, classes, typeclasses, contracts, Result types, HTML (and other) DSLs, syntax sugar for a variety of things (notably anonymous functions `=>` and Option types `?`), pattern matching (now in the compiler), method cascading, async/await, and more that I'm forgetting.
Some Lisp advocates may tell you that Lisp even does not have hygienic macros. Lisp dialects like Scheme have. Lisp usually has procedural macros, which transform source code (in the form of nested lists), not ASTs (like Nim).
That Nim has 'powerful hygienic macros' is fine, many languages have powerful macros.
As part of delivering an e-commerce solution in Scheme, I wrote a module which allowed for SQL queries to be written in s-expression syntax. You could unquote Scheme variables or expressions into the queries and it would do the right thing wrt quoting, so no connection.prepareStatement("INSERT INTO CUSTOMERS (NAME, ADDRESS) VALUES (?, ?)", a, b) type madness. Wanky, you bet. But oh so, so convenient.
It's not tied to complex DSLs and compilation. I was frustrated by the lack of f-strings in emacs lisp so I hacked a macro, now I have embedded scoped string interpolation. I can write (f "<{class} {fields}>"). Having these freedom is really not a niche, it's mentally and pragmatically important.
Most well-written Lisp code has its structure indicated primarily by the indentation. Some form of offset rule is the most commonly suggested alternative Lisp syntax. It does make the tree structure more obvious:
The left looks a lot like a typical AST for many languages, such as C, shortly after parsing. As the article points out, it's the easy access to this tree-list structure at runtime, that can be easily handled as data by the program itself, that gets Lisp programmers so excited.
Unfortunately, as with all standards, there's so many wonderful alternatives to choose from. I rather like Wisp: https://www.draketo.de/software/wisp
I wish all languages had the : operator that Wisp has. It's $ in Haskell. It simply says that what follows to the end of line is wrapped in brackets. So:
putStrLn $ somefunc $ process val = putStrLn (somefunc (process val))
They definitely didn't care about that. No language that uses ` , the most painful common character to write on most non-US keyboards put any consideration into that.
Outside of the US and a few other English-speaking countries? Not really. In Europe, US keyboards are nowhere to be found (the UK uses a different layout).
It's part of why it's always painful to buy a keyboard west of Oder for me, because Poland use ANSI layout as base, not ISO, at least as an option. After over a decade, even Microsoft finally learnt to default to "Polish (Programmers)" rather than "Polish (214)" keyboard when installing the OS (it was a common pitfall across whole Windows 9x).
Other languages in former Warsaw Pact tend to have "Programmers" layout available as well, because the national standard layouts tended to be designed for typewriters not programming and lack all kinds of characters people need - unless you go trigram route in C/C++.
For what its worth, Common Lisp at least does let you remap ` and such :)
I'm not talking about the physical ISO vs ANSI layouts. I mean the layout in software. It's not even close. Basically noone outside the US and Canada uses it. We're on HN, so obviously someone here will, but outside of this niche? Noone
> The left looks a lot like a typical AST for many languages....
Right, and this is what's meant by the oft-repeated statement "LISP has no syntax." Of course it has syntax in the sense that if you were to go in and delete one of those parens, the whole thing wouldn't compile, but the syntax it does have is 100% regular.
Lisp has quite a bit more syntax, since it has a two stage syntax. The first stage is a syntax for data, called s-expressions. The parentheses are a part of that data syntax. Examples for s-expressions:
12
1/3
rome
(rome is a city in italy)
(rome (city . italy) (capital-of italy))
(rome local-name "Roma")
and more. The s-expression syntax is extensible via reader macros.
Lisp programs then are made out of s-expressions and there is a syntax for Lisp. Additionally the Lisp syntax is extensible via macros.
Example:
(let ((a 10)
(b 10))
(declare (integer a b))
(+ a b))
Above is a LET expression for spanning a local lexical scope for variables.
The syntax for LET would be (in some kind of EBNF notation):
let ({var | (var [init-form])}*)
declaration*
form*
Who cares? Assembly is regular*, very much so. There is a single control flow construct, in conditional and unconditional flavors. There is a single language construct, of which the single control flow construct is but a special case. Code is Data and Data is Code. Everything is possible.
You don't need regularity or simplicity, you need automation. Programming Languages are tools to under-specify a problem\process\pattern\solution\structure and still get an executable that does something useful, meaning the parts that weren't specified were filled in by the machine (in any way necessary), meaning automation. In other words, "A low level programming language is that which requires attention to the irrelevant", per Alan Perlis.
In that light, lisp's syntax is awful, you're laying down the AST yourself instead of making a parser figuring it out from program text. You're paying precious attention to the extremely irrelevant, the fact that a function's body is nested under its name, the fact that a control construct wraps its code, instead of letting those trivial facts fade into the background of a nicely-designed grammar and actually focus on the code itself.
There is literally 0 need for making code-as-text reflects code-as-tree, Mathematica represents every single language construct as a nested list underneath, and yet it has an extremely rich and pleasant syntax, full of sugar and shortcuts and alternate forms. It can do this because it understands a very fundamental point that lisp syntax fans too often miss : text is irrelevant.
The string "(+ (* 4 10) 2)" is not an S expression, its the textual form of an S expression, the serialized form. You can't do anything useful with it without parsing, the actual S expression is a hierarchical object graph in memory, you can either get it from the text by quoting the text, or immediately evaluate it by entering the text directly into an interpreter (which gets the tree first). In all cases, there is no reason why quoting `while (cond) {exprs}` can't evaluate to the list (while cond (exprs)). You are already parsing the text anyway, quote is nothing but invoking the parser from within a program, so make the textual form convenient to humans while you're at it, a simplistic form benefits no one but the parser (a machine, the very thing that needs to work hard).
Lisp's big idea is just making the compiler's services available through an easy API, this has absolutely nothing to do with its crippled syntax, it can be done by any language of any syntax whatsoever, much better than the 1960s hack lisp fans has stockholm-ed themselves into loving. To the extent it's rarely done in mainstream languages, it reflects the designers' choice.
* : I'm assuming a remotely sane ISA, not the excrement known as x86.
You could stack them like this in every language but only in Lisp do people actually do it. Lisps bad habbit of stacking all the parentheses like that is what makes it so hard to read. It is easier to write that way, but it is very hard to understand how many contexts up you just moved.
> Lisps bad habbit of stacking all the parentheses like that is what makes it so hard to read.
How much Lisp have you written/read? Closing parens on separate lines would be a nightmare to read in any real-world code.
Also, what makes Lisp hard to read is lack of familiarity. It's not like C is easy to read for someone who's only ever written Lisp. Lispers don't find Lisp hard to read.
Sounds like a good opportunity to oil up Chesterton’s fence. You may want to consider the possibility that the conventions Lispers have been using for considerably longer than C or whatever language you like to bikeshed style guides in has existed were actually settled on for pragmatic reasons.
You’re not alone though. Many nascent Lispers go through the use reader macros to make Lisp look more like what they’re used to phase.
Not just nascent Lispers. I inherited something like this in a C codebase before (several times, actually, only one that I had to actually edit though):
#define BEGIN {
#define END }
#define INTEGER int
...
Yes, they even did it with the types. Made for a weird Pasctran language that the dev was apparently more comfortable with. I think they actually got a C translation of another program almost "for free" doing this. The real thing was they didn't want to write new code and didn't want to learn C, so they subjected everyone after them to this horror.
Moral of the story: If you can't be bothered to learn a language and its conventions, be honest and get another job.
Pasctran is an ancient cult. [1] Some say they're extinct. Others say its practitioners have just gone underground, now that society will no longer tolerate such things done in public. It's everywhere once you start looking though, insidiously contaminating our precious function bodies.
From the source to the original Bourne shell:
BEGIN
REG BOOL slash; slash=0;
WHILE !fngchar(*cs)
DO IF *cs++==0
THEN IF rflg ANDF slash THEN break; ELSE return(0) FI
ELIF *cs=='/'
THEN slash++;
FI
OD
END
I don't buy that, when you do the same in other languages you get:
for (var i = 1; i < 101; i++) {
if (i % 15 == 0) {
console.log("FizzBuzz");}
else if (i % 3 == 0) {
console.log("Fizz");}
else if (i % 5 == 0) {
console.log("Buzz");}
else {
console.log(i);}}
Why do you think nobody other than lispers writes code like this? Is it really necessary to write code that way? If it is better, why not does nobody else do it? They can also use colored bracers and tooling, while lispers has written code that way forever.
I'm sure a big reason people call lisp a "write only language" is because of this strange convention of stacking all parentheses in a big clump instead of formatting like normal.
Yet, for Python the layout is like above, with indentation being significant:
for i in range(1, 101):
if i % 15 == 0:
print("FizzBuzz")
elif i % 3 == 0:
print("Fizz")
elif i % 5 == 0:
print("Buzz")
else:
print(i)
It does not need the {} pairs then. Now, are we lost because the {} pairs are missing?
In Lisp you need to learn to apply the same idea of indentation being significant:
(loop for i from 1 upto 100 do
(cond ((zerop (mod i 15))
(write-line "FizzBuzz"))
((zerop (mod i 3))
(write-line "Fizz"))
((zerop (mod i 5))
(write-line "Buzz"))
(t
(write-line (princ-to-string i)))))
Just imagine the grouping parentheses are not there.
The disadvantages of Lisps are basically two:
a) there are more parentheses because of the nested lists being used to write code
b) one now needs to understand when (sin a) is actually code and when it is data.
The advantage of Lisp syntax:
a) code is already a simple nested data structure
b) the indentation&layout of code can be (and often is) computed from the data, while usually in Python the lines and indentation are significant
Do you want Python? Because this is how you get Python.
Joke aside, this is why I never understood this problem. With proper indentation it looks essentially like Python with a generous helping of your-father's-parentheses.
>when you do the same in other languages you get...
Your result should be unsurprising. Lisps have a minimal syntax that naturally entails high levels of nesting: (s-)expressions being used to represent functions, control structures, data, etc.
Why should a particular style convention appropriate for that kind of language necessarily transplant well to JavaScript-- a brace-delimited, Algol-inspired language with a lot of syntax?
I used to feel the same way but lisp gets easier to read with practice, and unless you're writing code in notepad.exe or nano or something, your editor will show you the matching paren.
IMO, because C syntax sucks and makes people actually take time to read and interpret the brackets. In erlang and I think others people don't newline between closing delimiters either. In lisp the brain processes the parens automatically.
Readability. By properly indenting the parentheses and putting them on lines you will at a glance see which contexts you just closed, that is how people format code in every mainstream language.
None of these 'mainstream' languages use a data structure for writing and manipulating code like Lisp. The use of Lisp is thus very different and Lisp programmers find a more compact notation more useful.
> you will at a glance see which contexts you just closed
the text editor does that for me
> how people format code in every mainstream language
Python code formatting looks different from Java code formatting.
Python code:
for h in range(0, height):
for w in range(0, width):
v = bw_image.getpixel((w, h))
if v >= avg:
bm_image.putpixel((w, h), white)
else:
bm_image.putpixel((w, h), black)
Lisp code just looks like that. Only with added parentheses (because Lisp expressions are written as nested lists) and prefix notation:
(dotimes (h height)
(dotimes (w width)
(setf v (get-pixel bw-image w h))
(if (>= v avg)
(put-pixel bm-image w h while)
(put-pixel bm-image w h black))))
Any language that uses closing symbols formats them that way, in Pyhton in Java, in C etc. Lisp is the only example where they don't properly indent closing symbols and instead just put them all together at the last statement.
Yet most Python code is written in the compact form I've showed you, where code indentation is significant. Lisp has the same model: the code indentation is significant. But Lisp has the structure encoded in nested lists. These nested lists are automatically formatted in the same space saving way as Python code. Due to the significant indentation, Python usually can avoid to have grouping characters/symbols.
I see also lots of Python code where data isn't written like you claim...
There is a real drawback to stacking more than three or four closing parens like this, without spacing them out: it's hard and in fact impossible to count them at a glance. Not an issue if you have a code editor with auto paren matching, but it can be an annoyance when reading LISP-like code on the web or elsewhere.
Above is much harder to read and wastes a huge amount of space on the screen. Imagine that lists of are much longer and deeper nested. Finding the corresponding parentheses is usually done by using the editor (select a whole expression, have blinking or colored parentheses, etc.).
The main difference between Lisp and most other programming languages is that programs are written as a data structure: nested lists. Not only that: they are not static lists, but we can compute with them - that's the main appeal. It's a programming language, where it's easy and common to manipulate lists, even programs as lists. Thus the notation has not only be useful for reading code, but also for input/output by humans and programs. There a compact notation is much more useful, since tools will often create huge amounts of nested lists. For example imagine a macro for some Lisp functionality, like a complex loop expression. The macro expanded code can often be ten times larger as the input expression, yet we may want to see it in a debugger -> write that expression in a compact way.
I let Lisp indent my code and the system-wide standard indentation makes it easier to spot parentheses errors, since all code has the same shape rules.
I want the expressions to use less vertical space, so that I can see that the opening parentheses of the CASE clauses align. Anything else is just visual clutter.
It is very common for programmers who are new to Lisp to close their parentheses in the former style. However, it is a crutch that they soon do away with if they stick to the language for any length of time.
The first option has both opening and closing brackets on their own lines, while the second has neither. Note that I consider the function name to be part of the opening bracket since it's distinct from the parameters.
which looked really weird to me first but I’ve found it to improve legibility a lot. (Of course there’s a reason it’s idiomatic in Haskell and not elsewhere.)
That the expression is ended I can see in the typical Lisp expression because of the block structure. The single parentheses does make the layout very cluttered and visually ugly. It gets even worse in typically larger Lisp code or data.
Same here. I never understood why a paren on the left of a function name “looks like fingernail clippings” but to the right it’s “just how code works.”
We have to try to understand the non-strawman position of those who are genuinely turned off by Lisp syntax. The problem for them isn't the position of the parenthesis or the lack of commas: those things are probably fine for almost everyone.
In Lisps, this notation represents all structures in the program: definitions of functions, types and variables, control statements and so on.
For the users who have some kind of problem with that, it wouldn't be any better with the op(arg, ...) notation; and I suspect that most would agree that it's even worse.
(For that matter, most programmers don't actually have experience nesting the f(x, y, ...) notation to the same depth that is seen in Lisp programs. Anyone comparing a simple one-or-two-liner f(x, y, ...) with a modicum of nesting to Lisp code that runs for pages and pages is doing apples and oranges.)
Clojure takes this criticism to heart, and at least uses different delimiters for indicate different kinds of syntactic structures. Like () for classic lists, [] for arrays, {} for maps. Helps with visually detecting different kinds of structures in your code.
Clojure also prunes the excessive parentheses whenever the structure is obvious from context. E.g. "(let ((a 1) (b 2)) ...)" becomes "(let [a 1 b 2] ...)" and Clojure just requires an even number of elements so that the pairing can be inferred.
Common Lisp has this in the `setq` and `psetq` macros. E.g. exchange `x` and `y`:
(psetq x y y x)
You can easily have a nesting reduced binding macro, like (var (x 1 y 1) (list x y)), with sequential binding.
Untested:
(defmacro var (pairs &body body)
(if (oddp (length pairs))
(error "~s: variables and init-forms must occur pairwise" 'var))
`(let* ,(loop for (var init) on pairs by #'cddr
unless (and var (symbolp var)
(not (eql var t))
(not (keywordp var)))
do
(error "~s: ~s isn't a variable name" 'var var)
collect (list var init))
,@body)))
Sadly, I'm not convinced that there is much more than the straw man, honestly. Folks are predisposed to think it is a hard to read language. And this is on large because everyone says so.
Similarly, python. Is only a readable language because the community insists it is.
Larry Wall's remark was less about prefix notation and more about how there's very little visual difference. I don't necessarily agree with the criticism because that's the price you pay for homoiconicity, and it's a price well worth paying; but it was a more serious complaint than "lmao parentheses".
See e.g. Clojure, that introduced #{} for sets, [] for vectors etc.
It's funny cause to me visual differences in syntax are just useless information overload. And another dimension of constraints to deal with. That's why I clinged to low syntax languages.. you write in semantics almost. The rest is problem solving (or even extending metadomain with macros or else to help yourself)
Well no but actually no. Mathematica is homoiconic, and it has tons of special syntax and sugar that all boils down to lists when you quote. Elixir also does this with a ruby-like syntax on top. Those are just the 2 I know of. Making the compiler available through a programmer-accessible API from within the program itself is the big idea, it has nothing to do with what the text grammar happens to be.
I was personally blown away when I read PG's summary of McCarthy's paper where he came up with a few primitives and built literally any function based on them, including the `eval' function itself. All of this based on some really basic primitives like `atom' and `lambda'.
If writing `eval' is possible in the same language you started with, then you must be able to write your own eval, no? Or redefine the primitives? But then what happens if you accidentally set DEFINE to be 5? Restart the buffer? Is there any way to get back to your original definition of DEFINE after corrupting it?
I was gonna respond that you can't redefine the primitives because they're not functions or macros but special forms. And this seems true in Common Lisp. I can't define a function named SETQ(the primitive for assignment in CL, also called set! in other lisps) because it's a reserved name. But then I thought I'd try it in Guile and there I was allowed to (define define 5). Even though define is also a special form in Guile. Which made me unable to define further things, telling me 5 is not a valid argument to APPLY. And no I don't know how to fix it...
Every language has the errors that it thinks you should not be allowed to make, and the errors that it trusts you to make on the assumption that you can determine whether you meant to make them better than the compiler. People complain on both sides of the permissiveness spectrum—they want a language that allows them to make exactly the errors that they want to make, but, of course, that set of errors differs from person to person!
In Common Lisp you can have namespaces, where you can have your own SETQ:
CL-USER 53 > (defpackage mylisp
(:use "COMMON-LISP") ; everything from Common Lisp
(:shadow CL:SETQ)) ; but not SETQ
#<The MYLISP package, 1/16 internal, 0/16 external>
CL-USER 54 > (in-package "MYLISP")
#<The MYLISP package, 1/16 internal, 0/16 external>
MYLISP 55 > (defun setq (a b c) (print (list a b c)))
SETQ
MYLISP 56 > (setq (+ 1 2) 4 5)
(3 4 5)
(3 4 5)
MYLISP 57 > (cl:setq a 12)
12
Most implementations also have a way to protect core symbols against redefinition and a way to do it anyway (possibly shooting yourself into the foot).
Very great presentation! I made a little toy some time ago [1] with YAML files compiled into Javascript code, in Lisp style, basically along the same lines as the article, and I use this project to manage my dotfiles. The basic syntax probably looks like this:
If the language subset can't loop, can't recurse, can only call functions that are guaranteed to finish in finite time, and only allows finite-length programs, then its programs will always halt in finite time.
The key to solving the halting problem is realizing that you don't need to solve the halting problem.
Problem : Code C has to run in finite time T, Solution : put a call to a watch dog routine at every loop start, at every subroutine start, and every n linear instructions in C. If the watch dog decides T has passed, it never returns control to the program (calls exit(-1), throws a special uncatchable exception,...). This can be automated by the compiler.
This is why AWS can give you a limited time for a Lambda or why every single multi-tasking OS can terminate an unresponsive process. They are not solving the halting problem.
If the language C is written in is bytecode-interpreted, you don't need to modify the guest program itself to achieve the described effect, you can make the interpreter itself keep track of T and refuse to evaluate further when it runs out. If the execution environment is preemptively concurrent, you don't need to modify the host program as well, simply make the watch dog preempts the guest at T.
Lisp is inspiring at first glance, but when you need to solve complex problems like references, pointers, macros, byte-compilation and native compilation it just is not expressive enough like C, C++, or Rust. Neither is Javascript. Lisp could not replace all other languages
I seriously can't downvote your comment hard enough.
> references
Everything in Lisp is passed by copied references (when talking in C++ terms), so that's kind of a solved problem.
> pointers
You don't have raw pointers in CL since it's a memory-managed language, unless you're dealing with foreign code, and then there's support for handling these in a meaningful way (see CFFI).
> macros
Seriously? The CL macro system with quote-unquote and the full language being available has been good enough to be an inspiration for syntactic macro systems in multiple other languages, such as Rust or Elixir.
> byte-compilation
You only compile to bytecode if you cannot compile to native code. You can see it in implementations like ECL (if GCC is not present) and CLISP (if not built to use GNU Lightning).
> native compilation
SBCL and CCL and ECL and Clasp and LispWorks and ACL all do that, it's a solved problem. Or do you mean that writing compilers in Lisp is impossible since the language is not expressive enough, at which point you can see the source code for Lisp compilers which is frequently written in Lisp?
> Neither is Javascript.
That's the only part of your comment that I think makes any sense.
> Lisp could not replace all other languages.
Who even posed the statement that it should? Are you trolling?
This is entirely inaccurate and a fundamental misunderstanding of Lisp, both as a language and as an ecosystem. Have you seen [0], for example? It's how to leverage the native assembler of a Common Lisp compiler to build a small JIT-like compiler in Common Lisp.
Did you know Lisp has a disassembled built-in, with the standard library function DISASSEMBLE [1]?
Why would Lisp support these things like "native compilation" are supposedly out of scope?
Never mind that Lisp solved references, pointers, macros, byte-compilation and native compilation before the calendar flipped from 1969.
Lisp was compiled by around 1960.
Peter Landing, in 1964, presented the SECD machine, a kind of bytecode based on Lisp-like lists, directly usable for Lisp implementations, in "The Mechanical Evaluation of Expressions". Numerous Lisps have historically had some kind of "Lisp Assembly Language" (LAP) as an alternative or in addition to native compilation.
ZetaLisp, in 1981, the variant of MacLisp running on Lisp machines, featured locatives, which allow the address of a location to be passed around, for simulating reference parameters or members.
What do you mean? Many lisps have these features, in fact lisps in general are well-regarded for their macro systems. I don't think any programming language can replace all other languages but of all options, I feel lisps which allow you to safely and cleanly extend the language to better reflect the problem domain cast an unusually wide shadow.
I'm saying that it's hard to build a basic lego block with another lego block. You have to use plastic, heat and a mold.
That's why we have a wide range of programming languages at different levels, and why it's common for people to say "high level" or "low level" languages.
It's unclear to me what your suggesting, or what question your asking really.
>Now, if we agree that the ability to manipulate source code is important to us, what kind of languages are most conducive for supporting it?
This is useful for compiler programmers, or maybe also those writing source code analyzers/optimizers, but is that it? On occasion I have had to write DSLs for the user input, but in these cases the (non-programmer) users didn't want to write Lisp so I used something like Haskell's parsec to parse the data.
The remote code example given in the post is compelling, but again seems a bit niche. I don't doubt that it's sometimes useful but is it reason enough to choose the language? Are there examples of real-life (non-compilers-related) Lisp programs that show the power of homoiconicity?
Same goes with the concept of "being a guest" in the programming language. I have never wanted to change "constant" to "c". Probably I'm not imaginative enough, but this has never really been an issue for me. Perhaps it secretly has though, and some of my problems have been "being a guest" in disguise.