From 1b88574232b2882d80e3c0c1de6a195733b9bef8 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 01:35:12 -0500 Subject: [PATCH 01/18] Add intro to post --- .../2023-02-04-interpreters-and-compilers.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 _posts/2023-02-04-interpreters-and-compilers.md diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md new file mode 100644 index 000000000..e79191903 --- /dev/null +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -0,0 +1,91 @@ +--- +title: "Interpreters and compilers" +layout: post +date: 2023-02-04 +--- + +## Intro + +Compilers and interpreters are not as different as people often make them out +to be. I don't mean in the [Futamura projection][futamura] sense, where +everything is a specializer. I mean this more as an observation: compilers +often contain interpreters and interpreters often contain +compilers[^futamura-related]. + +[futamura]: https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_projections + +[^futamura-related]: Perhaps this co-occurrence is related to the Futamura + projections. I haven't thought about that too much. But it's not the point + of this particular text. + +If you think about your favorite interpreters, you might notice that they often +have some middle stage that front-loads some of the interpretation work. +Sometimes this is does not exist at all: MRI (the main Ruby implementation) +used to interpret code right off the AST. Now, it's just completely invisible: +Ruby compiles ASTs to bytecode in memory and executes that. The change from AST +to bytecode was a big change in the amount of user program preprocessing, but +the only effect observable by Ruby programmers was a speed increase. + +Good data structures are crucial for language implementation performance. The +change in representation from AST to bytecode may not seem like a big one, but +the transformation from a pointer-heavy tree data structure to a compact linear +structure gives significant wins on modern hardware. It's kind of like going +from iterating over a linked list to iterating over an array; machine caches +were built for arrays and as long as you iterate somewhat predictably, reading +in the next byte of data is very fast. + +For whatever reason, people feel compelled to make a big hullabaloo about the +distinction between compilers and interpreters[^languages-implementations]. My +friend Kartik and I don't agree with this, even if the way he phrases it might +make it seem otherwise: + +[^languages-implementations]: Not to mention conflating languages and + implementations. But that's both an inference you can make from this post + and a rant for another day. + +> It would be cool to go from BF interpreter all the way to a real compiler. + +We had a conversation about this after reading Laurie Tratt's Brainfuck +interpreter post, [*Compiled and Interpreted Languages: Two Ways of Saying +Tomato*][ltbf]. Kartik continues about what he wants to see: + +[ltbf]: https://tratt.net/laurie/blog/2023/compiled_and_interpreted_languages_two_ways_of_saying_tomato.html + +> In the beginning it runs the code with zero prep. At the end it does a lot of +> prep before running the code. In between it does some intermediate amount of +> data structure initialization. + +Maybe just reading Laurie's post and this commentary gives you enough insight +about the nature of compilers and interpreters and you can close this tab, +satisfied. That would be totally great. If not, though, strap in. We're going +to do the whole enchilada. We're going to write a lot of different +interpreters. With each interpreter, we will identify a bottleneck in +interpretation and adjust both the data structures and the amount of +preprocessing to make that bottleneck go away. We will continue until we arrive +at a native code compiler. Then we will continue some more. + +## A lay of the land + +I looked at a couple of different small languages while trying to decide which +to use. I wanted something bigger than Brainfuck---inscrutable, too few +operations, not similar enough to other languages---but not big enough that +this work would take a lifetime. Ideally, we could even get it done in a couple +of months. + +* Tiger +* MinCaml +* Decaf +* ChocoPy +* GoLite +* Xi +* Wabbit + +After surveying the list, I landed on Wabbit. It's small, useful enough, and +does not include any features that might require significant unexpected design +work, like classes or concurrency. A close second was Tiger, since I am already +familiar with it, but I think it is needlessly big for this exercise. Extending +Wabbit into Tiger is left as an exercise for the reader. + +
+
+ From b903e23268c3407b05b1ebd97ee31ca219f2d8ad Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 01:38:08 -0500 Subject: [PATCH 02/18] Add Lisp note --- _posts/2023-02-04-interpreters-and-compilers.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index e79191903..1536bf744 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -79,6 +79,7 @@ of months. * GoLite * Xi * Wabbit +* Lisp After surveying the list, I landed on Wabbit. It's small, useful enough, and does not include any features that might require significant unexpected design @@ -86,6 +87,12 @@ work, like classes or concurrency. A close second was Tiger, since I am already familiar with it, but I think it is needlessly big for this exercise. Extending Wabbit into Tiger is left as an exercise for the reader. +I also avoided Lisp because I want the broadest appeal possible. It's too easy +to write off a post using Lisp as its target language because it's "only +possible with Lisp" or "only possible for functional languages" or something +else. Also, I have too much Lisp content on this blog for someone who never +really writes Lisp. +
From 44b9f9bf484185bcd074039558c95724a15561af Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 01:40:16 -0500 Subject: [PATCH 03/18] Add potato note --- _posts/2023-02-04-interpreters-and-compilers.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 1536bf744..e3384a8c4 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -4,6 +4,10 @@ layout: post date: 2023-02-04 --- + + ## Intro Compilers and interpreters are not as different as people often make them out From 227945148b334d0072c5fcef5e3a73909bcd1dad Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 10:38:10 -0500 Subject: [PATCH 04/18] Add a bit about Wabbit --- .../2023-02-04-interpreters-and-compilers.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index e3384a8c4..2ee885c44 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -97,6 +97,38 @@ possible with Lisp" or "only possible for functional languages" or something else. Also, I have too much Lisp content on this blog for someone who never really writes Lisp. +[Wabbit][wabbit] is a small programming language created by [David +Beazley][dabeaz] to teach compilers. The following example snippet by David +gives a taste for the features we will need to implement. It looks a little bit +like Go: + +[wabbit]: https://www.dabeaz.com/wabbit.html +[dabeaz]: https://www.dabeaz.com/index.html + +```go +/* fib.wb - Compute fibonacci numbers */ + +const LAST = 30; // A constant declaration + +// A function declaration +func fibonacci(n int) int { + if n > 1 { // Conditionals + return fibonacci(n-1) + fibonacci(n-2); + } else { + return 1; + } +} + +func main() int { + var n int = 0; // Variable declaration + while n < LAST { // Looping (while) + print fibonacci(n); // Printing + n = n + 1; // Assignment + } + return 0; +} +``` +
From 5136e73ee88db6055ad77db515d989b459eaeb78 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 11:02:57 -0500 Subject: [PATCH 05/18] Give a lay of the land --- .../2023-02-04-interpreters-and-compilers.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 2ee885c44..f53cfa2b1 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -129,6 +129,39 @@ func main() int { } ``` +Unlike many other tomes about compilers, we will not talk about lexing and +parsing here. We will instead start at the AST, assuming it has been neatly +generated for us by some parser. We skip this step because parsing is a +requirement for both interpreters and compilers, it distracts from the main +points of the post, is well-covered by other resources, and can be provided as +a little library for your perusal, if you are interested later on. + +We will instead do the following: + +* Write an AST interpreter +* Optimize the AST interpreter[^graal] +* Compile the AST to bytecode +* Write a bytecode interpreter +* Optimize the bytecode interpreter +* Compile the bytecode to machine code in memory +* Write the machine code to disk + +[^graal]: Some interpreters don't make it past the AST interpreter stage + because they don't need to! Compilers like Graal have put in the work to + make AST interpreters very very fast. This is an unusually impressive feat + and we don't want to do that much work here. + +At each stage, we will do a little more work up-front (before starting user +program execution) than before. This means that interpreter start-up will be +slower and user program execution will be faster[^user-needs]. + +[^user-needs]: Depending on the user program, this might be an acceptable + trade-off. It might also not be, and you should stop earlier in the + process. For example, programs like servers tend to be long-lived, so they + want to optimize the user code as much as possible. Programs for scripting + at the command line might value low start-up latency because they are + interactive. +
From 10bbae2d59311fabe5985739aa5f3e7bf9f2fb18 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 11:07:37 -0500 Subject: [PATCH 06/18] Give authors credit --- _posts/2023-02-04-interpreters-and-compilers.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index f53cfa2b1..d6299841a 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -147,9 +147,9 @@ We will instead do the following: * Write the machine code to disk [^graal]: Some interpreters don't make it past the AST interpreter stage - because they don't need to! Compilers like Graal have put in the work to - make AST interpreters very very fast. This is an unusually impressive feat - and we don't want to do that much work here. + because they don't need to! The authors of Graal, for example, have put in + the work to make AST interpreters very very fast. This is an unusually + impressive feat and we don't want to do that much work here. At each stage, we will do a little more work up-front (before starting user program execution) than before. This means that interpreter start-up will be From 76aa273278cff6af8f610d47be18c3211cd93903 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Mon, 6 Feb 2023 14:16:09 -0500 Subject: [PATCH 07/18] Add note about overhead vs optimizations --- _posts/2023-02-04-interpreters-and-compilers.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index d6299841a..938a40478 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -36,7 +36,15 @@ the transformation from a pointer-heavy tree data structure to a compact linear structure gives significant wins on modern hardware. It's kind of like going from iterating over a linked list to iterating over an array; machine caches were built for arrays and as long as you iterate somewhat predictably, reading -in the next byte of data is very fast. +in the next byte of data is very fast[^interpretive-overhead]. + +[^interpretive-overhead]: Interpreter overhead and program optimization are + similar but also different. Going from an AST to bytecode may not + fundamentally alter the user program's meaning but the bytecode + representation is more convenient to interpret. It's optimizing your + program---the compiler author's program---as opposed to the user program. + It's possible to do traditional compiler optimizations (strength reduction, + etc) on either representation of the user program. For whatever reason, people feel compelled to make a big hullabaloo about the distinction between compilers and interpreters[^languages-implementations]. My From dd1981aecfe18e15e15cfd0e5b434d80373d4df9 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 15:02:43 -0400 Subject: [PATCH 08/18] Add a little vague preamble that might actually turn into the blog post --- .../2023-02-04-interpreters-and-compilers.md | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 938a40478..1476ee1e8 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -8,6 +8,67 @@ date: 2023-02-04 Laurie's post? Or maybe I should call the interpreter potato and the compiler potato, pronounced differently. --> + + ## Intro Compilers and interpreters are not as different as people often make them out From ac126a7b3c91dc4ccc14b4cb05dc3e1aa69d5f00 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 15:04:34 -0400 Subject: [PATCH 09/18] Add note about original plan --- _posts/2023-02-04-interpreters-and-compilers.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 1476ee1e8..9513a97e3 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -61,12 +61,17 @@ little fuzzy when you treat the code as data. ## Epilogue -What this doesn't address is that some *languages* require more run-time glue -code, called "a runtime", to happen around the edges of your application code. -Features like reflection, dynamic dispatch, garbage collection, etc all add a -bit of runtime code into the mix. People who see implementations that include a -runtime tend to point their fingers and yell "interpreter!" but I think it's a -red herring. +The post I was originally going to write on this topic involved actually +writing an interpreter and iteratively transforming it into more of a compiler, +doing all of the steps that the projects mentioned above do. But there are only +so many hours in the day. + +Also, what this doesn't address is that some *languages* require more run-time +glue code, called "a runtime", to happen around the edges of your application +code. Features like reflection, dynamic dispatch, garbage collection, etc all +add a bit of runtime code into the mix. People who see implementations that +include a runtime tend to point their fingers and yell "interpreter!" but I +think it's a red herring. --> ## Intro From 858d016ab3140bcdefd374553d1807976bd76c7a Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 15:08:19 -0400 Subject: [PATCH 10/18] WIP: rip out guts of old post --- .../2023-02-04-interpreters-and-compilers.md | 221 +++++------------- 1 file changed, 53 insertions(+), 168 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 9513a97e3..5d361ab4d 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -1,14 +1,55 @@ --- -title: "Interpreters and compilers" +title: "Interpreters and compilers; or, Potato potato" layout: post -date: 2023-02-04 +date: 2023-06-30 --- - +## Intro + +Compilers and interpreters are not as different as people often make them out +to be. I don't mean in the [Futamura projection][futamura] sense, where +everything is a specializer. I mean this more as an observation: compilers +often contain interpreters and interpreters often contain +compilers[^futamura-related]. + +[futamura]: https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_projections + +[^futamura-related]: Perhaps this co-occurrence is related to the Futamura + projections. I haven't thought about that too much. But it's not the point + of this particular text. + +For whatever reason, people feel compelled to make a big hullabaloo about the +distinction between compilers and interpreters[^languages-implementations]. My +friend Kartik and I don't agree with this, even if the way he phrases it might +make it seem otherwise: + +[^languages-implementations]: Not to mention conflating languages and + implementations. But that's both an inference you can make from this post + and a rant for another day. + +> It would be cool to go from BF interpreter all the way to a real compiler. + +We had a conversation about this after reading Laurie Tratt's Brainfuck +interpreter post, [*Compiled and Interpreted Languages: Two Ways of Saying +Tomato*][ltbf]. Kartik continues about what he wants to see: + +[ltbf]: https://tratt.net/laurie/blog/2023/compiled_and_interpreted_languages_two_ways_of_saying_tomato.html + +> In the beginning it runs the code with zero prep. At the end it does a lot of +> prep before running the code. In between it does some intermediate amount of +> data structure initialization. + +Maybe just reading Laurie's post and this commentary gives you enough insight +about the nature of compilers and interpreters and you can close this tab, +satisfied. That would be totally great. If not, though, strap in. We're going +to do the whole enchilada. We're going to write a lot of different +interpreters. With each interpreter, we will identify a bottleneck in +interpretation and adjust both the data structures and the amount of +preprocessing to make that bottleneck go away. We will continue until we arrive +at a native code compiler. Then we will continue some more. + +## blah - -## Intro - -Compilers and interpreters are not as different as people often make them out -to be. I don't mean in the [Futamura projection][futamura] sense, where -everything is a specializer. I mean this more as an observation: compilers -often contain interpreters and interpreters often contain -compilers[^futamura-related]. - -[futamura]: https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_projections - -[^futamura-related]: Perhaps this co-occurrence is related to the Futamura - projections. I haven't thought about that too much. But it's not the point - of this particular text. - -If you think about your favorite interpreters, you might notice that they often -have some middle stage that front-loads some of the interpretation work. -Sometimes this is does not exist at all: MRI (the main Ruby implementation) -used to interpret code right off the AST. Now, it's just completely invisible: -Ruby compiles ASTs to bytecode in memory and executes that. The change from AST -to bytecode was a big change in the amount of user program preprocessing, but -the only effect observable by Ruby programmers was a speed increase. - -Good data structures are crucial for language implementation performance. The -change in representation from AST to bytecode may not seem like a big one, but -the transformation from a pointer-heavy tree data structure to a compact linear -structure gives significant wins on modern hardware. It's kind of like going -from iterating over a linked list to iterating over an array; machine caches -were built for arrays and as long as you iterate somewhat predictably, reading -in the next byte of data is very fast[^interpretive-overhead]. - -[^interpretive-overhead]: Interpreter overhead and program optimization are - similar but also different. Going from an AST to bytecode may not - fundamentally alter the user program's meaning but the bytecode - representation is more convenient to interpret. It's optimizing your - program---the compiler author's program---as opposed to the user program. - It's possible to do traditional compiler optimizations (strength reduction, - etc) on either representation of the user program. - -For whatever reason, people feel compelled to make a big hullabaloo about the -distinction between compilers and interpreters[^languages-implementations]. My -friend Kartik and I don't agree with this, even if the way he phrases it might -make it seem otherwise: - -[^languages-implementations]: Not to mention conflating languages and - implementations. But that's both an inference you can make from this post - and a rant for another day. - -> It would be cool to go from BF interpreter all the way to a real compiler. - -We had a conversation about this after reading Laurie Tratt's Brainfuck -interpreter post, [*Compiled and Interpreted Languages: Two Ways of Saying -Tomato*][ltbf]. Kartik continues about what he wants to see: - -[ltbf]: https://tratt.net/laurie/blog/2023/compiled_and_interpreted_languages_two_ways_of_saying_tomato.html - -> In the beginning it runs the code with zero prep. At the end it does a lot of -> prep before running the code. In between it does some intermediate amount of -> data structure initialization. - -Maybe just reading Laurie's post and this commentary gives you enough insight -about the nature of compilers and interpreters and you can close this tab, -satisfied. That would be totally great. If not, though, strap in. We're going -to do the whole enchilada. We're going to write a lot of different -interpreters. With each interpreter, we will identify a bottleneck in -interpretation and adjust both the data structures and the amount of -preprocessing to make that bottleneck go away. We will continue until we arrive -at a native code compiler. Then we will continue some more. +Last, it doesn't address why you might want to do more or less program +transformation up front for your workload. -## A lay of the land - -I looked at a couple of different small languages while trying to decide which -to use. I wanted something bigger than Brainfuck---inscrutable, too few -operations, not similar enough to other languages---but not big enough that -this work would take a lifetime. Ideally, we could even get it done in a couple -of months. - -* Tiger -* MinCaml -* Decaf -* ChocoPy -* GoLite -* Xi -* Wabbit -* Lisp - -After surveying the list, I landed on Wabbit. It's small, useful enough, and -does not include any features that might require significant unexpected design -work, like classes or concurrency. A close second was Tiger, since I am already -familiar with it, but I think it is needlessly big for this exercise. Extending -Wabbit into Tiger is left as an exercise for the reader. - -I also avoided Lisp because I want the broadest appeal possible. It's too easy -to write off a post using Lisp as its target language because it's "only -possible with Lisp" or "only possible for functional languages" or something -else. Also, I have too much Lisp content on this blog for someone who never -really writes Lisp. - -[Wabbit][wabbit] is a small programming language created by [David -Beazley][dabeaz] to teach compilers. The following example snippet by David -gives a taste for the features we will need to implement. It looks a little bit -like Go: - -[wabbit]: https://www.dabeaz.com/wabbit.html -[dabeaz]: https://www.dabeaz.com/index.html - -```go -/* fib.wb - Compute fibonacci numbers */ - -const LAST = 30; // A constant declaration - -// A function declaration -func fibonacci(n int) int { - if n > 1 { // Conditionals - return fibonacci(n-1) + fibonacci(n-2); - } else { - return 1; - } -} - -func main() int { - var n int = 0; // Variable declaration - while n < LAST { // Looping (while) - print fibonacci(n); // Printing - n = n + 1; // Assignment - } - return 0; -} -``` - -Unlike many other tomes about compilers, we will not talk about lexing and -parsing here. We will instead start at the AST, assuming it has been neatly -generated for us by some parser. We skip this step because parsing is a -requirement for both interpreters and compilers, it distracts from the main -points of the post, is well-covered by other resources, and can be provided as -a little library for your perusal, if you are interested later on. - -We will instead do the following: - -* Write an AST interpreter -* Optimize the AST interpreter[^graal] -* Compile the AST to bytecode -* Write a bytecode interpreter -* Optimize the bytecode interpreter -* Compile the bytecode to machine code in memory -* Write the machine code to disk - -[^graal]: Some interpreters don't make it past the AST interpreter stage - because they don't need to! The authors of Graal, for example, have put in - the work to make AST interpreters very very fast. This is an unusually - impressive feat and we don't want to do that much work here. - -At each stage, we will do a little more work up-front (before starting user -program execution) than before. This means that interpreter start-up will be -slower and user program execution will be faster[^user-needs]. - -[^user-needs]: Depending on the user program, this might be an acceptable - trade-off. It might also not be, and you should stop earlier in the - process. For example, programs like servers tend to be long-lived, so they - want to optimize the user code as much as possible. Programs for scripting - at the command line might value low start-up latency because they are - interactive. +
From b757d66c34a0be44b94aaa0e82cbfd19da4334d3 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 15:20:50 -0400 Subject: [PATCH 11/18] Clean up --- .../2023-02-04-interpreters-and-compilers.md | 72 +++++++------------ 1 file changed, 27 insertions(+), 45 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 5d361ab4d..a68133a8f 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -6,58 +6,31 @@ date: 2023-06-30 ## Intro -Compilers and interpreters are not as different as people often make them out -to be. I don't mean in the [Futamura projection][futamura] sense, where -everything is a specializer. I mean this more as an observation: compilers -often contain interpreters and interpreters often contain -compilers[^futamura-related]. - -[futamura]: https://en.wikipedia.org/wiki/Partial_evaluation#Futamura_projections - -[^futamura-related]: Perhaps this co-occurrence is related to the Futamura - projections. I haven't thought about that too much. But it's not the point - of this particular text. - For whatever reason, people feel compelled to make a big hullabaloo about the distinction between compilers and interpreters[^languages-implementations]. My -friend Kartik and I don't agree with this, even if the way he phrases it might -make it seem otherwise: +friend [Kartik](http://akkartik.name/) and I had a conversation about this +tendency after reading Laurie Tratt's Brainfuck interpreter post, [*Compiled +and Interpreted Languages: Two Ways of Saying Tomato*][ltbf]. [^languages-implementations]: Not to mention conflating languages and implementations. But that's both an inference you can make from this post and a rant for another day. -> It would be cool to go from BF interpreter all the way to a real compiler. - -We had a conversation about this after reading Laurie Tratt's Brainfuck -interpreter post, [*Compiled and Interpreted Languages: Two Ways of Saying -Tomato*][ltbf]. Kartik continues about what he wants to see: - [ltbf]: https://tratt.net/laurie/blog/2023/compiled_and_interpreted_languages_two_ways_of_saying_tomato.html -> In the beginning it runs the code with zero prep. At the end it does a lot of -> prep before running the code. In between it does some intermediate amount of -> data structure initialization. - -Maybe just reading Laurie's post and this commentary gives you enough insight -about the nature of compilers and interpreters and you can close this tab, -satisfied. That would be totally great. If not, though, strap in. We're going -to do the whole enchilada. We're going to write a lot of different -interpreters. With each interpreter, we will identify a bottleneck in -interpretation and adjust both the data structures and the amount of -preprocessing to make that bottleneck go away. We will continue until we arrive -at a native code compiler. Then we will continue some more. +Laurie's post does a great job of iteratively adding more ahead-of-time +preprocessing (compilation!) stages to a simple Brainfuck interpreter. If that +already makes you think enough about this, great. Feel free to close this tab. +If you want more, read on. -## blah +## Bigger languages -There's been a lot of back-and-forth about what it means to be an interpreter -and what it means to be a compiler. This post will show you a bunch of examples -and ask you "interpreter or compiler?" and hopefully you will realize the line -is fuzzier than you previously thought and maybe not always a useful -distinction. +This post will show you a bunch of examples and ask you "interpreter or +compiler?" and hopefully you will realize the line is fuzzier than you +previously thought and maybe not always a useful distinction. There are some projects like [Elk](https://github.com/cesanta/elk) that -run JavaScript right off the source code. I think most people would call this +run JavaScript right off the source text. I think most people would call this an interpreter. There are some languages like [Forth](https://en.wikipedia.org/wiki/Forth_(programming_language)) @@ -72,14 +45,15 @@ most people would still say interpreter, despite having a transformation pass from text to tree. If I recall correctly, people on the internet circa 2012 really loved to define "interpreter" as only the tree-walking kind. -Nowadays MRI turns the AST into bytecode before running it. This all happens +Nowadays MRI turns the AST into bytecode before running it. This reduces the +amount of pointer chasing and allows for some optimization. This all happens transparently to you, the programmer, and the bytecode never leaves the VM. Hmmm, things are getting a little fuzzier. There's another code transformation, this time from tree to linearized bytecode... CPython takes this a step further with `.pyc` files and Java with `.class` files (it's in the spec!). Does having an artifact on disk skew your perception -of what's going on? +of what's going on? Even if the artifact is "just bytecode"? Further, some Java runtimes like the [OpenJDK](https://github.com/openjdk/jdk) even turn the bytecode into machine code before running it, though the machine @@ -107,7 +81,8 @@ writing an interpreter and iteratively transforming it into more of a compiler, doing all of the steps that the projects mentioned above do. But there are only so many hours in the day. This is left as an extended exercise for the reader. Take a look at David Beazley's [Wabbit](https://www.dabeaz.com/wabbit.html) as -a good language target. +a good language target. Please let me know if you do this and I will happily +link it here. Also, what this doesn't address is that some *languages* require more run-time glue code, called "a runtime", to happen around the edges of your application @@ -117,9 +92,16 @@ include a runtime tend to point their fingers and yell "interpreter!" but I think it's a red herring. Last, it doesn't address why you might want to do more or less program -transformation up front for your workload. - - +transformation up front for your workload. Some very bright people at a large +social media company wanted to spin up a project for just-in-time (JIT) +compiling C++ because the cost of ahead-of-time (AOT) compiling a bazillion +lines of C++ was just too high. The same social media company, for a different +project, also wanted to spend a little *more* time compiling their Python +code to get better run-time performance. + +So... find your place on the [Pareto +frontier](https://en.wikipedia.org/wiki/Pareto_front) and do as much +compilation as you need.
From 828cbbfc63e14ac1a3e8f184950f35b1cd1a0292 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 15:26:15 -0400 Subject: [PATCH 12/18] s/Laurie/Laurence/g --- _posts/2023-02-04-interpreters-and-compilers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index a68133a8f..cfcbe95da 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -9,7 +9,7 @@ date: 2023-06-30 For whatever reason, people feel compelled to make a big hullabaloo about the distinction between compilers and interpreters[^languages-implementations]. My friend [Kartik](http://akkartik.name/) and I had a conversation about this -tendency after reading Laurie Tratt's Brainfuck interpreter post, [*Compiled +tendency after reading Laurence Tratt's Brainfuck interpreter post, [*Compiled and Interpreted Languages: Two Ways of Saying Tomato*][ltbf]. [^languages-implementations]: Not to mention conflating languages and @@ -18,7 +18,7 @@ and Interpreted Languages: Two Ways of Saying Tomato*][ltbf]. [ltbf]: https://tratt.net/laurie/blog/2023/compiled_and_interpreted_languages_two_ways_of_saying_tomato.html -Laurie's post does a great job of iteratively adding more ahead-of-time +Laurence's post does a great job of iteratively adding more ahead-of-time preprocessing (compilation!) stages to a simple Brainfuck interpreter. If that already makes you think enough about this, great. Feel free to close this tab. If you want more, read on. From 96d7b317030efe16e9f4e07e51d65afc31c92b74 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:13:42 -0400 Subject: [PATCH 13/18] nits --- _posts/2023-02-04-interpreters-and-compilers.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index cfcbe95da..ebfc98dd9 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -27,7 +27,9 @@ If you want more, read on. This post will show you a bunch of examples and ask you "interpreter or compiler?" and hopefully you will realize the line is fuzzier than you -previously thought and maybe not always a useful distinction. +previously thought and maybe not always a useful distinction. Or at least that +if you're confused, it's a system that includes both an interpreter and a +compiler. There are some projects like [Elk](https://github.com/cesanta/elk) that run JavaScript right off the source text. I think most people would call this @@ -66,6 +68,9 @@ if I told you that they have not one, but two different interpreters inside the compiler itself? That the constexpr tree-walking interpreter needed to be turned into a bytecode interpreter to improve compile times? +...and the interpreter that runs your x86? Is it not an interpreter if it's +written in digital circuits? + I think the main takeaway I am trying to push is that internet discourse about this has gotten a little silly and doesn't help people learn things. Interpreters tend to contain compilers and compilers tend to contain From 86caf8b5f3d0be1f572f071d2c592a5ea988294b Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:14:30 -0400 Subject: [PATCH 14/18] nit --- _posts/2023-02-04-interpreters-and-compilers.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index ebfc98dd9..72c2fd71e 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -84,10 +84,10 @@ little fuzzy when you treat the code as data. The post I was originally going to write on this topic involved actually writing an interpreter and iteratively transforming it into more of a compiler, doing all of the steps that the projects mentioned above do. But there are only -so many hours in the day. This is left as an extended exercise for the reader. -Take a look at David Beazley's [Wabbit](https://www.dabeaz.com/wabbit.html) as -a good language target. Please let me know if you do this and I will happily -link it here. +so many hours in the day, so this is left as an extended exercise for the +reader. Take a look at David Beazley's +[Wabbit](https://www.dabeaz.com/wabbit.html) as a good language target. Please +let me know if you do this and I will happily link it here. Also, what this doesn't address is that some *languages* require more run-time glue code, called "a runtime", to happen around the edges of your application From 95c4b3742f3d99834cd245bcc8e28cd20cfb8c3c Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:17:58 -0400 Subject: [PATCH 15/18] nit --- _posts/2023-02-04-interpreters-and-compilers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 72c2fd71e..c6b8fd1b9 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -10,7 +10,8 @@ For whatever reason, people feel compelled to make a big hullabaloo about the distinction between compilers and interpreters[^languages-implementations]. My friend [Kartik](http://akkartik.name/) and I had a conversation about this tendency after reading Laurence Tratt's Brainfuck interpreter post, [*Compiled -and Interpreted Languages: Two Ways of Saying Tomato*][ltbf]. +and Interpreted Languages: Two Ways of Saying Tomato*][ltbf] and I decided to +write a post of my own. [^languages-implementations]: Not to mention conflating languages and implementations. But that's both an inference you can make from this post From b0a7e55b73e4a3c6ffe92c22eea8d7de8eddcf3f Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:18:26 -0400 Subject: [PATCH 16/18] nit --- _posts/2023-02-04-interpreters-and-compilers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index c6b8fd1b9..b48e1707e 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -22,7 +22,7 @@ write a post of my own. Laurence's post does a great job of iteratively adding more ahead-of-time preprocessing (compilation!) stages to a simple Brainfuck interpreter. If that already makes you think enough about this, great. Feel free to close this tab. -If you want more, read on. +If you want more "real world" examples, read on. ## Bigger languages From 32b2170d7e9a201734513be4e1001d7889063a39 Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:18:41 -0400 Subject: [PATCH 17/18] nit --- _posts/2023-02-04-interpreters-and-compilers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index b48e1707e..1c356262d 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -24,14 +24,14 @@ preprocessing (compilation!) stages to a simple Brainfuck interpreter. If that already makes you think enough about this, great. Feel free to close this tab. If you want more "real world" examples, read on. -## Bigger languages - This post will show you a bunch of examples and ask you "interpreter or compiler?" and hopefully you will realize the line is fuzzier than you previously thought and maybe not always a useful distinction. Or at least that if you're confused, it's a system that includes both an interpreter and a compiler. +## Bigger languages + There are some projects like [Elk](https://github.com/cesanta/elk) that run JavaScript right off the source text. I think most people would call this an interpreter. From a75f1fc9ad45f66273e0bdd45da8a6b3b089f9fc Mon Sep 17 00:00:00 2001 From: Max Bernstein Date: Thu, 29 Jun 2023 17:19:56 -0400 Subject: [PATCH 18/18] nit --- _posts/2023-02-04-interpreters-and-compilers.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/_posts/2023-02-04-interpreters-and-compilers.md b/_posts/2023-02-04-interpreters-and-compilers.md index 1c356262d..254ddc547 100644 --- a/_posts/2023-02-04-interpreters-and-compilers.md +++ b/_posts/2023-02-04-interpreters-and-compilers.md @@ -32,9 +32,11 @@ compiler. ## Bigger languages -There are some projects like [Elk](https://github.com/cesanta/elk) that -run JavaScript right off the source text. I think most people would call this -an interpreter. +There are some projects like [Elk](https://github.com/cesanta/elk) that run +JavaScript right off the source text. This avoids allocation as much as +possible because it's designed for microcontrollers, but is slower than your +average JS runtime as a result. I think most people would call this an +interpreter. There are some languages like [Forth](https://en.wikipedia.org/wiki/Forth_(programming_language)) whose implementations generally read one word of input at a time and act on