Jason Barr

Posted on Jul 5, 2023 • Edited on Jul 11, 2023

Create Your Own Programming Language 9: Iteration

#programminglanguages #interpreters #compilers #javascript

In this installment of the Create Your Own Programming Language series we're going to add iteration to Wanda. We're also going to make some major improvements to the CLI at the end of the article.

As always, if you haven't read the previous article where we added conditionals, do that first and then continue below.

Ok, let's go!

Iteration in Wanda

The for form will handle iteration in Wanda, and it looks like this:

(for map ((i (range 5)))
  (+ i i))

The form takes an operator, a list of variables with their initializers, and a body with an arbitrary number of expressions.

Current operators in Wanda include each, map, filter, fold, and fold-r, but you can also define your own for operators by creating functions. They should be higher-order functions that take a function callback as the first argument, and the callback argument should take a parameter for each variable. Then the rest of the main functions' arguments should be the initializers for the callback's parameters.

For instance, here's an example implementation of a map operator:

(def map (fn lst)
  (if (nil? lst)
    lst
    (cons (fn (head lst)) (map fn (tail lst)))))

As you can see, the callback takes a single parameter and the higher-order function takes the callback and a list, which is the initializer for the callback parameter.

Here's how you'd use a for expression to sum up a list of numbers:

(for fold ((sum 0) (x (range 11)))
  (+ sum x))

A for expression desugars to a call expression that uses the operator as its function, constructs a lambda as the first argument to the operator, then passes in the initializers as the remaining arguments to the operator.

Inspiration for our for expressions comes chiefly from the Pyret language, which handles them similarly.

Pyret was inspired by Racket's list comprehensions, Ruby's blocks and iterators, and Smalltalk blocks.

New CLI Features

Here are the new features we're adding to the Wanda CLI:

History (up to 2000 lines)
Help info for the CLI, the wandac compiler, and the REPL
The ability to load a Wanda file from within the REPL
The ability to save a REPL session as a file

We'll get to those at the end of this article, but first let's implement for expressions!

An Easy Iterator

To make it easy to create an object to iterate over, let's add a range function to the core library.

In Python, a Range object doesn't actually contain all the numbers you iterate over. It computes them from the start, stop, and step values. We're not getting that fancy; our range function will just produce a list of numbers. It will work like Python's range function though, in that you can pass it 1, 2, or 3 arguments and it will calculate the range from them.

In lib/js/core.js, just below typeof, add an entry for range:

range: rt.makeFunction(
      (start, stop = undefined, step = 1) => {
        if (typeof stop === "undefined") {
          stop = start;
          start = 0;
        }

        let list = null;
        if (start < stop) {
          list = cons(start, list);
          for (let i = start + step; i < stop; i += step) {
            list.append(i);
          }
        } else if (stop < start) {
          list = cons(start, list);
          for (let i = start - step; i > stop; i -= step) {
            list.append(i);
          }
        }

        return list;
      },
      {
        // contract is variadic because language has no concept of default parameters
        contract: "(&(vector number) -> (list number))",
        name: "range",
      }
    ),

Changes to The Lexer and Reader

None. In fact, for the most part the lexer and reader are done. We may add some small things to them later, but there shouldn't be any major changes to either for the rest of this series.

Changes to The Parser

We need a ForExpression AST node and we need to parse them.

First, add a member to the ASTTypes enum in src/parser/ast.js:

export const ASTTypes = {
  // other members...
  ForExpression: "ForExpression",
};

Then add its constructor to the AST object:

export const AST = {
  // other constructors...
  ForExpression(op, vars, body, srcloc) {
    return {
      kind: ASTTypes.ForExpression,
      op,
      vars,
      body,
      srcloc,
    };
  },
}

In src/parser/parse.js we'll need a new case for the switch statement in parseList:

    // other cases...
    case "for":
      return parseForExpression(form);
    // default case

In parseForExpression we'll get the operator, the variable/initializer list, and the list of body expressions. The operator should be a symbol (though there's technically no reason why it couldn't be a lambda). Then we iterate over the list of variable/initializer pairs and parse them. Finally, we iterate over the body and parse each expression in it.

Here's parseForExpression:

const parseForExpression = (form) => {
  const [_, op, rawVars, ...body] = form;
  const srcloc = form.srcloc;
  const parsedOp = parseExpr(op);

  /** @type {import("./ast.js").ForVar[]} */
  let vars = [];

  for (let rawVar of rawVars) {
    const varName = parseExpr(rawVar.car);
    // need the head of the tail of the rawVar list
    const initializer = parseExpr(rawVar.cdr.car);

    vars.push({ var: varName, initializer });
  }

  /** @type {AST[]} */
  let parsedBody = [];

  for (let expr of body) {
    parsedBody.push(parseExpr(expr));
  }

  return AST.ForExpression(parsedOp, vars, parsedBody, srcloc);
};

That's it for changes to the parser. Now let's see how to type check a for expression.

Changes to The Type Checker

We actually only need to change 2 files in the type checker. First, we need to infer a type for the for expression.

We need to add a dispatch case to infer in src/typechecker/infer.js:

    // other cases...
    case ASTTypes.ForExpression:
      return inferForExpression(ast, env, constant);
    // default case

Now for inferForExpression.

We're actually going to deconstruct the for expression here and infer a type as if it were a function call. This means we need the operator as a function, a lambda as the first argument to the operator, and the list of arguments to follow the lambda.

We get the lambda's params by mapping over node.vars and constructing argument types from the initializers. If an initializer is a list or vector, we use the contained type.

Once we've got the lambda params we construct a LambdaExpression AST node using the params and the expression body. Then we construct an array of arguments for the call expression with the lambda as the first argument and mapping over node.vars again to get the initializers as the remaining arguments.

Then we construct a call expression from the operator and arguments and call infer on it.

Here's inferForExpression:

const inferForExpression = (node, env, constant) => {
  const lambdaArgs = node.vars.map((v) => {
    let varType = infer(v.initializer, env, constant);

    if (Type.isList(varType)) {
      varType = varType.listType;
    } else if (Type.isVector(varType)) {
      varType = varType.vectorType;
    }

    return { name: v.var, type: varType };
  });
  const lambda = AST.LambdaExpression(
    lambdaArgs,
    node.body,
    false,
    null,
    node.srcloc
  );
  const opArgs = [lambda, ...node.vars.map((v) => v.initializer)];

  return infer(AST.CallExpression(node.op, opArgs, node.srcloc), env, constant);
};

The process of constructing the CallExpression node in inferForExpression is similar to how we'll handle for expressions in the desugarer.

Next we need to handle the ForExpression node in src/typechecker/TypeChecker.js.

First, add a case to the switch statement in checkNode:

      // other cases...
      case ASTTypes.ForExpression:
        return this.checkForExpression(node, env);
      // default case

The checkForExpression method is pretty simple:

  checkForExpression(node, env) {
    const op = this.checkNode(node.op, env);
    return { ...node, op, type: infer(node, env) };
  }

That's it for changes to the type checker. It's a lot less than it's been in the past few articles where we've focused more on type checker features.

Now let's look at changes to the default visitor.

Changes to The Visitor

We need a dispatch case and default visitor for the ForExpression node in src/visitor/Visit.js.

First, add a case to the visit method's switch statement:

      // other cases...
      case ASTTypes.ForExpression:
        return this.visitForExpression(node);
      // default case

In visitForExpression we simply visit the operator, visit each variable and initializer in node.vars, and then visit each body expression.

Here's visitForExpression:

  visitForExpression(node) {
    const op = this.visit(node.op);

    let vars = [];

    for (let nodevar of node.vars) {
      const v = this.visit(nodevar.var);
      const initializer = this.visit(nodevar.initializer);

      vars.push({ var: v, initializer });
    }

    let body = [];

    for (let expr of node.body) {
      body.push(this.visit(expr));
    }

    return { ...node, op, vars, body };
  }

Now that there's a default visitor for the ForExpression node, we need to handle it in the desugarer.

Changes to The Desugarer

The process is similar to how we inferred a type for the expression: we construct a lambda, then construct a call expression with that lambda as its first argument.

Here's visitForExpression in src/desugarer/Desugarer.js:

  visitForExpression(node) {
    const op = this.visit(node.op);
    const lambdaArgs = node.vars.map((v) => ({ name: v.var }));
    const lambda = AST.LambdaExpression(
      lambdaArgs,
      node.body,
      false,
      null,
      node.srcloc
    );
    const callArgs = [lambda, ...node.vars.map((v) => v.initializer)];

    return AST.CallExpression(op, callArgs, node.srcloc);
  }

Now we don't need to worry about handling for expressions in the emitter! We do need to fix a bug I found, though.

Changes to The Emitter

The bug is in emitGlobalEnv in src/emitter/emitGlobalEnv.js.

When emitting the actual variable assignments, we currently don't emit the var keyword with them because we needed them to be globals in the REPL.

Now that we're emitting compiled files and have the option to emit a file that imports the global environment using ES2015 imports we need to change that slightly.

The reason is that ES2015 modules use strict mode by default, and in strict mode it throws an error if you assign to a variable without declaring it first with var, let, or const.

Since this is only an issue for compiled files, and not in the REPL or bundled files, we'll add var based on an optional boolean flag passed into the emitGlobalEnv function.

Here's the new version of emitGlobalEnv:

import path from "path";
import { ROOT_PATH } from "../../root.js";
import { makeGlobal } from "../runtime/makeGlobals.js";

export const emitGlobalEnv = (useVar = false) => {
  const globalEnv = makeGlobal();
  let code = `import { makeGlobal } from "${path.join(
    ROOT_PATH,
    "./src/runtime/makeGlobals.js"
  )}";
import { makeRuntime } from "${path.join(
    ROOT_PATH,
    "./src/runtime/makeRuntime.js"
  )}";

const globalEnv = makeGlobal();
${useVar ? "var " : ""}rt = makeRuntime();
`;

  for (let [k] of globalEnv) {
    code += `${useVar ? "var " : ""}${k} = globalEnv.get("${k}");\n`;
  }

  return code;
};

Now when we compile a file and run it as an ES2015 module we won't get an error because of undeclared variables.

Next we need to make a change to the AST printer to handle the new node type.

Changes to The Printer

We need to add the ForExpression node to the AST printer in src/printer/printAST.js.

First, let's add the case to the switch statement in the print method:

      // other cases...
      case ASTTypes.ForExpression:
        return this.printForExpression(node, indent);
      // default case

The printForExpression method is a little verbose because we have to iterate over both node.vars and node.body to print the subexpressions correctly.

Here's printForExpression:

  printForExpression(node, indent) {
    let prStr = `${prIndent(indent)}ForExpression:\n`;
    prStr += `${prIndent(indent + 2)}Operator:\n`;
    prStr += ` ${this.print(node.op, indent + 4)}\n`;
    prStr += `${prIndent(indent + 2)}Vars:\n`;

    for (let nodevar of node.vars) {
      prStr += `${prIndent(indent + 4)}Var: ${this.print(nodevar.var, 0)}\n`;
      prStr += `${prIndent(indent + 4)}Init: ${this.print(
        nodevar.initializer,
        0
      )}\n`;
    }

    prStr += `${prIndent(indent + 2)}Body:\n`;

    for (let expr of node.body) {
      prStr += this.print(expr, indent + 4) + "\n";
    }

    return prStr;
  }

That's it for changes to the printer! Now for expressions fully work within the language, so it's time to focus on our changes to the CLI.

Changes to The CLI

Remember, we're adding these new features to the CLI:

History (up to 2000 lines)
Help info for the CLI, the wandac compiler, and the REPL
The ability to load a Wanda file from within the REPL
The ability to save a REPL session as a file

In order to enable history, we're going to have to change how we get input in the REPL. Instead of using the readline-sync package, we're going to use Node's C++ API to directly access readline.

Don't worry, you won't have to write any C++ code to get this to work. There's a Node.js package that gives you access to the C++ API via JavaScript. Install the ffi-napi package with npm install ffi-napi.

The New Readline

We're going to put this in a new file in the CLI directory, src/cli/readline.js. This will handle both getting input and managing the history state when you fire up the REPL.

You'll need to import some dependencies:

import os from "os";
import fs from "fs";
import { join } from "path";
import ffi from "ffi-napi";

NOTE: The following code accesses libreadline directly, so I have no idea if it's cross platform. It probably isn't. I don't believe Windows includes libreadline by default, though I could be mistaken. I do my development, including the code for this series, on Linux so I haven't tried to make this run on Windows yet. If I do try it on Windows I'll post an update in a later article.

Ok, with that out of the way, first we use ffi to gain access to the readline and add_history primitives and store them in an object. We also set the path to the history file and a flag for if history has been loaded:

const rllib = ffi.Library("libreadline", {
  readline: ["string", ["string"]],
  add_history: ["int", ["string"]],
});

const HISTORY_FILE = join(os.homedir(), ".wanda-history");
let historyLoaded = false;

The readline function takes a prompt and returns the line given in response to the prompt. Most of what comes between those 2 things is managing history.

If the historyLoaded flag is false, we load the history. If the history file exists we read it, split on the end-of-line character, and filter out any blank lines.

If the history file doesn't yet exist or is blank, the array is empty.

Then we slice the remaining array so that a maximum of 2000 lines remain.

Next, loop over the array and add each line to the history.

After loading the history, we continue by prompting the user and getting text in reply to the prompt.

We add that line to the history, and append it to the end of the history file. Finally, return the text.

Here's the readline function:

export const readline = (prompt = ">") => {
  if (!historyLoaded) {
    let lines = [];

    if (fs.existsSync(HISTORY_FILE)) {
      lines = fs
        .readFileSync(HISTORY_FILE, { encoding: "utf-8" })
        .split(os.EOL)
        // remove blank lines
        .filter((line) => line !== "");
    }

    lines = lines.slice(Math.max(lines.length - 2000, 0));

    for (let line of lines) {
      rllib.add_history(line);
    }
  }

  const line = rllib.readline(prompt);

  if (line) {
    rllib.add_history(line);

    try {
      fs.appendFileSync(HISTORY_FILE, line + os.EOL, { encoding: "utf-8" });
    } catch (e) {
      // do nothing
    }
  }

  return line;
};

Getting The Version and Help Info

Next, in src/cli/utils.js, let's write 2 new functions: one to handle getting version info, and one to handle displaying help.

First, we need to import some values into the file:

import fs from "fs";
import { join } from "path";
import { ROOT_PATH } from "../../root.js";

getVersion retrieves the contents of package.json and parses them, then returns the version property.

export const getVersion = () => {
  const packageJson = JSON.parse(
    fs.readFileSync(join(ROOT_PATH, "./package.json"), {
      encoding: "utf-8",
    })
  );
  return packageJson.version;
};

printHelp shows a short introductory message, then loops over an object of commands and displays information based on the contents of each command's object.

A command's object looks like this:

{ alias?: string; description: string; usage?: string; }

Then after showing all the commands' info it shows a postscript message if one is included.

Here's printHelp:

export const printHelp = (commands, application, postscript = "") => {
  console.log(`**** ${application} v${getVersion()} help info ****`);
  console.log();
  console.log("Command:  |  Info:");
  console.log();

  for (let [name, command] of Object.entries(commands)) {
    console.log(`${name}`);
    command.alias && console.log(`             Alias: wanda ${command.alias}`);
    console.log(`             ${command.description}`);
    command.usage && console.log(`             Usage: ${command.usage}`);
  }

  console.log();
  postscript && console.log(postscript);
};

Commands

Ok, now we need help command objects for each of the REPL, the wanda CLI, and the wandac compiler.

Here are the commands for the REPL which should be at the top of src/cli/repl.js, including the new commands to load and save files in the REPL:

const COMMANDS = {
  ":quit": {
    description: "Quits the REPL with exit 0",
  },
  ":print-ast": {
    description:
      "Makes a printed representation of the AST show when you enter an expression",
  },
  ":print-ast -d": {
    description:
      "Like :print-ast, but shows the tree after the desugaring step, right before emitting code",
    usage: ":print-ast -d",
  },
  ":no-print-ast": {
    description: "Turns off AST printing if it's on",
  },
  ":save-file": {
    description: "Saves the current REPL session as a file",
    usage: "Prompts you for a path to save the file",
  },
  ":load-file": {
    description:
      "Loads the definitions from a file into the interactive session",
    usage: "Prompts you for a path to load the file from",
  },
  ":version": {
    description: "Prints the currently installed version of Wanda",
  },
  ":help": {
    description: "Shows this help message",
  },
};

Here are the commands for the wanda CLI, which you should put at the top of src/cli/run.js:

const COMMANDS = {
  load: {
    alias: "-l",
    description:
      "Loads a Wanda file into an interactive session so you can use its definitions",
    usage: "wanda load <filepath> or wanda -l <filepath>",
  },
  run: {
    alias: "-r",
    description: "Runs a Wanda file from the command line",
    usage: "wanda run <filepath> or wanda -r <filepath>",
  },
  repl: {
    alias: "-i",
    description: "Starts an interactive session with the Wanda REPL",
    usage: "wanda repl or wanda -i",
  },
  version: {
    alias: "-v",
    description: "Prints the current version number of your Wanda installation",
    usage: "wanda version or wanda -v",
  },
  help: {
    alias: "-h",
    description: "Prints this help message on the screen",
    usage: "wanda help or wanda -h",
  },
};

And here are the commands for the wandac compiler, which you should put at the top of src/cli/wandac.js:

const COMMANDS = {
  compile: {
    alias: "-c",
    description:
      "Compiles a single Wanda file to JavaScript that imports its dependencies",
    usage: "wandac compile <filepath> or wandac -c <filepath>",
  },
  build: {
    alias: "-b",
    description:
      "Compiles a Wanda file and builds it with bundled JavaScript dependencies",
    usage: "wandac build <filepath> or wandac -b <filepath>",
  },
  version: {
    alias: "-v",
    description:
      "Prints the current version number of your WandaC installation",
    usage: "wandac version or wandac -v",
  },
  help: {
    alias: "-h",
    description: "Prints this help message on the screen",
    usage: "wandac help or wandac -h",
  },
};

Changes to the REPL

Now let's go back to the REPL file.

First, make sure your imports look like this - there have been some changes:

import os from "os";
import vm from "vm";
import fs from "fs";
import { join } from "path";
import readlineSync from "readline-sync";
import { pprintAST, pprintDesugaredAST } from "./pprint.js";
import { println } from "../printer/println.js";
import { makeGlobalNameMap } from "../runtime/makeGlobals.js";
import { emitGlobalEnv } from "../emitter/emitGlobalEnv.js";
import { build } from "./build.js";
import { compile } from "./compile.js";
import { makeGlobalTypeEnv } from "../typechecker/makeGlobalTypeEnv.js";
import { countIndent, inputFinished } from "./utils.js";
import { readline } from "./readline.js";
import { getVersion, printHelp } from "./utils.js";

We're going to change the parameters the repl function takes. Instead of separate parameters for mode and file we're going to make them properties on a single options object and make them both optional.

Next, move these constants out of the repl function and put them at the top of the file:

// Create global compile environment
const compileEnv = makeGlobalNameMap();
const typeEnv = makeGlobalTypeEnv();

We need them to be at the module level now because we're going to use them in multiple functions.

You should now have this at the top of the repl function:

  // Build global module and instantiate in REPL context
  // This should make all compiled global symbols available
  const globalCode = build(emitGlobalEnv());
  vm.runInThisContext(globalCode);

Now we'll take all the code that was used to compile and run Wanda files when loading a file into an interactive session on invoking the wanda load <filename> command and move it to its own function, compileAndRunFromPath:

const compileAndRunFromPath = (path) => {
  const fileContents = fs.readFileSync(path, { encoding: "utf-8" });
  const compiledFile = compile(fileContents, path, compileEnv, typeEnv);
  vm.runInThisContext(compiledFile);
};

Then replace that code in the repl function with a call to the new function and pass it the path option:

  if (path) {
    // load file in REPL interactively
    compileAndRunFromPath(path);
  }

We need 2 additional new functions in src/cli/repl.js: one to get a file path from input in the REPL, and one to save a REPL session as its own file:

const saveAsFile = (session) => {
  const path = readlineSync.question("Enter the path to save the file at: ");
  const filePath = join(process.cwd(), path);

  try {
    fs.writeFileSync(filePath, session, { encoding: "utf-8" });
    console.log("File saved!");
  } catch (e) {
    console.log(
      `Error while saving file, please try again later: ${e.message}`
    );
  }
};

const getPathFromInput = () => {
  const path = readlineSync.question("Enter the path to load the file from: ");
  return join(process.cwd(), path);
};

Now, with the machinery in place, we turn back to the repl function.

Let's add a friendly welcome message with instructions on getting help that shows when you start a REPL session. Below the if (path) statement that calls compileAndRunFromPath, add this:

  console.log(
    `**** Welcome to the Wanda Programming Language v${getVersion()} interactive session ****`
  );
  console.log("Enter :help for more information");

Now we'll need to add a variable for the session contents to our series of variables just before the actual loop:

  let prompt = "> ";
  let input = "";
  let indent = 0;
  let session = "";

Next is the main loop with a nested try/catch. Here it is without the main contents:

  while (true) {
    try {
      // main contents
    } catch (e) {
      console.error(e.stack ? e.stack : e.message);
      input = "";
      indent = 0;
    }
  }

Now for the main contents. The first thing we need to do in the try block is read the input, indenting if it's multiline input:

      input += read(prompt + "  ".repeat(indent));

Now if the user enters a keyword that corresponds to a command we need to handle that. We'll extend our switch (input) statement to handle the new commands. Note that I've changed :print-desugared to just use a -d flag with the :print-ast command.

Here are the cases for the commands, switching on input:

        // If it's a command, execute it
        case ":quit":
          process.exit(0);
        case ":print-ast":
          mode = "printAST";
          input = "";
          break;
        case ":print-ast -d":
          mode = "printDesugared";
          input = "";
          break;
        case ":no-print-ast":
          mode = "repl";
          input = "";
          break;
        case ":save-file":
          saveAsFile(session);
          input = "";
          break;
        case ":load-file":
          compileAndRunFromPath(getPathFromInput());
          input = "";
          break;
        case ":version":
          console.log(getVersion());
          input = "";
          break;
        case ":help":
          printHelp(
            COMMANDS,
            "Wanda Interactive Session",
            "Enter an expression at the prompt for immediate evaluation"
          );
          input = "";
          break;

And the default case, which runs the code, remains the same except that we add completed input to the session variable and make sure incomplete input includes the indentation so the output files will have the same indentation as the REPL shows:

        // If it's code, compile and run it
        default:
          if (inputFinished(input)) {
            let compiled = compile(input, "stdin", compileEnv, typeEnv);
            let result = vm.runInThisContext(compiled);

            if (mode === "printAST") {
              console.log(pprintAST(input));
            } else if (mode === "printDesugared") {
              console.log(pprintDesugaredAST(input));
            }

            println(result);
            session += input + os.EOL + os.EOL;
            input = "";
            indent = 0;
          } else {
            indent = countIndent(input);
            input += os.EOL + "  ".repeat(indent);
          }

Or, if you need to see it all together, here's the complete repl function:

export const repl = ({ mode = "repl", path = "" } = {}) => {
  // Build global module and instantiate in REPL context
  // This should make all compiled global symbols available
  const globalCode = build(emitGlobalEnv());
  vm.runInThisContext(globalCode);

  if (path) {
    // load file in REPL interactively
    compileAndRunFromPath(path);
  }

  console.log(
    `**** Welcome to the Wanda Programming Language v${getVersion()} interactive session ****`
  );
  console.log("Enter :help for more information");

  let prompt = "> ";
  let input = "";
  let indent = 0;
  let session = "";

  while (true) {
    try {
      input += read(prompt + "  ".repeat(indent));

      switch (input) {
        // If it's a command, execute it
        case ":quit":
          process.exit(0);
        case ":print-ast":
          mode = "printAST";
          input = "";
          break;
        case ":print-ast -d":
          mode = "printDesugared";
          input = "";
          break;
        case ":no-print-ast":
          mode = "repl";
          input = "";
          break;
        case ":save-file":
          saveAsFile(session);
          input = "";
          break;
        case ":load-file":
          compileAndRunFromPath(getPathFromInput());
          input = "";
          break;
        case ":version":
          console.log(getVersion());
          input = "";
          break;
        case ":help":
          printHelp(
            COMMANDS,
            "Wanda Interactive Session",
            "Enter an expression at the prompt for immediate evaluation"
          );
          input = "";
          break;
        // If it's code, compile and run it
        default:
          if (inputFinished(input)) {
            let compiled = compile(input, "stdin", compileEnv, typeEnv);
            let result = vm.runInThisContext(compiled);

            if (mode === "printAST") {
              console.log(pprintAST(input));
            } else if (mode === "printDesugared") {
              console.log(pprintDesugaredAST(input));
            }

            println(result);
            session += input + os.EOL + os.EOL;
            input = "";
            indent = 0;
          } else {
            indent = countIndent(input);
            input += os.EOL + "  ".repeat(indent);
          }
      }
    } catch (e) {
      console.error(e.stack ? e.stack : e.message);
      input = "";
      indent = 0;
    }
  }
};

Now you can load and save files from inside the REPL.

Changes in Running The Wanda CLI

Since we're going to be running files from the command line now, we need to make sure we import everything needed for that. Here's the new list of imports in src/cli/run.js:

import vm from "vm";
import fs from "fs";
import { join } from "path";
import { repl } from "./repl.js";
import { makeGlobalNameMap } from "../runtime/makeGlobals.js";
import { emitGlobalEnv } from "../emitter/emitGlobalEnv.js";
import { build } from "./build.js";
import { compile } from "./compile.js";
import { makeGlobalTypeEnv } from "../typechecker/makeGlobalTypeEnv.js";
import { getVersion, printHelp } from "./utils.js";

We also need a new function runFile to compile and run the contents of a file passed to the CLI:

const runFile = (path) => {
  const fileContents = fs.readFileSync(path, { encoding: "utf-8" });
  const globalNs = makeGlobalNameMap();
  const typeEnv = makeGlobalTypeEnv();
  const globalCode = build(emitGlobalEnv());
  const compiledCode = compile(fileContents, path, globalNs, typeEnv);

  vm.runInThisContext(globalCode);
  return vm.runInThisContext(compiledCode);
};

In the run function, we'll have the command and alias cases fall through to a single handler for each command. I've also removed the option to start a REPL session with AST printing on because it makes more sense to me to have that just be an option you set when you're inside the interactive session.

Here's the new version of run. Note that I've also changed it from throwing exceptions on bad commands to just ending the process with an error code:

export const run = () => {
  switch (process.argv[2]) {
    case "-l":
    case "load":
      if (!process.argv[3]) {
        console.log(`load command requires file path as argument`);
        process.exit(1);
      }
      const path = join(process.cwd(), process.argv[3]);
      repl({ path });
      break;
    case "run":
    case "-r":
      if (!process.argv[3]) {
        console.log(`run command requires file path as argument`);
        process.exit(1);
      }
      return runFile(join(process.cwd(), process.argv[3]));
    case "-v":
    case "version":
      return console.log(getVersion());
    case "help":
    case "-h":
      return printHelp(
        COMMANDS,
        "Wanda Programming Language",
        "Just running wanda with no command also starts an interactive session"
      );
    case undefined:
    case "repl":
    case "-i":
      return repl();
    default:
      console.log("Invalid command specified");
      process.exit(1);
  }
};

Now you can run files directly from the command line.

Changes in Running The Compiler

First, we need to add an import for the new getVersion and printHelp functions:

import { getVersion, printHelp } from "./utils.js";

Most of the changes simply involve handling the new commands and aliases for the old commands, but we're also passing true into emitGlobalEnv in the default case because of the issue we fixed above. Here's the wandac function in src/cli/wandac.js:

export const wandac = () => {
  if (!process.argv[2]) {
    console.log(`wandac requires either a file path or command argument`);
    process.exit(1);
  }

  switch (process.argv[2]) {
    case "build":
    case "-b": {
      const pathname = join(process.cwd(), process.argv[3]);
      const compiledFile = compileFile(pathname);
      const globals = emitGlobalEnv();
      const code = globals + os.EOL + os.EOL + compiledFile;
      const bName = basename(pathname).split(".")[0];
      const outfile = bName + ".build" + ".js";
      const built = build(code, outfile, bName);

      fs.writeFileSync(outfile, built, { encoding: "utf-8" });
      break;
    }
    case "version":
    case "-v":
      getVersion();
      break;
    case "help":
    case "-h":
      printHelp(
        COMMANDS,
        "WandaC Compiler",
        "Just using wandac <filename> also compiles a single file"
      );
      break;
    default: {
      // should be a file path
      const pathname = join(process.cwd(), process.argv[2]);
      const compiledFile = compileFile(pathname);
      const globals = emitGlobalEnv(true);
      const code = globals + os.EOL + os.EOL + compiledFile;
      const outfile = basename(pathname).split(".")[0] + ".js";

      fs.writeFileSync(outfile, code, { encoding: "utf-8" });
      break;
    }
  }
};

Trying It Out

First, try out the new version and help commands with wanda and wandac. I think you'll agree they make the apps much more accessible than they used to be.

Now fire up a REPL session with wanda.

Enter this code into the REPL, one line at a time:

(for map ((i (range 5)))
  (+ i i))

Now enter :save-file and give it a filename at the prompt (I used examples/for.wanda). Your file should appear where you saved it!

You can also use the up arrow to cycle back through your history, and history is saved across sessions.

Try using the :load-file command and loading examples/inc.wanda from last time. Now you should be able to use the inc function just like if you'd defined it in your current session. Cool, right?

Conclusion

I hope this has been as fun for you as it's been for me.

As always, you can view the current state of the Wanda code as of the end of this article at the relevant tag in the GitHub repo.

Next time we're going to do something different. Instead of adding a new syntactic or type feature to Wanda, we're going to add an AST transformation and optimization.

We're going to implement an intermediate representation for the Wanda code between desugaring and code emitting called A Normal Form. I promise it's not as scary as it sounds.

We'll also add tail call optimization in the form of trampolining.

That will allow us to use essentially infinite recursion as long as the recursive call is in tail position. If you're not familiar with what that means, I'll explain all in the next article.

Stay tuned!

DEV Community