Nicollet.Net

Specification attempt

This article attempts to chalk up a specification of the main syntax elements of the (meta)language being designed. The language codename chosen here is Xed.

Xed is an expression language (meaning that all syntax constructs in the language are expressions). It’s also a pure functional language, and although some expressions may appear to behave as side-effects for readability purposes they are ultimately implemented as pure functional operations.

Every expression in the language executes in a context. That context is composed of a table of local variables (with the associated values), including the special values _ and $, a list of persistent entities and relationships and their initial values (as read from persistent storage when the script is executed) and a table of modifications applied to those persistent entities (commited to the persistent storage when the script finishes executing).

Value types

Xed can manipulate several types of objects :

• Base scalar types (integers, booleans), support the common arithmetic and boolean operators.
• Base vector types (strings, streams), support size-of, access, shift/push, traversal and concatenation.
• Base associative types (hashes), support size-of, access, set/unset, traversal and merge. Setting a new binding for a value hides the old one instead of replacing it.
• Objects, mostly equivalent to hashes except that the types of values that are associated are not restricted to a single type.
• Variants, equivalent to the polymorphic variant type in Objective Caml.
• Persistent data types, used when defining relationships.
• Closures (including patterns).
• Names.

Syntax and semantics

The syntax, with its associated semantics, of all relevant elements are detailed below.

Object semantics

The most fundamental semantics are those of objects. An object is an association that binds arbitrary values to names. Adding a new binding merely hides the existing binding for that name, so that removing a binding restores the previous one.

• [obj.empty]
  The empty object : { } is an expression that creates an empty object.

• [obj.make]
  Create an object by defining its members, using shorthand notation. Uses the form { name = expr; … } to bind an expression to each member. This form is best used when defining objects that contain simple values.

• [obj.add.short]
  Add members to an object using shorthand notation. Uses the form expr .+ { name = expr ; … } to bind an expression to each member. This form is best used when extending objects that contain simple values.

• [obj.add.def]
  Add a member to an object using the definition syntax. Uses the form expr def name args = expr end to bind the second expression to the provided name in the object created by the first expression. If arguments are present, defines a closure. In the second expression, the special variable $ refers to the object created by the first expression.

• [obj.add.rec]
  Add a member to an object using the recursive definition syntax. Uses the form expr def rec name args = expr and name args = expr … end to add all the specified members to the object defined by the first expression. The special variable $ refers to the object being created, and not the old one.

• [obj.undef]
  Un-define a member of an object. Creates a new object where that member does not exist anymore. If an earlier definition was hidden by the undefined member, that definition becomes available again. Uses the syntax expr undef name, or the abbreviated syntax expr .- name. Causes an error if no such definition exists. This approach is used to implement private members and data hiding.

• [obj.undef.all]
  Un-define a member of an object completely. Creates a new object where that does not have that member anymore (effectively removes the member and all the members hidden by it). Uses the syntax expr undef-all name.

• [obj.ext.add]
  Add an object as a member of another object. expr object name = <text> end-object is equivalent to expr def name = { $ = $ } <text> end. This syntax is an extension that is not available by default.

• [obj.mem]
  Extracts the selected member, using the syntax expr.name. Generates an error if the member does not exist.

Closure semantics

Closures carry with them state information extracted from the context. They do not, however, keep the list of modifications to the persistent state, which is provided to them when they are called and not when they are created. All local variables referring to the above context are carried over.

• [fun.make]
  Create a new closure, using the syntax { args -> expr }. There is no limit on the number of arguments, but pattern matching is not allowed.

• [fun.match]
  Create a new closure, using the syntax { pattern -> expr | pattern -> expr }. This function accepts a single argument, but that argument is pattern-matched. Patterns may be either variant type patterns or regular expression patterns, which bind variables that are available in the expressions.

• [fun.call]
  A function is called on an argument using the syntax expr expr.

Pipe semantics

A lot of operations are performed using pipes, which serve as a shorthand notation for many elementary operations.

• [pipe.call]
  A pipe can be used to call a function on an argument. For instance, a |> f is rewritten to f a. The exception is if f contains the special variable _, at which point it is rewritten as if it were equivalent to { _ -> f } a. This specificity of the right-hand operator is called the implicit lambda rule.

• [pipe.map]
  The expression s |: f, where s is a stream or hash, creates a new stream or hash (the same as the input) by applying f to every element of s. This expression is subject to the implicit lambda rule. If the input was a hash, the key-element association is unchanged.

• [pipe.filter]
  The expression s |? f keeps only the elements of the input stream or hash such that f x is true. This expression is subject to the implicit lambda rule. The key-element association of hashes is kept.

• [pipe.hash.filter]
  The expression s |# f filters a hash based on its keys instead of its values.

Note that the combination of the pipe.call operation and the fun.math operation allows an operation equivalent to the match-with or switch-case pattern-matching constructs.

Name, Variant, Exception semantics

The universal variant type works on the same basis as Objective Caml polymorphic variants. Exceptions are thrown and caught based on their name, and they may carry associated values. They are ultimately members of the variant type, and so they do not have to be defined.

• [name.fresh]
  Creates a fresh name, different from all the other names in the program. The syntax is simply fresh.

• [name.lit]
  Creates a name literal. The syntax is `name`

• [name.use]
  In any situation where a (syntactical) name is expected and a (semantical) name is available, one can use the approach (name) to instantiate it. For instance, obj undef (name) or `new[(name)]` or even `(name)`.

• [var.make]
  A new member of a variant type is created with expr::name, where the name can be chosen on the spot (no definition required). This may be used in pattern-matching expressions. Note that in orderto support variants bound to an unit value, a cleaner syntax is :name (equivalent to ()::name).

• [exn.throw]
  Throws a variant object as an exception. The syntax is merely throw expr. The current execution context is completely lost (with the exception of any values referenced by the thrown expression), meaning in particular that any modifications applied to the persistent data model are thrown away.

• [exn.catch]
  Catches an exception if it matches a certain variant name, using the syntax try expr catch { pattern -> expr | … }. If an exception is thrown by the first expression, then pattern-matching finds the corresponding expression. If one is found, it’s evaluated in the context of the try-catch expression. Otherwise, the exception is propagated upwards.

Note that the language is fully exception-safe: when an exception is thrown, no trace of the execution of the throwing code is left behind, except whatever is saved by the exception itself. This means that any side-effects are automatically rolled back, removing the need for a “finally” clause. Another important element is the fact that all exceptions must be caught before the escape the script (meaning no script will be interrupted by an unexpected exception).

Pattern semantics

Patterns are functions which map objects to objects. Such functions can be created and called by basic means, but the following syntactic sugar is available:

• [pattern.ext.def]
  Defines a pattern as a member of an object. The expression is expr pattern name args = <text> end-pattern is equivalent to expr def name args $ = $ <text> end. This syntax is an extension that is not available by default.

• [pattern.ext.use]
  Uses a pattern of an object. The expression is expr use-pattern name args and is equivalent to expr |> name args. This syntax is an extension that is not available by default.