Neverlang

Questions and issues to Walter Cazzola.

DSLs are used to solve several problems, such as typesetting documents and code (TEX/LATEX, lout, ...), to express and verify constraints in several domains (OCL, iLOG CP, C4J, ...) and to coordinate the computation and/or to data query (Linda, SQL, ...). In some cases, these are simply a bunch of programming features useless standalone embedded in a general purpose programming language or provided as external libraries (e.g., Linda and SQL). In these case performances and flexibility are often compromised especially when the DSL is realized as program transformation towards another high-level programming language.

A DSL integrating features from different programming languages and paradigms would be the best choice to express a concise and clean solution to a problem avoiding a cumbersome one due to the absence of a specific feature or to a farraginous general-purpose programming language. Unfortunately, to develop an ad hoc programming language implies a considerable effort.

To simplify and speed up the development of problem-tailored programming languages we developed Neverlang. The Neverlang tool originally basically reflects the fact that programming languages have a sectional definition and each language feature can be easily plugged and unplugged. A complete compiler/interpreter built up with Neverlang is the result of a compositional process involving several building blocks.

In this scenario, to design a domain specific language consists of implementing a set of slices (each of them coding a single programming feature and the necessary support code, such as type checking and code generation) and composing them together. The whole structure of the compiler/interpreter is the result of the composition of such a slices, in particular of the code necessary to compile/interpret each single feature.

How does it work?

The framework basically provides: a language for writing the building blocks and a mechanism for composing the blocks together and for generating the compiler/interpreter.

The basic units composing a programming language developed by using Neverlang are modules. Each module encapsulates a specific feature of the language, e.g., a module can encapsulate the syntactically aspect of a loop, the type checking code of a comparison, or the code generation for a method call. Roles define how modules composing together forming the compiler/interpreter. syntax, type-checking and evaluation are examples of roles. Finally, modules regarding the same language structure but with different roles are grouped together in slices.

The example below shows how Neverlang can be used to define a if-else conditional construct. Three roles (syntax, typechecking and evaluation) are involved; each role is defined in a separate module and combined together in the if slice.

if.nl

   module if_syntax {
     role(syntax) {
       Statement <--
          'if' '(' ExprB ')' '{' StatementL '}' 'else' '{' StatementL '}'
       Statement <-- 'if' '(' ExprB ')' '{' StatementL '}'
     }
   }

   module if_typeChecking {
    role(type-checking) {
       0 {
         if (!$1.type.equals("Boolean")) System.err.println("ERROR: ");
       }
       4 {
         if (!$5.type.equals("Boolean")) System.err.println("ERROR: ");
       }
     }
   }

   module if_eval {
     role(evaluation) {
       0 {
         if (new Boolean($1.eval)) $2.eval;
         else $3.eval;
       }
       4 {
         if (new Boolean($5.eval)) $6.eval;
       }
     }
   }

   slice if {
     module if_syntax with role syntax
     module if_eval with role evaluation
     module if_typeChecking with role type_checking
   }

The first role (syntax) contains two productions that define the syntax of the if-else structure with or without the else branch. Each production is composed by: terminals (surrounded by ') and nonterminals (e.g., StatementL and ExprB). These productions are bound to the nonterminal Statement, other productions bound to such a nonterminal can be defined in the syntax module of other slices.

All the other kind of modules (in our example those with role type-checking and evaluation) add some semantics action to the grammar rules defined in the module with role syntax in the slice. Each of these modules associates to the nonterminals the semantic actions necessary to carry out the corresponding compilation/interpretation phase.

The nonterminals are identified through their position in the productions numbering with 0 the top leftmost nonterminal and incrementing by one left-to-right and up-to-down all the nonterminals independently of repetitions and for the whole set of productions defined in the slice. The heading of the action just figures out where the semantic action is anchored in the productions.

The first enriches the head of the first production (position 0) and simply tests the evaluation of the boolean expression associated to the nonterminal in position 1 (ExprB) and accordingly to that respectively evaluates the nonterminal in position 2 or 3 through their eval attribute. The second action similarly behaves but refers to the case without the else branch and it is associated to the head of the second production (position 4).

The semantic action associated that compose the type-checking role just checks that the test Expression has a Boolean type and print an error otherwise.

The three modules are composed in a slice, the user has to define which role is implemented by each imported module.

All the slices are finally composed together in the resulted language. The example below shows the composition of 7 slices in a basic DSL language able to perform some basic tests and print results.``if`` slice a part, the other slices implements the compare operators the string and int types the print command.

if_lang.nl

 language IfLess{
    slices core less_then more_then equals string int print if
    roles syntax < type-checking < eval
 }

This is an example of a source code written in the produced language:

if_else_01.txt

 if (aaaa<=23) {
    if (1>aaaa) { print(err) }
    print(ok)
 } else { print(err) }

To support the computation of the various the interpretation phases, the developer may need some ancillary structures or services that concerns the whole compilation process affecting all the other modules crosswise. Simple examples are the symbol table -- a static table used to check which types are comparable --, and the code to deal with the memory management. To support this kind of behavior, we introduced a slightly different form of slice called endemic.

The example below shows the implementation of a simple endemic slice that implements a simple Var Table to map the identifier with their values.

module VarTable {
   role(endemic){}
}

slice VarTable {
  decl {
    import java.util.*;
    Hashtable valueTable = new Hashtable();
  }
  module VarTable with role endemic
}

The fields and methods defined in an endemic slice are accessible by all the modules in the language independently of the compiling/interpreting phase. To add/replace an endemic slice permits to easily redefine the whole behavior of the interpreter.

This an example how the value of a variable is retrieved using the Identifier and the VarTable.

role(syntax){
   Var <-- Identifier
}

0 {
  String value = VarTable.valueTable.get($1.eval);
  if (value == null){
    Logger.printErr("Warning!!! The variable "+$1.value+" must be declared.");
    ErrorControl.typeErr=true;
  } else { $0.value=value; }
}

Installation

REQUIREMENTS

In order to run the package in the system must be present:

• Java SDK 6.0
• AspectJ 1.6
• some common bash commands: dirname, which.

Optionally

• astyle (http://astyle.sourceforge.net/astyle.html) in order to format the generated source code

INSTALLATION

1. Download the version v0.9ß of the framework (TGZ).
2. Just unpack the tgz file and optionally add to the PATH environment variable «unpack dir»/Neverlang/bin/ where «unpack dir» is the directory where the package has been unpacked.

USAGE

To use the tool we provide two bash scripts: nlgc and nlg. The former reads the modules and generates the interpreter; the latter uses the generated interpreter given as first argument to execute the program files specified by the remaining arguments.

Launching the command below neverlang produced the compiler as a package of class in IfLess directory.

[12:44]cazzola@thor:~>nlgc -d IfLess if_less/*.nl

Now with the nlg command is possible to use the compiler created to interpreter the simple source code contained in if_less_01.txt

[12:45]cazzola@thor:~>nlg IfLess if_less_01.txt
 ok

Other options can be specified in order to use external classes.

nlgc -d <destDir> [-j <jar packages>] [ -a <suppPackDir> ] [-k] <file>+

<file>

the file composing the grammar

-d <destDir>	Directory of where the interpreter will be created. The directory will be overwritten if it already exists
-a <suppPackDir>
	specify the directory containing the Java sources that need to be compiled with the generated source code in order to create the interpreter
-j <jar packages>
	specify the needed Java packages not contained in the classpath. Multiple jar files are separated with ':'.
-k	Keep the source code generated, the cade will be intended if the astyle tool is present in the system

nlg [-j jars] <dir> <file>+ [-a <args>*]

<dir>

the directory created by NeverLangC containing the interpreter to be executed.

<file>

the file containing the source to be executed by the interpreter.

-a <args>	The arguments that will be pass to the interpreter
-j <jar packages>
	specify the needed Java packages not contained in the classpath. Multiple jar files are separated with ':'.

Getting started with Neverlang

The package LP there are the neverlang modules to support a programming language that mixes the Linda coordination language and the functional part of Python. Two versions are provided; they differ on the provided implementation of the tuple space: one thread-based and the other distributed.

In order to run the examples:

• unpack the neverlang package.

[12:45]cazzola@thor:~>tar -xzvf neverlang-v0.9beta.tgz
 ok

• add to the PATH environment variable «unpack dir»/Neverlang/bin/ where «unpack dir» is the directory where the package has been unpacked.

[12:45]cazzola@thor:~>export PATH=$PATH:~/Neverlang/bin/
 ok

• add the ./ directory to the CLASSPATH environment variable.
• unpack the examples package and go to the test directory of the first example.

[12:45]cazzola@thor:~>tar -xzvf linda-python.tgz
[12:45]cazzola@thor:~>cd LindaPythonExamples/LindaPyCom/test/
[12:45]cazzola@thor:~>./test.sh
 Starting source generation ...
 Rats! Parser Generator, v. 1.14.1, (C) 2004-2008 Robert Grimm
 Processing LindaPyCom/ratsBin//Parser.rats ...
 Generation end!
 Adding Support pack from ../LindaPySupport/ to LindaPyCom
 Compiling the sources...
 Compilation ended
 Removing the source files
 Parsing matrixMultiplication.txt ...
....
 ok

Note that, to run the second example contained in LindaPyComRmi there must be only an instance of rmiregistry running.

A plugin for Eclipse and few language extensions are under development, as well.

Neverlang Staff

The Neverlang project is led by Walter Cazzola.

• Ivan Speziale, ideated of framework's back-end;
• Davide Poletti, developed the first version of the framework;

1. Walter Cazzola and Ivan Speziale, Sectional Domain Specific Languages, in Proceedings of the 4th Domain Specific Aspect-Oriented Languages (DSAL'09), Charlottesville, Virginia, USA, March 2009, pp. 11-14, ACM. [PDF1].
2. Walter Cazzola and Davide Poletti, DSL Evolution through Composition, in Proceedings of the 7th ECOOP Workshop on Reflection, AOP and Meta-Data for Software Evolution (RAM-SE'10), Maribor, Slovenia, June 2010, ACM. [PDF2].