How to implement your DSL with RSyntaxtTextArea
This is a chapter about how to implement your DSL with RSyntaxtTextArea and is written without tables, figures, and footnotes even thought they are referenced. This will be added in a later version. Sorry for the inconvenience.
Implementing a custom domain-specific language (DSL) with a IDE or text editor may be a time consuming task. Using the model generated by Eclipse Modeling Framework (Eclipse MF) will make that easier. As last chapter mention, you could use the plug-in that Eclipse MF generates, but this paper is about how to implement a DSL with a custom workbench. In this case it will be showed by using RSyntaxTextArea (RSTA) by FifeSoft (http://www.fifesoft.com/). The aim is to implement the model generated by Eclipse EM with RSTA with changing as little as possible of the RSTA base code. Because of this we are depending on using JFlex to define the syntax highlightning. This may be seen by a drawback for RSTA as you have now two places to manage the keywords used in the language. Since a change in the Ecore-digram and CS-file will potensially affect the JFlex-file. Luckily this is a minor drawback compared to what RSTA gives us for free as a framework.
To implement your DSL into RSTA you need the model code and model editor generated from Eclipse MF, and RSyntaxTextArea and RSTALanguageSupport (which in turn need the AutoComplete library). The RSTA files can all be downloaded from FifeSoft. From the AutoComplete only the JAR is needed. Other then that JFlex is needed to generate the Java-parser for the syntax highlightning. The whole process of implementing the Eclipse model into RSTA is a comprehensive task, and it will be split into different task. To make it easier to follow and understand.
- Start of by separating all into different folders, so that you have it all nicely sorted. This will make it easier to update each of the individually part if you need to expand your language, or there is an new update for RSTA. As mention earlier you only need the JAR of AutoComplete, so that folder is not really necessary after you’ve used ANT to make a JAR.
- Then we will continue to make the syntax highlightning work. In RSTA this is done by a JFlex file with your keywords linked to the different keywords supported in RSTA. This can of course be modified so that you can add more colors (see image for example JFlex file). This is often in editors called a mode, and can be found in the mode folder inside RSyntaxTextArea. Next you need to parse the JFlex-file to generate a Java-file and remove two methods that’s been added twice. “This implementation was created using <a href=”http://www.jflex.de/”>JFlex</a> 1.4.1; however, the generated file was modified for performance. Memory allocation needs to be almost completely removed to be competitive with the handwritten lexers (subclasses of <code>AbstractTokenMaker</code>, so this class has been modified so that Strings are never allocated (via yytext()), and the scanner never has to worry about refilling its buffer (needlessly copying chars around). We can achieve this because RText always scans exactly 1 line of tokens at a time, and hands the scanner this line as an array of characters (a Segment really). Since tokens contain pointers to char arrays instead of Strings holding their contents, there is no need for allocating new memory for Strings.” In this case you need to remove the second definition of the zzRefill() and yyreset().
- Next before highlightning is done is to add your chosen syntax to the file SyntaxConstant.java in package org.fife.ui.rsyntaxtextarea. When this is done, what is left of RSyntaxTextArea is to make a JAR for the RSTALanguageSupport. As with the AutoComplete this is done with ANT.
- Rest of the implementation of the Eclipse model is done inside RSTALanguageSupport. API specification used for the code completion in RSTA can be added to the data folder following the syntax of CompletionXml.dtd. Examples can be seen by looking into one of the many languages already there. Also before continue be sure to add all necessary libraries to the ANT-file.
- Depending on what kind of features you want, there are few steps to take just to implement your own DSL. Start by adding the DSL to the language drop-down menu in DemoRootPane, so that it is easily loadable when the editor is running. This means that you need to have added a text-file with a sample of your DSL code. For example an Hello World. If a source tree is wanted for an editor take a look at step X.
- Next up is to add your languages parser, and that means adding your own language factory. This is done in two steps, first add your language to the LanguageSupportFactory.java, and then creating a folder for your language, and your own language support. Preferably DSL/DSLLanguageSupport.java. For reference there are sample language (should probably choose one who looks similar to your DSL or the XMLLanguageSupport.java as it is sort and fairly easy to comprehend). The main job for the LanguageSupport is to link the languages parser (not the Eclipse parser per se) with the RSTA.
- When creating the parser-file you need to be sure to import the Eclipse model. Then you need to build the class extending the AbstractParser class and implement the empty methods, for reference you should as mention look at some of the sample languages. Things to remember is to register your file-extension with the EPackage.Registry and the Resource.Factory.Registry (see example image with ThingML). Without this Java will not be able to load the DSL files for either parsing or compiling.
- Every time a user types something into the textarea (called RTextArea (footnote: RTextArea is a modified JTextArea with added features)), a method in the LanguageSupport called parse(RSyntaxTextDocument, String) is called. So inside this method, to start parsing the document that RSTA creates of the written code it need to be converted to an InputStream that the ResourceSetImpl can load. When loaded there are two main option to run, first the getErrors()-method need to be called to retrieve any syntax-errors. This should throw an ParseException to avoid crashing the loader later. The error-handling will be dealt with in the next step (X). If there are no syntax-errors the next step is to try to find context-errors using the resolveAll(). This will try to simulate run-time of the code. If there are any syntax-errors they can be retrieved by yet again calling the getErrors()-method, which will return a list of Diagnostics objects. This in turn need to be converted into DefaultParserNotice which is what RSTA uses to underline errors in the RTextArea. Every notice need to be added to the ParseResult object that the parse()-method returns. As stated earlier, if there are some context error the parser should throw a ParseException which should be caught inside the parse()-method. And it should create DefaultParseNotice similar to the context-context error, but with the result level as ParserNotice.ERROR. After all parsing is done, a firePropertyChange need to be called.
These steps are all that are needed to get your DSL up and running with RSTA with syntax highlightning, simple auto complete, and syntax and context error underlining. See step X for source tree, and step X to utilize the built in content assist for Eclipse.
- Adding the source tree can mainly be done in two ways. Here we will only cover the easiest of the two. In which case you need to create an own parser that will build the tree from scratch using tree nodes that extends SourceTreeNode. A good source tree example is the XML one, as it’s small with no extra features.
- The work inside the tree building class or method is up to you, as long as the AbstractSoruceTree (explained in the next task) can reach the root of the tree.
- Before testing the tree builder a DefaultCellTreeRenderer and a AbstractSourceTree need to be built. The initial CellRenderer is there to populate the tree and to avoid using Swing’s HTML rendering engine. As it is very slow. Still, there will be problems regarding speed with huge trees. The AbstractSourceTree (which is often named DSLOutlineTree.java) help you manage the tree and textarea interaction. In other words, when to update the tree, when to highlight words marked in the tree and so forth. This class will also retrieve and build a new tree when necessary.
- When all of these four separated classes have been made, the next step is to implement with the RTextArea, so that it gets generated when a user type in code in your new language. This is done in the refreshSourceTree() method inside the RootPane class.
This is all that needs to be done to implement a simple source tree for any given language. The only feature missing in RSTA in regards to source trees are the possibility of marking a word in the textarea and at the same time get it selected in the tree.
Last we have one of the useful features in RSTA, the content assist or auto-completion as the library is called in RSTA. In RSTA content assist is done in two ways (which can be mixed). There can be an XML-file with information about your language and keywords (as mention earlier), or by implementing them in code or while the program is running.
- Implementing code assistant by XML is explained in task X, so let us look at how to do it by code, and how to add the content assist already generated by Eclipse. Some code completion is already implemented in code as seen in three different CompletionProvider method; string, comment, and code. So if you only need simple completion, these can be added directly here.
- Adding content assist from the Eclipse model is a little bit more trickier. This is because you need to override the keystroke listener inside the inputmap. See an example in table X. This should be implemented using an AbstractAction. This ContentAssistAction need to get a loaded resource from the Eclipse model. See table X for an example on how it can be done. It is important to remember that the doCompletion() is called last, as this method triggers the content assist window. Letting the user see what can be selected.
In this chapter we have seen how to implement your own domain-specific language designed in Eclipse MF and EMFText with RSyntaxTextArea. The features we’ve seen now are the features that RSTA support out-of-the-box. File handling need to be implemented by the developer, but using the Java and Swing framework makes this easy.