Convert Java snippets to Xtend with Ctrl-Shift-V

Xtend is a dialect of Java which compiles into Java source code. The Xtend-integration in the Eclipse IDE allows developers to work simultaneously with Java and Xtend. As the syntax of both is similar, converting from Java to Xtend is not too hard, but it usually needs manual adjustments. Using the neat shortcut Ctrl-Shift-V (Ctrl+Shift+V/Cmd-Shift-V/Cmd+Shift+V) to paste Java code into Xtend files substantially reduces the manual editing. It helps to quickly get compilable Xtend code. Here are some examples of the automatic conversions.

Java type access:

EcoreUtil2.getContainerOfType(context, Model.class);

Pasted into an Xtend file:

EcoreUtil2.getContainerOfType(context, Model) 

The converter used the simpler Xtend syntax for accessing the type of Model and omitted the semicolon.

Java field with visibility:

private static final Logger logger = Logger.getLogger(
  CompositeEValidator.class);

Converted to Xtend:

static final Logger logger = Logger.getLogger(
  CompositeEValidator)

The default visibility for fields is private so the converter omitted it. Again, the short Xtend type access syntax was used.

Java name that is an Xtend keyword:

private String val;

Converted to Xtend:

String ^val

The Xtend keyword was automatically escaped.

Variable and cast in Java:

Model model = (Model) object;

Converted to Xtend:

var Model model = object as Model

The var/val keywords were added for mutable/final local variables and the Xtend type cast syntax was used (by the way, the type declaration of Model could be omitted due to Xtend's type inference).

Method in Java:

public void foo() { //...

Converted to Xtend:

def void foo() { //...

The default visibility for methods is public so it was omitted and the def keyword for method definitions was added.

Method overriding in Java:

public String toString() {

Converted to Xtend:

override String toString() {

As Xtend explicitly states overrides, the keyword was added (instead of def). Again, the visibility was omitted.

Default enum literals for Xtext generated EMF models

Xtext allows to easily build fully-fledged editors for domain specific languages (DSL) which are based on the Eclipse Modeling Framework (EMF). While existing Ecore models may be used, Xtext is also able to infer an Ecore model from the Xtext grammar. For complex Ecore models, it should certainly be considered to manage them independently, e.g. with Xcore. However, a few simple tweaks of grammar rules can influence the automatically generated Ecore model quite remarkably (e.g. the inheritance hierarchy) and may delay the point in time where it may be better to manage them independently. This example shows how to influence the generation of enumerators (enums) and default literals with grammar rules.

The code example below assumes that there's a model element with an optional "visibility" attribute of type enum and that the DSL must be able to capture the case where no visibility enum at all was given.

As the default literal of a generated enum is the first literal, an enum rule is added to the grammar which uses, say, "unspecified" for that. However, the user of the DSL editor should simply be able to enter nothing at all instead of "unspecified" which would look odd in the editor. This can be achieved by adding a second rule which returns the aforementioned enum, but only contains the literals allowed for the user. Please note that the order of the enum rules is important for generated models.

MyModelElement:
 // ...
 // user may specify enum or nothing at all
 visibility=Visibility? 
 // ...
;

// rule for the generated Ecore model
enum VisibilityEnum:
 unspecified | // first literal is the default
 public |
 private
;

// rule for the DSL editor
enum Visibility returns VisibilityEnum:
 public | 
 private
;

Multiple Validators in Xtext

Early validation of domain specific languages (DSLs) and suggestions on how to fix these errors ('quickfixes') are key for a good end user experience and for avoiding problems in the toolchain, for instance in generators that use DSL models: The more editor validations, the better. This description shows how to improve the maintainability and modularity of validation code for Xtext DSLs by simply splitting them. In the generated [MyDsl]JavaValidator, an annotation referring to other custom validators has to be added as shown below:

@ComposedChecks(validators =
        {MyCustomValidator.class, MyOthercustomValidator.class})
public class MyDslJavaValidator extends AbstractMyDslJavaValidator {

    // check method as usual inside the generated Java Validator
    @Check
    public void checkMyDslElement(MyEntity myEntity) { // ...

Please note that the custom validator has to override the register()-method. Apart from that, the checks look exactly as they would if they were in the generated validator.

// Example for a split custom validator written in Xtend
public class MyCustomValidator extends AbstractDeclarativeValidator {

    override register(EValidatorRegistrar registrar) {
        //not needed for classes used as ComposedCheck
    }

    // additional check method in separate validator
    @Check
    def void checkMyDslElement(MyEntity myEntity) {
       [...] // validation code

Clean Eclipse Preferences Tree with Multiple DSLs

As it is easy to build new domain specific languages (DSLs) with Xtext for Eclipse, enterprise projects may have multiple languages, which are usually related. Each language comes with its own Eclipse preference page, which are by default mixed with other preference pages. A simple way to keep the Eclipse preferences tree clean is to add a root page which aggregates the preference pages for individual (but possibly related) Xtext languages. This can be done by adding the snippet below to a plugin.xml file, for instance to the one of the base DSL's UI project.

<extension
        point="org.eclipse.ui.preferencePages">
    <page
            class="my.dsl.ui.MyDslExecutableExtensionFactory:org.eclipse.xtext.ui.editor.preferences.LanguageRootPreferencePage"
            id="my.id.root.ui"
            name="My DSLs">
        <keywordReference id="my.id.root.ui.keyword_root_pref"/>
    </page>
</extension>

<!-- add keywords for the search in the preferences page -->
<extension
point="org.eclipse.ui.keywords">
<keyword
        id="my.id.root.ui.keyword_root_pref"
        label="other keywords"/>
</extension>

Now, the id of the root page, in this case my.id.root.ui just has to be added to the plugin.xml files of the UI projects of the languages whose preferences should be aggregated. This can also be done in the graphical plugin.xml editor of the DSL's UI projects by navigating to the tab Extensions, selecting the first child node under org.eclipse.ui.preferencePages (which should be the DSL preference page) and pasting my.id.root.ui into the category text box.

Custom Syntax Error Messages with Quick Fix

Xtext editors for domain specific languages (DSLs) provide many error messages out of the box, such as syntactical errors, duplicate name errors or unresolvable references. For an improved user experience, some technical error messages from the editor (or, more specifically, from the Antlr parser that is used by the editor) may be customized. In many DSLs, identifiers (for DSL concepts like packages, entities and so on) are expected to conform to the regular expression of the terminal rule ID:

terminal ID : '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')*;

In addition, any keyword that is defined in other rules of the DSL grammar may not be used as an identifier. Keywords may be escaped with the caret (^) symbol, which is certainly arguable. This would be similar to using "class" as a name for a class in Java (if that would be possible). Here are some snippets of a DSL with a package concept:

package package // the second word is a reserved keyword and therefore not be valid as identifier
package ^package // okay, the keyword was escaped
package myPackage // okay unless 'myPackage' is a grammar keyword

The default error message when using a reserved keyword where an identifier is expected looks like this.

mismatched input 'package' expecting RULE_ID

This  message can be customized using Xtext's SyntaxErrorMessageProvider (written in Xtend):

class SyntaxErrorMessageProviderCustom extends SyntaxErrorMessageProvider { public static val String USED_RESERVED_KEYWORD = "USED_RESERVED_KEYWORD" @Inject IGrammarAccess grammarAccess /** * Customized error message for reserved keywords */ override getSyntaxErrorMessage(IParserErrorContext context) { val unexpectedText = context?.recognitionException?.token?.text if (GrammarUtil::getAllKeywords(grammarAccess.getGrammar()).contains(unexpectedText)) { println(context.defaultMessage) return new SyntaxErrorMessage(''' "«unexpectedText»" is a reserved keyword which is not allowed as Identifier. Please choose another word or alternatively confuse your co-workers by escaping it with the caret (^) character like this: "^«unexpectedText»".''', USED_RESERVED_KEYWORD) } super.getSyntaxErrorMessage(context) } }

The customized error message provider has to be bound in MyDslRuntimeModule.java like this:

/** * custom error messages for syntax errors */ public Class<X extends ISyntaxErrorMessageProvider> bindISyntaxErrorMessageProvider() { return SyntaxErrorMessageProviderCustom.class; }

A simple quickfix in MyDslQuickfixProvider could look like this:

/** * Provide a fix when reserved keywords are used as identifiers */ @Fix(SyntaxErrorMessageProviderCustom::USED_RESERVED_KEYWORD) def public void reservedKeywordUsed(Issue issue, IssueResolutionAcceptor acceptor) { val unexpectedText = issue.data?.get(0) acceptor.accept(issue, '''Change '«unexpectedText»' to '«unexpectedText.generateUniqueIdentifier».' ''', ''' Change '«unexpectedText»' to '«unexpectedText.generateUniqueIdentifier»', which is not a reserved keyword.''', "correction_linked_rename.gif", [ IModificationContext context | val xtextDocument = context.getXtextDocument xtextDocument.replace(issue.offset, issue.length, unexpectedText.generateUniqueIdentifier) ]) } def String generateUniqueIdentifier(String it) { val candidate = 'my' + it?.toFirstUpper?:'Name' var count = 1 val reserved = GrammarUtil::getAllKeywords(grammarAccess.getGrammar()) if (reserved.contains(candidate)) { while (reserved.contains(candidate + count)) { count = count + 1 } return candidate + count } return candidate }

This kind of customization has been available for a long time now. For more information, see Customizing error messages from Sebastian Zarnekow.

Xtext Grammar Visualization

When developing EBNF-style grammars, e.g. for Xtext, syntax graphs can be a great help. One of the new features in Xtext 2 is the Xtext Syntax Graph which can be selected in the Eclipse menu under Views - Xtext - Xtext Syntax Graph. In case the grammar has ambiguities, users get an error message similar to the following when trying to generate the code for the grammar by running the MWE2 workflow file.

Decision can match input such as [...] using multiple alternatives 1,2

If the error isn't obvious, the alternatives can be displayed graphically with ANTLRWorks (the graph analysis can be expensive and thus is not included in the standard Xtext Syntax Graph). Here is an example of an Eclipse Xtext project with a simple, ambiguous grammar and the Xtext Syntax Graph.

To analyze the grammar in ANTLRWorks, the executable jar file from the ANTLR website can be downloaded and run using the JRE (java -jar antlrworks-1.x.x.jar). ANTLRWorks expects the ANTLR grammar file from the Xtext project, which is called Internal[...].g. A debug-friendly version of this file may be generated by adding fragment = parser.antlr.DebugAntlrGeneratorFragment { } to your MWE2 workflow file and running it again.  The file is located in the Eclipse Xtext project in the src-gen folder in the [...].antlr.internal package (see first screenshot).
After opening it, the grammar can be checked by selecting Grammar - Check Grammar (Ctrl-R) from the menu. It will show an error message. Selecting the incorrect rule ruleModel and ticking both checkboxes for the alternatives (on the lower right on the screenshot) shows the ambiguity graphically.

Quickly formatting DSLs with Xtext

One typical task of developing new domain specific languages with Xtext is to customize the formatting in order to have a nice text layout. If the majority of grammar keywords should be formatted in the same way and only a few in a specific way, a generic formatter could be used. As an example, for DSLs with many key-value pairs (e.g. domain models), one might want to indent everything between curly braces and start a new line before each keyword. Instead of manually listing the keywords for the findKeywords()-method of the IGrammarAccess, GrammarUtil.getAllKeywords() can be used to get all the keywords, so a generic formatting method could look like this:

public class GenericFormatter {

    /**
     * In your implementation of
     * {@link org.eclipse.xtext.formatting.impl.AbstractDeclarativeFormatter#configureFormatting(org.eclipse.xtext.formatting.impl.FormattingConfig)}
     * you may call this generic formatting method first. It indents blocks between curly braces and sets a linewrap
     * before each keyword. Add your own behavior afterwards, e.g.
     * 
     * keywords = grammar.findKeywords(...);
     * for (final Keyword keyword : keywords) {
     * config.setNoLinewrap().before(keyword);
     * }
     */
    public static void genericFormatting(final FormattingConfig config, final IGrammarAccess grammar) {
        for (final Pair pair : grammar.findKeywordPairs("{", "}")) { //$NON-NLS-1$ //$NON-NLS-2$
            // a space before the first '{'
            config.setSpace(" ").before(pair.getFirst()); //$NON-NLS-1$
            // indentation between
            config.setIndentation(pair.getFirst(), pair.getSecond());
            // and a linewrap before the last '{'
            config.setLinewrap(1).before(pair.getSecond());
        }

        // linewrap before all keywords
        final Set allKeywords = GrammarUtil.getAllKeywords(grammar.getGrammar());
        final List keywords = grammar.findKeywords(allKeywords.toArray(new String[allKeywords.size()]));
        for (final Keyword keyword : keywords) {
            config.setLinewrap().before(keyword);
        }

    }
}

It could be called inside the configureFormatting()-method in [NameOfTheDSL]Formatter. Formatting code that overrides this behavior for specific keywords can be added after the call to the generic method like this:

keywords = grammar.findKeywords([keywords w/o wrap before]);
for (final Keyword keyword : keywords) {
    config.setNoLinewrap().before(keyword);
}

Migrating from Xtext 0.7.x to Xtext 1.0 in Eclipse Galileo (3.5)

Are you stuck with Eclipse Galileo in your project but still like to use the new features of Xtext 1.0? Xtext 1.0 works fine not only in the newest Eclipse Helios (3.6), but also in Eclipse Galileo (3.5). Galileo users can also benefit from Xtext 1.0 features. The eclipse plugins can be updated by using the Eclipse update mechanism and adding the Xtext update site http://download.itemis.com/updates/releases/ to the update manager.

After the update of the Xtext plugins, developers may refer to the Xtext documentation (Chapter 11. Migrating from Xtext 0.7.x to 1.0) for a manual on what has to be changed. If projects are heavily customized, the cleanest way to migrate, which is starting a new Xtext project and moving the artifacts from the old to the new projects, might not be desired and developers may prefer following the Migrating Step by Step guidelines. Before follwing these guidelines, I recommend to quickly create a sample project (New|Project...|Xtext Project) with the default settings and running the workflow (GenerateMyDsl.mwe2). This can be a handy reference, e.g. on the new folder structure for the recommended "Rename Package" refactoring. I also recommend to read section 11.2.5. Noteworthy API Changes before fixing compilation errors.

Grammars are backwards compatible. With Xtext 1.0, working on the grammar as a developer has become more convenient. Among other improvements, the content assist was enhanced and more hints for developers were added to prevent grammars with potentially unintended side-effects, e.g. if a rule contains only optional elements and may be consumed from the parser without object instantiation, the developer is given a hint to add an action that creates an object (see Actions in the Xtext documentation).

I was doing this migration for a heavily customized Xtext modeling project, and it turned out it took less effort than expected. The customer is happy with the new features of Xtext 1.0 and the complex project now loads in a bit more than half of the time it took before.

Xtext Quick Fix Variants

One of my favorite Eclipse features is the quick fix functionality. When there is an error in your source code, Eclipse may have a quick fix available - it shows up with a bulb symbol on the left editor margin and may offer you several actions to fix it. You can also use the shortcut [Ctrl-1] to activate it. Xtext ships with a quick fix API which makes it easy to provide quick fixes for your own DSL, so you can provide quick fixes for your customized validation errors and warnings, which is really nice.

Quick fix actions in Xtext may either manipulate the Xtext document directly, or modify the underlying semantic model, and Xtext takes care of changing the document. Isn't that neat? For pure text manipulation, the IModification interface can be used, while for working on the model you may implement ISemanticModification. The example below shows you how to use those two variants. In the example, we use a simple domain specific language for a tourist guide. The model may contain an arbitrary number of cities, and each city may contain zero or more sights, where a sight has a name and a description. We implemented two validation warnings: One checks whether the city's name starts with a capital letter, the other warns the user when a city doesn't have any sights. Quick fixes could be to capitalize the first letter (by changing the Xtext document) and, for the sake of the example, just add a generic sight, but this time by modifying the semantic model. To associate a quick fix with a certain validation, an identification code is used, so you have to define one for your validation like this:

public static final String INVALID_NAME = "xtext.workshop.advanced.quickfix.InvalidTypeName";

The check could look like this:

public class TouristguideDslJavaValidator extends
        AbstractTouristguideDslJavaValidator {
[...]

    @Check
    public void checkTypeNameStartsWithCapital(City city) {
        if (city.getName() == null || city.getName().length() == 0)
            return;
        if (!Character.isUpperCase(city.getName().charAt(0))) {
            warning("Name should start with a capital letter.",
                    TouristguideDslPackage.CITY__NAME, INVALID_NAME,
                    city.getName());
        }
    }
}

Note that the warning takes additional parameters (in the example only one is supplied) with "user data". Here you can supply an arbitrary number of Strings with user data that may be useful for the quick fix. Here is a step-by-step guide:

1. Import the plug-ins with the Touristguide Language (download examples). If you only want to try the quick fix but not implement it yourself, just download the finished projects where quick fixes are already implemented. Have a quick look at the Touristguide.xtext file in to understand how a valid .guide-file looks like.
2. Launch an Eclipse runtime application, create an new project, a new .guide-file and test if you get two validation warnings in the Problems view (Shift-Alt-Q X) for the following text:

   city "bonn" { }

3. Switch back to the Eclipse development environment workbench and review the validator TouristguideDslJavaValidator.java (you may cf. section 7.3 Quick Fixes of the Xtext 1.0.0 documentation)
4. Open the file TouristguideDslQuickfixProvider.java in the ui-Project. To implement the quick fix for the capital letters, you may implement methods like the ones below.

public class TouristguideDslQuickfixProvider extends DefaultQuickfixProvider {

    @Fix(TouristguideDslJavaValidator.INVALID_NAME)
    public void capitalizeName(final Issue issue,
                               IssueResolutionAcceptor acceptor) {
        // retrieve the 'user data' from the validation warning
        // upcase.png ... icon to display (in the icons folder)
        acceptor.accept(issue, "Capitalize name", "Capitalize the name \""
                        + issue.getData()[0] + "\".",
                "upcase.png", new IModification() {
                    public void apply(IModificationContext context)
                            throws BadLocationException {
                        IXtextDocument xtextDocument = context
                                .getXtextDocument();
                        String firstLetter = xtextDocument.get(
                                issue.getOffset() + 1, 1);
                        xtextDocument.replace(issue.getOffset() + 1, 1,
                                firstLetter.toUpperCase());
                    }
                });
    }

    @Fix(TouristguideDslJavaValidator.CITY_NOT_INTERESTING)
    public void addSightToCity(final Issue issue,
                               IssueResolutionAcceptor acceptor) {
        acceptor.accept(issue, "Add sight to make city more interesting",
                "Add a random sight, to make the city look more interesting.",
        
                // providing null for the icon name makes Eclipse use the
                // standard quick fix icon
                null, new ISemanticModification() {
                    public void apply(EObject element,
                                      IModificationContext context) throws Exception {
                        // we know that the warning applies to cities
                        City c = (City) element;
                        // programmatic modification of the model
                        Sight sight = TouristguideDslFactory.eINSTANCE
                                .createSight();
                        sight.setName("Central Station");
                        sight.setDescription("The famous central station of "
                                + Strings.toFirstUpper(c.getName()) + ".");
                        c.getSights().add(sight);
                        // Xtext automatically inserts text for the above
                    }
                });
    }
}

5. Restart the runtime workbench and have fun testing your new quick fixes :-)