Skip to content

Latest commit

 

History

History
933 lines (736 loc) · 29.7 KB

README.adoc

File metadata and controls

933 lines (736 loc) · 29.7 KB
Raw

CREST

Command-line API styled after JAX-RS

CREST allows you to get to the real work as quickly as possible when writing command line tools in Java.

  • 100% annotation based

  • Use Bean Validation on use input

  • Contains Several builtin validations

  • Generates help from annotations

  • Supports default values

  • Use variable substitution on defaults

  • Supports lists and var-ags

  • Supports any java type, usually out of the box

Simply annotate the parameters of any Java method so it can be invoked from a command-line interface with near-zero additional work. Command-registration, help text and validation is taken care of for you.

Example

For example, to do something that might be similar to rsync in java, you could create the following method signature in any java object.

package org.example.toolz;

import org.tomitribe.crest.api.Command;
import org.tomitribe.crest.api.Default;
import org.tomitribe.crest.api.Option;

import java.io.File;
import java.net.URI;
import java.util.regex.Pattern;

public class AnyName {

    @Command
    public void rsync(@Option("recursive") boolean recursive,
                      @Option("links") boolean links,
                      @Option("perms") boolean perms,
                      @Option("owner") boolean owner,
                      @Option("group") boolean group,
                      @Option("devices") boolean devices,
                      @Option("specials") boolean specials,
                      @Option("times") boolean times,
                      @Option("exclude") Pattern exclude,
                      @Option("exclude-from") File excludeFrom,
                      @Option("include") Pattern include,
                      @Option("include-from") File includeFrom,
                      @Option("progress") @Default("true") boolean progress,
                      URI[] sources,
                      URI dest) {

        // TODO write the implementation...
    }
}

Some quick notes on @Command usage:

  • Multiple classes that use @Command are allowed

  • Muttiple @Command methods are allowed in a class

  • @Command methods in a class may have the same or different name

  • The command name is derived from the method name if not specified in @Command

Executing the Command

Pack this class in an uber jar with the Crest library and you could execute this command from the command line as follows:

$ java -jar target/toolz-1.0.0-SNAPSHOT.jar rsync
Missing argument: URI...

Usage: rsync [options] URI... URI

Options:
  --devices
  --exclude=<Pattern>
  --exclude-from=<File>
  --group
  --include=<Pattern>
  --include-from=<File>
  --links
  --owner
  --perms
  --no-progress
  --recursive
  --specials
  --times

Of course, if we execute the command without the required arguments it will error out. This is the value of Crest — it does this dance for you.

In a dozen and more years of writing tools on different teams, two truths seem to prevail:

  • 90% of writing scripts is parsing and validating user input

  • Don’t do that well and you’ll be lucky if it gets more than six months of use

Computers are easy, humans are complex. Let Crest deal with the humans, you just write code.

Help Text

In the above example we have no details in our help other than what can be generated from inspecting the code. To add actual descriptions to our code we simply need to put an OptionDescriptions.properties in the same package as our class.

#code
# <option> = <description>
# <command>.<option> = <description>
# The most specific key always wins

recursive      = recurse into directories
links          = copy symlinks as symlinks
perms          = preserve permissions
owner          = preserve owner (super-user only)
group          = preserve group
times          = preserve times
devices        = preserve device files (super-user only)
specials       = preserve special files
exclude        = exclude files matching PATTERN
exclude-from   = read exclude patterns from FILE
include        = don't exclude files matching PATTERN
include-from   = read include patterns from FILE
progress       = this is not the description that will be chosen
rsync.progress = don't show progress during transfer

Some quick notes on OptionDescription.properties files:

  • These are Java java.util.ResourceBundle objects, so i18n is supported

  • Use OptionDescription_en.properties and similar for Locale specific help text

  • In DRY spirit, every @Command in the package shares the same OptionDescription ResourceBundle and keys

  • Use <command>.<option> as the key for situations where sharing is not desired

With the above in our classpath, our command’s help will now look like the following:

$ java -jar target/toolz-1.0.0-SNAPSHOT.jar rsync
Missing argument: URI...

Usage: rsync [options] URI... URI

Options:
  --devices                 preserve device files (super-user only)
  --exclude=<Pattern>       exclude files matching PATTERN
  --exclude-from=<File>     read exclude patterns from FILE
  --group                   preserve group
  --include=<Pattern>       don't exclude files matching PATTERN
  --include-from=<File>     read include patterns from FILE
  --links                   copy symlinks as symlinks
  --owner                   preserve owner (super-user only)
  --perms                   preserve permissions
  --no-progress             don't show progress during transfer
  --recursive               recurse into directories
  --specials                preserve special files
  --times                   preserve times

@Default values

Setting defaults to the @Option parameters of our @Command method can be done via the @Default annotation. Using as simplified version of our rsync example, we might possibly wish to specify a default exclude pattern.

@Command
public void rsync(@Option("exclude") @Default(".*~") Pattern exclude,
                  @Option("include") Pattern include,
                  @Option("progress") @Default("true") boolean progress,
                  URI[] sources,
                  URI dest) {

    // TODO write the implementation...
}

Some quick notes about @Option:

  • @Option parameters are, by default, optional

  • When @Default is not used, the value will be its equivalent JVM default — typically 0 or null

  • Add @Required to force a user to specify a value

Default values will show up in help output automatically, no need to update your OptionDescriptions.properties

Usage: rsync [options] URI... URI

Options:
  --exclude=<Pattern>      exclude files matching PATTERN
                           (default: .*~)
  --include=<Pattern>      don't exclude files matching PATTERN
  --no-progress            don't show progress during transfer

@Option Lists and Arrays

There are situations where you might want to allow the same flag to be specified twice. Simply turn the @Option parameter into an array or list that uses generics.

@Command
public void rsync(@Option("exclude") @Default(".*~") Pattern[] excludes,
                  @Option("include") Pattern include,
                  @Option("progress") @Default("true") boolean progress,
                  URI[] sources,
                  URI dest) {

    // TODO write the implementation...
}

The user can now specify multiple values when invoking the command by repeating the flag.

$ java -jar target/toolz-1.0.0-SNAPSHOT.jar rsync --exclude=".*\.log" --exclude=".*\.iml"  ...

@Default @Option Lists and Arrays

Should you want to specify these two exclude values as the defaults, simply use a comma , to separate them in @Default

@Command
public void rsync(@Option("exclude") @Default(".*\\.iml,.*\\.iml") Pattern[] excludes,
                  @Option("include") Pattern include,
                  @Option("progress") @Default("true") boolean progress,
                  URI[] sources,
                  URI dest) {

}

If you happen to need comma for something, use tab \t instead. When a tab is present in the @Default string, it becomes the preferred splitter.

@Command
public void rsync(@Option("exclude") @Default(".*\\.iml\t.*\\.iml") Pattern[] excludes,
                  @Option("include") Pattern include,
                  @Option("progress") @Default("true") boolean progress,
                  URI[] sources,
                  URI dest) {

}

If you happen to need both tab and comma for something (really????), use unicode zero \u0000 instead.

@Command
public void rsync(@Option("exclude") @Default(".*\\.iml\u0000.*\\.iml") Pattern[] excludes,
                  @Option("include") Pattern include,
                  @Option("progress") @Default("true") boolean progress,
                  URI[] sources,
                  URI dest) {

}

@Default and ${variable} Substitution

In the event you want to make defaults contextual, you can use ${some.property} in the @Default string and the java.lang.System.getProperties() object to supply the value.

@Command
public void hello(@Option("name") @Default("${user.name}") String user) throws Exception
    System.out.printf("Hello, %s%n", user);
}

Return Values

In the above we wrote to the console, which is fine for simple things but can make testing hard. So far our commands are still POJOs and nothing is stopping us from unit testing them as plain java objects — except asserting output writen to System.out.

Simply return java.lang.String and it will be written to System.out for you.

@Command
public String hello(@Option("name") @Default("${user.name}") String user) throws Exception
    return String.format("Hello, %s%n", user);
}

In the event you need to write a significant amount of data, you can return org.tomitribe.crest.api.StreamingOutput which is an exact copy of the equivalent JAX-RS StreamingOutput interface.

@Command
public StreamingOutput cat(final File file) {
    if (!file.exists()) throw new IllegalStateException("File does not exist: " + file.getAbsolutePath());
    if (!file.canRead()) throw new IllegalStateException("Not readable: " + file.getAbsolutePath());
    if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());

    return new StreamingOutput() {
        @Override
        public void write(OutputStream output) throws IOException {
            final InputStream input = new BufferedInputStream(new FileInputStream(file));
            try {
                final byte[] buffer = new byte[1024];
                int length;
                while ((length = input.read(buffer)) != -1) {
                    output.write(buffer, 0, length);
                }
                output.flush();
            } finally {
                if (input != null) input.close();
            }
        }
    };
}

Note a null check is not necessary for the File file parameter as Crest will not let the value of any plain argument be unspecified. All parameters which do not use @Option are treated as required

Stream injections

Command are often linked to console I/O. For that reason it is important to be able to interact with Crest in/out/error streams. They are provided by the contextual Environment instance and using its thread local you can retrieve them. However to make it easier to work with you can inject them as well.

Out stream (out and error ones) needs to be PrintStream typed and input is typed as a InputStream. Just use these types as command parameters and decorate it with @In/@Out/@Err:

public class IOMe {
    @org.tomitribe.crest.api.Command
    public static void asserts(@In final InputStream in,
                               @Out final PrintStream out,
                               @Err PrintStream err) {
        // ...
    }
}
Note
 using a parameter typed Environment you’ll get it injected as well but this one is not in crest-api.

Custom Java Types

You may have been seeing File and Pattern in the above examples and wondering exactly which Java classes Crest supports parameters to @Command methods. The short answer is, any. Crest does not use java.beans.PropertyEditor implementations by default like libraries such as Spring do.

After nearly 20 years of Java’s existence, it’s safe to say two styles dominate converting a String into a Java object:

  • A Constructor that take a single String as an argument. Examples:

    • java.io.File(String)

    • java.lang.Integer(String)

    • java.net.URL(String)

  • A static method that returns an instance of the same class. Examples:

    • java.util.regex.Pattern.compile(String)

    • java.net.URI.create(String)

    • java.util.concurrent.TimeUnit.valueOf(String)

Use either of these conventions and Crest will have no problem instantiating your object with the user-supplied String from the command-line args.

This should cover 95% of all cases, but in the event it does not, you can create a java.beans.PropertyEditor and register it with the JVM. Use your Google-fu to learn how to do that.

The order of precedence is as follows:

  1. Constructor

  2. Static method

  3. java.beans.PropertyEditor

Custom Validation

If we look at our cat command we had earlier and yank the very boiler-plate read/write stream logic, all we have left is some code validating the user input.

@Command
public StreamingOutput cat(final File file) {
    if (!file.exists()) throw new IllegalStateException("File does not exist: " + file.getAbsolutePath());
    if (!file.canRead()) throw new IllegalStateException("Not readable: " + file.getAbsolutePath());
    if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());

    return new StreamingOutput() {
        @Override
        public void write(OutputStream os) throws IOException {
            IO.copy(file, os);
        }
    };
}

This validation code, too, can be yanked. Crest supports the use of Bean Validation to validate @Command method parameters.

@Command
public StreamingOutput cat(@Exists @Readable final File file) {
    if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());

    return new StreamingOutput() {
        @Override
        public void write(OutputStream os) throws IOException {
            IO.copy(file, os);
        }
    };
}

Here we’ve eliminated two of our very tedious checks with Bean Validation annotations that Crest provides out of the box, but we still have one more to get rid of. We can eliminate that one by writing our own annotation and using the Bean Validation API to wire it all together.

Here is what an annotation to do the file.isFile() check might look like — let’s call the annotation simply @IsFile

package org.example.toolz;

import javax.validation.ConstraintValidator;
import javax.validation.ConstraintValidatorContext;
import javax.validation.Payload;
import java.io.File;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;

import org.tomitribe.crest.val.Exists;

import static java.lang.annotation.ElementType.ANNOTATION_TYPE;
import static java.lang.annotation.ElementType.FIELD;
import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.PARAMETER;
import static java.lang.annotation.RetentionPolicy.RUNTIME;

@Exists
@Documented
@javax.validation.Constraint(validatedBy = {IsFile.Constraint.class})
@Target({METHOD, FIELD, ANNOTATION_TYPE, PARAMETER})
@Retention(RUNTIME)
public @interface IsFile {
    Class<?>[] groups() default {};

    String message() default "{org.exampe.toolz.IsFile.message}";

    Class<? extends Payload>[] payload() default {};

    public static class Constraint implements ConstraintValidator<IsFile, File> {

        @Override
        public void initialize(IsFile constraintAnnotation) {
        }

        @Override
        public boolean isValid(File file, ConstraintValidatorContext context) {
            return file.isFile();
        }
    }
}

We can then update our code as follows to use this validation and eliminate all our boiler-plate.

@Command
public StreamingOutput cat(@IsFile @Readable final File file) {

    return new StreamingOutput() {
        @Override
        public void write(OutputStream os) throws IOException {
            IO.copy(file, os);
        }
    };
}

Notice that we also removed @Exists from the method parameter? Since we put @Exists on the @IsFile annotation, the @IsFile annotation effectively inherits the @Exists logic. Our @IsFile annotation could inherit any number of annotations this way.

As the true strength of a great library of tools is the effort put into ensuring correct input, it’s very wise to bite the bullet and proactively invest in creating a reusable set of validation annotations to cover your typical input types.

Pull requests are very strongly encouraged for any annotations that might be useful to others.

Maven pom.xml setup

The following sample pom.xml will get you 90% of your way to fun with Crest and project that will output a small uber jar with all the required dependencies.

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>toolz</artifactId>
  <version>0.3-SNAPSHOT</version>

  <dependencies>
    <dependency>
      <groupId>org.tomitribe</groupId>
      <artifactId>tomitribe-crest</artifactId>
      <version>0.3-SNAPSHOT</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>

    <!-- Add tomitribe-crest-xbean if you want classpath scanning for @Command -->
    <dependency>
      <groupId>org.tomitribe</groupId>
      <artifactId>tomitribe-crest-xbean</artifactId>
      <version>0.3-SNAPSHOT</version>
    </dependency>
  </dependencies>

  <build>
    <defaultGoal>install</defaultGoal>
    <plugins>
      <plugin>
        <artifactId>maven-shade-plugin</artifactId>
        <version>2.1</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>org.tomitribe.crest.Main</mainClass>
                </transformer>
              </transformers>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

  <repositories>
    <repository>
      <id>sonatype-nexus-snapshots</id>
      <name>Sonatype Nexus Snapshots</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
      <releases>
        <enabled>false</enabled>
      </releases>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
  </repositories>

</project>

Bean Parameter Binding

If you don’t want to inject in all your commands the same N parameters you can modelize them as an object. Just use standard parameters as constructor parameters of the bean:

public class ColorfulCmd {
    @Command
    public static void exec(final Color color) {
        // ...
    }
}

To identify Color as an "option aware" parameter just decorate it with @Options:

@Options
public class Color { // getters omitted for brevity
    private final int r;
    private final int g;
    private final int b;
    private final int a;

    public Color(@Option("r") @Default("255") final int r,
                 @Option("g") @Default("255") final int g,
                 @Option("b") @Default("255") final int b,
                 @Option("a") @Default("255") final int a) {
        this.r = r;
        this.g = g;
        this.b = b;
        this.a = a;
    }
}

Prefixing options

If you reuse the same parameter N times you’ll probably want to prefix options. If we take previous example (Params) you can desire to use --background.r and --foreground.r (same for g, b, a).

Just use @Option in the method parameter to do so:

public class ColorfulCmd {
    @Command
    public static void exec(@Option("background.") final Color colorBg, @Option("foreground.") final Color colorFg) {
        // ...
    }
}
Note
 the '.' is not automatically added to allow you use to another convention like '-' or '_' ones for instance.

Override defaults

If you reuse the same parameter model accross command parameter you’ll surely want to override some default in some cases. For that purpose just use @Defaults and define the mappings you want:

public class ColorfulCmd {
    @Command
    public static void exec(@Defaults({
                                @Defaults.DefaultMapping(name = "r", value = "0"),
                                @Defaults.DefaultMapping(name = "g", value = "0"),
                                @Defaults.DefaultMapping(name = "b", value = "0"),
                                @Defaults.DefaultMapping(name = "a", value = "0")
                            })
                            @Option("background.")
                            final Color colorBg,

                            @Defaults({
                                @Defaults.DefaultMapping(name = "r", value = "255"),
                                @Defaults.DefaultMapping(name = "g", value = "255"),
                                @Defaults.DefaultMapping(name = "b", value = "255"),
                                @Defaults.DefaultMapping(name = "a", value = "255")
                            })
                            @Option("foreground.")
                            final Color colorFg) {
        // ...
    }
}

Interceptors

Sometimes you need to modify the command invocation or "insert" code before/after the command execution. For that purpose crest has some light interceptor support.

Defining an interceptor is as easy as defining a class with:

public static class MyInterceptor {
    @CrestInterceptor
    public Object intercept(final CrestContext crestContext) {
        return crestContext.proceed();
    }
}

The constraint for an interceptor are:

  • being decorated with @CrestInterceptor

  • the method needs to be public

  • the method needs to table a single parameter of type CrestContext

Note
 you can pass @CrestInterceptor a value changing the key used to mark the interceptor.

To let a command use an interceptor or multiple ones just list them ordered in interceptedBy parameter:

@Command(interceptedBy = { MySecurityInterceptor.class, MyLoggingInterceptor.class, MyParameterFillingInterceptor.class })
public void test1(
         @Option("o1") final String o1,
         @Option("o2") final int o2,
         @Err final PrintStream err,
         @Out final PrintStream out,
         @In final InputStream is,
         @Option("o3") final String o3,
         final URL url) {
    // do something
}

Example for security

Crest provides a org.tomitribe.crest.interceptor.security.SecurityInterceptor which handles @RolesAllowed using the SPI org.tomitribe.crest.interceptor.security.RoleProvider to determine if you can call or not the command contextually.

Note
 RoleProvider is taken from Environment services. You can register it through org.tomitribe.crest.environments.SystemEnvironment constructor and just set it as environment on org.tomitribe.crest.environments.Environment.ENVIRONMENT_THREAD_LOCAL.

Here a sample command using it:

@RolesAllowed("test")
@Command(interceptedBy = SecurityInterceptor.class)
public static String val() {
    return "ok";
}

Maven Archetype

A maven archetype is available to quickly bootstrap small projects complete with the a pom like the above. Save yourself some time on copy/paste then find/replace.

mvn archetype:generate \
 -DarchetypeGroupId=org.tomitribe \
 -DarchetypeArtifactId=tomitribe-crest-archetype \
 -DarchetypeVersion=1.0.0-SNAPSHOT

Maven Plugin

If you don’t want to rely on runtime scanning to find classes but still want to avoid to list command classes or just reuse crest Main you can use Maven Plugin to find it and generate a descriptor used to load classes.

Here is how to define it in your pom:

<plugin>
  <groupId>org.tomitribe</groupId>
  <version>${crest.version}</version>
  <artifactId>crest-maven-plugin</artifactId>
    <executions>
      <execution>
        <goals>
          <goal>descriptor</goal>
        </goals>
      </execution>
    </executions>
</plugin>

DeltaSpike Annotation Processor

Adding this dependency to your project:

<dependency>
  <groupId>org.tomitribe</groupId>
  <artifactId>tomitribe-crest-generator</artifactId>
  <version>${crest.version}</version>
  <scope>provided</scope>
</dependency>

Crest Generator can integrates with DeltaSpike to generate binding pojo. It will split @ConfigProperty on first dot and create one binding per prefix.

Here is an example:

public class DeltaspikeBean {
    @Inject
    @ConfigProperty(name = "app.service.base", defaultValue = "http://localhost:8080")
    private String base;

    @Inject
    @ConfigProperty(name = "app.service.retries")
    private Integer retries;
}

It will generate the following binding:

package org.tomitribe.crest.generator.generated;

import java.util.Collections;
import java.util.Map;
import java.util.HashMap;

import org.apache.deltaspike.core.api.config.ConfigResolver;
import org.apache.deltaspike.core.spi.config.ConfigSource;
import org.tomitribe.crest.api.Default;
import org.tomitribe.crest.api.Option;

import static java.util.Collections.singletonList;

public class App {
    private String serviceBase;
    private Integer serviceRetries;

    public App(
        @Option("service-base") @Default("http://localhost:8080") String serviceBase,
        @Option("service-retries") Integer serviceRetries) {
        final Map<String, String> ____properties = new HashMap<>();
        this.serviceBase = serviceBase;
        ____properties.put("app.service.base", String.valueOf(serviceBase));
        this.serviceRetries = serviceRetries;
        ____properties.put("app.service.retries", String.valueOf(serviceRetries));
        ConfigResolver.addConfigSources(Collections.<ConfigSource>singletonList(new ConfigSource() {
            @Override
            public int getOrdinal() {
                return 0;
            }

            @Override
            public Map<String, String> getProperties() {
                return ____properties;
            }

            @Override
            public String getPropertyValue(final String key) {
                return ____properties.get(key);
            }

            @Override
            public String getConfigName() {
                return "crest-app";
            }

            @Override
            public boolean isScannable() {
                return true;
            }
        }));    }

    public String getServiceBase() {
        return serviceBase;
    }

    public void setServiceBase(final String serviceBase) {
        this.serviceBase = serviceBase;
    }

    public Integer getServiceRetries() {
        return serviceRetries;
    }

    public void setServiceRetries(final Integer serviceRetries) {
        this.serviceRetries = serviceRetries;
    }

}

Then you just need to reuse it ad a crest command parameter:

@Command
public void myCommand(@Option("app-") final App app) {
  // ...
}

The nice thing is it will integrate with crest of course but also with DeltaSpike. It means the previous code will also make DeltaSpike injection respecting App configuration (--app-service-base=…​ --app-service-retries=3 for instance).

If you create a fatjar using TomEE embedded it means you can handle all your DeltaSpike configuration this way and you just need to write a TomEE Embedded runner to get DeltaSpike configuration wired from the command line:

import org.apache.tomee.embedded.Main;

public final class Runner {
    @Command("run")
    public static void run(@Option("app-") App app) {
        Main.main(new String[] { "--as-war", "--single-classloader" } /*fatjar "as war" deployment*/);
        // automatically @Inject @ConfigProperty will be populated :)
    }
}

Potential enhancement(s):

  • option to generate TomEE Embedded main?

  • Tamaya integration on the same model?

  • Owner integration

  • …​

Cli module

Cli module aims to provide a basic integration with JLine.

All starts from org.tomitribe.crest.cli.api.CrestCli class. Current version is extensible through inheritance but already provides:

  • support of maven plugin commands (crest-commands.txt)

  • JLine integration

  • Basic pipping support (mycommand | jgrep foo)

  • History support is you return a file in org.tomitribe.crest.cli.api.CrestCli.cliHistoryFile

  • org.tomitribe.crest.cli.api.interceptor.interactive.Interactivable can be used to mark a parameter as required but compatible with interactive mode (ie the parameter is asked in interactive mode if missing).

Sample usage:

final CrestCli cli = new CrestCli();
cli.run();
Tip
 CrestCli also has a main(String[]) so it can be used directly as well.
Note
 if you don’t provide an exit command one is added by default.