Category Archives: Java

Introduction to Java Bytecode

Reading compiled Java bytecode can be tedious even for experienced Java developers. Why do we need to know about such low level stuff in the first place? Here is a simple scenario that happened to me last week: I had made some code changes on my machine long time ago, compiled a Jar and deployed it on a server to test a potential fix for a performance issue. Unfortunately, the code was never checked in to a version control system and for whatever reason, the local changes were deleted without a trace. After a couple of months, I needed those changes in source form again (which took quite an effort to come up with) but could not find them!

Luckily the compiled code still existed on that remote server. So with a sigh of relief I fetched the Jar again and opened it using a decompiler editor. Only one problem, the decompiler GUI is not a flawless tool, and out of the many classes in that Jar, for some reason, only the specific class I was looking to decompile caused a bug in the UI to be exercised whenever I opened it and the decompiler to crash!

Desperate times call for desperate measures… fortunately I was familiar with raw bytecode and I’d rather take some time manually decompiling some pieces of the code rather than work through the changes and testing them again. Since I still remembered at least where to look in the code, reading bytecode helped me pinpoint the exact changes and construct them back in source form. (I made sure to learn from my mistake and preserve them this time!)

The nice thing about bytecode is that you learn its syntax once and it applies on all Java supported platforms, because it is an intermediate representation of the code, and not the actual executable code for the underlying CPU. Moreover, bytecode is simpler than native machine code because the JVM architecture is rather simple, hence simplifying the instruction set. Yet another nice thing is that all instructions in this instruction set are fully documented by Oracle.

Before learning about the bytecode instruction set though, let’s get familiar with a few things about the JVM which are needed as a prerequisite.

JVM data types

Java is statically typed, which affects the design of the bytecode instructions such that an instruction expects itself to operate on values of specific types. For example, there are several add instructions to add two numbers: iadd, ladd, fadd, dadd. They expect operands of type, respectively, int, long, float and double. The majority of bytecode has this characteristic of having different forms of the same functionality but different depending on the operand types.

The data types defined by the JVM are:

  1. Primitive types:
    • Numeric types: byte (8-bit 2’s complement), short (16-bit 2’s complement), int (32-bit 2’s complement), long (64-bit 2’s complement), char (16-bit unsigned Unicode), float (32-bit IEEE 754 single precision FP), double (64-bit IEEE 754 double precision FP)
    • boolean type
    • returnAddress: pointer to instruction
  2. Reference types:
    • Class types
    • Array types
    • Interface types

The boolean type has limited support in bytecode. For example, there are no instructions that directly operate on boolean values. Boolean values are instead converted to int by the compiler and the corresponding int instruction is used.

Java developers should be familiar with all of the above types, except returnAddress which has no equivalent programming language type.

Stack-based architecture

The simplicity of the bytecode instruction set is largely due to Sun having designed a stack-based VM architecture, as opposed to a register-based one. There are various memory components used by a JVM process, but only the JVM stacks need to be examined in detail on to essentially be able to follow bytecode instructions:

PC register: for each thread running in a Java program, a PC register stores the address of the current instruction.

JVM stack: for each thread, a stack is allocated where local variables, method arguments and return values are stored. Here is an illustration showing stacks for 3 threads.

jvm_stacks

Heap: memory shared by all threads, and storing objects (class instances and arrays). Object deallocation is managed by a garbage collector.

heap.png

Method area: for each loaded class, stores the code of methods and a table of symbols (e.g. references to fields or methods) and constants known as the constant pool.

method_area.png

A JVM stack is composed of frames, each pushed onto the stack when a method is invoked and popped from the stack when the method completes (either by returning normally or by throwing an exception). Each frame further consists of:

  1. An array of local variables, indexed from 0 to its length minus 1. The length is computed by the compiler. A local variable can hold a value of any type, except long and double values which occupy two local variables.
  2. An operand stack used to store intermediate values that would act as operands for instructions, or to push arguments to method invocations.

stack_frame_zoom.png

Bytecode explored

With an idea about the internals of a JVM, we can look at some basic bytecode example generated from sample code. Each method in a Java class file has a code segment that consists of a sequence of instructions, each having the following format:

opcode (1 byte)      operand1 (optional)      operand2 (optional)      ...

That is an instruction consists of one-byte opcode and zero or more operands that contain the data to operate.

Within the stack frame of the currently executing method, an instruction can push or pop values onto the operand stack, and it can potentially load or store values in the array local variables. Let’s look at a simple example:

public static void main(String[] args) {
    int a = 1;
    int b = 2;
    int c = a + b;
}

In order to print the resulting bytecode in the compiled class (assuming it is in a file Test.class), we can run the javap tool:

javap -v Test.class

and we get:

public static void main(java.lang.String[]);
descriptor: ([Ljava/lang/String;)V
flags: (0x0009) ACC_PUBLIC, ACC_STATIC
Code:
stack=2, locals=4, args_size=1
0: iconst_1
1: istore_1
2: iconst_2
3: istore_2
4: iload_1
5: iload_2
6: iadd
7: istore_3
8: return
...

We can see the method signature for the main method, a descriptor which indicates that the method takes an array of Strings ([Ljava/lang/String; ) and has a void return type (V ). A set of flags follow which describe the method as public (ACC_PUBLIC) and static (ACC_STATIC).

The most important part is the Code attribute, which contains the instructions for the method along with information such as the maximum depth of the operand stack (2 in this case), and the number of local variables allocated in the frame for this method (4 in this case). All local variables are referenced in the above instructions except the first one (at index 0) which holds the reference to the args argument. The other 3 local variables correspond to variables a, b and c in the source code.

The instructions from address 0 to 8 will do the following:

iconst_1: Push the integer constant 1 onto the operand stack.

iconst_1.png

istore_1: Pop the top operand (an int value) and store it in local variable at index 1, which corresponds to variable a.

istore_1.png

iconst_2: Push the integer constant 2 onto the operand stack.

iconst_2.png

istore_2: Pop the top operand int value and store it in local variable at index 2, which corresponds to variable b.

istore_2.png

iload_1: Load the int value from local variable at index 1 and push it onto the operand stack.

iload_1.png

iload_2: Load int value from local variable at index 1 and push it onto the operand stack.

iload_2.png

iadd: Pop the top two int values from the operand stack, add them and push the result back onto the operand stack.

iadd

istore_3: Pop the top operand int value and store it in local variable at index 3, which corresponds to variable c.

istore_3.png

return: Return from the void method.

Each of the above instructions consists of only an opcode, which dictates exactly the operation to be executed by the JVM.

Method invocations

In the above example, there is only one method, the main method. Let’s assume that we need to a more elaborate computation for the value of variable c, and we decide to place that in a new method called calc:

public static void main(String[] args) {
    int a = 1;
    int b = 2;
    int c = calc(a, b);
}

static int calc(int a, int b) {
    return (int) Math.sqrt(Math.pow(a, 2) + Math.pow(b, 2));
}

Let’s see the resulting bytecode:

public static void main(java.lang.String[]);
  descriptor: ([Ljava/lang/String;)V
  flags: (0x0009) ACC_PUBLIC, ACC_STATIC
  Code:
    stack=2, locals=4, args_size=1
       0: iconst_1
       1: istore_1
       2: iconst_2
       3: istore_2
       4: iload_1
       5: iload_2
       6: invokestatic  #2         // Method calc:(II)I
       9: istore_3
      10: return

static int calc(int, int);
  descriptor: (II)I
  flags: (0x0008) ACC_STATIC
  Code:
    stack=6, locals=2, args_size=2
       0: iload_0
       1: i2d
       2: ldc2_w        #3         // double 2.0d
       5: invokestatic  #5         // Method java/lang/Math.pow:(DD)D
       8: iload_1
       9: i2d
      10: ldc2_w        #3         // double 2.0d
      13: invokestatic  #5         // Method java/lang/Math.pow:(DD)D
      16: dadd
      17: invokestatic  #6         // Method java/lang/Math.sqrt:(D)D
      20: d2i
      21: ireturn

The only difference in the main method code is that instead of having the iadd instruction, we now an invokestatic instruction, which simply invokes the static method calc. The key thing to note is that the operand stack contained the two arguments that are passed to the method calc. In other words, the calling method prepares all arguments of the to-be-called method by pushing them onto the operand stack in the correct order. invokestatic (or a similar invoke* instruction as will be seen later) will subsequently pop these arguments, and a new frame is created for the invoked method where the arguments are placed in its local variable array.

We also notice that the invokestatic instruction occupies 3 bytes by looking at the address which jumped from 6 to 9. This is because unlike all instructions seen so far, invokestatic includes two additional bytes to construct the reference to the method to be invoked (in addition to the opcode). The reference is shown by javap as #2 which is a symbolic reference to the calc method which is resolved from the constant pool described earlier.

The other new information is obviously the code for the calc method itself. It first loads the first integer argument onto the operand stack (iload_0). The next instruction i2d converts it to a double by applying widening conversion. The resulting double replaces the top of the operand stack.

The next instruction pushes a double constant 2.0d  (taken from the constant pool) onto the operand stack. Then the static Math.pow method is invoked with the two operand values prepared so far (the first argument to calc, and the constant 2.0d). When the Math.pow method returns, its result will be stored on the operand stack of its invoker. This can be illustrated below.

math_pow.png

The same procedure is applied to compute Math.pow(b, 2):

math_pow2.png

The next instruction dadd pops the top two intermediate results, adds them and pushes the sum back to the top. Finally, invokestatic invokes Math.sqrt on the resulting sum, and the result is cast from double to int using narrowing conversion (d2i). The resulting int is returned to main method, which stores it back to c (istore_3).

Instance creations

Let’s modify the example and introduce a class Point to encapsulate XY coordinates.

public class Test {
    public static void main(String[] args) {
        Point a = new Point(1, 1);
        Point b = new Point(5, 3);
        int c = a.area(b);
    }
}

class Point {
    int x, y;

    Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    public int area(Point b) {
        int length = Math.abs(b.y - this.y);
        int width = Math.abs(b.x - this.x);
        return length * width;
    }
}

The compiled bytecode for the main method is shown below:

 public static void main(java.lang.String[]);
   descriptor: ([Ljava/lang/String;)V
   flags: (0x0009) ACC_PUBLIC, ACC_STATIC
   Code:
     stack=4, locals=4, args_size=1
        0: new           #2       // class test/Point
        3: dup
        4: iconst_1
        5: iconst_1
        6: invokespecial #3       // Method test/Point."<init>":(II)V
        9: astore_1
       10: new           #2       // class test/Point
       13: dup
       14: iconst_5
       15: iconst_3
       16: invokespecial #3       // Method test/Point."<init>":(II)V
       19: astore_2
       20: aload_1
       21: aload_2
       22: invokevirtual #4       // Method test/Point.area:(Ltest/Point;)I
       25: istore_3
       26: return

The new instructions encountereted here are new , dup and invokespecial. Similar to the new operator in the programming language, the new instruction creates an object of the type specified in the operand passed to it (which is a symbolic reference to class Point). Memory for the object is allocated on the heap, and a reference to the object is pushed on the operand stack.

The dup instruction duplicates the top operand stack value, which means that now we have two references the Point object on the top of the stack. The next three instructions push onto the operand stack the arguments of the constructor (used to initialize the object), and then invoke a special initialization method called   which corresponds the contructor. The  method is where the fields x and y will get initialized. After the method is finished, the top three operand stack values are consumed, and what remains is the original reference to the created object (which is by now successfully initialized).

init.png

Next astore_1 pops that Point reference and assigns to local variable at index 1 (the a in astore_1 indicates this is a reference value).

init_store.png

The same procedure is repeated for creating and initializing the second Point instance, which is assigned to variable b .

init2.png

init_store2.png

The last step loads the references to the two Point objects from local variables at indexes 1 and 2 (using aload_1 and aload_2 respectively), and invokes the area method using invokevirtual, which handles dispatching the call to the appropriate method based on the actual type of the object. For example, if the variable a contained an instance of type SpecialPoint that extends Point, and the subtype overrides the area method, then the overriden method is invoked. In this case, there is no subclass, and hence only one area method is available.

area.png

Note that even though the area method accepts one argument, there are two Point references on the top of the stack. The first one (pointA  which comes from variable a) is actually the instance on which the method is invoked (otherwise referred to as this in the programming language), and it will be passed in the first local variable of the new frame for the area method. The other operand value (pointB) is the argument to the area method.

The other way around

You don’t need to master the understanding of each instruction and the exact flow of execution to gain an idea about what the program does based on the bytecode at hand. For example, in my case I wanted to check if the code employed a Java stream to read a file, and whether the stream was properly closed. Now given the below bytecode, it is relatively easy to determine that indeed a stream is used and most likely it is being closed as part of a try-with-resources statement.

 public static void main(java.lang.String[]) throws java.lang.Exception;
  descriptor: ([Ljava/lang/String;)V
  flags: (0x0009) ACC_PUBLIC, ACC_STATIC
  Code:
    stack=2, locals=8, args_size=1
       0: ldc           #2                  // class test/Test
       2: ldc           #3                  // String input.txt
       4: invokevirtual #4                  // Method java/lang/Class.getResource:(Ljava/lang/String;)Ljava/net/URL;
       7: invokevirtual #5                  // Method java/net/URL.toURI:()Ljava/net/URI;
      10: invokestatic  #6                  // Method java/nio/file/Paths.get:(Ljava/net/URI;)Ljava/nio/file/Path;
      13: astore_1
      14: new           #7                  // class java/lang/StringBuilder
      17: dup
      18: invokespecial #8                  // Method java/lang/StringBuilder."<init>":()V
      21: astore_2
      22: aload_1
      23: invokestatic  #9                  // Method java/nio/file/Files.lines:(Ljava/nio/file/Path;)Ljava/util/stream/Stream;
      26: astore_3
      27: aconst_null
      28: astore        4
      30: aload_3
      31: aload_2
      32: invokedynamic #10,  0             // InvokeDynamic #0:accept:(Ljava/lang/StringBuilder;)Ljava/util/function/Consumer;
      37: invokeinterface #11,  2           // InterfaceMethod java/util/stream/Stream.forEach:(Ljava/util/function/Consumer;)V
      42: aload_3
      43: ifnull        131
      46: aload         4
      48: ifnull        72
      51: aload_3
      52: invokeinterface #12,  1           // InterfaceMethod java/util/stream/Stream.close:()V
      57: goto          131
      60: astore        5
      62: aload         4
      64: aload         5
      66: invokevirtual #14                 // Method java/lang/Throwable.addSuppressed:(Ljava/lang/Throwable;)V
      69: goto          131
      72: aload_3
      73: invokeinterface #12,  1           // InterfaceMethod java/util/stream/Stream.close:()V
      78: goto          131
      81: astore        5
      83: aload         5
      85: astore        4
      87: aload         5
      89: athrow
      90: astore        6
      92: aload_3
      93: ifnull        128
      96: aload         4
      98: ifnull        122
     101: aload_3
     102: invokeinterface #12,  1           // InterfaceMethod java/util/stream/Stream.close:()V
     107: goto          128
     110: astore        7
     112: aload         4
     114: aload         7
     116: invokevirtual #14                 // Method java/lang/Throwable.addSuppressed:(Ljava/lang/Throwable;)V
     119: goto          128
     122: aload_3
     123: invokeinterface #12,  1           // InterfaceMethod java/util/stream/Stream.close:()V
     128: aload         6
     130: athrow
     131: getstatic     #15                 // Field java/lang/System.out:Ljava/io/PrintStream;
     134: aload_2
     135: invokevirtual #16                 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
     138: invokevirtual #17                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
     141: return
    ...

We see occurrences of java/util/stream/Stream where forEach is called, preceded by a call to InvokeDynamic with a reference to a Consumer. And then we see a chunk of bytecode that calls Stream.close along with branches that call Throwable.addSuppressed. This is the basic code that gets generated by the compiler for a try-with-resources statement.

Here’s the original source for completeness:

public static void main(String[] args) throws Exception {
    Path path = Paths.get(Test.class.getResource("input.txt").toURI());
    StringBuilder data = new StringBuilder();
    try(Stream lines = Files.lines(path)) {
        lines.forEach(line -> data.append(line).append("\n"));
    }

    System.out.println(data.toString());
}

Conclusion

Thanks to the simplicity of the bytecode instruction set and the near absence of compiler optimizations when generating its instructions, disassembling class files could be one way to examine changes into your application code without having the source, if that ever becomes a need.

 

Advertisements

Compact Strings in Java 9

One of the performance enhancements introduced in the JVM (Oracle HotSpot to be specific) as part of Java SE 9 is compact strings. It aims to reduce the size of String objects, hence reducing the overall footprint of Java applications. As a result, it can also reduce the time spent on garbage collection.

The feature is based on the observation that most String objects do not need 2 bytes to encode every character, because most applications use only Latin-1 characters. Hence, instead of having:

/** The value is used for character storage. */
private final char value[];

java.lang.String now has:

private final byte[] value;
/**
 * The identifier of the encoding used to encode the bytes in
 * {@code value}. The supported values in this implementation are
 *
 * LATIN1
 * UTF16
 *
 * @implNote This field is trusted by the VM, and is a subject to
 * constant folding if String instance is constant. Overwriting this
 * field after construction will cause problems.
 */
private final byte coder;

In other words, this feature replaces the char array value (where each element uses 2 bytes) with a byte array with an extra byte to determine the encoding (Latin-1 or UTF-16). This means that for most application that use only Latin-1 characters, only half the previous amount of heap is used. This feature is completely invisible to the user, and related API such as StringBuilder automatically make use of it.

To demonstrate this change in terms of the size used by a String object, I’ll be using Java Object Layout, a simple utility that can be used to visualize the structure of an object in the heap. For that matter, we are interested in determining the footprint of the array (stored in the variable value above), and not simply the reference (both a byte array reference and a char array reference use 4 bytes). The following prints this information using a JOL GraphLayout:

public class JOLSample {

    public static void main(String[] args) {
        System.out.println(GraphLayout.parseInstance("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz").toFootprint());
    }
}

Running the above against Java 8 and then against Java 9 shows the difference:

$java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

$java -cp lib\jol-cli-0.9-full.jar;. test.JOLSample
java.lang.String@4554617cd footprint:
     COUNT       AVG       SUM   DESCRIPTION
         1       432       432   [C
         1        24        24   java.lang.String
         2                 456   (total)

...

$java -version
java version "9"
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

$java -cp lib\jol-cli-0.9-full.jar;. test.JOLSample
java.lang.String@73035e27d footprint:
     COUNT       AVG       SUM   DESCRIPTION
         1       224       224   [B
         1        24        24   java.lang.String
         2                 248   (total)

Ignoring the 24-byte size of the internals of java.lang.String (header plus references), we see the size reduced to almost half with string compaction.

If we change the above String to use a UTF-16 character such as \u0780, and then re-run the above, both Java 8 and Java 9 show the same footprint because the compaction no longer occurs.

This feature can be disabled by passing the option -XX:-CompactStrings to the java command.

20 Examples of Using Java’s CompletableFuture

This post revisits Java 8’s CompletionStage API and specifically its implementation in the standard Java library, CompletableFuture. The API is explained by examples that illustrate the various behaviors, where each example focuses on a specific one or two behaviors.

Since the CompletableFuture class implements the CompletionStage interface, we first need to understand the contract of that interface. It represents a stage of a certain computation which can be done either synchronously or asynchronously. You can think of it as just a single unit of a pipeline of computations that ultimately generate a final result of interest. This means that several CompletionStages can be chained together so that one stage’s completion triggers the execution of another stage, which in turns triggers another, and so on.

In addition to implementing the CompletionStage interface, CompletableFuture also implements Future, which represents a pending asynchronous event, with the ability to explicitly complete this Future, hence the name CompletableFuture.

1. Creating a completed CompletableFuture

The simplest example creates an already completed CompletableFuture with a predefined result. Usually this may act as the starting stage in your computation.

static void completedFutureExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message");
    assertTrue(cf.isDone());
    assertEquals("message", cf.getNow(null));
}

The getNow(null) returns the result if completed (which is obviously the case), otherwise returns null (the argument).

2. Running a simple asynchronous stage

The next example is how to create a stage that executes a Runnable asynchronously:

static void runAsyncExample() {
    CompletableFuture cf = CompletableFuture.runAsync(() -> {
        assertTrue(Thread.currentThread().isDaemon());
        randomSleep();
    });
    assertFalse(cf.isDone());
    sleepEnough();
    assertTrue(cf.isDone());
}

The takeaway of this example is two things:

  1. A CompletableFuture is executed asynchronously when the method typically ends with the keyword Async
  2. By default (when no Executor is specified), asynchronous execution uses the common ForkJoinPool implementation, which uses daemon threads to execute the Runnable task. Note that this is specific to CompletableFuture. Other CompletionStage implementations can override the default behavior.

3. Applying a Function on previous stage

The below example takes the completed CompletableFuture from example #1, which bears the result string "message", and applies a function that converts it to uppercase:

static void thenApplyExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message").thenApply(s -> {
        assertFalse(Thread.currentThread().isDaemon());
        return s.toUpperCase();
    });
    assertEquals("MESSAGE", cf.getNow(null));
}

Note the behavioral keywords in thenApply:

  1. then, which means that the action of this stage happens when the current stage completes normally (without an exception). In this case, the current stage is already completed with the value “message”.
  2. Apply, which means the returned stage will apply a Function on the result of the previous stage.

The execution of the Function will be blocking, which means that getNow() will only be reached when the uppercase operation is done.

4. Asynchronously applying a Function on previous stage

By appending the Async suffix to the method in the previous example, the chained CompletableFuture would execute asynchronously (using ForkJoinPool.commonPool()).

static void thenApplyAsyncExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message").thenApplyAsync(s -> {
        assertTrue(Thread.currentThread().isDaemon());
        randomSleep();
        return s.toUpperCase();
    });
    assertNull(cf.getNow(null));
    assertEquals("MESSAGE", cf.join());
}

5. Asynchronously applying a Function on previous stage using a custom Executor

A very useful feature of asynchronous methods is the ability to provide an Executor to use it to execute the desired CompletableFuture. This example shows how to use a fixed thread pool to apply the uppercase conversion Function:

static ExecutorService executor = Executors.newFixedThreadPool(3, new ThreadFactory() {
    int count = 1;

    @Override
    public Thread newThread(Runnable runnable) {
        return new Thread(runnable, "custom-executor-" + count++);
    }
});

static void thenApplyAsyncWithExecutorExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message").thenApplyAsync(s -> {
        assertTrue(Thread.currentThread().getName().startsWith("custom-executor-"));
        assertFalse(Thread.currentThread().isDaemon());
        randomSleep();
        return s.toUpperCase();
    }, executor);

    assertNull(cf.getNow(null));
    assertEquals("MESSAGE", cf.join());
}

6. Consuming result of previous stage

If the next stage accepts the result of the current stage but does not need to return a value in the computation (i.e. its return type is void), then instead of applying a Function, it can accept a Consumer, hence the method thenAccept:

static void thenAcceptExample() {
    StringBuilder result = new StringBuilder();
    CompletableFuture.completedFuture("thenAccept message")
            .thenAccept(s -> result.append(s));
    assertTrue("Result was empty", result.length() > 0);
}

The Consumer will be executed synchronously, so we don’t need to join on the returned CompletableFuture.

7. Asynchronously consuming result of previous stage

Again, using the async version of thenAccept, the chained CompletableFuture would execute asynchronously:

static void thenAcceptAsyncExample() {
    StringBuilder result = new StringBuilder();
    CompletableFuture cf = CompletableFuture.completedFuture("thenAcceptAsync message")
            .thenAcceptAsync(s -> result.append(s));
    cf.join();
    assertTrue("Result was empty", result.length() > 0);
}

8. Completing a computation exceptionally

Now let us see how an asynchronous operation can be explicitly completed exceptionally, indicating a failure in the computation. For simplicity, the operation takes a string and converts it to upper case, and we simulate a delay in the operation of 1 second. To do that, we will use the thenApplyAsync(Function, Executor) method, where the first argument is the uppercase function, and the executor is a delayed executor that waits for 1 second before actually submitting the operation to the common ForkJoinPool.

static void completeExceptionallyExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message").thenApplyAsync(String::toUpperCase,
            CompletableFuture.delayedExecutor(1, TimeUnit.SECONDS));
    CompletableFuture exceptionHandler = cf.handle((s, th) -> { return (th != null) ? "message upon cancel" : ""; });
    cf.completeExceptionally(new RuntimeException("completed exceptionally"));
assertTrue("Was not completed exceptionally", cf.isCompletedExceptionally());
    try {
        cf.join();
        fail("Should have thrown an exception");
    } catch(CompletionException ex) { // just for testing
        assertEquals("completed exceptionally", ex.getCause().getMessage());
    }

    assertEquals("message upon cancel", exceptionHandler.join());
}

Let’s examine this example in detail:

  • First, we create a CompletableFuture that is already completed with the value "message". Next we call thenApplyAsync which returns a new CompletableFuture. This method applies an uppercase conversion in an asynchronous fashion upon completion of the first stage (which is already complete, thus the Function will be immediately executed). This example also illustrates a way to delay the asynchronous task using the delayedExecutor(timeout, timeUnit) method.
  • We then create a separate “handler” stage, exceptionHandler, that handles any exception by returning another message "message upon cancel".
  • Next we explicitly complete the second stage with an exception. This makes the join() method on the stage, which is doing the uppercase operation, throw a CompletionException (normally join() would have waited for 1 second to get the uppercase string). It will also trigger the handler stage.

9. Canceling a computation

Very close to exceptional completion, we can cancel a computation via the cancel(boolean mayInterruptIfRunning) method from the Future interface. For CompletableFuture, the boolean parameter is not used because the implementation does not employ interrupts to do the cancelation. Instead, cancel() is equivalent to completeExceptionally(new CancellationException()).

static void cancelExample() {
    CompletableFuture cf = CompletableFuture.completedFuture("message").thenApplyAsync(String::toUpperCase,
            CompletableFuture.delayedExecutor(1, TimeUnit.SECONDS));
    CompletableFuture cf2 = cf.exceptionally(throwable -> "canceled message");
    assertTrue("Was not canceled", cf.cancel(true));
    assertTrue("Was not completed exceptionally", cf.isCompletedExceptionally());
    assertEquals("canceled message", cf2.join());
}

10. Applying a Function to result of either of two completed stages

The below example creates a CompletableFuture that applies a Function to the result of either of two previous stages (no guarantees on which one will be passed to the Function). The two stages in question are: one that applies an uppercase conversion to the original string, and another that applies a lowercase conversion:

static void applyToEitherExample() {
    String original = "Message";
    CompletableFuture cf1 = CompletableFuture.completedFuture(original)
            .thenApplyAsync(s -> delayedUpperCase(s));
    CompletableFuture cf2 = cf1.applyToEither(
            CompletableFuture.completedFuture(original).thenApplyAsync(s -> delayedLowerCase(s)),
            s -> s + " from applyToEither");
    assertTrue(cf2.join().endsWith(" from applyToEither"));
}

11. Consuming result of either of two completed stages

Similar to the previous example, but using a Consumer instead of a Function (the dependent CompletableFuture has a type void):

static void acceptEitherExample() {
    String original = "Message";
    StringBuilder result = new StringBuilder();
    CompletableFuture cf = CompletableFuture.completedFuture(original)
            .thenApplyAsync(s -> delayedUpperCase(s))
            .acceptEither(CompletableFuture.completedFuture(original).thenApplyAsync(s -> delayedLowerCase(s)),
                    s -> result.append(s).append("acceptEither"));
    cf.join();
    assertTrue("Result was empty", result.toString().endsWith("acceptEither"));
}

12. Running a Runnable upon completion of both stages

This example shows how the dependent CompletableFuture that executes a Runnable triggers upon completion of both of two stages. Note all below stages run synchronously, where a stage first converts a message string to uppercase, then a second converts the same message string to lowercase.

static void runAfterBothExample() {
    String original = "Message";
    StringBuilder result = new StringBuilder();
    CompletableFuture.completedFuture(original).thenApply(String::toUpperCase).runAfterBoth(
            CompletableFuture.completedFuture(original).thenApply(String::toLowerCase),
            () -> result.append("done"));
    assertTrue("Result was empty", result.length() > 0);
}

13. Accepting results of both stages in a BiConsumer

Instead of executing a Runnable upon completion of both stages, using BiConsumer allows processing of their results if needed:

static void thenAcceptBothExample() {
    String original = "Message";
    StringBuilder result = new StringBuilder();
    CompletableFuture.completedFuture(original).thenApply(String::toUpperCase).thenAcceptBoth(
            CompletableFuture.completedFuture(original).thenApply(String::toLowerCase),
            (s1, s2) -> result.append(s1 + s2));
    assertEquals("MESSAGEmessage", result.toString());
}

14. Applying a BiFunction on results of both stages

If the dependent CompletableFuture is intended to combine the results of two previous CompletableFutures by applying a function on them and returning a result, we can use the method thenCombine(). The entire pipeline is synchronous, so getNow() at the end would retrieve the final result, which is the concatenation of the uppercase and the lowercase outcomes.

static void thenCombineExample() {
    String original = "Message";
    CompletableFuture cf = CompletableFuture.completedFuture(original).thenApply(s -> delayedUpperCase(s))
            .thenCombine(CompletableFuture.completedFuture(original).thenApply(s -> delayedLowerCase(s)),
                    (s1, s2) -> s1 + s2);
    assertEquals("MESSAGEmessage", cf.getNow(null));
}

15. Asynchronously applying a BiFunction on results of both stages

Similar to the previous example, but with a different behavior: since the two stages upon which CompletableFuture depends both run asynchronously, the thenCombine() method executes asynchronously, even though it lacks the Async suffix. This is documented in the class Javadocs: “Actions supplied for dependent completions of non-async methods may be performed by the thread that completes the current CompletableFuture, or by any other caller of a completion method.” Therefore, we need to join() on the combining CompletableFuture to wait for the result.

static void thenCombineAsyncExample() {
    String original = "Message";
    CompletableFuture cf = CompletableFuture.completedFuture(original)
            .thenApplyAsync(s -> delayedUpperCase(s))
            .thenCombine(CompletableFuture.completedFuture(original).thenApplyAsync(s -> delayedLowerCase(s)),
                    (s1, s2) -> s1 + s2);
    assertEquals("MESSAGEmessage", cf.join());
}

16. Composing CompletableFutures

We can use composition using thenCompose() to accomplish the same computation done in the previous two examples. This method waits for the first stage (which applies an uppercase conversion) to complete. Its result is passed to the specified Function which returns a CompletableFuture, whose result will be the result of the returned CompletableFuture. In this case, the Function takes the uppercase string (upper), and returns a CompletableFuture that converts the original string to lowercase and then appends it to upper.

static void thenComposeExample() {
    String original = "Message";
    CompletableFuture cf = CompletableFuture.completedFuture(original).thenApply(s -> delayedUpperCase(s))
            .thenCompose(upper -> CompletableFuture.completedFuture(original).thenApply(s -> delayedLowerCase(s))
                    .thenApply(s -> upper + s));
    assertEquals("MESSAGEmessage", cf.join());
}

17. Creating a stage that completes when any of several stages completes

The below example illustrates how to create a CompletableFuture that completes when any of several CompletableFutures completes, with the same result. Several stages are first created, each converting a string from a list to uppercase. Because all of these CompletableFutures are executing synchronously (using thenApply()), the CompletableFuture returned from anyOf() would execute immediately, since by the time it is invoked, all stages are completed. We then use the whenComplete(BiConsumer<? super Object, ? super Throwable> action), which processes the result (asserting that the result is uppercase).

static void anyOfExample() {
    StringBuilder result = new StringBuilder();
    List messages = Arrays.asList("a", "b", "c");
    List<CompletableFuture> futures = messages.stream()
            .map(msg -> CompletableFuture.completedFuture(msg).thenApply(s -> delayedUpperCase(s)))
            .collect(Collectors.toList());
    CompletableFuture.anyOf(futures.toArray(new CompletableFuture[futures.size()])).whenComplete((res, th) -> {
        if(th == null) {
            assertTrue(isUpperCase((String) res));
            result.append(res);
        }
    });
    assertTrue("Result was empty", result.length() > 0);
}

18. Creating a stage that completes when all stages complete

The next two examples illustrate how to create a CompletableFuture that completes when all of several CompletableFutures completes, in a synchronous and then asynchronous fashion, respectively. The scenario is the same as the previous example: a list of strings is provided where each element is converted to uppercase.

static void allOfExample() {
    StringBuilder result = new StringBuilder();
    List messages = Arrays.asList("a", "b", "c");
    List<CompletableFuture> futures = messages.stream()
            .map(msg -> CompletableFuture.completedFuture(msg).thenApply(s -> delayedUpperCase(s)))
            .collect(Collectors.toList());
    CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()])).whenComplete((v, th) -> {
        futures.forEach(cf -> assertTrue(isUpperCase(cf.getNow(null))));
        result.append("done");
    });
    assertTrue("Result was empty", result.length() > 0);
}

19. Creating a stage that completes asynchronously when all stages complete

By switching to thenApplyAsync() in the individual CompletableFutures, the stage returned by allOf() gets executed by one of the common pool threads that completed the stages. So we need to call join() on it to wait for its completion.

static void allOfAsyncExample() {
    StringBuilder result = new StringBuilder();
    List messages = Arrays.asList("a", "b", "c");
    List<CompletableFuture> futures = messages.stream()
            .map(msg -> CompletableFuture.completedFuture(msg).thenApplyAsync(s -> delayedUpperCase(s)))
            .collect(Collectors.toList());
    CompletableFuture allOf = CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()]))
            .whenComplete((v, th) -> {
                futures.forEach(cf -> assertTrue(isUpperCase(cf.getNow(null))));
                result.append("done");
            });
    allOf.join();
    assertTrue("Result was empty", result.length() > 0);
}

20. Real life example

Now that the functionality of CompletionStage and specifically CompletableFuture is explored, the below example applies them in a practical scenario:

  1. First fetch a list of Car objects asynchronously by calling the cars() method, which returns a CompletionStage. The cars() method could be consuming a remote REST endpoint behind the scenes.
  2. We then compose another CompletionStage that takes care of filling the rating of each car, by calling the rating(manufacturerId) method which returns a CompletionStage that asynchronously fetches the car rating (again could be consuming a REST endpoint).
  3. When all Car objects are filled with their rating, we end up with a List, so we call allOf() to get a final stage (stored in variable done) that completes upon completion of all these stages.
  4. Using whenComplete() on the final stage, we print the Car objects with their rating.
cars().thenCompose(cars -> {
    List<CompletionStage> updatedCars = cars.stream()
            .map(car -> rating(car.manufacturerId).thenApply(r -> {
                car.setRating(r);
                return car;
            })).collect(Collectors.toList());

    CompletableFuture done = CompletableFuture
            .allOf(updatedCars.toArray(new CompletableFuture[updatedCars.size()]));
    return done.thenApply(v -> updatedCars.stream().map(CompletionStage::toCompletableFuture)
            .map(CompletableFuture::join).collect(Collectors.toList()));
}).whenComplete((cars, th) -> {
    if (th == null) {
        cars.forEach(System.out::println);
    } else {
        throw new RuntimeException(th);
    }
}).toCompletableFuture().join();

Since the Car instances are all independent, getting each rating asynchronously improves performance. Furthermore, waiting for all car ratings to be filled is done using a more natural allOf() method, as opposed to manual thread waiting (e.g. using Thread#join() or a CountDownLatch).

Working through these examples helps better understand this API. You can find the full code of these examples on GitHub.

 

What Is Project Amber in Java?

In this post, we’re going to delve into some details of the features being targeted in Project Amber, which was introduced in early 2017 by its lead and language architect Brian Goetz. This project aims to add some really cool beans to the Java programming language that improve the developer’s productivity when writing Java code, namely:

As this is still work in progress, other features could be added to the project in the future. Even though these features may seem to have been addressed late in the Java timeline, it’s worth considering that the Java team has historically been rather cautious in introducing new features to evolve the language, as Goetz explains in a talk about the project.

Local-variable type inference

In Java 5 generics were introduced with the possibility for the compiler to infer type arguments during generic method calls, as shown in the below example:

private <T> void foo(T t) {
    ...
}

public void bar() {
    this.foo(1);  // with implicit type, compiler infers type argument
    this.<Integer>foo(1); // with explicit typing
}

Further enhancements to type inference were done over the next releases, including the diamond operator in Java 7, enhancements in Java 8 along with lambda and stream support, and in Java 9 allowing the diamond operator in anonymous classes when the inferred type is denotable. With this feature (specified in JEP 286), the compiler will be able to infer declaration types of local variables, subject to certain limitations. The main requirement for the compiler is that the initializer needs to be included with the variable declaration. With this enhancement, a statement like:

File inputFile = new File("input.txt");

could be written as:

var inputFile = new File("input.txt");

Like any other form of type inference, the main benefit is avoiding the redundant typing of the variable type when it can be easily known from the right hand side of the assignment. It is important to remember here that the var keyword does not mean that the variable is dynamically typed. It is just a syntax to avoid writing the manifest type of the local variable; the static typing nature of Java remains intact. Strictly speaking, var is a reserved type name that gets desugared to the variable initializer type by the compiler.

Not every form of local variable declaration can use var to infer the declaration or manifest type of the variable. The following cases do not allow the use of var:

  • Local variables without initializers, such as File inputFile;
  • Local variables initialized to null.
  • Initializers that expect a target type, such as a lambda, a method reference or an array initializer.

As such, below are examples where type inference is not allowed:

// not allowed, lambda expression needs an explicit target-type
var func = s -> Integer.valueOf(s);

// not allowed, method reference needs an explicit target-type
var biConsumer = LogProcessor::process;

// not allowed, array initializer needs an explicit target-type
var array = { 1, 2 };

The majority of local variable declarations in typical code could benefit from this feature. For example, in the OpenJDK codebase, only 13% of local variables cannot be re-written using var. Therefore, the cost of broadening local type inference to include cases like the above may be too high compared to the amount of applicable code that can further benefit from it.

This feature is included in the planned Java 10 release, which is expected to be available in March 2018.

Enhanced enums

This feature enhances enums in two aspects. First, it allows declaring a generic enum, which combines the flexibility and type safety of generics with the simplicity and powerful semantics of an enum. Second, it enhances enums so that an enum constant that is declared as generic or overrides behavior via a class body gets its own type information, along with its own state and behavior.

In some use cases, we may need to define enum constants where each is bound to a certain type. A typical example is an enum that contains mappings to Java types, where a generic enum can be used shown in the below JsonType example:

public enum JsonType<T> {
    STRING<String>(String.class),
    LONG<Long>(Long.class),
    DOUBLE<>(Double.class), // can use a diamond operator to infer
    BOOLEAN<>(Boolean.class),
    ...

    final Class<T> mappedClass;

    JsonType(Class<T> mappedClass) {
        this.mappedClass = mappedClass;
    }

    public T convert(Object o) {
        ...
    }
}

In this case, the enum constant STRING has a sharper type JsonType<String>, enum constant LONG is of type JsonType<Long>, and so on. One could further customize each enum constant with additional state and/or methods:

public enum JsonType<T> {
    STRING<>(String.class),

    // LONG is has an anonymous class as type now,
    // we can use a diamond as long as inferred type is denotable
    LONG<>(Long.class) {
        public String desc = "Long JSON type";

        public boolean isLongValue(JSONValue value) {
            ...
        }
    },
    DOUBLE<>(Double.class),
    BOOLEAN<>(Boolean.class),
    ...
}

Since the enum constant LONG has a class body, its type is an anonymous class whose supertype is JsonType<Long>.

This feature is discussed more in JEP 301 which still has a “Candidate” status, so not all risks may have been addressed and is not expected to reach JDK 10.

Enhancements to lambda expressions

These are a couple of additional features added to lambda along with improving type inference for methods involving lambdas as arguments. The first feature is the ability to use an underscore to denote an unused parameter in a lambda:

BiFunction<Integer, String, String> biss = (i, _) -> String.valueOf(i);

As of Java 9, an underscore can no longer be used as an identifier, and with this feature it now carries a special meaning in the context of lambdas.

The second feature introduced in this JEP for lambdas is the ability to shadow variables declared in the enclosing scope of a lambda by re-using the same variable names to declare the lambda parameters:

int i = 0;
// can declare lambda parameter named i, shadowing the local variable i
BiFunction<Integer, String, String> biss = (i, _) -> String.valueOf(i);

Currently it is not allowed to re-use i in the lambda expression because lambdas are lexically scoped and generally do not allow shadowing variables. Last but not least, improving overload resolution for methods invoked with either a lambda or a method reference as argument is optionally targeted in this project. This should fix false compilation errors that may be commonly encountered when writing methods that accept functional interfaces:

m(Predicate<String> ps) { ... }
m(Function<String, String> fss) { ... }

m(s -> false) // error due to ambiguity, although Predicate
              // should have been inferred

class Foo {
    static boolean g(String s) { return false }
    static boolean g(Integer i) { return false }
}

m(Foo::g) // error due to ambiguity, although boolean g(String s)
          // should have been selected

Pattern matching

The next feature in the Amber project introduces a powerful construct called a pattern. The motivation behind this feature is the commonly used boilerplate code shown below:

String content = null;

if (msg instanceof JsonMessage) {

    content = unmarshalJson((JsonMessage) msg);

} else if (msg instanceof XmlMessage) {

    content = unmarshalXml((XmlMessage) msg);

} else if (msg instanceof PlainTextMessage) {

    content = ((PlainTextMessage) msg).getText();

} ...

Each condition branch checks if the object is of a certain type, then casts it to that type and extracts some information from it. Pattern matching is a generalization of this “test-extract-bind” technique and can be defined as:

  • a predicate that can be applied to a target
  • a set of binding variables that are extracted from the object matching the predicate

Instead of the above, we could apply a type-test pattern on the object msg using a new matches operator. As a first step, this would remove the redundant cast:

String content = null;

if (msg matches JsonMessage json) {

    content = unmarshalJson(json);

} else if (msg matches XmlMessage xml) {

    content = unmarshalXml(xml);

} else if (msg matches PlainTextMessage text) {

    content = text.getText();

} ...

The existing switch statement already makes use of the simplest form of patterns: the constant pattern. Given an object whose type is allowed in a switch statement, we test the object if it matches any of several constant expressions. Now the switch statement would also benefit from type-test patterns:

String content;

switch (msg) {

    case JsonMessage json:      content = unmarshalJson(json); break;

    case XmlMessage xml:        content = unmarshalXml(xml); break;

    case PlainTextMessage text: content = text.getText(); break;

    ...

    default:                    content = msg.toString();
}

This switch could be now be considered a “type switch” but it’s actually a generalized switch that can take other types of patterns; in this example, type-test patterns. In addition to generalizing a switch statement, this feature suggests a further improvement by allowing a switch to be also used as an expression instead of a statement, making the above look even more readable:

String content = switch (msg) {

    case JsonMessage json      -> unmarshalJson(json);

    case XmlMessage xml        -> unmarshalXml(xml);

    case PlainTextMessage text -> text.getText();

    ...

    default                    -> msg.toString();
}

Another kind of a pattern that can be used for matching and that is further proposed in this JEP is a deconstruction pattern. It matches an expression against a certain type, and extracts variables based on the signature of an existing constructor in the matched type:

// Using a deconstruction pattern with nested type patterns
if (item matches Book(String isbn, int year)) {
    // do something with isbn and published
}

What’s really nice about this is that the component String isbn is itself a type-test pattern that matches the isbn property of the Book object against a String and extracts it into the isbn variable, and the same for the int year pattern. This means patterns may be nested within each other. Another example is nesting a constant pattern in a deconstruction pattern:

// Using a deconstruction pattern with a nested constant pattern
if (item matches Book("978-0321349606")) {
    ...
}

Finally, within a future work scope of this JEP are sealed types, which allow defining types whose subtypes can be limited by the programmer. With this feature, a type could be marked as “sealed” to mean that the subtypes are restricted to a known set. It is similar to a final type, but instead gives the programmer a way to restrict the hierarchy of child types. For example, in an bookstore application we may only need to handle certain items like books, DVDs, etc. In this case we could seal our BookstoreItem parent class. This can be very useful because it removes the burden of handling default cases whenever we are switching over the possible subtypes.

Data classes

A lot of times all that a class is responsible of is holding data. And with the current way to writing such classes, we usually end up writing too much boilerplate to customize methods based on the state of the object, e.g. by overriding equals(), hashCode(), toString(), etc. Furthermore, there is nothing in the language that just tells the compiler or the reader that this class is a simple data holder class.

In order to define such semantics, this feature seeks to introduce data classes of the following form (syntax and semantics still under discussion; for a comprehensive discussion see this document on the OpenJDK site):

__data class Point(int x, int y) { }

which would be translated into the following at compile time:

final class Point extends java.lang.DataClass {
    final int x;
    final int y;

    public Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    // destructuring pattern for Point(int x, int y)
    // state-based equals, hashCode, and toString
    // public read accessors for x and y
}

The general idea is to have the compiler generate all that boilerplate for the programmer (similar to what is done with enums), and have the programmer still able to override methods like equals() or implementing interfaces. At this point, some design decisions are in progress, for example it is undecided whether immutability will be enforced or if mutability could be allowed, which would definitely impact the implementation of this feature along with the thread-safety of such classes.

Summary

Project Amber aims to bring features that can make writing Java code more readable and concise, and target specific use cases such as using generic enums or data classes. Local variable type inference enables the programmer to defer thinking about the variable type to whenever it is initialized. Enhanced enums allows a more flexible approach to solving specific problems with less code. Lambda leftovers improves lambda support with a couple of small changes. Pattern matching provides a powerful construct to writing conditional logic and reduces boilerplate coding. And finally data classes allow the programmer to segregate plain data carriers from other classes. So far, only local variable type inference is planned in the Java 10 release, but with the accelerated release timeline of Java, the others can be rolled out as soon as they are ready.

15 Productivity Tips for Eclipse Java IDE Users

With over 10 years of releases and countless extensions and plugins, Eclipse remains one of the most popular IDEs for developers working across several domains. Especially those in the Java ecosystem, where Eclipse provides a solid environment for developing, debugging and running applications. In this post, I’d like to share my favorite features in Eclipse that help me be more productive in my daily work as a Java developer. These features don’t require any plugin to be installed if the very first tip is followed.

1. Use Eclipse Oxygen

There are many improvements in the last release of Eclipse alone, and with the latest build released just a few days ago, the IDE supports Java 9 out of the box. Some of these improvements are:

  • Showing the last returned value while debugging a Java program. This shows you the value returned by the last method after stepping through the program statements.
  • Better Java 8 support, with numerous bug fixes for lambda expression type inference.
  • A new Java index that significantly reduces tooling performance, such as when loading the type hierarchy of platform classes and interfaces.
  • Support for JUnit 5.

You can find a list of Java tooling improvements in Oxygen here.

2. Switch editors using Ctrl+Tab

If you’re used to switching tabs in browsers and editors like Notepad++, then do yourself a favor and change the keybinding for switching between Java source files. Go to Window -> Preferences -> Keys, then search for “Next editor” and “Previous editor”, and override their bindings to Ctrl+Tab and Ctrl+Shift+Tab respectively (note: I’m using Windows).

3. Group related projects in working sets before choosing multiple workspaces

If you work on many different projects, at some point you may need to use multiple workspaces to separate them. But before reaching that level, you can just group related projects into a working set. This way you don’t have to switch workspaces or have two Eclipse windows using different workspaces. It also keeps your projects organized and accessible from within the same view. For example, I typically keep a working set for sample projects for quick experimentation, and then typically a working set for each group of related modules in a Maven project. One thing that you may need to do is change the Package Explorer view to view these working sets as shown below.

Image title

I often end up with many working sets containing projects I’m not using. In this case, I can simply close a working set by right-clicking on it from the Package Explorer, and selecting Close Project. This reduces memory consumption in the IDE and makes it as if these closed projects do not exist in your workspace anymore until you re-open them. Only when I have too many working sets, or I have projects that considerably differ from each other that I rarely switch between, then I separate them in different workspaces.

4. Set the “incremental” option in the search dialog

When you hit Ctrl+F to find text in a source file, check the Incremental checkbox in the search dialog to make the occurrence of the searched text appear as you type. This seemingly minor detail helps me avoid typing too many characters and then hitting return to find what I want.

Image title

5. Use navigation and search shortcuts

A few shortcuts to help understanding your code (using Windows). These are so useful that eventually they’ll easily become second nature:

  • F3 or Ctrl+Left click: goes to declaration of element
  • Ctrl+T: view type hierarchy and implementation methods
  • Ctrl+Alt+H: view call hierarchy of selected element
  • Ctrl+Shift+G: search workspace for all references to selected element
  • Ctrl+Shift+T: search for a class, interface or enum
  • Ctrl+Shift+R: search for a resource (e.g. text file)

6. Use the File Search feature

This is really helpful if you want to search files in your workspace for text. The search can be filtered by file type and scope of the search, such as searching only the selected project.

Image title

7. Use Ctrl+Space and Ctrl+1 for content assist and quick fixes

Ctrl+Space allows for auto-completion. It can also be used to override methods and generate getters /setters and constructors.

Ctrl+1 can be very handy in quick and smart fixes such as:

  • assigning constructor parameters to new or existing class fields
  • assigning a statement to a local variable
  • renaming a variable, field or class, etc.

8. Use code generation actions

Alt+Shift+S can be used to quickly add common code:

  • generating getters/setters and constructors
  • overriding hashCode() and equals()
  • overriding toString()

9. Ctrl+3 is your friend

As with any modern IDE, there are many keybindings to do all sorts of actions in Eclipse. But the most important is probably Ctrl+3 for Quick Access which is an entry point to virtually all actions you can do. I typically use it to open a view, but it can also be used to do refactoring, creating a new project, and lots of others.

10. Download the sources of libraries

If you’re using Maven, you can download the source code of your dependencies. There is preference under Window -> Preferences -> Maven that when selected automatically fetches the source artifacts of dependencies. You can also download sources manually by right-clicking on the dependency in the Maven Dependencies tree and selecting Maven -> Download Sources. Usually this also makes Javadoc comments available in your IDE when you hit F2, so you no longer need to browse it separately. There is similar way to do this in Gradle.

11. Use shortcuts to run, debug and inspect code

These again should become second nature while debugging:

  • Ctrl+F11 to run the last application launched
  • F11 to debug the last application launched
  • F5 to step into
  • F6 to step over, i.e. go to next line
  • F7 to step return to caller
  • F8 to resume until next breakpoint
  • Ctrl+Shit+I to evaluate selected expression
  • Use the Display view to write and execute code within the current debug context

12. Pinpoint program suspension with conditional breakpoints and watchpoints

Often you can make your program suspend on a line of code only when a certain condition is met. This reduces time spent on debugging, as long as you don’t overuse the feature (too many breakpoints, especially conditional ones, can make the program run slower in debug mode; in this case, you can either disable or delete unneeded breakpoints).

Image title

13. Save your run configurations for later re-use

I often need to build multi-module projects with different parameters, or run unit tests for a specific project, or configure some parameters for running a main class. Instead of switching to the command line, take the time to configure an appropriate run or debug configuration within the IDE. Ideally I never want to have a separate command line and everything should be done in the IDE, especially that it comes with all major build tools and SCM plugins already bundled.

14. Leverage code coverage support

In Eclipse Oxygen, the Eclipse EclEmma tool based on the JaCoCo code coverage library is integrated as part of the IDE, which allows you to have information on code coverage when running your unit tests. To run a program or unit test with coverage, right-click on the class to run and select Coverage As -> Java Application or JUnit Test.

15. For large workspaces with lots of dependent projects, disable Build Automatically

If you have lots of projects that depend on each other, the Build Automatically default behavior can be time-consuming because it would trigger an “internal” build upon saving. In this case, you can uncheck it from under the Project menu, which makes you in control of when to manually build your project. For example, after saving all needed modifications, the developer can manually do the internal build using Ctrl+B or using Project -> Build All, or do a full build from scratch using whatever build tool he or she is using.

Conclusion

And those were 15 habbits for improving productivity which I hope to be useful for Java developers using Eclipse! Of course many of them may depend on the developer’s preferences, and there are others that are not mentioned.

Check out what’s new in Eclipse Oxygen for Java developers. You can also follow the Eclipse Java IDE for more tips and features.

 

Java 9 Besides Modules

Java 9 is about modularization of the JDK (known as Project Jigsaw). But like previous releases, it is also about lots of other features at the level of the language, the platform API, JDK tools and the JVM.

Image result for java 9

The most important feature of Java 9 is the Java Platform Module System (JPMS). There are other interesting features like improvements to the Process API and new tools like jshell. Over the past couple of years, I wasn’t paying much attention to the other “smaller” changes until I attended this interesting speech at Devoxx France. Now that JDK 9 has been announced Feature Complete earlier this year, this post compiles all those features that are interesting enough for a wide range of developers, and provides detail on each of them. Of course not everything is mentioned below (you can look at the complete feature set here, which is currently frozen).

Process API Updates (JEP 102)

Among the API changes is the introduction of the ProcessHandle interface which makes common operations on native processes much easier.

Retrieve PID of current process

Before Java 9, there is no standard solution to get the native ID of the current process. One could use java.lang.management.ManagementFactory as follows:

// Using a combination of JMX and internal classes
java.lang.management.RuntimeMXBean runtime = java.lang.management.ManagementFactory.getRuntimeMXBean();
java.lang.reflect.Field jvm = runtime.getClass().getDeclaredField("jvm");
jvm.setAccessible(true);
sun.management.VMManagement mgmt = (sun.management.VMManagement) jvm.get(runtime);
java.lang.reflect.Method pid_method = mgmt.getClass().getDeclaredMethod("getProcessId");
pid_method.setAccessible(true);

int pid = (Integer) pid_method.invoke(mgmt);

or rely on parsing a command result:

// Using jps and parsing the result, requires JDK tool on path
Process proc = Runtime.getRuntime().exec(new String[] { "jps", "-l" });

if (proc.waitFor() == 0) {
    InputStream in = proc.getInputStream();
    int available = in.available();
    byte[] outputBytes = new byte[available];

    in.read(outputBytes);
    String output = new String(outputBytes);

    final String[] lines = output.split("\\r?\\n");
    for (String l : lines) {
        if (l.endsWith(TestClass.class.getName())) {
            System.out.println("Your pid is " + l.split(" ")[0]);
        }
    }
}

Starting Java 9, we can use the pid() method of the current ProcessHandle:

long pid = ProcessHandle.current().pid();
Checking if a process is current running

Prior to the introduction of ProcessHandle, checking if a process is alive could be done by running a command such as ps -ef and parsing its output to see if the process is listed. Another workaround relies on the exitValue() method of the Process class, which throws an IllegalThreadStateException if the process has not yet terminated.

Starting Java 9, we can use the isAlive() method:

Process process = ...
boolean isAlive = process.toHandle.isAlive();


// Or given a pid
Optional processHandle = ProcessHandle.of(pid);
boolean isAlive = processHandle.isPresent() && processHandle.get().isAlive();
Retrieving process information

Similarly, if we wanted to retrieve basic information about a certain process, there is no built-in solution. One could parse a command result or rely on a third-party library to do it. In Java 9, we can retrieve a ProcessHandle.Info instance which contains these information:

Optional processHandle = ProcessHandle.of(pid);
if(processHandle.isPresent()) {
    ProcessHandle.Info processInfo = processHandle.get().info();
    System.out.println("Process arguments: " + Arrays.toString(processInfo.arguments().orElse(new String[0])));
    System.out.println("Process executable: " + processInfo.command().orElse(""));
    System.out.println("Process command line: " + processInfo.commandLine().orElse(""));
    System.out.println("Process start time: " + processInfo.startInstant().orElse(null));
    System.out.println("Process total cputime accumulated: " + processInfo.totalCpuDuration().orElse(null));
    System.out.println("Process user: " + processInfo.user().orElse(""));
}
Running post-termination code

Another convenient feature in the new process API is the ability to run code upon process termination. This can be done using the onExit method in either a Process or a ProcessHandle, which returns a CompletableFuture.

Process process = Runtime.getRuntime().exec(command);
process.onExit().thenRun(() -> { System.out.println("Finished!"); });
Getting children and parent processes

A few other methods further make it easier to navigate process trees:

Method name Description
children() Returns a Stream that captures the current direct children.
descendants() Returns a Stream that captures the descendant processes.
allProcesses() A static method that returns a Stream of all processes currently visible.

Note: It is important to keep in mind that the operating system can restrict some of these API, and for the above three methods, that processes are created and terminate asynchronously. There is no guarantee that a process in the stream is alive or that no other processes may have been created since the inception of the snapshot.

Milling Project Coin (JEP 213)

Five items in this JEP introduce small changes to the language:

@SafeVarargs annotation can be applied to private methods

This annotation was introduced in Java 7 to allow a programmer to signal to the compiler that a variable arity method performs safe operations on its varargs parameter. A bit of context about the annotation: due to the type-unsafe nature of mixing generic types with array creation, and because varargs are translated into arrays behind the scenes, the compiler generates a warning for varargs method that use such generic types, as well as warning in all method invocations where there is a generic array creation.

As an example, the following method causes a compiler warning:

m(new ArrayList());  // WARNING type safety

..

static void m(List... stringLists) { // WARNING type safety
    Object[] array = stringLists;
    List tmpList = Arrays.asList(42);
    array[0] = tmpList; // Semantically invalid, but compiles without warnings
    String s = stringLists[0].get(0); // Oh no, ClassCastException at runtime!
}

To make the compiler ignore this (although it would be the wrong thing to do in this case), we can annotate the method with @SafeVarargs.

So what changed in Java 9? Before Java 9, @SafeVarargs could be applied to either static or final methods. In Java 9, the annotation can also be used on private methods.

Allow effectively final variables to be used in a try-with-resources statement

Try-with-resources statements require a local variable declaration for the resource:

// Before Java 9
try(FileReader fileReader = new FileReader("input")) {
   ...
}

Now it is possible to use the statement on a resource without declaring a local variable for it, as long as the variable referencing the resource is final or effectively final.

// Java 9
FileReader fileReader = new FileReader("input")
... // code that does not re-assign fileReader
try(fileReader) {
   ...
}
Allow operator in anonymous classes

For reasons related to the compiler’s type inference implementation, diamond operators were not allowed when instantiating anonymous classes in Java 7. With this change, as long as the compiler can infer the type argument of the anonymous class, we can use the operator:

Comparator compareByHeight = new Comparator<>() {
    @Override
    public int compare(Person p1, Person p2) {
        return p2.getHeight() - p1.getHeight();
    }
}
Forbid the underscore as an identifier

In Java 8, Java compilers started to issue a warning on using an underscore as an identifier, and an error when used in lambdas. With Java 9, an error is thrown in all cases underscores are used:

$ javac -source 9 Lambda.java 
Lambda.java:2: error: as of release 9, '_' is a keyword, and may not be used as an identifier 
    public int a(int _); 
                     ^ 
Lambda.java:4: error: '_' used as an identifier 
        t(_ -> 0); 
          ^ 
  (use of '_' as an identifier is forbidden for lambda parameters) 
2 errors
Private methods in interfaces

When default methods were being added to interfaces in Java 8, private methods were being considered as well. But it was later postponed. Now with Java 9, interfaces can have private methods (static and instance) to allow non-abstract methods to share code:

interface Greeter {
    ...

    default void greetSomeone(String personName, boolean female) {
        System.out.println("Hi " + getTitle(female) + " " + personName);
    }

    default void farewellSomeone(String personName, boolean female) {
        System.out.println("Bye " + getTitle(female) + " " + personName);
    }

    private String getTitle(boolean female) {
        return female ? "Ms" : "Mr";
    }
}

New Doclet API (JEP 221)

The Doclet API which allowed client applications to customize the output of javadoc underwent a re-design as defined in JEP 221. The goal was to make use of the Language Model API introduced in Java 6 as a standard of representing program elements, as well as the DocTree API to represent documentation elements in the source code, and get rid of using the old language model that was part of the old Doclet API. The outcome is a migration of the old API from the com.sun.javadoc package to a new jdk.javadoc.doclet package.

Also, the standard doclet that the javadoc tool uses by default to generate HTML documentation has been adapted to use the new API.

JShell: The Java Shell (JEP 222)

JShell is a REPL (Read-Eval-Print-Loop) tool that allows snippets of code to be run without having to place them in classes. It is similar to what exists in other JVM based languages such as Groovy or Scala. One of the motivations behind JShell was that “the number one reason schools cite for moving away from Java as a teaching language is that other languages have a REPL and have far lower bars to an initial Hello, world! program”. Meh… a bit debatable maybe. But a more convincing rationale would be to facilitate quick prototyping of new code without having to compile and run and without having to open an IDE.

In addition to the command line tool, JShell comes with an API to allow other tools to integrate JShell’s functionality.

Some rules such as ending statements with semi-colons and checked exceptions are relaxed. You can even declared variables of some type that you define after declaring the variable. The class path and module path can also be changed during the session, or when starting JShell the first time (using --class-path and --module-path).

|  Welcome to JShell -- Version 9-ea
|  For an introduction type: /help intro

jshell> 1
$1 ==> 1

jshell> System.out.println("Statement without semi-colon")
Statement without semi-colon

jshell> import java.util.regex.*

jshell> boolean match = Pattern.matches("a*b", "aaab")
match ==> true

jshell> import java.io.*

jshell> FileReader fr = new FileReader("input.txt")
fr ==> java.io.FileReader@42e26948

jshell> /vars
|    int $1 = 1
|    boolean match = true
|    FileReader fr = java.io.FileReader@42e26948

jshell> /save session.txt

jshell> /open session.txt
Statement without semi-colon

jshell> /env --class-path  lib/commons-lang3-3.5.jar
|  Setting new options and restoring state.
Statement without semi-colon
Statement without semi-colon


jshell> /env
|     --class-path lib\commons-lang3-3.5.jar


jshell> import org.apache.commons.lang3.StringUtils

jshell> Foo x
|  created variable x, however, it cannot be referenced until class Foo is declared

jshell> class Foo {}
|  created class Foo
|    update replaced variable x, reset to null

jshell> /exit
|  Goodbye

New Versioning Scheme (JEP 223)

Throughout the past 20+ years, the versioning of Java releases was inconsistent and sometimes confusing. The first two major releases were JDK 1.0 and JDK 1.1. From 1.2 till 1.5, the platform was referred to as J2SE (for the Standard Edition). Then starting 1.5, the versioning changed to become Java 5, then Java 6, and so on. However, when you run java -version with an installed Java 8, the output still shows 1.8 instead of 8. The current versioning scheme for releases, introduced after Oracle acquired Sun, goes as follows:

  • For Limited Update Releases (no critical security fixes), release numbers will multiples of 20.
  • For Critical Patch Updates (fixes security vulnerabilities), the release number will be calculated by adding multiples of five to the prior Limited Update and when needed adding one to keep the resulting number odd.
Version numbers

Starting Java 9, the versioning will be consistent with semantic versioning, and version numbers have the format $MAJOR.$MINOR.$SECURITY(.$otherpart)? where:

  • $MAJOR is the major version number, and is incremented when a major version is released that typically changes the platform specification. For JDK 9, this value will be 9.
  • $MINOR is the minor version number, and is incremented for releases that contain bug fixes and enhancements to standard APIs.
  • $SECURITY is the security level, and is incremented for releases that contain critical security fixes. This version is not reset to zero when the minor version number is incremented.
  • $otherpart consists of one or more versions that can be used by JVM providers to indicate a patch with a small number of non-security fixes.
Version strings

The version string will be the version number with some other information such as early-access release identifier or the build number:

$VNUM(-$PRE)?\+$BUILD(-$OPT)?

$VNUM-$PRE(-$OPT)?

$VNUM(+-$OPT)?

where:

  • $PRE is a pre-release identifier.
  • $BUILD is the build number
  • $OPT is optional information such as the timestamp.

For comparison, the versioning for JDK 9 using both the existing and upcoming schemes is shown in the below table:

                      Existing                New
Release Type    long           short    long           short
------------    --------------------    --------------------
Early Access    1.9.0-ea-b19    9-ea    9-ea+19        9-ea
Major           1.9.0-b100      9       9+100          9
Security #1     1.9.0_5-b20     9u5     9.0.1+20       9.0.1
Security #2     1.9.0_11-b12    9u11    9.0.2+12       9.0.2
Minor #1        1.9.0_20-b62    9u20    9.1.2+62       9.1.2
Security #3     1.9.0_25-b15    9u25    9.1.3+15       9.1.3
Security #4     1.9.0_31-b08    9u31    9.1.4+8        9.1.4
Minor #2        1.9.0_40-b45    9u40    9.2.4+45       9.2.4

The new versioning scheme is fully documented in the Runtime.Version class and version information can be accessed from it:

System.out.println(Runtime.version().toString());  // 9-ea+167
System.out.println(Runtime.version().major());     // 9
System.out.println(Runtime.version().minor());     // 0
System.out.println(Runtime.version().security());  // 0

Javadoc Search and HTML5 (JEPs 224-225)

Have you noticed anything new in the content of Javadoc pages for Java 9 API so far? Look at the top right of the main frame. That’s right, there is now a search box that you can use to search for classes, methods, etc. It’s hard to believe it took them so much to implement it. The pages are also in HTML5 by default instead of HTML 4.

The searching is done locally, and the things that can be searched are:

  • Modules, packages, types and members.
  • Text that is marked with the tag @index.

New HotSpot Diagnostic Commands (JEP 228)

The following JVM diagnosis commands were added:

  • print_class_summary: print all loaded classes and their hierarchy
  • print_codegenlist: show the queue of methods to be compiled in C1 and C2 compilers
  • print_utf8pool: print string table
  • datadump_request: signal the JVM to do a data-dump request for JVMTI
  • dump_codelist: print all compiled methods in code cache that are alive
  • print_codeblocks: print code cache layout and bounds
  • set_vmflag: set VM flag option using the provided value

Here is a sample test that sends a command to print the string table using the jcmd utility:

C:\Users\manouti>jcmd
6448 sun.tools.jcmd.JCmd
6392 test.ProcessHandleExample

C:\Users\manouti>jcmd ProcessHandleExample VM.stringtable
6392:
StringTable statistics:
Number of buckets       :     60013 =    480104 bytes, avg   8.000
Number of entries       :      2478 =     59472 bytes, avg  24.000
Number of literals      :      2478 =    162896 bytes, avg  65.737
Total footprint         :           =    702472 bytes
Average bucket size     :     0.041
Variance of bucket size :     0.042
Std. dev. of bucket size:     0.204
Maximum bucket size     :         3

Create PKCS12 Keystores by Default (JEP 229)

Starting Java 9, keystores are created using the PKCS12 format instead of JKS, because it offers more stronger cryptographic algorithms. This change is backward compatible, so applications accessing existing keystores continue to work.

Multi-Release JAR Files (JEP 238)

One of the most interesting features introduced in Java 9 is the multi-release Jar (MRJAR) which allows bundling code targeting multiple Java releases within the same Jar file. By setting Multi-Release: true in the MANIFEST.MF file, the file becomes a multi-release Jar and the Java runtime will pick the appropriate versions of classes depending on the current major version running. The structure of such a file is illustrated as follows:

jar root
  - A.class
  - B.class
  - C.class
  - D.class
  - META-INF
     - versions
        - 9
           - A.class
           - B.class
        - 10
           - A.class
  • On JDKs < 9, only the classes in the root entry are visible to the Java runtime.
  • On a JDK 9, the classes A and B will be loaded from the directory root/META-INF/versions/9, while C and D will be loaded from the base entry.
  • On a JDK 10, class A would be loaded from the directory root/META-INF/versions/10.

A multi-release Jar file allows projects to maintain different versions of their code targeting different Java platforms, while being able to distribute the code as one Jar, with a single version (e.g. Maven artifact version). This relaxes the common restriction of writing backward compatible code, and allows developers to benefit from new language and API changes incrementally as they add new code.

This feature naturally requires modifications to some APIs used to process Jar files, such as JarFile and URLClassLoader. Also many JDK tools have been adapted to be aware of the new format, such as java, javac and jar.

As an example, the jar command can be used to create a multi-release Jar containing two versions of the same class compiled for both Java 8 and Java 9, albeit with a warning telling that the classes are identical:

C:\Users\manouti>jar --create --file MR.jar \
              -C sampleproject-base demo \
              --release 9 -C sampleproject-9 demo
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar

This creates an MRJAR named MR.jar with the following contents:

jar root
  - demo
     - SampleClass.class
  - META-INF
     - versions
        - 9
           - demo
              - SampleClass.class

Let us now create a class called Main that prints the URL of the SampleClass, and add it for the Java 9 version:

package demo;

import java.net.URL;

public class Main {

    public static void main(String[] args) throws Exception {
		URL url = Main.class.getClassLoader().getResource("demo/SampleClass.class");
		System.out.println(url);
	}
}

If we compile this class and re-run the jar command, we get an error:

C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo \
                     --release 9 -C sampleproject-9 demo
entry: META-INF/versions/9/demo/Main.class, contains a new public class not found in base entries
Warning: entry META-INF/versions/9/demo/Main.java, multiple resources with same name
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar
invalid multi-release jar file MR.jar deleted

It turns out that the jar tool prevents adding public classes to versioned entries if they are not added to the base entries as well. This is done so that the MRJAR exposes the same public API for the different Java versions. Note that at runtime, this rule is not required. It may be only applied by tools like jar. In this particular case, the purpose of Main is to run sample code, so we can simply add a copy in the base entry. If the class were part of a newer implementation that we only need for Java 9, it could be made non-public.

To add Main to the root entry, we first need to compile it to target a pre-Java 9 release. This can be done using the new --release option of javac (see JEP 247 – Compile for Older Platform Versions):

C:\Users\manouti\sampleproject-base\demo>javac --release 8 Main.java
C:\Users\manouti\sampleproject-base\demo>cd ../..
C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo \
                     --release 9 -C sampleproject-9 demo

Running the Main class shows that the SampleClass gets loaded from the versioned directory:

C:\Users\manouti>java --class-path MR.jar demo.Main
jar:file:/C:/Users/manouti/MR.jar!/META-INF/versions/9/demo/SampleClass.class

Remove the JVM TI hprof Agent (JEP 240)

The hprof JVM native agent was removed. Before Java 9, it could be used to dump the heap or profile the CPU. The reason it was removed is the existence of better alternatives. For example, jmap can do the heap dump, while JVisualVM can be used to profile running applications.

To demonstrate the impact of removing this agent, we can run a Java program with the hprof agent enabled (i.e. using the option -agentlib:hprof) on Java 8 and then on Java 9 (which is added on my system path):

C:\Users\manouti>D:\Dev\Java\jdk1.8.0_121\bin\java.exe -agentlib:hprof test.ProcessHandleExample
Running...
Dumping Java heap ... allocation sites ... done.

C:\Users\manouti>java -agentlib:hprof test.ProcessHandleExample
Error occurred during initialization of VM
Could not find agent library hprof on the library path, with error: Can't find dependent libraries

In all cases, this agent was not an official part of the JDK and it was rarely used by existing applications.

Remove the jhat Tool (JEP 241)

The jhat tool which could be used to browse a heap dump in a Web browser was removed since better alternatives exist. The tool was marked experimental and subject to removal in previous releases.

Compile for Older Platform Versions (JEP 247)

Before Java 9, we used to apply -source to tell it to use the selected language specification and -target to generate a certain version of bytecode. However, this can still lead to runtime issues as the compiler will link compiled classes to platform APIs of the current version of the JDK (unless you override the boot classpath). In Java 9, these options are replaced with one simple option --release to be able to compile for an older version.

--release is equivalent to -source N -target N -bootclasspath

JDK 9 makes this possible by maintaining some signature data about APIs from old releases, specifically under $JDK_HOME/lib/ct.sym.

G1 as Default Garbage Collector (JEP 248)

Prior to Java 9, the default garbage collector was typically the Parallel GC on server VMs and the Serial GC on client ones. On Java 9, server VMs will use G1 as the default, which was introduced in Java 7. G1 is a parallel and low-pause garbage collector that works especially well for multi-core machines with big heap sizes. For an overview of the G1 collector, see http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html. In addition to this feature, the Concurrent Mark Sweep (CMS) collector was deprecated.

Multi-Resolution Images (JEP 251)

A new interface MultiResolutionImage is added with a base implementation BaseMultiResolutionImage that can encapsulate several image variants with different sizes. This interface can be used to select the best image variant given certain width and height values.

Compact Strings (JEP 254)

An internal optimization is applied to the String class to reduce memory consumption. The idea is that most String objects contain characters that do not need 2 bytes to represent. The change consists of replacing the internal character array with a byte array, plus an extra byte that denotes the encoding of the byte array: either Latin-1 which takes up 1 byte, or UTF-16 which takes up 2 bytes. The String class will determine which encoding based on the content to be stored.

This is expected to reduce the size of heap memory in existing applications that rely heavily on strings, and should also reduce time spent on garbage collection. This change is internal and does not affect the external API of String and its related classes such as StringBuilder or StringBuffer.

Stack-Walking API (JEP 259)

Prior to Java 9, access to the thread stack frames was limited to an internal class sun.reflect.Reflection. Specifically the method sun.reflect.Reflection::getCallerClass. Some libraries relies on this method which is deprecated. An alternative standard API is now provided in JDK 9 via the StackWalker class, and is designed to be efficient by allowing lazy access to the stack frames. Some applications may use this API to traverse the execution stack and filter on classes. Two methods are of interest in this class:

  • public T walk(Function function); which allow traversing a stream of stack frames for the current thread, starting from the top frame, and applying the given Function on the stream.
  • public Class getCallerClass();which returns the class that invoked the method that calls this method.

This class is thread-safe, so multiple threads can use the same instance to walk their stacks.

For example, the following prints all stack frames of the current thread:

package test;

import java.lang.StackWalker.StackFrame;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import java.util.stream.Collectors;

public class StackWalkerExample {

    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
        Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
        fooMethod.invoke(null, (Object[]) null);
    }
}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        List stack = StackWalker.getInstance()
                .walk((s) -> s.collect(Collectors.toList()));
        for(StackFrame frame : stack) {
            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
        }
    }
}

Output:

test.BarHelper 26 bar

test.FooHelper 19 foo

test.StackWalkerExample 13 main

The following prints the current caller class. Note that in this case, the StackWalker needs to be created with the option RETAIN_CLASS_REFERENCE, so that Class instances are retained in the StackFrame objects. Otherwise an exception would occur.

public class StackWalkerExample {

    public static void main(String[] args) {
        FooHelper.foo();
    }

}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        System.out.println(StackWalker.getInstance(Option.RETAIN_CLASS_REFERENCE).getCallerClass());
    }
}

Output:

class test.FooHelper

A couple of other options allow stack traces to include implementation and/or reflection frames. This may be useful for debugging purposes. For instance, the first example includes some reflection to invoke FooHelper, but the reflection methods were not shown in the output. We can add the SHOW_REFLECT_FRAMES option to the StackWalker instance upon creation, so that the frames for the reflective methods are printed as well:

package test;

import java.lang.StackWalker.Option;
import java.lang.StackWalker.StackFrame;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import java.util.stream.Collectors;

public class StackWalkerExample {

    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
       Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
        fooMethod.invoke(null, (Object[]) null);
    }
}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        List stack = StackWalker.getInstance(Option.SHOW_REFLECT_FRAMES)       // show reflection methods
                .walk((s) -> s.collect(Collectors.toList()));
        for(StackFrame frame : stack) {
            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
        }
    }
}

Output:

test.BarHelper 27 bar

test.FooHelper 20 foo

jdk.internal.reflect.NativeMethodAccessorImpl -2 invoke0

jdk.internal.reflect.NativeMethodAccessorImpl 62 invoke

jdk.internal.reflect.DelegatingMethodAccessorImpl 43 invoke

java.lang.reflect.Method 563 invoke

test.StackWalkerExample 14 main

Note that line numbers for some reflection methods may not be available so StackFrame.getLineNumber() may return negative values.

Encapsulate Internal APIS (JEP 260)

Types whose packages start with sun. (and some starting with com.sun.) are internal to the Java platform. The Java development team has long discouraged their use since they are not supported and vary from implementation to another. Now, most of these classes have been encapsulated in modules that don’t export them for code outside the JDK. This means that existing code that rely on them will break; however this decision was based on the analysis that these API are very rarely, or either they have official replacements that exist prior to JDK 9. Exceptionally, the following internal APIs remained exported and placed in a separate module, and for those have supported replacements in JDK 9 they have been deprecated and may be encapsulated or removed in JDK 10:

  • sun.misc.{Signal,SignalHandler}
  • sun.misc.Unsafe
  • sun.reflect.Reflection::getCallerClass
  • sun.reflect.ReflectionFactory.newConstructorForSerialization

These APIs have been exported to public use because they are heavily used by some libraries, notably Unsafe. They are used to perform intrinsic JVM operations that are otherwise impossible to achieve.

Reactive Stream Flow API (JEP 266)

A standard set of interfaces corresponding to the reactive-streams specification have been introduced nested under the Flow class. These interfaces define a publish-subscribe mechanism that allows asynchronous flow-controlled communication between producers and consumers of data streams.

  1. Flow.Publisher: specifies a producer of data items and has one method subscribe(Flow.Subscriber subscriber) that adds a subscriber to be sent items.
  2. Flow.Subscriber: specifies a consumer of data items, and defines four methods:
    1. onSubscribe(Flow.Subscription subscription): invoked upon subscription of the subscriber by the publisher, and passes an instance of a Flow.Subscription to allow the subscriber to control the flow.
    2. onNext(T item): Invoked when a new item is received by the subscriber.
    3. onComplete(): invoked when all items have been sent by the publisher and no further items are to be received.
    4. onError(Throwable throwable): invoked when an error occurs on this subscription. No subsequent items are received.
  3. Flow.Subscription: represents a subscription and can be used by the subscriber to control the flow of data, and defines two methods:
    1. request(long n): can be invoked by the consumer to demand up to n items to be sent from the publisher. The number may be < n if the publisher finished sending all items.
    2. cancel(): invoked to cancel the subscription so that no more items are received.
  4. Flow.Processor: implements both a Publisher and a Subscriber and is used to transform data items. It can be used as a medium between a publisher and a subscriber.

Using this model of communication, the subscriber is more in control of the flow of data so that the rate of messages can be handled more efficiently. There is one additional utility class SubmissionPublisher that can be used by item generators that want to publish their data. This class implements the Flow.Publisher and AutoCloseable interfaces and can be closed to complete sending its items.

Convenience Factory Methods for Collections (JEP 269)

The interfaces List, Set and Map have been enriched for factory methods for immutable collections:

  1. 12 Overloaded of(...) factory methods for Set and List. One with a varargs parameter.
  2. 11 Overloaded of(...) factory methods for Map that take key and value arguments. Plus one that takes a varargs of Entry objects ofEntries(Entry... entries).

The returned collections are instances of nested types defined under java.util.ImmutableCollections. This class is package-private so it cannot used to check if the collection is immutable. This is left as an implementation detail of your application.

Enhancements to Streams

Apart from collections, new methods were added to java.util.stream.Stream. The first two are normally intended to be used when the stream is ordered:

Stream takeWhile(Predicate predicate)

This method returns, for an ordered stream, a stream that consists of the longest prefix of elements that match the give predicate. The returned stream consists of the prefix of elements matching the predicate in that order. If the original stream is unordered, the order of elements is nondeterministic; the implementation is free to return any subset of elements that matches the predicate. However, if it happens that all elements match the predicate then the returned stream will consist of the same sequence of elements of the original (in the same order). If no element matches the predicate, an empty stream is returned.

An example that returns the first 5 integers of an ordered infinite stream:

Stream infiniteInts = Stream.iterate(0, i -> i + 1);
infiniteInts.takeWhile(i -> i < 5).forEach(System.out::println);

Output:

0

1

2

3

4

Example with an unordered stream:

Stream unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.takeWhile(i -> i < 5).forEach(System.out::println);

Output:

3

2

1

0

Note that if we run this last example, it may return any other result that excludes 5.

Stream dropWhile(Predicate predicate)

This method does the opposite. It returns, for an ordered stream, a stream consisting of the elements remaining after dropping the longest prefix of elements matching the predicate. If the original stream is unordered, the behavior is indeterministic; the implementation is free to drop any subset of elements matching the predicate and return the remaining elements in the stream.

Example using an ordered stream:

Stream finiteInts = Stream.iterate(0, i -> i < 10, i -> i + 1);  // iterate takes a seed element, a Predicate, and an UnaryOperator
finiteInts.dropWhile(i -> i < 5).forEach(System.out::println);

Output:

5

6

7

8

9

Example using an ordered stream:

Stream unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.dropWhile(i -> i < 5).forEach(System.out::println);

Output:

5

4


Note that if we run this last example, it may return some other stream that contains 5.

Stream iterate(T seed, Predicate hasNext, UnaryOperator next)

This method returns an ordered stream stream that applies iteratively the predicate starting on an initial value, until the predicate returns a false with the subsequent increment specified by the UnaryOperator. It is conceptually similar to the traditional for loop:

for (T index = seed; hasNext.test(index); index = next.apply(index)) {

   ...

}

Stream ofNullable(T t)

This method returns the given element if not null; otherwise an empty stream.

Enhancements to Optional

Three new methods were added to Optional:

void ifPresentOrElse(Consumer action, Runnable emptyAction)

If a value is present, performs the given action with the value, otherwise performs the given empty-based action.

Optional or(Supplier supplier)

If a value is present, returns an Optional describing the value, otherwise returns an Optional produced by the supplying function.

Stream stream()

If a value is present, returns a sequential Stream containing only that value, otherwise returns an empty Stream.

Enhanced Deprecation (JEP 277)

The following elements have been added to the @Deprecated annotation to provide more information about the deprecation status:

1. forRemoval() If true, the deprecated API is intended to be removed in a future JDK release.

2. since() which indicated since which release this API has been deprecated.

As part of this JEP, some existing API elements have been planned for deprecated, such as legacy collections (e.g. Vector and Hashtable); some deprecated elements have been marked as for removal such as Thread.stop() and Thread.destroy().

Furthermore, a new utility jdeprscan has been added in the JDK tools to scan a Jar file or a set of classes for usage of deprecated code. Note that it only scans for deprecated code from the standard libraries.

Spin-Wait Hints (JEP 285)

This feature introduces a new method in java.lang.Thread called onSpinWait() which allows application code to provide a hint to the JVM that it running in a spin-loop, meaning it is busy waiting for some event to occur. The JVM can benefit from this hint to execute some intrinsic code that leverages hardware platform specific instructions. For example, x86 processors can execute a PAUSE instruction to indicate spin-wait, which can improve performance.

Note that the method onSpinWait does nothing. It is simply marked as an intrinsic candidate (using the jdk.internal.HotSpotIntrinsicCandidate annotation) to indicate to the JVM that it can replace it with native code optimized for the target platform. The method is just a hint, so the JVM is free to simply do nothing. A typical scenario where this method may be used is illustrated in the docs:

class EventHandler {
    volatile boolean eventNotificationNotReceived;
    void waitForEventAndHandleIt() {
        while ( eventNotificationNotReceived ) {
            java.lang.Thread.onSpinWait();
        }
        readAndProcessEvent();
    }

    void readAndProcessEvent() {
        // Read event from some source and process it
         . . .
    }
}

Applet API Deprecated (JEP 289)

Due to the decreasing support of Java plug-ins by Web browsers, the Applet API is deprecated. However, there is no intention to remove it in the next major release, so there is no forRemoval = true in the @Deprecated.