Category Archives: Programming

Java 9 Besides Modules

Java 9 is about modularization of the JDK (known as Project Jigsaw). But like previous releases, it is also about lots of other features at the level of the language, the platform API, JDK tools and the JVM.

Image result for java 9

The most important feature of Java 9 is the Java Platform Module System (JPMS). There are other interesting features like improvements to the Process API and new tools like jshell. Over the past couple of years, I wasn’t paying much attention to the other “smaller” changes until I attended this interesting speech at Devoxx France. Now that JDK 9 has been announced Feature Complete earlier this year, this post compiles all those features that are interesting enough for a wide range of developers, and provides detail on each of them. Of course not everything is mentioned below (you can look at the complete feature set here, which is currently frozen).

Process API Updates (JEP 102)

Among the API changes is the introduction of the ProcessHandle interface which makes common operations on native processes much easier.

Retrieve PID of current process

Before Java 9, there is no standard solution to get the native ID of the current process. One could use java.lang.management.ManagementFactory as follows:

// Using a combination of JMX and internal classes
java.lang.management.RuntimeMXBean runtime = java.lang.management.ManagementFactory.getRuntimeMXBean();
java.lang.reflect.Field jvm = runtime.getClass().getDeclaredField("jvm");
jvm.setAccessible(true);
sun.management.VMManagement mgmt = (sun.management.VMManagement) jvm.get(runtime);
java.lang.reflect.Method pid_method = mgmt.getClass().getDeclaredMethod("getProcessId");
pid_method.setAccessible(true);

int pid = (Integer) pid_method.invoke(mgmt);

or rely on parsing a command result:

// Using jps and parsing the result, requires JDK tool on path
Process proc = Runtime.getRuntime().exec(new String[] { "jps", "-l" });

if (proc.waitFor() == 0) {
	InputStream in = proc.getInputStream();
	int available = in.available();
	byte[] outputBytes = new byte[available];

	in.read(outputBytes);
	String output = new String(outputBytes);

	final String[] lines = output.split("\\r?\\n");
	for (String l : lines) {
		if (l.endsWith(TestClass.class.getName())) {
			System.out.println("Your pid is " + l.split(" ")[0]);
		}
	}
}

Starting Java 9, we can use the pid() method of the current ProcessHandle:

long pid = ProcessHandle.current().pid();
Checking if a process is current running

Prior to the introduction of ProcessHandle, checking if a process is alive could be done by running a command such as ps -ef and parsing its output to see if the process is listed. Another workaround relies on the exitValue() method of the Process class, which throws an IllegalThreadStateException if the process has not yet terminated.

Starting Java 9, we can use the isAlive() method:

Process process = ...
boolean isAlive = process.toHandle.isAlive();


// Or given a pid
Optional processHandle = ProcessHandle.of(pid);
boolean isAlive = processHandle.isPresent() && processHandle.get().isAlive();
Retrieving process information

Similarly, if we wanted to retrieve basic information about a certain process, there is no built-in solution. One could parse a command result or rely on a third-party library to do it. In Java 9, we can retrieve a ProcessHandle.Info instance which contains these information:

Optional processHandle = ProcessHandle.of(pid);
if(processHandle.isPresent()) {
    ProcessHandle.Info processInfo = processHandle.get().info();
    System.out.println("Process arguments: " + Arrays.toString(processInfo.arguments().orElse(new String[0])));
    System.out.println("Process executable: " + processInfo.command().orElse(""));
    System.out.println("Process command line: " + processInfo.commandLine().orElse(""));
    System.out.println("Process start time: " + processInfo.startInstant().orElse(null));
    System.out.println("Process total cputime accumulated: " + processInfo.totalCpuDuration().orElse(null));
    System.out.println("Process user: " + processInfo.user().orElse(""));
}
Running post-termination code

Another convenient feature in the new process API is the ability to run code upon process termination. This can be done using the onExit method in either a Process or a ProcessHandle, which returns a CompletableFuture.

Process process = Runtime.getRuntime().exec(command);
process.onExit().thenRun(() -> { System.out.println("Finished!"); });
Getting children and parent processes

A few other methods further make it easier to navigate process trees:

Method name Description
children() Returns a Stream that captures the current direct children.
descendants() Returns a Stream that captures the descendant processes.
allProcesses() A static method that returns a Stream of all processes currently visible.

Note: It is important to keep in mind that the operating system can restrict some of these API, and for the above three methods, that processes are created and terminate asynchronously. There is no guarantee that a process in the stream is alive or that no other processes may have been created since the inception of the snapshot.

Milling Project Coin (JEP 213)

Five items in this JEP introduce small changes to the language:

@SafeVarargs annotation can be applied to private methods

This annotation was introduced in Java 7 to allow a programmer to signal to the compiler that a variable arity method performs safe operations on its varargs parameter. A bit of context about the annotation: due to the type-unsafe nature of mixing generic types with array creation, and because varargs are translated into arrays behind the scenes, the compiler generates a warning for varargs method that use such generic types, as well as warning in all method invocations where there is a generic array creation.

As an example, the following method causes a compiler warning:

m(new ArrayList());  // WARNING type safety

..

static void m(List... stringLists) { // WARNING type safety
	Object[] array = stringLists;
	List tmpList = Arrays.asList(42);
	array[0] = tmpList; // Semantically invalid, but compiles without warnings
	String s = stringLists[0].get(0); // Oh no, ClassCastException at runtime!
}

To make the compiler ignore this (although it would be the wrong thing to do in this case), we can annotate the method with @SafeVarargs.

So what changed in Java 9? Before Java 9, @SafeVarargs could be applied to either static or final methods. In Java 9, the annotation can also be used on private methods.

Allow effectively final variables to be used in a try-with-resources statement

Try-with-resources statements require a local variable declaration for the resource:

// Before Java 9
try(FileReader fileReader = new FileReader("input")) {
   ...
}

Now it is possible to use the statement on a resource without declaring a local variable for it, as long as the variable referencing the resource is final or effectively final.

// Java 9
FileReader fileReader = new FileReader("input")
... // code that does not re-assign fileReader
try(fileReader) {
   ...
}
Allow <> operator in anonymous classes

For reasons related to the compiler’s type inference implementation, diamond operators were not allowed when instantiating anonymous classes in Java 7. With this change, as long as the compiler can infer the type argument of the anonymous class, we can use the operator:

Comparator compareByHeight = new Comparator<>() {
    @Override
    public int compare(Person p1, Person p2) {
        return p2.getHeight() - p1.getHeight();
    }
}
Forbid the underscore as an identifier

In Java 8, Java compilers started to issue a warning on using an underscore as an identifier, and an error when used in lambdas. With Java 9, an error is thrown in all cases underscores are used:

$ javac -source 9 Lambda.java 
Lambda.java:2: error: as of release 9, '_' is a keyword, and may not be used as an identifier 
    public int a(int _); 
                     ^ 
Lambda.java:4: error: '_' used as an identifier 
        t(_ -> 0); 
          ^ 
  (use of '_' as an identifier is forbidden for lambda parameters) 
2 errors
Private methods in interfaces

When default methods were being added to interfaces in Java 8, private methods were being considered as well. But it was later postponed. Now with Java 9, interfaces can have private methods (static and instance) to allow non-abstract methods to share code:

interface Greeter {
    ...

    default void greetSomeone(String personName, boolean female) {
        System.out.println("Hi " + getTitle(female) + " " + personName);
    }

    default void farewellSomeone(String personName, boolean female) {
        System.out.println("Bye " + getTitle(female) + " " + personName);
    }

    private String getTitle(boolean female) {
        return female ? "Ms" : "Mr";
    }
}

New Doclet API (JEP 221)

The Doclet API which allowed client applications to customize the output of javadoc underwent a re-design as defined in JEP 221. The goal was to make use of the Language Model API introduced in Java 6 as a standard of representing program elements, as well as the DocTree API to represent documentation elements in the source code, and get rid of using the old language model that was part of the old Doclet API. The outcome is a migration of the old API from the com.sun.javadoc package to a new jdk.javadoc.doclet package.

Also, the standard doclet that the javadoc tool uses by default to generate HTML documentation has been adapted to use the new API.

JShell: The Java Shell (JEP 222)

JShell is a REPL (Read-Eval-Print-Loop) tool that allows snippets of code to be run without having to place them in classes. It is similar to what exists in other JVM based languages such as Groovy or Scala. One of the motivations behind JShell was that “the number one reason schools cite for moving away from Java as a teaching language is that other languages have a REPL and have far lower bars to an initial Hello, world! program”. Meh… a bit debatable maybe. But a more convincing rationale would be to facilitate quick prototyping of new code without having to compile and run and without having to open an IDE.

In addition to the command line tool, JShell comes with an API to allow other tools to integrate JShell’s functionality.

Some rules such as ending statements with semi-colons and checked exceptions are relaxed. You can even declared variables of some type that you define after declaring the variable. The class path and module path can also be changed during the session, or when starting JShell the first time (using --class-path and --module-path).

|  Welcome to JShell -- Version 9-ea
|  For an introduction type: /help intro

jshell> 1
$1 ==> 1

jshell> System.out.println("Statement without semi-colon")
Statement without semi-colon

jshell> import java.util.regex.*

jshell> boolean match = Pattern.matches("a*b", "aaab")
match ==> true

jshell> import java.io.*

jshell> FileReader fr = new FileReader("input.txt")
fr ==> java.io.FileReader@42e26948

jshell> /vars
|    int $1 = 1
|    boolean match = true
|    FileReader fr = java.io.FileReader@42e26948

jshell> /save session.txt

jshell> /open session.txt
Statement without semi-colon

jshell> /env --class-path  lib/commons-lang3-3.5.jar
|  Setting new options and restoring state.
Statement without semi-colon
Statement without semi-colon


jshell> /env
|     --class-path lib\commons-lang3-3.5.jar


jshell> import org.apache.commons.lang3.StringUtils

jshell> Foo x
|  created variable x, however, it cannot be referenced until class Foo is declared

jshell> class Foo {}
|  created class Foo
|    update replaced variable x, reset to null

jshell> /exit
|  Goodbye

New Versioning Scheme (JEP 223)

Throughout the past 20+ years, the versioning of Java releases was inconsistent and sometimes confusing. The first two major releases were JDK 1.0 and JDK 1.1. From 1.2 till 1.5, the platform was referred to as J2SE (for the Standard Edition). Then starting 1.5, the versioning changed to become Java 5, then Java 6, and so on. However, when you run java -version with an installed Java 8, the output still shows 1.8 instead of 8. The current versioning scheme for releases, introduced after Oracle acquired Sun, goes as follows:

  • For Limited Update Releases (no critical security fixes), release numbers will multiples of 20.
  • For Critical Patch Updates (fixes security vulnerabilities), the release number will be calculated by adding multiples of five to the prior Limited Update and when needed adding one to keep the resulting number odd.
Version numbers

Starting Java 9, the versioning will be consistent with semantic versioning, and version numbers have the format $MAJOR.$MINOR.$SECURITY(.$otherpart)? where:

  • $MAJOR is the major version number, and is incremented when a major version is released that typically changes the platform specification. For JDK 9, this value will be 9.
  • $MINOR is the minor version number, and is incremented for releases that contain bug fixes and enhancements to standard APIs.
  • $SECURITY is the security level, and is incremented for releases that contain critical security fixes. This version is not reset to zero when the minor version number is incremented.
  • $otherpart consists of one or more versions that can be used by JVM providers to indicate a patch with a small number of non-security fixes.
Version strings

The version string will be the version number with some other information such as early-access release identifier or the build number:

$VNUM(-$PRE)?\+$BUILD(-$OPT)?

$VNUM-$PRE(-$OPT)?

$VNUM(+-$OPT)?

where:

  • $PRE is a pre-release identifier.
  • $BUILD is the build number
  • $OPT is optional information such as the timestamp.

For comparison, the versioning for JDK 9 using both the existing and upcoming schemes is shown in the below table:

                      Existing                New
Release Type    long           short    long           short
------------    --------------------    --------------------
Early Access    1.9.0-ea-b19    9-ea    9-ea+19        9-ea
Major           1.9.0-b100      9       9+100          9
Security #1     1.9.0_5-b20     9u5     9.0.1+20       9.0.1
Security #2     1.9.0_11-b12    9u11    9.0.2+12       9.0.2
Minor #1        1.9.0_20-b62    9u20    9.1.2+62       9.1.2
Security #3     1.9.0_25-b15    9u25    9.1.3+15       9.1.3
Security #4     1.9.0_31-b08    9u31    9.1.4+8        9.1.4
Minor #2        1.9.0_40-b45    9u40    9.2.4+45       9.2.4

The new versioning scheme is fully documented in the Runtime.Version class and version information can be accessed from it:

System.out.println(Runtime.version().toString());  // 9-ea+167
System.out.println(Runtime.version().major());     // 9
System.out.println(Runtime.version().minor());     // 0
System.out.println(Runtime.version().security());  // 0

Javadoc Search and HTML5 (JEPs 224-225)

Have you noticed anything new in the content of Javadoc pages for Java 9 API so far? Look at the top right of the main frame. That’s right, there is now a search box that you can use to search for classes, methods, etc. It’s hard to believe it took them so much to implement it. The pages are also in HTML5 by default instead of HTML 4.

The searching is done locally, and the things that can be searched are:

  • Modules, packages, types and members.
  • Text that is marked with the tag @index.

New HotSpot Diagnostic Commands (JEP 228)

The following JVM diagnosis commands were added:

  • print_class_summary: print all loaded classes and their hierarchy
  • print_codegenlist: show the queue of methods to be compiled in C1 and C2 compilers
  • print_utf8pool: print string table
  • datadump_request: signal the JVM to do a data-dump request for JVMTI
  • dump_codelist: print all compiled methods in code cache that are alive
  • print_codeblocks: print code cache layout and bounds
  • set_vmflag: set VM flag option using the provided value

Here is a sample test that sends a command to print the string table using the jcmd utility:

C:\Users\manouti>jcmd
6448 sun.tools.jcmd.JCmd
6392 test.ProcessHandleExample

C:\Users\manouti>jcmd ProcessHandleExample VM.stringtable
6392:
StringTable statistics:
Number of buckets       :     60013 =    480104 bytes, avg   8.000
Number of entries       :      2478 =     59472 bytes, avg  24.000
Number of literals      :      2478 =    162896 bytes, avg  65.737
Total footprint         :           =    702472 bytes
Average bucket size     :     0.041
Variance of bucket size :     0.042
Std. dev. of bucket size:     0.204
Maximum bucket size     :         3

Create PKCS12 Keystores by Default (JEP 229)

Starting Java 9, keystores are created using the PKCS12 format instead of JKS, because it offers more stronger cryptographic algorithms. This change is backward compatible, so applications accessing existing keystores continue to work.

Multi-Release JAR Files (JEP 238)

One of the most interesting features introduced in Java 9 is the multi-release Jar (MRJAR) which allows bundling code targeting multiple Java releases within the same Jar file. By setting Multi-Release: true in the MANIFEST.MF file, the file becomes a multi-release Jar and the Java runtime will pick the appropriate versions of classes depending on the current major version running. The structure of such a file is illustrated as follows:

jar root
  - A.class
  - B.class
  - C.class
  - D.class
  - META-INF
     - versions
        - 9
           - A.class
           - B.class
        - 10
           - A.class
  • On JDKs < 9, only the classes in the root entry are visible to the Java runtime.
  • On a JDK 9, the classes A and B will be loaded from the directory root/META-INF/versions/9, while C and D will be loaded from the base entry.
  • On a JDK 10, class A would be loaded from the directory root/META-INF/versions/10.

A multi-release Jar file allows projects to maintain different versions of their code targeting different Java platforms, while being able to distribute the code as one Jar, with a single version (e.g. Maven artifact version). This relaxes the common restriction of writing backward compatible code, and allows developers to benefit from new language and API changes incrementally as they add new code.

This feature naturally requires modifications to some APIs used to process Jar files, such as JarFile and URLClassLoader. Also many JDK tools have been adapted to be aware of the new format, such as java, javac and jar.

As an example, the jar command can be used to create a multi-release Jar containing two versions of the same class compiled for both Java 8 and Java 9, albeit with a warning telling that the classes are identical:

C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demo
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar

This creates an MRJAR named MR.jar with the following contents:

jar root
  - demo
     - SampleClass.class
  - META-INF
     - versions
        - 9
           - demo
              - SampleClass.class

Let us now create a class called Main that prints the URL of the SampleClass, and add it for the Java 9 version:

package demo;

import java.net.URL;

public class Main {

    public static void main(String[] args) throws Exception {
		URL url = Main.class.getClassLoader().getResource("demo/SampleClass.class");
		System.out.println(url);
	}
}

If we compile this class and re-run the jar command, we get an error:

C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demoentry: META-INF/versions/9/demo/Main.class, contains a new public class not found in base entries
Warning: entry META-INF/versions/9/demo/Main.java, multiple resources with same name
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar
invalid multi-release jar file MR.jar deleted

It turns out that the jar tool prevents adding public classes to versioned entries if they are not added to the base entries as well. This is done so that the MRJAR exposes the same public API for the different Java versions. Note that at runtime, this rule is not required. It may be only applied by tools like jar. In this particular case, the purpose of Main is to run sample code, so we can simply add a copy in the base entry. If the class were part of a newer implementation that we only need for Java 9, it could be made non-public.

To add Main to the root entry, we first need to compile it to target a pre-Java 9 release. This can be done using the new --release option of javac (see JEP 247 – Compile for Older Platform Versions):

C:\Users\manouti\sampleproject-base\demo>javac --release 8 Main.java
C:\Users\manouti\sampleproject-base\demo>cd ../..
C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demo

Running the Main class shows that the SampleClass gets loaded from the versioned directory:

C:\Users\manouti>java --class-path MR.jar demo.Main
jar:file:/C:/Users/manouti/MR.jar!/META-INF/versions/9/demo/SampleClass.class

Remove the JVM TI hprof Agent (JEP 240)

The hprof JVM native agent was removed. Before Java 9, it could be used to dump the heap or profile the CPU. The reason it was removed is the existence of better alternatives. For example, jmap can do the heap dump, while JVisualVM can be used to profile running applications.

To demonstrate the impact of removing this agent, we can run a Java program with the hprof agent enabled (i.e. using the option -agentlib:hprof) on Java 8 and then on Java 9 (which is added on my system path):

C:\Users\manouti>D:\Dev\Java\jdk1.8.0_121\bin\java.exe -agentlib:hprof test.ProcessHandleExample
Running...
Dumping Java heap ... allocation sites ... done.

C:\Users\manouti>java -agentlib:hprof test.ProcessHandleExample
Error occurred during initialization of VM
Could not find agent library hprof on the library path, with error: Can't find dependent libraries

In all cases, this agent was not an official part of the JDK and it was rarely used by existing applications.

Remove the jhat Tool (JEP 241)

The jhat tool which could be used to browse a heap dump in a Web browser was removed since better alternatives exist. The tool was marked experimental and subject to removal in previous releases.

Compile for Older Platform Versions (JEP 247)

Before Java 9, we used to apply -source to tell it to use the selected language specification and -target to generate a certain version of bytecode. However, this can still lead to runtime issues as the compiler will link compiled classes to platform APIs of the current version of the JDK (unless you override the boot classpath). In Java 9, these options are replaced with one simple option --release to be able to compile for an older version.

--release is equivalent to -source N -target N -bootclasspath

JDK 9 makes this possible by maintaining some signature data about APIs from old releases, specifically under $JDK_HOME/lib/ct.sym.

G1 as Default Garbage Collector (JEP 248)

Prior to Java 9, the default garbage collector was typically the Parallel GC on server VMs and the Serial GC on client ones. On Java 9, server VMs will use G1 as the default, which was introduced in Java 7. G1 is a parallel and low-pause garbage collector that works especially well for multi-core machines with big heap sizes. For an overview of the G1 collector, see http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html. In addition to this feature, the Concurrent Mark Sweep (CMS) collector was deprecated.

Multi-Resolution Images (JEP 251)

A new interface MultiResolutionImage is added with a base implementation BaseMultiResolutionImage that can encapsulate several image variants with different sizes. This interface can be used to select the best image variant given certain width and height values.

Compact Strings (JEP 254)

An internal optimization is applied to the String class to reduce memory consumption. The idea is that most String objects contain characters that do not need 2 bytes to represent. The change consists of replacing the internal character array with a byte array, plus an extra byte that denotes the encoding of the byte array: either Latin-1 which takes up 1 byte, or UTF-16 which takes up 2 bytes. The String class will determine which encoding based on the content to be stored.

This is expected to reduce the size of heap memory in existing applications that rely heavily on strings, and should also reduce time spent on garbage collection. This change is internal and does not affect the external API of String and its related classes such as StringBuilder or StringBuffer.

Stack-Walking API (JEP 259)

Prior to Java 9, access to the thread stack frames was limited to an internal class sun.reflect.Reflection. Specifically the method sun.reflect.Reflection::getCallerClass. Some libraries relies on this method which is deprecated. An alternative standard API is now provided in JDK 9 via the StackWalker class, and is designed to be efficient by allowing lazy access to the stack frames. Some applications may use this API to traverse the execution stack and filter on classes. Two methods are of interest in this class:

  • public T walk(Function<Stream, T> function); which allow traversing a stream of stack frames for the current thread, starting from the top frame, and applying the given Function on the stream.
  • public Class<?> getCallerClass();which returns the class that invoked the method that calls this method.

This class is thread-safe, so multiple threads can use the same instance to walk their stacks.

For example, the following prints all stack frames of the current thread:

1  package test;
2
3  import java.lang.StackWalker.StackFrame;
4  import java.lang.reflect.InvocationTargetException;
5  import java.lang.reflect.Method;
6  import java.util.List;
7  import java.util.stream.Collectors;
8
9  public class StackWalkerExample {
10
11    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
12        Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
13        fooMethod.invoke(null, (Object[]) null);
14    }
15 }
16
17 class FooHelper {
18    protected static void foo() {
19        BarHelper.bar();
20    }
21 }
22
23 class BarHelper {
24    protected static void bar() {
25        List stack = StackWalker.getInstance()
26                .walk((s) -> s.collect(Collectors.toList()));
27        for(StackFrame frame : stack) {
28            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
29        }
30    }
31 }

Output:

test.BarHelper 26 bar

test.FooHelper 19 foo

test.StackWalkerExample 13 main

The following prints the current caller class. Note that in this case, the StackWalker needs to be created with the option RETAIN_CLASS_REFERENCE, so that Class instances are retained in the StackFrame objects. Otherwise an exception would occur.

public class StackWalkerExample {

    public static void main(String[] args) {
        FooHelper.foo();
    }

}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        System.out.println(StackWalker.getInstance(Option.RETAIN_CLASS_REFERENCE).getCallerClass());
    }
}

Output:

class test.FooHelper

A couple of other options allow stack traces to include implementation and/or reflection frames. This may be useful for debugging purposes. For instance, the first example includes some reflection to invoke FooHelper, but the reflection methods were not shown in the output. We can add the SHOW_REFLECT_FRAMES option to the StackWalker instance upon creation, so that the frames for the reflective methods are printed as well:

1  package test;
2
3  import java.lang.StackWalker.Option;
4  import java.lang.StackWalker.StackFrame;
5  import java.lang.reflect.InvocationTargetException;
6  import java.lang.reflect.Method;
7  import java.util.List;
8  import java.util.stream.Collectors;
9
10 public class StackWalkerExample {
11
12    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
13       Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
14        fooMethod.invoke(null, (Object[]) null);
15    }
16 }
17
18 class FooHelper {
19    protected static void foo() {
20        BarHelper.bar();
21    }
22 }
23
24 class BarHelper {
25    protected static void bar() {
26        List stack = StackWalker.getInstance(Option.SHOW_REFLECT_FRAMES)       // show reflection methods
27                .walk((s) -> s.collect(Collectors.toList()));
28        for(StackFrame frame : stack) {
29            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
30        }
31    }
32 }
33

Output:

test.BarHelper 27 bar

test.FooHelper 20 foo

jdk.internal.reflect.NativeMethodAccessorImpl -2 invoke0

jdk.internal.reflect.NativeMethodAccessorImpl 62 invoke

jdk.internal.reflect.DelegatingMethodAccessorImpl 43 invoke

java.lang.reflect.Method 563 invoke

test.StackWalkerExample 14 main

Note that line numbers for some reflection methods may not be available so StackFrame.getLineNumber() may return negative values.

Encapsulate Internal APIS (JEP 260)

Types whose packages start with sun. (and some starting with com.sun.) are internal to the Java platform. The Java development team has long discouraged their use since they are not supported and vary from implementation to another. Now, most of these classes have been encapsulated in modules that don’t export them for code outside the JDK. This means that existing code that rely on them will break; however this decision was based on the analysis that these API are very rarely, or either they have official replacements that exist prior to JDK 9. Exceptionally, the following internal APIs remained exported and placed in a separate module, and for those have supported replacements in JDK 9 they have been deprecated and may be encapsulated or removed in JDK 10:

  • sun.misc.{Signal,SignalHandler}
  • sun.misc.Unsafe
  • sun.reflect.Reflection::getCallerClass
  • sun.reflect.ReflectionFactory.newConstructorForSerialization

These APIs have been exported to public use because they are heavily used by some libraries, notably Unsafe. They are used to perform intrinsic JVM operations that are otherwise impossible to achieve.

Reactive Stream Flow API (JEP 266)

A standard set of interfaces corresponding to the reactive-streams specification have been introduced nested under the Flow class. These interfaces define a publish-subscribe mechanism that allows asynchronous flow-controlled communication between producers and consumers of data streams.

  1. Flow.Publisher: specifies a producer of data items and has one method subscribe(Flow.Subscriber subscriber) that adds a subscriber to be sent items.
  2. Flow.Subscriber: specifies a consumer of data items, and defines four methods:
    1. onSubscribe(Flow.Subscription subscription): invoked upon subscription of the subscriber by the publisher, and passes an instance of a Flow.Subscription to allow the subscriber to control the flow.
    2. onNext(T item): Invoked when a new item is received by the subscriber.
    3. onComplete(): invoked when all items have been sent by the publisher and no further items are to be received.
    4. onError(Throwable throwable): invoked when an error occurs on this subscription. No subsequent items are received.
  3. Flow.Subscription: represents a subscription and can be used by the subscriber to control the flow of data, and defines two methods:
    1. request(long n): can be invoked by the consumer to demand up to n items to be sent from the publisher. The number may be < n if the publisher finished sending all items.
    2. cancel(): invoked to cancel the subscription so that no more items are received.
  4. Flow.Processor: implements both a Publisher and a Subscriber and is used to transform data items. It can be used as a medium between a publisher and a subscriber.

Using this model of communication, the subscriber is more in control of the flow of data so that the rate of messages can be handled more efficiently. There is one additional utility class SubmissionPublisher that can be used by item generators that want to publish their data. This class implements the Flow.Publisher and AutoCloseable interfaces and can be closed to complete sending its items.

Convenience Factory Methods for Collections (JEP 269)

The interfaces List, Set and Map have been enriched for factory methods for immutable collections:

  1. 12 Overloaded of(...) factory methods for Set and List. One with a varargs parameter.
  2. 11 Overloaded of(...) factory methods for Map that take key and value arguments. Plus one that takes a varargs of Entry objects ofEntries(Entry<? extends K, ? extends V>... entries).

The returned collections are instances of nested types defined under java.util.ImmutableCollections. This class is package-private so it cannot used to check if the collection is immutable. This is left as an implementation detail of your application.

Enhancements to Streams

Apart from collections, new methods were added to java.util.stream.Stream. The first two are normally intended to be used when the stream is ordered:

Stream takeWhile(Predicate<? super T> predicate)

This method returns, for an ordered stream, a stream that consists of the longest prefix of elements that match the give predicate. The returned stream consists of the prefix of elements matching the predicate in that order. If the original stream is unordered, the order of elements is nondeterministic; the implementation is free to return any subset of elements that matches the predicate. However, if it happens that all elements match the predicate then the returned stream will consist of the same sequence of elements of the original (in the same order). If no element matches the predicate, an empty stream is returned.

An example that returns the first 5 integers of an ordered infinite stream:

Stream infiniteInts = Stream.iterate(0, i -> i + 1);
infiniteInts.takeWhile(i -> i < 5).forEach(System.out::println);

Output:

0

1

2

3

4

Example with an unordered stream:

Stream unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.takeWhile(i -> i < 5).forEach(System.out::println);

Output:

3

2

1

0

Note that if we run this last example, it may return any other result that excludes 5.

Stream dropWhile(Predicate<? super T> predicate)

This method does the opposite. It returns, for an ordered stream, a stream consisting of the elements remaining after dropping the longest prefix of elements matching the predicate. If the original stream is unordered, the behavior is indeterministic; the implementation is free to drop any subset of elements matching the predicate and return the remaining elements in the stream.

Example using an ordered stream:

Stream finiteInts = Stream.iterate(0, i -> i < 10, i -> i + 1);  // iterate takes a seed element, a Predicate, and an UnaryOperator
finiteInts.dropWhile(i -> i < 5).forEach(System.out::println);

Output:

5

6

7

8

9

Example using an ordered stream:

Stream unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.dropWhile(i -> i < 5).forEach(System.out::println);

Output:

5

4


Note that if we run this last example, it may return some other stream that contains 5.

Stream iterate(T seed, Predicate<? super T> hasNext, UnaryOperator next)

This method returns an ordered stream stream that applies iteratively the predicate starting on an initial value, until the predicate returns a false with the subsequent increment specified by the UnaryOperator. It is conceptually similar to the traditional for loop:

for (T index = seed; hasNext.test(index); index = next.apply(index)) {

   ...

}

Stream ofNullable(T t)

This method returns the given element if not null; otherwise an empty stream.

Enhancements to Optional

Three new methods were added to Optional:

void ifPresentOrElse(Consumer<? super T> action, Runnable emptyAction)

If a value is present, performs the given action with the value, otherwise performs the given empty-based action.

Optional or(Supplier<? extends Optional<? extends T>> supplier)

If a value is present, returns an Optional describing the value, otherwise returns an Optional produced by the supplying function.

Stream stream()

If a value is present, returns a sequential Stream containing only that value, otherwise returns an empty Stream.

Enhanced Deprecation (JEP 277)

The following elements have been added to the @Deprecated annotation to provide more information about the deprecation status:

1. forRemoval() If true, the deprecated API is intended to be removed in a future JDK release.

2. since() which indicated since which release this API has been deprecated.

As part of this JEP, some existing API elements have been planned for deprecated, such as legacy collections (e.g. Vector and Hashtable); some deprecated elements have been marked as for removal such as Thread.stop() and Thread.destroy().

Furthermore, a new utility jdeprscan has been added in the JDK tools to scan a Jar file or a set of classes for usage of deprecated code. Note that it only scans for deprecated code from the standard libraries.

Spin-Wait Hints (JEP 285)

This feature introduces a new method in java.lang.Thread called onSpinWait() which allows application code to provide a hint to the JVM that it running in a spin-loop, meaning it is busy waiting for some event to occur. The JVM can benefit from this hint to execute some intrinsic code that leverages hardware platform specific instructions. For example, x86 processors can execute a PAUSE instruction to indicate spin-wait, which can improve performance.

Note that the method onSpinWait does nothing. It is simply marked as an intrinsic candidate (using the jdk.internal.HotSpotIntrinsicCandidate annotation) to indicate to the JVM that it can replace it with native code optimized for the target platform. The method is just a hint, so the JVM is free to simply do nothing. A typical scenario where this method may be used is illustrated in the docs:

class EventHandler {
    volatile boolean eventNotificationNotReceived;
    void waitForEventAndHandleIt() {
        while ( eventNotificationNotReceived ) {
            java.lang.Thread.onSpinWait();
        }
        readAndProcessEvent();
    }

    void readAndProcessEvent() {
        // Read event from some source and process it
         . . .
    }
}

Applet API Deprecated (JEP 289)

Due to the decreasing support of Java plug-ins by Web browsers, the Applet API is deprecated. However, there is no intention to remove it in the next major release, so there is no forRemoval = true in the @Deprecated.

Jar Normalizer Maven Plugin

When building a project with Maven, the generated binary artifact would typically differ from one build to another, even if nothing has changed in the project. Systems relying on binary fingerprints (e.g. checks on MD5 hash) would detect different artifacts although the source is exactly the same. The change in the binary Jar files results from different timestamps, possibly different ordering of the entries, among other reasons. One way to solve this is by “fixing” the timestamp of the Jar entries (to some constant value), sorting the entries of the manifest, etc.

The jar-timestamp-normalize-maven-plugin is a Maven plugin which provides this capability. It consists of a single goal called normalize which can be bound to the package lifecycle phase and configured in the project’s POM:

<plugins>
    <plugin>
        <groupId>com.github.manouti</groupId>
        <artifactId>jar-timestamp-normalize-maven-plugin</artifactId>
        <version>1.0-SNAPSHOT</version>
        <executions>
            <execution>
                <id>jar-normalize</id>
                <goals>
                    <goal>normalize</goal>
                </goals>
                <phase>package</phase>
            </execution>
        </executions>
    </plugin>
</plugins>

Normalization mainly consists of the following steps:

  • Alphabetically sort all files and directories in the Jar.
  • Setting the last modified time of all Jar entries to a specific timestamp (default value is 1970-01-01 00:00:00AM but can be changed by passing -Dtimestamp as a system property).
  • Reordering (alphabetically) of attributes in the manifest except for Manifest-Version which always comes first.
  • Removing comments from the pom.properties file which contain a timestamp that causes the Jar to differ from one build to another.

Once invoked, the goal will generate the output file next to the original artifact (named artifactId-version-normalized.jar), i.e. in the project.build.directory directory.

List of configuration parameters:

Parameter Description
outputDirectory Where to store the output artifact, defaults to the project output directory.
timestamp Fixed timestamp for artifact entries, defaults to 1970-01-01 00:00:00AM. Can be using -Dtimestamp property.
normalizedArtifactId Artifact name to be used in the output file name, default to the project’s artifactId.
normalizedClassifierName Classifier to be used in the output file name, defaults to normalized.

Project home: https://github.com/manouti/jar-timestamp-normalize-maven-plugin

Regular Expressions

A regular expression, or regex, is a pattern of characters that can be matched against a string. It is useful for searching within text for one or more specific substrings that match the pattern. A typical use case I encounter a lot is when I need to search for a specific pattern in a huge file (possibly multiple files), and occasionally I need to replace all occurrences of this pattern with another string. Many editors support regular expressions for searching files. Most programming languages (e.g. Java, .NET, PHP) also have native support for regular expressions, although the syntax of the regex may vary from one language to another.

search

 

String Constants and Metacharacters

The most basic form of regex is a string constant that has no special characters. For example, the expression foo will match exactly the substring foo in food as shown in the below pseudocode.

"food".match("foo")  // "foo"

The following characters are interpreted as special characters: ([{\^$-|*+.?}])

To force using one of these characters as a normal character, a backslash (“\”) must be used.

"Value: 20.5".match("20\.5")  // "20.5"

 

Character Classes

There are some predefined character classes that can be used in regular expressions:

Type Description
. Any character except new line characters (\n and \r)
\d A digit: [0-9]
\D A non-digit: [^0-9]
\s A whitespace character: [ \r\n\t\f]
\S A non-whitespace character: [^ \r\n\t\f]
\w A word character: [a-zA-Z0-9_]
\W A non-word character: [^a-zA-Z0-9_]

Examples

"hamburger".match(".urger")  // "burger"

"abc1".match("ab.\d")  // "abc1"

"abc1".match(".\d")  // "c1"

"abc1n".match(".\d\D") // "c1n"

"abc123".match(".\d\D") // no match

"abc@123".match("\w\W\d") // "c@1"

The user can also specify the range or set of characters to be matched in the regex. For example, [agn] would match either a, g, or n. [a-g] would match any character between a and g, inclusive. On the other hand, [^aeiou] would match any character that is not a vowel.

"In 1991".match("19[0-9][0-9]")  // "1991"

"In 2003".match("200[^0-5]")     // no match

"In 2009".match("200[^0-5]")     // "2009"

The pipe character (|) is equivalent to a logical OR operator, causing the regex to match either the left side or the right side of the expression:

"Deep Purple".match("Blue|Purple")  // "Purple"

 

Quantifiers

Quantifiers are used to repeat the pattern in the regular expression. The basic quantifiers are: ? * +
The user can also specify the number of occurrences of a pattern in a string using curly brackets.

Quantifier Description
x? x, zero or one occurrence
x* x, zero or more occurrences
x+ x, one or more occurrences
x{n} x, exactly n occurrences

Examples

"1-22-333".match("\d{3}")           // "333"

"1-22-333@".match("\d+-\d{3}@\d*")  // "22-333@"

"-333@".match("\d+-\d{3}@\d*")      // no match

"-333@7".match("-?\d{3}@")          // "-333@"

Greedy vs. Lazy Matching

There are two types of matching that can be done when using quantifiers: greedy or lazy. In most regex implementations, a match can be made lazy by adding a question mark to the quantifier as shown below.

"honolulu".match("ho.+lu")   // "honolulu" (greedy)

"honolulu".match("ho.+?lu")  // "honolu" (lazy)

 

Boundary Matchers

A boundary matcher specifies where in the string the pattern should be matched, for example at the start or end of the string. The below table lists some of these matchers:

Character Description
^ The start of a line
$ The end of a line
\A The start of the string (don’t care about new lines)
\Z The end of the string (ignore final line terminator, if any)
\z The end of the string
\b A word boundary
\B A non-word boundary
\< The start of a word (similar to \b, but only the start of the word
\> The end of a word (similar to \b, but only the end of the word

Examples

"guitar".match("^guitar$")        // "guitar"

"guitar  ".match("^guitar$")      // no match

"guitar  ".match("^guitar\s*")    // "guitar  "

"This is a guitar".match("\bguitar\b")     // "guitar"

"This is a guitarist".match("\bguitar\b")  // no match

"This is a guitar".match("\bguitar\B")     // no match

"This is a guitarist".match("\bguitar\B")  // "guitar"

 

Groups

A capturing group is used to group a substring within the matched pattern as a single unit. It can be referenced back within the regular expression. A group is marked by placing the characters inside parentheses, and is referenced by a number depending on the total number of groups in the regex. For example, in (\d+)-(\w+)@\d-\1, (\d)+ is the first group and (\w+) is the second group. The first group is also referenced at the end of the regex.

"d1d1".match("(\w\d)\1")    // "d1d1"

"d1d2".match("(\w\d)\1")    // no match

 

Object Serialization/Deserialization Using XStream

XStream is a simple Java library for serializing objects to XML and vice-versa. With few lines of code, it can convert object graphs into an XML representation, or deserialize XML into an object graph. This is done in a fast and memory efficient way. By default, XStream relies on XPP (XML Pull Parser) which is a fast XML parsing library. It can also use a standard DOM parser or a StAX parser (since Java 6) that is provided with the library.

First, let’s consider some simple classes to serialize and deserialize.

public class Student {
    private int id;  // fields can be private
    private String name;
    private int age;
    private List courseInfos = new ArrayList();

    // getters and setters
    // no need for a default constructor

    @Override
    public String toString() {
        return "Student [id=" + id + ", name=" + name + ", age=" + age
	    + ", courseInfos=" + courseInfos + "]";
    }
}

public class CourseInfo {
    private String courseId;
    private String title;

    // getters and setters

    @Override
    public String toString() {
        return "CourseInfo [courseId=" + courseId + ", title=" + title + "]";
    }
}

The below example shows how to serialize an object representing a student into XML.

XStream xstream = new XStream(new DomDriver());

xstream.alias("student", Student.class);
xstream.alias("course", CourseInfo.class);

Student student = new Student();
student.setId(123);
student.setName("Alex");
student.setAge(24);

CourseInfo mathCourse = new CourseInfo();
mathCourse.setCourseId("MATH201");
mathCourse.setTitle("Calculus II");

CourseInfo chemCourse = new CourseInfo();
chemCourse.setCourseId("CHEM200");
chemCourse.setTitle("Introductory chemistry");

student.getCourseInfos().add(mathCourse);
student.getCourseInfos().add(chemCourse);

String serializedStudent = xstream.toXML(student);
System.out.println(serializedStudent);

We create an instance of the XStream class. In this case, we tell XStream to use a DOM parser instead of the default XPP parser. Then we create the Student object to serialize and call the toXML method to get a string containing the XML representation. The alias method at lines 3 and 4 specify the names of the XML elements corresponding to the classes. If no aliases are specified, then the fully qualified class names will be used, which would increase the size of the XML.

When the above code executes, the output is the following:

<student>
  <id>123</id>
  <name>Alex</name>
  <age>24</age>
  <courseInfos>
    <course>
      <courseId>MATH201</courseId>
      <title>Calculus II</title>
    </course>
    <course>
      <courseId>CHEM200</courseId>
      <title>Introductory chemistry</title>
    </course>
  </courseInfos>
</student>

To convert the above XML back into a Student object, we call the fromXML method as follows:

Student student = (Student) xstream.fromXML(serializedStudent);
System.out.println(student);

which would output the following:

Student [id=123, name=Alex, age=24, courseInfos=[CourseInfo [courseId=MATH201, title=Calculus II], CourseInfo [courseId=CHEM200, title=Introductory chemistry]]]

XStream is mainly used to read and write XML but also supports other formats like JSON.