Moving reflection-based code from Java 8 to 11 and 17

During the migration of one of the libraries from Java 8 to Java 11 and then to Java 17 I ran into a number of problems with the reflection functionality that was used in this library. In my article I'm going to describe these problems and how they can be solved.

Java 8

In the library reflection mechanism was used to access private fields, methods and constructors of JDK internal classes, for example, to manipulate annotations in runtime. Here are some examples of reflection usages without actual manipulation code:

import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.Map;

public class ReflectionUsage {

    public static void main(String[] args) throws NoSuchMethodException, NoSuchFieldException, ClassNotFoundException {
        Method declaredAnnotationsMethod = Field.class.getDeclaredMethod("declaredAnnotations");
        declaredAnnotationsMethod.setAccessible(true);

        Field declaredAnnotationsField = Field.class.getDeclaredField("declaredAnnotations");
        declaredAnnotationsField.setAccessible(true);

        Class<?> annotationDataClass = Class.forName("java.lang.Class$AnnotationData");
        Constructor<?> annotationDataConstructor = annotationDataClass.getDeclaredConstructor(Map.class, Map.class, int.class);
        annotationDataConstructor.setAccessible(true);

        Method annotationDataMethod = Class.class.getDeclaredMethod("annotationData");
        annotationDataMethod.setAccessible(true);
    }
}

Java 11

After I migrated the library to Java 11, I started getting warnings like the followings:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by ...ReflectionUsage ... to method java.lang.reflect.Field.declaredAnnotations()
WARNING: Please consider reporting this to the maintainers of ...ReflectionUsage
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

These warnings are caused by the modular structure introduced in Java 9: reflective access to internal JDK modules is considered to be Illegal, but in Java 9-11 such access is still allowed - it only produces warnings.

To see all warnings produced due to Illegal access of the code, you can use the option:

--illegal-access=warn

Java 17

InaccessibleObjectException

After I migrated the library to Java 17, I started seeing runtime errors InaccessibleObjectException instead of Illegal access warnings. An example of InaccessibleObjectException stack trace:

Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Unable to make private java.util.Map java.lang.reflect.Field.declaredAnnotations() accessible: module java.base does not "opens java.lang.reflect" to unnamed module @...
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
    at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
    at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
    at ...ReflectionUsage.main(ReflectionUsage.java:13)

To resolve these exceptions it's required to use the option:

--add-opens

From the Oracle article:

If you have to allow code on the class path to do deep reflection to access nonpublic members, then use the --add-opens runtime option.
Some libraries do deep reflection, meaning setAccessible(true), so they can access all members, including private ones. You can grant this access using the --add-opens option on the java command line. No warning messages are generated as a result of using this option.

So I used the following options for the library:

--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED

NOTE
In Java 16 and prior it's also possible to fix this error by using the option:

--illegal-access=permit

but its support has been removed in Java 17

NoSuchFieldException

Fix with --add-opens option helped me with InaccessibleObjectException but then I started getting NoSuchFieldException when trying to access private field declaredAnnotations from java.lang.reflect.Field class:

Exception in thread "main" java.lang.NoSuchFieldException: declaredAnnotations
    at java.base/java.lang.Class.getDeclaredField(Class.java:2610)
    at ...ReflectionUsage.main(ReflectionUsage.java:15)

for the sentence:

Field declaredAnnotationsField = Field.class.getDeclaredField("declaredAnnotations");

NoSuchFieldException was caused by the feature "Filtering for classes with security sensitive fields" introduced in Java 12 JDK-8210522:

Core reflection has a filtering mechanism to hide security and integrity sensitive fields and methods from Class getXXXField(s) and getXXXMethod(s).
The filtering mechanism has been used for several releases to hide security sensitive fields such as System.security and Class.classLoader.
This CSR proposes to extend the filters to hide fields from a number of highly security sensitive classes in java.lang.reflect and java.lang.invoke.

I found two possible workarounds to access private fields of java.lang.reflect.Field class (in my case I needed access to declaredAnnotations field, but the same approach can be used to access other fields that are not available because of JDK-8210522)

Workaround #1: use VarHandle (source):

var lookup = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
VarHandle declaredAnnotationsField = lookup.findVarHandle(Field.class, "declaredAnnotations", Map.class);

Workaround #2: invoke getDeclaredFields0 method for java.lang.reflect.Field class using reflection (source):

Method getDeclaredFields0 = Class.class.getDeclaredMethod("getDeclaredFields0", boolean.class);
getDeclaredFields0.setAccessible(true);

Field[] fields = (Field[]) getDeclaredFields0.invoke(Field.class, false);
Field declaredAnnotationsField;
for (Field field : fields) {
    if ("declaredAnnotations".equals(field.getName())) {
        declaredAnnotationsField = field;
        declaredAnnotationsField.setAccessible(true);
        break;
    }
}

NOTE
Main goal of JDK-8210522 feature is to prevent reflective access to several internal fields, so workarounds described above can stop working with any new JDK release so if you have the same problem - probably you need to think about re-writing your code to use a different approach.

Conclusion

In this article I've covered only reflection related aspects of migration Java applications to the newest LTS releases of Java (JDK 11, JDK 17), but there are certainly many more potential issues and problems during this upgrade.

Cover image: Photo by Andrew Ly on Unsplash