How to safely access the URLs of all resource files in the classpath in Java 9+?

Question

We learned from the release notes of Java 9 that

The application class loader is no longer an instance of java.net.URLClassLoader (an implementation detail that was never specified in previous releases). Code that assumes that ClassLoader::getSytemClassLoader returns a URLClassLoader object will need to be updated.

This breaks old code, which scans the classpath as follows:

Java <= 8

URL[] ressources = ((URLClassLoader) classLoader).getURLs();

which runs into a

java.lang.ClassCastException: 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to 
java.base/java.net.URLClassLoader

So for Java 9+ the following workaround was proposed as a PR at the Apache Ignite Project, which works as intended given adjustments in the JVM runtime options: --add-opens java.base/jdk.internal.loader=ALL-UNNAMED. However, as mentioned in the comments below, this PR was never merged into their Master branch.

/*
 * Java 9 + Bridge to obtain URLs from classpath...
 */
private static URL[] getURLs(ClassLoader classLoader) {
    URL[] urls = new URL[0];

    try {
        //see https://github.com/apache/ignite/pull/2970
        Class builtinClazzLoader = Class.forName("jdk.internal.loader.BuiltinClassLoader");

        if (builtinClazzLoader != null) {
            Field ucpField = builtinClazzLoader.getDeclaredField("ucp");
            ucpField.setAccessible(true);

            Object ucpObject = ucpField.get(classLoader);
            Class clazz = Class.forName("jdk.internal.loader.URLClassPath");

            if (clazz != null && ucpObject != null) {
                Method getURLs = clazz.getMethod("getURLs");

                if (getURLs != null) {
                    urls = (URL[]) getURLs.invoke(ucpObject);
                }
            }
        }

    } catch (NoSuchMethodException | InvocationTargetException | NoSuchFieldException | IllegalAccessException | ClassNotFoundException e) {
        logger.error("Could not obtain classpath URLs in Java 9+ - Exception was:");
        logger.error(e.getLocalizedMessage(), e);
    }
    return urls;
}

However, this causes some severe headache due to the use of Reflection here. This is kind of an anti-pattern and is strictly criticized by the forbidden-apis maven plugin:

Forbidden method invocation: java.lang.reflect.AccessibleObject#setAccessible(boolean) [Reflection usage to work around access flags fails with SecurityManagers and likely will not work anymore on runtime classes in Java 9]

Question

Is there a safe way to access the list of all resource URLs in the class- / module path, which can be accessed by the given classloader, in OpenJDK 9/10 without using sun.misc.* imports (e.g. by using Unsafe)?

UPDATE (related to the comments)

I know, that I can do

 String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));

to obtain the elements in the classpath and then parse them to URLs. However - as far as I know - this property only returns the classpath given at the time of the application launch. However, in a container environment this will be the one of the application server and might not be sufficient, e.g. then using EAR bundles.

UPDATE 2

Thank your for all your comments. I will test, if System.getProperty("java.class.path") will work for our purposes and update the question, if this fullfills our needs.

However, it seems that other projects (maybe for other reasons, e.g Apache TomEE 8) suffer the same pain related to the URLClassLoader- for this reason, I think it is a valueable question.

UPDATE 3

Finally, we did switch to classgraph and migrated our code to this library to resolve our use-case to load ML resources bundled as JARs from the classpath.

If you want the URLs to the elements of the class path then look at the system property java.class.path. — Alan Bateman
It's easy to split the value of the java.class.path property and create a file URL to each element, that is exactly what the application class loader does when it is initialized. — Alan Bateman
System.getProperty("java.class.path") still works and you can make URLs of it with ease. But the key point is, that’s the class path. The module path might be more involved and nobody says, that it must be representable as a list of URLs at all. I suppose, that’s one of the reasons for moving away from the URLClassLoader as application loader; you are not supposed to assume that you are running on a bunch of URLs. — Holger
“this property only returns the classpath given at the time of the application launch”—exactly and since Java 9 is going to forbid hacking the class path, that’s the only class path for the application class loader. But when you are talking about a container, you are not talking about the application loader at all. As long as you haven’t changed your application server, the container loader likely still is aURLClassLoaders. In the end, it seems to be an XY problem. What do you want to do with those URLs? — Holger
By the way, the PR you mention was actually not merged but the team has since followed a similar approach as yours in their IgniteUtils.java#classLoaderUrls(), although with a bug that I have now raised here — sujit

Rich Rich · Accepted Answer · 2018-04-09T10:51:01

I think this is an XY problem. Accessing the URLs of all resources on the classpath is not a supported operation in Java and is not a good thing to try to do. As you have already seen in this question, you will be fighting against the framework all the way if you try to do this. There will be a million edge cases that will break your solution (custom classloaders, EE containers, etc. etc.).

Please could you expand on why you want to do this?

If you have some kind of plugin system and are looking for modules that interface with your code which may have been provided at runtime, then you should use the ServiceLoader API, i.e.:

A service provider that is packaged as a JAR file for the class path is identified by placing a provider-configuration file in the resource directory META-INF/services. The name of the provider-configuration file is the fully qualified binary name of the service. The provider-configuration file contains a list of fully qualified binary names of service providers, one per line. For example, suppose the service provider com.example.impl.StandardCodecs is packaged in a JAR file for the class path. The JAR file will contain a provider-configuration file named:
META-INF/services/com.example.CodecFactory
that contains the line:
com.example.impl.StandardCodecs # Standard codecs

How to safely access the URLs of all resource files in the classpath in Java 9+?

Question

2 Answers