Spring Component scan fails to find jar entries in WEB-INF/classes with embedded Tomcat

Description Component Scan can't find annotated beans in the Web application.

Reproducer A small sample application for the issue has been created: GitHub Repository (Java 17, Gradle).

Exception Stack Trace:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'beanFromXML' defined in ServletContext resource [/WEB-INF/applicationContext.xml]: Cannot resolve reference to bean 'beanFromScan' while setting bean property 'beanFromScan'
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:377)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:135)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1711)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1460)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:336)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:307)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:334)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.instantiateSingleton(DefaultListableBeanFactory.java:1122)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingleton(DefaultListableBeanFactory.java:1093)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:1030)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:987)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:627)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:394)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:274)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:126)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4008)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4436)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:772)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1203)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1193)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:749)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:203)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:415)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:870)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        at org.apache.catalina.startup.Tomcat.start(Tomcat.java:437)
        at com.test.app.runner.AppRunner.run(AppRunner.java:42)
        at com.test.app.runner.AppRunner.main(AppRunner.java:131)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at com.test.app.runner.JarLauncher.main(JarLauncher.java:38)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'beanFromScan' available
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:925)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1361)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:365)
        ... 51 common frames omitted

How to reproduce Clone https://github.com/osnsergey/Spring62TestWebApp, use README.md commands to run the sample.

Comment From: jhoeller

@osnsergey this looks like a highly custom launcher which makes it hard to reason about on my end. There is nothing wrong with such a custom setup, I would just appreciate some debugging on your end: specifically, what kind of resources PathMatchingResourcePatternResolver sees in its code path and why it does not match the class file that you expect it to match.

PathMatchingResourcePatternResolver has been revised quite a bit in 6.2, so I'm not surprised by a side effect here. We've fixed a few things in 6.2.2 already, largely around jar caching. If you could point me to where PathMatchingResourcePatternResolver still mismatches in your scenario (even just a reasonable guess), that would make it much easier to fix this for 6.2.3 (due in two weeks).

Comment From: osnsergey

@jhoeller the debugging shows the following:

rootDirResources is filled by ParallelWebappClassLoader from embedded Tomcat (in PathMatchingResourcePatternResolver).

412:    protected Set<Resource> doFindAllClassPathResources(String path) throws IOException {
        Set<Resource> result = new LinkedHashSet<>(16);
        ClassLoader cl = getClassLoader();
        Enumeration<URL> resourceUrls = (cl != null ? cl.getResources(path) : ClassLoader.getSystemResources(path));
        while (resourceUrls.hasMoreElements()) {
            URL url = resourceUrls.nextElement();
            result.add(convertClassLoaderURL(url));
        }
        if (!StringUtils.hasLength(path)) {
            // The above result is likely to be incomplete, i.e. only containing file system references.
            // We need to have pointers to each of the jar files on the class path as well...
            addAllClassLoaderJarRoots(cl, result);
        }
        return result;
    }

cl (default web app class loader) is object of class ParallelWebappClassLoader.

rootDirResources = getResources(rootDirPath); // line 689 in PathMatchingResourcePatternResolver

the resource URL for the base path resource in the rootDirResources contains ! at the end of WEB-INF/classes:

URL [jar:file:/C:/Work//Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/classes!/com/test/app/]

this URL where the Component Scan should get annotated bean candidates.

jarEntriesCache is filled by Spring

this.jarEntriesCache.put(jarFileUrl, entriesCache); //line 900 in PathMatchingResourcePatternResolver

the entriesCache doesn't contain ! at the end of WEB-INF/classes:

0 = "META-INF/"
1 = "META-INF/MANIFEST.MF"
2 = "META-INF/context.xml"
3 = "META-INF/services/"
4 = "META-INF/services/org.apache.juli.logging.Log"
5 = "WEB-INF/"
6 = "WEB-INF/applicationContext.xml"
7 = "WEB-INF/classes/"
8 = "WEB-INF/classes/com/"
9 = "WEB-INF/classes/com/test/"
10 = "WEB-INF/classes/com/test/app/"
11 = "WEB-INF/classes/com/test/app/runner/"

Here (in PathMatchingResourcePatternResolver):

810:        if (separatorIndex >= 0) {
            jarFileUrl = urlFile.substring(0, separatorIndex);
            rootEntryPath = urlFile.substring(separatorIndex + 2);  // both separators are 2 chars
            NavigableSet<String> entriesCache = this.jarEntriesCache.get(jarFileUrl);
            if (entriesCache != null) {
                Set<Resource> result = new LinkedHashSet<>(64);
                // Search sorted entries from first entry with rootEntryPath prefix
817:                for (String entryPath : entriesCache.tailSet(rootEntryPath, false)) {
818:                    if (!entryPath.startsWith(rootEntryPath)) {
                        // We are beyond the potential matches in the current TreeSet.
820:                        break;
                    }

urlFile contains file:/C:/Work/Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar!/WEB-INF/classes!/com/test/app/ jarFileUrl contains file:/C:/Work/Spring62TestWebApp/spring62webapp/result/release/spring62webapp.jar rootEntryPath contains WEB-INF/classes!/com/test/app/ entryPath contains WEB-INF/classes/

The code on lines 817 - 820 skips bean candidates from the com/test/app base path because rootEntryPath contains ! after WEB-INF/classes but the entriesCache doesn't contain the ! after WEB-INF/classes.

Comment From: jhoeller

Thanks for the thorough analysis! I'll try to cover this scenario for 6.2.3.

Comment From: jhoeller

It seems that we simply need to clean the path before matching it against jar entries in the cache. I haven't seen any other encounters of that difference in the common code paths there, so I'm going with the simplest possible change. This will be available in the upcoming 6.2.3 snapshot; it makes your repro setup pass for me locally.