Describe the bug I recently upgraded Spring dependencies in Spring Config Server as below : spring-boot-starter-parent : from 2.3.12.RELEASE to 2.7.5 spring-cloud-dependencies : from Greenwich.RELEASE to 2021.0.5
After Upgrading I found that whenever our Spring Config Client enabled application is making a request to load properties file from Spring Config Server,Spring Config Server becomes unresponsive and response extremely slow(> 5 mins)
Here is the client configuration of the file in my Spring Config Client application which is making request to Config Server:
spring:
application:
name: myappsearchpattern
config:
import: optional:configserver:https://www.myconfigserver.com/myconfigserver
cloud:
config:
enabled: true
On further debugging I found that when the above file is loaded by Spring Config Server,Spring Config Server is parsing the file and is acting as Config Client as well there by making recursive calls to Config Server url mentioned in spring.config.import.
Sample In the logs of config server,we are able to see the below logs which we usually see only in Spring Config Client:
org.springframework.cloud.config.client.ConfigServerConfigDataLoader - Fetching config from server at : https:www.myconfigserver.com/myconfigserver
Its weird that config server is making call to config server url which it is fetching from configuration files of config client applicationss.
Temporary Workaround I tried disabling config client configuration in Spring Config server by setting spring.cloud.config.enabled as false but it didnt work.I finally had to override org.springframework.boot.context.config.ConfigDataLocationResolver and org.springframework.boot.context.config.ConfigDataLoader and return empty data when these classes were called by Config Server internally.
I was expecting that Spring Config Server should be only loading the configuration file and in no way should be acting as Config Client.
Just to remind you we started seeing this issue only after ugrading the Spring dependencies as mentioned above.
Comment From: ryanjbaxter
Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.
Comment From: spring-cloud-issues
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
Comment From: mdtamim
@ryanjbaxter I have created a Spring Config Server app and a client app.
Config Server Git Repo : https://github.com/mdtamim/ConfigRefreshServerTestApp Config Client properties file link which I want to load in the client app : https://github.com/mdtamim/SpringConfigClienApp/blob/main/src/main/resources/application.yml
Reproducing Issue Just install the config Server in your local and hit the below URL from the browser to load the config client app's properties file http://localhost:8441/configserver/ConfigRefreshClient/default
Make sure you don't change any port and context path related properties in application.yml as same has been configured in Config client's properties file.
Observation
Your response will be extremely slow and in the application logs of config server you will encounter below log
Fetching config from server at : http://localhost:8441/configserver
This will be logged multiple times indicating config server is calling itself recursively while loading properties file of client app
Comment From: ryanjbaxter
The problem is this https://github.com/mdtamim/SpringConfigClienApp/blob/main/src/main/resources/application.yml#L5
That spring.config.import
configuration should live in the application.yaml on the applications classpath or else the config server is going to resolve it.
Comment From: mdtamim
@ryanjbaxter can you please elaborate on what should I do?What i understood is I should be keeping Spring cloud Config related properties in application.yaml?
Comment From: mdtamim
FYI,what I tried this time is moving all the config related properties in application.yaml and other configuration in application-e1.yml.And when I tried loading application-e1 ,config server is loading application.yaml as well and is making recursive calls.
Comment From: ryanjbaxter
You need to remove spring.config.import: optional:configserver:http://localhost:8441/configserve
from application.yaml
in the config server git repo and place it in the clients application.yaml which is on the classpath.
Comment From: mdtamim
This is the application.yml of my config server git repo : https://github.com/mdtamim/ConfigRefreshServerTestApp/blob/main/src/main/resources/application.yml
It don't have any config client related properties.
Here is my config client : https://github.com/mdtamim/SpringConfigClienApp/blob/main/src/main/resources/application.yaml
weather you load application.yaml or application-e1.yaml from client app.,response is extremely slow
Comment From: ryanjbaxter
you config server is pointing to this /src/main/resources
in https://github.com/mdtamim/SpringConfigClienApp. So when it serves up /src/main/resourcesapplication.yaml
it will process the spring.config.import statement which points to the config server again, which causes the config server to call itself. That spring.config.import statement should only be locally.
Comment From: mdtamim
That spring.config.import statement should only be locally.
Where should I place this property in config client app so that its local to client and not available to config server?As you see if I am keeping these in application.yml of config client,config server is loading it.
Comment From: ryanjbaxter
Take a step back and explain why you are pointing the confit server to that directory in the repo
Comment From: mdtamim
So we have many applications(also acting as Spring config client) which are fetching their properties file from config server.We keep their properties file in client application's 'src/main/resources' as per spring standard.Spring also recommends to keep spring config client properties in application.yml(earlier we kept in bootstrap.yml).
So now to make config server aware of the properties file location we have to point the search location to 'src/main/resources'
Just for your information we are using Spring Config for really long time.Its just now that we have started seeing this slowness after upgradation((details added in the issue already).
Comment From: ryanjbaxter
Are you also building the jar with the configuration in src/main/resources? If so that pretty much defeats the purpose of the config server, which is allow you to externalize the configuration. By using the config server the configurations for the app should live outside of the application itself. (With the exception of a few specific configuration properties like spring.config.import
, server.port
, stuff like that)
Comment From: mdtamim
@ryanjbaxter I got your point. So in our project we have a config server and many new teams come and want to use our CRS server.And we dont want to manage all their properties file in our Config Server.How it used to work is they used to keep their properties file in their client app only and we used to onboard their repos location in our Config Server.We get benefitted in below ways :
- Once we added their repo in Config server,they dont come and commit in Cloud config repo for any new changes in ther properties file.So less maintanance needed for Config Server given all the client apps frequently change their properties file in their own repo.
- They used to update their application properties in their repo and they could just restart their application and Config server would load the latest properties file via Config Server from the client repo itself.
- Client applications can just disable spring.cloud.config.enabled if they dont want to use Cloud config.No additional changes will be needed as properties file are already in their repos.
So now we will have to make changes in all the repo and remove all the configuration files from their app and keep it in Config Server. Any changes in these properties file will need commit in the Config Server repo.
I personally feel, config server should not be making call to itself because I don't see any valid reason/scenario to have this logic in the config server code.
Comment From: ryanjbaxter
If it helps your use case you can add spring-cloud-starter-bootstrap
to return the old behavior. I don't advise what you are doing but if that is how you want to use it, that sounds like the best path forward.
Comment From: mdtamim
@ryanjbaxter I have implemented a workaround where I am able to use the latest Spring dependencies.In this workaround I am overriding below two classes :
- org.springframework.boot.context.config.ConfigDataLocationResolver=com.demo.DisableCloudConfig
- org.springframework.boot.context.config.ConfigDataLoader=com.demo.DisableCloudConfig.DisableResourceLoader
And here is the implementation of DisableCloudConfig class :
import org.springframework.boot.context.config.*;
import org.springframework.cloud.config.client.ConfigServerConfigDataLocationResolver;
import org.springframework.core.Ordered;
import java.util.List;
public class DisableCloudConfig implements ConfigDataLocationResolver<ConfigDataResource>, Ordered {
@Override
public boolean isResolvable(ConfigDataLocationResolverContext context, ConfigDataLocation location) {
return location.hasPrefix(ConfigServerConfigDataLocationResolver.PREFIX);
}
@Override
public List<ConfigDataResource> resolve(ConfigDataLocationResolverContext context, ConfigDataLocation location) throws ConfigDataLocationNotFoundException, ConfigDataResourceNotFoundException {
return List.of(new DisableResource());
}
@Override
public List<ConfigDataResource> resolveProfileSpecific(ConfigDataLocationResolverContext context, ConfigDataLocation location, Profiles profiles) throws ConfigDataLocationNotFoundException {
return List.of(new DisableResource());
}
@Override
public int getOrder() {
return -1;
}
public static class DisableResource extends ConfigDataResource{
private DisableResource(){
super(true);
}
}
public static class DisableResourceLoader implements ConfigDataLoader<DisableResource> {
@Override
public ConfigData load(ConfigDataLoaderContext context, DisableResource resource) {
return ConfigData.EMPTY;
}
}
}
As a consumer of Spring Cloud Config dependencies ,my feedback/suggestion would be - If we can stop Spring Config Server dependency to make a call to itself while resolving the properties files from client,the Spring Config Clients can keep their properties file in their repos and still benefit from this amazing Spring Cloud Config framework.This will only increase the usability of this framework to a great extent specially for those who are reluctant to keep their properties file in another repo for few good reasons.
Do you think this suggestion makes sense and workable?
Comment From: ryanjbaxter
Did you try adding spring-cloud-starter-bootstrap
and continue using bootstrap.yaml
? To me that is the best way forward for you.
Comment From: mdtamim
We moved away from bootstrap.yaml as we slowly upgraded the Spring Config dependencies.
I can use that but isn't it deprecated and will be removed in future?
Comment From: ryanjbaxter
No it is not deprecated. If you find somewhere that says that please let me know.
Comment From: mdtamim
With Spring Cloud Vault 3.0 and Spring Boot 2.4, the bootstrap context initialization (bootstrap.yml, bootstrap.properties) of property sources was deprecated. Instead, Spring Cloud Vault favors Spring Boot’s Config Data API which allows importing configuration from Vault. With Spring Boot Config Data approach, you need to set the spring.config.import property in order to bind to Vault. You can read more about it in the Config Data Locations section. You can enable the bootstrap context either by setting the configuration property spring.cloud.bootstrap.enabled=true or by including the dependency org.springframework.cloud:spring-cloud-starter-bootstrap.
This is what i got in https://docs.spring.io/spring-cloud-vault/docs/current/reference/html/#new-in-3.0.0
I can make changes in all our client applications use bootstrap.yml if there are no plans of removing support of bootstrap.yml in future.
Comment From: ryanjbaxter
Just in Vault, see https://github.com/spring-cloud/spring-cloud-vault/issues/662