Spring Cloud Config Extra config import fetched through cloud config server property isn't injected during app start

Library versions

Spring Boot 2.7.12 Spring Cloud 2021.0.6 Spring Cloud AWS 2.4.1

---

I have a Spring Cloud Config setup with a Git backend. Generally, property files fetching function well. My aim is store secrets in AWS and reference them from property files, so I added additional spring.config.import: "aws-secretsmanager:/test/springconfig" directive within the properties file for the Spring Cloud Config Server.

This option only works when I add such a configuration to an application property itself as follows:

spring:
  application:
    name: my-java-client
  config:
    import: 
    - "configserver:"
    - "aws-secretsmanager:/test/springconfig"

My intention is to house both configurations and sensitive data (in the form of AWS references like "aws-secretsmanager:/test/springconfig") in the Git backend. So, the properties file in Git backend would look like

server.datasource.url: jdbc:postgresql://dev:5432/example
spring.config.import: "aws-secretsmanager:/test/springconfig"
some.dummy: value

This is what I get now:

{
"name": "configserver:git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml",
"properties": {
"server.datasource.url": {
"value": "jdbc:postgresql://dev:5432/example",
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:1:24"
},
"some.dummy": {
"value": "value",
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:2:13"
},
"aws.secretsmanager.region": {
"value": "us-east-1",
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:5:13"
},
"management.endpoints.web.exposure.include": {
"value": "env",
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:7:44"
},
"management.endpoint.env.enabled": {
"value": true,
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:8:34"
},
"spring.config.import": {
"value": "aws-secretsmanager:/test/springconfig",
"origin": "Config Server git@github.com:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:10:23"
}
}
},

The problem is that I don't get retrieved values from aws secret while storing spring.config.import: "aws-secretsmanager:/test/springconfig" in Git backend for Cloud Config server. I would appreciate any guidance on potential options to achieve this result. I have read that this can be done using Spring Cloud Vault, but it seems excessive for our needs as we would like to use AWS facilities for this purpose.

Thanks, Oleksandr

Comment From: ryanjbaxter

Yes you cannot put spring.config.import in configuration files coming from external sources. It needs to be placed in the application yaml locally of the client.

The config server itself does support AWS Secrets Manager as a backend though https://docs.spring.io/spring-cloud-config/docs/4.0.2/reference/html/#_aws_secrets_manager

Comment From: okulbida

Thanks @ryanjbaxter I'd like to manage both configurations and references to AWS Secrets Manager secrets within property files. For example:

management.endpoints.web.exposure.include: env
management.endpoint.env.enabled: true
aws-secretsmanager:/test/springconfig

I am seeking a way to store all application configuration references, including secrets, in one location. I am interested in the most straightforward and efficient method to achieve this. Could you please suggest an appropriate solution that minimizes complexity?

Comment From: ryanjbaxter

What version of Spring Cloud are you using?

Comment From: okulbida

Currently I have this

<spring-cloud.version>2021.0.6</spring-cloud.version>
<spring-cloud-aws.version>2.4.4</spring-cloud-aws.version>

Comment From: ryanjbaxter

Can you try your original project with Spring Cloud 2022.0.3?

Comment From: okulbida

@ryanjbaxter
Is it possible to have multiple backends for config server ? Option to have both git and something like this would help a lot to manage secrets in proper way https://docs.spring.io/spring-cloud-config/docs/current/reference/html/#_aws_secrets_manager_backend

Comment From: ryanjbaxter

Yes you can use a composite https://docs.spring.io/spring-cloud-config/docs/current/reference/html/#composite-environment-repositories

Comment From: okulbida

Thanks @ryanjbaxter I tested with Spring Cloud 2022.0.3 and it didn't help. For some reason I also cannot run awsparamstore or aws-secretsmanager composite type (multiple git, native types work).

Here is my current config

application.yaml:

server.port: 8888

management.endpoints.web.exposure.include: env
management.endpoint.env.enabled: true

aws.secretsmanager.region: us-east-1
aws.paramstore.region: us-east-1

credentials:
  profile:
    name: default

spring:
  profiles:
    include: composite
  # config:
  #   import:
  #   - 'aws-secretsmanager:/test/springconfig'
  #   - 'aws-parameterstore:/spring-cloud/server/privatekey'
  cloud:
    config:
      server:
        composite:
        - type: awsparamstore
          region: us-east-1
          prefix: /dev 
          profile-separator: '/'    
        - type: git
          uri: git@github.com:repo/configs.git
          basedir: configs-location
          clone-on-start: true
          default-label: main
          timeout: 5
          refresh-rate: 1
          ignoreLocalSshSettings: true
          search-paths:
            - "configs-location/shared"
            - "configs-location/{application}"

pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.1.0</version>
    <relativePath/> <!-- lookup parent from repository -->
  </parent>
  <groupId>io.example</groupId>
  <artifactId>configserver</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>spring-cloud-config-server</name>
  <description>Spring Config server providing config information</description>
  <properties>
    <java.version>17</java.version>
    <spring-cloud.version>2022.0.3</spring-cloud.version>
    <spring-cloud-aws.version>2.4.4</spring-cloud-aws.version>
    <amazon-awssdk-ssm.version>2.20.69</amazon-awssdk-ssm.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-config-server</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-config</artifactId>
    </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-secrets-manager-config</artifactId>
            <version>${spring-cloud-aws.version}</version>
        </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
            <version>${spring-cloud-aws.version}</version>
        </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
        <version>2.2.6.RELEASE</version>
    </dependency>
        <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>ssm</artifactId>
      <version>${amazon-awssdk-ssm.version}</version>
    </dependency>

  </dependencies>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-dependencies</artifactId>
        <version>${spring-cloud.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
    </plugins>
  </build>

</project>

ERROR:

config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616244539+03:00 2023-06-02T16:44:58.609Z ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616293659+03:00
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616304029+03:00 java.lang.IllegalStateException: Error processing condition on org.springframework.cloud.config.server.config.CompositeRepositoryConfiguration.searchPathCompositeEnvironmentRepository
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616310859+03:00   at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:60) ~[spring-boot-autoconfigure-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616317126+03:00   at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616323933+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForBeanMethod(ConfigurationClassBeanDefinitionReader.java:183) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616329747+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:144) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616335625+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616342099+03:00   at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:427) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616348214+03:00   at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:287) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616354789+03:00   at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:344) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616360413+03:00   at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:115) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616366608+03:00   at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:771) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616372949+03:00   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:589) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616379074+03:00   at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616401782+03:00   at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:733) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616408682+03:00   at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:435) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616414915+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616420954+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1305) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616426666+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1294) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616432147+03:00   at io.example.configserver.ConfigServerApplication.main(ConfigServerApplication.java:13) ~[classes!/:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616438333+03:00   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616443910+03:00   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616449824+03:00   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616455529+03:00   at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616461058+03:00   at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616467082+03:00   at org.springframework.boot.loader.Launcher.launch(Launcher.java:95) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616472374+03:00   at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616478058+03:00   at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616483715+03:00 Caused by: java.lang.NullPointerException: Cannot invoke "Object.hashCode()" because "key" is null
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616508433+03:00   at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616515734+03:00   at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:888) ~[spring-beans-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616521508+03:00   at org.springframework.cloud.config.server.composite.CompositeUtils.getEnvironmentRepositoryFactoryTypeParams(CompositeUtils.java:77) ~[spring-cloud-config-server-4.0.3.jar!/:4.0.3]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616530630+03:00   at org.springframework.cloud.config.server.composite.OnSearchPathLocatorPresent.getMatchOutcome(OnSearchPathLocatorPresent.java:46) ~[spring-cloud-config-server-4.0.3.jar!/:4.0.3]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616536604+03:00   at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47) ~[spring-boot-autoconfigure-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616542537+03:00   ... 25 common frames omitted
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616547774+03:00

Comment From: ryanjbaxter

Can you change type: awsparamstore to type: awsParameterStore and try again?

Comment From: okulbida

thanks for your reply, this was solved in different way