Describe the bug Originally asked here, i want to fetch text and excel files
I am using minio as s3 client to integrate with awss3 in spring cloud config. The connection is established successfully and I am able to access propertysources using http://localhost:8080/config-server-service/dev/master for files with.properties, .yml, .yaml or .jsonextension
I have tried using the standard config urls provided at the spring cloud config i.e. /{application}/{profile}/{label}/{path}
For instance, i used http://localhost:8080/config-server-service/prod/test/test1.txt to fetch test1.txt file contents inside test folder of minio with prod profile.
However, the request throws a 404 error. The detailed logs are attached below for reference:
Host: localhost:8080
Connection: keep-alive
sec-ch-ua: "Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Linux"
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: security_authentication=Fe26.2**a84f225ac3c9fa344b6d276dc2d142c67f4dc78543705577cd5c5de24f470b47*UijhHhd7tN6efBvaMszEng*rjAHa4KoTHd_NYeJW4M7Zqm8IuAgQ_wW8l88XYRLxVGQmIyDzvK6WUN3w5xzRnSrinNEJH8ZI6tVGUWYlZUYtLnNWd_CqZlXDOX1bYHc7u8btoBL7-KIsXyyWU837_ImcPhs8c8wZzHJwRrGBNpXtkMxeQBPIeH2G_-tsncE9blZLUesMRA9NWn9sOUJLN7xU2ByhTVYczatOwszGPnPzzKZlp9qOwY-hg5yFB9WdnQ**0f1695b96263093906fbc9a5ab0ef70b46ed18374a153738c0082b5ff762cd5c*v2HX3YWPQnTk4mAidQfIRRo_lQJFd8waWbNXYcRDhR0; JSESSIONID=6D14C332CC525CE613BAE2FEDAAE21C0
]
2023-11-08T15:43:40.286+05:30 DEBUG 214768 --- [nio-8080-exec-1] org.apache.tomcat.util.http.Parameters : Set query string encoding to UTF-8
2023-11-08T15:43:40.287+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.a.t.util.http.Rfc6265CookieProcessor : Cookies: Parsing b[]: security_authentication=Fe26.2**a84f225ac3c9fa344b6d276dc2d142c67f4dc78543705577cd5c5de24f470b47*UijhHhd7tN6efBvaMszEng*rjAHa4KoTHd_NYeJW4M7Zqm8IuAgQ_wW8l88XYRLxVGQmIyDzvK6WUN3w5xzRnSrinNEJH8ZI6tVGUWYlZUYtLnNWd_CqZlXDOX1bYHc7u8btoBL7-KIsXyyWU837_ImcPhs8c8wZzHJwRrGBNpXtkMxeQBPIeH2G_-tsncE9blZLUesMRA9NWn9sOUJLN7xU2ByhTVYczatOwszGPnPzzKZlp9qOwY-hg5yFB9WdnQ**0f1695b96263093906fbc9a5ab0ef70b46ed18374a153738c0082b5ff762cd5c*v2HX3YWPQnTk4mAidQfIRRo_lQJFd8waWbNXYcRDhR0; JSESSIONID=6D14C332CC525CE613BAE2FEDAAE21C0
2023-11-08T15:43:40.288+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.a.catalina.connector.CoyoteAdapter : Requested cookie session id is 6D14C332CC525CE613BAE2FEDAAE21C0
2023-11-08T15:43:40.289+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.a.c.authenticator.AuthenticatorBase : Security checking request GET /config-server-service/prod/test/test1.txt
2023-11-08T15:43:40.289+05:30 DEBUG 214768 --- [nio-8080-exec-1] org.apache.catalina.realm.RealmBase : No applicable constraints defined
2023-11-08T15:43:40.292+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.a.c.a.jaspic.AuthConfigFactoryImpl : Loading persistent provider registrations from [/tmp/tomcat.8080.14180384723947386722/conf/jaspic-providers.xml]
2023-11-08T15:43:40.292+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.a.c.authenticator.AuthenticatorBase : Not subject to any constraint
2023-11-08T15:43:40.299+05:30 DEBUG 214768 --- [nio-8080-exec-1] org.apache.tomcat.util.http.Parameters : Set encoding to UTF-8
2023-11-08T15:43:40.299+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : GET "/config-server-service/prod/test/test1.txt", parameters={}
2023-11-08T15:43:40.307+05:30 DEBUG 214768 --- [nio-8080-exec-1] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to org.springframework.cloud.config.server.resource.ResourceController#retrieve(String, String, String, ServletWebRequest, boolean)
2023-11-08T15:43:40.322+05:30 DEBUG 214768 --- [nio-8080-exec-1] .m.m.a.ExceptionHandlerExceptionResolver : Using @ExceptionHandler org.springframework.cloud.config.server.resource.ResourceControllerAdvice#notFound(NoSuchResourceException)
2023-11-08T15:43:40.322+05:30 DEBUG 214768 --- [nio-8080-exec-1] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [org.springframework.cloud.config.server.resource.NoSuchResourceException: Not found: test1.txt]
2023-11-08T15:43:40.322+05:30 DEBUG 214768 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed 404 NOT_FOUND
Sample I have tried to debug the scenario and found the following:
- ResourceController handles the api request, which calls retrieve method
- Resource is fetched using findOne method in GenericResourceRepository
- Inside findOne method, location object variable has a path variable which translates to the s3 url:
/s3://<bucket-name>/<folder-name> - Please note a leading
/appended at the start of the url - when I tried with aws cli, the following url gives the appropriate response:
s3://<bucket-name>/<folder-name>
I am using the following dependencies/version inside the project - java-version: 17 - spring-cloud-version: 2022.0.4 - s3: 2.20.52 - spring-cloud-aws-context: 2.4.2 - spring-cloud-aws-autoconfigure: 2.2.6.RELEASE
Even adding Accept header of application/octet-stream did not work
Comment From: ryanjbaxter
Can you try using Spring Cloud AWS 3.0.3? It is the version that is compatible with the version of Spring Boot and Cloud that you are using.
Comment From: kssingh600
Thanks a ton @ryanjbaxter for the inputs. I have tried now and it works as expected.
I have another project wherein i am using Java 8 along with the following dependencies:
- spring Boot(2.7.17)
- spring-cloud-config-server(2021.0.8). It is compatible with Java 8
should i be using Spring Cloud AWS 3.0.3 there as well? If not, can you please tell compatible versions for java 1.8
Thanks
Comment From: ryanjbaxter
No I don’t believe spring cloud AWS 3.0.x is compatible with spring boot 2.7.x. I believe you will need to use spring cloud AWS 2.4.x
Comment From: anish-yesuraj-ascenthr
Thanks a ton @ryanjbaxter for the inputs. I have tried now and it works as expected.
I have another project wherein i am using Java 8 along with the following dependencies:
- spring Boot(2.7.17)
- spring-cloud-config-server(2021.0.8). It is compatible with Java 8
should i be using Spring Cloud AWS 3.0.3 there as well? If not, can you please tell compatible versions for java 1.8
Thanks
@kssingh600 - I am trying a similar setup and facing the same problem. can you please share your pom.xml. Thanks
Comment From: ryanjbaxter
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.0.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>s3-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>s3-demo</name>
<description>s3-demo</description>
<properties>
<java.version>17</java.version>
<spring-cloud.version>2022.0.5-SNAPSHOT</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-s3</artifactId>
<version>3.0.3</version>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-autoconfigure</artifactId>
<version>3.0.3</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.17.195</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<pluginRepository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<releases>
<enabled>false</enabled>
</releases>
</pluginRepository>
</pluginRepositories>
</project>