how to connect to S3 using Spring Cloud AWS?
I'm pretty new to Spring in general, as well as Amazon AWS, so I apologize in advance if I'm doing something horribly wrong.
I can not seem to get my Spring Boot application to communicate with AWS services in any way when deployed onto Elastic Beanstalk. I've been following the guide here to try to set my project up correctly.
I'm essentially taking the code they provide and attempting to run from my project:
@Component
public class Storage {
@Autowired
private ResourceLoader resourceLoader;
Logger logger = LoggerFactory.getLogger(this.getClass());
public Storage() {
}
public void writeResource() throws IOException {
Resource resource = this.resourceLoader.getResource("s3://server-images/rootFile.log");
WritableResource writableResource = (WritableResource) resource;
try (OutputStream outputStream = writableResource.getOutputStream()) {
outputStream.write("test".getBytes());
}
}
}
Super, seems simple enough. I also set my application.properties file:
cloud.aws.credentials.accessKey = A******A
cloud.aws.credentials.secretKey = i******K
cloud.aws.credentials.instanceProfile = true
cloud.aws.region.auto = true
so I set the key and secret in the profile, and then lastly, I add the stuff to the build.gradle file to get the Spring Cloud stuff:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.2.4.RELEASE")
classpath("io.spring.gradle:dependency-management-plugin:0.5.2.RELEASE")
}
}
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'spring-boot'
apply plugin: 'war'
apply plugin: "io.spring.dependency-management"
war {
baseName = 'server'
// version = '0.0.1'
}
dependencyManagement {
imports {
mavenBom 'org.springframework.cloud:spring-cloud-aws:1.0.2.RELEASE'
}
}
//jar {
// baseName = 'gs-spring-boot'
// version = '0.1.0'
//}
repositories {
mavenCentral()
jcenter()
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
configurations {
providedRuntime
}
dependencies {
// tag::jetty[]
// compile("org.springframework.boot:spring-boot-starter-web") {
// exclude module: "spring-boot-starter-tomcat"
//}
compile("org.springframework.boot:spring-boot-starter-web")
providedRuntime("org.springframework.boot:spring-boot-starter-tomcat")
// Spring Cloud AWS
compile 'org.springframework.cloud:spring-cloud-starter-aws'
//compile("org.springframework.boot:spring-boot-starter-jetty")
// end::jetty[]
// tag::actuator[]
compile("org.springframework.boot:spring-boot-starter-actuator")
compile("org.springframework.boot:spring-boot-starter-thymeleaf")
// end::actuator[]
testCompile("junit:junit")
}
task wrapper(type: Wrapper) {
gradleVersion = '2.4'
}
Lastly, I don't know how much this matters, but I gave "AmazonS3FullAccess" and "AmazonEC2FullAccess" to the elasticbeanstalk role in AWS. I also set permissions for "Everyone" to list/upload/delete.
So far, All I get is:
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.NullPointerException
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:295)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:102)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:68)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:113)
at org.springframework.boot.context.web.ErrorPageFilter.access$000(ErrorPageFilter.java:59)
at org.springframework.boot.context.web.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:106)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:673)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:516)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1086)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:659)
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1558)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1515)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: null
at app.data.Storage.readTerminator(Storage.java:44)
at app.controllers.CodeController.getCode(CodeController.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
... 54 common frames omitted
As I can see, the resourceLoader isn't getting anything
Maybe there is some permission problems? Maybe I have misconfigured it? I figure this shouldn't be this hard, so I'm definitely doing something wrong! If anyone has any ideas, they are more than welcome.
Brian.
If the cause is wrong permissions you should check if your user(you created in IAM) has corresponding permissions(Policies):
AmazonS3FullAccess, AWSConnector
Hope this will help.
Check if your bucket policy allows you to put un-encrypted objects . If this is not the reason, there may be other reasons why AWS S3 would not allow your objects to be uploaded. When you use the native AWS library, they set a lot of defaults, but spring libraries may not be setting those defaults.
I was getting the same error, until I figured out that my bucket policy was blocking any un-encrypted objects from being uploaded.
{
"Sid": "RequiredEncryptedPutObject",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": [
"AES256",
"aws:kms"
]
}
}
},
上一篇: 我如何混合打字和非打字的演员？