== Resource handling
The Spring Framework provides a `org.springframework.core.io.ResourceLoader` abstraction to load files from the filesystem,
servlet context and the classpath. Spring Cloud AWS adds support for the https://aws.amazon.com/s3/[Amazon S3] service
to load and write resources with the resource loader and the `s3` protocol.

The resource loader is part of the context module, therefore no additional dependencies are necessary to use the resource
handling support.

=== Configuring the resource loader with Spring Boot

Resource loader is enabled by default when `spring-cloud-starter-aws` is added as the dependency.

=== Configuring the resource loader with XML
Spring Cloud AWS does not modify the default resource loader unless it encounters an explicit configuration with an XML namespace element.
The configuration consists of one element for the whole application context that is shown below:

[source,xml,indent=0]
----
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	   xmlns:aws-context="http://www.springframework.org/schema/cloud/aws/context"
	   xsi:schemaLocation="http://www.springframework.org/schema/cloud/aws/context
	   http://www.springframework.org/schema/cloud/aws/context/spring-cloud-aws-context.xsd">

	<aws-context:context-credentials>
    		...
        </aws-context:context-credentials>

	<aws-context:context-resource-loader/>
</beans>
----

=== Downloading files
Downloading files can be done by using the `s3` protocol to reference Amazon S3 buckets and objects inside their bucket. The
typical pattern is `s3://<bucket>/<object>` where bucket is the global and unique bucket name and object is a valid object
name inside the bucket. The object name can be a file in the _root_ folder of a bucket or a nested file within a directory
inside a bucket.

The next example demonstrates the use of the resource loader to load different resources.

[source,java,indent=0]
----
public class SimpleResourceLoadingBean {

	@Autowired
	private ResourceLoader resourceLoader;

	public void resourceLoadingMethod() throws IOException {
		Resource resource = this.resourceLoader.getResource("s3://myBucket/rootFile.log");
		Resource secondResource = this.resourceLoader.getResource("s3://myBucket/rootFolder/subFile");

		InputStream inputStream = resource.getInputStream();
		//read file
	}
}
----

=== Uploading files
Since Spring Framework 3.1 the resource loader can also be used to upload files with the `org.springframework.core.io.WritableResource`
interface which is a specialization of the `org.springframework.core.io.ResourceLoader` interface. Clients can upload files
using the `WritableResource` interface. The next example demonstrates an upload of a resource using the resource loader.

[source,java,indent=0]
----
public class SimpleResourceLoadingBean {

    @Autowired
    private ResourceLoader resourceLoader;

    public void writeResource() throws IOException {
        Resource resource = this.resourceLoader.getResource("s3://myBucket/rootFile.log");
        WritableResource writableResource = (WritableResource) resource;
        try (OutputStream outputStream = writableResource.getOutputStream()) {
            outputStream.write("test".getBytes());
        }
    }
}
----

==== Uploading multi-part files
Amazon S3 supports https://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html[multi-part uploads] to
increase the general throughput while uploading. Spring Cloud AWS by default only uses one thread to upload the files and
therefore does not provide parallel upload support. Users can configure a custom `org.springframework.core.task.TaskExecutor`
for the resource loader. The resource loader will queue multiple threads at the same time to use parallel multi-part uploads.

The configuration for a resource loader that uploads with 10 Threads looks like the following

[source,xml,indent=0]
----
<beans ...>
  <aws-context:context-resource-loader task-executor="executor" />
  <task:executor id="executor" pool-size="10" queue-capacity="0" rejection-policy="CALLER_RUNS" />
</beans>
----

[WARNING]
====
Spring Cloud AWS consumes up to 5 MB (at a minimum) of memory per thread. Therefore each parallel thread will incur
a memory footprint of 5 MB in the heap, and a thread size of 10 will consume therefore up to 50 mb of heap space. Spring Cloud
AWS releases the memory as soon as possible. Also, the example above shows that there is no `queue-capacity` configured,
because queued requests would also consume memory.
====

==== Uploading with the TransferManager
The Amazon SDK also provides a high-level abstraction that is useful to upload files, also with multiple threads using
the multi-part functionality. A `com.amazonaws.services.s3.transfer.TransferManager` can be easily created in the application
code and injected with the pre-configured `com.amazonaws.services.s3.AmazonS3` client that is already created
with the Spring Cloud AWS resource loader configuration.

This example shows the use of the `transferManager` within an application to upload files from the hard-drive.

[source,java,indent=0]
----
public class SimpleResourceUploadingBean {

	@Autowired
	private AmazonS3 amazonS3;

	public void withTransferManager() {
		TransferManager transferManager = TransferManagerBuilder.standard()
		                                                        .withS3Client(this.amazonS3)
		                                                        .build();
		transferManager.upload("myBucket","filename",new File("someFile"));
	}
}
----

=== Searching resources
The Spring resource loader also supports collecting resources based on an Ant-style path specification. Spring Cloud AWS
offers the same support to resolve resources within a bucket and even throughout buckets. The actual resource loader needs
to be wrapped with the Spring Cloud AWS one in order to search for s3 buckets, in case of non s3 bucket the resource loader
will fall back to the original one. The next example shows the resource resolution by using different patterns.

[source,java,indent=0]
----
public class SimpleResourceLoadingBean {

	private ResourcePatternResolver resourcePatternResolver;

 	@Autowired
    public void setupResolver(ApplicationContext applicationContext, AmazonS3 amazonS3){
        this.resourcePatternResolver = new PathMatchingSimpleStorageResourcePatternResolver(amazonS3, applicationContext);
    }

 	public void resolveAndLoad() throws IOException {
 		Resource[] allTxtFilesInFolder =  this.resourcePatternResolver.getResources("s3://bucket/name/*.txt");
 		Resource[] allTxtFilesInBucket =  this.resourcePatternResolver.getResources("s3://bucket/**/*.txt");
 		Resource[] allTxtFilesGlobally =  this.resourcePatternResolver.getResources("s3://**/*.txt");
 	}
}
----

[WARNING]
====
Resolving resources throughout all buckets can be very time consuming depending on the number of buckets a user owns.
====

=== Using CloudFormation
CloudFormation also allows to create buckets during stack creation. These buckets will typically have a generated name
that must be used as the bucket name. In order to allow application developers to define _static_ names inside their
configuration, Spring Cloud AWS provides support to resolve the generated bucket names.
Application developers can use the `io.awspring.cloud.core.env.ResourceIdResolver` interface to resolve the
physical names that are generated based on the logical names.

The next example shows a bucket definition inside a CloudFormation stack template. The bucket will be created with a name
like _integrationteststack-sampleBucket-23qysofs62tc2_

[source,json,indent=0]
----
{
	"Resources": {
		"sampleBucket": {
			"Type": "AWS::S3::Bucket"
		}
	}
}
----

Application developers can resolve that name and use it to load resources as shown in the next example below.

[source,java,indent=0]
----
public class SimpleResourceLoadingBean {

	private final ResourceLoader loader;
	private final ResourceIdResolver idResolver;

	@Autowired
	public SimpleResourceLoadingBean(ResourceLoader loader, ResourceIdResolver idResolver) {
		this.loader = loader;
		this.idResolver = idResolver;
	}

	public void resolveAndLoad() {
		String sampleBucketName = this.idResolver.
			resolveToPhysicalResourceId("sampleBucket");
		Resource resource = this.loader.
			getResource("s3://" + sampleBucketName + "/test");
	}
}
----

=== IAM Permissions
red
Following IAM permissions are required by Spring Cloud AWS:

[cols="2"]
|===
| Downloading files
| `s3:GetObject`

| Searching files
| `s3:ListObjects`

| Uploading files
| `s3:PutObject`
|===

Sample IAM policy granting access to `spring-cloud-aws-demo` bucket:

[source,json,indent=0]
----
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::spring-cloud-aws-demo"
        },
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::spring-cloud-aws-demo/*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::spring-cloud-aws-demo/*"
        }
    ]
}
----
