score:3
Accepted answer
I have done this in past, Amazon s3 needs 5Mb chunks, I was returning tuple at last, you could change as per your requirement.
val client = new AmazonS3Client(new BasicAWSCredentials(access_key, secret_key))
def my_parser = BodyParser {
val consume_5MB = Traversable.takeUpTo[Array[Byte]](1024 * 1024 * 5) &>> Iteratee.consume()
val rechunkAdapter: Enumeratee[Array[Byte], Array[Byte]] = Enumeratee.grouped(consume_5MB)
multipartFormData(Multipart.handleFilePart({
case Multipart.FileInfo(partName, file_name, content_type) => {
val object_id = java.util.UUID.randomUUID().toString().replaceAll("-", "")
val object_id_key = if (content_type.getOrElse("").contains("video") || content_type.getOrElse("").contains("audio")) object_id else object_id + file_name.substring(file_name.lastIndexOf('.'))
var position = 0
val etags = new java.util.ArrayList[PartETag]()
val initRequest = new InitiateMultipartUploadRequest(bucket, object_id_key)
val initResponse = client.initiateMultipartUpload(initRequest)
println("fileName = " + file_name)
println("contentType = " + content_type)
(rechunkAdapter &>> Iteratee.foldM[Array[Byte], Int](1) { (c, bytes) =>
Future {
println("got a chunk! :" + c + " size in KB: " + (bytes.length / 1024));
val is = new java.io.ByteArrayInputStream(bytes)
val uploadRequest = new UploadPartRequest()
.withBucketName(bucket).withKey(object_id_key)
.withUploadId(initResponse.getUploadId())
.withPartNumber(c)
.withFileOffset(position)
.withInputStream(is)
.withPartSize(bytes.length)
etags.add(client.uploadPart(uploadRequest).getPartETag)
position = position + bytes.length
c + 1
}
}).map { v =>
try {
val compRequest = new CompleteMultipartUploadRequest(
bucket,
object_id_key,
initResponse.getUploadId(),
etags)
client.completeMultipartUpload(compRequest)
println("Finished uploading " + file_name)
client.setObjectAcl(bucket, object_id_key, com.amazonaws.services.s3.model.CannedAccessControlList.PublicRead)
(object_id_key, file_name, content_type.getOrElse("application/octet-stream"))
} catch {
case e: Exception => {
println("S3 upload Ex " + e.getMessage())
val abortMPURequest = new AbortMultipartUploadRequest("xxxxxxx", object_id, initResponse.getUploadId())
client.abortMultipartUpload(abortMPURequest);
("error", file_name, content_type.getOrElse("application/octet-stream"))
}
}
}
}
}))
}
Source: stackoverflow.com
Related Query
- Upload Files directly to S3 chunk-by-chunk using Play Scala using Iteratees
- Using Iteratees and Enumerators in Play Scala to Stream Data to S3
- One-to-many relationship between tables on different files using Play with Scala and Slick
- where to keep external files in play framework while using scala
- How to read and process a file chunk by chunk for each step of the process using Play Iteratees
- Error loading large JSON files using Scala Play Framework 2
- How to exclude routes and ReverseRoutes files auto generated using scoverage for play scala framework?
- Playframework 2.3.x: Upload videos and large files using rest Web-Services with Scala
- Upload a file from Play webapp to another website using java.io.FileInputStream with Scala
- Validating form data with image upload using multipart in Play framework 2 with scala
- Play 2.x : Reactive file upload with Iteratees
- How to load JSON file using Play with Scala
- Read Parquet files from Scala without using Spark
- What does the word "Action" do in a Scala function definition using the Play framework?
- Scala Play framework: Duplicate mappings of compiled css files
- How can I fix missing conf files when using shadowJar and Scala dependencies?
- Parsing multipart HTTP form data with file upload content using Scala
- Play 2 Scala - Best way to upload a big CSV file with Iteratee in order to process each line reactively
- Scala : Way to use a function directly into the println(...) using string interpolation
- How to show images using Play framework and Scala in the view page
- POST request using play ws in Scala
- What is the fastest way to compile Scala files using maven?
- Scala Play upload file within a form
- scala play framework file upload error
- Scala Play 2.0.2 multiple file upload
- Parsing a Json String in Scala using Play framework
- How to list out all the files in the public/images directory in a Play Framework 2 Scala application?
- How do I build multiple jar files using scala sbt
- Spring Data Neo4j in Play 2.x app using Scala
- Problems with running Android APK file when merging dex files using Scala
More Query from same tag
- Macro Annotations in Scala 3
- Spark Mapvalues vs. Map
- reverse effect of explode function
- Play 2 / Scala - Generic Reactive Mongo CRUD - json serialization
- Some questions about difference between call-by-name and 0-arity functions
- Spark-SQL : How to read a TSV or CSV file into dataframe and apply a custom schema?
- (SBT) How to disable default resolver and only use the company internal resolver?
- How to create pair of keys of Map
- DynamoDB - remove from set for items based on a non-key condition
- Get the elements from different arraytype columns and build a column with heterogeneous data in Spark
- Implementing an abstract method with a trait, inconsistent compiler behaviour?
- How do you get an object associated with a Future Actor?
- How to convert a single character into Scala's Char object?
- Rolling Upgrades and Dynamic Software Update for JVM / Scala
- Are scala Objects and Collections immutable by default?
- Best practice : How to retrieve all EntityIds based on some condition applied on the state?
- What is the ? type?
- Scala type erasure for pattern matching
- Multiply elements of list of lists with each other
- How to parse JSON in Spark with fasterxml without SparkSQL?
- Seq[AnyVal] returned as type instead of Seq[Int]
- Find 73 palindromes
- GroupByKey faster than CombineByKey
- Singletons as Synthetic classes in Scala?
- Any there Disadvantages to Ignoring the Completion of a Future?
- How to idiomatically turn a Seq of tuples into a Map in scala?
- Scala pass a generic function into another generic function confusion
- Passing today's date in Gatling Body using session.set but no success
- Play Framework request attributes with typed key
- How to save and reuse Scala utility code