0
votes

I need to append the string content to google cloud storage file.

Actually I'm doing string operation that contains a file size of more than 30 MB.

I'm running my project in google app engine. When I try to hold the entire content in stringbuilder its throwing java heap space error in google app engine.

Is there any way to append the content in google cloud storage file instead of increasing the java heap space?

1

1 Answers

1
votes

Streaming data out in Java is usually accomplished through a Writer pattern. As you build your data, you "write" it out on a stream. Keeping the entire thing in memory leads to the problems you've experienced.

I'm not sure which library you're using to access Google Cloud Storage, but most of them provide a way to write objects a few bytes at a time.

App Engine's legacy Google Cloud Storage API provides an AppEngineFile object with a write() method that you can repeatedly invoke as you create your object. The new Google Cloud Storage Client Library provides a GcsOutputChannel that can do the same.

I might have misunderstood your question, though. Are you asking about creating an object in Google Cloud Storage and then appending content to it afterwards? GCS does not allow for appending to files once they've been fully created, with the limited exception of the "compose" functionality (which would work, except that you can only compose an object a certain number of times before you reach a maximum composition level).