Streaming data out in Java is usually accomplished through a Writer pattern. As you build your data, you "write" it out on a stream. Keeping the entire thing in memory leads to the problems you've experienced.
I'm not sure which library you're using to access Google Cloud Storage, but most of them provide a way to write objects a few bytes at a time.
App Engine's legacy Google Cloud Storage API provides an AppEngineFile
object with a write()
method that you can repeatedly invoke as you create your object. The new Google Cloud Storage Client Library provides a GcsOutputChannel
that can do the same.
I might have misunderstood your question, though. Are you asking about creating an object in Google Cloud Storage and then appending content to it afterwards? GCS does not allow for appending to files once they've been fully created, with the limited exception of the "compose" functionality (which would work, except that you can only compose an object a certain number of times before you reach a maximum composition level).