How to log Google Cloud Storage response headers while uploading from Google App Engine

Question

I want to improve logging of my app looking at App Engine Getting Started tutorial available here: https://cloud.google.com/java/getting-started-appengine-standard/using-cloud-storage

I stuck on how to log response headers from Google Cloud Storage for every upload.

How do I access low-level properties of a HTTP response?

This is the helper method from the official tutorial:

  public String uploadFile(Part filePart, final String bucketName) throws IOException {
    DateTimeFormatter dtf = DateTimeFormat.forPattern("-YYYY-MM-dd-HHmmssSSS");
    DateTime dt = DateTime.now(DateTimeZone.UTC);
    String dtString = dt.toString(dtf);
    final String fileName = filePart.getSubmittedFileName() + dtString;

    // the inputstream is closed by default, so we don't need to close it here
    BlobInfo blobInfo =
        storage.create(
            BlobInfo
                .newBuilder(bucketName, fileName)
                // Modify access list to allow all users with link to read file
                .setAcl(new ArrayList<>(Arrays.asList(Acl.of(User.ofAllUsers(), Role.READER))))
                .build(),
            filePart.getInputStream());
    // return the public download link
    return blobInfo.getMediaLink();
  }

Answer 1

Here the code for requesting the API:

import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.util.Iterator;
import java.util.List;
import java.net.URL;

public class OracleConnection extends HttpServlet { //extends only usefull for Cloud Run

    //With function, the name can be different. Not with Cloud Run. Here a GET request
    public void doGet(HttpServletRequest request,
                      HttpServletResponse response) throws IOException {
        String token = request.getParameter("token");
        String bucket = request.getParameter("bucket");
        String fileName = request.getParameter("file");

        String getUrl = "https://www.googleapis.com/storage/v1/b/" + bucket + "/o/" + fileName + "?alt=media";

        URL url = new URL(getUrl);
        HttpURLConnection con = (HttpURLConnection) url.openConnection();
        con.setRequestMethod("GET");
        con.setRequestProperty("Authorization", "Bearer " + token);

        con.getRequestProperties().forEach((s, strings) -> System.out.println(s + " -> " + strings));
        //Do the query
        int status = con.getResponseCode();

        StringBuilder fullResponseBuilder = new StringBuilder("Response code \n");
        fullResponseBuilder.append(con.getResponseCode())
                .append(" ")
                .append(con.getResponseMessage())
                .append("\nHeaders\n");

        con.getHeaderFields().entrySet().stream()
                .filter(entry -> entry.getKey() != null)
                .forEach(entry -> {
                    fullResponseBuilder.append(entry.getKey()).append(": ");
                    List headerValues = entry.getValue();
                    Iterator it = headerValues.iterator();
                    if (it.hasNext()) {
                        fullResponseBuilder.append(it.next());
                        while (it.hasNext()) {
                            fullResponseBuilder.append(", ").append(it.next());
                        }
                    }
                    fullResponseBuilder.append("\n");
                });

        Reader streamReader = null;

        if (con.getResponseCode() > 299) {
            streamReader = new InputStreamReader(con.getErrorStream());
        } else {
            streamReader = new InputStreamReader(con.getInputStream());
        }

        BufferedReader in = new BufferedReader(streamReader);
        String inputLine;
        StringBuilder content = new StringBuilder();
        while ((inputLine = in.readLine()) != null) {
            content.append(inputLine);
        }

        in.close();

        fullResponseBuilder.append("Response: ")
                .append(content);
        //File content is in 'content' string.
        System.out.println(fullResponseBuilder.toString());
        response.getWriter().println(fullResponseBuilder.toString());
        response.setStatus(HttpServletResponse.SC_OK);
    }
}

I call it with this command (deployed in a jetty server, it's just for providing examples of params)

curl "localhost:8080/?bucket=<mybucket>&file=<my full path file>&token=$(gcloud config config-helper --format='value(credential.access_token)')"

The response look like that:

Response code
200 OK
Headers
Alt-Svc: quic=":443"; ma=2592000; v="46,43,39",h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
Server: UploadServer
X-Goog-Generation: 1567533725338357
Pragma: no-cache
X-Goog-Metageneration: 1
Date: Tue, 24 Sep 2019 10:03:36 GMT
X-Goog-Hash: crc32c=Yqnzlw==,md5=kAQK2I6n1zJq0o/LkH7G/w==
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
ETag: CPWV/ZaeteQCEAE=
Content-Disposition: attachment
X-Goog-Storage-Class: REGIONAL
X-GUploader-UploadID: AEnB2UruP4LxUKRnX7tB6uRBAwnEbKJgfhuoWBWQa2_aArD7L-mUC96Fr9-cA0w90bMLwsZFE5joxT392Hd-Ab-XTwgkybvMPw
Vary: X-Origin, Origin
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Content-Length: 14
Content-Language: en
Content-Type: text/csv
Response: <your file content>
``