Jsoup HTTP POST with payload

Question

I am trying to make this HTTP request via jsoup as given here:

http://api.decarta.com/v1/[KEY]/batch?requestType=geocode

And here is my code for that:

String postUrl = postURLPrefix + apiKey + "/batch?requestType=geocode";
String response = Jsoup.connect(postUrl).timeout(60000).ignoreContentType(true)
        .header("Content-Type", "application/json;charset=UTF-8")
        .method(Connection.Method.POST)
        .data("payload", jsonPayload.toString())
        .execute()
        .body();

jsonPayload.toString() gives this:

{
  "payload": [
    "146 Adkins Street,Pretoria,Pretoria,Gauteng",
    "484 Hilda Street,Pretoria,Pretoria,Gauteng",
    "268 Von Willich Street,Centurion,Centurion,Gauteng",
    ...
  ]
}

Which is a perfectly valid JSON.

However, jsoup each time returns HTTP status code 400 (malformed).
So, how do I send proper HTTP POST with JSON payload using jsoup if this is possible at all? (Please note that it's payload and not an ordinary key-value pair in URL)

Answer 1

Use latest JSOUP library.

If you are using maven, then add below entry to pom.xml

<dependency>
        <groupId>org.jsoup</groupId>
        <artifactId>jsoup</artifactId>
        <version>1.10.2</version>
</dependency>

And below code will solves your problem.

String postUrl=postURLPrefix+apiKey+"/batch?requestType=geocode";
            System.out.println(postUrl);
            String response= Jsoup.connect(postUrl).timeout(60000).ignoreContentType(true)
                    .method(Connection.Method.POST)
                    .requestBody("payload",jsonPayload.toString())
                    .execute()
                    .body();

Answer 2

What you need is to post raw data. That functionality has been implemented but it hasn't been added yet. Check this pull request https://github.com/jhy/jsoup/pull/318 . Do you really need to use jsoup for this? I mean you could use HttpURLConnection (this is what jsoup uses underneath) to make the request and then pass the response to jsoup as a string.

Here is an example of HttpURLConnection taken (but simplified and added json/raw data) from www.mkyong.com

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;

public class Main {

    public static void main(String[] args) {

        try {

            String url = "http://www.google.com";

            URL obj = new URL(url);
            HttpURLConnection conn = (HttpURLConnection) obj.openConnection();
            conn.setReadTimeout(5000);
            conn.addRequestProperty("Accept-Language", "en-US,en;q=0.8");
            conn.addRequestProperty("User-Agent", "Mozilla");
            conn.addRequestProperty("Referer", "google.com");

            conn.setDoOutput(true);

            OutputStreamWriter w = new OutputStreamWriter(conn.getOutputStream(), "UTF-8");

            w.write("SOME_JSON_STRING_HERE");
            w.close();

            System.out.println("Request URL ... " + url);

            int status = conn.getResponseCode();

            System.out.println("Response Code ... " + status);

            BufferedReader in = new BufferedReader(new InputStreamReader(
                    conn.getInputStream()));
            String inputLine;
            StringBuffer html = new StringBuffer();

            while ((inputLine = in.readLine()) != null) {
                html.append(inputLine);
            }

            in.close();
            conn.disconnect();

            System.out.println("URL Content... \n" + html.toString());
            System.out.println("Done");

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Answer 3

This is how I did that with jsoup v1.14.3 and Kotlin (also applies to Java).
It sends two parameters key and nodeID as a JSON object (assuming the server accepts that):

val document = Jsoup.connect("the/target/url")
    .userAgent("Mozilla")
    .header("content-type", "application/json")
    .header("accept", "application/json")
    .requestBody("""{"key": "Hello", "nodeID": 4324}""")
    .ignoreContentType(true)
    .post()

You can also send the parameters in url-encoded format (assuming the server accepts that):

val document = Jsoup.connect("the/target/url")
    .userAgent("Mozilla")
    .header("content-type", "application/x-www-form-urlencoded")
    .header("accept", "application/json")
    .data("key", "Hello")
    .data("nodeID", "4324")
    // OR
    //.data(mapOf("key" to "Hello", "nodeID" to "4324"))
    .ignoreContentType(true)
    .post()