ArduinoJson does not deserialize string read in from an HTTP GET but does deserialize a String Literal

Question

I am using a ESP8266 to make an HTTP GET request to the APi provided by https://openaq.org/ and then uses the ArduinoJson https://arduinojson.org/ library to deserialize the data and extract the sensor reading.

I can obtain the Json data and print out the string but when I try to deserialize it, it doesn't work

If I copy and paste the output that I have printed to the serial monitor back into the code and then pass that to be deserialized then it does work, see the currently commented out line

//line = "{\"meta\":{\"name\":\"openaq-api\",\"license\":\"CC BY 4.0\",\"website\":\"https://docs.openaq.org/\",\"page\":1,\"limit\":1,\"found\":1955},\"results\":[{\"location\":\"Birmingham Acocks Green\",\"parameter\":\"pm25\",\"date\":{\"utc\":\"2019-06-26T12:00:00.000Z\",\"local\":\"2019-06-26T13:00:00+01:00\"},\"value\":5,\"unit\":\"µg/m³\",\"coordinates\":{\"latitude\":52.437165,\"longitude\":-1.829999},\"country\":\"GB\",\"city\":\"Birmingham\"}]}";

I thought at first it might be the " that need escaping but that hasn't made any difference. I also thought it might be that the string didn't have a null terminator so wasn't working properly but tried concatenating a string to the end

line = line + "";

but again that had no effect

void setup() {


const char* host = "api.openaq.org";
const int httpsPort = 443;

const char* fingerprint = "47 03 D4 71 84 CD E2 47 5D 4C 04 52 61 28 83 84 E6 FF 66 53";
String url = "/v1/measurements?location=Birmingham%20Acocks%20Green&parameter=pm25&limit=1";
  // put your setup code here, to run once:
    Serial.begin(115200);

  /*
   * 
   * 
   * Connect to Wifi
   * 
   */


  Serial.println();
  Serial.print("connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());


  //Attempt to connect to the host and port specified
  WiFiClientSecure client;
  Serial.print("Connecting to ");
  Serial.println(host);
  if (!client.connect(host, httpsPort)){
    Serial.println("Connection failed");
    return;
  }

   //Compare the Certificate returned with the one we manually obtained
   if (client.verify(fingerprint, host)) {
    Serial.println("certificate matches");
  } else {
    Serial.println("certificate doesn't match");
    return;
  }

  Serial.print("Requesting url ");
  Serial.println (url);

    client.print(String("GET ") + url + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +
               "User-Agent: ArduinoESP8266\r\n" +
               "Connection: close\r\n\r\n");

  Serial.println("request sent");


   // Check HTTP status
  char status[32] = {0};
  client.readBytesUntil('\r', status, sizeof(status));
  // It should be "HTTP/1.0 200 OK" or "HTTP/1.1 200 OK"
  if (strcmp(status + 9, "200 OK") != 0) {
    Serial.print(F("Unexpected response: "));
    Serial.println(status);
    return;
  }

  // Skip HTTP headers
  char endOfHeaders[] = "\r\n\r\n";
  if (!client.find(endOfHeaders)) {
    Serial.println(F("Invalid response"));
    return;
  }

  String line = client.readStringUntil('\n');
  String toReplace = "\"";
  String replacedBy = "\\\"";
  line.replace(toReplace, replacedBy);



  //line = "{\"meta\":{\"name\":\"openaq-api\",\"license\":\"CC BY 4.0\",\"website\":\"https://docs.openaq.org/\",\"page\":1,\"limit\":1,\"found\":1955},\"results\":[{\"location\":\"Birmingham Acocks Green\",\"parameter\":\"pm25\",\"date\":{\"utc\":\"2019-06-26T12:00:00.000Z\",\"local\":\"2019-06-26T13:00:00+01:00\"},\"value\":5,\"unit\":\"µg/m³\",\"coordinates\":{\"latitude\":52.437165,\"longitude\":-1.829999},\"country\":\"GB\",\"city\":\"Birmingham\"}]}";

  Serial.println(line);
  int line_len = line.length() + 1;
  char char_array[line_len];

  line.toCharArray(char_array,line_len);


  //send the String object to the input of the DeserializeJson
  DynamicJsonDocument doc(1200);
  deserializeJson(doc,char_array);
  JsonObject obj = doc.as<JsonObject>();

  String nameAQ = obj[String("meta")];

  Serial.println(nameAQ);

  JsonArray results = obj["results"];
  JsonObject results_0 = results[0];

  int results_0_value = results_0["value"];

  Serial.println(results_0_value);


}

When I Serial.print() the content of meta i expect to return {"name":"openaq-api","license":"CC BY 4.0","website":"https://docs.openaq.org/","page":1,"limit":1,"found":1955} but it outputs null

when I Serial.print() the results_0_value int I expect it to return whatever value is in the JSON but it outputs 0

Weird Workshop Weird Workshop · Accepted Answer · 2019-06-26T20:28:47

Have managed to fix this. I removed the lines to escape the " in the JSON

  String toReplace = "\"";
  String replacedBy = "\\\"";
  line.replace(toReplace, replacedBy);

I put these in earlier on in the developmnent to fix a problem and I think that the earlier problem was something else but left these in. They look like they are needed as they are shown in the ArduinoJSon Assistant https://arduinojson.org/v6/assistant/

ArduinoJson does not deserialize string read in from an HTTP GET but does deserialize a String Literal

1 Answers