2
votes

I have just started with Web Programming & Arduino. I am trying to learn how an Arduino communicates with a Web Server.

I have this Arduino UNO that communicates with my Web Server via the SimComm SIM808 GSM-GPRS module. I have a file called retrieve.php on my Web Server that basically fetches data from a database & delivers them to whoever makes a request.

My Arduino program is reading the file, but it's reading more than it should.

retrieve.php echos two values 1 & 1. You can have a look at it here

Problem is when I make my Arduino call this file, it reads everything & this is what I end up getting

GSM Shield testing.

status=READY
status=ATTACHED

100.107.219.185
H
Number of data received:
1

Data received:
H
TTP/1.1 200 OK
Date: Sat, 04 Jun 2016 13:07:05 GMT
Server: Apache
X-Powered-By: PHP/5.5.35
Content-Length: 2
Connection: close
Content-Type: text/html

11
CLOSED

My Arduino code is given below:-

#include "SIM900.h"
#include <SoftwareSerial.h>
#include "inetGSM.h"
//#include "sms.h"
//#include "call.h"

//To change pins for Software Serial, use the two lines in GSM.cpp.

//GSM Shield for Arduino
//www.open-electronics.org
//this code is based on the example of Arduino Labs.

//Simple sketch to start a connection as client.

InetGSM inet;
//CallGSM call;
//SMSGSM sms;

char msg[50];
int numdata;
char inSerial[50];
int i=0;
boolean started=false;

void setup()
{
     //Serial connection.
     Serial.begin(9600);
     Serial.println("GSM Shield testing.");
     //Start configuration of shield with baudrate.
     //For http uses is raccomanded to use 4800 or slower.
     if (gsm.begin(2400)) {
          Serial.println("\nstatus=READY");
          started=true;
     } else Serial.println("\nstatus=IDLE");

     if(started) {
          //GPRS attach, put in order APN, username and password.
          //If no needed auth let them blank.
          if (inet.attachGPRS("TATA.DOCOMO.INTERNET", "", ""))
               Serial.println("status=ATTACHED");
          else Serial.println("status=ERROR");
          delay(1000);

          //Read IP address.
          gsm.SimpleWriteln("AT+CIFSR");
          delay(5000);
          //Read until serial buffer is empty.
          gsm.WhileSimpleRead();

          //TCP Client GET, send a GET request to the server and
          //save the reply.
          numdata=inet.httpGET("www.boat.esy.es", 80, "/retrieve.php", msg, 1);
          //Print the results.
          Serial.println("\nNumber of data received:");
          Serial.println(numdata);
          Serial.println("\nData received:");
          //Serial.println(msg);

          char* content = strstr(msg,"\r\n\r\n");
          content = content+4;
          Serial.println(content);
     }
};

void loop()
{
     //Read for new byte on serial hardware,
     //and write them on NewSoftSerial.
     serialhwread();
     //Read for new byte on NewSoftSerial.
     serialswread();
};

void serialhwread()
{
     i=0;
     if (Serial.available() > 0) {
          while (Serial.available() > 0) {
               inSerial[i]=(Serial.read());
               delay(10);
               i++;
          }

          inSerial[i]='\0';
          if(!strcmp(inSerial,"/END")) {
               Serial.println("_");
               inSerial[0]=0x1a;
               inSerial[1]='\0';
               gsm.SimpleWriteln(inSerial);
          }
          //Send a saved AT command using serial port.
          if(!strcmp(inSerial,"TEST")) {
               Serial.println("SIGNAL QUALITY");
               gsm.SimpleWriteln("AT+CSQ");
          }
          //Read last message saved.
          if(!strcmp(inSerial,"MSG")) {
               Serial.println(msg);
          } else {
               Serial.println(inSerial);
               gsm.SimpleWriteln(inSerial);
          }
          inSerial[0]='\0';
     }
}

void serialswread()
{
     gsm.SimpleRead();
}

And this is my PHP code

<?php 

    $server = "mysql.hostinger.in";
    $user = "xxxxxxxxxx";
    $password = "xxxxxx";
    $db = "xxxxxxx";

    $lck;
    $ign;

    $conn = mysqli_connect($server,$user,$password,$db);

    if($conn->connect_error)
    {
        die("Connection Failed:" . $conn->connect_error);
    }

    $query = "SELECT unlck_lck, ignition FROM BOATOP";
    $res = $conn->query($query);

    if($res->num_rows>0)
    {
        $row = $res->fetch_assoc();
        $lck = $row["unlck_lck"];
        $ign = $row["ignition"];

        echo $lck;
        echo $ign;
    }

    mysqli_close($conn);


?>

I want my Arduino to read only the output echoed by my PHP file. But currently it's reading everything.

And I also need to mention one thing here. For my Arduino GSM Shield, I haven't used the examples or libraries that come with the Arduino installation pack since the libraries weren't compatible with my GSM Shield. Instead, I have used some libraries I found online. I didn't post those files here because they would make my post really long. Please let me know if you need to have a look at them as well.

UPDATE 1

Content showing null

Arduino modification

content = strstr(msg,"\r\n\r\n");
if(content == NULL) {
    Serial.println("ERROR IN CONTENT READING");
} else {
    content = content + 4;
    Serial.println("The Content Is:");
    Serial.println(content);
}

Result:

GSM Shield testing.

status=READY
status=ATTACHED

100.110.244.218

Number of data received:
50

Data received:

ERROR IN CONTENT READING
HTTP/1.1 200 OK
Date: Sat, 04 Jun 2016 17:17:12 GMT
Server: Apache
X-Powered-By: PHP/5.5.35
Content-Length: 2
Connection: close
Content-Type: text/html

11
CLOSED
1

1 Answers

2
votes

Your issue is actually not one. You should read more on the HTTP protocol specification to fully understand what your problem is.

When you have your PHP script return some output, it is being served by the HTTP server. Your client is communicating with the server with a well documented protocol such as:

  1. client→server:

Please give me the page at this address!

GET www.boat.esy.es/retrieve.php HTTP/1.1
  1. server→client:

Hey, here's a few details about your request, followed by your content

HTTP/1.1 200 OK
Date: Sat, 04 Jun 2016 13:07:05 GMT
Server: Apache
X-Powered-By: PHP/5.5.35
Content-Length: 2
Connection: close
Content-Type: text/html

11

the context being given usually helps your webbrowser better handle your data. But in the embedded environment it can help you too. For example, if Content-Length is too big for your variable, you can detect that and show an error that you cannot hold that much data into memory!

So as a simple solution in your arduino code, you need to read the output of the webserver, discarding every line it outputs until it reaches \r\n\r\n which in the HTTP protocol is the separator between headers and content. Then all the following data will be your content as it has been output by your PHP code.

A way to do it would be to do:

numdata=inet.httpGET("www.boat.esy.es", 80, "/retrieve.php", msg, 1);
// look up substring '\r\n\r\n' within msg, and allocate content with it
char * content = strstr(msg, "\r\n\r\n");
if (content == NULL) {
    Serial.println("Couldn't find the contents within the message!");
} else {
    // then we skip the '\r\n\r\n' substring to only keep the contents
    content = content+4; // we skip 4 characters
    Serial.print("The content is: ");
    Serial.println(content);
}

If you were to try to extract the content-length header, you could do the same:

// lookup the content-length header
char* content = strstr(msg, "Content-Length: ");
// skip the size of the looked up string
content = content+strlen("Content-Length: ");
// store the size of the contents in number of characters
int nb_chars = atoi(content);
// lookup the content
content = strstr(msg, "\r\n\r\n");
// check that the substring has been found
if (content == NULL) {
    Serial.println("Couldn't find the contents within the message!");
} else {
    // skip the header-content delimiter
    content = content+4;
    // and do useful things with the extracted data
    if (nb_chars == 2) {
        Serial.print("The content is:");
        Serial.println(content);
    } else {
        Serial.println("There's been an error processing the request.");
    }
}

You'll find a running version of the code there:


As a test you might want to print the ASCII codes of each character of the string, to test it, so in place of your original Serial.println(msg) try:

for (int c=0; c<strlen(msg); ++c) {
    Serial.print((int)msg[c], HEX);
    Serial.print(" ");
}
Serial.println("");

which should produce the following output:

0048 0054 0054 0050 002F 0031 002E 0031 0020 0032 0030 0030 0020 004F 004B 000D 000A 0044 0061 0074 0065 003A 0020 0053 0061 0074 002C 0020 0030 0034 0020 004A 0075 006E 0020 0032 0030 0031 0036 0020 0031 0033 003A 0030 0037 003A 0030 0035 0020 0047 004D 0054 000D 000A 0053 0065 0072 0076 0065 0072 003A 0020 0041 0070 0061 0063 0068 0065 000D 000A 0058 002D 0050 006F 0077 0065 0072 0065 0064 002D 0042 0079 003A 0020 0050 0048 0050 002F 0035 002E 0035 002E 0033 0035 000D 000A 0043 006F 006E 0074 0065 006E 0074 002D 004C 0065 006E 0067 0074 0068 003A 0020 0032 000D 000A 0043 006F 006E 006E 0065 0063 0074 0069 006F 006E 003A 0020 0063 006C 006F 0073 0065 000D 000A 0043 006F 006E 0074 0065 006E 0074 002D 0054 0079 0070 0065 003A 0020 0074 0065 0078 0074 002F 0068 0074 006D 006C 000D 000A 000D 000A 0031 0031 000D 000A 
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 ^^^^ ^^^^ ^^^^ ^^^^

Here, the 000D 000A are respectively \r\n. So you should find the 000D 000A 000D 000A sequence at the same position as highlighted with the ^^^^ ^^^^ ^^^^ ^^^^.

That should help you double check that the string produced by the server is the one we expect when parsing with strstr().

HTH