SAS and Curl :Host not found error

Question

I am trying to establish communication between SAS and web by sending html headers and hence be able to communicate using GET/POST over http/https.

I am attempting to connect using Curl using the below code:

filename curl pipe 'c:\curl\curl http://support.sas.com" lrecl=8192;
data _null_;
infile curl;
input;
file log;
put _infile_;
run; This code snippet runs perfectly fine when i execute it using my home internet connection. But when i try and execute the same using my worplace internet it gives an error: Host not found. I understand that when I run it on my workplace it assumes support.sas.com to be a part of the local lan hence the error. But how to get to the actual website and get the source code in sas log.

I am facing the same issue when i use filename socket.

Please advice.

Thanks Abhishek

Answer 1

It's possible to achieve this but unfortunately it's not all that simple. Below is the macro I use to do this kind of thing. I suggest reading through the whitepaper that is linked to in the header.

Once you understand what they're doing in the whitepaper, the below code should make more sense. Look at the 2 examples I provide below the macro. The first example is simply accessing a regular HTTPS web page. The second example I use to pull data that requires a login via a user/password form on a web page. I've obviously changed the domains, page names, etc in the 2nd example so it's not going to work for you but it should give you a starting point.

I should also add that I have also used CURL to pull data from web services that allowed me to specify the user/pass in the request URL. AFAIK if you can't specify the login details in the request URL then the only way to do it is using the below technique.

Finally, this also will require you to sort out your proxy settings as per @Laurent de Walick mentions above.

/*****************************************************************************
**  PROGRAM: CMN_MAC.HTTP.SAS
**
**  PERFORMS A HTTP REQUEST AND SAVES THE RESULT TO A FILE. 
**  BASED ON WHITEPAPER FOUND AT: http://www2.sas.com/proceedings/sugi30/100-30.pdf
**  
**  PARAMETERS: SEE http://www2.sas.com/proceedings/sugi30/100-30.pdf
**
******************************************************************************
**  HISTORY:
**  1.0 MODIFIED: 7-MAY-2014  BY:RP
**  - CREATED
*****************************************************************************/

%macro http(http=%nrstr(), save_file=, hostname=, delimiter=%str(|),fileref=browser, termstr=CRLF);
  %local str1 str2 pos http;

  %let str1=; 
  %let str2=;
  %do %while (%index(&http,&delimiter)>0);
    %let pos=%index(&http,&delimiter);
    %let str1=&str1%qsubstr(&http, 1, &pos-1)/;
    %let str2=&str2%qsubstr(&http, 1, &pos-1)'0d0a'x;
    %let http=%qsubstr(&http, &pos+1, %length(&http)-&pos);
  %end;

  filename &fileref socket "&hostname:80" termstr=&termstr lrecl=8192;
  data _null_;
    retain chunked;
    length rec2 $10000 chunked $1;
    infile &fileref length=len;
    if _n_=1 then do;
    file &fileref;
      put %unquote(&str1);
      file &save_file;
      put '>>> HTTP Request:' '0d0a'x;
      put %unquote(&str2);
      put '0d0a'x '<<< HTTP Response:' '0d0a'x;
    end; 

    file &save_file recfm=n;
    input record $varying8192. len;
    put record $varying8192. len '0d0a'x;
    if record='Transfer-Encoding: chunked' then chunked='Y';
    if len=0 and chunked='Y' then do while (1); %*** look for & process chunks;
      input chunksize $varying8. len / record $varying8192. len;
      if len ne 0 then do;
        l2=len+countc(record, '0a'x);
        rec2=tranwrd(record, '0a'x, '0d0a'x);
        put rec2 $varying10000. l2;
      end;
      else stop; *** found null record at end of file, end loop;
    end;
  run;
%mend;

A simple example:

%http(
hostname=support.sas.com,
save_file='c:\support_sas.txt',
http=%nrstr(
'GET / HTTP/1.1' |
'Host: support.sas.com' |
'Connection: Close' | )
); 

A more complex example:

/*
** INSTRUCTIONS: 
** 1) IN FIREFOX, INSTALL THE LIVEHTTPHEADERS PLUG IN
** 2) ONCE ITS ENABLED, GO INTO THE OPTIONS AND MAKE SURE ITS TURNED ON IN THE SIDEBAR
** 3) MAKE SURE THE SIDE-BAR IS VISIBLE VIEW->OPTIONS->SIDEBAR
** 4) OPEN A PRIVATE BROWSER WINDOW.  
** 5) PASTE IN THE URL.  ENTER THE SIGNON DETAILS AND GET TO THE PAGE YOU DESIRE
** 6) COPY/PASTE THE POST REQUEST THAT WAS MADE FROM THE SIDEBAR.
** 7) IF THE POST REQUEST HAS A LINE THAT SAYS ACCEPT-ENCODING GZIP THEN DELETE IT
** 8) MAKE SURE THAT YOU MOVE THE REQUEST PARAMETERS FROM THE VERY FIRST LINE TO THE LINE UNDER THE 
**    CONTENT LENGTH.  ADJUST THE CONTENT LENGTH ACCORDINGLY.
** 9) MAKE SURE YOU CHANGE THE CONNECTION TO CLOSE RATHER THAN KEEP ALIVE.
** 10) I THINK THATS IT.
** 
** EXAMPLE:
** 
** 
** URL I WANTED TO RETRIEVE THAT WAS BEHIND XXX LOGIN PAGE:
** 
** http://BLAH_tools.someplace.com/soap.php?id=9fb7692e120e5a7fc6ad1af44b
** 
** THE STUFF I COPIED AND PASTED FROM THE LIVEHTTPHEADERS SIDEBAR:
** 
** POST /soap.php?id=9fb7692e120e5a7fc6ad1af44b HTTP/1.1
** Host: BLAH_tools.someplace.com
** User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
** Accept: text/html,application/xhtml+xml,application/xml;q=0.9,* / *;q=0.8
** Accept-Language: en-US,en;q=0.5
** Accept-Encoding: gzip, deflate
** Referer: http://live.x.com/?v=2&r=ovL29scF90b29scy5qdWJpbGVlLnRzcy9hcHBsaWNhdGlvbi9zb2FwX3Bvc3GhwP3VuaXF1ZV9pZD0yYTMxOWZiNzY5MmUxMjBlNWE3ZmM2YWQxYWY0NGIxZiZtb2RlPXNs
** Cookie: XXX_key=1e43eeaec0fbc7e506579c
** Connection: keep-alive
** Cache-Control: max-age=0
** Content-Type: application/x-www-form-urlencoded
** Content-Length: 46
** key=7e1e43eeaec0fbc7e506579ce7fc8edd&postdata=
** 
** THE CALL TO THE MACRO AFTER CHANGES HAD BEEN MADE TO THE HEADERS:
*/

 %http(hostname  = BLAH_tools.someplace.com,
       save_file = 'c:\xxx.txt',
       http      = %nrstr(
 'POST /application/soap_post.php HTTP/1.1' |
 'Host: BLAH_tools.someplace.com' |
 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0' |
 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,* / *;q=0.8' |
 'Accept-Language: en-US,en;q=0.5' |
 'Referer: http://live.x.com/?v=2&r=ovL29scF90b29scy5qdWJpbGVlLnRzcy9hcHBsaWNhdGlvbi9zb2FwX3Bvc3GhwP3VuaXF1ZV9pZD0yYTMxOWZiNzY5MmUxMjBlNWE3ZmM2YWQxYWY0NGIxZiZtb2RlPXNs' |
 'Cookie: XXX_key=1e43eeaec0fbc7e506579c' |
 'Connection: close' |
 'Cache-Control: max-age=0' |
 'Content-Type: application/x-www-form-urlencoded' |
 'Content-Length: 100' |
 |
 'id=9fb7692e120e5a7fc6ad1af44b&key=1e43eeaec0fbc7e506579c&postdata=' |
   )
 );

** 
** FINALLY - NOTE THAT TO PARAMETERIZE THE REQUEST VARIABLES MY FINAL LINE LOOKED LIKE THIS:
*; 
"id=&iMyID%nrstr(%bquote(&)key=1e43eeaec0fbc7e506579c%bquote(&)postdata=)" |