IS.HTTPCLIENT - D3 9.1 and above
D3 9.1 and above includes support for both normal sockets and SSL sockets. This provide this version of D3 the ability to access both Http:// and Https:// without requiring an external OS routine.
Since D3 9.1 provide support at the socket level using CFUNCTION SOCKET.BUILTIN, you normally would have to handle the complete http protocol yourself.
The subroutine included with this article removes that set of complexity by creating a single subroutine that you can call.
SUBROUTINE IS.HTTPCLIENT(HTTP.METHOD,URL,HEADER.DATA,POST.DATA,RESP.HTTP.STATUS,RESP.HEADERS,RESP.DATA,ERROR)
- HTTP.METHOD
This is the HTTP Method you are asking the web server to process. Normally, this is either 'GET' or 'POST' - URL
This is the "Http://" or "Https://" want to request data from or send data to. - HEADER.DATA<1,n>
Http Request Header Name - This is the header name that the developer wants to included with the request to the server. - HEADER.DATA<2,n>
Http request Header data - This is the data you want to include with this header name - POST.DATA<-1>
This is the data you want to send to the web server. Each AM (Char(254)) will be converted to a CRLF. - RESP.HTTP.STATUS
Http response returned by the web server. That values are between 100-500. Most successful request will return with a value of 200 or 201 as per the HTTP protocol. - RESP.HEADER<1,N>
Header Names returned with the response data. This is provided by the web server, and may include important information. - RESP.HEADER<2,N>
Value for the header data - RESP.DATA<n>
This is the data returned by the web server based on the URL, HTTP.METHOD, and POST.DATA. Each CRLF will be converted into an AM mark (char(254)) - ERROR
All other communication errors, parsing error, or anything not related to the HTTP protocol will be returned here. If there is no error, then this will return '' or '0'
Example:
CALL IS.HTTPCLIENT("GET","http://www.google.com","","",HTTP.STATUS,"",HTTP.RESP,ERROR) IF (ERROR<1> GT 0) THEN CRT "Error ": ERROR STOP END * CRT HTTP.RESP[1,200]
D3 CFUNCTION
D3 allows developers to create external C library that you can bind and access in the D3 Environment. Starting in 7.x there was a built in library for Socket calls called SOCKET.BUILTIN. This allowed developers to access non-encrypted sockets, but this meant that SSL enabled web services could not be accessed.
Starting in 9.1, the SOCKET.BUILTIN added the %CONNECT_SSL command that took care of making the connection to an SSL enabled socket on a web server. It also handled all the encryption and decryption that was needed to make SSL communication work.
This requires that OpenSSL be installed on machines that you want to use %CONNECT_SSL.
CFUNCTION binding also requires you to FlashBASIC the subroutine. If you currently do not FlashBASIC your programs, this may become a logistic issue. This requires that all programs that call this subroutine be FlashBASICed, as well, all the way to the main line program that was executed from TCL or EXECUTE.
OpenSSL for Windows
If you are running D3/Windows, then you will need to install a windows version of OpenSSL.
One place you can get a precompiled version of OpenSSL for Windows is:
Download the Win32 version of OpenSSL and copy libeay32.dll and ssleay32.dll into the D3Programs directory, and restart the D3 VME.
OpenSSL on Linux
OpenSSL can be found on almost all Linux/Unix machines natively, and will not likely require any updates.
OpenSSL Note:
OpenSSL is open source, and is up to you as a System Admin to keep it up to date.
HTTP Protocol
Working with the HTTP Protocol is actually very simple. It is a text-only protocol which allows you to build everything you need in one large string, and then pass it to the socket.
You then need to read the data being returned and then parse out the headers from the data. This again is pretty easy, but keep in mind the response back may be a very large string… 2MB or 10 MB depending on the web service or what you are trying to download.
This subroutine does not try to optimize processing of Very Large strings and will trying to start all that data in memory. This may result in a Overflow Runaway issues or, in rare cases, a Monitor Abort. If you know you need to handle Very Large Strings, then you might want to change the way read of the information is handled to write the Very Larges string as blocks into a temp file.
This program does not do that, but can be easily modified to do so.
License Notes:
This program is free software in the public domain; you can redistribute it and/or modify it in any way you wish. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Commercial support and warranties of this software and code can be purchased through International Spectrum.
This program is not the only means to do this, but is a starting point or working sample that can be extended.