AmiWest 2013 Lesson 4

Simple IP Clients & Servers

Simple network access is easy

Developing applications that use TCP/IP "sockets" for network communications have a reputation of being a challenging task. But AmigaOS and its Roadshow TCP/IP stack provides a couple of built-in shorthand mechanisms for easily creating simple network clients and servers: the TCP: handler and Roadshow's Superserver.

Using these Roadshow shortcuts, creating simple network client and server applications for single user or light-duty uses is no more complicated than reading or writing a file. The real challenge is understanding the protocol to be used (f.e. HTTP communications with web browsers) and being careful not to create a security risk on your or another user's Amiga.

An Internet Client

First, we will look at creating a small client to retrieve some simple information from an Internet website. Similar applications could be made to retrieve basic information like stock quotes, email contents, server conditions or other basic interactions. Aminet has a number examples of such clients written in ARexx that use the TCP: device that can be examined.

The Roadshow shortcut we will use is its built-in TCP: handler. This is mechanism that creates a virtual filesystem device ("TCP:") whenever Roadhsow gets online. To interact with a remote server, one just opens a file with the name being the URL of the server and port to be accessed.

Connect to the Server

To begin with, one opens a connection to the internet server as if one were opening a file. As described above, when Roadshow gets online, it creates a virtual TCP: device. To open a connection, one opens a file to the URL and port in question using this format:

TCP:<server URL>/<port number>

For example, to connect to the website "www.wunderground.com" one would use:

	fileh = fopen("TCP:www.wunderground.com/80","r+");

The file name starts with TCP:, then the website URL and finally the port number - here "80" is site's HTTP port (used by almost all websites).

Typically, the port number reflects the type of port or protocol that is to be used. Such protocols could be FTP (file transfer), POP/SMTP (email) or many others. Port 80 is usually used for HTTP communications with websites.

If the file open request is successful, your Amiga is then connected to that server and any further interaction with the "fileh" file handle communicates directly with that port on that server. In this case, we would have connected to a websserver.

Talk to the Server

Once one has connected to a server, knowledge of the communications protoccol for that type of connection is required.

For each of those, a server expects a certain format of interaction to proceed and respond. Fortunately, common protocols are publicly documented.

http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers

For each protocol there is usually an "RFC" document that describes how one uses the protocol, interacts with the port and accomplishes that protocol's goals.

In the case of "HTTP" (or a "Hypertext Transfer Protocol") connection to a webserver, we are usually expected to submit a "GET" request. Here are a couple more complete references on the HTTP protocol:

http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol

http://tools.ietf.org/html/rfc2616

To keep things simple, lets just look at submitting a "GET" request, such as a web browser, like IBrowse, would do. A simple HTTP version 1.1 GET request would include the following lines:

	GET /<path> HTTP/1.1
	host: <domain>
	user-agent: IBrowse/2.4 (AmigaOS 4.1; PPC; 68K build)
	Pragma: no-cache
	Accept-Language: en, *
	Accept: text/html;level=3
	Accept: text/html;version=3.0
	Accept: */*
	<blank line>

The <path> would be the rest of the URL (if any) on that system and <domain> is the name of the server.

Using the "WeatherUnderground" website URL from above and their pathname for searching for airport conditions, we can find the current conditions at Washington's Dulles International Airport with these domain & path values:

   URL = www.wunderground.com
   PATH = cgi-bin/findweather/getForecast?query=IAD

The GET request syntax can be transcribed into C code which builds the entire request in a string and then sends it to the server with a simple "fprintf" to our open file handle, as follows:

	// build web req text
	strcpy(line,"GET /");
	strcat(line,path);
	strcat(line," HTTP/1.1\r\n");
	strcat(line,"host: ");
	strcat(line,domain);
	strcat(line,"\r\n");
	strcat(line,"user-agent: IBrowse/2.4 (AmigaOS 4.1; PPC; 68K build)\r\n");
	strcat(line,"Pragma: no-cache\r\n");
	strcat(line,"Accept-Language: en, *\r\n");
	strcat(line,"Accept: text/html;level=3\r\n");
	strcat(line,"Accept: text/html;version=3.0\r\n");
	strcat(line,"Accept: */*\r\n");
	strcat(line,"\r\n");
 
	// send web req line to server
	fprintf(fileh,"%s",line);

As you can see, the variables "path" and "domain" are used to fill in some blanks in the request. The blank line at the end completes the GET request, after which the server will reply using the same file handle.

Listen to the Server

Once the GET request has been sent to the webserver, the server will start sending back either the requested webpage (or other content) or an error page. To see what sort of content is returned by the server, you can combine the domain and path values above and enter them in your web browser:

   www.wunderground.com/cgi-bin/findweather/getForecast?query=IAD

Once the page is loaded, viewing the page source (f.e., IBrowse menu item "Page/Display Source...") will show you the same text that your program will receive after sending the get request.

As such, your program needs to start reading from the same file handle the request was sent with. This loop will print out the first hundred lines returned:

	// read in the response
	count = 0;
	while( (fgets(inStr,MAX_STR,fileh) != NULL) && (count<100) )
	{
		++ count;
		printf("line %ld = %s\n",count,inStr);
 
		// process lines received here !
	}

Within this loop your applcation can parse the received lines for whatever data your application is tryng to obtain. In the case of our example program below, the program searches for a string precedes the weather information we want.

After the returned content has been read, output or otherwise processed, simply closing the file handle will close the connection to the server:

	fclose(fileh);

EXAMPLE ONE: IPClient.c

All of these elements are combined in the following example program "IPClient.c" that asks you for a IATA airport code (f.e., IAD = Washington Dulles, FRA=Frankfurt, SYD=Sydney), then parses & prints out the weather conditions from the returned page.

Please Note: this example program worked with the wunderground.com website in 2013-2014, as the served webpages change over time, it is likely the parsing in this example is likely to fail and the program no longer return useful information.

/*
************************************************************
**
** Created by: CodeBench 0.41 (12.10.2013)
** Project: IPclient
** Date: 12-10-2013 18:51:10
**
************************************************************
*/
 
#include <stdio.h>
#include <string.h>
 
#include <proto/exec.h>
#include <proto/dos.h>
 
/*****************************************************************************
 *		GLOBAL VARIABLES
 *****************************************************************************/
 
STATIC CONST_STRPTR version USED = "$VER: IPclient v.02 (12.10.2013)";
 
char URL[200] = "www.wunderground.com/cgi-bin/findweather/getForecast?query=";
char domain[200];
char path[200];
char fname[256] = "";
#define MAX_STR 2056
char line[MAX_STR] = "";
char inStr[MAX_STR];
uint16 uLen;
uint16 dLen;
uint16 count = 0;
char *found;
FILE *fileh;
 
/*****************************************************************************
 *		PROGRAM START
 *****************************************************************************/
 
int main(int argc,char **argv)
{
	// get URL from user
	printf("IPclient example\n");
	printf("======================\n");
 
	printf("   enter airport code >");
	fgets(line,sizeof(line),stdin);
	line[strlen(line)-1] = '\0';		// strip \n off end of string
	printf("======================\n");
 
	// append airport code on URLprefix
	strcat(URL,line);
 
	// parse domain from path
	uLen = strlen(URL) - 2;
	strcpy(domain,strtok(URL,"/\n"));
	dLen = strlen(domain);
	if (uLen>dLen)
		strcpy(path,strtok(NULL,"\n"));
	else
		printf("   No chars remain for path\n");
	printf("   URL domain >%s<\n",domain);
	printf("   URL path >%s<\n",path);
 
	// build URL filename from domain name
	strcpy(fname,"TCP:");
	strcat(fname,domain);
	strcat(fname,"/80");
 
	// open file access to URL
	printf("      Opening URL >%s<\n",fname);
	fileh = fopen(fname,"r+");
	if (fileh == NULL)
	{
		printf("Couldn't open connection domain server via \"%s\"\n",fname);
		return RETURN_ERROR;
	};
	printf("      TCP: file opened\n");
 
	// build web req text
	strcpy(line,"GET /");
	strcat(line,path);
	strcat(line," HTTP/1.1\r\n");
	strcat(line,"host: ");
	strcat(line,domain);
	strcat(line,"\r\n");
	strcat(line,"user-agent: IBrowse/2.4 (AmigaOS 4.1; PPC; 68K build)\r\n");
	strcat(line,"Pragma: no-cache\r\n");
	strcat(line,"Accept-Language: en, *\r\n");
	strcat(line,"Accept: text/html;level=3\r\n");
	strcat(line,"Accept: text/html;version=3.0\r\n");
	strcat(line,"Accept: */*\r\n");
	strcat(line,"\r\n");
 
	// send web req line to server
	fprintf(fileh,"%s",line);
	strcpy(line,"");
	printf("      Web req sent\n");
	printf("======================\n");
 
	// read in the response
	while( (fgets(inStr,MAX_STR,fileh) != NULL) && (count<100) )
	{
		++ count;
		//printf("line %d = %s\n",count,inStr);
 
		// look for name of airport
		found = strstr(inStr,"og:title");
		if (found != NULL)
		{
			found = strtok(inStr,"=");
			found = strtok(NULL,"=");
			printf("WeatherUnderground.com reports:\n");
			printf("   Airport   = %s\n",strtok(NULL,"|")+1);
			printf("   Temp      =%s F\n",strtok(NULL,"&"));
			found = strtok(NULL,"|");
			printf("   Condition =%s\n",strtok(NULL,"\""));			
		}
	}
	printf("======================\n");
 
	// close file
	fclose(fileh);
 
	return RETURN_OK;
}

Designing Your Network Client

As you see, the above code and the example program, the mechanics of connecting to a internet server and obtaining data are relatively trivial. The real challenge for the developer lies in understanding the protocol used and in processing the data sent and received.

Public webservers make for fairly rich and easy targets to interact with. You can explore simple interaction with any webbrowser and mimic that interaction with your code. In many cases, you can control the feedback just by crafting the web request as we did above and then parse the results.

Naturally, the more complicated the interactions, like logging into a website, would require much more knowledge of the protocols used more complicated code that is beyond the scope of this tutorial.

But there are a number of basic topics you should consider when designing a web client!

Server Suitability:

Unless you are connecting to your own server or it invites such connections (some publish API's for you to make such connections), your application's "visit" may not be welcome or even legal.

At the very least, your program should "tread lightly" - do not abuse the server with unnecesary or intrusive requests. Furthermore, if your client uses the resources of someone else's server, you should credit the server within your application and documentation.

Server Changes:

As with any programming that interacts with external inputs, your code should provide for the possibility the connection may not succeed or that the returned results may not be what was expected.

As servers and websites change over time, the processing and parsing of your program may often have to change accordingly. To maintain your program and deal with such changes, you may want to consider how to make changing those things easy (by the user?). The parsing strings could be kept in program tooltypes or config text file, you could provide a GUI for managing the parsing, employ an external, editable ARexx macro.

Protocol Details & Vagaries:

Our example code presented a very simple interaction with a web server using the a widely supported version of the HTTP protocol. As one develops an application and uses a protocol, one should get familiar with the details of that protocol. Typically a "RFC" document (as linked above) will describe the details of how things are to work and how they might fail. Your application should be careful to comply with the details of the protocol ("Are we sending a CR-LF or LF-CR?!").

Ideas

The above example was just a quick, simple exercise in demosntrating the use of the AmigaOS Roadshow TCP: handler to access a web server. There are many ways applications could be created to have simple interactions with all sorts internet services, such as:

· Check stock quotes from a financial site.
· Get package tracking information from a shipping site.
· Check for new emails on a POP email server.
· Convert currencies with a financial or travel web site.
· Check for files on an FTP server.
· Send messages with a SMTP email server.
· Look up words on a dictionary website.

You could also write your own server for another Amiga and interact with your own client. A remote controlled media player?