XML-RPC is meant to abstract away how a remote procedure call is translated into an exchange of XML documents over HTTP so that you as a user of XML-­RPC don’t have to understand the underlying process. That’s the theory with XML-­RPC and especially with SOAP, an expansive elaboration on XML-­RPC out of which it originally evolved. In practice, with the right tools and under certain circumstances, consuming services with XML-­RPC or SOAP is a very simple, trouble-­free experience.

At other times, however, you’ll find yourself having to know more about the underlying protocol than you really need to know. For that reason, in the following sections I’ll show you techniques for making sense of what XML is actually being exchanged and how it’s being exchanged over HTTP. This discussion is meant as an explication of XML-­RPC in its own right but also as preparation for studying the yet more complicated SOAP later in the chapter. But first, let’s look at two tools that I use to analyze XML-­RPC and SOAP: Wireshark and curl.

Wireshark (http://www.wireshark.org/) is an open source network protocol analyzer that runs on Windows, OS X, and Linux. With it, you can analyze network traffic flowing through your computer, including any HTTP traffic—making it incredibly useful for seeing what’s happening when you are using web APIs (or, if you are curious, merely surfing the Web). Refer to the Wireshark site for instructions about how to install and run Wireshark for your platform.

	Tip
	With Wireshark, I found it helpful to turn off the Capture Packets in Promiscuous Mode option. Also, for studying web service traffic, I filter for only HTTP traffic—otherwise, there is too much data to view.

curl (http://curl.haxx.se/) is another highly useful command-­line tool for working with HTTP—among many other things:

Go to http://curl.haxx.se/download.html to find a package for your platform. Be sure to look for packages that support SSL—you’ll need it when you come to some examples later this chapter. Remember in particular the following documentation:

To learn these tools, I suggest using curl to issue an HTTP request and using Wireshark to analyze the resulting traffic. For instance, you can start with the following:

curl http://www.yahoo.com

to see how to retrieve the contents of a web page. To see the details about the HTTP request and response, turn on the verbose option and make explicit what was implicit (that fetching the content of http://www.yahoo.com uses the HTTP GET method):

curl -v -X GET http://www.yahoo.com

You can get more practice studying Wireshark and the Flickr API by performing some function in the Flickr UI or in the Flickr API Explorer and seeing what HTTP traffic is exchanged. Try operations that don’t require any Flickr permissions, and then try ones that require escalating levels of permissions. You can see certainly see the Flickr API being invoked and when HTTP GET vs. HTTP POST is used by Flickr—and specifically what is being sent back and forth.

I’ll teach you more about curl in the context of the following examples.

When you look at the documentation for the XML-­RPC request format for Flickr ?(http://?www.flickr.com/services/api/request.xmlrpc.html) and for the response format (http://www.flickr.com/?services/api/response.xmlrpc.html), you’ll find confirmation that the transport mechanism is indeed HTTP (just as it for the REST request and response). However, the request parameters and response are wrapped in many layers of XML tags. I’ll show you how to use Wireshark and curl to confirm for yourself what’s happening when you use XML-­RPC.

Here I use Wireshark to monitor what happens when I run the Python example that uses the flickr.photos.search method and then use curl to manually duplicate the same request to show how you can formulate XML-­RPC requests without calling an XML-­RPC library per se. Again, I’m not advocating this as a practical way of using XML-­RPC but as a way of understanding what’s happening when you do use XML-­RPC.

When I ran the Python program and monitored the HTTP traffic, I saw the following request (an HTTP POST to /services/xmlrpc/):

POST /services/xmlrpc/ HTTP/1.0

It had the following HTTP request headers:

Host: api.flickr.com
User-Agent: xmlrpclib.py/1.0.1 (by www.pythonware.com)
Content-Type: text/xml
Content-Length: 415
         

and the following request body (reformatted here for clarity):

<?xml version='1.0'?>
<methodCall>
  <methodName>flickr.photos.search</methodName>
  <params>
    <param>
      <value><struct>
        <member>
          <name>per_page</name>
          <value><int>3</int></value>
        </member>
        <member>
          <name>api_key</name>
          <value><string>[API-KEY]</string></value>
        </member>
        <member>
          <name>tags</name>
          <value><string>flower</string></value>
        </member>
      </struct></value>
    </param>
  </params>
</methodCall>
         

The HTTP response (edited here for clarity) was as follows:

HTTP/1.1 200 OK
Date: Sun, 26 Aug 2007 04:33:29 GMT
Server: Apache/2.0.52
[...some cookies....]
Content-Length: 1044
Connection: close
Content-Type: text/xml; charset=utf-8

<?xml version="1.0" encoding="utf-8" ?>
<methodResponse>
  <params>
    <param>
      <value>
        <string>
          &lt;photos page=&quot;1&quot; pages=&quot;485823&quot;
          perpage=&quot;3&quot; total=&quot;1457468&quot;&gt;
          &lt;photo id=&quot;1237314286&quot; owner=&quot;41336703@N00&quot;
          secret=&quot;372291c5f7&quot; server=&quot;1088&quot; farm=&quot;2&quot;
          title=&quot;250807 047&quot; ispublic=&quot;1&quot; isfriend=&quot;0&quot;
          isfamily=&quot;0&quot; /&gt;
          &lt;photo id=&quot;1236382563&quot; owner=&quot;70983346@N00&quot;
          secret=&quot;459e79fde3&quot; server=&quot;1376&quot; farm=&quot;2&quot;
          title=&quot;Darling daisy necklace&quot; ispublic=&quot;1&quot;
          isfriend=&quot;0&quot; isfamily=&quot;0&quot; /&gt;
          &lt;photo id=&quot;1237257850&quot; owner=&quot;39312862@N00&quot;
          secret=&quot;fa9d15f9c3&quot; server=&quot;1272&quot; farm=&quot;2&quot;
          title=&quot;Peperomia species&quot; ispublic=&quot;1&quot;
          isfriend=&quot;0&quot; isfamily=&quot;0&quot; /&gt;
          &lt;/photos&gt;
        </string>
      </value>
    </param>
  </params>
</methodResponse>
         

To make sense of the interchange, it’s useful to study the XML-­RPC specification (http://www.xmlrpc.com/spec) to learn that the Flickr XML-­RPC request is passing in one struct that holds all the parameters. The request uses HTTP POST. What comes back in the response is an entity-­encoded XML <photos> element (the results that we wanted from the API), wrapped in a series of XML elements used in the XML-­RPC protocol to encapsulate the response. This process of serializing the request and deserializing the response is what an XML-­RPC library does for you.

We can take this study of XML-­RPC one more step. You can use curl (or another HTTP client) to confirm that you can synthesize an XML-­RPC request independently of any XML-­RPC library to handle the work for you. This is not a convenient way to do things, and it defeats the purpose of using a protocol such as XML-­RPC—but this technique is helpful for proving to yourself that you really understand what is really happening with a protocol.

To wit, to call flickr.photos.search using XML-­RPC, you need to send an HTTP POST request to http://api.flickr.com/services/xmlrpc/ whose body is the same as what I pulled out using Wireshark. The call, formulated as an invocation of curl, is as follows:

curl -v -X  POST --data-binary "<?xml version='1.0' encoding='UTF-8'?>
  <methodCall><methodName>flickr.photos.search</methodName><params><param><value>
  <struct><member><name>per_page</name><value><int>3</int></value></member><member>
  <name>api_key</name><value><string>[API-KEY]</string></value></member><member>
  <name>tags</name><value><string>flower</string></value></member></struct></value>
  </param></params></methodCall>"  http://api.flickr.com/services/xmlrpc/
         

	Note
	To write curl invocations that work from the command line of Windows, OS X, and Linux, I rewrote the XML to use single quotes to allow me to use double quotes to wrap the XML.

You can issue this request through curl to convince yourself that you are now speaking and understanding XML-­RPC responses!

An XML-­RPC library is supposed to hide the details you just looked at from you. One of the major practical problems that I have run into when using XML-­RPC (and SOAP) is understanding for a given language and library how exactly to formulate a request. Notice some important lines from the examples. An essentialist rendition of the Python example is as follows:

server = ServerProxy("http://api.flickr.com/services/xmlrpc/")
args = {'api_key': API_KEY, 'tags':'flower', 'per_page':3}
rsp = server.flickr.photos.search(args)
rsp = rsp.encode('utf-8')
tree = et.XML(rsp)
print "total number of photos: %s" %(tree.get('total'))
         

Besides the mechanics of calling the right libraries, you had to know how to pass in the URL endpoint of the XML-­RPC server—which is usually not too hard—but also how to package up the parameters. Here, I had to use a Python dictionary, whose keys are the names of the Flickr parameters. I then call flickr.photos.search as a method of server and get back XML.

The PHP example can be boiled down to this:

$server = new XML_RPC_Client('/services/xmlrpc','http://api.flickr.com',80);
$myStruct = new XML_RPC_Value(array(
   "api_key" => new XML_RPC_Value($API_KEY, "string"),
   "tags" => new XML_RPC_Value('flower',"string"),
   "per_page" => new XML_RPC_Value('2',"string"),
   ), "struct");
$msg = new XML_RPC_Message('flickr.photos.search', array($myStruct));
$resp = $server->send($msg);
$val = $resp->value()->scalarval();
         

Again, I knew what I had to tell PHP and the PEAR::XML_RPC library, and once someone provides you with skeletal code like I did here, it’s not hard to use. However, it has been my experience with XML-­RPC and especially SOAP that it takes a lot of work to come up with the incantation that works. Complexity is moved from having to process HTTP and XML directly (as you would have using the Flickr REST interface) to understanding how to express methods and their parameters in the way a given higher-­level toolkit wants from you.