The Internet is quite something isn’t it? It brings many different pieces of data together through an interface that all of us are familiar with (at least those that read my blog): a web browser. If we need to find the nearest Tube station or but stop in London, we can go to the Transport for London web site. If we wanted the latest news, we can go our favourite’s news provider’s web site, which has all sorts of neat pictures and articles to satiate our hunger. If we wanted to cook something new and interesting for dinner tonight, we can fire up our favourite cooking web site, or, more interestingly, use Google to find what we’re looking for (the latter can apply for most anything you’re looking for on this wild and wacky world-wide web of computers, networks and you).
What I have just described is of course, the Internet. You’re familiar with it. Perhaps you even landed on this post through a search engine hoping to read about Representational State Transfer (REST). It’s (mostly) a fantastic and accessible interface to a library of information from all over the world, all walks of life. Blissfully built upon a RESTful architecture using the Internet Protocol Suite (TCP/IP), Hypertext Transfer Protocol (HTTP), and Uniform Resource Identifiers (URIs) you have what has been referred to as the Human Web[1].
The Human Web
Simply put, the Human Web is the collective set of resources available on the Internet that was designed for human consumption.
Beyond the Internet’s underlying architecture, what makes this possible is the Hypertext Mark-up Language (HTML). HTML is a set of tags that can be used on information to describe its structure. For example, <H1> is an HTML tag to refer to a level one heading. The title of this post is REST: Tale of Two Webs, so I could reasonably do something like <H1>REST: Tale of Two Webs</H1> to convey that the content between the H1 tags is some sort of heading. But how does an Internet consumer see all there is to see on the web and never see these tags? The answer to that lies on the tools we use.
The other side of the HTML coin is the Web Browser. Web browsers, like Microsoft Internet Explorer, Mozilla Firefox, and Apple Safari, are computer programs that take in HTML input and output that data in a way that a human can easily understand, according to the specification published by the World Wide Web Consortium[2]. The Web Browser reads in the <H1> HTML tag, and understands that it should make that content of that tag big, bold, and whatever other styling the browser deems appropriate to represent a Level 1 Heading. (It is worth noting that HTML producers can optionally provide a style guide to the browser as to how to represent any HTML tag on-screen, and the browser user can, in most cases, tell the browser to use provided styling or their own custom styling set up in browser preferences, ignoring the author’s styling suggestion.)
With HTML as the language and the Web Browser as the tool, we can get what we need from the Internet in a way that makes sense to us. Let’s walk through the steps typically used to find something on the Internet:
1. Open a web browser on your computer.
2. Type: http://google.com (this is the URI).
3. Submit the request by clicking GO or pressing the ENTER key.
4. The request makes its way to the server associated with the Google.com host name and a response is returned based on the requester (computer web browser) and URI requested (the root default resource belonging to Google.com): an HTML representation of the Google Home Page.
5. The response makes its way back to the web browser in the form of an HTML document.
6. The browser renders the HTML document to you in a way that you can easily interact with.
7. Repeat from step 2 for another specific URI, click on one of the links presented or submit any forms required and continue from step 4, or proceed to step 8 if you have nothing more to do.
8. Close the web browser.
That’s how a human’s journey on the web typically goes. The Human Web alive and ticking.
It’s hard to argue with the success of the Human Web. The Internet’s resources meant for human consumption is growing and evolving. It started as simple static pages. Nowadays we have personalised content, interactive media, and rich user interfaces within the safe harbours of the web browser sandbox[3]. And there seems to be no limit to what can be done in the Human Web, as full-blown operating systems and office software suites are on the horizon as well.
Yes, humans have it good, don’t we? A strong open architecture underpins a set system where we can go out and get what we need when we need it, all things considered. So at this point, I ask myself:
Is there potential here outside of the scope of human use?
Is there a way that computer systems can leverage this architecture to interact and collaborate?
The Programmable Web
You could point out the work being done by the W3C and the WS-I on the Web Services Stack Specification[4] but frankly, I think this approach is riddled with serious issues. To pick one example, the Simple Object Access Protocol (SOAP) is by all intents a repeat of the HTTP protocol (making SOAP over HTTP redundant), causing the spec to be needlessly complicated (and by consequence difficult to deploy and maintain). Moreover, the WS-I stack is pretty anti-RESTful. It uses the URI more as a point of entry to a set of operations instead of a unique identifier for a resource. And the response you get is typically something more or less what you requested, with enough ambiguity to cause subsequent requests because of lack of information or receiving what you did and didn’t need.
No, this doesn’t look like the Human Web at all. So let’s take a step back from the path that’s been trodden and ask the questions again:
Is there potential here outside of the scope of human use?
Is there a way that computer systems can leverage this architecture to interact and collaborate?
It is from a clean slate and an analysis of the web as it is utilized by you and me through our web browsers that I consider something referred to as the Programmable Web[5]. Similar to the Human Web, it is the collective set of resources available on the Internet, but this time as designed for computing consumption. Imagine being able to programmatically get the data you need, from an internal or external service, just as easily as you do using Google or the like yourself. This is what the Programmable Web looks like. But is it possible? Is it feasible? I think so, and without all the needless complexities as well. To apply the same model as the Human Web to the Programmable Web, we need a similar approach to how we expose our data, along with tools like HTML and web browsers, for our computing platforms. What follows is my take on what we could use to solve both these problems and provide a clean guideline for web services.
Much like a human-intended HTML resource, exposing an API as well-defined URI resources that meaningfully support the HTTP methods is the first step. If I have a User Service, I’m going to want a URI that corresponds to a user. For example, I would have something like http://localhost/my-user-service/user/100 where the /user/100 portion refers to the User with User ID 100. A GET request would return some sort of representation of a specific user, whereas a POST request of the same slightly modified representation will update a specific user. Clean resource exposure through well-defined URIs is the first place to start.
Next, we need a meaningful and effective way in which to encode our representations. HTML works well for the Human Web because it’s a published specification for document parts that allows web browsers to easily implement and render. For the Programmable Web, I submit XML as a good choice for how to encode machine-friendly representations of our resources. XML lets us structure data in a meaningful way both to a human observer and a computer program (via an XML parser). XML over HTTP is easily handled. And XML is also language-agnostic, opening our service up to be cross-platform.
To continue the User Service example, I would encode my User representation in XML, so that an HTTP GET request for the http://localhost/my-user-service/user/100 URI would return:
<user>
<id>100</id>
<name>Daniel Silva</name>
<favourite-arch>SOAP</favourite-arch>
</user>
But, there’s a mistake! SOAP is not my favourite. We have to take care of that immediately by posting the following to the http://localhost/my-user-service/user/100 URI via HTTP POST:
<user>
<id>100</id>
<name>Daniel Silva</name>
<favourite-arch>REST</favourite-arch>
</user>
And my service should understand that by doing an HTTP POST request to the URI http://localhost/my-user-service/user/100 what I want to do is update the user. Note the lack of some specific form URI or form handler that you might find in traditional web applications that operate at this level.
It is worth noting that while XML works as the means for representing your data in a machine-friendly way it is not the only way. Alternatives such as JSON, YAML, and GroovyMarkup work potentially as well. (There are many more that just these, and you can even write your own custom flat-file format if you’d like.) However I could say a couple things in favour of XML. You can use XML Schemas (XSDs) to ensure data integrity, and XML parsers are highly-available, making your service data friendlier to consumption. But you are certainly not limited to XML. The point is to make your data machine-friendly somehow.
The final piece of the puzzle is the client. Web browsers make it very easy for humans to browse the Human Web. Is there, or could there be, a similar tool for the Programmable Web? Unless the XML that is returned from every URI is standardized in some way, creating some new language in the process (Object-XML perhaps), there is no way to really do this. (SIDEBAR: I will tackle this issue more head-on in a future post and explore what Object-XML might look like.) Therefore, we could go one of two routes. We could publish a schema and let our consumers deal with the parsing bit, or provide our own custom client instead. While you could do both, I think it’s best to do the latter only.
Custom clients are the [short-term] answer to the web browser problem in the Programmable Web. The designer of the service is best capable of doing this work. The designer can also implement a client for his service in many different languages, giving his service true cross-platform operability. But the biggest positive of this approach is that it encapsulates the web’s architecture and data, much the same way a web browser does for the Human Web. As a consumer of the Human Web, you’re expected to have a web browser and know a URI. That’s it. Custom clients provide the same notion to the Programmable Web: consumers would be expected to have the client and, optionally, know the URI. (However, the URI could be hidden in the implementation because the client is specific to a particular URI set as I describe.) Using the custom client, consumers make what looks like a local function call that in its implementation reaches out over the web to get whatever it is you requested. It’s beautiful, really. Isn’t it?
Thus, in summary, to make your application a part of the Programmable Web just as the Human Web:
1. Adjust your API exposure to be more resource-oriented.
2. XML is analogous to HTML: Expose clean URIs that return XML representations of your data.
3. Custom clients are analogous to the Web Browser: Create custom clients that read your XML and translate it transparently to the object model of choice, in the appropriate computer language.
The benefits of this approach are tremendous. While I could write a whole new post about it, I’ll just quickly highlight three benefits. Coding to the Programmable Web provides quick cross-platform interoperability. You can, for example, utilize the Java platform for the back-end heavy lifting while you delegate the front-end capabilities of a web site to the Ruby platform, for which it is well-suited. You can also leverage the architecture of none other than the World Wide Web to design distributed applications, where key bits are done in different, focused computing environments, vastly improving the scalability of your system. Lastly, I’ll again reiterate the simplicity of it all. By providing a machine-friendly response and a client that can read it in, you have scalable cross platform web services, without the need for sticky, gooey fluff.
It’s a wonder to me this approach hasn’t been more popular to-date!
Conclusion
A “Tale of Two Webs” is a bit of a farce, really. There aren’t two webs, but rather at least two generic response types: human-readable (e.g., HTML) and machine-friendly (e.g., XML). The point I wanted to make in drawing attention to a notion of the Programmable Web is as follows: the machine-friendly stuff is surprisingly lacking. Yes, some companies on the Web are producing XHTML, a stricter form of HTML that can be machine friendly while being human-readable (through a web browser). However XHTML’s base (HTML) is still a tag library for marking up parts of a document (e.g., headings, subtitles, and paragraphs) and not the contents of the document itself (the data, e.g., planes, trains, and automobiles). While it is fashionable to “scrape” content (create complex parsers that read in HTML data and spit out machine-friendly versions of it) I submit to you there shouldn’t be the need at all for this, and making it easier will only enhance the quality of the Web and our applications at-large.
In my next instalment, I will go over a RESTful Seasons’ Greetings service. It will be a simple service that will return an optionally-customized greeting appropriate for the holidays. We’ll break down the parts, explain why they are meaningful as they are, and bring together this notion of the Human and Programmable Webs together through one web application. I’m a Java developer so I will be doing this in Java, but the concepts really are cross-platform. And I will be providing tarballs of the entire source, so that you can build it and deploy it on your own machine.
I’m personally very excited, and I hope you are, too. See you next time!
[1] RESTful Web Services, http://oreilly.com/catalog/9780596529260/
[2] While this is mostly true, it’s not 100% true because each browser implements portions of the specification, which may or may not behave the same way. Browsers also provide functionality outside of the specification as a way to drive users to their platform (as opposed to a competitor).
[3] By design it’s supposed to be safe, but an improper web browser implementation can open your computer to published and lesser-known dangers. Always practice safe browsing.
[4] W3C Web Services Activity, http://www.w3.org/2002/ws/
[5] RESTful Web Services, http://oreilly.com/catalog/9780596529260/