This article covers the various ways that common web related tasks can be automated, and how the evolution of technologies for sharing information are changing the way people use the world wide web. It covers:
- RSS and XML streams
- APIs and online services
- Emerging task automation technologies
The reader should take away an understanding of how these technologies can enable them to share information more effectively to reach a larger audience, as well as an appreciation of the power of the web as an information sharing environment.
They may also be lucky enough to experience a "light bulb moment" when they realize that the sum of the parts on offer is far greater than expected!
RSS and XML are Driving Web Mashups and Automation
The XML format enables the structured exchange of data in a very flexible way. It is flexible to the point that different services can structure the information in a proprietary way, whilst providing a data stream that can be validated and used by third parties.
One of the first service types to use XML online was RSS (Really Simple Syndication), which allows publishers to provide a feed of items in a structured fashion such that feed consumers can use that information in a coherent manner.
As an example, Suite101 provides RSS feeds of recent articles, which can be used to provide a continuously updated list of articles on a Squidoo Lens. Other information publishing services, from Yahoo! to Twitter, also provide feeds in XML or RSS that can be consumed on third party sites.
For those that don’t, there’s Feed43.com - a service which can scrape a web page, and provide a user-structured feed that can be shared. When the Feed43.com site is accessed, using the specific identification number for the feed, it will provide an XML/RSS stream containing the information that it finds (based on pattern matching on the target page.)
Introducing Yahoo Pipes
Pipes are a way of combining multiple feeds in innovative ways. When a Pipe is run, it consumes feeds, and based on the operations that the user has defined (look-ups, web searches, feed splicing, etc.) produces a single feed that can be passed to a third party, and used in the same way as any other feed.
So, for example, a Pipe could consume a Twitter feed, use it to derive keywords based on tags, lookup Amazon books with those keywords, and produce a feed of books that could be posted to a web site. Each time the Pipe is run, the list is refreshed, providing an ever-changing list of Tweet-contextual books.
Pipes are, however, run on demand. If there’s no human interaction, they’re not run, which limits their use to fairly specific situations.
Ifttt – the Latest Technology to Automate Web Tasks
The Ifttt service (if this then that) takes the concept of automation via feeds to a new level. Unlike other run on demand services, Ifttt runs as a daemon. In other words, every 15 minutes, the user-constructed task is executed, and the results produced.
On top of this, the results can be directed to a Twitter account, sent via email, or SMS text message.
So, a user could choose to receive an email every time their favorite feed is updated, or create a task that Tweets every photo they post to Flickr, or blogs every time they send a Tweet with a specific tag.
They could even use a Yahoo Pipe that consumes a feed built via Feed43.com, and triggers an action via Ifttt – automatically, every 15 minutes, without any other interaction whatsoever.
Clearly, this tool, still in its infancy, and still in a invitation-only open beta, has the potential to change the way we think about repetitive research tasks on the web.
What ties all these possibilities together is something called an API, and almost every online service has one – and they are usually accessed via HTTP (a simple web request) and can produce a feed of structured information that can be processed with a Pipe, via Ifttt, or directly by the web programmer.
In fact, most of the points discussed here would not be possible without two vital components that underpin the world wide web – XML and APIs.
Guy Lecky-Thompson - Guy W. Lecky-Thompson is the author of several technical and non-technical books, and writer at large. He has written for Dr. Dobbs ...
Join the Conversation