Termed as one of the brightest ideas in search customization, Yahoo Pipes, was launched sometime back in late ‘07. Indeed the promise was arresting. A novel and exponentially powerful mechanism to customize, aggregate, process the information on the web and possibly create a channel of communication or media that has all the cumulative powers of social networking, grid computing and parallel processing. It seems really powerful when you compare it to the mainstream information processing channels, in the fact that it creates a flexible environment with the compounding power of ‘customization’ to suit individual needs, i.e. common user’s needs (going by the claim of the program designers)! Also recently ‘pipes’ migrated to iPhone to add the power of ‘ubiquitous and distributed access’ to ‘automated data syndication’. Today there is a big buzz on ‘Cloud Computing’ and fundamentally it is little different than what Yahoo Pipes proposed.

Yahoo!’s new Pipes service is a milestone in the history of the internet. It’s a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant.” While it’s still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.

I want to talk about the implications for that marvelous aspect of the fundamental UNIX design: the pipe, and its ability to connect small independent programs so that they could collectively perform functions beyond the capability of any of them alone. What is the equivalent of the pipe in the age of the web? …This is one of the REALLY BIG IDEAS that is going to shape the next five or ten years of computing.
– O’Reilly Radar

Yahoo! Pipes goes beyond what just pipes are and what pipes do though as the application provides functions (or as they are called in the app - modules) that will perform a variety of different actions. There are modules available to prompt the user for input (a variety of input types), different operators to count, loop, cut, count, sort and merge data along with a variety of string and date functions. Because of this already broad base of available functions, Yahoo! Pipes is more akin to a shell scripting environment for the web rather than just a simple conduit between applications. It works like a visual procedural programming language with the output of the process dropping out at the bottom, in the form of text output, RSS, SMS alerts of even JSON. You can use feeds, user input or other pipes as input.
– TechCrunch

Heady…isn’t it. Now to top it all, it has a swanky user controlled API for managing and customizing these ‘Feed sources’ and programmable ‘Processing units’. You could manually ‘cut ‘n splice’ your feeds with all the ‘drag ‘n drop’ resources at your disposal! What an awesome proposition! Users or rather the ‘Information Consumers’ of today’s digital society would just love to use this…

But that’s where the biggest hole was. Even after the launch of the public beta, it still remained a ‘geek toy’. You would only hear of it in the conferences and biztalks. The application failed to seep into the everyday usage of millions of netizens. Some would argue that it was never publicized or marketed well. And that’s partly true!

What is more intriguing is what some of the users had to say about it:

Working with these damned gui workflow components is annoying.
Why not give us a restricted set language instead so I can actually do something useful?

Fascinating and quite well implemented. It isn’t for joe public yet but it should help the many programmers out there who aren’t RSS aware become RSS aware.

Wow, I read the entire article and all of the comments and I still don’t understand it. I guess I’m the ‘joe Public’ you are all talking about.

Tubes! It’s a series of tubes!

Well these were picked from the ‘comments thread’ selectively (there were many that said very good things about it). That almost explains why this post is being written! Yes, I am talking about those who are not nerd enough to go and put a rave review about how great a concept ‘Yahoo Pipes’ are, on some famed Tech Blogs.

Whatever the argument is, whether it is a good first step by Yahoo! or it was the start of a great era in ‘automated syndication engines’….I just wanted to drive one central point home. In the frenzy of creative pleasures, the technocrats create technologies without caring who they are building for. To say that ‘Pipes’ will revolutionize the way people access, process and consume information, it will obviously have to be adopted by a vast majority of people first. I especially like what one user said: “Why not give us a restricted set of language instead so I can actually do something useful?” Common users want to do things in a simple way, they want guidance (at least initially) until they realize the value of the offering and get on to the learning curve. Exposing the complex intricacies of data syndication and custom processing does not resonate well with how these user seek information, behave with an information artifact or utilize the information in their own context.
There is an important lesson in it for all future and present web application developers. Think for your Users!!!

For latest updates on pipe, please visit the ‘pipe blog’