Archive for RSS aggregator

RSS Syndication has long been a feature of blogs and websites, allowing publishers to share their work with a broad audience, and aggregate, arrange and publish that of others. Until now, though, syndicating and broadcasting online video and rich media hasn\’t been the simplest of prospects. All of that changed yesterday. Splashcast has already established itself as a simple way to mash-up and publish online video, audio, images, documents and even PowerPoint presentations directly from a web application, and to a simple, easy-to-navigate web widget. Yesterday, however, a major update to the service introduced the ability to create online playlists from any RSS-enclosed media on the web. What that means for independent publishers and the kids at MySpace alike is …

A new social media network with RSS aggregation capabilities has arrived on the scene, promising to provide a one-stop solution for the promotion of all of your online media. Whether you are a blogger, NewsMaster, podcaster or online video-maker a crucial part of your work is in drawing attention to your content. Shared RSS feeds, social bookmarking, social networks and social news are all avenues that you might have explored in promoting your work, and each have their merits in boosting traffic to your website. But what if you could promote all of your media using a combination of these approaches, from a single easy-to-use platform? Now you can. In what appears to be a unique remix of existing social …

myFeedz is a new service aimed at personalizing and adding a democratic, social dimension to the RSS feed reading experience. Here we have an interesting attempt to roll together the success of social news sites such as Digg, services that quickly learn your viewing preferences such as Stumble Upon and the familiar process of aggregating your online news into a single space using a feed reader, such as Google Reader. The result is a new breed of feed reader that delivers to you a personalized stream of news generated at the intersection of your own interests and other users\’ ratings, so that – theoretically at least – you get the cream of the latest news in your personal areas of …

Yahoo! Pipes, is essentially a very powerful RSS feed remixer, which goes well and beyond the original newsmastering concept I described a few years ago. Potentially, Yahoo! Pipes is a highly disruptive visual programming environment that puts in the hands of many people the ability to create web mashups and web-based applications that combine data from different sources with much greater ease and effectiveness.

Yahoo-pipes-edit-by-Robin-Good-465.jpg
Photo credit: Jack Parry

Put simply, Yahoo! Pipes is a way of visually manipulating data feeds from around the web, and mashing them together into new interactive creations.

Yahoo! Pipes lets you drag and drop different information feeds – for example, the latest news items from your favourite news sources, or search queries from the online shop of your choice – and combine them through a series of filters into so-called “pipes”.

The term “pipes” comes from the Unix operating system terminology and refers to the ability to connect sources of data to filters and utilities.

A pipe is a way of constructing ad-hoc workflows composed of any number of inputs, filters, and manipulation tools. And the beauty of the whole system is that they all use a very simple input and output method, so there’s a nearly infinite set of ways you can combine and recombine them.

Wikipedia says: “In Unix-like computer operating systems, a pipeline is the original software pipeline: a set of processes chained by their standard streams, so that the output of each process feeds directly as input of the next one. Filter programs are often used in this configuration. The concept was invented by Douglas McIlroy for Unix shells and it was named by analogy to a physical pipeline.

While Yahoo! describes its new service as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant” the real potential of Yahoo! Pipes sits all in the promise of “turning the web into a programmable environment for everyone.

Pipes examples currently in circulation include a New York Times Thru Flickr application that matches images from photo sharing website Flickr with news items from the New York Times, and an aggregated news alert pulling in the latest headlines from Yahoo, Google, MSN, Findory, Bloglines and Technorati.

In essence here is a service that promises to let everyday people (perhaps with a slightly geeky streak) grab different data from all over the web and manipulate it to do their bidding, simply by dragging and dropping elements around a visual interface.

But how does it work?

Who is likely to use it?

What do experts think about it?

In this mini-guide to Yahoo! Pipes, me and Michael Pick have summarized, organized and commented the key facts, issues, and thoughts that have emerged from the Web in the few hours that have passed since Yahoo! Pipes launch.

Read on to find out:

  • What it is – what exactly this tool is capable of

  • Usability – the latest feedback on how to go about using Yahoo Pipes!
  • Examples – how the tool has been put to use so far, and for what
  • Positive buzz – the positive feedback bouncing around the blogosphere about Yahoo! Pipes
  • Concerns and caveats – the potential issues users might face
  • Related tools – the rollcall of other similar services and tools out there already
  • Here, the details:

    yahoopipes.jpg
    Photo credit: Mateusz Zagorski

    What are Yahoo Pipes?

    Mashups are applications that merge data from different sources and bring them together to serve a new purpose. If you’ve ever run into an online map that displays photos of key sights, or clicked on a person’s name to make an instant Skype call, you might well have come into contact with mashups.

    The problem with mashups is that you need to be a code genius programmer to have a hope of making your own. This is where web widgets come in handy, given that they give you the chance to plug this mashup content right into your website as easily as you might embed a YouTube video. But still, as much as you can fiddle with widgets and bend what they do slightly to your needs, the level of control you have is limited to say the least.

    yahoo_pipes1.jpg
    Photo credit: Richard MacManus

    Enter Yahoo! Pipes, which, while certainly not as easy to get to grips with as widgets, certainly take the pain out of mashing up your online data sources. If this sounds a little geeky, the long and the short of it is this – it is. While I would trust my grandmother to throw together a widget or two, Yahoo! Pipes would probably present her with a few more problems.

    So what we have in Yahoo! Pipes is a way of programming without really programming. Instead of typing in code, you drag and drop tools and filters around a visual interface, filling in the details as you go. As Yahoo! Pipes is based on a social network paradigm, there are lots of prefabricated elements that you can grab from other users and change slightly to suit your needs.

    So while a lot of the more complicated mashups being put together with Yahoo! Pipes are likely to be from the geek fraternity, less tech-savvy users will be able to go in, copy and tweak the work already done for them by the caring, sharing members of the community. Let’s say that someone had already made a news aggregator that pulls in the latest headlines from generic sources like Yahoo News, CNN and the BBC. It wouldn’t be too difficult to go on in there, use this mashup as a starting point, and swap the news feeds for something more suited to your needs – Tech news sites, for instance.

    Usability

    Obviously the chief appeal of Yahoo Pipes! is in the fact that it takes coding out of the equation of creating mashups, and instead brings a user-friendly visual interface into play. But just how user friendly is it, and who is likely to be able to make use of the service in its early beta stage?

    pipes_edit_interface.jpg
    Photo credit: Brady Forrest

    Reports vary as to the ease of use of the service, with some suggesting that it is a piece of cake to get stuck in and working with mashups, others suggest that it is not quite ready for primetime, and is more likely to be of use to the tech-savvy. Brady Forrest notes that:

    The Pipes editor/creator is a really amazing piece of software. You are presented with a white, graphpaper-esque canvas to build your pipe on. A toolbox with a slew of potential modules is on the left, tabs for working with multiples pipes are across the top, and a debugger on the bottom. To build the Pipe you drag modules onto the canvas, enter the relevant data and connect them together via “wires”.

    Brady Forrest, O’Reilly Radar

    His enthusiasm is mirrored by Nik Cubrilovic, who emphasizes not only the interface design, but the speed and simplicity with which it can be put to use:

    The beauty of the application is with its simplicity – a user can take any sources, user input requests or the above mentioned module and drag+drop them into place and then connect the pipes. Within minutes I had built an application (also known as a pipe, they should probably change the name as not everything can be a pipe) that would search for ‘Techcrunch’ in a variety of feeds, bring that data together, sort it and filter it for unique results.

    Nik Cubrilovic – TechCrunch

    Richard MacManus is slightly more reserved about this ease-of-use, and while he seems enthusiastic about the service, brings in the first hint that it might not be entirely suitable for Joe Public to make use of:

    The UI seems a little geeky and kind of reminds me of Ning (not sure if that’s a compliment or not, as Ning never took off). But I’ve long thought that RSS remix feeds are the future of RSS – and certainly one way to try and filter information overload. So this is a great move by Yahoo to release an RSS remix service to the early adopter crowd.

    Richard MacManus, Read / Write Web

    This praise mixed with a gentle warning about Yahoo! Pipes’ complexity also finds its way into Anil Dash’s coverage of the service. The message seems clear – this is not going to have the same market penetration as MySpace or YouTube:

    Pipes combines a remarkably sophisticated development environment with some core social features such as the ability to clone or share the web services you produce. The service is fairly approachable, but somewhat complex once you get just under the surface, and should be moderately successful while radically raising the bar for other tools in its category.

    Anil Dash, Dashes.com

    Even Tim O’Reilly, who authored a veritable love sonnet to Yahoo! Pipes stresses this same point:

    It’s not quite as easy as drag and drop. I have to understand the query syntax of the sites I want to search, and modify the URL-builder modules to use that syntax rather than the syntax of the sites I’m replacing. But it’s relatively easy once you play around a bit.

    Tim O’Reilly, O’Reilly Radar

    In short then Yahoo! Pipes would seem to be something less than rocket science, but certainly taxing enough to make several tech luminaries issue caveats about its complexity and slight inaccessibility to the technologically uninitiated.

    Examples

    The following list includes some of the Yahoo! Pipes currently in circulation:

    • Apartment Near Something – allows you to input what you would like to be near (a mountain, a shopping mall), which city you would like to be in or close to, and how far you would be willing to travel, before spitting out geographical data to fit your query.

  • A Yahoo blog feed aggregator
  • A hot deal search that searches numerous shopping search engines to get you the best price on the item of your choice
  • Aggregated news feeds from the top online news sites
  • Browse the other Pipes available at Yahoo! Pipes
  • Positive buzz

    There will be more than a few champagne corks popped at Yahoo! HQ tomorrow, given that the response to Yahoo! Pipes has been resoundingly positive, and as widespread as any PR agent could dream of. Perhaps the least restrained, and glowing of the responses comes from Tim O’Reilly, who sees the service as a new step forward in the evolution of the web. He writes:

    Yahoo!’s new Pipes service is a milestone in the history of the internet. It’s a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant.” While it’s still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.

    Tim O’Reilly – O’Reilly Radar

    pipesaggregator.jpg
    Photo credit: Tim O’Reilly

    Pete Cashmore is less grandiose, but nevertheless sees the service as a step in the right direction. He notes that:

    Pipes is still a little geeky, admittedly, but it’s a great first step in creating a mashup tool for the masses.

    Pete Cashmore, Mashable

    Overall, however, the feedback on the service falls somewhere between these two examples, making Yahoo! Pipes’ reception an incredibly positive affair. It is easy to see how, while the service might not be a killer app in terms of market penetration, it is certainly leading the way and hinting at the shape of things to come for the evolving social, malleable web.

    Concerns and caveats

    While the mood is a largely positive one out in the tech blogosphere, a couple of concerns and caveats have risen from the jollity. Chief among them is the previously mentioned issue of usability:

    Now, while I say Pipes opens up mashup programming to the non-programmer, it’s not entirely for the faint of heart. At minimum, you need to be able to look at a URL line and parse out the parameters (so, for example, you can use Pipes’ “URL builder” module to construct input to a site’s query function), understand variables and loops, and so on. But you don’t really need to know these things to get started

    Tim O’Reilly, O’Reilly Radar

    yahoopipes-1.jpg
    Photo credit: Niall Kennedy

    The other possible issue raised is from a content publishers perspective. While consumers might welcome the ability to weed out advertising content from blog posts, for instance, this could potentially threaten the livelihood of those pro-bloggers that rely on this important revenue stream:

    Yahoo! Pipes makes it easy to remove advertising from feeds or otherwise reformat your content. I already know a few publishers who hold back the publishing the full content of their posts for fear of easy resyndication and brand dilution, and if Pipes becomes popular publishers might hold back a bit further or ban Yahoo! Pipes outright. A Yahoo! Mail user searching for a new feed subscription will likely choose an identical feed labeled “No Ads!!!” associated with their favorite brands.

    Niall Kennedy – Niell Kennedy.com

    On the one hand the usability issue threatens the extent to which Yahoo! Pipes will be taken up by everyday web users, and on the other, its powerful ability to filter and refine the content that passes through it has content producers a little concerned about how it might facilitate both ad-dodging and unchecked wholesale copying of content by unscrupulous sploggers.

    Related tools

    Several predecessors and alternatives to Yahoo! Pipes have been discussed in it’s ongoing coverage. Anil Dash suggests:

    Plagger: an open-source, installable feed routing system created by Tatsuhiko Miyagawa which performs much of the core functionality of Pipes and is customizable, but lacks the user interface and integrated development environment (IDE) which distinguish Pipes.

    Ning: Perhaps the archetypal social application platform for the web. Headed by Gina Bianchini, Ning has thus far defined the feature set for end-user creation of web applications, though the focus has not been on creating web services.

    Jamie Pitts suggests that Yahoo! Pipes leans heavily on the interface design of Apple Quartz Composer and Propellerhead’s Reason applications.

    Ivan Pope throws a recommendation in the direction of Dappit, a data mapping web app that serves a similar function to Yahoo! Pipes.

    Conclusions

    yahoopipesshot-1.jpg
    Photo credit: Pete Cashmore

    Pipes is a hosted service that lets you remix feeds and create new data mashups in a visual programming environment. The name of the service pays tribute to Unix pipes, which let programmers do astonishingly clever things by making it easy to chain simple utilities together on the command line.”

    Here is a tool with the potential to change the way non-programmers interact with data on the web, allowing for the relatively easy mixing, matching and filtering of data streams into new information services and products which we have started to invent only in recent times.

    Nevertheless, this is not yet, a tool quite ready to be unleashed on the general public.

    While Web widgets bring mash ups to the masses, anyone that wants to get really stuck into the possibilities offered up by Yahoo! Pipes is going to have to know at least a little bit about the way web applications and protocols work behind the scenes.

    Now, while I say Pipes opens up mashup programming to the non-programmer, it’s not entirely for the faint of heart. At minimum, you need to be able to look at a URL line and parse out the parameters (so, for example, you can use Pipes’ “URL builder” module to construct input to a site’s query function), understand variables and loops, and so on. But you don’t really need to know these things to get started.

    What’s really lovely about this is that, like the Unix shell, Pipes provides a gradual introduction to web programming. You start out by modifying someone else’s pipe just a bit, then branch out into something more adventurous.
    (Source: Tim O’Reilly)

    That said, given that the site is emphasizing its social network aspect, and that many users are already freely sharing their mashups, it should not prove too difficult to modify and tweak the groundwork laid by others in remixing RSS feeds for your own needs and interests. What that means is that nonetheless some technical prowess is required to make the best out of Yahoo! Pipes, by simply cloning and refining on the Pipes work done by others, many more will have the opportunity to more easily learn and familiarize themselves with “piping”.

    Yahoo! Pipes is the first tangible tool that will allow many of you to customize, re-arrange and engineer new data views of the web as well as compelling new services that mix complementary and isolated data components available out there.

    Yahoo! Pipes is indeed, at least in historical terms, a milestone technology for the Web. It opens up opportunities that are orders of magnitude larger than what is typically possible today and it gives also to the less-technically equipped the means to mashup and remix content and information sources in ways and fashions not possible until now.

    Additional resources

    If you are hungry to learn more about Yahoo! Pipes, you might want to check out the following websites:

  • Tim O’Reilly’s resounding praise for Yahoo! Pipes
  • Nik Cubrilovic’s Tech Crunch feature on the service
  • Niall Kennedy’s coverage of Pipes
  • Pete Cashmore’s Mashable post on Pipes
  • Richard MacManus’ Read/Write Web coverage of Pipes
  • Anil Dash’s review of the service
  • Brady Forrrest’s Modules for building pipes and Deconstructing a pipe
    tutorials for the service, over at O’Reilly Radar
  • Rok Hrastnik marketing insight into Yahoo! Pipes

  • Beyond NewsMastering: Yahoo! Pipes Is The Internet RSS Remixer – Overview And Reports – Originally published by Robin Good on MasterNewMedia.org

    For corporate newsmasters OPML reading lists are an excellent way of gathering and custom-distributing thematic collections of RSS Feeds within the organization.

    By being able to effectively aggregate, organize, label, and provide selected access to thematic RSS feeds collections, corporate digital information librarians are now given true power to provide highly customized and tailored information feeds to their key stakeholders.

    Not only.

    Such capabilities provide also the means to create multiple definitive source of RSS feeds on a given topic, be it the latest communication software or just a hand picked selection on the best news sources out there. Such features support the use of a RSS feed library also and well beyond the walls of internal communications, research and intelligence to the ability to serve and distribute high-value content feeds collection on any designated theme to the public at large or to paying audiences.

    What RSS does for websites, OPML does for RSS feeds. Which is to say that it gives end-users the capacity to subscribe, with a single click, to constantly updated information sources and access them from a single place, such as an RSS aggregator.

    Increasingly, information professionals across a number of industries and sectors are making use of RSS feeds to pass along the latest information to their clients and end-users. These curators of information can make use of OPML to quickly and effectively import and export vast selections of feeds, organized into headings and subheadings, making the task of sharing a much less daunting one.

    Blogbridge, whose powerful RSS reader / aggregator runs on Mac, Windows and Linux, has developed a solution for these information professionals: The Blogbridge Feed Library.

    Here the details:

    The Blogbridge Library is a powerful but surprisingly intuitive way to create vast, well organized libraries of feeds for a range of applications. This simple, easy-to-use platform could be of great use to librarians and PR professionals alike, not to mention those working in independent publishing and the blogosphere.

    In this video review I walk you through the interface, along with the simple process of putting together your own library of RSS feeds using OPML reading lists.

    BlogBridge Library – Video Review

    In this video review I take a look at how you can:

    • Find your way around Blogbridge Library’s main interface

  • Export and Import OPML reading lists to and from your library to add and share content
  • Search your library through a range of criteria, or via a tag cloud
  • Customize your library in terms of its users and the micro-organizations into which you can group them
  • BlogBridge Library – Purpose

    As more and more organizations turn to RSS as a great way to gather up-to-the-minute, sometimes mission critical data from around the web there is an increasing need for a simple means to organize these feeds.

    As it is possible to create a feed from any website, it has never been more important to find a way to aggregate, organize and recommend reliable sources of this information to clients and end-users.

    BlogBridge Library’s main function is to provide and efficient way of going about this very task.

    As BlogBridge’s own site describes it, BlogBridge Library creates a:

    flexible web based structure to showcase Feeds, Reading Lists and Podcasts to employees in your company, or members of your organization.

    It will be the ‘store‘ where users can browse and search for recommendations of content to read with their Aggregators. And, here’s the important point: these are recommendations by people in your organization for people in your organization.

    Effectively, BlogBridge Library brings Robin Good’s concept of the NewsMaster to within the walls of your own organization. While the role of the NewsMaster on the web is to hand-pick the best information for a specific target audience, using BlogBridge Library, librarians or information professionals can specifically target vast lists of recommended sources of data entirely geared to the people that they work with.

    The Interface

    foldersinterface.jpg

    The BlogBridge Library interface uses an intuitive, instantly recognizable folder-based format to display information. Anyone familiar with Windows, Mac OS or the various Linux GUIs will not find navigation an issue.

    This familiar format, with folders nested within folders, is enhanced by the library paradigm, so that working through or putting together an OPML reading list is rather like taking a look at books on a shelf, which can be pulled out and explored in more depth at your leisure.

    The home page arranges categorized reading lists in rows of four folders, with four tabs that allow you to easily explore a further four rows, meaning that a given category can contain sixteen lists. These lists can in turn contain further lists, so that the only limit of the amount of feeds contained in your library is that of the license you have subscribed to.

    By being able to quickly scan through entire collections using thumbnails and a rich graphical interface makes the exploration of OPML lists seem a lot less technical, and will be sure to make end-users comfortable regardless of their previous experience of working with RSS news feeds.

    It looks quite evident that a lot of time has gone into making the interface as accessible and instantly recognizable as possible, as to attempt to ground its potential success in its very simplicity and accessibility.

    From the very top level BlogBridge Library folders, it is possible to explore the RSS feeds within them, right down to the ability to preview individual posts within those feeds.

    There is no need to actually subscribe to a feed until you have given it thorough test within BlogBridge Library. In fact, it would be quite possible to use BlogBridge Library as your sole source of RSS content review, and this will appeal to those end-users that just want a quick way to get to the news you have gathered for them.

    Powerful Search

    library search.jpg

    End-users will also welcome the powerful array of search options available to them, facilitating content discovery even within such a large array of content sources and supporting information professionals in their quest for finding the best sources on a specific topic /theme. In the BlogBridge Feed Library it is possible to search by the titles of RSS feeds, by folder or user names, which helps you narrow down and pinpoint search results considerably.

    Furthermore, search results can be limited to title, description, tags, URLs or a combination of the above, so that users can expect to bring back only the information relevant to them at any given time. While this is a simple function, it is an essential one to users attempting to navigate any sizable collection of reading lists, and serves as an excellent complement to the already strong visual navigation methods available.

    The BlogBridge Feed Library provides an effective tag cloud to allow those wanting to wade through information in a more visual fashion. As all of your RSS feeds can be tagged at the point of creation, this makes it very easy to create targeted and well-organized information quickly and efficiently, aiding the easy location of key data at later times.

    Organizing Users and Administrative Functions

    organizations.jpg

    The administrator access facilities in the BlogBridge Feed Library, allow you to easily organize other users acces rights to the library. The library administrator can in fact add new users, assign them to different “organizations“, implement access restrictions, security, integrate images, biographical data and more.

    The ability to create “organizations” within the BlogBridge Feed Library is a powerful aid when in need to create a series of interconnected RSS feed libraries for a number of clients or departments within your workplace.

    Each “organization” can be customized as to which RSS feeds it will be given access to, and this makes the BlogBridge Feed Library very flexible for those looking to work with a large group of end-users.

    Furthermore, individuals can be assigned to manage specific areas of the feed library, so that the expertise spread throughout your organization can be fed right back into the reading lists gathered in your collection.

    In this fashion, you can also have your video production and new research department set itself up to automatically collect the best RSS feeds available from online news video sources, your PR department do the same for the most relevant important PR sources in their sector, R&D collect its unique mix of research and development feeds and the organization can start to manage the incoming information flow with some true organization and intelligence.

    By allowing customization of data at a user and group level, the BlogBridge Feed Library avoids a one-size-fits-all approach to information architecture and allows for a number of flexible functions to be achieved from a single account or installation.

    Conclusion

    The BlogBridge Feed Library is an easy to use, powerful tool for anyone looking to quickly and easily gather and share a vast amount of OPML reading lists.

    Primarily, I can see it being of interest to information professionals such as librarians, but also to organizations that rely heavily upon the monitoring of closely targeted, constantly updated information on specific themes and topics from across the web.

    This second group is a vast one, and might range from pro-bloggers right up to PR, R6D, competitive intelligence, business analysis and marketing departments with a need to keep abreast of the latest changes in their field.

    The BlogBridge Library does a very effective job of demystifying and simplifying the process of browsing and organizing feeds through the use of its clear, familiar and primarily visual means of navigation. It manages to make accessible what might otherwise be considered the territory of geeks and IT administrators, and that is in and of itself a major achievement.

    Leveraging in full the potential of OPML and RSS, the Blogbridge Library provides excellent interoperability allowing the ability to import and export data in both of these standards-based formats.

    I personally found Blogbridge Library to be a pleasure to use, and didn’t encounter any noticeable bugs or issues in setting up a simple library, or in exploring the RSS feeds arranged in Blogbridge’s own Expert Guides. As such, I highly recommend this easy-to-use, thoughtfully designed RSS content aggregation and delivery platform.

    Specifications and Pricing

    The Blogbridge Library is entirely browser-based and platform independent, which is to say that you can use it regardless of which computer or operating system you are accessing it from.

    There are two versions of the Blogbridge Library – one that is free to use, and installed on the end-users own server, and another that is hosted at Blogbridge’s own servers, and which comes in a number of different account sizes all offered at what I would consider very reasonable prices.

    pricing.jpg

    As you can see, the main differences are in:

    • The number of librarian, administrative accounts made available to the user, ranging from 5 to unlimited.

  • The amount of individual OPML feeds allowed, ranging from 100 to unlimited
  • The ability to export OPML reading lists from the library (which is not included in the Entry-Level account due to the extra load on the Blogbridge server). All accounts can obviously import and export OPML reading lists.
  • The pricing, which ranges from free for Non Profits, NGOs and academics, up to $149 / month for those with an effectively unlimited Production account
  • Additional resources

  • Take a Visual Tour of the service
  • Explore full details of licensing and pricing
  • Download the Feed Library application