Social network integration the hard way

There is a jungle of social and blogging services out there, and it's hard to pick one to use and impossible to use them all. Some people asked me how I deal with it and the short answer is I use several of them and tie them together using RSS feeds and APIs. Now for the long answer with pretty graphics.

Main goals were:

1. Keep using the preferred service for each task: Picasaweb for pictures, Vimeo for videos, Blogger for blogging etc.
2. Simplify subscription down to 0 clicks; support e-mail as alternative to feeds for technophobes.
3. Merge data streams down to two, personal and work. Keep it simple, but don't mix the unmixable.
4. Reach people where they are, don't invite them to use services they don't want to use. Have data, will travel.
5. No manual intervention.
6. To the extent possible, use own domain names: companies come and go, names are here to stay.

The solution's broad lines are:

1. Extract feeds from everywhere I create content, by whatever means necessary.
2. Create two Posterous blogs and import everything there nightly using the API.
3. From there export things where they will be read using posterous own extensive export features.

This is a graph of how things are plumbed, first the so called serious stuff:

And then the recreational or personal stuff:

Now for the gory details. Why Posterous as the central hub? It has some killer features such as: 

1. Create as many blogs as you want, in this case two. No need to have grandma check out the latest on hadoop, or the uber-geek be briefed on baby's latest trick. 
2. 0-click subscription: if grandma doesn't do feeds and doesn't do subscriptions but does email, you can force subscribe her via email, a few clicks for you but 0 for her. It's a little invasive but the only thing that worked with some technophobe friends. Use with care.
3. Export to XYZ. Posterous supports a lot of different targets for content delivery — called autopost. This way you don't need to convert everybody to Posterous, which will never happen, you can just reach them where they are. I know, everyone is on Facebook. But most people I talked to have ended up befriending everyone, they are overwhelmed by the newsfeed and again I didn't want to mix geek content with baby pictures. One wrinkle: can't get the export to Flickr to work. I've been on and off with support, but no good news so far.
4. In the opposite direction, they also have a much trumpeted import feature, but it is one-time only, for migrating your pre-existing blog, whereas export is ongoing, from when you set it up until you stop it. Why this asymmetry, I don't know — it looks like a lock-in play. Another "post everywhere" type service is ping.fm. Friendfeed, soup.io and also Tumblr, to a degree, really push in the opposite direction: "give me all your content, we'll take good care of it". They have ongoing import and little or no export. If someone gets both right, they might be onto something. Also, Posterous doesn't support importing from a generic RSS feed, only specific services of their choosing — so much for supporting standards. Anyway, Posterous has a read/write API and I am a computer person, so I wrote my own import from RSS and ATOM. You are welcome to try it out. I decided to import short form or abridged versions of most feeds to avoid copyright and rendering issues.  
5. Custom domains. I can switch to a different service with less pain since I own the domain names. You never know how long a startup is going to last — hopefully Posterous will be around for a long time, but one never knows.

Instead of straight sharing from Google reader I use two tags that are marked as public, which gives them a public web page with feeds. This is because I want to keep two separate streams, one for grandma and one for the uber-geek, and that can be accomplished with a tag for each. I wish I used this more often, I will try.

The feed from a Google bookmark list is the only unimplemented part of this plan: neither it is a feature provided by Google, nor several feed generators I tried have managed to generate a feed from that page. Feed generators are web services that check a page periodically for changes and extract what's new. I use page2rss with some success and Google Reader used to have a built in one, now retired. Neither those two nor others I tested worked for Google Bookmarks. No updates were ever generated. I thought I had cracked the problem, since that page is Ajax-heavy but has a static version for crawlers and other robots. Switching to that version apparently didn't help the generators. My testing was not extensive, I've got other business to attend, but if anyone has a solution please share. Yes I could switch to delicious but the integration of bookmarks and search is, well, too delicious to give up. Come on Google, every web page should have its own feed already.

Finally you might ask about that intermediate step through a Friendfeed group I have set up. This is so for historical reasons, since Friendfeed groups were my hubs of choice before the switch to Posterous. I run into some issues pulling the Pandora and Netflix feeds myself, so I left it to Friendfeed to do that and pulled the latter instead.

Is this network of data streams the nirvana of social interoperability? Of course not. First, it's way too hard to set up. Only a stubborn geek can pull this one off. Second, the feeds are incomplete and unidirectional. Comments may be lost in the process, and certainly any conversation that develops can  be fragmented. Like when a friend sees a new collection of pictures in my Posterous and follows the link to Picasaweb and enters a comment there — if he has an account. That comment will be only there, neither visible on Posterous nor any other of my social streams. If for some reason that picture is particularly good, my friends on Facebook will never know. It is for social web companies to decide that interoperability is in their best interest but if they wait a bit longer, it might just be too late.