Friday
Jan022009
A River of TWiT
Friday, January 2, 2009 at 8:48AM 
TWiT started as two podcasts way back when in 2005. Today it's a dozen shows, a live, nearly 24x7 video stream, a 24x7 audio stream, two very active chatrooms, a microblog for listeners, and a forum for contributors. In short, it's a large and growing community of tech enthusiasts built around downloadable and streaming audio and video content.
My experience with Twitter, the TWiT Army, the TWiT chatrooms, and the real-time feeds at Friendfeed, plus my extensive conversations with Steve Gillmor and the other participants at Bearhug Camp, have inspired me to attempt one more live feed: a live text stream of links, comments, and notes related to the live programming. The chatrooms offer something similar, but they're more conversation focused. I see this stream more like the live-blogging that occurs during a Steve Jobs keynote, or the tweets surrounding a major news event, like the recent election. Think of it as real-time show notes created by me, our show hosts, and our community. Let's call this stream the "river." This is an experiment, but I think it could be very useful, both informative and entertaining.
bear has set up a Jabber server for us at twit.im. That's the engine that would power this river. Now it's time to think about how we'll implement the user interface, both for readers and contributors. I'd like to make it easy enough to use that readers naturally become contributors.
Aside from installing a server and buying the domain twit.im, we haven't done anything else. So the following is purely speculative. I'd like to get your comments on it before we begin implementation. Consider the following an RFC.
The best interface to this river would, I imagine, be an IM client, but as with any stream there might be other ways to view it, in a Friendfeed room, on a dedicated web site, as a ticker on live.twit.tv, etc. I think these instances can be created by the TWiT community as needed over time. And since the river is just an XMPP stream it should be very simple - the API already exists. I would also like to have a real-time RSS feed of the full-river - that should make it pretty easy to write viewing tools. But initially, any Jabber-compatible IM client, GTalk, Trillian, Pidgin, iChat, or Adium, would work fine.
People post to the river by adding river@twit.im to their IM client. They'll also get the river content fed back on that channel. I think we'll encourage the use of hashtags so posts can be categorized: #link, #note, #location, #wisecrack, and the like. We might even want to require that every post be tagged to make it easier to filter the river. Some folks, for instance, might only want links, others might just want wisecracks, and so on.
More importantly, we'd also need some commands.
FOLLOW/UNFOLLOW/FOLLOWING - so users can control whose posts they see
BLOCK/UNBLOCK/BLOCKING - to prevent spamming, by handle and IP address I think
TRACK/UNTRACK/TRACKING - to allow you to watch for particular content on the river
Are there any other commands we need? Ultimately a search of some kind will be important, but we can get this for free by piping the RSS of the river into a Friend Feed room (as an example).
By implementing FOLLOW, BLOCK, and TRACK we can open up the server to anyone who wants to participate, since users will be able to precisely control the content of their feed, just as they currently do with Twitter (only better).
I would also like to be able to create special-use instances of the river. For example, at my Macworld keynote, I'd like to be able to create a one-time use backchannel, say macworld@twit.im, that attendees could post to and follow, and that, perhaps, I could put up on screen.
At this time we don't have any plans to let users register for twit.im accounts. I don't want to bog the server down with additional duties. It's going to be busy enough as it is. But if you have an existing GTalk or Jabber account you'll be able to use that.
What do you think? Have I missed anything? Is there any functionality you'd like to add? Please add your comments below. Thanks!
Reader Comments (48)
XMPP automatically time stamps every message. Done!
Some issues have already been discussed by others but here are some of my concerns/thoughts:
1) Ability to view stream at a later time when listening to podcast, for ex. Having this avail via archive, even if then one cannot easily filter it, could be very helpful to see those links mentioned...kinda like SN's transcripts...this would thus need timestamps as others have said.
2) How would we view the stream? I have IRC chat and Army open in tabs in Opera, and video in IE window over that...having yet another app open that I'd need to switch to on this small laptop isn't practical, especially with my eyesight issues...I use the http://techguy.startan.net" rel="nofollow">techguy.startan.net to view video as I can make that window almost as small as the video itself and thus see it and IRC at same time...if the stream were incorporated in that site as a scroll above the video, then it might work.
3) Tags might need to be kept short to make usage easier, such as #loc vs #location, etc.
4) Having only certain ppl/ID's authorized to post is a must, and being able to select which ones you see could be handy as well.
5) once an item gets "posted", there may well be a need to be able to remove it later, such as when a typo or incorrect data gets into stream...we'd not want a typo on link that landed us on a incorrect/naughty site to remain, right?
That's it for now...
Agreed. I think this idea is extremely exciting and thought-provoking, but in fact almost all of my interaction with Twit is through offline podcast listening on a portable player (either in my car or in my pocket while gardening, working out, etc.). I think the video feed is very neat but rarely watch it. I do listen to the audio feed from time to time either at work or while working around the house (courtesy of a Squeezebox).
Bottom line: any live content I see as a value-add, not the essential service I'm using with TWIT. With that said, if there was the ability to play back the video or to otherwise consume live streams (video, audio, text or ALL) TIME-SHIFTED, I think that would be very neat. I would only use it rarely, but I hope that the end result of this is a page that would essentially let you play back all of the live components (ex. the ultimate would be a web page for an episode of the show that showed you the video, chat and text streams as they happened during the recording).
Hope my comment wasn't taken the wrong way. More addressing the "slipping back into the mold of traditional broadcast media". I guess what I meant was that even though it's "free" content, you would need to find ways to tweek and moderate it so it's not a distraction to the main show. I fully trust that you always keep the core audience in mind as that's why you try all of these things to make the experience that much better. And face it, we all want to try new stuff like this. That's why we listen to your shows.
Hello Leo - sounds like you're on to something. Security and data ownership is always a concern. Using something like OpenID for access might keep some of the spammers out (yeah right). Another question would be who "owns" the information posted and what can they do with it?
Dave
Good points, especially regarding have to have another application installed. I use Twhirl off and on but most off since I does not show prior posts. The only IM client I have is google chat which means I would have to have IE open for the video, mirc running and either gmail open in IE or Firefox window. I rarely use the # sign in the army as it is since I have shorten my posts.
I like the idea of the main interface being irc like. Then you can have bots control the chanel and push the content to twitter and the other outlets.
There are a number of web based Irc clients for those who don't want a full client installed. And the bot can handle #follow commands etc. As well as filter conternt before it's pushed to twitter and the like.
I agree with you. I think an IRC interface, maybe a moderated channel for some of the shows, (only voices can post) and open channels otherwise, and bots to grab and post, even with an XMPP interface would work. I also think it would be nice to have several channels, as the population grown on irc now, during some show's it's impossible to follow the conversations because of the number of them going on, so have one for gen chat, have one for show specific chats, etc.
I think the idea is for general users to supply info to "mods" who will then post the info to stream, thus keeping junk/spam under some control. A link to another site that is validated by someone like Darth_Emma would go thru, but a random submit by Listener-xxx wouldn't go direct to stream, ok?
The whole idea is to have a live stream of trusted users creating this river in real time during the recording of the netcast. Later when you're listening you can then pull up the river for that specific netcast and have that real-time list of links and comments (from trusted sources so it's not noisy) and follow along as if it was in real-time.
I hope that made sense,
Chris Heath
I have thought a lot about this and a few points I think are worth noting first....
Your analogy of the firehose is very appropriate - huge volumes at high speed. But not many of us can properly drink from a firehose...in fact it would overwhelm us and we would look elsewhere for our drink...
So we need to think about how the firehose is paired down into digestible intelligent feeds and how that is best presented and controlled
Second, you have an extremely valuable brand that must be protected from abuse either inadvertently or via deliverate action. To that end I think you need to be a little more conservative than Steve Gillmour or Robert Scoble as your public brand/personna is much broader and directly related to ongoing advertising revenue. Thus the feeds derived from the firehose need to have some level of control ...possibly more than what the purists would like but hey...it is not their brand we are dealing with. You need to ensure that the two major TWIT channels of information, the show notes and the chat forum are tightly controlled for appropriateness as they will be the major carriers of your brand.
Third, we need to just recognise that XMPP is just a real time pipe or trnasport mechanism and the real value to be added here is how best to slice and dice and present the streams.
So....I am thinking that there should be a number of streams or channels that are defined by you and sent out as standard flows from twit.im
These could include:
- A raw feed of everything...for people to innovate off
- A chat/comment stream...a la TWIT Army
- A TWIT controlled show notes/links stream. This can also carry ads in sync with the ads within your shows...this is the main feed you show on your video stream and also the one you most publicise in your forums such as the Radio Show and any commercial television deals that you do....
Next I am thinking that a client like TWEETDECK would be a good paradigm for how these feeds could be consumed...one pane for the chat stream, one pane for the show notes/links stream. It would be great to have an AIR client that had the two main streams running beside a window with the video feed in it
On the video feed you could have also have a permanent side panel on the feed (inserted via the tricaster??) that shows the notes/links feed and/or the chat stream flowing past...this would be useful when you distribute your shows to commercial networks as it will be like a CNBC feed with the ticker.
I have so much more to say here but my experience as a designer of trading systems in capital markets is that my clients are always dealing with a firehose of data and the art is not to just present the firehose but to think about how best to present and manage the firehose to ensure easy access and to also enable use by not drowing the consumer in the volume....
Perhaps you can schedule a GoToMeeting session so that we can all join in and whiteboard some ideas .....that is an great way to moderate a large meeting and enable people to present ideas and discuss options
Anyway...just some thoughts
I like IRC style, I like the idea of a Pirillo-esque scrolling feed/river, but I don't like standalone IM client. As information will be coming pretty quickly, I like the idea of being able to pull individual feeds out of the "River" into my own "Pond" if you will, where I can look at my handpicked links, comments if i can't keep up with them. I know that the RSS piped into friend feed will provide all this for archival purpose, but I like being able to choose what I like on the fly, so that once the podcast is done I can save it for if I want to download the podcast later and look at my personal stream while listening. This might be beyond the scope of what you are trying to accomplish, but just my musings :)
Adding value to your content...Crowd Sourcing and Interactive Ranking:
I watch the shows pre-recorded. I assume the video will be on-demand sometime soon. Hash tags will be great to filter, but here is the feature I would think would be extremely helpful, though technically tricky possibly.
Each tag or comment should have an approximate time-code that matches up with the live video stream.
Each comment as an "object" should be right mouse clickable as it streams across
Right mouse click would expose a small menu: Example: 1) Funny, 2)Helpful, 3)Abusive etc
A server would collect and rank the top Funny, top Helpful, and etc
Question: Can Stickam pass script calls like you could in Video Flash?
If I had a Client that could replay Video on Demand and add the Funniest, or the most Helpful that would be extremely helpful. Also, You might insert a "Best of" feed post broadcast later into the actual video as a CNN type crawl. (See Obama the Magic N***gro feed on Fox for why not to insert it live)
I think this would make the video extremely valuable as tool and resource. It would leverage crowd sourcing aspects and add tremendous value to pre-recorded feeds.
Also it would give the crowd generated content a ranking, give lurkers and quiet listeners an easy task and allow less witty, or less knowledgeable viewers a way to participate.
And it would give Twit Members a way to see the best of the TwitCrowd. A Blog or Site would list these rankings. A simple script might someday be able call up that segment when the comment was made. By restricting the voting ranking to live sessions only, you could control demand and spamming.
Its been years since I engineered interactive TV, but I recall flash video enables this and I think this should be possible yes?
If you are going to offer an archived stream there needs to be a way to keep links from being broken or the link content from being compromised. Imagine what would happen if a link was suddenly pointing to malware. Yuck! How about doing some sort of tinyurl like link library that would be under TWIT control for later editing or redirecting.
That sounds very cool Leo. I would love to participate in a live IM stream of content. Just have it work with Adium or iChat and I am in.
Conceptually this sounds like a 'stream of consciousness' (or in this case 'river of consciousness') for the net. Indexing would be crucial, but the idea is fabulous.
earlier thoughts that may be applicable:
some applications i would like to see with this river, in a way, correspond to my thoughts a decade ago about incorporating a feed that streams directly to an led ticker. back then, perl or python seemed like it was doable -yet very restrictive. i've searched all over the internet and ebay but either the components of a finished product aren't there or you have to deal with an expensive product relegated to talking with a salesperson in addition to figuring out what you had to deal with -ie. serial port/usb port/comm port/ethernet and if you had the necessary drivers to do it. lots of guesswork and gamble for me at the time.
other thoughts:
i think it would be great to have an adobe air application as well as an iphone application.
in all these implementations, i agree that having it versatile in accessibility is key. as long as there is a way for any device to pull data from the internet would make it really great.
and as you have stated, refining and tailoring the data for each user in realtime would help make it very viable.
Very interesting and a great idea...
Is there any way to get that feed into the tricaster? If the Tricaster could act on what sort of content it was seeing from the feed and put up the appropriate CG up, and possibly link to that story or ad page which is then clickable (we're streaming in Flash so IT SHOULD be relativly easy but then again, I have no experience with the Tricaster).
So if in the XMPP feed (for shownotes), we might have an audible ad for http://www.audible.com/twit" rel="nofollow">http://www.audible.com/twit, the tricaster would then see this as an ad and put the relavent audible CG up. If the content was a link to a story or a website you're discussing, it would put up a CG that is predefined (let's say the general TWiT one) with a clickable link to that story.
I think it be also useful to distribute the river as a subtitle file when the shows are posted, instead of have to find the relevant links and stuff in friend feed. That way also the feed will be preserved along with the show.
(Sorry for being off topic here)
Leo, Regarding your comment on TWiT #197, where you mention doing real time reactions to an Apple Event with silhouettes of you and your panelists a la Mystery Science Theater 3000.
This sounds very interesting! What if you did record an audio version of this which could be listened to with the video once Apple releases the Keynote to the podcast feed?
This would be a really fun idea!
Leo,
Here's a link for a beta Wiffiti screen tagged @River that you could try.
http://tinyurl.com/leoriver" rel="nofollow">http://tinyurl.com/leoriver
You can txt a message by sending @River <message> to 87884 or any Tweet with @River (or any other word we/you choose to tag) would get displayed on the @river Wiffiti screen.
This is a "lean in" and chat or "lean back" and watch experience.
[...] Leo Laport, host of the amazing This Week in Tech, on starting to use a live text stream of links, comments, and notes via Twitter, FriendFeed, TwitArmy etc for his show. [...]
[...] also am working on a page to view the TWiT Army and eventually TWiT.im river stream and TWiT Live video stream on one [...]