A few months back I started working on thredr.com, a service that keeps track of online news conversations around topics that I’m most interested in, such as Tablet PCs, Microsoft developer trends, and soon Robotics. It’s very much like TechMeme in that sense.

The way it works is that it culls through about a thousand or so RSS feeds applying various criteria for how to group or list stories. Of course, the idea is to try to place the most interesting stories first–all done automatically, several times a day.

One bit of feedback I got early on from friends is that they thought there would be no way an algorithm could generate an interesting enough collection of stories. Creating the list by hand would be so much better. The arugment goes that a human can better editorialize.

My response: The algorithm is doing no more than what you tell it, so in fact, you can editorialize through the algorithm. Want to emphasize links from more popular blogs, for instance, because that’s where you think the more interesting stories are? Then simply change the algorithm to do so. Likewise, if you don’t care for the conversational tone of posts coming from forums, then don’t scan through them or demphasize them.

You see, whether an algorithm is being applied or not, there’s still plenty of editorializing going on. It’s just that it’s automated.

I’d argue that the creators and managers of TechMeme, for instance, are in fact editors. It’s simply that they apply their editorial instincts via an algorithm. They decide how sites should be ranked, they decide which sites to scan, which ones to give preference to, and so on.

Take a site like ValleyWag, for example. TechMeme promotes ValleyWag stories all the time, giving them comparable weight to let’s say an article from SearchEngineLand. The result is that as TechMeme readers we’ll often see stories about some rumored attack on someone alongside breaking stories on an acquisition or major service. That’s TechMeme’s choice. That’s their editorial decision. The algorithm simply follows the instructions the “editors” at TechMeme have spelled out for it. If I were on the editorial board of TechMeme I’d make a different decision. That’s just me. Others would create a different mix.

This is a note to myself to add the Tablet PC Nursing blog to thredr.

I also need to rework the clustering algorithm for the Tablet PC topic. There’s not enough linking going on right now even though there are plenty of posts.

Likewise, in the Apple space just about everyone uses partial RSS feeds so I need to scrape more of the web pages directly.

Every now and then I take inventory about which products and services I really use and rely upon.

In terms of the latest trending products, my list is pretty thin.

At the top is my blog, which you’re reading here. It’s still my number one way of archiving my thoughts and experiences that other can read. It’s meant to be public and shareable.

Next up is Twitter. Some call it a micro-blogging service, some liken it to IM. Both are right. It’s a must have for pushing out small posts as well as keeping track of what’s going on.

In third place are social news sites, such as TechMeme and Thredr. TechMeme I think most people know about. It clusters conversations taking place in the blogosphere and various media sources. Thredr is something I started, which is like TechMeme, but tracks topics that TechMeme doesn’t cover, such as news around R/C vehicles, Apple, Tablet PCs, Microsoft developers, digital photography, and soon Robotics. I check each of these three of four times a day to see what’s going on.

I’m also a fan of video. So I often check what’s going on with Qik as well as a couple other sites, though I’ve noticed that I’m doing this less and less. I’m getting too busy I think.

What’s most notable at least to me, is that Facebook doesn’t make my list. It’s too much of a walled garden to be interesting and there’s tons of junk applets that make it too MySpace-ish for me. I wouldn’t notice if Facebook fell off the edge of the earth. I’m not alone in this. Many of my friends that were giving Facebook a try have pretty much stopped using it too. No doubt students will still use it because of its strong social networking value, but for the rest of us, it’s not all that.

Likewise, I gave FriendFeed a short spin. Outside of the fact that I could see it becoming the epicenter for many online conversations, I don’t have that much desire to sit and chat and page through lists of….stuff.

Lastly, I might throw in an honorable mention to Linked In. Whereas Facebook reminds me of a highly organized high school event (that would place MySpace in the middle school social rung I guess), Linked In has maintained a fairly professional feel about it. I don’t use it all that much, but every now and then I get a link request and visit the site. It does maintain a network for me that’s a step above my email contact list in terms of who I might know. My email list is still more valuable overall, but still, I see how Linked In fits in.

What’s all this mean? Who knows. Since I often guess wrong on trends like this, I’ll use reverse psychology and predict that Facebook is going to be acquired for more money than any other startup and that Twitter and TechMeme–both services which I think have the greater value–will continue on their independent paths.

Update: Robert Scoble argues the locked-in feeling I get with Facebook has great value to Microsoft.

Henry Work over at TechCrunch lists out about 100 top tech bloggers that he culled from TechMeme’s Leaderboard. I’m sure it’s obvious to those of us that follow tech news that this isn’t exactly a great way to determine much of anything–except for those who are commonly on TechMeme. This probably says more about the editorializing of the TechMeme algorithm than anything else.

I’ll give Henry this, if we’re talking “Web 2.0″ for the most part, then TechMeme’s Leaderboard probably gives a good idea of the major trends in the industry. But there’s so much more to tech than TechMeme covers. Lots more. That’s one reason I started Thredr, because there are topics I (or my friends) like to follow that don’t show up on TechMeme. What kind of topics am I talking about? Well, you have to go no farther than Microsoft-oriented software developers. Most of the news around what’s going on in this group rarely makes it to TechMeme. So does much of the news around the iPhone or MIDs or the OLPC. Yes, there are sprinklings of news, but there is so much more to read about, to learn from.

And then there are tech bloggers that are sure not to make any typical tech blogger list. For example, what about Chris Anderson’s DIYDrones.com. Come on, you can’t get any techier than flying robots can you?

So my list of top tech bloggers is probably much different than anything Henry would come up with. That’s fine. That’s why thredr means so much to me and my daily reading habits.

Looks like something’s awry with the Tablet PC topic on Thredr. I’m in the midst of a big project today and tomorrow so I’m not sure if I’ll get to it in the next day or so, but I’ll try my best to figure out what’s going on fixing it.

ReadWriteWeb looks at some alternatives to Techmeme for finding tech news. A few of the suggested alternatives: Rssmeme, HackerNews, and FriendFeed.

Yeah, and thredr didn’t make the cut :-). No big surprise there. Checking Google Analytics, yesterday it looks like there were 27 unique visitors. Almost half from Los Angeles interesting enough and all others scattered about the world. No matter, I use it all the time to keep track of various topics that interest me and so far I haven’t seen anything that’s more iPhone friendly–which is one of my top ways to check online happenings.

I’m also a bit surprised that alltop and popurls didn’t make the list, but maybe they are a different beasts.

Oh, and no mention of Twitter, my other fav.

Scott Hanselman looks at blog stats and trying to figure out who’s reading his blog.

From his post is a great quote: “The only way to know if a human is reading your blog is if they are talking with you.”

So true.

That’s one reason I like services like TechMeme and Thredr. They give you overviews of the major conversations at hand.

Robert, your post is exactly why I wrote thredr. I want to go beyond the professional journalists promoting this or that social media property that I see on TechMeme. I want to track other conversations I’m interested in, such as those around Tablet PCs, Apple, Microsoft development, and now R/C and drone flying. Robots are up after that. Can’t forget the Robots!

Update: For those of you visiting here from Robert’s blog, I didn’t think about a trackback link showing up in his comments list. Didn’t mean to spam the topic. I just get carried away by this topic because I’ve been having so much fun with thredr.

The last couple days I’ve been working on tracking conversations around Remote Control (R/C) planes, cars, and helicopters. You can follow my progress at www.thredr.com/rc.

Creating a useful list of interesting R/C content has been a bit of a challenge. Why? The big issue is that sites don’t link very much. Further, several of them appear to be more silos than open platforms. This isn’t right or wrong, it’s just the way they are. Like in other verticals, I’ve noticed that many of the sites on the same topic often coalesce around one pattern. In some verticals, linking is common (such as in the Tablet PC space). Others don’t. Some push out full posts/content. Some don’t. Some are blog oriented. Some are forum oriented.

In the case of the R/C field, there are a couple key forums, several of which require registration. And others sites lack RSS feeds or blogs. They just manually enter “posts” into HTML pages.

The challenge then is to automatically mine the interesting material from all this varied content.

So far, thredr is following about 30 feeds and a couple forums. The most active content and therefore interesting content is coming from RCGroups and DIYDrones.

I hope I can add in some of the helicopter and other sites without too much trouble, but I’m not sure how this is going to all work out. Ultimately, a site that has a conversation lister should help draw traffic, but I can see where they are reluctant to open up their content because they are concerned that people will simply republish it as their own–on spam sites and the like.

I’ll see how far I can take this. It means I may have to do some alternative things to get around these legacy posting models. This isn’t necessarily a bad thing, because it’ll encourage me to step back and try to create some unique tracking formats–some of which are going to be quite exciting. At least for me.

Using Microsoft’s Syndicated Client Experience (SCE) SDK, I put together a prototype desktop “reader” for thredr.com.

Think in terms of the New York Times Reader experience crossed with the micro-conversation tracking provided by thredr. With thredrReader you can get a “desktop”-style experience while reading through the stories and posts gathered by thredr.

The SCE Starter Kit has everything to build a reader like I’ve shown here. There are some things you need to do to get it to work. For instance, the reader doesn’t use a standard RSS feed; it uses an extended format. This wasn’t too much of a hassle for thredr, since there’s a custom app generating the content already, so I simply added a couple functions to render the RSS feed that the reader needs. Right now it just outputs text. I’m still working on fleshing out the content so that it can handle images and other content.

There are a couple issues I’m working through. First and foremost, the model assumed in the reader doesn’t quite match that of conversation aggregators such as thredr. With a reader, such as the New York Times Reader, the NYT owns all their content so has no problem republishing it all in digital format. In an aggregator, this is not true. Further, many source feeds don’t contain the full content anyway, so without doing some extra work, the whole story isn’t available anyway.

This isn’t a huge deal, because the text excerpts can simply be a link to the originating web content to view the rest of the story, but this isn’t the optimal experience offline. A prefetch of this content for offline viewing may be needed.

Similarly, there’s a whole copyright issue. Thredr can provide a text snippet or image thumbnail, just like a search engine will, but it can’t present more in the reader. My current approach is to have a flag that signals whether content from a particular site can be replicated in the reader. I have no problem with my content being duplicated in the reader, but I can appreciate the fact that there are others that would not like this. No problem. It shouldn’t copy the content by default. This should be a special case.

One nice thing about the reader over using the browser to read the content is that it is searchable. This feature is more useful than I thought it would be. In fact, I’ve began to wonder about twisting it into a filtered view where you specify want you want to watch for.

So, I’m still thinking through what it means to make the conversations as accessible as possible. Thoughts, anyone?

I set up Thredr last night so it would automatically build the conversations going on among Tablet PC/MID/Eee PC/ModBook/UMPC fans, Apple enthusiasts, and Microsoft-oriented developers every two hours.

Thredr has become one of the top places I check for news in these three topics. It gives a quick glance at what people are talking about.

I still have a little work to do with thredr. Cosmetically I know I need to restrict the text’s maximum width. I just haven’t gotten to it. I also need to some additional feeds in each of the topics to increase the quality of the linking. And I need to tweak the ranking/grouping algorithm a bit too to group related conversations. There’s probably more to do.

Bob’s still working on the front end so you can create your own micro-conversation topics too. That should be fun to use.

We’re about to go live with what we have on thredr–our micro-conversation tracker. And this is as good as time as any to look back to what I’ve learned over the last couple weeks.

First, there’s the RSS. You wouldn’t believe all the messy feeds out there. Some from small sites. Some from major news organizations. Despite all the clamouring for web standards, you can see that people don’t follow them. I’ve given up being too restrictive here. It is what it is.

Also, when it comes to RSS, there are definitely patterns within various communities. For instance, in the Apple community it’s quite common to only provide a partial feed. I don’t get this, but oh, well. Microsoft developers are on the other end of the spectrum–they typically share everything. There are also lots more of them, sharing much more content.

I also noticed that outside of a few key sites, linking isn’t all that common in photography blogs (and with many Apple sites too). This surprised me. There aren’t as many “conversations” in blogs as I expected. Instead, these two communities are more active in forums. In fact, it wasn’t until thredr started reading through forums, that the richness of the conversations started showing up in these areas. This is also true in the finance area, but I’m haven’t found the magic recipe yet.

How useful have I found thredr? I’m probably a biased sample, but thredr has pretty much done what I expected. It gives me a quick glance at what’s going on in various communities. In fact, it’s become my number one way to check news in the Apple and Microsoft developer spaces. The top news stories consistently migrate to the top of the page in both of these areas. The one problem I’ve had is that some of the big breaking stories stay too long at the top, but I’m working on that.

In terms of the Tablet page, it really came to life once I added Eee PCs, OLPCs, and the like. Not only is the variety more interesting, it’s also showing me how active these other markets are right now. It makes sense, but you can really see it in the volume of conversations going on in these areas.

All this being said, I’m still a fan of techmeme and I visit it several times a day. It’s richness goes far beyond thredr–of course, the two are designed with two different goals in mind.

Finally, are “conversations” the way to track news. Not completely. I still think the more reading you do, the better off you are. Thredr provides just one slice of the news. There are other ways to tracks what’s going on. AllTop.com lists the most recent headlines from a variety of top sites. And Google Reader let’s you pull a variety of feeds together in any way you like. Each has its value.

So this weekend or next, I’ll flip the switch on thredr and let it run automatically. I’m thinking I’ll rebuild the conversation list every hour or so. Hopefully that’ll be frequent enough.

I think I fixed the RSS feeds for thredr. Hopefully, they’ll work in all browsers now.

I needed to url encode the links and html encode the titles. I thought I had, but I guess I didn’t. Ooops.

Let me know you have any further problems with them.

Thanks for the nudge Matt Faulkner to get this fixed.

It must be Tuesday (or at least it was yesterday), because there’s one more news aggregator site online now. This one is called NewsPond. TechMeme, TechCrunch, Mashasble have the breaking news. After all, this is a “2.0″ friendly site and that’s where the 2.0 news usually is. Kind of ironic, isn’t it?

Newspond is TechMeme-like in that it pulls together and ranks news stories and blog posts. However, unlike TechMeme it has a decidely un-Google-like design, comments, and many more topics than TechMeme.

While working on Thredr I can see how NewsPond came up with these design decisions to not only differentiate it, but also to try to make it more “attractive” and useful.

The UI really sticks out. A tad on the thick side, it makes you wonder if it’s not time to forgo the “efficient” Google look and render pages more like magazines. Personally, it’s not either here or there, but I wonder if that’s what people want and expect now. Are the days of the light Google layout over? Me thinks not, but we’ll have to see.

I do like Newspond’s support for multiple topics. This is where TechMeme needs to go even further. A bit about this in a moment.

What about the commenting system? Good idea. I think it brings people back to Newspond–although again for me I don’t find this the most useful. But I can appreciate it’s social value. We are in the 2.0 era after all.

OK, so back to the issue of multiple topics. For those that have been following Bob and my work with Thredr, this is a big deal to me. This is why I started working on Thredr in the first place.

I love TechMeme, but unfortunately it doesn’t give the level of coverage I want and need for the types of news I follow. In particular, I want to track the top conversations going on in the Tabletsphere. TechMeme has bits of this, but there’s much more out there. So “necessity” being the mother of invention, I decided to write my own TechMeme-like conversation collector. It’s still under development–as a part-time project–but I’m actively using it to track Tablet news and we’ve put it online (even in its unfinished state) so others can use it if they’d like.

Like Newspond and unlike TechMeme, thredr supports a collection of topics–which by design are fairly easy to expand. So far the topics thredr tracks include: All things Apple, Finance, Tablet PCs/UMPCs/OLPCs/MIDs/Surface, Photography, and the Microsoft developer community. We hope to be adding more topics down the road and even opening this up to others.

Some reactions we’ve gotten about thredr: It’s look is too plain; too Google like. That’s my fault. Bob keeps reminding me that this isn’t 1999 anymore and to get with it. Yeah, you can tell I’m an engineer. I want to focus most on the content. But I get the point. I need to work on this. It’s not all that obvious to me what it should be like anyway. I figure what’s really needed in the design will bubble up. I’m kind of slow that way :-).

And personally, I like the thin look. It works really well on my iPhone, for instance, which has become one of my favorite ways to check for news on the go or even at home. I can check it in 15 seconds and then move on if there’s nothing of interest at the time. I think this pattern is going to become more common for more people too. So I admit I’ve kept the design somewhat mobile minded. 

People want to create their own sets of topics. I agree. That’s the reason I wanted to work on this project in the first place. Bob is working on the pieces to make this happen. There’s no ETA yet for this, but when he gets it going, we’ll let you know.

What about comments or voting or some other social design? We haven’t put in anything yet. Still thinking. Here’s my problem with this. As soon as you have a commenting or voting system, you have to have a sign in process. Not only do I hate to log into yet another system, at this point I don’t want to deal with any of that from an infrastructure standpoint. For now, it’s all about the content. Tomorrow, maybe this will be different and we’ll leverage readers to suggest, prune, merge, rank, split, and comment upon stories. That’ll have to come later.

And one more thing. In Michael Arrington’s comments about Newspond, I think he misses a big point as to why these news aggregators make sense. For those of us that digest lots of news, we often turn to RSS readers, however, the plain simple truth is that we quickly wind up with too much stuff to read and then stop reading altogether. I’ve heard this from many people. I’m guilty of this too.

What I find most useful is a digested view that gives me the “top” news stories of the day. TechMeme does this very well, particularly for Web 2.0ish type of news. I’m working on thredr to make it worthy in this respect.

What’s so amazing to me is how many news aggregators are popping up. When we started thredr, it seemed like nothing was going on. Now it seems like a new one is launching every week. I wonder what next Tuesday will bring :-).

(By the way, if you do try out thredr, yes, I know the RSS feeds are broken outside of viewing them in Safari. I’ve been meaning to fix that. And yes, I know that there’s no UI element to take you “back” to the home page. I need to fix these. Sorry.)

I’m really liking how thredr is collecting up Tablet PC conversations.

It’s picking up most–if not all–the top stories I’m finding by hand. The one thing I have noticed though, is that sometimes big stories get split up into more than entry. I need to add to the merging code to fix this.

I have added a few more blogs to the feed list. These include Mac ModBook sites and Nokia Tablet blogs. The breadth helps to make the content more varied and interesting–yet still “on topic,” I think.