Technorati Blog

Technorati has a number of initiatives in the works to improve the data in our search indexes and analytics systems. Web spam sites (splogs) have long been an issue that we've been working to address. The days when pings came only from legitimate blogs are long gone. Including all of the spam and duplicates, Technorati receives over 8 million pings per day. Over 90% are recognized and blocked as soon as they're received. The remainder is allowed into the system and selectively processed - a large portion is determined to be spam later.

Recently, we've been focusing on link farms and pornography sites that have been getting into the system. Link farms are networks of sites linking to each other and other sites with the intention of raising search rankings. Sometimes, these sites link to legitimate blogs to "camouflage" these intentions or simply because the content has been stolen from another site. During a recent scrub of the system, a number of legitimate blogs were misidentified as spam. The flags set on those blogs were reversed, so going forward they are being indexed correctly again. However, some of the link and post data scrubbed from our search and analytics systems could not be reverted. We're working on upgrades to make that data handling better managed but in the meantime, there are some gaps in certain blog's data which may affect the authority of blogs they linked to. Additionally, some blogs suffered authority drops due to being the beneficiary of camouflaged links from spam sites being removed (wittingly or not); when those spam sites were removed, so was a portion of the authority of the legitimate blogs they linked to.

We have a number of technology initiatives in the works to improve the scaling characteristics and data quality of our systems. More news will be arriving on that in the weeks and months ahead.

I guess you could say we’re on a bit of a roll. In June, we launched Technorati Media, our ad network. At the end of August, we acquired BlogCritics, a great site and community of blog authors. Today, we’re launching a private alpha of Technorati Engage, a self-service advertising network for blogs and social media sites.

Step one for Technorati Media was to get the network up and running with a smaller group of bloggers. But our ultimate goal is to help bloggers at all levels make money, while at the same time creating an effective and safe vehicle for brand advertisers.

So step two is to open up something that works for everyone, and that is ideally suited to the long tail. While the audiences here are smaller, the levels of engagement, influence and expertise in the audience can be exponentially higher. It’s also an incredibly challenging space for advertisers to target and buy. With Technorati Engage, advertisers can very easily achieve the necessary levels of targeting and critical mass.

To help power this new platform, we’ve acquired AdEngage, an online advertising network and advertising exchange.

AdEngage has been running since 2004, serves more than 12 billion ad impressions across more than 4,000 sites, and has grown rapidly each year since its inception. They’ve established a good reputation with publishers: dependable, accessible, and they pay on time. They’ve built an interface that is very easy to use for advertisers and publishers.
Most importantly, AdEngage was founded by the publisher of an independent site, who was fed up with his ad network, expressly for publishers of independent sites. It’s designed from the ground up with your challenges in mind.

So when will all of this happen?

First we’re launching the Technorati Engage private alpha, which offers AdEngage’s existing text and PhoText products. Next month, with the public beta launch, we’ll add a 125x125 square, which is one of the leading ad formats in the blogosphere. We’ll quickly move to full display capabilities, with the most popular IAB display ad sizes becoming available for Q1.

How will this work?

You simply enter into the exchange, are reviewed to ensure your site meet Technorati’s content quality standards, and your inventory is made available. We’re reviewing and evaluating the applying blog and social media sites before they are accepted, as well as only giving access to qualified advertisers, to ensure a safe place for brand and engagement marketing. Advertisers are then able to directly target, buy, create, and see the performance of their ads. You can sign up now and come back and grab the ad code next month.

While Technorati Engage will focus on blogs and social media, AdEngage will continue to operate as a standalone entity and grow its existing business with its current self-serve advertising network.

On October 1st, we’re teaming up to support DonorsChoose.org in their 2nd Annual Blogger Challenge DonorsChoose.org is dedicated to getting our kids the materials, resources and experiences they need to learn. They’re challenging the blogosphere to compete to see who can rally the most support for public schools. Across the blogosphere, bloggers are creating giving pages that list specific classroom requests in public schools--and then encouraging their readers to donate to those classroom requests.

Technorati is sponsoring the "generosity rankings" – which also means that at the end of the challenge we’ll be broadcasting the results showing which bloggers drove the most generosity. You can see the current giving contest here.

During the last DonorsChoose.org Blogger Challenge, blog readers donated $420,000 toward classroom projects benefitting 75,000 students in low-income communities. This year, the need is even more urgent: the rough road ahead for the US economy means an even rougher road for public schools. With your participation, you and your readers can help thousands of public school kids. It’s easy:

HOW YOUR BLOG CAN HELP

1. Pick a few classroom requests posted on DonorsChoose.org and add them to a challenge page which takes 1-2 minutes to set up.

2. Do a post on October 1 encouraging your readers to donate to any of the classroom requests on each of your challenge page. Your readers can give as little as $5.

3. Publish a widget which pulls in the classroom requests you have selected and shouts out the readers who have donated to those requests. Simply select the category to which your blog belongs to grab the appropriate widget.

If you have additional questions or need help getting started, feel free to contact DonorsChoose.org directly at bloggers@DonorsChoose.org.

BACKGROUND ON THE CHARITY
DonorsChoose.org grew out of a high school in the Bronx where teachers saw their students going without the materials needed to learn. Our website provides an easy way for everyday people to address this problem. Public school teachers post project requests that range from a $100 classroom library, to a $600 digital projector, to a $1,000 trip to the zoo. People like you can choose which projects to fund and then get photos and thank-you letters from the classroom.

BACKGROUND ON THE 2008 DONORSCHOOSE.ORG BLOGGER CHALLENGE
In October of 2007, bloggers competed to see who could rally the most support for public schools via DonorsChoose.org. Blog readers gave $420,000 to classroom projects benefitting 75,000 students in low-income communities. While A-list bloggers like Engadget and TechCrunch inspired great generosity, smaller blogs with really engaged readers generated even more! In fact, it was a personal blog from Brooklyn, TomatoNation who brought in a whopping $100,000.

Thank you so much for your support.

I’m very happy to announce that we released the 2008 State of the Blogosphere report this morning. If you missed my talk at Blog World Expo on Saturday, you can see the study here.

We’ve been publishing this report since Dave Sifry wrote the first one in 2004.
This year, we wanted to go beyond the numbers and deliver deeper insights into bloggers and the state of blogging today. In addition to analyzing the data from the Technorati Index, for the first time, we’ve reached out to the blogosphere to understand the role of blogging in their lives; tools, time and resources used for their blogs; and how blogging has impacted them personally, professionally and financially.

So what did Technorati measure this year and why?

There’s a wide range of estimates of the number of global blogs as well as blog readership (including ours), but all the numbers agree that blogs are a global phenomenon that has hit the mainstream. Further, as the blogosphere grows in size and influence, the lines between what is a blog and what is a mainstream site become less clear. Larger blogs are taking on more characteristics of mainstream sites and mainstream sites are incorporating styles and formats from the blogosphere.

We feel that the real story now lies with the Active Blogosphere. The trends, stories and behaviors here influence not only the rest of the blogosphere but mainstream media as well.

Technorati defines the Active Blogosphere as: The ecosystem of interconnected communities of bloggers and readers at the convergence of journalism and conversation.

So how do we determine who’s active? Some blogs are more integral to the blogosphere than others: How frequently does this blog post? Is this blog linking to others and are others linking to it? Does this blogger post original, opinion, or reactive content? These are all factors that factor into a blog’s authority and determine its place in the active blogosphere.

In short, these are the bloggers that are making the space tick.

The study goes live over the course of this week:

Live today: Overview, and Who are the Bloggers?
Tuesday: The What and Why of Blogging
Wednesday: The How of Blogging
Thursday: Blogging for profit
Friday: Brands in the blogosphere

Late last week part of our indexing system underwent a brief outage. During that time a number of blogs experienced a halt in indexing. Since then, fixes have been implemented and indexing is occurring again. However, because of the outage, a backlog has built up in our spider queues. The spider queues keep track of pings as they come in, so no pings were lost, we just have to process them.

We are catching up as quickly as we can, but in the meantime, please expect delays in indexing, particularly for those on the Blogspot and Wordpress blog platforms. We appreciate your patience as we catch up and thank you for your understanding.

Today, we’ve announced the acquisition of Blogcritics.org. If you’re a blogger, you might be familiar with them (and they’ve been part of the Technorati Media network since June). If you’re not, Blogcritics is an online community of thousands of bloggers, and an award winning site. They’re publishing everything from music reviews to articles on politics and technology – to a monthly audience of more than a million.

Why did we do it? It just made sense – as we’ve stated more times than you probably care to hear, our mission is to help bloggers and the people who read blogs. Blogcritics shares this mission, executed in their own unique way by providing a large stage for bloggers to express themselves while giving readers a great array of high quality blog content.

What’s in it for us? Blogcritics brings us closer to an open community of bloggers and the audience that follows them. It also gives us a lot more advertising inventory. What’s in it for them? Our combined resources will help that community grow and expose their work to an even wider audience. We’ll also work more closely with Blogcritics authors so they can monetize their own blogs. What’s in it for our advertisers? They’ll be able to run more fully integrated programs across the site and its related blogs.

I’m thrilled to welcome Blogcritics and their great community of bloggers to the Technorati team. We’re excited at what the coming months hold for both properties.

We're committed to improving our search results and the overall user experience and are taking steps to reduce the amount of spam and non-blog entries that make it into the Technorati index. We’ve made some improvements in how we identify legitimate blogs in order to filter out the spam.

What was the problem?

Large volumes of splogs (spam blogs) and non-blogs ping us in ever increasing numbers. While only a small percentage get through our filters, it is still enough to negatively impact the Technorati experience for everyone. Splogs can show up in search results and blog reactions. Additionally, the high volume of junk pings slows down our systems.

What does this mean for me?

New blogs

In the past, simply pinging Technorati initiated an indexing of a site. This open door policy meant a lot of splogs and non-blogs would enter the system and masquerade as blogs. In order to better filter out spam as well as inadvertent pings to Technorati, we have implemented a few simple review measures to evaluate a site prior to indexing it as a blog.

The review process entails verifying that the site falls within the Technorati Blog Quality Guidelines.

Existing, indexed blogs

As an active blogger already in the Technorati index, we will continue to crawl and index your blog as before. We hope that your Technorati experience will improve. Our service should be faster, search results and blog reactions should be cleaner, and the number of legitimate blogs accidentally caught by our spam filters should decrease.

Links from new sites will be attributed to blog reactions and Technorati Authority when the new site is approved and indexed.

Technorati is experiencing a problem with our search result updating infrastructure. We continue to crawl and save data, however, post search results are stale and temporarily stuck at about 3pm Pacific Fri. Aug 15. Link results (reactions) are stuck at Thu. Aug 14.

We have identified the root cause and are actively working on the issue. We expect to have the system caught up during the evening hours.

No data is being lost, but the most recent posts and reactions are not reflected in results at the moment.

UPDATE:

We have restored our post and tag search results. Link (reactions) results are catching up. We expect the system to be fully restored late this evening.

I was in Chicago last week to participate in ad-tech. The content and speakers struck me as particularly good this time around, with a major focus on social media.

The media shift of the past few years is fundamental – you can’t underestimate this – and it’s critical that brands adapt to life in this new environment. There was definitely an air of urgency on the part of everyone present to figure it all out.

Overwhelmingly, the two main themes I heard were:

Brands need to be part of or at least adjacent to the conversation

Brands need to go where their audiences are versus trying to bring audiences to them

A few highlights and how-tos from the sessions I attended:

The six drivers of brand credibility in social media environments*

• Trust
• Authenticity
• Transparency
• Affirmation
• Listening
• Responsiveness

The conversation is less about brands and more about the issues and topics that surround brands, or that are passion points for the audiences of those brands.

Every brand is different: You might need to blog, you might need to listen and interact or you might simply need to be present alongside the conversation.

Speaking of execution:

The microsite was declared dead. Rising up in its place are media that function as the microsite, but do it one better by putting that content and interactivity where your audiences ARE: conversational ads and channels, widgets.

Even the most universally loved brands have their critics. Look at this new era not as a problem to solve but as an unprecedented opportunity to truly know what people think about you, and to engage with them.

The long tail is where you find influence. Even if a blogger has a relatively small number of followers, the level of influence and trust is exponentially higher than with large, mainstream media

And finally, don’t wait for a crisis to get started. The case studies are there: conversational strategies are working.

“We’re not serving them dinner anymore, we’re at the dinner party.”
- Richard Binhammer, Dell, Inc

Technorati is bringing you that much closer to attending Web 2.0 Expo NYC next month – we’ve got free tickets to give away! As a media sponsor for the event, Technorati has complimentary promotional tickets for the conference taking place Sept. 16-19 at the Javits Center in NYC.

For your chance to snag a ticket, email WebExNY@technorati.com by August 12, 2008. You’ll be entered into the drawing, and notified by August 14.

Good luck!