In my ongoing quest to penetrate the mysterious veil of social media measurement, I got interested in Klout a couple of weeks ago. Klout, for the uninitiated, is a score derived by a thorough analysis of all of one’s activities and behaviors in the social media environment as well as the connections one makes and the connections and activities of one’s connections. Following the thorough analysis, one is assigned a Klout Score between 1 and 100.

Once I grokked the basics of Klout, I was rolling along on behalf of the agency, watching as our Klout score inched up from 24 when I first discovered it to a high of 40 last week. Then, when I checked my email today, I had received an update from Klout. Suddenly, overnight our Klout score had dropped to 30 with no explanation other than Klout had just recalibrated everything.

Touting it as a new, more “transparent” and more “accurate” system, Klout explained the change this way:

Today we’re releasing a new scoring model with insights to help you understand changes in your influence. This project represents the biggest step forward in accuracy, transparency and our technology in Klout’s history.

Sounds good, doesn’t it? Klout further explained:

We analyze 2.7 billion pieces of content and connections daily. Reaching this scale, we’ve introduced significant upgrades to our platform, allowing us to handle this explosive growth. Now, we can add more networks and other sources of your influence much, much faster.

Insights help you understand why your Score changed. Each day, you can see which subscore and people in your network caused that change. You can also view insights on your friends’ profiles.

That makes sense, kind of. I mean, if you’re analyzing 2.7 billion piece of content a day, well, that’s a lot. But it still didn’t explain the precipitous drop in NewmanPR’s score. The worst part was this bit:

A majority of users will see their Scores stay the same or go up but some users will see a drop. In fact, some of our Scores here at the Klout HQ will drop — our goal is accuracy above all else. We believe our users will be pleased with the improvements we’ve made. Below is a distribution of the Score changes. You’ll note large decreases in Score are rare.

Apparently, a drop of 10 points, as in our case, is the second-rarest of the rare, the rarest being -15, according to an attractive but meaningless bar chart Klout provided.

I’m reminded of a well-known cruise guidebook writer who was notorious for changing his scoring system every few years, seemingly on a whim. He didn’t revisit the several hundred cruise ships included in his guidebook and assessed their merits all over again, he simply recalculated their scores based on his new system. Usually such a re-scoring was accompanied by press release screed about how the quality of cruising was declining, and now four-star service was being sold as five-star and he wasn’t going to stand by and watch the industry be cheapened, etc.

I think Klout was in a similar situation — too many people’s scores were getting too high too fast as they figured ways to game the system, as humans are wont to do. So Klout recalibrated everybody and those of us who came on later and rose rapidly were treated to a fall.

So, does a Klout score mean anything? Maybe. After all, a company that analyzes 2.7 billion pieces of content daily can’t be all wrong — can they?