Technorati colo move

I just posted an update regarding the status of Technorati.com and our colo move.

The short answer is, we've moved 2/3 of the old colo, search and tags are working, but slow, and blog claiming and blog lookups are not happy.

Our priorities are:

1. Stability and performance of the site.


2. Making sure blog claiming works.


3. Continued ping processing and indexing.

We will continue to move servers and network equipment over the next two weeks, but, at this point, we have enough capacity to run almost everything out of the new colo. Getting this far has been a ton of work for the ops team, late nights, long weekends, system pages at all hours of the day and night. I really appreciate their continued hard work and positive attitude. Ops and 7x24x365 duty is not for everyone. Throw a colo move of this magnitude into the mix and it is inspiring to hear them respond, "it's all part of the job".

Thanks guys, and thanks to the core engineering team and product team who continue to help out in any way they can.

Onward.

Technroati Tags: Technorati, TechnoratiNews, ColoMove

#1 on Google for search "dorion" - I win

For the 1st time I am the #1 result for the search "dorion" on Google. Can't tell you how cool this feels.

Google search for "dorion"



Technorati Tags: dorion, SEO

Richard Jalichandra launches a new blog

Richard Jalichandra, AKA the bossman, RJ, the Jali Green Giant (ok, I just made that last one up).

Follow him on Twitter at jalichandra.

Technorati Tags: Jalichandra, Richard Jalichandra.

What's the fastest way to turn an oil tanker?

A tidal wave.

Rather than focus on the laborious task of changing direction, it might very well be better to consider the absurd. If you can make a tidal wave, you can turn a lot of oil tankers. Of course, you need to figure out how not to sink them.

I was at a dinner party tonight with a bunch of people who, perhaps without knowing it, figured out how to make tidal waves. I suspect they did not focus on the specific problem of turning a single oil tanker. In fact, I think Niklas and Janus knew, once they had learned how to make tidal waves, they could do a lot more than turn ships. Thanks for a pleasant and illuminating evening.

Mass of a multi-dimensional cube

Recently I've been thinking a lot about multi-dimensional arrays. Don't ask me why, I can't tell you.

It's quite common to comment on the sparsity in a multi-dimensional array. In data warehousing this can be be quite helpful or the bane of your data management strategies. Keeping track of indexes and pointers can also be a royal pain.

Given a few dimensions, and knowing the expected size of any of the individual key spaces lends itself to the easy calculation of the volume of the multi-dimensional space.

You have two dimensions. Friends and Cities. You have 100 Friends. You are tracking 100 Cities. The possible combination of these keys is 100 x 100 or 10,000 cells in the space. But what if you wanted to estimate how many of your friends had visited each of the cities. This would certainly be much smaller.

So, you might estimate that most of your friends have probably visited 10 of the cities. Thus, there are only 10 x 100 cells filled in. But then, many of them may have visited the same cities, so you want to think about how many of the cities have been visited by any friends. Maybe most of the cities have been visited by 5 of your friends on average. Certainly there is a high likelihood that some of the cities haven't been visited by any of your friends.

Now, imagine that you have 6 or more dimensions. You are actually trying to model a much more complex set of keys and their combinations where not all combinations are likely to happen. Thus, you have lots of sparsity. But what about where you do have combinations?

Well, what's been racing through my mind is a visualization of the clusters, the density of the combinations and, that leads me to the notion of calculating the "mass of the multi-dimensional cube". Ok, it's really a multi-dimensional array as a cube has three dimensions, but data warehousing seems to use the cube concept.

There you have it, nothing conclusive but I thought I would coin my phrase.

Technorati indexes new White House Blog

My colleague Ian Kallen put up an excellent post about the change in Washington and, in particular, the new White House Blog. He was far too humble in stating his contribution to our new crawler architecture that is rolling out this month.

Excellent post, Ian, and incredible job on the new crawler. I know it has been a long time coming.

On a separate note, I just have to comment on the title of the White House Blog. All it says is:

Blog

Go figure.

Technorati Tags: White House Blog, Spidaman, Ian Kallen

Montecito Tea Fire links - Nov '08

I grew up in Montecito and my parents still live there. Their home is safely outside the optional evacuation point, though the fire is burning only 2 miles from their home.

I found this post on Wildfire Today that has some news updates and links to Google maps.

Here is the closeup link with an updating fire map.



Technorati Tags:

TeaFire, Montecito Fire, Santa Barbara Fire,

Technorati VP of Engineering talks with ZDNet

At the tail end of October, I had a chance to speak with Sumi Das over at ZDNet to talk about Technorati as part of their CIO Sessions.

We spoke for longer than the interview play time, but they did a great job editing it to keep it relevant to the series.

It was pretty cool to be able to look back on what we have accomplished in the 4 years I have been with Technorati and to be able share a little insight with those who might be interested.

I haven't seen myself on video in a long time and it was a bit freaky how much my body language made me feel like I was watching my own brother.

It's a quick watch, so if you'd like to know a little about the evolution of Technorati, give it a try.

CIO Sessions: Technorati VP of engineering: Dorion Carroll

In this episode of CIO Sessions, Dorion Carroll, vice president of engineering for Technorati, talks to ZDNet correspondent Sumi Das about the challenges with scaling operations as the blogosphere continues to grow. He also discusses how they’re able to index millions of blog posts in near real time, surviving the economic downturn and what differentiates the company from its biggest competitor, Google.

Technorati Tags:

CIO Sessions,
Sumi Das,
Scalability

Wordle word cloud: United States Constitution

Wordle.net has a fun new style of word cloud. It's really easy. I just created one for the United States Constitution.

Technorati Tags:

Wordle, USConstitution, Constitution

Google Blogger hosts 2% of web's malware, says Sophos

Google has created the perfect storm for malware and other malicious content with Blogger as a free content system, AdSense for quick monetization, and their search engine to deliver it all rapidly to the entire web.

read more | digg story

First read on PC Pro

Technorati Tags:

Malware, Blogger

Synopsis of the OMMA Behavioral panel I just spoken on

This was a nice little surprise from Pingie when I got back from the OMMA panel on BT



OMMA Behavioral Panel: Out of the Black Box: BT’s Next Practices

From the agenda: The classic ‘black box’ of BT tags and tracks user content consumption across the Web to target them later with relevant messages. But, how can marketers realize similar targeting efficiencies outside of the typical online ad network, in RSS feeds, email, mobile and widgets? As your content and marketing becomes more portable and customizable by the user, what targeting techniques are available now and on the way? Moderator: Roman Bukary, Truviso. Speakers: Bill Flitter, Pheedo; Eddie Smith, SocialMedia Networks; Dorion Carroll, Technorati; Elgin Kim, Nokia Interactive; Dave Martin, Ignited Media.

Not really a transcription, but got the gist of most of what was being said.

Being at the conference and on such a great panel really got me thinking about the future of advertising and content. I'd really like to see how we navigate the issues between privacy and utility. Clearly, the more an ad network knows about you, the better they can target ads, but, the riskier it will be that someone crosses the privacy line. Right now, I think it is safer to target the behavior profile rather than the individual profile and, more importantly, to be able to filter by context and end user initiated engagement.

Technorati Tags:

BehavioralTargeting, OMMA

Peter Hirshberg interviews Lawrence Lessig about Revolution

At the Aspen Ideas Festival,

Peter Hirshberg interviewed Lawrence Lessig about how his ideas have expanded beyond copyright policy and into the realm of a much needed revolution.

The U.S. Congress is broken. Stanford Law Professor and cyber visionary Lessig has set out on a simple mission: fix it.

The 3 minute interview covers some of Lessig's thinking about revolution in governance. In this instance, he is using the word, revolution, as in the revolution of the hands of a clock. They return to where they started. He is suggesting we need to return to founding ideals rather than requiring a radical change. This, in fact, is really what the American Revolution was about, though it also resulted in a radical change.

The basic premise in Lessig's view is, if our representatives were free of the dependency on campaign fund raising, they would be able to spend more time on addressing the issues of their constituency.

Thanks to Sarah Kennon from FORA.tv for pointing this out to me. Worth a listen.

Technorati Tags:

PeterHirshberg
LawrenceLessig
AspenIdeas

The Iron Man cometh

/film has a video clip of Tony Stark testing his flight controls. Pretty sweet. Should be a summer of good flicks, but I'm pyched about this one. I've always been an Iron Man fan.

1426700227

Speed and stability can make a huge difference

Over at Perry Web (Technorati? OK, Maybe I Do Care.) they noticed some of the improvements we've made in performance and stability on Technorati.

We've put a lot of effort into our search infrastructure over the last 3 quarters and I am extremely proud of our Operations team. Not only is our back end performance under 1 second almost all the time, the team was able to achieve this with very little additional expense.

I'm very excited to see someone posting good news since I suspect there a lot of others who have noticed but not thought to blog about it.

Finally, the other obvious thing that makes a huge difference is great people. That's what we have and we are looking to grow our team. Check out the job listings.

Technorati Tags:

Technorati,
Performance,
Perry Web

Update:

Case in point, here's a snapshot of my blog page on Technorati that shows it took 49 seconds between me hitting the Publish Post button and the post being available in the Technorati index.

Great suggestions on fighting scrapers, reposters and sploggers

Miriam Schwab raises a very important issue about scrapers, reposters, and sploggers ripping off her content in WordPressGarage is being scraped! I want to stop them…now!.

The comments are very informative and I recommend reading through them all.

The basic problem is, you find your content being republished on some other site without your permission and want to stop them. Some of the suggestions are:

• Alyk starts with the soft approach suggesting that you email them and ask them nicely to stop. This might actually work now and then, but for the hard core sploggers, you'll never find an email address and emailing may just tell them to add a live email address to their email spam lists. Yikes!
  


• Jennifer suggests filing a TOS complaint with their hosting provider. She also suggests reading:
  
  http://www.plagiarismtoday.com/2007/09/05/autodiscovery-and-rss-scraping/#more-623
  


• Jonathan Bailey suggests filing a DMCA with their hosting provider or file a complaint with their advertisers.
  
  Jonathan seems to have some solid background on this problem. You might just want to check out his blog, Plagiarism Today.
  


• Israel Jobs suggests:
  report them as spam to Google here:
  
  https://www.google.com/webmasters/tools/spamreport?hl=en
  
  Harder: submit a dmca complaint to Google:
  
  http://www.google.com/dmca.html
  

Having fought spam for many years, the best way to deal with this is to go after their pocket book. Track down an AdSense ID, file a complaint with Google or their other advertisers. If they can't make money easily, they'll look for other prey.

Technorati Tags: Scrapers, Reposters, Sploggers, Blogging, Fighting Back

Must read post about blogger stats by Josh Bancroft

Josh Bancroft over at TinyScreenfuls puts together a great roundup of stats that matter to bloggers with Google Analytics screen shots and meaningful context. The comments are helpful too.

Highly recommended.

Technorati Tags: Stats,
Bloggers,
Blogging

Evel Knievel has left the stadium

Just read this on Jalopnik.

Alas, Evel Knievel is gone. He definitely lit up the imagination of a pre-teen. Jumping buses, jumping sharks, jumping ridiculous distances, obstacles and natural phenomena.

Who amongst us would seek fame and accept such a personal, physical price. Very few, I expect.

Hope that last big jump landed.

EVEL KNIEVEL DAREDEVIL TRIBUTE 1938-2007

Technorati Tags: Fame,
Faith,
Glory,
Evel Knievel,
Robert Craig

American Pie

I don't blog often. I try to make each post something worth reading. This one brings together a visual interpretation of Don McLean's American Pie and the fact that I was turned onto this by my father-in-law.

Sent to me by a friend of mine..thought you might enjoy it.

Subject: FW: American Pie

http://www.youtube.com/watch?v=Ycgegp0KdE4

And I did.

Technorati Tags: American Pie

Customizing the Technorati Link Count Widget

Matt over at pixelspread talks about how to customize the look and feel of the Technorati Link Count widget. He does a nice job of making the result match the look and feel of his blog.




Oh, and since, at the time of this writing, no one had linked to the post, I thought I would to watch it change.

I use the widget on my posts. Most of them have no link love :(

Technorati Tags: Link Count Widget

Universal search comparison, explanation, and SEO challenges

It's a bit of a long read, but worth it if you want to understand the differences and similarities amongst Google Universal Search, Ask 3D, Yahoo!, and Microsoft Live.

Lee Odden of Online Marketing Blog writes about it.

Small Business Guide To Optimizing Universal Search

At the end, Lee talks about new challenges in SEO to help optimize for universal search. Particularly intereszting are the comments about embedding video and images about product launches with relevant text, descriptions, and keywords or tags to help bump the product into multiple universal search opportunities.

Technorati Tags: SEO,
Universal Search,
Marketing