The Napoleon Dynamite Problem

Or maybe, “The One Million Dollar Algorithm”.

A competition to improve the recommendation engine of the online DVD rental company, Netflix, has been running in to problems.

As the contestants edge toward an improvement rate of 10% (the point at which the $1,000,000 prize will be awarded), their progress grinds to a halt thanks to a small selection of films that are notoriously divisive and difficult to predict. The New York Times reports that this problem is being called the Napoleon Dynamite Problem:

Mathematically speaking, “Napoleon Dynamite” is a very significant problem for the Netflix Prize. Amazingly, Bertoni has deduced that this single movie is causing 15 percent of his remaining error rate. […] And while “Napoleon Dynamite” is the worst culprit, it isn’t the only troublemaker. A small subset of other titles have caused almost as much bedevilment among the Netflix Prize competitors. When Bertoni showed me a list of his 25 most-difficult-to-predict movies, I noticed they were all similar in some way to “Napoleon Dynamite” — culturally or politically polarizing and hard to classify, including “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

Tags: