Metapost: Roger Ebert on the Five-Star Rating System

So why am I posting a link to a blog posting from Roger Ebert on this here review site? This is a webcomic site, not a movie site! No doubt some you want to sit me down, pour me a nice, cold glass of milk, and say: “El Santo, webcomics are COMICS on THE WEB.”

And I’d drink that milk because it’s a particularly warm day in Seattle today, and because I need some calcium. But Ebert comments on an interesting grievance that may be of interest to the many reviewers who drop by this site: “You give out too many stars.”

Roger Ebert on giving out stars.

Really a little man jumping and clapping.

Hence, I’m posting this link because I get a lot of questions — surprise! — on why certain webcomics are ranked the way they do. Ebert discusses the topic of star ratings, and he elaborates on these nifty points:

  • Gene Siskel boiled it down: “What’s the first thing people ask you? Should I see this movie? They don’t want a speech on the director’s career. Thumbs up–yes. Thumbs down–no.”
  • Once the scent of blood is in the water, the sharks arrive. I like to write as if I’m on an empty sea. I don’t much care what others think. “The Women” scored an astonishingly low 28 score at Metacritic. “Sex and the City” scored 53. How could “The Women” be worse than SATC? See them both and tell me. I am never concerned about finding myself in the minority.
  • I have quoted countless times a sentence by the critic Robert Warshow (1917-1955), who wrote: “A man goes to the movies. The critic must be honest enough to admit that he is that man.” If my admiration for a movie is inspired by populism, politics, personal experience, generic conventions or even lust, I must say so. I cannot walk out of a movie that engaged me and deny that it did.
  • I cringe when people say, “How could you give that movie four stars?” I reply, “What in my review did you disagree with?” Invariably, they’re stuck for an answer. One thing I try to do is provide an accurate account of what you will see, and how I feel about it. I cannot speak for you. Any worthwhile review is subjective.



At the end, he recalls the Little Man rating system from Mick LaSalle of the San Francisco Chronicle. The Webcomic Overlook follows a similar philosophy to the Little Man, though I use little Mario stars rather than fanciful little men.

The meanings of the various Little Man icons have been subject to interpretation, even here at the Chronicle.

First, the easy ones:

Little Man #1: THE LITTLE MAN JUMPING OUT OF HIS SEAT CLAPPING: This is reserved for hands-down great films. But it also, in practice, ends up going to A-minus films, because we don’t have the equivalent of a three and a half star rating. With A-minus it can go either way. Basically, I try not to give this rating unless I love it.

Little Man #2: THE CLAPPING LITTLE MAN: This is a definite endorsement. No B-minuses here. This is telling people that they will enjoy this movie. So anything that’s definitely good and definitely not great resides here.

We’ll skip little man #3, save him for the end, and go to…

Little Man #4: THE SLEEPING LITTLE MAN: This is instruction not to go. Definitely bad, definitely not worth seeing. But maybe, just maybe, there’s a single decent scene in there (there usually is), some glimmer of something that might have been.

Little Man #5: THE EMPTY CHAIR: This doesn’t mean, of course, that the critic actually left. As you know, we can never leave. But it means that the movie is a complete bomb with nothing redeeming about it. However, in some cases, the extent of the awfulness can give the movie a kind of purity — like, say, Bo Derek and Anthony Quinn in “Ghosts Can’t Do It.” This is probably why the kiss-of-death rating is actually not this rating but the SLEEPING little man rating. Some people actually like the idea of going to EMPTY CHAIR movies.

Which brings us to the ambiguous one . . .

Little Man #3: THE “ALERT” LITTLE MAN: Because we don’t have half-stars, the Alert or Interested Little Man takes up the mid-range and is used for everything from almost-OK-but-not-quite to almost-no-good-but-not quite, which is actually quite a wide range. Every so often, an editor decides that the Interested Little Man should be considered a positive review, the equivalent of a two-and-a-half star rating, and that’s actually how I try to use it — for reviews that are, on balance, positive, but the buyer should beware. A decent but unexceptional genre film would come into this range. If it doesn’t transcend the genre, it will mean that someone not drawn to the genre wouldn’t like it; hence the Interested Little Man.

The temptation is to overuse this rating as a way of being wishy-washy, but it’s worth remembering that for the reader, it’s the most disappointing rating, in that it’s journalistically the least interesting. It gets used a lot, because in reality a lot of movies end up in this zone, but it can’t be used as an excuse for indecisiveness.

I think that about sums it up.

So there you go, ladies and gents! The Webcomic Overlook follows a variation of the rating system that Roger Ebert calls “the only rating system that makes sense.” If questions still persist, I may have to replace those stars with the tiny fellas. Maybe some guy crashed on a sofa and laughing softly while clicking on the laptop touchpad button.


About El Santo

Somehow ended up reading and reviewing almost 300 different webcomics. Life is funny, huh? Despite owning two masks, is not actually a luchador.

Posted on September 15, 2008, in The Webcomic Overlook, webcomics and tagged . Bookmark the permalink. 5 Comments.

  1. That’s an interesting thought. If I ever get around to doing my own review site, maybe I’ll draw little comic-me icons that show my response to the comic instead of stars (or cookies, or flaming arrows, or whatever.)

  2. Did you ever notice ratings are the most useful at the extremes? Which is a way of saying YES! or NO! The MAYBEs are hard to parse.

  3. Interesting indeed.

    I have been working on a variant of Mike’s (mpd57) hat system for my own star-rating purposes. I was thinking it might offer a more subjective take on comics but… it may just be that I’d like to avoid the middle road and maybes, since that’s what I see a lot of.

  4. Well, here’s my take on things: I avoid halves. I don’t give out half ratings, and the above little man explanation explains why.

    Doing halves effectively makes the system a 10-star rating system.

    At least when I took courses on public polling, the best system is considered a 3-point system (which signifies “No,” “Indifferent,” and “Yes”). The next best would be 5. And then 7, and so on.

    Ebert is right on the money when the question on most people’s minds just boils down to “Well, did you like it or not?”

    When you start going from 4 to 4.5, you start to mentally rank comics on preference. Such as, “Well, I gave McNinja a 4.5, and Pibgorn a 4. I like Joe Loves Crappy Movies more than Pibgorn and less than McNinja. Do I give it a 4.25?” And you just can’t do it that way. The decimals are just too hard to keep track of.

    Ratings are bunk, anyway. There’s 2,000+ words of opinion already out there to tell ya what I really think.

    • I never heard of a 7-star system. I tried using a 10-star system. It was too hard. I don’t know when to give 10 stars because it is like calling it perfect then I realized that there is no point of 10th star if I never used it. I guess that I haven read enough books to use the 10-star system.

      Is it a matter of experience or preference?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: