How accurate are ratings?


The question is sometimes asked "Why aren't ratings shown to four figures, to reduce the number of players with the same rating?"

We use a ratings system designed for chess, where the most likely result between similarly rated players is a draw. If they are both rated 150, they each earn 150 ratings points for the draw. In Scrabble, by far the most likely outcome of a similar game is that one player wins, earning 200 ratings points whereas the loser earns 100 ratings points.

But doesn't the fact that most players are rated on their last 150 games mean that ratings will even themselves out?

No. Simulations suggest that at least 300 games would be necessary for that. A Scrabble rating based on only 150 games is probably accurate to within plus or minus 3 points. So a player with a published rating of 150 probably has a "true rating" between 147 and 153.

Also, like is not always being compared with like. For example, Player A's rating of 120 may be based on 150 games all played within the last six months, whereas Player B's rating of 120 is based on only 15 games played during the last three years (Player B plays in a rated event once a year, at the local tournament).

For these reasons, it might be statistically more appropriate to round to the nearest 10 (so that a rating of 146 or 154 would be 150). For practical purposes the ABSP accepts the inaccuracies involved in publishing ratings to the nearest whole number.