ChessMaster Ratings

Originally posted by dottewell
Out of interest, why did Ratava suddenly leave?

He said he quit because he said online chess was a life waster, However I think his parents kindve forced him to make that decision

Originally posted by Sicilian Smaug
Couldnt you just tell us if you know the answer?

I pasted that link in another forum yesterday and just copy/pasted it into this so people can compared.

Anyways, there was some site which had other computers listed with ratings, maybe 150 computers including hand held but i have no idea what that site was. The link was on chessbase, 2 years ago i believe.

Originally posted by Wulebgr
Ratings calculated against a narrow range of opponents have no reliability. No matter how many Chessmaster personalities you test yourself against, you cannot arrive at a reasonable rating estimate that will predict your OTB rating.

Chessmaster's match against Christensen is too few games and too few opponents to estimate rating, although it reveals than ...[text shortened]... MB of RAM. Certainly the software would be stronger running on an AMD 5000+ with 2 Gig of RAM.

My stance on this subject is that chessmaster rating are inflated and OTB = CM-200/300 about.

However, this idea which someone presented to me is very intresting.

First I want to get to your answers. I'm assuming the CM ratings are exact OTB ratings for now: Narrow range of opponents:

My Otb rating is 1700 about after 48 games. I don't think anyone would doubt this. I have played approximately, 15 different opponents. Isn't that a narrow range of opponents?

CM has tons of opponents around and range. In fact, I played more different opponent in CM then i did in OTB. So I don't get how you can say they aren't reliable.

That ssdf site plays tons of game with those programs to get that rating, so those ratings are reliable. CM has played 380 games( how much more can you want) Plus, how many man vs machine games have there been? Usually the machine dominates and they aren't running on super computers, except Hydra. Most of them are running on normal 3 GHz which a lot of people have now days. So if that ssdf list say Fritz 8 is 27XX then I believe it because how many GM beat fritz in the man vs machine match. And since they played tons of games with fritz and other programs vs CM, then CM rating should be right.

Vlad plays is just strange. I would crush him but some people below him I couldn't beat regulary. Maybe you don't play enough OTB?

3.4 GHz is faster then what CM was running on for those ssdf tests.


K, so this is what i'm think about althought I don't completely agree with that CM ratings = OTB ratings.

Top GM's have played programs such as junior, fritz, shredder etc and mostly lose to them. Therefore, the ratings for those programs are around 2600-2800 OTB range. I'm sure everyone will agree with this. Now if they take those programs and play it vs CM for 376 games, then they would get a decent estimate of what CM otb rating would be. It says CM 9000 is rated at 2714. Its a bit high, i think so i'll give it 2600. CM 9k did beat Larry C and maybe some other GM's played vs it also? I just know of Larry C. So anyways CM 9k rating must be around the 2700 OTB range. And the top personality in CM is around 2700 on a 1200 Mhz computer. So using this theory the CM ratings should be exactly equal to the OTB ratings. Maybe its just exactly equal for the top personality and then as you go down and play easier people it's not exactly the same. I think that is the case. It the same for at the very top,but as you play easier personalities its not exactly the same.

Plus those SSDF numbers are from swedish people and they know their math/stats 🙂

man vs machine results from the past

http://home.interact.se/~w100107/Man%20vs%20Machine%20%20%20%2019%20Nov.htm

I seem to remember 2000+ people here playing non-rated games against CM and beating it easily? maybe it only requires familiarity about how it plays?

I have not much to say about CM top-personality myself, but the ones under 2000 pretty far from any national rating I've come across.

also, I don't think the cpu-speed has much of an effect on playing strength, as it should probably grow exponentially to have any kind of real effect. because of the nature of search trees.

Chessmaster is my primary computer chess simulation. (The latest and greatest) The clinics helped me kick the dust off, and for someone like me who hasn't been serious about chess in years, it helps my game a lot.

Though I'm still struggling to beat a 1000 rated player. I'll get there. One day 😉

Originally posted by RahimK
My stance on this subject is that chessmaster rating are inflated and OTB = CM-200/300 about.

However, this idea which someone presented to me is very intresting.

First I want to get to your answers. I'm assuming the CM ratings are exact OTB ratings for now: Narrow range of opponents:

My Otb rating is 1700 about after 48 games. I don't think anyone w Plus those SSDF numbers are from swedish people and they know their math/stats 🙂

Yes, Vlad's play is truly strange. 😲 I should be more honest. I reach a winning position in nearly every game against Vlad, but often blow it. One problem stems from responding to Vlad's instantaneous moves. At the end of a ten minute blitz game, Vlad sill has ten minutes on the clock. Vlad plays a couple offbeat openings that I struggle against with human players, and want to practice, but Vlad deviates from the book lines on move 4 or 5. Human players rated over 1800 do not deviate from the book line of their favorite openings this early. I suspect Vlad does so because that's how the programmers chose to weaken the engine.

Chessmaster personalities all stem from a single engine (with its strengths and weaknesses), but these personalities are tweaked in peculiar ways. It took several decades (mid-1950s to mid-1980s) for computer programming theory to reach the point where a computer can play a competent game of chess. Once computers reached this level of competency, they were at master strength. It then required a mere ten years longer before the machines were beating GMs routinely. After another ten years, GM victories have become rare. All of the top chess playing software today is at least as strong as the top humans (2750+).

The SSDF ratings are reliable for computer vs. computer comparisons. Engines that do better against other engines than they do against humans will have inflated ratings, but they are otherwise reliable. These ratings test the engines at full strength, not their weakened "personalities".

At their top levels, I'd say that advertised ratings of most chess software are accurate. (Those stand alone chess computers are another matter.) However, computer programming still has a long way to go in creating a weakened engine that plays a realistic game. All the Chessmaster personalites, Fritz "friend" and "sparring" mode, and multitudes of alternative methods for weakening GM-strength software leaves something to be desired. As with other ratings (RHP for example), the rating scheme of Chessmaster has internal consistency (Vlad should outplay Jon), but these personalities in the 1600-2000 range make gross errors that are rare among humans in the same rating group. At the same time, some of these personalties will play certain parts of the game far stronger than a human in that range.

I started collecting free engines and running engine tournaments in Winboard and Arena about two years ago. My hope was to find some weaker engines against which to spar. I believe that a weak engine playing at full strength might play a more "realistic" game than a strong engine playing at weakened strength. Near the bottom of my rating list (from engine tournaments on my box, and the occasional human-engine game) we find:

Program Elo + - Games Score Av.Op. Draws

91 Zephyr : 1869 116 124 40 25.0 % 2060 10.0 %
92 Awe153A : 1862 113 120 39 26.9 % 2035 12.8 %
93 Piranha 0.5 : 1852 62 64 165 20.3 % 2090 6.7 %
94 T.rex 1.8.5 : 1769 77 84 158 12.7 % 2104 2.5 %
95 Wulebgr : 1759 116 126 47 18.1 % 2022 10.6 %
96 Lime_v11 : 1715 145 171 64 7.8 % 2144 3.1 %

My rating is a little inflated here, but not completely outside the realm of realistic possibilities. I've beaten all the engines near me at the bottom, except Zephyr, which I haven't played. I win by playing a solid positional game, and avoiding tactical blunders. Against some of these engines, it can help to sacrifice a pawn for mobility, because it will overestimate its chances and help me reach the endgame I need to bring home the point. These engines do not leave book early, and do not make gross tactical blunders. They do lack positional understanding.

At the top of the list, the ratings are a bit deflated:

Program Elo + - Games Score Av.Op. Draws

1 Hiarcs 9 : 2469 39 39 257 67.5 % 2342 23.7 %
2 Hiarcs8 Bareev : 2467 96 92 45 74.4 % 2282 28.9 %
3 Fritz 7 : 2465 52 51 163 69.9 % 2318 18.4 %
4 Fruit 2.1 : 2433 63 62 82 57.9 % 2378 32.9 %
5 Fritz 8 : 2429 50 50 158 61.7 % 2346 19.6 %
6 Spike10a : 2402 60 59 112 60.7 % 2327 19.6 %
7 Delfi 4.6 : 2393 89 86 50 70.0 % 2245 28.0 %

I set the ELOStat to run 2200 as the average. The folks at SSDF set their average higher.

In short, I would say that if you want a computer to play at 1800, you will have an easier time finding it in software written by a hobbyist who has made the best program that he can, than you will in the work of professional teams of programmers who have attempted to weaken their GM-crushing engine.

Originally posted by Wulebgr
Yes, Vlad's play is truly strange. 😲 I should be more honest. I reach a winning position in nearly every game against Vlad, but often blow it. One problem stems from responding to Vlad's instantaneous moves. At the end of a ten minute blitz game, Vlad sill has ten minutes on the clock. Vlad plays a couple offbeat openings that I struggle against with human ...[text shortened]... ional teams of programmers who have attempted to weaken their GM-crushing engine.

Yeah, but how do we measure our handheld's strength when there's no USCF rating in the literature? Doesn't any organization measure every chess computer (handheld, stand alone, etc...)??? I would forever be friends with the person who shows me a list like this that's recent. hehe

Originally posted by Wulebgr
Yes, Vlad's play is truly strange. 😲 I should be more honest. I reach a winning position in nearly every game against Vlad, but often blow it. One problem stems from responding to Vlad's instantaneous moves. At the end of a ten minute blitz game, Vlad sill has ten minutes on the clock. Vlad plays a couple offbeat openings that I struggle against with human ...[text shortened]... ional teams of programmers who have attempted to weaken their GM-crushing engine.

http://www.chessbase.com/newsdetail.asp?newsid=1141


I just came across this Human vs Computer comparising. It has to do with SSDF computer vs computer comparision.


So we all agree that CM playing at its top level is around 2700 OTB ?

I agree with that.

And CMs rating at a weaker level is inflated?

I would agree with that and say its about 200 points inflated.

Originally posted by powershaker
Yeah, but how do we measure our handheld's strength when there's no USCF rating in the literature? Doesn't any organization measure every chess computer (handheld, stand alone, etc...)??? I would forever be friends with the person who shows me a list like this that's recent. hehe

Did you try searching for handheld chess computer ratings or chess computer ratings? They used to have a list of 150+ programs from a couple of years ago. I just came across the website by luck one time.

Originally posted by RahimK
Did you try searching for handheld chess computer ratings or chess computer ratings? They used to have a list of 150+ programs from a couple of years ago. I just came across the website by luck one time.

I've typed in everything and can't seem to find one. The only thing I found is that Virtual Kasparov on the Ninetendo gameboy - at full strength - is rated 1395 USCF by Kasaprov. It's a cute game. I played it. But, my other handheld - the stronger one - is no where to be found. It only says the same thing: 1700 ELO. My third - Mephisto - was found to be rated at 1900. But, I played my 1700 e-chess against the 1700 T.C. personality on Chessmaster 9000, and T.C. crushed it! Then, I played T.C. and crushed him. So, I'm confused.

Chessmaster 'character' ratings are a load of bull. The one you plyed was more realistically 1500, and your handheld thing most probably then 1300-1400.

My chessmaster program (CM9000) screwed up when I drew with it- it gave me a rating of 33000 and so I do not believe the ratings to be accurate! Anyway, the more normal ratings for the personalities are completely inaccurate- I find that some with ratings of 1400 or 1500 are harder to defeat than those with ratings of 1700. My friend, with CM10000 said that it gave him a rating of 20000!

Originally posted by RahimK
[b]My stance on this subject is that chessmaster rating are inflated and OTB = CM-200/300 about.

However, this idea which someone presented to me is very intresting.

Chessmaster personalities all stem from a single engine (with its strengths and weaknesses), but these personalities are tweaked in peculiar ways. It took several decades (mid-1950s to mid-198 ...[text shortened]... e rare. All of the top chess playing software today is at least as strong as the top humans (2750+).

Even though GMs are often beaten by the computers, Kasparov, a few years ago discovered that as long as you make obscure moves that intend to improve your position, and not go into sharp tactical variations, the computer will only dither around and waste time- allowing the player to improve his position infinitely.

Originally posted by RahimK
My stance on this subject is that chessmaster rating are inflated and OTB = CM-200/300 about.

However, this idea which someone presented to me is very intresting.

First I want to get to your answers. I'm assuming the CM ratings are exact OTB ratings for now: Narrow range of opponents:

My Otb rating is 1700 about after 48 games. I don't think anyone w ...[text shortened]... Plus those SSDF numbers are from swedish people and they know their math/stats 🙂

Interestingly, I have adjusted a few of the Chessmaster's personalites traits and they can now normally beat the orginal CM! In fact, one I called 'super master' can beat the CM 60 to 70 percent of the time. Strangely, if you change the attacker/ defender to 100 % attack, it is very weak even if the playing strength is tuned to 100% (as it attempts pointless sacrifices) but 100% defender is very strong.