I see a lot of disucssion about people having X% match with engines. I've always figured that good players would have high engine matches simply because in many cases, there are only some 1 or 2 good moves.
So, for the heck of it, I took a game that I recently finished:
Game 5599012
and I matched it up against Chessmaster: GM Edition.
After leaving the "book" (according to CM) there were 27 more moves. Of those, I got:
- 11 first choice matches
- 10 more top 3 matches
- 6 moves not in the top 3
That gives me 21 out of 27 top 3 matches, or 77.77%; Gee, that ratio sounds pretty high... even to the level of suspicious. no?
Now 2 things I can guarantee you:
1) I'm no great player; and
2) I certainly did not (and never do) use an engine during the game
I mean, seriously- can anyone look at that game and think there was engine use going on? I played well, for me. But I still only won because of some silly mistakes on his part, including trapping his own rook for no good reason.
I guess my question is: Where exactly do you draw the line between suspicious and non-suspicious and how do you correct for the fact that there might only be a couple of good moves much or the time and that good players are naturally going to make good moves, as the computer would?
Originally posted by sh76How long did it have for each move?
I see a lot of disucssion about people having X% match with engines. I've always figured that good players would have high engine matches simply because in many cases, there are only some 1 or 2 good moves.
So, for the heck of it, I took a game that I recently finished:
Game 5599012
and I matched it up against Chessmaster: GM Edition.
After le ...[text shortened]... he time and that good players are naturally going to make good moves, as the computer would?
well, anyway you decide to go, a good sanity check is to do the exact same test for the greats, like capablanca, and see how they do. that way, whatever parameters your specific system has, the threshold will be adjusted accordingly.
best would be to run through a number of pre-computer era CC champs. but of the otb players I believe capablanca has had the highest matchup rate.
at least it should give you a some kind of a rule of thumb.
Originally posted by wormwoodOkay; but Capablanca had only a few minutes to make each move- and he didn't have the analyze board feature... and he was playing under the pressure of people watching, etc. There's got to be at least come correction for that. No?
well, anyway you decide to go, a good sanity check is to do the exact same test for the greats, like capablanca, and see how they do. that way, whatever parameters your specific system has, the threshold will be adjusted accordingly.
best would be to run through a number of pre-computer era CC champs. but of the otb players I believe capablanca has had the highest matchup rate.
at least it should give you a some kind of a rule of thumb.
Originally posted by sh76You need to use a better database.
I see a lot of disucssion about people having X% match with engines. I've always figured that good players would have high engine matches simply because in many cases, there are only some 1 or 2 good moves.
So, for the heck of it, I took a game that I recently finished:
Game 5599012
and I matched it up against Chessmaster: GM Edition.
After le ...[text shortened]... he time and that good players are naturally going to make good moves, as the computer would?
www.chesslive.de probably has several times the amount of games as the db pre-loaded into an engine.
Also it's consistently high matchups over time.
If you had 1 game randomly looked at & it was top 1 match = 100% for 20 moves out of book then you'd probably have a few more analysed to see if it was an anomaly or if there is a consistent, high match up in many games.
You'd never be banned on the basis of a couple of high matchup games which have forcing tactical lines & obvious 1st & 2nd choice moves, so there's nothing to worry about!
Originally posted by wormwoodNever mind the greats, has anyone tried this on some engine v engine games? Considering the amount of hot air expended on this issue around here and the overuse of the word "statistics" I thought it was about time a proper statistician had a look. Well, I am one, with letters after my name and all. I am currently analysing a tournament held in Vienna in 1922 (no engines there but Reti does have a suspicious oil leak) and the 15th World Computer Chess Championship (plenty engines there but Loop's blunder rate is suspiciously high).
well, anyway you decide to go, a good sanity check is to do the exact same test for the greats, like capablanca, and see how they do. that way, whatever parameters your specific system has, the threshold will be adjusted accordingly.
best would be to run through a number of pre-computer era CC champs. but of the otb players I believe capablanca has had the highest matchup rate.
at least it should give you a some kind of a rule of thumb.
Before I start the serious stuff I need to know (or work out for myself) some stuff. If anyone knows of a reason for using 30 seconds per move (why not 10, 20, 40, 50 etc?) and/or using the top three engine choices (why not 1, 2, 4 or more?) can they please PM me with said reason? I don't want to discuss the methodology in open forum since I suspect that would lead to people getting argumetative and the thread being deleted.
2 edits
Originally posted by wormwoodhere's a list of otb GM matchups with rybka 2.3.2 at 14 ply (it should be longer than 30 seconds) for the first choice of the engine, taken from http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?pid=103791
well, anyway you decide to go, a good sanity check is to do the exact same test for the greats, like capablanca, and see how they do. that way, whatever parameters your specific system has, the threshold will be adjusted accordingly.
best would be to run through a number of pre-computer era CC champs. but of the otb players I believe capablanca has had the highest matchup rate.
at least it should give you a some kind of a rule of thumb.
it's very interesting to see morphy have a higher matchup than kasparov, but this could have to do with the selection of games:
Shirov 59,00% +1400 P. 34 G. 59% concur with "best moves" by Rybka 2.3.2 at 14 Ply Forwards, from more than 1400 positions from 34 games.
Svidler 57,76 % +1400 P. 33 G.
Beliavsky 57,34% +1400 P. 33 G.
Zappa 57,16% 1000 P. 12 G.
Khalifman 57,00 % +1400P. 37 G.
Kramnik 56,94% +2400 P. 59 G.
Ivkov 56,82% +1300 P. 33 G.
Aronian 56,78% +1600 P. 33 G.
Leko 56,48 % +1500 31 G.
Uhlman 56,27% +1300P. 35 G.
Fritz (deep, x3d) 56,25 % 600P. 13 G.
Morphy 56,24% +1800 P. 56 G.
Eichborn 56,10% +900 P. 29 G.
Kolisch 55,69% +1000 P. 30 G.
Ivanchuk 55,49 % +1500 40 G.
Karjakin 55,45 % +1500 P. 33 G.
Stein 55,31 % +1300P. 33 G.
Anand 54,87% +1900 P. 46 G.
Korchnoi 54,85% +1200 P. 33 G.
Geller 54,70 % +1300 P. 35 G.
Tal 54,66% +2600 P. 68 G.
Capablanca 54,55% +2900 P. 66 G.
Rubinstein 54,54% +1400 P. 34 G.
Pillsbury 54,52 % +1300 P. 34 G.
Matulovic 54,50 % +1300 P. 33 G.
Ponomariov 54,37% +1800 P. 40 G.
Kasparov 54,21% +3100 P. 69 G.
Dubois 54,20% +1000P. 34 G.
Buckle 54,16% +1000P. 25 G.
Lasker 54,02% +3600 P. 91 G.
Carlsen 54,02 % +1200 P. 28 G.
Von der Lasa 54,01% +900P. 32 G.
Fischer 54,00% +2900 P. 72 G.
Portisch 53,99% +1200P. 33 G.
Smyslov 53,91% +1700 P. 42 G.
Larsen 53,91% +1400 P. 35 G.
Petrov 53,78% +600P. 16 G.
Romanishin 53,72% +1300 P. 33 G.
Radjabov 53,66% +1100 P. 29 G.
Karpov 53,57% 3500 P. 74 G.
Kasimdzhanov 53,52% +1400 P. 40 G.
Paulsen 53,46% +1500 P. 39 G.
Spassky 53,33% +1200 P. 33 G.
Nezhmetdinov 53,13% +1300P. 32 G.
Alekhine 53,09% +3000 P. 68 G.
Gunsberg 53,05 % +1200P. 33G.
Euwe 52,85% +1600 P. 45 G.
Fine 52,71% +1300P. 31 G.
Najdorf 52,38 % +1000P. 28 G.
Short 52,21% +1100 P. 27 G.
Kieseritzky 52,20 %+1300 P. 33 G.
Harrwitz 52,08 % +1000 P. 25 G.
Topalov 52,00% +1400 P. 30 G.
Reshevsky 51,92% +1300 P. 30 G.
Neumann 51,69 % +1000 P. 30 G.
Steinitz 51,65% +1500P. 52 G.
Janowski 51,5 % +1300 P. 33 G.
Winawer 51,44 % +1300 P. 32 G.
Bronstein 51,12% +1200 P. 31 G.
Botvinnik 51.10% +2600 P. 60 G.
Taimanov 51,05 % +900 P. 23 G.
Keres 51,04 % +1200 P. 30 G.
Greco 51,03% +600 P. 37 G.
Maroczy 50,98 % +1100 P. 35 G.
Zukertort 50,82 % +1100 P. 28 G.
Marshall 50,21 % +1100 P. 31 G.
Petrosian 50,05% +2000 P. 51 G.
Kamsky 49,91 % +1100 P. 26 G.
Tarrasch 49,65% +1400 P. 38 G.
Suhle 49,63% +600 P. 21 G.
Chigorin 49,61 % +2100 P. 55 G.
Anderssen 48,97% +3300 P. 99 G.
Blackburne 48,45% +1600 P. 41 G.
Lowenthal 48,44 % +1200 P. 29 G.
Mason 48,34 % +1600 P. 34 G.
La Bourdonnais 48,19% +1300 P. 31 G.
Cochrane 48,00% +700 P. 28 G.
Staunton 47,98% +2400 P. 61 G.
McDonnell 47,79% +1300 P. 32 G.
Saint Amant 47,40% +1500 P. 34 G.
Philidor 44,41% +900 P. 26 G.
1 edit
Originally posted by Kepleryes but they only included in the list because it was the one Kasparov had played. It's a very old version. engines were a lot weaker back then.
The really interesting thing in that list is the entry for Deep Fritz which gets a slightly lower match up % than Kramnik!