16 Jul '08 19:21>1 edit
I suggested that applying the system no1marauder was told to use to gather evidence against suspecte3d cheats to a correspondence chess master who could not have used an engine might provide evidence for or against such players playing like engines. Someone suggested I should do it and post the results so here they are. I have not put this in the original thread to avoid it getting lost or possibly even deleted.
First, the how. I suspect most of you know how no1marauder has been gathering evidence by now but i shall describe the method here just to make sure you all know exactly what I did. I played each game through from the start, allowing an engine (HIARCS 12.1 in this case) to analyse each move for 30 seconds. I set it up to display the top 12 engine choices (no1 uses less but I wanted to see more of what the engine was doing) and then recorded all moves that matched the 1st, 2nd and 3rd choices. Finally, I calculated an engine match up percentage.
Second, the who. I decided to look at some of the games of Hans Berliner for the following reasons:
1. Berliner was CC World Champion in 1965 long before any chess engine would have been any use so he should come up clean as a whistle.
2. Berliner later went on to program chess engines (notably Hitech) and therefore should know something about chess engines and how they calculate.
3. Berliner wrote a book entitled "The System" that outlines his method for finding a move. He describes his system as a position evaluation function that ideally will give a numerical value for any position. He also comments that he has tried to program his Hitech system to use his evaluation function. On the basis of this I reckon that Berliner is the best candidate for possible engine-like play.
4. Berliner returned to play a 50 year Jubilee Waorld Champions Tournament in 2003 when engine use was already regarded as an issue.
5th CC World Championship Final
I had access to 15 games played by Berliner. He drew 4 and won 11.
311 moves total.
1st choice matches: 150
2nd choice matches: 55
3rd choice matches: 49
Total matches 254
Match up percentage = 82%
82% is fairly high but is just a number it does not really reveal much. I found that watching the games unfold and how the engine assessed the positions was very instructive.
Berliner always played 1. d4 as white and remarks on this in his book. As black he played Nf6 or g6 and aimed for King's Indian Defence type positions, not for the kingside attack possibilities but for solidity and safeness of position. Once out of the opening the engine generally assesses the position as about equal, giving a value within 0.3 of 0. At this point there will be many candidate moves that have approximately the same value to choose from. Berliner seems to choose from this 5, 6 or possibly more moves one that is as safe as possible. The chosen moves sometimes match, sometimes do not. In fact, in this stage of the game the match up percentage is about 50%. It seems that Berliner was in the habit of waiting for his opponent to make an error and then capitalising on it. This point is often easily seen in the engine analysis - the score suddenly changes from ~0 to ~1 (or ~-1 if playing black). At this point Berliner's play changes, his match up percentage increases dramatically and he often chooses the engine 1st choice. Berliner never blundered in any of these games although he did make mistakes, but these were not game losing mistakes unlike those of his opponents. Engine like? I don't think so but i have no doubt others will disagree.
Onwards then, to the age of engines.
50 Year Jubilee World Champions' Tournament.
This was a tournament played between past CC World Champions. It is important to remember that Berliner was 74 at the time of this tournament.
8 games played by Berliner. Won 1, drew 5, lost 2.
288 moves total.
1st choice matches: 220
2nd choice matches: 18
3rd choice matches: 6
Total matches: 244
Match up percentage: 85%
Again, this is quite high and is higher than in 1965, an again I found watching the games and the analysis instructive.
The first thing I noticed is that Berliner's opponents do not make mistakes for him to capitalise on anymore. The games tend to stay in that state of almost equality for many moves without really varying. I can think of two reasons for the lack of mistakes. First, this may be a consequence of having world champoions play each other. Maybe what separates a CC World Champion from the also-rans is simply the lack of mistakes. Second, maybe we are seeing evidence of engine use.
The second thing I noticed was that Berliner match up percentage in the sort of quiet position that dominates these games has shot up dramatically from 1965. He is now matching HIARCS' choices far more often whereas in 1965 in these quiet, equal positions he was only matching about 50% of the time. It is also notable that his highest match ups (both 92% ) happened in the two games he lost. I doubt this indicates total reliance on an engine since he is still making minor errors, and some that are major enough to lose. However, I strongly suspect he was making use of his own Hitech system.
So, make of that what you will.
First, the how. I suspect most of you know how no1marauder has been gathering evidence by now but i shall describe the method here just to make sure you all know exactly what I did. I played each game through from the start, allowing an engine (HIARCS 12.1 in this case) to analyse each move for 30 seconds. I set it up to display the top 12 engine choices (no1 uses less but I wanted to see more of what the engine was doing) and then recorded all moves that matched the 1st, 2nd and 3rd choices. Finally, I calculated an engine match up percentage.
Second, the who. I decided to look at some of the games of Hans Berliner for the following reasons:
1. Berliner was CC World Champion in 1965 long before any chess engine would have been any use so he should come up clean as a whistle.
2. Berliner later went on to program chess engines (notably Hitech) and therefore should know something about chess engines and how they calculate.
3. Berliner wrote a book entitled "The System" that outlines his method for finding a move. He describes his system as a position evaluation function that ideally will give a numerical value for any position. He also comments that he has tried to program his Hitech system to use his evaluation function. On the basis of this I reckon that Berliner is the best candidate for possible engine-like play.
4. Berliner returned to play a 50 year Jubilee Waorld Champions Tournament in 2003 when engine use was already regarded as an issue.
5th CC World Championship Final
I had access to 15 games played by Berliner. He drew 4 and won 11.
311 moves total.
1st choice matches: 150
2nd choice matches: 55
3rd choice matches: 49
Total matches 254
Match up percentage = 82%
82% is fairly high but is just a number it does not really reveal much. I found that watching the games unfold and how the engine assessed the positions was very instructive.
Berliner always played 1. d4 as white and remarks on this in his book. As black he played Nf6 or g6 and aimed for King's Indian Defence type positions, not for the kingside attack possibilities but for solidity and safeness of position. Once out of the opening the engine generally assesses the position as about equal, giving a value within 0.3 of 0. At this point there will be many candidate moves that have approximately the same value to choose from. Berliner seems to choose from this 5, 6 or possibly more moves one that is as safe as possible. The chosen moves sometimes match, sometimes do not. In fact, in this stage of the game the match up percentage is about 50%. It seems that Berliner was in the habit of waiting for his opponent to make an error and then capitalising on it. This point is often easily seen in the engine analysis - the score suddenly changes from ~0 to ~1 (or ~-1 if playing black). At this point Berliner's play changes, his match up percentage increases dramatically and he often chooses the engine 1st choice. Berliner never blundered in any of these games although he did make mistakes, but these were not game losing mistakes unlike those of his opponents. Engine like? I don't think so but i have no doubt others will disagree.
Onwards then, to the age of engines.
50 Year Jubilee World Champions' Tournament.
This was a tournament played between past CC World Champions. It is important to remember that Berliner was 74 at the time of this tournament.
8 games played by Berliner. Won 1, drew 5, lost 2.
288 moves total.
1st choice matches: 220
2nd choice matches: 18
3rd choice matches: 6
Total matches: 244
Match up percentage: 85%
Again, this is quite high and is higher than in 1965, an again I found watching the games and the analysis instructive.
The first thing I noticed is that Berliner's opponents do not make mistakes for him to capitalise on anymore. The games tend to stay in that state of almost equality for many moves without really varying. I can think of two reasons for the lack of mistakes. First, this may be a consequence of having world champoions play each other. Maybe what separates a CC World Champion from the also-rans is simply the lack of mistakes. Second, maybe we are seeing evidence of engine use.
The second thing I noticed was that Berliner match up percentage in the sort of quiet position that dominates these games has shot up dramatically from 1965. He is now matching HIARCS' choices far more often whereas in 1965 in these quiet, equal positions he was only matching about 50% of the time. It is also notable that his highest match ups (both 92% ) happened in the two games he lost. I doubt this indicates total reliance on an engine since he is still making minor errors, and some that are major enough to lose. However, I strongly suspect he was making use of his own Hitech system.
So, make of that what you will.