Originally posted by DawgHaus
As I recall GMs typically match post-opening-book engine moves 45-65% of the time, depending strongly on the type of position (Tactical vs. Positional, etc.). Assuming that's the range, a match percentage of 80% for one game (let's say) would strongly indicate cheating. Alternatively, an engine-match percentage of 68% over 100 games may be equally damning ...[text shortened]... t a certain point, it becomes silly to claim that the match percentage could be due to chance.
Leaving aside how statistics works and whether it is an exact science or not, I strongly suspect that the deciding factor is not a high match up with an engine. There are several reasons for that but the primary one must be that engines simply don't make moves that match each other! There are differences in software and hardware to take account, differences which are effectively unknown to any detection system. To my mind detection should depend on a low match up in certain positions.
In some positions the move is actually or effectively forced so a high match up should be expected.
There are positions, often highly tactical in nature, where calculation will reveal the best move or moves and we should not be surprised if strong players (the Soviet system of chess training relied heavily on calculation of concrete variations) and engines find the same moves.
Then there are positions where nothing is obviously forcing or winning or necessary, where a good human player will choose from a small range of moves that are reasonable but will not often make superficially good moves that subsequently turn out to be bad. In short, they have a kind of positional intuition based on experience. Weaker players and engines do not possess this intuition and so make moves that a strong player would never make. In these positions there is a good chance that strong players will match each other often but weaker players and engines will often fail to find a sensible move in the critical position.
This must reduce the load of the game mods considerably since they only need to look at certain types of position and can ignore the rest. It would also produce the "stand out like a sore thumb" criteria for banning. You would in effect have an apparently strong player (according to rating, wins or whatever) playing like a patzer in those positions where there are one or two obviously reasonable moves. They would be banned for a low match up in those positions rather than a high match up. This would also explain why it takes so long to catch the cheats, they have to have built up a sufficiently large number of games with the required positions in them before the statistical method can even begin to be applied.