How to detect engine cheats - a guide
22 Jun '09 10:022 editsAdvantages of top 3 matchup analysis:
1) There is a significant body of statistics which illustrate what is humanly possible both from pre-computer era CC World Championships and also Super GM OTB games. The upper-end thresholds are remarkably consistent.
2) All you need to do high quality analysis is a reasonably powerful pc, a decent modern engine and access to an online database such as www.chesslive.de
3) The method itself is very simple.
The 30 second top 3 matchup allows for reasonably practical analysis (ie games don't take 3 weeks to do!) but also yields very high quality results, in keeping with those achieved by many other Games Moderators.
You can go for instance 60 seconds per move, but I tried this with several games and results were virtually identical to 30 secs but you spend many hours on 1 game!
4) All non-database moves are included. You don't need to remove moves you subjectively consider are forcing or obvious. 'Only legal' moves are also taken into account in the threshold stats on post #4 of this thread.
Any argument that suggests that these moves should be exempt from final figures ignores the fact that they frequently appear in pre-1980's CC WC & Super GM OTB matches. Try analysing Capablanca-Alekhine 1927 or Fischer-Spassky 1972 and then tell me there aren't plenty of forced exchanges.
The 'theory' as such is based on a false premise, unless you think all the top player legitimate games have an un-representatively low amount of these moves!
Disadvantages are:
1) It takes about an hour or more to analyse and write up one game. 20+ games are needed as evidence, so, unless you are chained to your pc this will take a week or 2 to collect data on one suspect.
How to select a suspect for analysis
With the effort and time needed, you want to make certain you have grounds to suspect the user you are investigating may be using an engine on a consistent basis in their games.
It is logical to go for the most blatant cheats, as these should be the easiest to detect. Some people will only use an engine on rare occasions - these people cannot be caught with the top 3 matchup method.
There are some give-aways:
-They are within the top 2 % or so of the highest rated players on the site. A controversial point, but proven by experience. This is where the blatant cheats live.
-In their earlier games they played like a patzer losing to 1400 rateds. A few month's later they are playing like a GM, beating 2300's.
-The number of games in progress and the move frequency in these games. It could be that you have stumbled upon a Super GM, dishing-out a free, unofficial online blitz simul. This is probably quite unlikely and even so, analysis can prove if this is a GM or not.
-No losses in many completed games against all-comers. Engines don't get tired and very rarely make blunders. Humans do. The lack of mistakes in many games over time is a key indicator of engine use.
-They cannot realise when a position is clearly drawn and so play on. Their engine says that they have a 0.25 advantage in the position, so of course they attempt to go for the non-existent win.
-They repeatedly go for sharp winning lines in games where there are clear and simple ways to maintain the advantage by simplifying or playing 'safe & solid' human-like moves.
-The engine seems to have no plan in closed positions. This only really applies to the weakest engines these days. Many of the moves that modern engines play in these positions are similar to those which strong human players would choose.
Since we are going for the most blatant cheats, limit yourself to players in the top 1 or 2% highest rated. There is little point going after an 1800 with a suspicion they may be getting 'help'.
So, you now have a suspect. Next up is the crucial; which games are you going to analyse?
1) There is a significant body of statistics which illustrate what is humanly possible both from pre-computer era CC World Championships and also Super GM OTB games. The upper-end thresholds are remarkably consistent.
2) All you need to do high quality analysis is a reasonably powerful pc, a decent modern engine and access to an online database such as www.chesslive.de
3) The method itself is very simple.
The 30 second top 3 matchup allows for reasonably practical analysis (ie games don't take 3 weeks to do!) but also yields very high quality results, in keeping with those achieved by many other Games Moderators.
You can go for instance 60 seconds per move, but I tried this with several games and results were virtually identical to 30 secs but you spend many hours on 1 game!
4) All non-database moves are included. You don't need to remove moves you subjectively consider are forcing or obvious. 'Only legal' moves are also taken into account in the threshold stats on post #4 of this thread.
Any argument that suggests that these moves should be exempt from final figures ignores the fact that they frequently appear in pre-1980's CC WC & Super GM OTB matches. Try analysing Capablanca-Alekhine 1927 or Fischer-Spassky 1972 and then tell me there aren't plenty of forced exchanges.
The 'theory' as such is based on a false premise, unless you think all the top player legitimate games have an un-representatively low amount of these moves!
Disadvantages are:
1) It takes about an hour or more to analyse and write up one game. 20+ games are needed as evidence, so, unless you are chained to your pc this will take a week or 2 to collect data on one suspect.
How to select a suspect for analysis
With the effort and time needed, you want to make certain you have grounds to suspect the user you are investigating may be using an engine on a consistent basis in their games.
It is logical to go for the most blatant cheats, as these should be the easiest to detect. Some people will only use an engine on rare occasions - these people cannot be caught with the top 3 matchup method.
There are some give-aways:
-They are within the top 2 % or so of the highest rated players on the site. A controversial point, but proven by experience. This is where the blatant cheats live.
-In their earlier games they played like a patzer losing to 1400 rateds. A few month's later they are playing like a GM, beating 2300's.
-The number of games in progress and the move frequency in these games. It could be that you have stumbled upon a Super GM, dishing-out a free, unofficial online blitz simul. This is probably quite unlikely and even so, analysis can prove if this is a GM or not.
-No losses in many completed games against all-comers. Engines don't get tired and very rarely make blunders. Humans do. The lack of mistakes in many games over time is a key indicator of engine use.
-They cannot realise when a position is clearly drawn and so play on. Their engine says that they have a 0.25 advantage in the position, so of course they attempt to go for the non-existent win.
-They repeatedly go for sharp winning lines in games where there are clear and simple ways to maintain the advantage by simplifying or playing 'safe & solid' human-like moves.
-The engine seems to have no plan in closed positions. This only really applies to the weakest engines these days. Many of the moves that modern engines play in these positions are similar to those which strong human players would choose.
Since we are going for the most blatant cheats, limit yourself to players in the top 1 or 2% highest rated. There is little point going after an 1800 with a suspicion they may be getting 'help'.
So, you now have a suspect. Next up is the crucial; which games are you going to analyse?
Back to Top