How to detect engine cheats - a guide

How to detect engine cheats - a guide

Only Chess

Cookies help us deliver our Services. By using our Services or clicking I agree, you agree to our use of cookies. Learn More.

D
Losing the Thread

Quarantined World

Joined
27 Oct 04
Moves
87415
09 Nov 09

Originally posted by rja211077
hence the need for a large sample of games (i recall squelch recommended 20+) - but i expect Russ would want more for banning purposes.

as others have pointed out, it's fairly easy to be very accurate in non-complex positions.

btw - the "birthday paradox"?!
In order for the chances of two people in a group having the same birthday to be bigger than one half you need 23 people. A naive guess would assume that you'd need 175 or so, consequently it's known as the birthday paradox.

It's caused unsafe convictions in murder cases when mothers were convicted after having more than one child die of cot death, Search for Sally Clark on Wikipedia, they give the details there. The Royal Statistical Society made a statement about this which you can look at here: http://www.rss.org.uk/main.asp?page=1225

The scope for error in engine cheat detection is lower, and less serious, they don't go to gaol. Even so I've got a slight concern that raw match-ups with engines may not quite tell the whole story - the chance of an honest player hitting an engine move on one move is dependent on whether they hit on the previous move - for example if there is mate in 2. Over many games you can assume this sorts itself out, but the basic statistical model seems to assume that each move is an independent trial. I don't know how much of a problem this is, just that it could present difficulties. I kind of assume the site admins / games mods have thought of this stuff and fixed any potential statistical problems.

As Squelchbelch noted many engine users find their use running away with them so that if you are suspicious about a user now but can't prove anything what you need to do is wait for their engine use to run away with them. Also, occasionally, it's possible to tell what engine a player's using, if you can do that there's no doubt,

S

Joined
14 Jul 06
Moves
20541
09 Nov 09

Originally posted by nimzo5
very interesting thread, however I think for the sake of really getting a good anti cheat system the following should be considered;

1) it seems that using capablanca, alekhine or any of the pre computer masters would give you lower than normal average % of matching moves since classical principles and lack of appreciation of dynamic qualities of games (so ...[text shortened]... nd worth using further since anything to put in check computer engine use would be welcome.
1) The pre computer era CC World Champions had months to complete games.
They played to the best of their (human) abilities.
What reasons are there why modern CC players often play so much more like engines & less like humans & have much higher matchup rates than these pre-computer era greats?

Humans and engines play very differently at a basic level.
For instance, engines go for a clean win following a very sharp line, whereas even the best unassisted human will go for the safe & steady option, if it at least maintains his position or seems to improve it.

2) the engine moves vs "home analysis" argument (or having a so-called cyborg database on your home system) is really desperate logic.
Ok, maybe for one or two games where you have studied particular openings, the matchup rates could be quite high, but 20+ games all selected objectively???
Highly unlikely that matchup rates would be high, unless of course your opponent is using an engine to generate plenty of top 3 engine choice moves.
In such a "cyborg database" case, Admin could easily request to see these thousands upon thousands of pre-prepared game files as evidence.

This argument requires a conspiracy of engine use by both players to find suspicion in a player's games selected over time.

3) I also match the engine suspects against similarly (highly) rated players as game selection criteria, so that the positions tend to be more balanced & have far less obvious top 3 moves.

4) This is why I have analysed 2 of the most recent World CC Championships which were guaranteed to have finished before the onset of strong chess programs.
The player's thresholds then were remarkably consistent, both with each of the pre-computer CC Wc's and also the Super GM WC's that I analysed.
It seems that even with modern preparation, Kramnik & Topalov can only achieve very much "human" results, according to my thresholds.
These are in stark comparison with the engine matchup rates of some modern CC players!

5) CC matchup rates post 1990 are going to be less use in determining engine use thresholds, simply because in modern CC tourneys engines are frequently used!
I have no idea what point you are trying to make here & your rationale is.... interesting!

n
Ronin

Hereford Boathouse

Joined
08 Oct 09
Moves
29575
09 Nov 09

don't misunderstand I think this is a good way to analyze computer use. I am just sounding out on a couple assumptions this method makes and wondering how it might skew the results.

S

Joined
14 Jul 06
Moves
20541
09 Nov 09

The idea is to
a) find the most blatant and annoying cheats
and
b) reduce (as far as possible) false positives.
I think the original post in this thread & also my supplementary answers show that top 3 matchup - done under the strict criteria already set out - is a very good way of achieving these objectives.

t

Joined
05 Jan 05
Moves
42096
10 Nov 09

I'd look at their rated record versus their blitz record against rated players. In blitz you don't have time to run an engine on 0-5 second per move under 5 minutes. If they are highly rated in tournament play then it should translate to blitz.

h

Joined
25 Apr 06
Moves
5939
10 Nov 09

Originally posted by tcvegas
I'd look at their rated record versus their blitz record against rated players. In blitz you don't have time to run an engine on 0-5 second per move under 5 minutes. If they are highly rated in tournament play then it should translate to blitz.
Nonsense, do you think online blitz is in any way comparable to online corr 😛

w
If Theres Hell Below

We're All Gonna Go!

Joined
10 Sep 05
Moves
10228
10 Nov 09

Originally posted by tcvegas
If they are highly rated in tournament play then it should translate to blitz.
it doesn't. and if there were more bold enough people outing their crap blitz (or other) ratings instead of hiding them, it would be common knowledge.

nobody wants to parade their beer gut or bingo wings, instead they prefer to look good. only the ones who do well flaunt their blitz ratings. the rest prefer to stay quiet, and some even flat out lie about their ratings.

Joined
25 Jul 07
Moves
27760
10 Nov 09

Originally posted by wormwood
it doesn't. and if there were more bold enough people outing their crap blitz (or other) ratings instead of hiding them, it would be common knowledge.

nobody wants to parade their beer gut or bingo wings, instead they prefer to look good. only the ones who do well flaunt their blitz ratings. the rest prefer to stay quiet, and some even flat out lie about their ratings.
Totally agree, and I can out myself to support the point. My FICS blitz rating is currently around 1100, best 1274. My current FICS standard rating is 1700, but I haven't played on FICS for ages as I play here now. My rating current rating here is 1880. I wouldn't claim that was good by any means, but it's significantly more than 1100.

Basically I don't play well when I play fast. I don't have time to idiot-check every move, and I make lots of silly mistakes. In OTB games my main problem used to be time trouble, but I'm (hopefully) getting better now ...

e4

Joined
06 May 08
Moves
42492
10 Nov 09

Agree that playing blitz over the net is a poor guide. Well I hope so.

On Uchess I'm Beatle currently 1517 Won 222 lost 86. D2.
On the other site I play I've been 1912 but dropped back to 1640 because I think
I'm great at blitz when I've has a few.

Much better at OTB blitz, though age is telling after a few pints.
In 1987 I won a strong blitz tournament in Edinburgh. It made the newspapers
and I was famous for a day.

I use to love my OTB blitz sessions. Remember playing Danny Kopec all
Christmas day in 1982. That made us both very popular with Mrs greenpawn.
(DK won most).

I also know, and I'm sure other players do as well, some very good players
who are quite bad a blitz, it just does not suit them. It means nothing.

Cheats:
Some are so blatant you get the impression they do it just for a laugh.
I'm sure the no profile, non-posters do.

n
Ronin

Hereford Boathouse

Joined
08 Oct 09
Moves
29575
10 Nov 09

I have always been stronger at classical time controls- currently there is about a 400 point difference between my ratings. Part of it is the willingness to sling pieces after a few (or many cocktails), part is my choice of openings (closed positions) and lastly the simple fact that I dont calculate as fast as I did 10 years ago.

That being said I would think a typical player would be a lot closer, especially a young one.

w
If Theres Hell Below

We're All Gonna Go!

Joined
10 Sep 05
Moves
10228
11 Nov 09

here's a GM with 1700 blitz rating on ICC:

http://www.chessclub.com/finger/cendrier

for those who are not familiar with ICC ratings, nakamura is currently 3400 in the same exact blitz pool.

h

Joined
25 Apr 06
Moves
5939
11 Nov 09
1 edit

Originally posted by wormwood
here's a GM with 1700 blitz rating on ICC:

http://www.chessclub.com/finger/cendrier

for those who are not familiar with ICC ratings, nakamura is currently 3400 in the same exact blitz pool.
Did he have some brain damaging accident 5 years ago? (his rating top is 2300+)

w
If Theres Hell Below

We're All Gonna Go!

Joined
10 Sep 05
Moves
10228
11 Nov 09
1 edit

Originally posted by heinzkat
Did he have some brain damaging accident 5 years ago? (his rating top is 2300+)
I have no idea. I saw someone say he's quite old, but have no idea if it's true. and even the 2300 in the normal ICC blitz pool isn't that high for a GM either. I've seen other GMs in the 1800s and 1900s as well, although I don't remember if it was in the blitz or 5-minute pool.

there are surprisingly many titled players with no blitz rating at all there, even though they're frequently online. I suspect many of them are simply bad at speed chess, and don't want people to see it. as it would probably affect their coaching business as well.

Chess Librarian

The Stacks

Joined
21 Aug 09
Moves
113630
11 Nov 09

Originally posted by nimzo5
very interesting thread, however I think for the sake of really getting a good anti cheat system the following should be considered;

1) it seems that using capablanca, alekhine or any of the pre computer masters would give you lower than normal average % of matching moves since classical principles and lack of appreciation of dynamic qualities of games (so ...[text shortened]... nd worth using further since anything to put in check computer engine use would be welcome.
I think you made some excellent points. I find the fact that someone would question your motives to be a little unnerving.

I recently had a game where I played on a little too long in a drawn position because I was following some general Glenn Flear analysis on two minor pieces vs rook with two pawns each on the same side of the board, and it took me a bit to see that the only pawn I could pass <insert joke here 😉> was my h-pawn, and I had the wrong color bishop.

In retrospect, I played on longer than I should have. Based on this thread, my opponent would probably assume I was using a computer, which is not true.

A good cheat system will view statistics as a starting point, not as a conclusion.

Paul

n
Ronin

Hereford Boathouse

Joined
08 Oct 09
Moves
29575
11 Nov 09

Originally posted by wormwood
I have no idea. I saw someone say he's quite old, but have no idea if it's true. and even the 2300 in the normal ICC blitz pool isn't that high for a GM either. I've seen other GMs in the 1800s and 1900s as well, although I don't remember if it was in the blitz or 5-minute pool.

there are surprisingly many titled players with no blitz rating at all there ...[text shortened]... don't want people to see it. as it would probably affect their coaching business as well.
Many international titled players use icc to watch games and talk to friends not to play chess. Not sure now, but they used to get a free account, also many use a secret account to test new ideas vs other strong opposition