Originally posted by PhySiQ
If its at all possible.. I'd like to get my hands on the software to help find cheats. I'll probably be removed from the site shortly after getting it - but I'd be willing to stir the martini.
It sounds as if it needs a real good stir.
Q
Erm to steer this back on topic, in answer to your question...
I won't distribute Batch Analyzer to anyone else purely because it was given to me on that basis.
Typically the suspect will be one of the top 1st percentile highest rated.
What happens is I select 20 of the suspects most recently completed games vs higher rateds (here that would be 1900+) which all have 35 or more total moves.
Long games vs higher rateds should ensure plenty of fairly balanced positions, where engine move selection is more telling.
This objective selection makes the process reasonable & fair.
I paste the 20 game PGN's in to Batch Analyzer.
The analysis takes between 10 & 20 hours, depending on the non-database move sample size.
During that time, my pc is running flat-out on all cores. To stop it overheating, I run the batches overnight, with the window open & a stand-up fan pointing straight at the pc tower.
Next day you generally either have blatantly high match rate results or not. There are very few "in-betweens".
Ideally you need several of these volunteer analysts.
That's not the end of the story though. You then need legitimately strong OTB players (FM+ level ideally) to look through the games to find smoking-gun engine play.
David Tebb & Northern Lad are 2 that spring to mind. There are only a couple of 2100+ FIDE rateds on the site, and they seem to have lost interest.
Hopefully Russ had a Page Viewer Log running, whereby he can access all the times & dates of a user's activity whilst on his site. That can be extremely helpful if coupled with high engine match rates in many games over time.
If all this in in place, then booting obvious engine users is like shelling peas.
If any one part of this process fails then you may as well just allow an engine free-for-all, with laughable leader boards, very few decent real-world players & pointless tournaments apart from banded ones below the intermediate level.