I would go on with it-- even if he says his finger slipped or
something, you were already ahead in the game. It seems unfair that
you should have to take a draw because he was careless.
If his finger didn't slip, and he just didn't see where his move was
leading, there's absolutely no issue: the game should go on, and his
asking for a draw is out of bounds.