Andrew, I could be wrong, but I believe the draw offer needs to be acted upon right away. If you accept, the game is drawn. If you decline by making another move, the offer is gone, and after that either party can make a new offer when it's their turn to move.
I'm fairly sure you can't go back after making a few more moves and decide to accept an earlier draw offer.
Did I read your post correctly? I'm sure I'll hear about it if I have this wrong !!