Not everybody will repsect the vacation flag. They believe that with the time bank system people shouldn't accept games with time banks that they can't play to. They believe that when the flag falls you lose.
Also, people have been known to abuse the flag to halt for time so some people are now very warey of it. I don't think you can dispute it, but you can take it up with the admins if you like. What was the game ID?
I'm afraid that the vacation flag does not prevent people from claiming victory over you if your alloted time frame expires. This is especially true in tournament games where automatic timeout win will happen two days after your time is up, whether your opponent wants it or not.
Consider it a lesson learnt.