Yes, I've used the "cancel payment" regularly when guests add a night and I don't want to have them pay their last payment... and then ONE MORE payment after that. So I cancel the outstanding payment and create a new one. Guest gets an email saying the cancelled payment was cancelled; they get another one immediately with the new payment request even if the request is due in 6 months, so I've learned to add some verbiage to the description to make sure the guest knows they'll be reminded again 2 weeks before the payment is due.
Haven't done a "mark as paid", but assume it's similar in that the guest is no longer bugged about that payment.
I have used the mark as paid extensively since my payment terms were pay at check in and it worked every time without a problem.
If you look at your payment overview you can see who paid how much when and if it is one of two or three payments. If someone brings cash or check you can mark this as paid and they will show as fully paid for VRBO and you.
Now with the new system you have to have at least 25% down payment. However, since the fee is taken right away from that you still could have the rest at check in or when you have guest arriving and checking out before VRBO even sends the first payment.