Our agreement states we charge additional for extra cleaning if needed due to what we consider to be anything beyond normal cleaning to re-rent the house.
We use the damage deposit for that...never had a problem although we've sometimes gotten some pushback until we send photos that our cleaning person sends us. Document exactly WHAT the additional time is needed for. That's what the deposit is there for.....IMHO!
I take into consideration additional factors including how much was paid for the entire stay, how much trouble the guests were, and whether I would rent to them again. And one must keep in mind that to distinguish the difference between a normal cleaning and excessive, one may need to have experience, probably at least 10 guests who have come and gone, to say what is normal.
If the guests paid several thousand dollars I am far more likely to absorb the cost of extra cleaning than if it was under $1K.
The issue of the lanai door is not something I would hold against the guests. Even if there are instructions provided, one needs to assume that the guests will not read the instructions and guestbook, and basic functions such as door locks need to be idiot proof. If this door requires special instructions, I would post the instructions at the door where they cannot be missed. I know that signs can be tacky and would attempt to modify the door so operation of the lock mechanism will be obvious to someone who has never seen a sliding door before.
Probably 10% of my guests require extra cleanup time afterwards. I have always absorbed the expense. But never has the extra charge been $50, let alone $250. It sounds as if you can easily document that the cleaning time and expense were far greater than normal. You may be disinclined to rent again to someone who left your property a filthy mess. I would lean strongly toward withholding the funds from the security deposit.