This problem is solved by having paper towels (air dryers suck) and placing the paper towel trash bin next to the door so that you can use your hand-full of towels as a barrier between the handle and throw the towels away as you leave.