InputHandler usage

I'm certain this has been done before, but I'm struggling with it.  Say I have an InputHandler with children InputHandlers.

I only want the children InputHandlers to be called if the parent InputHandler did not handle the action.  Is there a

nice easy way to do this, or will it involve overriding key (as in important, not keyboard) methods in the InputHandler class?

Attached InputHandlers are not meant to work this way. What about using a chain of responsibility application side?

That's what I was afraid of.  I'm trying to abstract away as much of the handling away from the application as possible,

but it doesn't look like that's going to happen.  Thanks, irrisor.