Hi @TooMuchJava,
just a guess cause your problem description is lacking some information for good assumptions.
If you use the newest version of lemur (V.1.11) then in some cases the mouse ray is calculated incorrectly - thus not reaching any lemur element.
Check: Buttons and DragHandler without effect after upgrading to Lemur 1.11 - #17 by Aufricer - the last entries of the threat
Otherwise you may need to give more detailed explaination of your problem to get more advice.