Yep. That’s what makes restitution so hard. I’ve read a couple of game physics books at this point and most of them spend the majority of text on this very issue. Collision detection is largely considered a ‘separate issue’ and indeed I have a book entirely on that subject.
The thing is collision detection is basically a solved problem… it’s just really hard in the general case. So you may have an entire book on doing it efficiently.
Contact restitution is not a solved problem. There are several distinct approaches and each have their tradeoffs. For example, do you just push things out as you’ve done? (and basically never be able to stack anything) or do you rewind time to the first estimated collision, resolve it at the collision time, then continue forward until the next collision, then repeat? (and deal with the performance issues, potential to never resolve, etc.)
…or some hybrid of the two approaches.
Or do you treat all of the contacts as constraints in some giant combinatorial matrix and use magic to solve them simultaneously. (Jacobian matrices?)
My mphys library (soon to be released I hope) is using the first approach (push everything out based on penetration) but it sums those “pushes” per object per frame. So if a ball is flying directly into a corner (simultaneous contacts) it doesn’t shoot out at a weird angle as the two constraints will average out to a single push. It has its own down sides but it made things a LOT more stable. Prior to that change, it was impossible for a table to land on all four legs. One leg would win the constraint and flip the table at an odd angle.
I still can’t stack things more than two objects high without ugly jitter… but I have ideas for solving that, too.
You will spend your life on contact solvers… and it will be especially frustrating because invariably you will find that your contact solver fails sometimes because of bad inputs from your collision detection. (Hint: when a sphere hits the edge of a box, what normal should be used? It turns out to be very important…)
Edit: on this last point, I ended up setting up a special test program where I can load wireframe shapes and maneuver them into each other… contact information is then displayed as arrows and lines of penetration. Being able to microtest each and every edge case (sometimes literally ‘edge’ case) has been sooooo useful.)