Just noticed this project and I must say great job!
Assuming this uses bullet for the physics it’s gotta be quite similar to what I implemented for LSF ship destruction a while back. That usecase didn’t need any texturing work though, since the modules were batched together discrete custom meshes.
If the wireframe display is the physics debug appstate you currently have hundreds of boxes as colliders. Have you considered using a greedy algorithm to convert the cubes into big rectangles? That way a giant wall with no holes would be a single box collider.
That brought me a far bit more performance I think, but it needed to be done in an executor thread to avoid hitching during battles.