The physics space ray test will only perform ray tests between physical objects. If you have objects in your scene graph that aren’t also in your physics space and you want to ray test against them, you’re out of luck if you use this method. Beyond that, I’m not sure if Bullet supports any sort of ray test filtering.
I’ve only used the physics space ray tracing a handful of times, but if I remember correctly, using the physics ray test can have slower performance compared to using the standard ray collision. I should add that in my situation, I was casting multiple rays every frame, so this could be why I was noticing a significant difference.
Edit: Also here’s a link to a thread discussing a few different ways to filter your colllidable objects Selective Raytracing