Just posting this here for visibility.
Here is the short version:
Even if no lights are needed when rendering geometries the WorldLightList still gets filtered. At least the *ShadowRenderer suffer from this.