So, to show something as modal (note: the element must be currently hidden):
[java]
Element.showAsModal(boolean showWithEffect);
[/java]
To set the background properties:
[java]
screen.getModalBackground().setColor(ColorRGBA color, boolean useSeparateAlpha);
screen.getModalBackground().resetColor(boolean useSeparateAlpha);
screen.getModalBackground().setAlpha(float alpha);
screen.getModalBackground().resetAlpha();
[/java]
There is nothing else to do, the modal background is managed on element hide, etc.