You can definitely use papiermache with electric lights as both bases and as lampshades.  Although the books I have looked at never actually specify what is used as a fire retardant, although usage is mentioned.

Realistically the heat output from a night light, is pretty low, the resistance needed to produce the glow is minimal and as a result the element doesn't get white hot.  With a long life bulb the heat output is even less, the glow is produced using a powder system and not forcing electricity along a resistant element.

HOWEVER.  I would be very wary of applying PM to a plastic night light shade or removing the shade and adding your own, although there is little chance of fire it is not beyond the realms of possibility.  If the worst was to happen with gift you had made ..........

What I would be inclined to do instead is approach it from a different angle, why not make a night light screen,  Construct a frame using dowelling or thin wood, Make an insert for each section of the screen with opaque card or pm laminate and cut out a scene/shapes in each one.  Put the screen around the night lights so that you have enough light to illuminate the holes.  I would aim for a square frame with four screens that have enough distance from the light source to prevent any issue over safety.  They could be put on a table in the childs room.