There are 3 different ways to create overlays in Protowire.

Quick Overlay

With this overlay you put the design on a separate artboard. This overlay can be shared with multiple artboards. Create a link to the overlay artboard and select Overlay in the popover. It automatically adds a dark background which you can change the opacity of. Setting the opacity to 0 will hide it. Checking "Is a Toggle" will cause the trigger layer to open and close the overlay as a toggle. If you want a separate layer to close the overlay, create a link and select the Close button that appears at the top. You can link overlays to other overlays. If the overlay is the same size and position as the previous one it will replace it rather than stacking it on top.

Layer Animation on a Separate Artboard

The next 2 approaches give you move control over how the overlay is animated. In this approach the layer animation is on a separate artboard so it can be shared by multiple artboards. Turn the artboard that the overlay will appear on into a symbol. Add a symbol instance of this symbol to a new artboard. Add your overlay on top of this. Create a link from the non-overlay artboard to the overlay artboard. Having a symbol instance of an artboard tells Protowire that you want to create an animated overlay and will show you the animation popover. In the popover you would select "End State" for "Current design is". And then in the animation editor you would move your overlay to its start state using the Off-Artboard button to have it just on the edge of the artboard. To have a close button close the overlay, you could create a link within the animation editor and link to the start state. Editing the animation from any of these artboards will edit the same animation.

Layer Animation on the Same Artboard

You setup a layer animation like the previous approach but you do it on the same artboard that triggers it. This overlay can't be shared by multiple artboards. Design your overlay and and create a link to trigger the animation by selecting the Animation button that appears at the top. Animate the layers like the previous approach. You can hide the overlay layer so you can work on the rest of the design and it will still appear in the prototype.