[DevLog] Door Improvement - Editor Edition

In our previous game, placing doors was relatively difficult. The reason it was difficult was because you had to create a hole in the wall before placing a door.

The pain of placing doors and windows

Since our game is tile-based, holes also had to be made in tile units.

However, doors are not sized exactly to fit a tile. In other words, the door image inevitably had to include graphics to fill the wall gap.

Also, to finely adjust the position of a door, we needed another version of the image with a different background. To support placing a door offset by half a tile, we had to create both versions like above.

Still, we developed while enduring this hassle. Game development produces mountains of issues, and unless they’re critical, we often overlook them.

The door issue that finally stood out

Walls implemented not with tilemaps, but with mesh renderers were added to our game. And we needed to attach doors to these walls.

While tilemaps let us make holes on a grid, meshes require more complex calculations. We’d even need to create a tool for carving holes. So we started looking for a simpler method.

Improvement 1

Through some research, we found people use stencil buffers in such cases. Stencil buffers are special rendering buffers used for masking, allowing certain areas to be rendered or not.

In the example above, only areas marked with 1 in the stencil buffer are rendered. Unity's ShaderLab supports the stencil feature, so we used it to modify our door like this:

We used a texture for the darker area of the door, masking it with stencil so that the wall behind wouldn’t render. The door frame was floated slightly in front of the wall to avoid z-fighting. ←! It was a convenient method, and we used it for a while, but even this had its problems.

A new problem

Characters would sometimes render between the door and the wall. It didn’t happen often—it only occurred when the Y position was aligned just right—but it was very annoying as a developer.

This happened because there was a tiny actual gap between the frame and the wall. But if we pushed the frame closer to the wall, z-fighting would occur.

Improvement 2

So we created a new mask texture. This mask included not just the hole but also the frame area. In other words, we cut out the frame too.

This allowed us to place the frame flush with the wall without causing z-fighting. But it meant needing an extra texture, which was a bit inconvenient.

Then we came up with the idea of using the frame texture itself for masking. We painted the hole section of the frame texture with alpha 1. It's barely visible to the eye, but since it’s not fully transparent, the hole still gets cut.

Final Result

All the hassle was resolved, and we can now place holed objects freely and conveniently.

Thanks for reading!