Imaging Sonar Game Prototype

Depth Panorama

I needed to generate a 360° panorama.
Using a Scene Capture Cube would not allow for independent control over horizontal and vertical resolution. It was also too expensive as it renders a full 360° view while I only need to update a small slice each frame.

Instead, I captured the panorama incrementally by rotating a narrow-FOV Scene Capture 2D and stitching the slices together.

However, the default Scene Capture 2D does not support independent control over horizontal and vertical FOV. To solve this, I implemented a custom Scene Capture 2D with a modified projection matrix, enabling full control over both axes.

The slice FOV needed to be narrow because higher FOVs introduce noticeable perspective warping toward the edges. This distortion creates visible seams between slices and at high FOVs, can make the final image difficult to read.

I found a horizontal FOV of 2 degrees to be a good balance between coverage and image quality.

I looked into using Panini projection to reduce distortion at higher FOVs, but I was already happy with the results so I chose to focus on the topographic map and passive sonar instead.

Separating horizontal and vertical distance

When using standard depth, the distance is taken along the view ray. This means that deeper areas of the ocean floor appear farther away, even if they are directly below the submarine.

For a top-down view, this creates distortion. Points that are deeper are pushed outward when sorting pixels by distance. Standard depth also does not give enough information to determine how deep a point is relative to the submarine.

To address this, I separate horizontal and vertical distance into two channels using a post-process material. I get the vector from the camera to each pixel in world space.

I store the horizontal distance, which is the length of the XY components, in the red channel. I store the vertical distance, which is the Z component, in the green channel.

By doing this, I can sort pixels based purely on horizontal distance from the submarine, producing an undistorted top-down view, while using the vertical distance for the color.

I divide both distances by the maximum sonar range to normalize them.

Turning the panorama into a top down image

I already have the data I need in the panorama, but it isn’t arranged as a top-down view. So I rearrange it into one.

I do this by sorting the pixels based on their horizontal distance, stored in the red channel. Because this value is already normalized, I can use it directly to set each pixel’s V coordinate.

For each pixel, I add 1.0 / PanoramaVerticalResolution to the red channel and store the vertical distance from the panorama in the green channel.

Using 1.0 / PanoramaVerticalResolution means the final red channel values would still be similar if I were to change the panorama resolution.

When multiple pixels map to the same coordinate, their red values accumulate. This naturally highlights surfaces like walls or steep slopes, where many pixels share the same horizontal distance.

For the green channel, I keep the lowest vertical distance from the pixels that end up at the same coordinate. This saves the information about the point closest to the submarine.

To make the top-down view intuitive, I warp the UV into a disc shape which puts the submarine at the center. This way, distances radiate outward from the center, making it easy to understand what’s forward, behind, or to the sides of the submarine.

Color and final touches

The resulting image contains all the information we need, but it’s not very readable or visually clear. To make it easier to interpret, I put the green channel through a color ramp to create a red->green->blue gradient. Red indicates heights similar to the submarine and potential collision risks, while green and blue show safer distances. I then use the red channel to control the emissive strength, highlighting slopes and walls.

When the submarine isn't moving, sonar updates show very similar data, making it hard to see what’s being updated. To highlight the current update area, I add a rotating fade using the circular U coordinate to reduce emissive strength over distance. Pixels fade out over time and become bright again once the pixel is updated, giving visual feedback on the area of the map that's being updated.

Generating the map texture

I generate a topographic map so the player can use it together with the imaging sonar to figure out their position relative to the terrain.

To do this, I take the landscape heightmap and process it in a compute shader. I also send in parameters for the contour line spacing, index line spacing, and the thickness of the lines.

To place the contour lines, I step the height values based on the contour line spacing so I can detect edges between levels. For example, with a spacing of 1.5 meters, the values become 0, 1.5, 3.0, 4.5…

To detect where lines should appear, I sample neighboring pixels within a radius defined by the line thickness. If a neighbor has a different stepped value,an edge has been found and the pixel contributes to a line. The closer the neighbor is, the stronger the contribution, and I keep the highest value. This produces a soft edge that effectively works as anti-aliasing.

I generate both contour lines and index lines at the same time so I can reuse the sampled values and avoid duplicate work.

Index lines use a larger step interval, based on a multiple of the contour spacing. The contour lines are stored in the red channel and index lines in green.

Since I already have the data, I also store the stepped heightmap and the original heightmap in the blue and alpha channels. This gives me flexibility later when making the display material for the map, without needing to go back and change the compute shader.

Heightmap Resolution Issue
The original heightmap is lower resolution than the final map texture, which makes the source resolution visible in the result. Without any changes, the contour lines appear blocky and follow the original pixel grid.

To reduce this, I sample a few neighboring pixels and average the result to blur the data slightly. This removes the stair-stepping effect, giving me smooth lines.

What is is passive sonar

The passive sonar is a waterfall graph that shows the direction towards certain objects in the world. The idea is that these objects would be points of interest or dangers like torpedoes.

On the graph the vertical axis is time, where new information is at the top and scrolls down with time.

The horizontal axis is the angle to the object. 0° is the center of the center and 90° right is the right edge of the image.

In the video, you can see the submarine ( Green Box ), 3 targets ( Red ) and the passive sonar waterfall graph that’s tracking the 3 targets.

How the passive sonar works

I use noise to give a hint about the distance from the objects.

I send the submarine position and object positions to a compute shader and calculate the angle to each object to place the signal along the horizontal axis.

At that position, I add noise to the top row of the texture. The noise is strongest at the object’s angle and fades outward from that point. The spread depends on distance, so closer objects produce tighter and stronger signals.

Each frame, the texture scrolls downward to create the waterfall effect. I also add a layer of ambient noise so the display remains active even when there are no strong signals present.