Equirectangular image editing: region removal using graph cuts

I've been doing some work recently in editing equirectangular 360 panoramic images. Specifically, using graph cuts (max-flow/min-cut) to find a good way to remove a slice of the sphere. This allows the removal of large or small slices, so anything from small pieces of equipment to entire crews could be removed. However, as it uses a graph cuts technique to find a good cut, there are certain types of image for which it is more likely to produce good results e.g. outdoor, natural scenes. For my tests I've been using forest scenes, like this creative commons one created by Peter Gawthrop:

I've written some software that allows the removal of an image slice, available on my github. It's a Qt application that makes use of a number of other projects, specifically Graphcut Textures as implemented by Nghia Ho.

How it works

A 360 panorama is essentially a sphere. The viewer is at the centre of this sphere, onto which the image is mapped. If we wanted to remove an element of the panorama, we could remove a slice of the sphere and then stretch the remaining content to fill the hole. The issue here is that a hard edge will be seen at the cut point.

For example here's the original image, viewed as a rectilinear:

If we perform a simple rectangular cut on the equirectangular, we'll end up with a very noticeable, hard line where the cut took place:

Instead, we can specify a region we want to remove and then let graphcuts find a good seam to disguise this removal. In the tool (available above), we specify a region to remove as well as an amount we're willing to lose in order to find a decent cut:

The red region indicates the section to be removed from the panorama. The "overlap size" specifies how many additional columns can be used to find a good cut. Specifying more here means losing more columns, but provides more flexibility as the algorithm looks for a good cut. Here's the cut (in green) for the overlap in the above image:

This is the 400 columns to the left of the cut, overlaid with the 400 columns to the right of the cut. Max-flow/min-cut is then used to find the best seam between them, as described in the graphcut textures paper. Here's the result of the above cut in the original panorama, viewed as rectilinear:

This cut is fairly impressive. Some artefacts can be seen, for example the branch at the centre top of the image ends abruptly where the cut has taken place. However overall the effect is fairly good.

An issue is that a cut results in a stretching of the rest of the content. We've removed a fairly large amount of the sphere, so the rest of the image must stretch to avoid having a missing slice. In a forest scene, this doesn't introduce too many issues - the trees could plausibly be more squat than in the original. However in certain circumstances, for example if a person was in the scene, this may not be acceptable. In such cases, techniques such as non-homogeneous stretching or additive seam carving could be used to avoid scaling the salient elements.