SurfCut: FreeBoundary Surface Extraction
 11k Downloads
Abstract
We present SurfCut, an algorithm for extracting a smooth simple surface with unknown boundary from a noisy 3D image and a seed point. In contrast to existing approaches that extract smooth simple surfaces with boundary, our method requires less user input, i.e., a seed point, rather than a 3D boundary curve. Our method is built on the novel observation that certain ridge curves of a front propagated using the Fast Marching algorithm are likely to lie on the surface. Using the framework of cubical complexes, we design a novel algorithm to robustly extract such ridge curves and form the surface of interest. Our algorithm automatically cuts these ridge curves to form the surface boundary, and then extracts the surface. Experiments show the robustness of our method to errors in the data, and that we achieve higher accuracy with lower computational cost than comparable methods.
Keywords
Segmentation Surface extraction Fast Marching methods Minimal path methods Cubicle complexes1 Introduction
In this paper, we build on Fast Marching algorithms to create an algorithm for extracting the boundary of a surface from a 3D image and a single seed point, and a corresponding algorithm to extract the surface. We validate our algorithm on seismic images for extracting fault surfaces, which form surfaces with freeboundaries. This has wide ranging applications in the oil industry [9]. Although we validate our method with such images, our method is general and can be used to extract any simple surface with boundary from an image that contains noisy local measurements (possibly from an edge map) of the surface.
The contributions of this work are: 1. We introduce the first algorithm, to the best of our knowledge, to extract a closed space curve in 3D forming the boundary of a surface from a single seed point based on Fast Marching. 2. We introduce a new algorithm to extract a surface given its boundary and a noisy image that produces a topologically simple surface whose boundary is the given space curve. Both curve and surface extraction have \(O(N \log N)\) complexity, where N is the number of pixels. 3. We provide a fully automated algorithm using the algorithms above to extract all such surfaces from a 3D image. 4. We test our method on challenging datasets, and we quantitatively outperform comparable stateoftheart in freeboundary surface extraction.
1.1 Related Work
Surface Extraction: Active surface methods [10, 11, 12], based on level set methods [13], their convex counterparts [14], graph cut methods [15, 16], and other image segmentation methods partition the image into volumes and the surfaces enclose these volumes. These methods have been used widely in segmentation. However, they are not applicable to our problem since we seek a surface, whose boundary is a 3D curve, that does not enclose a volume nor partition the image.
Our method builds on the Fast Marching (FM) Method [3]. This method propagates an initial surface (for example, a seed point) within an image in the direction of the outward normal with speed proportional to a function defined at each pixel of the image. The end result of the method is a distance function, which gives the shortest path length (measured as a path integral of the inverse speed) from any pixel to the initial surface. The method is known to have better accuracy than discrete algorithms based on Dijkstra’s algorithm. Shortest paths from any pixel to the initial surface can be obtained from the distance function [2] (see also [17]). This has been used in 2D images to compute edges of images when derivatives of the image are noisy. A limitation of this approach is that it requires the user to input two points  the initial and ending point of the edge. In [4], the ending point is automatically detected. However, these methods are not directly applicable to extracting a surface forming an edge in 3D.
Attempts have been made to use minimal paths to obtain edges that form a surface. In [7], minimal paths are used to extract a surface edge topologically equivalent to a cylinder. The user inputs the two boundary curves of the cylinder and minimal paths joining the two curves are computed conveniently using the solution of a partial differential equation. Surface extraction with less intensive user input was attempted in [6]. There, a patch of a sheetlike surface is computed with a user provided seed point and a bounding box, with the assumption that the patch slices the box into two pieces. The algorithm extracts a curve that is the intersection of the surface patch with the bounding box using the distance function to the seed point obtained with Fast Marching. Once this boundary curve is obtained, the patch is computed using [7]. The obvious drawbacks of this method are that only a patch of the desired surface is obtained, and a bounding box, which may be cumbersome to obtain, must be given by the user.
An alternative approach to obtaining a surface along image edges from its boundary is with the use of minimal surface theory [8, 18]. It is argued that minimal surfaces are more natural extensions of the 2D shortest path problem to 3D. The minimal weighted area surface interpolating the boundary is obtained by solving a linear program. The drawback of this method is that the user must input the boundary of the surface, which our method addresses, and it is computationally expensive as we show in experiments.
Another approach for surface extraction, which does not require user input, is the approach by [1]. There, a local differential operator (based on the smoothed Hessian matrix) is used to compute the likelihood of a pixel belonging to the desired surface. Then, connected components of the maxima of this likelihood are computed, obtaining several surfaces within the image. This method is convenient since it is fully automated. However, it is sensitive to noise, and estimates the boundary of the surface inaccurately, as we show in experiments. This approach has been applied to seismic images for extracting fault surfaces [9], and it is regarded as the stateoftheart in that field.
Cubical Complexes for Thinning: Our method is a discrete algorithm and is based on the framework of cubical complexes [19]. This framework allows for performing operations analogous to topological operations in the continuum. It has been used for thinning surfaces in 3D based on their geometry [20] to obtain skeletons (or medial representations [21]) of geometrical shapes. This is proven to be robust to noise or fine topological features. Our novel algorithms use concepts from cubical complex theory. In contrast to [20], our method is designed to robustly extract ridges of a function or data defined on a surface (defined by Fast Marching), rather than geometrical properties of a surface.
1.2 Overview of Method
Our algorithm consists of the following steps (see Fig. 2): (i) Weighted Distance to Seed Point Computation: From a given seed point on the surface, the Fast Marching algorithm is used to propagate a front to compute shortest path distance from any point in the image to the seed point (Sect. 3.1). (ii) Ridge Curve Extraction: At samples of the propagating front, the ridge points of the Euclidean path distance of minimal paths to the seed point are computed by removing points on the front from least to greatest distance while preserving topology (Sect. 3.2). This results in a closed curve that lies on the surface of interest. (iii) Surface Boundary Detection: At snapshots, a graph is formulated from curves from the previous step, and is cut along locations where the Euclidean distance between points on adjacent curves are small, resulting in the outer boundary of the surface when a cost threshold is exceeded (Sect. 3.3). (iv) Surface Extraction: Finally, points in the image excluding the cut curve are removed from highest to lowest based on weighted distance to the seed point while preserving topology  resulting in the desired surface (Sect. 3.4).
2 Cubical Complexes Theory
In this section, we review notions from cubical complex theory. This theory defines topological notions (and computational methods) for discrete data that are analogous to topological notions in the continuum. The notion of free pairs, i.e., those parts of the data that can be removed without changing topology of the data, is pertinent to our algorithms. Since the algorithms we define in the next sections require the extraction of lower dimensional structures (curves from surfaces, and surfaces from volumes), it is important that the algorithms are guaranteed to produce lower dimensional structures. The theory of cubicle complexes [20] guarantees such lower dimensional structures are generated while having homotopy equivalence to the original data.
Our data (either a curve, surface or volume) will be represented discretely by a cubical complex. A cubical complex consists of basic elements, called faces, of ddimensions, e.g., points (0faces), edges (1faces), squares (2faces) and cubes (3faces). Formally, a dface is the cartesian product of d intervals of the form \((a,a+1)\) where a is an integer. We can now define a cubical complex:
Definition 1
A ddimensional cubical complex is a finite collection of faces of ddimensions and lower such that every subface of a face in the collection is contained in the collection.
Our algorithms will consist of simplifying cubicle complexes by an operation that is analogous to the continuous topological operation called a deformation retraction, i.e., the operation of continuously shrinking a topological space to a subset. For example, a punctured disk can be continuously shrunk to its boundary. Therefore, the boundary circle is a deformation retraction of the punctured disk, and the two are said to be homotopy equivalent. We are interested in an analogous discrete operation, whereby faces of the cubicle complex can be removed while preserving homotopy equivalence. Free faces, defined in cubical complex theory, can be removed simplifying the cubicle complex, while preserving a discrete notion of homotopy equivalence. These are defined formally as:
Definition 2
Let X be a cubicle complex, and let \(f, g \subset X\) be faces.
g is a proper face of f if \(g\ne f\) and g is a subface of f.
g is free for X, and the pair (g, f) is a free pair for X if f is the only face of X such that g is a proper face of f.
The definition directly provides a constanttime operation to check whether a face is free. For example, if a cubicle complex X is a subset of the 3dim complex formed from a 3D image grid, a 2face is known to be free by only checking whether only one 3face containing the 2face is contained in X.
In the next section, we construct cubicle complexes for the evolving front produced from the Fast Marching algorithm, and retract this front by removing free faces to obtain a lower dimensional curve that lies on the surface that we wish to obtain. We also retract a volume to obtain the surface of interest.
3 FreeBoundary Surface Extraction
In this section, we present our algorithm for extracting the boundary curve of a freeboundary surface from a noisy local likelihood map of the surface defined in a 3D image. Then we present our algorithm for surface extraction. The former algorithm consists of retracting the fronts (closed surfaces) generated by the Fast Marching algorithm to obtain curves on the freeboundary surface of interest. We therefore review Fast Marching in the first subsection before defining our novel algorithms for freeboundary surface extraction.
3.1 Fronts Localized to the Surface Using Fast Marching
We use the Fast Marching Method [3] to generate a collection of fronts that grow from a seed point and are localized to the surface of interest. We denote by \(\phi : \mathbb {Z}_n^3 \rightarrow \mathbb {R}^+\), where \(\mathbb {Z}_n = \{ 0, 1, \ldots , n1 \}\), a noisy function defined on each pixel of the given image grid. It has the property that (in the noiseless situation) a small value of \(\phi (x)\) indicates a high likelihood of the pixel x belonging to the surface of interest.
3.2 Retracting Fronts for Curves on the Surface
If we choose the seed point p to be on the freeboundary surface of interest, the front generated by Fast Marching will travel the fastest when \(\phi \) is small (i.e., along the surface) and travel slower away from the surface, and thus the front is elongated along the surface at each time instant (see Fig. 3). Our algorithm is based on the following observation: points along the front at a time instant that have traveled the furthest (with respect to Euclidean path length), i.e., traveled the longest time, compared to nearby points tend to lie on the surface of interest. This is because points traveling along locations where \(\phi \) is low (on the surface) travel the fastest, tracing out paths that have large arclength.
This property can be more readily seen in the 2D case (see Fig. 3): suppose that we wish to extract a curve from a seed point, and we do so by using Fast Marching to propagate a front. At each time, the points on the front that travel the furthest with respect to Euclidean path length lie on the 2D curve of interest. This has been noted in the 2D case by [4]. In the 3D case (see Fig. 3), we note this generalizes to ridge points ^{1} of the Euclidean path length \(d_E\) (defined next) likely lie on the surface of interest. To define Euclidean path length \(d_E\), define a front \(F = \{ x\in \mathbb {Z}_n^3 \,:\, U(x) \in [D, D+\varepsilon ) \}\) where \(\varepsilon >0\) is small. The function \(d_E : F\rightarrow \mathbb {R}^+\) is such that \(d_E(x) \) is the Euclidean path length of the minimal weighted path (w.r.t to the distance U) from x to p. Note that \(d_E\) is easy to obtain by keeping track of another distance \(U_E\) that solves the eikonal equation with the right hand side of the first equation in (1) equal to 1, while propagating the front to compute U.

\(C_F'\) contains all 2faces f in \(\mathbb {Z}_n^3\) between any 3faces \(g_1, g_2\) with the property that one of \(g_1,g_2\) has all its 0subfaces with \(U < D\) and one does not.

Each face f of \(C_F'\) has cost equal to the average of \(U_E\) over 0subfaces of f.

\(C_F\) removes from \(C_F'\) face f with minimum cost and the smallest local minima with distance (determined by the seed point and f) away from f.
3.3 Stopping Criteria and Surface Boundary Extraction
To determine when to stop the process of extracting ridge curves, and thus obtain the outer boundary of the surface of interest, we make the following observation. Parts of the curves generated from the previous section move slowly, i.e., become close together with respect to Euclidean distance at the boundary of the surface. This is because the speed function \(1/\phi \) becomes small outside the surface. Hence, for the curves \(c_i\) generated, we aim to detect the locations where the distance between points on adjacent curves becomes small. To formulate an algorithm robust to noise, we formulate this as a graph cut problem [15].

vertices V are 0faces in all the 1complexes \(c_i\) formed from ridge extraction

edges E are \((v_1,v_2)\) where \(v_1, v_2\in V\) are such that \(v_1, v_2\) are connected by a 1face in some \(c_i\) or \(v_1\) is a 0face in \(c_i\) and \(v_2\) is the closest (in terms of Euclidean distance) 0face in \(c_{i+1}\) to \(v_1\)

a cost \(v_jv_k\) is assigned to each edge \((v_j,v_k)\) where \(v_j\) and \(v_k\) belong to different \(c_i\) (so that the min cut will be where adjacent curves are close)

for edges \((v_j,v_k)\) such that \(v_j\) and \(v_k\) belong to the same \(c_i\), the cost is the minimum Euclidean distance between segment \((v_j,v_k)\) and segments on \(c_{i+1}\)

the source is the seed point p, and the sink is the last ridge curve \(c_l\).
We wish to obtain a cut of G (separating G into two disjoint sets) with minimum total cost defined as the sum of all costs along the cut. In this way, we obtain a cut of the ridge curves along locations where the distance between adjacent ridge curves is small. The process of obtaining ridge curves from the Fast Marching front is stopped when the cost divided by the cut size is less than a prespecified threshold. This cut then forms the outer boundary of the surface. The computational cost of the cut (compared to other parts of the algorithm) is negligible as the graph size is typically less than \(0.5\,\%\) of the image. Figure 4 shows an example of a cut that is obtained. Figure 5 shows a synthetic example.
3.4 Surface Extraction
Given the surface boundary curve determined from the previous section, we provide an algorithm that determines a surface going through locations of small \(\phi \) and whose boundary is the given curve. Our algorithm uses the cubicle complex framework and has complexity \(O(N\log N)\). Although there is another algorithm, [8], for this task, it is computationally expensive as we show in Sect. 4.
4 Experiments
Supplementary and executables are available^{2}. We quantitatively assess our method by presenting an experimental protocol and comparing against a competing algorithm. To the best of our knowledge, there is no other algorithm that extracts both the boundary of the freesurface and the surface given a seed point. Existing methods with user interaction require user input of the surface boundary. Therefore, we compare our method in an interactive setting and automated setting (with seed points automatically initialized) to [1]. [1] returns all surfaces by detecting connected components of maxima of the local surface likelihood map. In an interactive setting, we choose the surface returned by [1] that is near to the user provided seed point (and best fits ground truth) to provide comparison to our method. In an automated setting, we use a seed point extraction algorithm (described later) to initialize our surface extraction.
4.1 Dataset and Parameters
We evaluate our method on fault surface extraction from 3D seismic images, which are cluttered and have subtle edges. Faults are edges that form surfaces with boundary, which typically have curvature. Several faults may exist within the volume. We test on two separate datasets with image sizes \(463\times 951 \times 651\). We have obtained ground truth segmentations (human annotated) of two faults within each image. We compute \(\phi (x)\) (a local edge map) by computing the eigenvalues of the smoothed Hessian and choosing \(\phi (x)\) to be small when one eigenvalue is small compared to the other two, as described in [1].
Our algorithm, given the local surface likelihood \(\phi \), requires only one parameter, the threshold on the cut cost. In all experiments, we choose this to be \(T=5\). This is not sensitive to the data (see Supplementary).
4.2 Evaluation Protocol
4.3 Evaluation
Computational Cost: We analyze runtimes on a dataset of size \(463\times 951\times 651\). The runtime of our algorithm depends on the size of the surface. To extract one surface, our algorithm takes on average 10 min (9 min for the boundary extraction and 1 min for the surface extraction). Automated seed point extraction takes about 3 min Therefore, the total cost of our algorithm for extracting 6 faults is about 1 h We note that after seed point extraction, the computation of surfaces can be parallelized. In comparison, [1] takes about 2.5 h on the same dataset. Even though the method [8] requires manual input of the boundary curve of the surface, we state the time of [8] for surface extraction. Using Gurobi’s stateoftheart linear programming implementation, the method takes over 10 h for a single surface (and the time grows drastically with increasing image sizes). Ours takes 1 min given the boundary (both achieve similar accuracies). Our solution may not achieve the minimal surface as in [8], but it does achieve a surface with high fidelity to the surface of interest. Speeds are reported on a single Pentium 2.3 GHz processor.
5 Conclusion
We have provided a general method for extracting a smooth simple (without holes) surface with unknown boundary in a 3D image with noisy local measurements of the surface, e.g., edges. Our novel method takes as input a single seed point, and extracts the unknown boundary that may lie in 3D. It then uses this boundary curve to determine the entire surface efficiently. We have demonstrated with extensive experiments on noisy and corrupted data with possible interruptions that our method accurately determines both the boundary and the surface, and the method is robust to seed point choice. In comparison to an approach which extracts connected components of edges in 3D images, our method is more accurate in both surface and boundary measures. The computational cost of our algorithm is less than competing approaches.
A limitation of our method is in extracting multiple intersecting surfaces. Our boundary extraction method may extract boundaries of one or both parts of the intersecting surfaces depending on the data. However, if given the correct boundary of one of the surfaces, our surface extraction produces the relevant surface. This limitation of our boundary extraction is the subject of future work. This is important in seismic images, where surfaces can intersect.
Footnotes
Notes
Acknowledgements
This work was supported by KAUST OCRF2014CRG362140401, and the Visual Computing Center at KAUST.
Supplementary material
Supplementary material 1 (mp4 4595 KB)
Supplementary material 3 (mp4 12191 KB)
References
 1.Schultz, T., Theisel, H., Seidel, H.P.: Crease surfaces: from theory to extraction and application to diffusion tensor MRI. IEEE Trans. Vis. Comput. Graph. 16(1), 109–119 (2010)CrossRefGoogle Scholar
 2.Cohen, L.D., Kimmel, R.: Global minimum for active contour models: a minimal path approach. Int. J. Comput. Vis. 24(1), 57–78 (1997)CrossRefGoogle Scholar
 3.Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Nat. Acad. Sci. 93(4), 1591–1595 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
 4.Kaul, V., Yezzi, A., Tsai, Y.: Detecting curves with unknown endpoints and arbitrary topology using minimal paths. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1952–1965 (2012)CrossRefGoogle Scholar
 5.Mille, J., Bougleux, S., Cohen, L.D.: Combination of piecewisegeodesic paths for interactive segmentation. Int. J. Comput. Vis. 112(1), 1–22 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
 6.Benmansour, F., Cohen, L.D.: From a single point to a surface patch by growing minimal paths. In: Tai, X.C., Mørken, K., Lysaker, M., Lie, K.A. (eds.) SSVM 2009. LNCS, vol. 5567, pp. 648–659. Springer, Heidelberg (2009). doi: 10.1007/9783642022562_54 CrossRefGoogle Scholar
 7.Ardon, R., Cohen, L.D., Yezzi, A.: A new implicit method for surface segmentation by minimal paths: applications in 3D medical images. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 520–535. Springer, Heidelberg (2005). doi: 10.1007/11585978_34 CrossRefGoogle Scholar
 8.Grady, L.: Minimal surfaces extend shortest path segmentation methods to 3D. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 321–334 (2010)CrossRefGoogle Scholar
 9.Hale, D., et al.: Fault surfaces and fault throws from 3D seismic images. In: 2012 SEG Annual Meeting, Society of Exploration Geophysicists (2012)Google Scholar
 10.Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22(1), 61–79 (1997)CrossRefzbMATHGoogle Scholar
 11.Yezzi, A., Kichenassamy, S., Kumar, A., Olver, P., Tannenbaum, A.: A geometric snake model for segmentation of medical imagery. IEEE Trans. Med. Imaging 16(2), 199–209 (1997)CrossRefGoogle Scholar
 12.Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
 13.Osher, S., Sethian, J.A.: Fronts propagating with curvaturedependent speed: algorithms based on HamiltonJacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
 14.Pock, T., Schoenemann, T., Graber, G., Bischof, H., Cremers, D.: A convex formulation of continuous multilabel problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 792–805. Springer, Heidelberg (2008). doi: 10.1007/9783540886907_59 CrossRefGoogle Scholar
 15.Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In: Eighth IEEE International Conference on Computer Vision, ICCV 2001, Proceedings, vol. 1, pp. 105–112. IEEE (2001)Google Scholar
 16.Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (TOG) 23, pp. 309–314. ACM (2004)Google Scholar
 17.Ulen, J., Strandmark, P., Kahl, F.: Shortest paths with higherorder regularization. IEEE Trans. Pattern Anal. Mach. Intell. 37(12), 2588–2600 (2015)CrossRefGoogle Scholar
 18.Grady, L.: Computing exact discrete minimal surfaces: extending and solving the shortest path problem in 3D with application to segmentation. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 69–78. IEEE (2006)Google Scholar
 19.Kovalevsky, V.A.: Finite topology as applied to image analysis. Comput. Vis. Graph. Image Process. 46(2), 141–161 (1989)CrossRefGoogle Scholar
 20.Chaussard, J., Couprie, M.: Surface thinning in 3D cubical complexes. In: Wiederhold, P., Barneva, R.P. (eds.) IWCIA 2009. LNCS, vol. 5852, pp. 135–148. Springer, Heidelberg (2009). doi: 10.1007/9783642102103_11 CrossRefGoogle Scholar
 21.Siddiqi, K., Pizer, S.: Medial Representations: Mathematics, Algorithms and Applications, vol. 37. Springer, Netherlands (2008)zbMATHGoogle Scholar
 22.Eberly, D., Gardner, R., Morse, B., Pizer, S., Scharlach, C.: Ridges for image analysis. J. Math. Imaging Vis. 4(4), 353–373 (1994)CrossRefGoogle Scholar
 23.Lindeberg, T.: Edge detection and ridge detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 117–156 (1998)CrossRefGoogle Scholar
 24.Kolomenkin, M., Shimshoni, I., Tal, A.: Multiscale curve detection on surfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 225–232(2013)Google Scholar
 25.Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4. IEEE (2011)Google Scholar