Multidimensional sum-up rounding for integer programming in optimal experimental design

Abstract

We present a numerical method for approximating the solution of convex integer programs stemming from optimal experimental design. The statistical setup consists of a Bayesian framework for linear inverse problems for which the direct relationship is described by a discretized integral equation. Specifically, we aim to find the optimal sensor placement from a set of candidate locations where data are collected with measurement error. The convex objective function is a measure of the uncertainty, described here by the trace or log-determinant of the posterior covariance matrix, for the discretized linear inverse problem solution. The resulting convex integer program is relaxed, producing a lower bound. An upper bound is obtained by extending the sum-up rounding approach to multiple dimensions. For this extension, we analyze its accuracy as a function of the discretization mesh size for a rectangular domain. We show asymptotic optimality of the integer solution defining the upper bound for different experimental design criteria (A- and D-optimal), by proving the convergence to zero of the gap between the upper and lower bounds as the mesh size goes to zero. The technique is illustrated on a two-dimensional gravity surveying problem for both A-optimal and D-optimal sensor placement where our designs yield better results compared with a thresholding rounding approach.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Alexanderian, A., Petra, N., Stadler, G., Ghattas, O.: A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized \(l_0\)-sparsification. SIAM J. Sci. Comput. 36, A2122–A2148 (2014)

    Article  Google Scholar 

  2. 2.

    Alexanderian, A., Petra, N., Stadler, G., Ghattas, O.: A fast and scalable method for a-optimal design of experiments for infinite-dimensional Bayesian nonlinear inverse problems. SIAM J. Sci. Comput. 38(1), A243–A272 (2016)

    MathSciNet  Article  Google Scholar 

  3. 3.

    Atkinson, K., Han, W.: Theoretical Numerical Analysis, vol. 39. Springer, New York (2005)

    Google Scholar 

  4. 4.

    Balas, E., Ceria, S., Dawande, M., Margot, F., Pataki, G.: Octane: a new heuristic for pure 0–1 programs. Operat. Res. 49, 207–225 (2001)

    MathSciNet  Article  Google Scholar 

  5. 5.

    Berry, J., Hart, W.E., Phillips, C.A., Uber, J.G., Watson, J.P.: Sensor placement in municipal water networks with temporal integer programming models. J. Water Resour. Plan. Manag. 132, 218–224 (2006)

    Article  Google Scholar 

  6. 6.

    Berthold, T.: RENS–the optimal rounding. Math. Prog. Comput. 6, 33–54 (2014)

    Article  Google Scholar 

  7. 7.

    Blacker, T.: Meeting the challenge for automated conformal hexahedral meshing. In: 9th International Meshing Roundtable, pp. 11–20 (2000)

  8. 8.

    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  9. 9.

    Cox, D.R., Reid, N.: The Theory of the Design of Experiments. Chapman & Hall/CRC, Boca Raton (2000)

    Google Scholar 

  10. 10.

    Cressie, N., Wikle, C.K.: Statistics for Spatio-Temporal Data. Wiley, New York (2015)

    Google Scholar 

  11. 11.

    Drăgănescu, A.: Multigrid preconditioning of linear systems for semi-smooth newton methods applied to optimization problems constrained by smoothing operators. Optim. Methods Softw. 29(4), 786–818 (2014)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems, vol. 375. Springer, New York (1996)

    Google Scholar 

  13. 13.

    Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning, vol. 1. Springer, New York (2001)

    Google Scholar 

  14. 14.

    Geoga, C.J., Anitescu, M., Stein, M.L.: Scalable Gaussian process computations using hierarchical matrices (2018). arXiv preprint, arXiv:1808.03215

  15. 15.

    Hammer, P.L., Johnson, E.L., Peled, U.N.: Facet of regular 0–1 polytopes. Math. Program. 8, 179–206 (1975)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  17. 17.

    Iyengar, S.S., Brooks, R.R.: Distributed Sensor Networks, Second Edition: Sensor Networking and Applications. Chapman and Hall/CRC, Boca Raton (2016)

    Google Scholar 

  18. 18.

    Kaipio, J., Somersalo, E.: Statistical and Computational Inverse Problems, vol. 160. Springer, New York (2006)

    Google Scholar 

  19. 19.

    Kendall, E.A.: The Numerical Solution of Integral Equations of the Second Kind. Cambridge University Press, Cambridge (1997)

    Google Scholar 

  20. 20.

    Kirsch, A.: An Introduction to the Mathematical Theory of Inverse Problems. Springer, New York (2011)

    Google Scholar 

  21. 21.

    Krause, A., Leskovec, J., Guestrin, C., VanBriesen, J., Faloutsos, C.: Efficient sensor placement optimization for securing large water distribution networks. J. Water Resour. Plan. Manag. 134, 516–526 (2008)

    Article  Google Scholar 

  22. 22.

    Lodi, A., Bonami, P., Cornuéjols, G., Margot, F.: A feasibility pump for mixed integer nonlinear programs. Math. Program. 119, 331–352 (2009)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Nannicini, G., Belotti, P.: Rounding-based heuristics for nonconvexminlps. Math. Program. Comput. 4, 1–31 (2012)

    MathSciNet  Article  Google Scholar 

  24. 24.

    Patera, A.T.: A Spectral Element Method for Fluid Dynamics: Laminar Flow in a Channel Expansion, vol. 54. Elsevier, Amsterdam (1984)

    Google Scholar 

  25. 25.

    Pukelsheim, F.: Optimal Design of Experiments. Classics in Applied Mathematics, vol. 50. SIAM (2006)

  26. 26.

    Sager, S.: Sampling decision in optimum experimental design in the light of Pontryagin’s maximum principle. SIAM J. Control Optim. 51, 3181–3207 (2013)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Sager, S., Bock, H.G., Diehl, M.: The integer approximation error in mixed-integer optimal control. Math. Program. Ser. A 133, 1–23 (2012)

    MathSciNet  Article  Google Scholar 

  28. 28.

    Wang, Y., Yagola, A.G., Yang, C.: Computational Methods for Applied Inverse Problems. Higher Education Press, Beijing (2012)

    Google Scholar 

  29. 29.

    Watson, J.P., Greenberg, H.J., Hart, W.E.: A multiple-objective analysis of sensor placement optimization in water networks. In: Proceedings of the World Water and Environment Resources Congress. American Society of Civil Engineers (2004)

  30. 30.

    Welch, W.J.: Algorithmic complexity: three NP-hard problems in computational statistics. J. Stat. Comput. Simul. 15(1), 17–25 (1982)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Wielandt, H.: Error bounds for eigenvalues of symmetric integral equations. Proc. Sympos. Appl. Math 6, 261–282 (1956)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Wolsey, L.A.: Faces for a linear inequality in 0–1 variables. Math. Program. 8, 165–178 (1975)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Yu, J., Zavala, V.M., Anitescu, M.: A scalable design of experiments framework for optimal sensor placement. J. Process Control 67, 44–55 (2017)

    Article  Google Scholar 

  34. 34.

    Zhang, Yongjie, Bajaj, Chandrajit: Adaptive and quality quadrilateral/hexahedral meshing from volumetric data. Comput. Methods Appl. Mech. Eng. 195(9–12), 942–960 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

This material was based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research (ASCR) under Contract DE-AC02-06CH11347. We acknowledge partial NSF funding through awards FP061151-01-PR and CNS-1545046. We thank Noemi Petra and Victor Zavala for feedback on this topic. We also thank the two anonymous referees whose comments have helped significantly improve the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mihai Anitescu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Also, Preprint ANL/MCS 9032-1218.

Government License: The submitted manuscript has been created by UChicago Argonne, LLC, Operator of Argonne National Laboratory (“Argonne”). Argonne, a U.S. Department of Energy Office of Science laboratory, is operated under Contract No. DE-AC02-06CH11357. The U.S. Government retains for itself, and others acting on its behalf, a paid-up nonexclusive, irrevocable worldwide license in said article to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, by or on behalf of the Government.

A Other rounding strategies

A Other rounding strategies

A.1 Another sum-up rounding procedure for rectangular domains

We present the sum-up rounding algorithm II based on the following compatible two-level decomposition, with concepts defined in Definition 1. We use the notation

$$\begin{aligned} k_1(n_i) = \lfloor \sqrt{n_i} \rfloor ,\quad \text{ and }\quad k(n) = k_1(n_1)k_1(n_2)..k_1(n_P). \end{aligned}$$
  1. (i)

    On \([l^i_1, l^i_2]\) for \(i\!=\!1,2,\ldots ,P\), group the first \(k_1(n_i)\) intervals \(\{{\mathcal {I}}_{i, j}\}_{j=1}^{k_1(n_i)}\) as \({\mathcal {G}}_{i,1}\), group the next \(k_1(n_i)\) intervals \(\{{\mathcal {I}}_{i, j}\}_{j=k_1(n_i)+1}^{2k_1(n_i)}\) as \({\mathcal {G}}_{i,2}\), and so forth until we get \({\mathcal {G}}_{i,k_1(n_i)}\). The remaining intervals \(\{{\mathcal {I}}_{i, j}\}_{j=k_1(n_i)^2+1}^{n}\) are grouped as \({\mathcal {G}}_{i, last}\), and the number of intervals in \({\mathcal {G}}_{i, last}\) equals \(n_i-k_1(n_i)^2\). Note that

    $$\begin{aligned} \sqrt{n_i}-1<k_1(n_i)=\lfloor \sqrt{n_i}\rfloor \le \sqrt{n_i}. \end{aligned}$$

    We can bound the number of intervals in the last group by

    $$\begin{aligned} n_i-(\sqrt{n_i})^2&\le n_i-k_1(n_i)^2< n_i-(\sqrt{n_i}-1)^2\\ 0&\le n_i-k_1(n_i)^2 < 2\sqrt{n_i}, \end{aligned}$$

    so the cardinality of \({\mathcal {G}}_{i,.}\) is \({\mathcal {O}}(\sqrt{n_i})\), and its size is \({\mathcal {O}}(1/\sqrt{n_i})\).

  2. (ii)

    Consider a subdomain \(V_j\) of the form

    $$\begin{aligned} \prod _{\begin{array}{c} i=1,2,\ldots ,P\\ j_i\in \{1,2,\ldots ,k_1(n_i), last\} \end{array}}{\mathcal {G}}_{i,j_i}. \end{aligned}$$

This decomposition has the following parameters and properties, in reference to Definition 1.

$$\begin{aligned}&k(n)=\prod _{i=1}^P \lfloor \sqrt{n_i} \rfloor ,\; {\tilde{k}}(n)=\prod _{i=1}^P \lceil \sqrt{n_i} \rceil ,\;r(n)=k(n), \; \end{aligned}$$
(56)
$$\begin{aligned}&\rho (V_j)=\sqrt{\sum _{i=1}^P \left( \frac{(l_2^i-l_1^i)}{\lfloor \sqrt{n_i} \rfloor } \right) ^2 }, \quad j=1,2,\ldots ,k(n) \end{aligned}$$
(57)

Theorem 5

Under the assumptions of Theorem 1, there exists a C such that the sum-up rounding algorithm II construction satisfies

$$\begin{aligned} \Big |\sum _{k=1}^n f(x_k)\left( w^n(x_k)-{\tilde{w}}^n(x_k)\right) \varDelta _x \Big | \le \frac{C}{n^{1/2P}}. \end{aligned}$$

Proof

We use the definitions of the sum-up rounding procedure parameters (56)–(57), and the inequalities (21)–(22) to infer the following inequalities:

$$\begin{aligned} \frac{1}{\sqrt{n_i} } \le c_1^{-\frac{1}{2}} n^{-\frac{1}{2P}}, \; i=1,2,\ldots ,P; \quad \frac{1}{r(n)} = \prod _{i=1}^P \frac{1}{\lfloor \sqrt{n_i} \rfloor } {\mathop {\le }\limits ^{(21)}} \frac{2^\frac{P}{2}}{\sqrt{n}}. \end{aligned}$$
(58)

For the maximum diameter of \(V_j\) we obtain from (57) and (21)

$$\begin{aligned} \max _{j=1,2,\ldots ,k(n)} \rho (V_j)&\le \sqrt{P} \frac{\max _{i=1,2,\ldots ,P} (l_2^i-l_1^i)}{\frac{1}{2} \min _{i=1,2,\ldots ,P} \sqrt{n_i}}\nonumber \\&{\mathop {\le }\limits ^{(22)}} \sqrt{P} \frac{\max _{i=1,2,\ldots ,P} (l_2^i-l_1^i)}{\frac{1}{2} \sqrt{c_1}} n^{-\frac{1}{2P}}. \end{aligned}$$
(59)

We also obtain

$$\begin{aligned} 1-\frac{k(n)r(n)}{n}=1-\prod _{i=1}^{P}\frac{\lfloor \sqrt{n_i} \rfloor ^2}{n_i}\le 1-\prod _{i=1}^{P}\left( 1-\frac{2}{\sqrt{n_i}}\right) {\mathop {\le }\limits ^{(22)}} 1 - \left( 1-2c_1^{-\frac{1}{2}}n^{-\frac{1}{2P}} \right) ^P. \end{aligned}$$

In turn, from the mean value theorem applied to \((1-x)^P\) for \(x \in [0,1]\) and the last inequality, we have

$$\begin{aligned} 1-(1-x)^P \le Px, \; \forall x \in [0,1] \Rightarrow 1-\frac{k(n)r(n)}{n} \le 2P c_1^{-\frac{1}{2}}n^{-\frac{1}{2P}}. \end{aligned}$$
(60)

We now use Theorem 1 along with (21)–(22), (58), (59), and (60) to obtain the statement of this theorem for the sum-up rounding algorithm II with the choice

$$\begin{aligned} C= & {} \max _{x\in V}|f(x)|\mu (V) 2^{\frac{P}{2}} + 2L\mu (V) \sqrt{P} \frac{\max _{i=1,2,\ldots ,P} (l_2^i-l_1^i)}{\frac{1}{2} \sqrt{c_1}}\\&+\,4\max _{x\in V}|f(x)|\mu (V) P c_1^{-\frac{1}{2}}. \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Anitescu, M. Multidimensional sum-up rounding for integer programming in optimal experimental design. Math. Program. 185, 37–76 (2021). https://doi.org/10.1007/s10107-019-01421-z

Download citation

Keywords

  • Optimal experimental design
  • Sum-up rounding
  • Integer programming
  • Bayesian inverse problem

Mathematics Subject Classification

  • 62K05
  • 90C10
  • 15A29