Ensemble Selection in Expensive Optimization Problems

AUTHOR(S):

TITLE

Ensemble Selection in Expensive Optimization Problems

ABSTRACT

Computationally intensive simulations are being extensively used across engineering and science in various design optimization problems. To alleviate the high computational load associated with each simulation run metamodels are used, as they provide predicted objective values at a lower computational cost. However, the optimal metamodel variant is typically unknown and is problem-dependant. In an attempt to alleviate this ensembles use multiple metamodels concurrently and aggregate their predictions. However the optimal ensemble configuration is also problem-dependant and typically unknown. To address this issue, this paper proposes an approach in an optimal ensemble configuration is selected during the search out of a family of candidate ensembles, without a need for user intervention or a-priori domain knowledge. Performance analysis shows that the proposed approach improved the search effectiveness over a range of test problems.

KEYWORDS

expensive optimization problems, metamodels, ensembles, computational intelligence

REFERENCES

[1].D. B¨uche, N. N. Schraudolph, and P. Koumoutsakos, “Accelerating evolutionary algorithms with Gaussian process fitness function models,” IEEE Transactions on Systems, Man, and Cybernetics–Part C, vol. 35, no. 2, pp. 183–194, 2005.

[2].A. R. Conn, K. Scheinberg, and P. L. Toint, “On the convergence of derivative-free methods for unconstrained optimization,” in Approximation Theory and Optimization: Tributes to M.J.D. Powell, A. Iserles and M. D. Buhmann, Eds. Cambridge; New York: Cambridge University Press, 1997, pp. 83–108.

[3].A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust RegionMethods. Philadelphia, Pennsylvania: SIAM, 2000.

[4].K. A. de Jong, Evolutionary Computation:A Unified Approach. Cambridge, Massachusetts: MIT Press, 2006.

[5].M. Drela and H. Youngren, XFOIL 6.9 User Primer, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, 2001.

[6].T. Goel, R. T. Haftka, W. Shyy, and N. V. Queipo, “Ensembles of surrogates,” Structural andMultidisciplinary Optimization, vol. 33, pp. 199–216, 2007.

[7].D. Gorissen, T. Dhaene, and F. De Turck, “Evolutionary model type selection for global surrogate modeling,” The Journal of Machine Learning Research, vol. 10, pp. 2039–2078, 2009.

[8].R. M. Hicks and P. A. Henne, “Wing design by numerical optimization,” Journal of Aircraft, vol. 15, no. 7, pp. 407–412, 1978.

[9].Y. Jin, M. Olhofer, and B. Sendhoff, “A framework for evolutionary optimization with approximate fitness functions,” IEEE Transactions on evolutionary computation, vol. 6, no. 5, pp. 481–494, 2002.

[10].J. Muller and R. Pich´e, “Mixture surrogate models based on dempster-shafer theory for global optimization problems,” Journal of Global Optimization, vol. 51, no. 1, pp. 79–104, 2011.

[11].J. Muller and C. A. Shoemaker, “Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems,” Journal of Global Optimization, vol. 60, pp. 123–144, 2014.

[12].M. J. D. Powell, “On trust region methods for unconstrained minimization without derivatives,” Mathematical Programming, Series B, vol. 97, pp. 605– 623, 2003.

[13].A. Ratle, “Optimal sampling strategies for learning a fitness model,” in The 1999 IEEE Congress on Evolutionary Computation–CEC 1999. Piscataway, New Jersey: IEEE, 1999, pp. 2078–2085.

[14].R. G. Regis and C. A. Shoemaker, “A quasi-multistart framework for global optimization of expensive functions using response surface models,” Journal of Global Optimization, vol. 56, pp. 1719–1753, 2013.

[15].D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, 4th ed. Boca Raton, Florida: Chapman and Hall, 2007.

[16].P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y. P. Chen, A. Auger, and S. Tiwari, “Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization,” Nanyang Technological University, Singapore and Kanpur Genetic Algorithms Laboratory, Indian Institute of Technology Kanpur, India, Technical Report KanGAL 2005005, 2005.

[17].Y. Tenne, “A computational intelligence algorithm for simulation-driven optimization problems,” Advances in Engineering Software, vol. 47, pp. 62–71, 2012.

[18].“An optimization algorithm employing multiple metamodels and optimizers,” International Journal of Automation and Computing, vol. 10, no. 3, pp. 227–241, 2013.

[19].“An algorithm for computationally expensive engineering optimization problems,” International Journal of General Systems, vol. 42, no. 5, pp. 458– 488, 2013.

[20].Y. Tenne and C. K. Goh, Eds., Computational Intelligence in Expensive Optimization Problems, ser. Evolutionary Learning and Optimization. Berlin: Springer, 2010, vol. 2.

[21].F. A. C. Viana, G. Venter, and V. Balabanov, “An algorithm for fast optimal Latin hypercube design of experiments,” International Journal of Numerical Methods in Engineering, vol. 82, no. 2, pp. 135– 156, 2009.

[22].H.-Y.Wu, S. Yang, F. Liu, and H.-M. Tsai, “Comparison of three geometric representations of airfoils for aerodynamic optimization,” in Proceedings of the 16th AIAA Computational Fluid Dynamics Conference. Reston, Virginia: American Institute of Aeronautics and Astronautics, 2003, AIAA 2003- 4095.

Cite this paper

Yoel Tenne. (2017) Ensemble Selection in Expensive Optimization Problems. International Journal of Mechanical Engineering, 2, 134-141