A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 6 Issue 3
May  2019

IEEE/CAA Journal of Automatica Sinica

• JCR Impact Factor: 6.171, Top 11% (SCI Q1)
CiteScore: 11.2, Top 5% (Q1)
Google Scholar h5-index: 51， TOP 8
Turn off MathJax
Article Contents
Zhiming Lv, Linqing Wang, Zhongyang Han, Jun Zhao and Wei Wang, "Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 838-849, May 2019. doi: 10.1109/JAS.2019.1911450
 Citation: Zhiming Lv, Linqing Wang, Zhongyang Han, Jun Zhao and Wei Wang, "Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 838-849, May 2019.

# Surrogate-Assisted Particle Swarm Optimization Algorithm With Pareto Active Learning for Expensive Multi-Objective Optimization

##### doi: 10.1109/JAS.2019.1911450
Funds:  This work was supported by the National Natural Sciences Foundation of China (61603069, 61533005, 61522304, U1560102) and the National Key Research and Development Program of China (2017YFA0700300)
• For multi-objective optimization problems, particle swarm optimization (PSO) algorithm generally needs a large number of fitness evaluations to obtain the Pareto optimal solutions. However, it will become substantially time-consuming when handling computationally expensive fitness functions. In order to save the computational cost, a surrogate-assisted PSO with Pareto active learning is proposed. In real physical space (the objective functions are computationally expensive), PSO is used as an optimizer, and its optimization results are used to construct the surrogate models. In virtual space, objective functions are replaced by the cheaper surrogate models, PSO is viewed as a sampler to produce the candidate solutions. To enhance the quality of candidate solutions, a hybrid mutation sampling method based on the simulated evolution is proposed, which combines the advantage of fast convergence of PSO and implements mutation to increase diversity. Furthermore, $\varepsilon$-Pareto active learning ($\varepsilon$-PAL) method is employed to pre-select candidate solutions to guide PSO in the real physical space. However, little work has considered the method of determining parameter $\varepsilon$. Therefore, a greedy search method is presented to determine the value of $\varepsilon$ where the number of active sampling is employed as the evaluation criteria of classification cost. Experimental studies involving application on a number of benchmark test problems and parameter determination for multi-input multi-output least squares support vector machines (MLSSVM) are given, in which the results demonstrate promising performance of the proposed algorithm compared with other representative multi-objective particle swarm optimization (MOPSO) algorithms.

•  [1] G. R. Zavala, A. J. Nebro, F. Luna, and C. A. C. Coello, " A survey of multi-objective metaheuristics applied to structural optimization,” Struct. Multidiscipl. Optim., vol. 49, no. 4, pp. 537–558, Apr. 2014. [2] M. Park, M. Nassar, and H. Vikalo, " Bayesian active learning for drug combinations,” IEEE Trans. Biomed. Eng., vol. 60, no. 11, pp. 3248–3255, Nov. 2013. [3] D. Ronco, C. Comis, R. Ponza, and E. Benini, " Aerodynamic shape optimization of aircraft components using an advanced multi-objective evolutionary approach,” Comput. Methods Appl. Mech. Eng., vol. 285, pp. 255–290, Mar. 2015. [4] J. Snoek, I. Larochelle, and R. P. Adans., " Practical Bayesian optimization of machine learning algorithms,” in Proc. 25th Int. Conf. NIPS, Lake Tahoe, Nevada, 2012, pp. 2951–2959. [5] H. D. Wang, Y. C. Jin, and J. O. Jansen, " Data-driven surrogate-assisted multiobjective evolutionary optimization of a trauma system,” IEEE Trans. E Comput., vol. 20, no. 6, pp. 939–952, Dec. 2016. [6] Q. L. Zhu, Q. Z. Lin, W. N. Chen, K. C. Wong, C. A. Coello Coello, J. Li, J. Chen, and J. Zhang, " An external archive-guided multiobjective particle swarm optimization algorithm,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2794–2808, Sep. 2017. [7] Q. Z. Lin, S. B. Liu, Q. L. Zhu, C. Y. Tang, R. Z. Song, J. Y. Chen, C. A. Coello Coello, K. C. Wong, and J. Zhang, " Particle swarm optimization with a balanceable fitness estimation for many-objective optimization problems,” IEEE Trans. E Comput., vol. 22, no. 1, pp. 32–46, Feb. 2018. [8] W. J. Kong, T. Y. Chai, J. L. Ding, and Z. W. Wu, " A real-time multiobjective electric energy allocation optimization approach for the smelting process of magnesia,” Acta Autom. Sinica, vol. 40, no. 1, pp. 51–61, Jan. 2014. [9] W. Hu and G. G. Yen, " Adaptive multiobjective particle swarm optimization based on parallel cell coordinate system,” IEEE Trans. E Comput., vol. 19, no. 1, pp. 1–18, Feb. 2015. [10] A. Trivedi, D. Srinivasan, K. Sanyal, and A. Ghosh, " A survey of multiobjective evolutionary algorithms based on decomposition,” IEEE Trans. E Comput., vol. 21, no. 3, pp. 440–462, Jun. 2017. [11] Q. Kang, S. W. Feng, M. C. Zhou, A. C. Ammari, and K. Sedraoui, " Optimal load scheduling of plug-in hybrid electric vehicles via weight-aggregation multi-objective evolutionary algorithms,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 9, pp. 2557–2568, Sep. 2017. [12] R. T. Haftka, D. Villanueva, and A. Chaudhuri, " Parallel surrogate-assisted global optimization with expensive functions — a survey,” Struct. Multidiscip. Optim., vol. 54, no. 1, pp. 1–11, Apr. 2016. [13] M. Tabatabaei, J. Hakanen, M. Hartikainen, K. Miettinen, and K. Sindhya, " A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods,” Struct. Multidiscip. Optim., vol. 52, no. 1, pp. 1–25, Jul. 2015. [14] Y. Jin, " Surrogate-assisted evolutionary computation: recent advances and future challenges,” Swarm E Comput., vol. 1, no. 2, pp. 61–70, Jun. 2011. [15] C. Ma and L. Y. Qu, " Multiobjective optimization of switched reluctance motors based on design of experiments and particle swarm optimization,” IEEE Trans. Energy Convers., vol. 30, no. 3, pp. 1144–1153, Sep. 2015. [16] X. R. Ye, H. Chen, H. M. Liang, X. J. Chen., and J. X. You, " Multi-objective optimization design for electromagnetic devices with permanent magnet based on approximation model and distributed cooperative particle swarm optimization algorithm,” IEEE Trans. Magn., vol. 54, no. 3, pp. 1–5, Mar. 2018. [17] H. X. Jie, Y. Z. Wu, J. J. Zhao, J. W. Ding, and L. liang, " An efficient multi-objective PSO algorithm assisted by Kriging metamodel for expensive black-box problems,” J. Glob. Optim., vol. 67, no. 1–2, pp. 399–423, Jan. 2017. [18] F. Bittner and I. Hahn, " Kriging-assisted multi-objective particle swarm optimization of permanent magnet synchronous machine for hybrid and electric cars,” in Proc. IEEE Int. Elect. Mach. Drives Conf., Chicago, IL, USA, 2013, pp. 15–22. [19] H. D. Wang, Y. C. Jin, and J. O. Jansen, " Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems,” IEEE Trans. Cybern., vol. 47, no. 9, pp. 2664–2677, Jun. 2017. [20] M. Parno, T. Hemker, and K. R. Fowler, " Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems,” Eng. Optim., vol. 44, no. 5, pp. 521–535, Sep. 2012. [21] Z. C. Cao, C. R. Lin, M. C. Zhou, and R. Huang, " Scheduling semiconductor testing facility by using cuckoo search algorithm with reinforcement learning and surrogate modeling,” IEEE Trans. Autom. Sci. Eng.. 2018. DOI: 10.1109/TASE.2018.2862380. [22] X. Y. Sun, D. W. Gong, Y. C. Jin, and S. S. Chen, " A new surrogate-assisted interactive genetic algorithm with weighted semisupervised learning,” IEEE Trans. Cybern., vol. 43, no. 2, pp. 685–698, Apr. 2013. [23] P. Campigotto, A. Passerini, and R. Battiti, " Active learning of Pareto fronts,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 3, pp. 506–519, Mar. 2014. [24] D. Reker, P. Schneider, and G. Schneider, " Multi-objective active machine learning rapidly improves structure-activity models and reveals new protein-protein interaction inhibitors,” Chem. Sci., vol. 7, pp. 3919–3727, Mar. 2016. [25] M. Zuluaga, G. Sergent, A. Krause, and M. Püschel, " Active learning for multi-objective optimization,” in Proc. Int. Conf. Mach. Learning, Atlanta, GA, USA, 2013, pp. 462–470. [26] M. Zuluaga, A. Krause, and M. Puschel, " ε-PAL: an active learning approach to the multi-objective optimization problem,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 3619–3650, Aug. 2016. [27] D. F. Wang, and L. Meng, " Performance analysis and parameter selection of PSO algorithms,” Acta Autom. Sinica, vol. 42, no. 10, pp. 1552–1561, Jan. 2016. [28] C. E. Rasmussen and H. Nickisch, " Gaussian processes for machine learning (GPML) toolbox,” J. Mach. Learn. Res., vol. 11, pp. 3011–3015, Nov. 2010. [29] S. Z. Martínez and C. C. Coello, " A multi-objective particle swarm optimizer based on decomposition,” in Proc. Genetic Evol. Comput., Dublin, Ireland, 2011, pp. 69–76. [30] A. J. Nebro, J. J. Durillo, J. Garcia-Nieto, C. A. coello coeelo, F. Luna, and R. Aeba, " SMPSO: a new PSO-based metaheuristic for multi-objective optimization,” in Proc. IEEE Symp. Comput. Intell. Miulticriteria Decision Making, Nashville, TN, USA, 2009, pp. 66–73. [31] Y. Tian, R. Cheng, X. Y. Zhang, and Y. C. Jin, " PlatEMO: A MATLAB platform for evolutionary multi-objective optimization,” IEEE Comput. Intell. Mag., vol. 12, pp. 73–87, Nov. 2017. [32] Q. F. Zhang, A. M. Zhou, S. Z. Zhao, P. Suganthan, W. D. Liu, and S. Tiwari,, " Multiobjective optimization test instances for the CEC-2009 special session and competition,” Nanyang Technol. Univ., Singapore, Tech. Rep., 2008. http://www.ntu.edu.sg/home/epnsugan/. [33] Q. F. Zhang, A. M. Zhou, and Y. C. Jin, " RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm,” IEEE Trans. E Comput., vol. 21, no. 1, pp. 41–63, Feb. 2008. [34] S. Xu, X. An, X. D. Qiao, L. J. Zhu, and L. Li, " Multi-output least-squares support vector regression machines,” Pattern Recognit. Lett., vol. 34, no. 9, pp. 1078–1084, Jul. 2013. [35] J. Bergstra and Y. Bengio, " Random search for hyper-parameter optimization,” J. Mach. Learn. Res., vol. 13, no. 1, pp. 281–305, Feb. 2012. [36] S. Chen, " Multi-output regression using a locally regularised orthogonal least square algorithm,” IEEE Proc. Vision Image Signal Process., vol. 149, no. 4, pp. 185–195, Aug. 2002. [37] Z. Y. Han, Y. Liu, J. Zhao, and W. Wang, " Real time prediction for converter gas tank levels based on multi-output least square support vector regressor,” Control Eng. Pract., vol. 20, no. 12, pp. 1400–1409, Dec. 2012.

### Catalog

###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(4)  / Tables(7)