Istrazivanja i projektovanja za privreduJournal of Applied Engineering Science

FIXED-BUDGET APPROXIMATION OF THE INVERSE KERNEL MATRIX FOR IDENTIFICATION OF NONLINEAR DYNAMIC PROCESSES


DOI: 10.5937/jaes0-31772 
This is an open access article distributed under the CC BY 4.0
Creative Commons License

Volume 20 article 915 pages: 150-159

Nikita Antropov
Reshetnev Siberian State University of Science and Technology, Institute of Computer Science and Telecommunications, Department of Systems Analysis and Operations Research, Krasnoyarsk, Russian Federation

Evgeny Agafonov
Reshetnev Siberian State University of Science and Technology, Institute of Computer Science and Telecommunications, Department of Systems Analysis and Operations Research, Krasnoyarsk, Russian Federatio; Siberian Federal University, School of Petroleum and Natural Gas Engineering, Department of Fuel Supply and Combustibles, Krasnoyarsk, Russian Federation

Vadim Tynchenko*
Siberian Federal University, School of Petroleum and Natural Gas Engineering, Department of Technological Machines and Equipment of Oil and Gas Complex, Krasnoyarsk, Russian Federation; Reshetnev Siberian State University of Science and Technology, Institute of Computer Science and Telecommunications, Information-Control Systems Department, Krasnoyarsk, Russian Federation

Vladimir Bukhtoyarov
Siberian Federal University, School of Petroleum and Natural Gas Engineering, Department of Technological Machines and Equipment of Oil and Gas Complex, Krasnoyarsk, Russian Federation; Reshetnev Siberian State University of Science and Technology, Institute of Computer Science and Telecommunications, Department of Information Technology Security, Krasnoyarsk, Russian Federation

Vladislav Kukartsev
Siberian Federal University, Institute of Space and Information Technologies, Department of Computer Science, Krasnoyarsk, Russian Federation; Reshetnev Siberian State University of Science and Technology, Engineering and Economics Institute, Department of Information Economic Systems, Krasnoyarsk, Russian Federation

The paper considers the identification of nonlinear dynamic processes using kernel algorithms. Kernel algorithms rely on a nonlinear transformation of the input data points into a high-dimensional space that allows solving nonlinear problems through the construction of kernelized counterparts of linear methods by replacing the inner products with kernels. A key feature of the kernel algorithms is high complexity of the inverse kernel matrix calculation. Nowadays, there are two approaches to this problem. The first one is based on using a reduced training data sample instead of a full one. In case of kernel methods, this approach could cause model misspecification, since kernel methods are directly based on training data. The second one is based on the reduced-rank approximations of the kernel matrix. A major limitation of this approach is that the rank of the approximation is either unknown until approximation is done or it is predefined by the user, both of which are not efficient enough. In this paper, we propose a new regularized kernel least squares algorithm based on the fixed-budget approximation of the kernel matrix. The proposed algorithm allows regulating the computational burden of the identification algorithm and obtaining the least approximation error. We have shown some simulations results illustrating the efficiency of the proposed algorithm compared to other algorithms. The application of the proposed algorithm is considered on the identification problem of the input and output pressure of the pump station.

View article

The reported study was funded by RFBR, project number 19-37-90040.

1. Liu, Q., Chen, W., Hu, H., Zhu, Q., Xie Z. (2020). An optimal NARX Neural Network Identification Model for a Magnetorheological Damper With Force-Distortion Behavior. Frontiers in Materials. DOI: 10.3389/fmats.2020.00010

2. Tavoosi, J., Mohammadzadeh, A., Jermsittiparsert, K. (2021). A review on type-2 fuzzy neural networks for system identification. Soft Computing, vol. 25, 7197-7212, DOI: 10.1007/s00500-021-05686-5

3. Li, J., Ding, F. (2021). Identification methods of nonlinear systems based on the kernel functions. Nonlinear Dynamics, vol. 104, 2537-2552, DOI: 10.1007/s11071-021-06417-z

4. Ning, H., Qing, G., Tian, T., Jing, X. (2019). Online Identification of Nonlinear Stochastic Spatiotemporal System With Multiplicative Noise by Robust Optimal Control-Based Kernel Learning Methods. IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 2, p. 389-404, DOI: 10.1109/TNNLS.2018.2843883

5. Zhang, T., Wang, S., Huang, X., Jia, L. (2020). Kernel Recursive Least Squares Algorithm Based on the Nyström Method With k-Means Sampling. IEEE Signal Processing Letters, vol. 27, p. 361-365, DOI: 10.1109/LSP.2020.2972164

6. Mazzoleni, M., Scandella, M., Formentin, S., Previdi, F. (2020). Enhanced kernels for nonparametric identification of a class of nonlinear systems. European control Conference (ECC), p. 540-545, DOI: 10.23919/ECC51009.2020.9143785

7. Blanken, L., Oomen, T. (2020). Kernel-based identification of non-causal systems with application to inverse model control. Automatica, vol. 114, p. 108830, DOI: 10.1016/j.automatica.2020.108830

8. Huh, M. (2015). Kernel-Trick Regression and Classification. Communications for Statistical Applications and Methods, vol. 22, no. 2, 201-207, DOI: 10.5351/CSAM.2015.22.2.201

9. Rojo-Álvarez J.L., Martínez-Ramón M., Muñoz-Marí J., Camps-Valls G. (2018). Kernel Functions and Reproducing Kernel Hilbert Spaces. Digital Signal Processing with Kernel Methods, IEEE, p. 165-207, DOI: 10.1002/9781118705810.ch4

10. Dey, A.U., Harit, G., Hafez, A.H.A. (2018). Greedy Gaussian Process Regression Applied to Object Categorization and Regression. Proceeding of the 11th Indian Conference. In proceeding of the 11th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP 2018), no. 51, p. 1-8, DOI: 10.1145/3293353.3293404

11. Wenzel, T., Santin, G., Haasdonk, B. (2021). A novel class of stabilized greedy kernel approximation algorithms: Convergence, stability and uniform point distribution. Journal of Approximation Theory, vol. 262, 105508.

12. Harbrecht, H., Jakeman, J.D., Zaspel, P. (2021). Cholesky-Based Experimental Design for Gaussian Process and Kernel-Based Emulation and Calibration. Communications in Computational Physics, vol. 29, no. 4, p. 1152-1185, DOI: 10.4208/cicp.OA-2020-0060

13. Zhang, H., Jiang, H., Wang, S. (2020). Kernel Least Mean Square Based on the Sparse Nyström Method. 2020 IEEE International Symposium on Circuits and Systems (ISCAS), p. 1-5, DOI: 10.1109/ISCAS45731.2020.9181116

14. Lei, D., Tang, J., Li, Z., Wu, Y. (2019). Using Low-Rank Approximations to Speed Up Kernel Logistic Regression Algorithm. In IEEE Access, vol. 7, p. 84242-84252, DOI: 10.1109/ACCESS.2019.2924542

15. Niu, W., Xia, K., Zu, B., Bai, J. (2017). Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation. Computational Intelligence and Neuroscience, vol. 2017, 3678487, DOI: 10.1155/2017/3678487

16. He, L., Zhang, H. (2018). Kernel K-Means sampling for Nyström Approximation. IEEE Transactions on Image Processing, p. 2108-2120, DOI: 10.1109/TIP.2018.2796860

17. Li, M., Bi, W., Kwok, J., Lu, B. (2015). Large-scale Nyström kernel matrix approximation using randomized SVD. IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 1, p. 152-164, DOI: 10.1109/TNNLS.2014.2359798

18. Harbrecht, H.,Peters, M., Schneider, R. (2012). On the low-rank approximation by the pivoted Cholesky decomposition. Applied Numerical Mathematics, vol. 62, no. 4, 428-440, DOI: 10.1016/j.apnum.2011.10.001

19. Seth, S., Príncipe, J. C. (2009). On speeding up computation in information theoretic learning. 2009 International Joint Conference on Neural Networks, p. 2883-2887, DOI: 10.1109/IJCNN.2009

20. Saunders, C., Gammerman, A., Vovk, V. (1998). Ridge regression learning algorithm in dual variables. Proceedings of the 15th International Conference on. Machine Learning (ICML), p. 515-521.

21. Kocijan, J. (2016). Modeling and Control of Dynamic Systems Using Gaussian Process Models. Advances in Industrial Control, Springer, Switzerland, DOI: 10.1007/978-3-319-21021-6

22. Rasmussen, C. E., Williams, C. K. I. (2006). Gaussian processes for machine learning. The MIT Press, Cambridge, Massachusetts.

23. Golub, G. H., Van Loan, Ch. F. (1996). Matrix computations, 3 edition. The Johns Hopkins University Press, Baltimore and London, 3 edition.

24. Wang, R., Li, Y. (2018). On the Numerical Rank of Radial Basis Function Kernels in High Dimensions. SIAM Journal on Matrix Analysis and Applications, vol. 39, no. 4, 1810-1835, DOI: 10.1137/17M1135803

25. Press, W. H., Teukolsky, S. A., Vetterling, W. T., Flannery, B. P. (1992). Numerical recipes in C. Cambridge University Press, Second edition.

26. Narendra, K. S., Parthasarathy, K. (1990). Identification and control of dynamical systems using neural networks. IEEE Transactions of Neural Networks, vol. 1, no. 1, 4-27.DOI: 10.1109/72.80202