A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 7 Issue 5
Sep.  2020

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 17.6, Top 3% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
Xia Chen, Zhan-Li Sun, Kin-Man Lam and Zhigang Zeng, "A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1455-1464, Sept. 2020. doi: 10.1109/JAS.2020.1003006
Citation: Xia Chen, Zhan-Li Sun, Kin-Man Lam and Zhigang Zeng, "A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1455-1464, Sept. 2020. doi: 10.1109/JAS.2020.1003006

A Local Deviation Constraint Based Non-Rigid Structure From Motion Approach

doi: 10.1109/JAS.2020.1003006
Funds:  The work was supported by the National Natural Science Foundation of China (61972002), and Open Grant from Anhui Province Key Laboratory of Non-Destructive Evaluation (CGHBMWSJC07)
More Information
  • In many traditional non-rigid structure from motion (NRSFM) approaches, the estimation results of part feature points may significantly deviate from their true values because only the overall estimation error is considered in their models. Aimed at solving this issue, a local deviation-constrained-based column-space-fitting approach is proposed in this paper to alleviate estimation deviation. In our work, an effective model is first constructed with two terms: the overall estimation error, which is computed by a linear subspace representation, and a constraint term, which is based on the variance of the reconstruction error for each frame. Furthermore, an augmented Lagrange multipliers (ALM) iterative algorithm is presented to optimize the proposed model. Moreover, a convergence analysis is performed with three steps for the optimization process. As both the overall estimation error and the local deviation are utilized, the proposed method can achieve a good estimation performance and a relatively uniform estimation error distribution for different feature points. Experimental results on several widely used synthetic sequences and real sequences demonstrate the effectiveness and feasibility of the proposed algorithm.

     

  • loading
  • [1]
    X. Wang and H. Duan, “Hierarchical visual attention model for saliency detection inspired by avian visual pathways,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 2, pp. 540–552, 2019. doi: 10.1109/JAS.2017.7510664
    [2]
    Y. Tian, X. Li, K. Wang, and F.-Y. Wang, “Training and testing object detectors with virtual images,” IEEE/CAA J. Autom. Sinica, vol. 5, no. 2, pp. 539–546, 2018. doi: 10.1109/JAS.2017.7510841
    [3]
    Z. Q. Zhao, P. Zheng, S. T. Xu, and X. Wu, “Object detection with deep learning: A review,” IEEE Trans. Neural Networks and Learning Systems, 2019. doi: 10.1109/TNNLS.2018.2876865
    [4]
    D. Herrera, F. Roberti, M. Toibero M, and R. Carelli, “Human interaction dynamics for its use in mobile robotics: Impedance control for leaderfollower formation,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 696–703, 2017. doi: 10.1109/JAS.2017.7510631
    [5]
    Z. Q. Zhao, H. Glotin, Z. Xie, J. Gao, and X. Wu, “Cooperative sparse representation in two opposite directions for semi-supervised image annotation,” IEEE Trans. Image Processing, vol. 21, no. 9, pp. 4218–4231, 2012. doi: 10.1109/TIP.2012.2197631
    [6]
    D. S. Huang, H. H. S. Ip, and Z. Chi, “A neural root finder of polynomials based on root moments,” Neural Computation, vol. 16, no. 8, pp. 1721–1762, 2004. doi: 10.1162/089976604774201668
    [7]
    D. S. Huang, “A constructive approach for finding arbitrary roots of polynomials by neural networks,” IEEE Trans. Neural Networks, vol. 15, no. 2, pp. 477–491, 2004. doi: 10.1109/TNN.2004.824424
    [8]
    X. Sun, S. Shen, H. Cui, L. Hu, and Z. Hu, “Geographic, geometrical and semantic reconstruction of urban scene from high resolution oblique aerial images,” IEEE/CAA J. Autom. Sinica, vol. 6, no. 1, pp. 118–130, 2019. doi: 10.1109/JAS.2017.7510673
    [9]
    N. Hao, H. Liao, Y. Qiu, and J. Yang, “Face super-resolution reconstruction and recognition using non-local similarity dictionary learning based algorithm,” IEEE/CAA J. Autom. Sinica, vol. 3, no. 2, pp. 213–224, 2016. doi: 10.1109/JAS.2016.7451109
    [10]
    Z. L. Sun, K. M. Lam, and Q. W. Gao, “An effective missing-data estimation approach for small-size image sequences,” IEEE Computational Intelligence Magazine, vol. 10, no. 3, pp. 10–18, 2015. doi: 10.1109/MCI.2015.2437311
    [11]
    S. Kumar, A. Cherian, Y. Dai, and H. Li, “Scalable dense non-rigid structure-from-motion: A grassmannian perspective,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 254–263.
    [12]
    Z. Zhou, F. Shi, J. Xiao, and W. Wu, “Non-rigid structure-from-motion on degenerate deformations with low-rank shape deformation model,” IEEE Trans. Multimedia, vol. 17, no. 2, pp. 171–185, 2015. doi: 10.1109/TMM.2014.2384396
    [13]
    C. Bregler, A. Hertzmann, and H. Biermann, “Recovering non-rigid 3D shape from image streams,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 690–696.
    [14]
    J. Xiao, J. Chai, and T. Kanade, “A closed-form solution to non-rigid shape and motion recovery,” Int. J. Computer Vision, vol. 67, no. 2, pp. 233–246, 2006. doi: 10.1007/s11263-005-3962-9
    [15]
    L. Torresani, A. Hertzmann, and C. Bregler, “Nonrigid structure-frommotion: estimating shape and motion with hierarchical priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878–892, 2008. doi: 10.1109/TPAMI.2007.70752
    [16]
    Z. L. Sun and M. K. Lam, “Depth estimation of face images based on the constrained ICA model,” IEEE Trans. Information Forensics and Security, vol. 6, no. 2, pp. 360–370, 2011. doi: 10.1109/TIFS.2011.2118207
    [17]
    Q. Dong and H. Wang, “Latent-smoothness nonrigid structure from motion by revisiting multilinear factorization,” IEEE Trans. Cybernetics, vol. 49, no. 9, pp. 3557–3570, 2018.
    [18]
    Y. Dai, H. Li, and M. He, “A simple prior-free method for non-rigid structure-from-motion factorization,” Int. J. Computer Vision, vol. 107, no. 2, pp. 101–122, 2014. doi: 10.1007/s11263-013-0684-2
    [19]
    S. Kumar, “Jumping manifolds: Geometry aware dense non-rigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2019, pp. 5346–5355.
    [20]
    J. W. Lu, G. Wang, W. H. Deng, and K. Jia, “Reconstruction-based metric learning for unconstrained face verification,” IEEE Trans. Information Forensics and Security, vol. 10, no. 1, pp. 79–89, 2015. doi: 10.1109/TIFS.2014.2363792
    [21]
    A. Agudo, F. Moreno-Noguer, B. Calvo, and J. M. M. Montiel, “Sequential non-rigid structure from motion using physical priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 5, pp. 979–994, 2016. doi: 10.1109/TPAMI.2015.2469293
    [22]
    A. M. Gallardo, T. Collins, A. Bartoli, and F. Mathias, “Dense nonrigid structure-from-motion and shading with unknown,” in Proc. IEEE Int. Conf. Computer Vision, 2017, pp. 3884–3892.
    [23]
    A. Agudo, M. Pijoan, and F. Moreno-Noguer, “Dust: Dual union of spatio-temporal subspaces for monocular multiple object 3D reconstruction,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2017, pp. 1513–1521.
    [24]
    A. Agudo, M. Pijoan, and F. Moreno-Noguer, “Image collection popup: 3D reconstruction and clustering of rigid and non-rigid categories,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 2607–2615.
    [25]
    I. Akhter, Y. Sheikh, and S. Khan, “In defense of orthonormality constraints for nonrigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 1534–1541.
    [26]
    I. Akhter, Y. A. Sheikh, S. Khan, and T. Kanade, “Trajectory space: A dual representation for nonrigid structure from motion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1442–1456, 2011. doi: 10.1109/TPAMI.2010.201
    [27]
    P. F. U. Gotardo and A. M. Martinez, “Computing smooth time trajectories for camera and deformable shape in structure from motion with occlusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 2051–2065, 2011.
    [28]
    P. F. U. Gotardo and A. M. Martinez, “Non-rigid structure from motion with complementary rank-3 spaces,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 3065–3072.
    [29]
    X. Zhou, M. Zhu, G. Pavlakos, S. Leonardos, K. G. Derpanis, and K. Daniilidis, “Monocap: Monocular human motion capture using a CNN coupled with a geometric prior,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 41, no. 4, pp. 901–914, 2018.
    [30]
    Y. Wang, X. Yan, M. Jiang, and J. Zheng, “3D Non-rigid structure from motion based on space approximation in trajectory space,” Int. J. Robotics and Automation, vol. 33, no. 2, pp. 111–117, 2018.
    [31]
    Z. Lin, M. Chen, and Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” UIUC Technicial Report UIUL-ENG-09-2214, 2010.
    [32]
    M. R. Hestenes, “Multiplier and gradient methods,” J. Optimization Theory and Applications, vol. 4, no. 5, pp. 303–320, 1969. doi: 10.1007/BF00927673
    [33]
    D. P. Bertsekas, “Constrained optimization and Lagrange multiplier methods,” New York: Academic Press, 2014.
    [34]
    C. Li, C. L. Wang, and J. Wang, “Convergence analysis of the augmented Lagrange multiplier algorithm for a class of matrix compressive recovery,” Applied Mathematics Letters, vol. 59, pp. 12–17, 2016. doi: 10.1016/j.aml.2016.02.022
    [35]
    O. C. Hamsici, P. F. U. Gotardo, and A. M. Martinez, “Learning spatially-smooth mappings in non-rigid structure from motion,” in Proc. European Conf. Computer Vision, 2012, pp. 260–273.
    [36]
    M. Lee, J. Cho, and S. Oh, “Consensus of non-rigid reconstructions”, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 4670–4678.
    [37]
    P. F. U. Gotardo, and A. M. Martinez, “Kernel non-rigid structure from motion”, in Proc. IEEE Int. Conf. Computer Vision, 2011, pp. 802–809.

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(6)  / Tables(4)

    Article Metrics

    Article views (1206) PDF downloads(51) Cited by()

    Highlights

    • An effective model is constructed by considering both the overall estimation error and the variance of reconstruction errors for each frame.
    • An Augmented Lagrange Multipliers (ALM) iterative algorithm is developed to optimize the local deviation-constrained-based estimation model.
    • A convergence analysis is carried out in detail for the model optimization.

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return