SCIENCE CHINA Information Sciences, Volume 61, Issue 11: 112209(2018) https://doi.org/10.1007/s11432-017-9421-3

## Discriminative graph regularized broad learning system for image recognition

• AcceptedMar 30, 2018
• PublishedOct 17, 2018
Share
Rating

### Abstract

Broad learning system (BLS) has been proposed as an alternative method of deep learning. The architecture of BLS is that the input is randomly mapped into series of feature spaces which form the feature nodes, and the output of the feature nodes are expanded broadly to form the enhancement nodes, and then the output weights of the network can be determined analytically. The most advantage of BLS is that it can be learned incrementally without a retraining process when there comes new input data or neural nodes. It has been proven that BLS can overcome the inadequacies caused by training a large number of parameters in gradient-based deep learning algorithms. In this paper, a novel variant graph regularized broad learning system (GBLS) is proposed. Taking account of the locally invariant property of data, which means the similar images may share similar properties, the manifold learning is incorporated into the objective function of the standard BLS. In GBLS, the output weights are constrained to learn more discriminative information, and the classification ability can be further enhanced. Several experiments are carried out to verify that our proposed GBLS model can outperform the standard BLS. What is more, the GBLS also performs better compared with other state-of-the-art image recognition methods in several image databases.

### Acknowledgment

This work was supported in part by National Natural Science Foundation of China (Grant No. 61572540), Macau Science and Technology Development Fund (FDCT) (Grant Nos. 019/2015/A, 024/2015/AMJ, 079/2017/A2), and the University Macau MYR Grants.

### References

[1] Feng S, Chen C L P. A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers. IEEE Trans Fuzzy Syst, 2018, 26: 117-130 CrossRef Google Scholar

[2] Chen C L P, Zhang C Y, Chen L. Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning. IEEE Trans Fuzzy Syst, 2015, 23: 2163-2173 CrossRef Google Scholar

[3] Cai S J, Zhang L, Zuo W M, et al. A probabilistic collaborative representation based approach for pattern classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. Google Scholar

[4] Xie Z, Zeng Z, Zhou G. Topic enhanced deep structured semantic models for knowledge base question answering. Sci China Inf Sci, 2017, 60: 110103 CrossRef Google Scholar

[5] Qu W, Wang D, Feng S. A novel cross-modal hashing algorithm based on multimodal deep learning. Sci China Inf Sci, 2017, 60: 092104 CrossRef Google Scholar

[6] Pao Y H, Takefuji Y. Functional-link net computing: theory, system architecture, and functionalities. Computer, 1992, 25: 76-79 CrossRef Google Scholar

[7] Pao Y H, Phillips S M, Sobajic D J. Neural-net computing and the intelligent control of systems. Int J Control, 1992, 56: 263-289 CrossRef Google Scholar

[8] Igelnik B, Yoh-Han Pao B. Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans Neural Netw, 1995, 6: 1320-1329 CrossRef PubMed Google Scholar

[9] Klassen M, Pao Y H, Chen V. Characteristics of the functional-link net: a higher order delta rule net. In: Proceedings of International Conference on Neural Networks, San Diego, 1988. Google Scholar

[10] Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks, 1991, 4: 251-257 CrossRef Google Scholar

[11] Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2: 359-366 CrossRef Google Scholar

[12] Chen C L P. A rapid supervised learning neural network for function interpolation and approximation. IEEE Trans Neural Netw, 1996, 7: 1220-1230 CrossRef PubMed Google Scholar

[13] Chen C L P, LeClair S R, Pao Y H. An incremental adaptive implementation of functional-link processing for function approximation, time-series prediction, and system identification. Neurocomputing, 1998, 18: 11-31 CrossRef Google Scholar

[14] Chen C L P, Wan J Z. A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction. IEEE Trans Syst Man Cybern B, 1999, 29: 62-72 CrossRef PubMed Google Scholar

[15] Chen C L P, Zhang C Y. Data-intensive applications, challenges, techniques and technologies: A survey on Big Data. Inf Sci, 2014, 275: 314-347 CrossRef Google Scholar

[16] Chen C L P, Liu Z. Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture. IEEE Trans Neural Netw Learning Syst, 2018, 29: 10-24 CrossRef PubMed Google Scholar

[17] Chen C L P, Liu Z, Feng S. Universal Approximation Capability of Broad Learning System and Its Structural Variations.. IEEE Trans Neural Netw Learning Syst, 2018, : 1-14 CrossRef PubMed Google Scholar

[18] Feng S, Chen C L P. Fuzzy Broad Learning System: A Novel Neuro-Fuzzy Model for Regression and Classification.. IEEE Trans Cybern, 2018, : 1-11 CrossRef PubMed Google Scholar

[19] Miao S, Wang J, Gao Q. Discriminant structure embedding for image recognition. Neurocomputing, 2016, 174: 850-857 CrossRef Google Scholar

[20] Fang Y, Wang R, Dai B. Graph-Based Learning via Auto-Grouped Sparse Regularization and Kernelized Extension. IEEE Trans Knowl Data Eng, 2015, 27: 142-154 CrossRef Google Scholar

[21] Yan S, Xu D, Zhang B. Graph Embedding and Extensions: A General Framework for Dimensionality Reduction. IEEE Trans Pattern Anal Mach Intell, 2007, 29: 40-51 CrossRef Google Scholar

[22] Liao Y, Wang Y, Liu Y. Graph Regularized Auto-Encoders for Image Representation. IEEE Trans Image Process, 2017, 26: 2839-2852 CrossRef PubMed ADS Google Scholar

[23] Gao Q X, Ma J J, Zhang H L. Stable Orthogonal Local Discriminant Embedding for Linear Dimensionality Reduction. IEEE Trans Image Process, 2013, 22: 2521-2531 CrossRef PubMed ADS Google Scholar

[24] Xue H, Chen S, Yang Q. Discriminatively regularized least-squares classification. Pattern Recognition, 2009, 42: 93-104 CrossRef Google Scholar

[25] Peng Y, Wang S, Long X. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing, 2015, 149: 340-353 CrossRef Google Scholar

[26] Huang G B, Mattar M, Berg T, et al. Labeled faces in the wild: a database forstudying face recognition in unconstrained environments. In: Proceedings of Workshop on Faces in `Real-Life' Images: Detection, Alignment, and Recognition, Marseille, 2008. Google Scholar

[27] Guo P, Chen C L P, Lyu M R. Cluster number selection for a small set of samples using the Bayesian Ying-Yang model. IEEE Trans Neural Netw, 2002, 13: 757-763 CrossRef PubMed Google Scholar

[28] Guo P, Lyu M R, Chen C L P. Regularization parameter estimation for feedforward neural networks. IEEE Trans Syst Man Cybern B, 2003, 33: 35-44 CrossRef PubMed Google Scholar

[29] Naseem I, Togneri R, Bennamoun M. Linear regression for face recognition. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 2106-2112 CrossRef PubMed Google Scholar

[30] Wright J, Yang A Y, Ganesh A. Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell, 2009, 31: 210-227 CrossRef PubMed Google Scholar

[31] Zhang L, Yang M, Feng X C. Sparse representation or collaborative representation: which helps face recognition? In: Proceedings of International Conference on Computer Vision, Barcelona, 2011. Google Scholar

[32] Peng X, Zhang L, Yi Z. Learning locality-constrained collaborative representation for robust face recognition. Pattern Recognition, 2014, 47: 2794-2806 CrossRef Google Scholar

[33] Yao B P, Jiang X Y, Khosla A, et al. Human action recognition by learning bases of action attributes and parts. In: Proceedings of International Conference on Computer Vision, Barcelona, 2011. Google Scholar

[34] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014,. arXiv Google Scholar

[35] Chi Y, Porikli F. Classification and Boosting with Multiple Collaborative Representations. IEEE Trans Pattern Anal Mach Intell, 2014, 36: 1519-1531 CrossRef PubMed Google Scholar

[36] Diba A, Mohammad A, Pirsiavash H, et al. Deepcamp: deep convolutional action & attribute mid-level patterns. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016. Google Scholar

[37] Wang L M, Qiao Y, Tang X O, et al. Actionness estimation using hybrid fully convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. Google Scholar

[38] Razavian A S, Azizpour H, Sullivan J, et al. CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, 2014. Google Scholar

[39] Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: delving deep into convolutional nets. 2014,. arXiv Google Scholar

[40] Cimpoi M, Maji S, Vedaldi A. Deep filter banks for texture recognition and segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 2015. Google Scholar

• Figure 1

The structure of BLS. First, the input features are randomly mapped into a series of features spaces. Second, the random features are transformed to the enhancement nodes. At the output layer, all the features are connected together to link the label layer.

• Figure 2

Sample from three face databases. Samples of three subjects in (a) the ORL databases, (b) the ExYaB database, and (c) the UMIST database.

• Figure 3

(Color online) Sample images from the four challenging visual database. The first row shows ten kinds of action in Standford $40$ Actions database; the second row shows various kinds of birds in CUB200-2011 database; the third row shows ten kinds of flowers in Flower 102 database; the forth row shows ten kinds of objects in Caltech 256 database.

• Figure 4

(Color online) Effects of parameters combinations of $(\lambda_1,\lambda_2)$ in IGBLS on UMIST. (a) ${\rm~Tn}=5$, (b) ${\rm~Tn}=10$, (c) ${\rm~Tn}=15$.

• Figure 5

(Color online) Effects of parameters combinations of $(\lambda_1,\lambda_2)$ in IPGBLS on UMIST. (a) ${\rm~Tn}=5$, (b) ${\rm~Tn}=10$, (c) ${\rm~Tn}=15$.

• Figure 6

(Color online) Effects of parameters combinations of $(k1,k2)$ in IPGBLS on UMIST. (a) ${\rm~Tn}=5$, (b) ${\rm~Tn}=10$, (c) ${\rm~Tn}=15$.

• Table 1   Parameters setting of BLS, IGBLS, IPGBLS on different databases
 2*Number BLS IGBLS IPGBLS $N_w$ $N_f$ $N_e$ $N_w$ $N_f$ $N_e$ $N_w$ $N_f$ $N_e$ 3*ORL ${\rm~Tn}~=~5$ $20$ $25$ $500$ $15$ $20$ $380$ $20$ $20$ $400$ ${\rm~Tn}~=~6$ $10$ $26$ $460$ $12$ $16$ $400$ $20$ $20$ $640$ ${\rm~Tn}~=~7$ $20$ $20$ $400$ $20$ $20$ $240$ $22$ $20$ $500$ 3*ExYaB ${\rm~Tn}~=~10$ $30$ $60$ $3000$ $20$ $50$ $2000$ $40$ $50$ $1300$ ${\rm~Tn}~=~20$ $30$ $60$ $4000$ $20$ $50$ $3000$ $30$ $35$ $3000$ ${\rm~Tn}~=~30$ $30$ $60$ $5000$ $30$ $40$ $2000$ $34$ $40$ $2000$ 3*UMIST ${\rm~Tn}~=~5$ $10$ $9$ $400$ $11$ $9$ $900$ $11$ $9$ $900$ ${\rm~Tn}~=~10$ $30$ $20$ $300$ $15$ $10$ $300$ $12$ $9$ $860$ ${\rm~Tn}~=~15$ $10$ $9$ $575$ $10$ $9$ $300$ $11$ $11$ $400$
•

Algorithm 1 Discriminative GBLS models

indent 4.0em

Random ${\boldsymbol~W}_{e_i}$, $\pmb{\beta}_{e_i}$, $i=1,2,\ldots,N_w$;

Calculate ${\boldsymbol~Z}_i=\phi(\boldsymbol{XW}_{e_i}+\pmb{\beta}_{e_i})$, $i=1,2,\ldots,N_w$;

Set the feature mapping group ${\boldsymbol~Z}^n=[{\boldsymbol~Z}_1,{\boldsymbol~Z}_2,\ldots,{\boldsymbol~Z}_n]$;

Step 2:

indent 4.0em

Random ${\boldsymbol~W}_h$, $\pmb{\beta}_h$;

Calculate ${\boldsymbol~H}^m=\xi({\boldsymbol~Z}^n{\boldsymbol~W}_h+\pmb{\beta}_h)$;

Step 3:

indent 4.0em

Set ${\boldsymbol~A}=[{\boldsymbol~Z}^n,{\boldsymbol~H}^m]$;

Construct the adjacent matrix ${\boldsymbol~V}$ by 7 and 12; then calculate the graph regulation term by 8 and 16;

Calculate the output weight matrix ${\boldsymbol~W}$ by 11 and 18.

Require:training set $\{\boldsymbol{X,Y}\}$, the feature mapping function $\phi(\cdot)$, the activation function $\xi(\cdot)$, the number of feature mapping groups $N_w$, feature nodes $N_f$, enhancement nodes $N_e$, regularization parameter $(\lambda_1,\lambda_2)$ and NNs $(k1,~k2)$;

Output:Output weight ${\boldsymbol~W}$;

Step 1:

• Table 2   Recogniton results and running time of standard BLS, IGBLS, IPGBLS on different databases$^{\rm~a)}$
 2*Number BLS IGBLS IPGBLS Training (%) Testing (%) Time (s) Training (%) Testing (%) Time (s) Training (%) Testing (%) Time (s) 3*ORL ${\rm~Tn}~=~5$ $100$ $95.00$ $0.44$ $100$ $96.11$ $0.32$ $100$ $\mathbf{97.50}$ $0.35$ ${\rm~Tn}~=~6$ $100$ $97.50$ $0.37$ $100$ $98.11$ $0.31$ $100$ $\mathbf{98.13}$ $0.35$ ${\rm~Tn}~=~7$ $100$ $98.33$ $0.41$ $100$ $\mathbf{99.17}$ $0.39$ $100$ $99.14$ $0.35$ 3*ExYaB ${\rm~Tn}~=~10$ $100$ $85.60$ $2.96$ $100$ $88.35$ $1.62$ $100$ $\mathbf{90.12}$ $2.25$ ${\rm~Tn}~=~20$ $100$ $95.59$ $4.05$ $100$ $96.61$ $2.76$ $100$ $\mathbf{97.28}$ $2.87$ ${\rm~Tn}~=~30$ $100$ $97.17$ $5.67$ $100$ $98.82$ $2.82$ $100$ $\mathbf{98.98}$ $2.96$ 3*UMIST ${\rm~Tn}~=~5$ $100$ $84.21$ $0.38$ $100$ $87.16$ $0.24$ $100$ $\mathbf{88.21}$ $0.26$ ${\rm~Tn}~=~10$ $100$ $96.53$ $0.70$ $100$ $97.60$ $0.26$ $100$ $\mathbf{98.13}$ $0.27$ ${\rm~Tn}~=~15$ $100$ $98.18$ $0.78$ $100$ $98.54$ $0.36$ $100$ $\mathbf{99.27}$ $0.31$

a) The bold number means the best result.

• Table 3   The recognition accuracy and running time of competing algorithms on ExYaB$^{\rm~a)}$
 2*Method ${\rm~Dim}=84$ ${\rm~Dim}=150$ ${\rm~Dim}=300$ Accuracy (%) time (s) Accuracy (%) time (s) Accuracy (%) time (s) SVM $94.9$ $5.87$ $96.4$ $6.58$ $97.0$ $8.26$ NN $85.8$ $3.89$ $90.0$ $4.09$ $91.6$ $4.76$ SRC $95.5$ $180.90$ $96.8$ $205.02$ $97.9$ $261.38$ LRC $94.5$ $4.28$ $95.1$ $4.72$ $95.9$ $6.49$ CRC$\_$RLS $95.0$ $2.12$ $96.3$ $2.64$ $97.9$ $3.72$ GELM $94.45$ $6.10$ $95.21$ $6.90$ $96.69$ $7.41$ BLS $93.40$ $0.87$ $95.05$ $1.73$ $96.41$ $2.25$ IGBLS $95.69$ $0.83$ $96.90$ $1.87$ $98.21$ $2.31$ IPGBLS $\mathbf{96.11}$ $1.12$ $\mathbf{97.67}$ $1.91$ $\mathbf{98.36}$ $2.46$

a) The bold number means the best result.

• Table 4   The recognition accuracy and running time of competing algorithms on the LFW database with threefeatures$^{\rm~a)}$
 2*Method FFT Gabor LBP 2*Average (%) Accuracy (%) time (s) Accuracy (%) time (s) Accuracy (%) time (s) SVM $5.8$ $9.62$ $42.4$ $9.74$ $18.5$ $10.96$ $22.3$ SRC $33.6$ $2230$ $68.7$ $2236$ $61.6$ $2238$ $54.6$ LRC $13.9$ $19.35$ $25.4$ $20.05$ $26.3$ $23.27$ $21.9$ CRC$\_$RLS $14.0$ $9.11$ $25.4$ $9.22$ $26.3$ $9.44$ $21.9$ LCCR $22.2$ $11.49$ $64.6$ $11.62$ $66.5$ $11.74$ $51.1$ GELM $33.5$ $5.23$ $67.8$ $4.96$ $58.86$ $5.68$ $53.4$ BLS $35.2$ $0.28$ $69.90$ $0.33$ $65.3$ $0.57$ $56.8$ IGBLS $38.34$ $0.37$ $\mathbf{71.81}$ $0.35$ $66.44$ $0.53$ $58.86$ IPGBLS $\mathbf{38.85}$ $0.37$ $71.32$ $0.41$ $\mathbf{66.76}$ $0.48$ $\mathbf{58.98}$

a) The bold number means the best result.

• Table 5   Classification performance and running time of different classification on four databases$^{\rm~a)}$
 2*Method Standford 40 Flower 102 CUB200-2011 Caltech 256 Accuracy (%) Time (s) Accuracy (%) Time (s) Accuracy (%) Time (s) Accuracy (%) Time (s) SVM $79.0$ $26.97$ $90.9$ $30.67$ $75.4$ $51.63$ $80.1$ $228.52$ Kernel SVM $79.8$ $296.01$ $92.2$ $377.24$ $76.6$ $691.84$ $81.3$ $3085$ NSC $74.7$ $47.16$ $90.1$ $67.36$ $74.5$ $98.38$ $80.2$ $487.27$ CRC$\_$RLS $78.2$ $25.78$ $93.0$ $30.74$ $76.2$ $49.36$ $81.1$ $234.27$ SRC $78.7$ $2655$ $93.2$ $3228$ $76.0$ $5282$ $81.3$ $25535$ CROC $79.1$ $56.32$ $93.1$ $74.71$ $76.2$ $109.76$ $81.7$ $490.53$ ProCRC $80.9$ $26.82$ $94.8$ $32.48$ $78.3$ $52.27$ $83.3$ $234.69$ GELM $78.7$ $54.63$ $90.3$ $55.11$ $76.7$ $57.75$ $81.8$ $69.89$ BLS $81.4$ $14.32$ $95.3$ 20.26 $78.8$ $20.74$ $84.0$ $26.37$ IGBLS $81.7$ $15.61$ $95.1$ $20.33$ $79.5$ $21.88$ $84.6$ $28.65$ IPGBLS $\mathbf{82.3}$ $15.92$ $\mathbf{95.6}$ $21.47$ $\mathbf{80.4}$ $23.46$ $\mathbf{84.9}$ $30.42$

a) The bold number means the best result.

• Table 6   Comparsions to the state-of-the-arts on four challenging visual datasets$^{\rm~a)}$
 2*Algorithms Recognition rate (%) Standford 40 Flower 102 CUB200-2011 Caltech 256 IGBLS $81.7$ $95.1$ $79.5$ $84.6$ IPGBLS $\mathbf{82.3}$ $\mathbf{95.6}$ $\mathbf{80.4}$ $84.9$ DeepCAMP [36] $52.6$ – – – A-FCN [37] $79.7$ – – – CNN-SVM [38] – $74.7$ – – CNNaug-SVM [38] – $86.8$ $66.7$ – FV-CNN [40] – – $61.8$ – VGG19 [34] – – – $\mathbf{85.1}$ CNN-S [39] – – – $77.6$

a) The bold number means the best result.

Citations

• #### 0

Altmetric

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有