Factorized Bilinear Models for Image Recognition


Fig.1 Expanding FB model structure and DropFactor technology.


In this paper, we propose a novel Factorized Bilinear (FB) layer to model the pairwise feature inter- actions by considering the quadratic terms in the transformations. Compared with existing methods that tried to in- corporate complex non-linearity structures into CNNs, the factorized parameterization makes our FB layer only re- quire a linear increase of parameters and affordable computational cost. To further reduce the risk of overfitting of the FB layer, a specific remedy called DropFactor is devised during the training process. We also analyze the connection between FB layer and some existing models, and show FB layer is a generalization to them. Finally, we validate the effectiveness of FB layer on several widely adopted datasets including CIFAR-10, CIFAR-100 and ImageNet, and demonstrate superior results compared with various state-of-the-art deep models.


  • Paper: arXiv
  • Code: Github Code
  • Citation

    @article{li2016factorized,   title={Factorized Bilinear Models for Image Recognition},   author={Li, Yanghao and Wang, Naiyan and Liu, Jiaying and Hou, Xiaodi},   booktitle={International Conference on Computer Vision},   year={2017} }


    Fig. 2 Top-1 error (%) of different methods on CIFAR-10 and CIFAR-100 datasets using moderate data augmentation (flip/translation).

    Fig. 3 Comparisons of different methods by single center-crop error on the ImagNet validation set.