Recently I was preparing for a lecture on autoencoders and I asked myself what is the relevant background literature one needs to read to have a general overview of the topic. The Ian Goodfellow’s and colleagues’ book came to my mind first. It is a good book, still books can be biased due to authors’ research preferences.

An unbiased approach would be to look at bibliographic data. I went through Google Scholar and Microsoft Academic Knowledge base to get a list of highly cited papers. Below I show some results using Google Scholar, as it has a biggest data base.

This first-order analysis is a useful tool to get a general overview of the field.

Next steps can be to look into clustering of papers and compare applied vs fundamental (out-of vs within group) citations.

a clickable Citation ~ Year plot for Autoencoder-related papers


Top-20 references:

Citations Title Authors Year
4870 Reducing the dimensionality of data with neural networks GE Hinton & RR Salakhutdinov 2006
1481 Extracting and composing robust features with denoising autoencoders P Vincent et al. 2008
1373 Spectral hashing Y Weiss et al. 2009
1153 Building high-level features using large scale unsupervised learning QV Le 2013
865 Kernel PCA and De-noising in feature spaces. S Mika et al. 1998
705 Auto-encoding variational bayes DP Kingma & M Welling 2013
676 Multimodal deep learning A Khosla & J Nam 2011
567 Multimodal learning with deep boltzmann machines N Srivastava & RR Salakhutdinov 2012
564 Semi-supervised recursive autoencoders for predicting sentiment distributions R Socher & EH Huang 2011
536 Domain adaptation for large-scale sentiment classification: A deep learning approach X Glorot et al. 2011
438 Contractive auto-encoders: Explicit invariance during feature extraction S Rifai & P Vincent 2011
404 Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection. R Socher & EH Huang 2011
373 Autoencoders, minimum description length and Helmholtz free energy RS Zemel 1994
357 On optimization methods for deep learning A Coates 2011
304 Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure. R Salakhutdinov & GE Hinton 2007
304 A novelty detection approach to classification N Japkowicz et al. 1995
224 Learning a deep compact image representation for visual tracking N Wang & DY Yeung 2013
224 Measuring invariances in deep networks I Goodfellow et al. 2009
214 Stacked convolutional auto-encoders for hierarchical feature extraction J Masci et al. 2011
208 The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training. D Erhan et al. 2009