Pre-Trained Models: Past, Present and Future

A deep look into the history of pre-training, especially its special relation with transfer learning and self-supervised learning, is taken to reveal the crucial position of PTMs in the AI development spectrum.