The Lottery Ticket Hypothesis

EasyChair Preprint 4819

Abstract

Machine learning is a branch of artificial intelligence (AI) which helps the systems to gain abilities in such a way that they can automatically learn and improve from the experience without being precisely programmed. Moreover, it focuses on the development of computer programs so that they can access data and use it to learn by adapting to the environment themselves.

Deep learning is a subset of machine learning, which uses neural networks with many layers. Here, we have used Neural Network pruning to reduce the size of neural networks by compression. We find that these methods can reduce parameter counts of pre-trained subnetworks by 90%. Such techniques help in reducing the size, improving the computational performance and at the same time accuracy remains the same if not better.

A standard pruning technique naturally exhibits subnetworks whose initializations make the sub-networks capable of training efficiently. On the basis of the results above we vocalize the Lottery Ticket Hypothesis. These networks contain sub-networks which when trained in isolation are able to reach the test accuracy equivalent to the original network in a similar number of iterations. We present an algorithm to find the winning tickets which is supported by a few experiments in favour of the lottery ticket hypothesis.

Keyphrases: Artificial Intelligence, Pruning, deep learning, neural network

@booklet{EasyChair:4819,
  author    = {Devansh Punj and Mehar Mutreja and Kunal Khandelwal},
  title     = {The Lottery Ticket Hypothesis},
  howpublished = {EasyChair Preprint 4819},
  year      = {EasyChair, 2020}}