This paper introduces stochastic sparse adversarial attacks (SSAA), simple, fast and purely noise-based targeted and untargeted L_0 attacks of neural network classifiers (NNC). SSAA are devised by exploiting a simple small-time expansion idea widely used for Markov processes and offer new examples of L_0 attacks whose studies have been limited. They are designed to solve the known scalability issue of the family of Jacobian-based saliency maps attacks to large datasets and they succeed in solving it. Experiments on small and large datasets (CIFAR-10 and ImageNet) illustrate further advantages of SSAA in comparison with the-state-of-the-art methods. For instance, in the untargeted case, our method called Voting Folded Gaussian Attack (VFGA) scales efficiently to ImageNet and achieves a significantly lower L_0 score than SparseFool (up to 25 lower) while being faster. Moreover, VFGA achieves better L_0 scores on ImageNet than Sparse-RS when both attacks are fully successful on a large number of samples. Codes are publicly available through the link https://github.com/SSAA3/stochastic-sparse-adv-attacks