The Symmetric group S_n manifests itself in large classes of quantum systems as the invariance of certain characteristics of a quantum state with respect to permuting the qubits. The subgroups of S_n arise, among many other contexts, to describe label symmetry of classical images with respect to spatial transformations, e.g. reflection or rotation. Equipped with the formalism of geometric quantum machine learning, in this work we propose the architectures of equivariant quantum convolutional neural networks (EQCNNs) adherent to S_n and its subgroups. We demonstrate that a careful choice of pixel-to-qubit embedding order can facilitate easy construction of EQCNNs for small subgroups of S_n. Our novel EQCNN architecture corresponding to the full permutation group S_n is built by applying all possible QCNNs with equal probability, which can also be conceptualized as a dropout strategy in quantum neural networks. For subgroups of S_n, our numerical results using MNIST datasets show better classification accuracy than non-equivariant QCNNs. The S_n-equivariant QCNN architecture shows significantly improved training and test performance than non-equivariant QCNN for classification of connected and non-connected graphs. When trained with sufficiently large number of data, the S_n-equivariant QCNN shows better average performance compared to S_n-equivariant QNN . These results contribute towards building powerful quantum machine learning architectures in permutation-symmetric systems.