Despite being studied extensively for a few decades, handwritten character recognition (HCR) is still considered a challenging learning problem in pattern recognition, and there is very limited research on script independent models. This is mainly because of similarity in structure of characters, different handwriting styles, noisy datasets, diversity of scripts, focus of the conventional research on handcrafted feature extraction techniques, and unavailability of public datasets and code-repositories to reproduce the results. On the other hand, deep learning has witnessed huge success in different areas of pattern recognition, including HCR, and provides an end-to-end learning. However, deep learning techniques are computationally expensive, need large amount of data for training and have been developed for specific scripts only. To address the above limitations, we have proposed a novel generic deep learning architecture for script independent handwritten character recognition, called HCR-Net. HCR-Net is based on a novel transfer learning approach for HCR, which partly utilizes feature extraction layers of a pre-trained network. Due to transfer learning and image-augmentation, HCR-Net provides faster and computationally efficient training, better performance and better generalizations, and can work with small datasets. HCR-Net is extensively evaluated on 40 publicly available datasets of Bangla, Punjabi, Hindi, English, Swedish, Urdu, Farsi, Tibetan, Kannada, Malayalam, Telugu, Marathi, Nepali and Arabic languages, and established 26 new benchmark results while performed close to the best results in the rest cases. HCR-Net showed performance improvements up to 11% against the existing results and achieved a fast convergence rate showing up to 99% of final performance in the very first epoch. HCR-Net significantly outperformed the state-of-the-art transfer learning techniques...