This work proposes NePhi, a neural deformation model which results in approximately diffeomorphic transformations. In contrast to the predominant voxel-based approaches, NePhi represents deformations functionally which allows for memory-efficient training and inference. This is of particular importance for large volumetric registrations. Further, while medical image registration approaches representing transformation maps via multi-layer perceptrons have been proposed, NePhi facilitates both pairwise optimization-based registration as well as learning-based registration via predicted or optimized global and local latent codes. Lastly, as deformation regularity is a highly desirable property for most medical image registration tasks, NePhi makes use of gradient inverse consistency regularization which empirically results in approximately diffeomorphic transformations. We show the performance of NePhi on two 2D synthetic datasets as well as on real 3D lung registration. Our results show that NePhi can achieve similar accuracies as voxel-based representations in a single-resolution registration setting while using less memory and allowing for faster instance-optimization.