Thermal images can be obtained as either grayscale images or pseudo colored images based on the thermal profile of the object being captured. In this work, we explore what an optimal encoder decoder might look like for creating a thermal-optical fused domain image. We compare the results from several different encoder-decoder structures with different networks to answer this question. This output images obtained from our method provides information of both domains jointly in a colorized image. We call this a cross domain colorized image. We also present a robust registration method for thermal and optical pairs, which can work despite changes in resolution and the make of the thermal imager. Lastly, we present a unique public dataset with registered thermal-visual image pairs containing around 1800 images as a part of this work, collected over a period of 2 years. We compare our results with prior literature, show how our results are different and discuss on some future work that can be explored further in this domain.