Spectral data acquired through remote sensing are invaluable for environmental and resource studies. However, these datasets are often marred by nuisance phenomena such as atmospheric interference and other complexities, which pose significant challenges for accurate analysis. We show that an autoencoder architecture, called symmetric autoencoder (SymAE), which leverages symmetry under reordering of the pixels, can learn to disentangle the influence of these nuisance from surface reflectance features on a pixel-by-pixel basis. The disentanglement provides an alternative to atmospheric correction, without relying on radiative transfer modelling, through a purely data-driven process. More importantly, SymAE can generate virtual hyperspectral images by manipulating the nuisance effects of each pixel. We demonstrate using AVIRIS instrument data that these virtual images are valuable for subsequent image analysis tasks. We also show SymAE’s ability to extract intra-class invariant features, which is very useful in clustering and classification tasks, delivering state-of-the-art classification performance for a purely spectral method.