AI On-board of Satellites, Towards Autonomous Scientific Instruments

AI On-board of Satellites, Towards Autonomous Scientific Instruments - Vit Ruzicka, NASA - Jet Propulsion Laboratory I recently finished my PhD at the University of Oxford focusing on deployment of ML models on-board of satellites. I would like to share my research and outline several future points that I am currently exploring as a Postdoc at NASA JPL. In 2023, our ML model was deployed on-board of a satellite by D-Orbit. We successfully tested our efficient and tiny foundational model RaVAEn and also achieved the world’s first training of a ML model using PyTorch on-board of a satellite. We used few-shot learning with a small annotated dataset of cloudy tiles. While on-board inference often uses frozen model inference and libraries such as ONNX or TensorRT, for training we interestingly needed PyTorch to keep the model weight changing during training. Since then I focused on designing ML models for imaging spectroscopy (hyperspectral) data. This data is a key enabler for a number of detection tasks, such as the detections of faint traces of methane in the atmosphere, or the identification of minerals present at the surface of the Earth. NASA’s recent mission EMIT uniquely provides global coverage. I am currently researching the deployment of efficient ML models to enable instrument autonomy.