MARS-MINDS — Martian Intelligent Navigation and Decision System

Context

Mars mission planning has historically been a deeply manual exercise — surface classification, terrain reasoning, dust-storm forecasting, and safe-landing analysis each conducted as independent workflows by separate teams against independent data pipelines. As the cadence of planetary missions accelerates and the data volume from orbital and surface instruments compounds, the manual approach cannot keep up. The opportunity was to build a unified planetary AI platform that produces a single coherent decision surface from the raw HiRISE and Curiosity imagery to a benchmark that exceeds the published baselines on each task.

Challenge

The state of the art at the time of the work — the 2022 Mars-TRP baseline — reported 88 per cent accuracy on the 25-class Curiosity surface classification task. The published architectures were end-to-end single networks, dust-storm detection sat below 90 per cent on small label sets, habitat-suitability analysis was largely unpublished, and safe-landing analysis was generally addressed as a downstream rule-based layer over the surface model. There was no public benchmark for a unified system. The architectural question was whether a single end-to-end model could be pushed past 95 per cent on the 25-class task, or whether the path forward was to decompose the problem.

Approach

The architecture rejected the end-to-end single-network hypothesis early. We decomposed the problem into four task heads under a single reinforcement-learning controller. Mission Planning uses a Vision Transformer backbone with task-specific heads for surface, terrain, and rover-pose reasoning. Dust Storm Insight uses a ConvLSTM head over sequential HiRISE imagery. Habitat Building uses an EfficientNetV2 plus ConvNeXt-V2 hybrid head with a Mask R-CNN segmentation layer. Safe Landing uses a multi-modal fusion head over visible-light, thermal, and elevation channels.

The controller is trained with PPO over a reward shaped against ground-truth mission-outcome trajectories with safety penalties on hazardous decisions. The decomposition allowed each head to be improved independently against a head-specific evaluation harness — a property the end-to-end network does not have. The combined system is deployed on a Tesla T4 GPU inference target with a unified throughput of more than five thousand inferences per minute and sub-200 millisecond per-model inference latency.

Delivery Ownership

This engagement followed a principal-led delivery model. I owned the strategy, architecture, data-pipeline construction, model training, evaluation harness, and the reinforcement-learning controller end to end. The data pipeline ingested 21 NASA and ESA planetary datasets totalling more than 600 GB through the NASA Planetary Data System and JPL HiRISE pipelines. The work was conducted as an MSc thesis at NED University of Engineering and Technology, under the Department of Computer Information System Engineering, with academic supervision but full operational ownership at the principal level.

Outcomes

Authorship status

The MSc thesis is complete. Five manuscripts derived from the work are in active submission or preparation: a Q1 manuscript on Mission Planning under Vision Transformer architectures, a Q1 manuscript on dust-storm forecasting from HiRISE imagery, a Q1 manuscript on habitat suitability and dust storm exposure assessment (AstroVisionNet), an IEEE W-category manuscript on autonomous safe landing using a DenseNet framework (MarsLanderNet), and an IEEE W-category manuscript on InceptionV3 surface feature classification. A sixth manuscript on the multi-agent reinforcement-learning controller is in preparation.

Technology Stack

PyTorch 2.x, JAX for the controller training loop, HuggingFace Transformers for the vision backbones, Vision Transformers, EfficientNetV2, ConvNeXt-V2, Swin-V2, YOLO v11, Mask R-CNN, ResNet-50-FPN, Grad-CAM for explainability, OpenCV for preprocessing, NASA HiRISE data pipelines, Tesla T4 GPU inference target, Triton Inference Server, FastAPI for the inference API surface.

Services Delivered

Planetary AI & Remote Sensing / Agentic AI Architecture & Build / LLM Fine-Tuning & Alignment (controller PPO) / LLMOps & Production AI Platform.

Why it matters

The Mission Planning module is the first publicly verifiable system to break the 99 per cent ceiling on the Curiosity 25-class task. The architecture choice of a five-module platform with a reinforcement-learning controller, instead of a single end-to-end network, is the load-bearing decision because it allows independent improvement of each module without retraining the whole. That property is what makes the system maintainable across the lifetime of a mission programme rather than a one-shot benchmark exercise. For a deeper engineering discussion of that architectural decision, see the companion Insight: Why MARS-MINDS uses five models and a controller, not an end-to-end network.