Home | Research/Projects | Mentorship| Geopolitical Strategy
Directory
Spacecraft Position Estimation with Terrain Relative Kalman Filtering [View Manuscript]
An accurate, vision-based, Kalman Filtering framework for spacecraft position estimation in GNSS-denied orbit scenarios.
 3D error.png)
Figure 1. 3D Orbit trajectories for the EKF and UKF compared to true state.
Problem: Accurate knowledge of a spacecraft’s position is critical for mission success; however, there are a number of circumstances that may cause GNSS (Global Navigation Satellite System) services to be unavailable, potentially for extended periods of time. In these instances, the ability to leverage vision-based estimation can be critical to the successful execution of the spacecraft’s mission.
Approach: Employ Extended and Unscented Kalman Filters (EKF and UKF) with novel terrain-relative residuals to achieve sub-kilometer position estimates for a simulated spacecraft in LEO.
Results: We achieve a 90% reduction in initial position error over the duration of the 7-second flyover of Lake Seneca. We achieve sub-kilometer estimates with both the EKF and UKF, but we ultimately discover the UKF is most effective with a lowest error of 424 m from the true position.
 x error.png)
 y error.png)
 z error.png)
Figure 2. X, Y, and Z errors for each filter compared.
Technical details:
- We simulate the true trajectory with two-body motion equations + the J2 perturbation and atmospheric drag (nominal trajectory intentionally excludes perturbations)
- Leverage OpenCV and pyproj to create a physically consistent orbital flyover and generate measurement residuals
- Use MATLAB’s native Python interface to run the filtering loop in MATLAB while leveraging key Python libraries to develop residuals from the data.
Demo: See the project repository’s README for setup instructions.
A Deep Semantic Segmentation Network for Mars Rover Traversibility
*T-1st for Best Machine Learning Project in Georgia Tech’s graduate Machine Learning course (Spring 2026)
Semantic Segmentation model for Martian terrain type, trained solely on unlabeled Earth imagery.
Figure 3. Final Segmentation output compared with ground truth labels
Problem: Current Martian rovers rely on classical (geometry-based) computer vision, lacking semantic understanding of terrain type. This can cause critical mission failures, like the 2009 Spirit Rover embedding. Furthermore, when preparing to deploy a rover on unexplored terrain, there may not be labeled data from that environment to train on. The ability to use an accessible Earth environment to train a model to perform on novel terrain is both exciting and challenging.
Approach: We use weakly supervised learning to generate pseudo-labels for the unlabeled Mars analog dataset, BASEPROD (located in the Bardenas Semi-Desert, Spain). These pseudo-labels were then used to train a DeepLabV3 model for semantic segmentation of four specific terrain types on Mars. We then validate these results by testing against the AI4Mars dataset.
Figure 4. Overall Training and Deployment Pipeline
Results:
Figure 5. Mean Intersection over Union for pseudo labels compared with hand annotations
Figure 6. Pseudo Labels Compared with raw BaseProd images
Figure 7. Final Confusion Matrix after deploying on Mars (i.e., testing against AI4Mars)
Technical details:
- Trained a custom multi-layer perceptron (MLP) for DINO ViT-S/14 to learn pseudo masks using just 134 hand-drawn labels.
- Fine-tuned DeepLabV3 (ResNet-50 backbone) with auxiliary loss on the 31,707 images and corresponding pseudo-labels
- Weighted the classes during training to remove bias against less common terrain types
Acknowledgments:
This work was done in collaboration with Trisha Singh, Noah Fischer, Steven Baker, and Sameer Panghal. Sincere thanks to them for their hard work on this project.
Links:
- Project GitHub report for code and more details.
Interactive Orbital Mechanics Simulator
An interactive graphical orbital mechanics simulator based on 2-body Newtonian mechanics. Allows the user to simulate a constellation of spacecraft orbiting Earth and intuitively understand the scale and dynamics.
Key features
- Command line arguments to change the following simulation parameters:
- The number of spacecraft in the simulated constellation
- The mass of the spacecraft in the simulation
- Spacing between spacecraft
- Initial orbit radius and velocity of each spacecraft
- Simulation time scale
- Accompanying MATLAB script to plot spacecraft trajectories
- Simulation log to confirm the simulation parameters
- “WASD + Up/Down” controls for changing the camera view
Figure 8. 3D plot of 2 spacecraft in the simulated constellation
Technical Details
- Leveraged C++ OpenGL for graphics rendering
- Employed custom classes to facilitate the addition of an arbitrary number of spacecraft
- Implemented the Verlet Integrator method to propagate the orbital mechanics
Links
2D formation controller for fish-like bio-robots
Two degree of freedom PD controller for formation control of schooling hydrofoils. Provides closed-loop control to command a pitching hydrofoil to navigate to and hold formation at a target position in the wake of another hydrofoil.
Video 1: Simulation of a closed-loop PD hydrofoil tracking
Figure 9. Trailing hydrofoil position (in chord lengths) over time
Results: Enabled the study of the hydrodynamic mechanisms that govern fish schooling in various formations of choice. This tool will enable researchers in the Lehigh Unsteady Flow Interactions Laboratory to:
- Understand the most energetically efficient relative positions for fish to school in.
- Learn whether certain formations are physically possible to hold with realistic constraints
- Analyze the feedback control signals and find patterns that may allow for open-loop (feedback-less) control.
Other Projects
Portfolio of past projects here