Spiking Neural Network Actor-Critic Proximal Policy Optimization Control for Autonomous UAV Navigation Through Constrained Openings in Civil Infrastructure and Buildings

Summary

This paper proposes a spiking neural network (SNN) actor-critic framework trained with proximal policy optimization (PPO) to navigate a simulated UAV through three sequential rectangular windows in a constrained 3D corridor, motivated by autonomous inspection of confined civil infrastructure. The authors implement leaky integrate-and-fire neurons with surrogate gradients in snnTorch, a Gaussian-policy actor and value critic (6→256→64 layers), Monte Carlo returns combined with PPO clipping, and a custom UAVWindowEnv with reward shaping plus curriculum learning that shrinks windows from 60×60 to 20×20 over 800 episodes. Training ran 3000 episodes with a single fixed seed: 1913 full successes (63.77% overall), mean reward 304.83, 2.10 windows passed per episode, and block-wise success rates rising from 45–55% to 90–95% in the final block. Main limitations: no ANN or non-spiking baseline despite energy efficiency being the core motivation; the spike and energy metrics defined in §3.8 are never reported; all results are single-seed training-time aggregates with no held-out evaluation or statistical testing; and dynamics are simplified Newtonian kinematics without perception or noise. The work is a reasonable proof of concept for continuous spiking control, but its conclusions remain preliminary.