Spatial Intelligence through combined sensing, using computer vision and on board electromagnetics modelling to enhance signals intelligence and situational awareness

As the cost of antenna arrays continues to decrease, there is an emerging opportunity for the combination of flexible electromagnetics models, modular and distributed antenna array design, and computer vision to provide enhancements to the situational awareness of platform operators. Combining computer vision, efficient electromagnetics modelling and radio channel measurements can provide a consistent understanding of the local environment for improved signals intelligence and situational awareness using commercial off the shelf (COTS) mobile AI modules. In this paper, an open source electromagnetics model is presented for simulation of distributed antenna arrays on complex platforms (LyceanEM), and the use of computer vision based spatial mapping (DepthAI) to populate a channel model for improved direction finding using a 4x4 MIMO spatial intelligence node powered by an Nvidia Orin AGX. A combination of simulated and measured signals will be used to demonstrate the initial workflow for onboard environment mapping, characterisation, and direction finding within the node. The influence of the channel model on signals localisation accuracy will be examined and the fusion of radio and vision based spatial characterisation.