High Accuracy Localization Technology and Integrated Localization Device

Aiming at design and manufacture of commercial big plane independently, development of the CAR-CA(collaborative augmented reality for commercial aircraft design & maintenance) system for design of the cockpit and maintenance of the key components is put forward.

The main job includes developing an localization device serving two parts of the system:
1. Interactive augmented reality system for cockpit design & pilot training.
2. Interactive augmented reality system for training of technicians to detach and assembling of the key components.This device is used to get high accuracy localization result of the HMD on the operators head, making sure the augmented reality of the virtual dashboard and virtual engine parts runs correctly.

We developed and improved the VisTracker-barcode system according to the paper "Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker" of L Naimark and E Foxlin.

Our Device:


Patterns & Environment:



Augmented Reality of Plane Dash Board:


Multi-State EKF with IMU-Camera Capture Synchronization

To address the problem of data synchrony between an IMU and a camera, an algorithm based on multi-state EKF is proposed, which can largely alleviate the negative in°uence of synchronous data on the system's localization accuracy.

Synchronization Problem:


Multi-State EKF with Synchronization Flowchart: