NLOS Tracking

Official codes of CVPR 2023 Paper | Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking

Prepreation

Environment

Create a new environment and install dependencies with requirement.txt:

conda create -n NLOS_Tracking

conda activate NLOS_Tracking

conda install --file requirements.txt

Data

The NLOS-Track dataset can be downloaded from kaggle.

The file structure in project root should be as follow:

project_root
|   README.md
|   requirements.txt
|   train.py
+---data
+---utils
+---configs
|   ...
+---dataset
    +---render
    |   +---0000
    |   |      configs.yaml
    |   |      route.mat
    |   |      video_128.npy
    |   |      001.png
    |   |      002.png
    |   |      ...
    |   +---0001
    |       ...
    +---real-shot
        +---0000
        |      route.mat
        |      video_128.npy
        +---0001
            ...

Data Loading and Visualization

Follow the code blocks in data_playground.ipynb to load and visualize the dataset.

Usage

Train

Before training, fill the missing items in configuration files.

Create a new configuration file in ./configs for training:

python train.py --cfg_file=new_cfg --model_name=PAC_Net

or directly use default.yaml by default:

python train.py --model_name=PAC_Net --pretrained -b 64 -lr_b 2.5e-4 --gpu_ids=0,1 --port=8888

Test

Follow the code blocks in test.ipynb to test a trained model.

Citation

@article{wang2023nlosTrack,
  author   = {Wang, Yihao and Wang, Zhigang and Zhao, Bin and Wang, Dong and Chen, Mulin and Li, Xuelong},
  title    = {Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking},
  journal  = {CVPR},
  year     = {2023},
}