The Scenario

Picture yourself as an engineer at a trailblazing tech company that will revolutionise urban mobility with a combined autonomous taxi and delivery service. Your goal is to provide a seamless, safe, and efficient way for people and goods to get from point A to point B in urban environments. The key to making this possible? An impeccable navigation system.

A key component of this navigation system is robust and safe visual odometry.

In order to decide which visual odometry approach to choose for the autonomous vehicle, you are tasked with implementing a visual odometry pipeline, and testing and analysing its performance.

Assumptions

When solving the task, you can make the following assumptions:

The autonomous vehicle is equipped with a front-facing stereo camera with known calibration parameters, as in the KITTI dataset.
Your Tasks

This assignment consists of three Tasks. We summarise them below, followed by more comprehensive description further down.

Implement a full visual odometry pipeline, using a feature-based approach of your choice.
Experimentally evaluate and discuss the performance of your implemented system on the KITTI dataset (more about KITTI below).
Add an experiment of your choice, for example:
Compare your performance with another algorithm from a publicly available VO system (it is ok to simply download from github or install via pip, you do not have to reimplement this 3rd party method yourself).
Combine your VO with place recognition or localisation to obtain a full visual SLAM system.
Conduct a detailed ablation study of your chosen algorithm, i.e. systematically vary a meaningful selection of parameters and discuss the observed influence on the performance.
If you have other ideas, talk to your lecturer.
Task 1 Description – Implementation

Implement a full visual odometry pipeline, using a feature-based approach of your choice.
In your report, clearly describe the approach you implemented, including the underlying algorithmic and mathematical concepts.
In your report, discuss the differences of your method compared to relevant other approaches. Explain the advantages and disadvantages of your chosen method, compared to the other methods, in the context of an urban autonomous driving scenario.
Clearly state the weaknesses you expect from your chosen method, e.g. the failure cases you expect in certain conditions.
Remarks:

Since you have access to a calibrated stereo camera, you can choose between very different VO approaches, based on different types of correspondences: 2D-2D, 2D-3D, and 3D-3D.
You will not be marked for the choice of method, but for the discussion of advantages and disadvantages.
You can use all existing functions of OpenCV or other libraries and do not have to reimplement everything from scratch. You should however clearly describe what algorithm is used by the functions you call. For example, if you use OpenCVs PnP solver, you should explain the PnP problem in your descriptions (see point 2 above).
Task 2 Description – Evaluation

Experimentally evaluate the performance of your implemented system on the KITTI dataset by comparing against the ground truth pose information.
Choose meaningful test sequences from the KITTI dataset and meaningful evaluation metrics.
In your report, clearly explain your experimental setup and evaluation protocol. This should contain the following points:
Explain which sequences from the KITTI dataset you chose and why.
Discuss the metrics you used and why you chose them.
Discuss the obtained results of your evaluation, including the following:
Clearly present the performance of your algorithm and interpret the results.
Discuss failure cases, and the relative strengths and weaknesses of your algorithm.
Discuss if your experimental findings were aligned with the strengths and weaknesses you expected and discussed as part of Task 1.