Submission: ZeroPose/HB

Download submission
Submission name
Submission time (UTC) Aug. 23, 2023, 2:24 a.m.
User jianqiujianqiu
Task Model-based 2D segmentation of unseen objects
Dataset HB
Description
Evaluation scores
AP:0.453
AP50:0.594
AP75:0.500
AP_large:0.550
AP_medium:0.466
AP_small:0.000
AR1:0.531
AR10:0.570
AR100:0.571
AR_large:0.709
AR_medium:0.579
AR_small:0.009
average_time_per_image:3.628

Method: ZeroPose

User jianqiujianqiu
Publication https://arxiv.org/abs/2305.17934
Implementation
Training image modalities None
Test image modalities RGB
Description

Submitted to: BOP Challenge 2023

Training data: GSO dataset rendered by Megapose

Onboarding data: No need for onboarding

Used 3D models: Default, CAD

Notes: These results are from the ZeroPose method without multi-hypothesis refinement. Abstract: We present a CAD model-based zero-shot pose estimation pipeline called ZeroPose. Existing pose estimation methods remain to require expensive training when applied to an unseen object, which greatly hinders their scalability in the practical application of industry. In contrast, the proposed method enables the accurate estimation of pose parameters for previously unseen objects without the need for training. Specifically, we design a two-step pipeline consisting of CAD model-based zero-shot instance segmentation and a zero-shot pose estimator. For the first step, there is a simple but effective way to leverage CAD models and visual foundation models SAM and Imagebind to segment the interest unseen object at the instance level. For the second step, we based on the intensive geometric information in the CAD model of the rigid object to propose a lightweight hierarchical geometric structure matching mechanism achieving zero-shot pose estimation. Extensive experimental results on the seven core datasets on the BOP challenge show that the proposed zero-shot instance segmentation methods achieve comparable performance with supervised MaskRCNN and the zero-shot pose estimation results outperform the SOTA pose estimators with better efficiency.

If you have any questions, feel free to contact us at jianqiuer@gmail.com.

Computer specifications RTX 3090