SceneNN: A Scene Meshes Dataset with aNNotations

Binh-Son Hua¹, Quang-Hieu Pham², Duc Thanh Nguyen³, Minh-Khoi Tran², Lap-Fai Yu⁴, and Sai-Kit Yeung⁵

¹The University of Tokyo ²Singapore University of Technology and Design ³Deakin University
⁴George Mason University ⁵The Hong Kong University of Science and Technology

We introduce an RGB-D scene dataset consisting of more than 100 indoor scenes. Our scenes are captured at various places, e.g., offices, dormitory, classrooms, pantry, etc., from University of Massachusetts Boston and Singapore University of Technology and Design.
All scenes are reconstructed into triangle meshes and have per-vertex and per-pixel annotation. We further enriched the dataset with fine-grained information such as axis-aligned bounding boxes, oriented bounding boxes, and object poses.

News

Aug 11, 2022: We have updated the links to the homepage and the data.
Oct 28, 2019: Our paper about a new object dataset named ScanObjectNN (derived from SceneNN and ScanNet) for real-world object classification is accepted to ICCV 2019 as an oral! See you in Seoul!
Feb 25, 2019: Our paper JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds got accepted to CVPR 2019 as an oral!
Jan 08, 2019: We presented the paper Real-time Progressive 3D Semantic Segmentation for Indoor Scenes at WACV 2019 in Hawaii!
Oct 02, 2018: We had fun running the tutorial at IROS 2018: Creating and Understanding 3D Annotated Scene Meshes!
June 07, 2018: Code and data of our paper Pointwise Convolutional Neural Networks (CVPR 2018) are released!
May 21, 2018: We are at ICRA 2018 for the tutorial: Creating Annotated Scene Meshes for Training and Testing Robot Systems.
Mar 18, 2018: MagixHome: our synthetic 3D scene modeling platform is available now!
Jan 22, 2018: We organize the RGB-D-to-CAD Object Retrieval contest at Eurographics 2018 Workshop on 3D Object Retrieval.
Oct 13, 2017: Experimental WebGL annotation tool is here.
Aug 04, 2017: The SceneNN annotation tool paper is accepted to TVCG and presented at Pacific Graphics 2017. See here.
Jan 16, 2017: We organize a new SHREC 2017 track named RGB-D Object-to-CAD Retrieval.
Oct 28, 2016: SceneNN received Best Paper Honorable Mention at 3DV 2016!

Dataset & Tools

CVPR 2018 (evaluating semantic segmentation with NYU-D v2 40 classes)

HKUST server
(Training data, 8.7 GB)

Google Drive
(Mirror link, 8.7 GB)

Google Drive
(Raw PLY files, 4.0 GB)

3DV 2016 (original instance annotation of 100+ scenes)

HKUST server

Annotation tool
(Windows x64)

RGBD reconstruction

Publication

JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields

Quang-Hieu Pham, Duc Thanh Nguyen, Binh-Son Hua, Gemma Roig, and Sai-Kit Yeung

Computer Vision and Pattern Recognition CVPR 2019 (Oral)
Project Code
Real-time Progressive 3D Semantic Segmentation for Indoor Scenes

Quang-Hieu Pham, Binh-Son Hua, Duc Thanh Nguyen, and Sai-Kit Yeung

WACV 2019
Project

Pointwise Convolutional Neural Networks

Binh-Son Hua, Minh-Khoi Tran, and Sai-Kit Yeung

Computer Vision and Pattern Recognition (CVPR) 2018

Project Paper Code Bibtex

@inproceedings{hua-pointwise-cvpr18,
    title = {Pointwise Convolutional Neural Networks},
    author = {Binh-Son Hua and Minh-Khoi Tran and Sai-Kit Yeung},
    booktitle = {Computer Vision and Pattern Recognition (CVPR)},
    year = {2018}
}

SceneNN: A Scene Meshes Dataset with aNNotations

Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, and Sai-Kit Yeung

International Conference on 3D Vision (3DV) 2016. Best Paper Honorable Mention.

Paper Supplemental Slides Poster Bibtex

@inproceedings{scenenn-3dv16,
    author = {Binh-Son Hua and Quang-Hieu Pham and Duc Thanh Nguyen and Minh-Khoi Tran and Lap-Fai Yu and Sai-Kit Yeung},
    title = {SceneNN: A Scene Meshes Dataset with aNNotations},
    booktitle = {International Conference on 3D Vision (3DV)},
    year = {2016}
}

A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation

Duc Thanh Nguyen, Binh-Son Hua, Lap-Fai Yu, and Sai-Kit Yeung

IEEE Transactions on Visualization and Computer Graphics (TVCG) 2017

(presented at Pacific Graphics 2017)

Paper Video Bibtex

@article{anno-tvcg17,
    author = {Thanh Nguyen, Duc and Hua, Binh-Son and Yu, Lap-Fai and Yeung, Sai-Kit},
    title = {A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation},
    journal = {IEEE Transactions on Visualization and Computer Graphics (TVCG)},
    year = {2017}
}

Technical Report

SHREC'18: RGB-D Object-to-CAD Retrieval

Organizers: Quang-Hieu Pham, Binh-Son Hua, Lap-Fai Yu, Duc Thanh Nguyen, and Sai-Kit Yeung

Eurographics Workshop on 3D Object Retrieval 2018
Paper Homepage

SHREC'17: RGB-D to CAD Retrieval with ObjectNN Dataset

Organizers: Binh-Son Hua, Quang-Trung Truong, Minh-Khoi Tran, Quang-Hieu Pham, Lap-Fai Yu, Duc Thanh Nguyen, and Sai-Kit Yeung

Eurographics Workshop on 3D Object Retrieval 2017

Paper Slides Homepage Bibtex

@misc{objectnn-shrec17,
    author = {Binh-Son Hua and Quang-Trung Truong and Minh-Khoi Tran and Quang-Hieu Pham and
              Asako Kanezaki and Tang Lee and HungYueh Chiang and Winston Hsu and
              Bo Li and Yijuan Lu and Henry Johan and Shoki Tashiro and Masaki Aono and
              Minh-Triet Tran and Viet-Khoi Pham and Hai-Dang Nguyen and Vinh-Tiep Nguyen and
              Quang-Thang Tran and Thuyen V. Phan and Bao Truong and Minh N. Do and Anh-Duc Duong and
              Lap-Fai Yu and Duc Thanh Nguyen and Sai-Kit Yeung},
    title = {SHREC'17: RGB-D to CAD Retrieval with ObjectNN Dataset},
    booktitle = {Eurographics Workshop on 3D Object Retrieval},
    year = {2017}
}

Discussion

Please email us at scenenn [at] gmail.com for any inquiries. You can also post to the discussion board below.

Acknowledgements

We are grateful to the anonymous reviewers for their constructive comments. We thank Fangyu Lin for his assistance with the data capture and development of the WebGL viewer, and Guoxuan Zhang for his help with the early version of the annotation tool.

Lap-Fai Yu is supported by the University of Massachusetts Boston StartUp Grant P20150000029280 and by the Joseph P. Healey Research Grant Program provided by the Office of the Vice Provost for Research and Strategic Initiatives & Dean of Graduate Studies of the University of Massachusetts Boston. This research is supported by the National Science Foundation under award number 1565978. We also acknowledge NVIDIA Corporation for graphics card donation.

Sai-Kit Yeung is supported by Singapore MOE Academic Research Fund MOE2013-T2-1-159 and SUTD-MIT International Design Center Grant IDG31300106. We acknowledge the support of the SUTD Digital Manufacturing and Design (DManD) Centre which is supported by the National Research Foundation (NRF) of Singapore. This research is also supported by the National Research Foundation, Prime Minister's Office, Singapore under its IDM Futures Funding Initiative.