This is a framework for generating incremental 3D scene graphs from data generated using the VirtualHome simulator.
This is based on SceneGraphFusion c++ framework. For the network part of the code, check here.
apt update
apt install git
# cmake
sudo apt install python3-pip
pip3 install cmake
# create ssh key and associate to your account in order to clone this project.
# * Generate new ssh key [link](https://docs.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
# * Adding a new ssh key to your account [link](https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account)
# Eigen3 3.3.90
# OpenCV 4.0.0
# This project will try to install them locally if they were not found.
loading meshes for online rendered view generation.
apt install libassimp-dev
With GUI:
# OpenGL & Glfw3
apt install freeglut3-dev libglfw3-dev
With graph prediction:
# Onnxruntime
# See https://github.com/microsoft/onnxruntime for more information
git clone --recursive -b v1.8.2 https://github.com/microsoft/onnxruntime
cd onnxruntime
# For Linux
./build.sh --config RelWithDebInfo --build_shared_lib --parallel
cd build/Linux/RelWithDebInfo
sudo make install
build the basic segmentation system
git clone {this_repo}
cd SceneGraphFusion
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make
build with GUI or graph prediction, pass these options in cmake:
cmake -DBUILD_GRAPHPRED=ON -DBUILD_GUI=ON ..
./exe_GraphSLAM --pth_in path/to/3RScan/squence/
# or with GUI
./exe_GraphSLAM_GUI --pth_in path/to/3RScan/squence/
# to see usage and options
./exe_GraphSLAM --help
# to run with graph prediction (need to build with graph predict)
./exe_GraphSLAM_GUI --pth_in path/to/3RScan/squence/ --pth_model /path/to/the/traced/model/folder/
you can download the traced model of our pretrained network here. The model is trained with 20 NYUv2 object classes used in ScanNet benchmark, and with 8 support types of predicates.
In some platforms the CMake config for Assimp is called assimpConfig.cmake
and
in others AssimpConfig.cmake
. This mismatch can lead to CMake not finding the
library despite being available in the system. If this is the case, edit
cmake/UseAssimp.cmake
to look like this:
find_package(assimp QUIET)
OPTION(WITH_ASSIMP "Build with Assimp support?" ${assimp_FOUND})
IF(WITH_ASSIMP)
MESSAGE(STATUS "WITH Assimp")
find_package(assimp REQUIRED)
INCLUDE_DIRECTORIES(${assimp_INCLUDE_DIRS})
ENDIF()
See assimp/assimp#3455, microsoft/vcpkg#14256 for more details