FMB: A Functional Manipulation Benchmark for Generalizable Robotic Learning

Jianlan Luo*, Charles Xu*, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, and Sergey Levine
* Equal Contribution

The International Journal of Robotics Research (IJRR) 2024

Paper Video Code Usage Dataset Material Eval Procedure
Overview
Our benchmark for studying robotic learning for functional manipulation consists of a variety of easily reproducible 3D printed objects, each one requiring a sequence of grasping, reorientation, and assembly behaviors. Generalization can be evaluated on test objects and varied positions, as well as more complex multi-stage assembly tasks. We also provide an imitation learning system that provides a basic set of policies for each skill, allowing researchers to use our tasks as a toolkit for studying any portion of the pipeline.

Dataset Overview

Overview
Our dataset consists of objects in diverse appearance and geometry. It requires multi-stage and multi-modal fine motor skills to successfully assemble the pegs onto a unfixed board in a randomized scene. We collected a total of 22,550 trajectories across two different tasks on a Franka Panda arm. We record the trajectories from 2 global views and 2 wrist views. Each view contains both RGB and depth map.

Visit the dataset page for more detail and links to download the dataset.

Single-Object Multi-Stage Manipulation Task

Overview
The Single-Object Multi-Stage Manipulation Task consists of 54 different assembly objects and 3 assembly boards of various shapes, sizes, and colors. The robot has to grasp the object, perform a series of reorientation actions with an environment fixture, then insert into the board. The time horizon for this task ranges from 20 to 40 seconds. To make the task managable, we break down each end-to-end trajectory into seperate primitives such as grasp, place on fixture, regrasp, rotate, move to board, and insert. Examples of each trajectories can be visualized below.

View a Random Trajectory

Sample
Side 1 View
Side 2 View
Wrist 1 View
Wrist 2 View
loading...

Multi-Object Multi-Stage Manipulation Task

Overview
The Multi-Object Multi-Stage Manipulation Task consists of 3 sets of assembly object. Each set contains 4 interlocking objects that need to be assembled together sequentially using the same primitives as the previous task. The time horizon for this task is even longer, and can easily exceed 100 seconds to fully assemble one board. Examples of each trajectories can be visualized below.

View a Random Trajectory

Sample
Side 1 View
Side 2 View
Wrist 1 View
Wrist 2 View
loading...

Reproducing the Benchmark

All the objects in our benchmark are designed to be 3D printable. We provide the CAD files for all the objects and links to material needed in the Material and CAD Files section.

A detailed description on how to setup the workspace can be found in the Workspace Setup section.

We also provide a detailed description of the evaluation procedure in the Evaluation Procedure section.

Others Using FMB

We are excited to see the community is using FMB for their research. If you are using FMB in your work, please let us know by emailing functionalmanipulationdataset@gmail.com and we would love to feature your work here.

Sun, Jiankai, et al. "ARCH: Hierarchical Hybrid Learning for Long-Horizon Contact-Rich Robotic Assembly."