FMB: A Functional Manipulation Benchmark for Generalizable Robotic Learning

Jianlan Luo^*, Charles Xu^*, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, and Sergey Levine

^* Equal Contribution

Our benchmark for studying robotic learning for functional manipulation consists of a variety of easily reproducible 3D printed objects, each one requiring a sequence of grasping, reorientation, and assembly behaviors. Generalization can be evaluated on test objects and varied positions, as well as more complex multi-stage assembly tasks. We also provide an imitation learning system that provides a basic set of policies for each skill, allowing researchers to use our tasks as a toolkit for studying any portion of the pipeline.

Dataset Overview

Our dataset consists of objects in diverse appearance and geometry. It requires multi-stage and multi-modal fine motor skills to successfully assemble the pegs onto a unfixed board in a randomized scene. We collected a total of 22,550 trajectories across two different tasks on a Franka Panda arm. We record the trajectories from 2 global views and 2 wrist views. Each view contains both RGB and depth map.

Visit the dataset page for more detail and links to download the dataset.

Single-Object Multi-Stage Manipulation Task

The Single-Object Multi-Stage Manipulation Task consists of 54 different assembly objects and 3 assembly boards of various shapes, sizes, and colors. The robot has to grasp the object, perform a series of reorientation actions with an environment fixture, then insert into the board. The time horizon for this task ranges from 20 to 40 seconds. To make the task managable, we break down each end-to-end trajectory into seperate primitives such as grasp, place on fixture, regrasp, rotate, move to board, and insert. Examples of each trajectories can be visualized below.

View a Random Trajectory

Sample

Side 1 View

Side 2 View

Wrist 1 View

Wrist 2 View

Multi-Object Multi-Stage Manipulation Task

The Multi-Object Multi-Stage Manipulation Task consists of 3 sets of assembly object. Each set contains 4 interlocking objects that need to be assembled together sequentially using the same primitives as the previous task. The time horizon for this task is even longer, and can easily exceed 100 seconds to fully assemble one board. Examples of each trajectories can be visualized below.