split string | arm string | chain_idx int64 | seed int64 | extra_attempt_budget int64 | instructions list | instructions_completed int64 | full_chain_success bool | budget_left int64 | env_steps int64 | n_layer_interventions int64 | events_json string |
|---|---|---|---|---|---|---|---|---|---|---|---|
heldout | flower_alone | 0 | 700,000 | 0 | [
"rotate_blue_block_right",
"move_slider_right",
"lift_red_block_slider",
"place_in_slider",
"turn_off_lightbulb"
] | 5 | true | 0 | 311 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 76}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 75}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 78}, {"sub_... |
heldout | flower_alone | 2 | 700,002 | 0 | [
"lift_pink_block_slider",
"place_in_slider",
"open_drawer",
"rotate_red_block_right",
"lift_red_block_table"
] | 5 | true | 0 | 441 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 67}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 4}, {"sub_idx": 2, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 72}, {"sub_idx": 3, "type... |
heldout | flower_alone | 4 | 700,004 | 0 | [
"open_drawer",
"lift_pink_block_table",
"place_in_slider",
"turn_on_lightbulb",
"rotate_blue_block_left"
] | 5 | true | 0 | 355 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 59}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 112}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 35}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 5 | 700,005 | 0 | [
"push_blue_block_left",
"close_drawer",
"lift_red_block_slider",
"place_in_slider",
"turn_off_lightbulb"
] | 5 | true | 0 | 300 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 68}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 75}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 90}, {"sub_idx": 3,... |
heldout | flower_alone | 9 | 700,009 | 0 | [
"push_red_block_left",
"lift_blue_block_table",
"place_in_slider",
"turn_on_lightbulb",
"move_slider_right"
] | 5 | true | 0 | 287 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_red_block_left", "status": "success", "steps": 60}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 60}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 31}, {"sub_idx": ... |
heldout | flower_alone | 11 | 700,011 | 0 | [
"lift_pink_block_slider",
"place_in_slider",
"turn_on_lightbulb",
"rotate_blue_block_right",
"open_drawer"
] | 5 | true | 0 | 377 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 112}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 4}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 61}, {"sub_idx": 3... |
heldout | flower_alone | 14 | 700,014 | 0 | [
"lift_red_block_slider",
"stack_block",
"move_slider_left",
"turn_off_led",
"open_drawer"
] | 5 | true | 0 | 419 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 71}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 57}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 174}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 15 | 700,015 | 0 | [
"move_slider_left",
"push_pink_block_left",
"open_drawer",
"turn_on_lightbulb",
"lift_blue_block_table"
] | 0 | false | 0 | 360 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 16 | 700,016 | 0 | [
"push_red_block_left",
"lift_pink_block_slider",
"stack_block",
"turn_off_led",
"open_drawer"
] | 2 | false | 0 | 466 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_red_block_left", "status": "success", "steps": 53}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 53}, {"sub_idx": 2, "type": "attempt", "subtask": "stack_block", "status": "fail", "steps": 360}] |
heldout | flower_alone | 19 | 700,019 | 0 | [
"move_slider_left",
"lift_red_block_table",
"place_in_slider",
"turn_on_lightbulb",
"close_drawer"
] | 2 | false | 0 | 495 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 46}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 89}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "fail", "steps": 360}] |
heldout | flower_alone | 20 | 700,020 | 0 | [
"open_drawer",
"turn_on_lightbulb",
"push_red_block_left",
"move_slider_left",
"push_into_drawer"
] | 5 | true | 0 | 368 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 58}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 71}, {"sub_idx": 2, "type": "attempt", "subtask": "push_red_block_left", "status": "success", "steps": 68}, {"sub_idx": 3, "type... |
heldout | flower_alone | 21 | 700,021 | 0 | [
"move_slider_right",
"turn_off_lightbulb",
"lift_pink_block_table",
"place_in_drawer",
"lift_pink_block_drawer"
] | 5 | true | 0 | 247 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 70}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 67}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 69}, {"sub_idx":... |
heldout | flower_alone | 22 | 700,022 | 0 | [
"rotate_pink_block_right",
"open_drawer",
"lift_blue_block_table",
"place_in_slider",
"turn_off_lightbulb"
] | 3 | false | 0 | 678 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 177}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 70}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 71}, {"sub_idx":... |
heldout | flower_alone | 23 | 700,023 | 0 | [
"turn_on_led",
"move_slider_left",
"lift_blue_block_table",
"stack_block",
"unstack_block"
] | 5 | true | 0 | 235 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 40}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 64}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 55}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 24 | 700,024 | 0 | [
"rotate_blue_block_right",
"open_drawer",
"turn_off_led",
"lift_blue_block_table",
"place_in_slider"
] | 5 | true | 0 | 372 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 94}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 83}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 63}, {"sub_idx": 3, "type"... |
heldout | flower_alone | 25 | 700,025 | 0 | [
"lift_blue_block_slider",
"stack_block",
"turn_off_led",
"unstack_block",
"push_blue_block_left"
] | 4 | false | 0 | 630 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 85}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 69}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 49}, {"sub_idx": 3, "type":... |
heldout | flower_alone | 26 | 700,026 | 0 | [
"push_blue_block_right",
"open_drawer",
"lift_pink_block_slider",
"place_in_slider",
"push_into_drawer"
] | 5 | true | 0 | 419 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 111}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 72}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 96}, {"sub_idx": ... |
heldout | flower_alone | 27 | 700,027 | 0 | [
"push_red_block_right",
"turn_on_lightbulb",
"push_into_drawer",
"move_slider_left",
"close_drawer"
] | 5 | true | 0 | 478 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_red_block_right", "status": "success", "steps": 152}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 51}, {"sub_idx": 2, "type": "attempt", "subtask": "push_into_drawer", "status": "success", "steps": 88}, {"sub_idx": 3... |
heldout | flower_alone | 29 | 700,029 | 0 | [
"lift_red_block_table",
"stack_block",
"unstack_block",
"open_drawer",
"turn_on_led"
] | 5 | true | 0 | 284 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 50}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 22}, {"sub_idx": 2, "type": "attempt", "subtask": "unstack_block", "status": "success", "steps": 78}, {"sub_idx": 3, "type": ... |
heldout | flower_alone | 33 | 700,033 | 0 | [
"turn_off_led",
"open_drawer",
"push_pink_block_left",
"lift_blue_block_table",
"place_in_slider"
] | 4 | false | 0 | 627 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 39}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 72}, {"sub_idx": 2, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 90}, {"sub_idx": 3, "type": "... |
heldout | flower_alone | 34 | 700,034 | 0 | [
"move_slider_right",
"push_blue_block_left",
"open_drawer",
"lift_red_block_table",
"stack_block"
] | 5 | true | 0 | 489 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 62}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 65}, {"sub_idx": 2, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 63}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 38 | 700,038 | 0 | [
"open_drawer",
"move_slider_right",
"push_blue_block_right",
"lift_blue_block_table",
"place_in_slider"
] | 5 | true | 0 | 497 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 54}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 125}, {"sub_idx": 2, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 140}, {"sub_idx": 3, "... |
heldout | flower_alone | 43 | 700,043 | 0 | [
"rotate_pink_block_right",
"lift_pink_block_table",
"place_in_slider",
"open_drawer",
"lift_blue_block_slider"
] | 5 | true | 0 | 453 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 134}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 95}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 40}, {"sub_i... |
heldout | flower_alone | 45 | 700,045 | 0 | [
"rotate_pink_block_right",
"turn_off_led",
"lift_pink_block_table",
"place_in_slider",
"open_drawer"
] | 3 | false | 0 | 764 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 253}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 64}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 87}, {"sub_idx"... |
heldout | flower_alone | 46 | 700,046 | 0 | [
"turn_off_lightbulb",
"push_pink_block_left",
"move_slider_left",
"open_drawer",
"lift_pink_block_table"
] | 5 | true | 0 | 386 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 57}, {"sub_idx": 1, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 111}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 55}, {"sub_idx": ... |
heldout | flower_alone | 49 | 700,049 | 0 | [
"lift_red_block_table",
"place_in_drawer",
"rotate_blue_block_right",
"lift_pink_block_slider",
"stack_block"
] | 5 | true | 0 | 297 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 50}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 34}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 86}, {"sub_idx... |
heldout | flower_alone | 50 | 700,050 | 0 | [
"open_drawer",
"push_blue_block_right",
"move_slider_left",
"push_into_drawer",
"lift_blue_block_drawer"
] | 5 | true | 0 | 384 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 53}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 106}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 77}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 51 | 700,051 | 0 | [
"rotate_red_block_right",
"lift_pink_block_slider",
"stack_block",
"turn_on_led",
"open_drawer"
] | 5 | true | 0 | 313 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_red_block_right", "status": "success", "steps": 74}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 81}, {"sub_idx": 2, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 46}, {"sub_idx": ... |
heldout | flower_alone | 52 | 700,052 | 0 | [
"turn_on_led",
"push_pink_block_left",
"push_into_drawer",
"lift_red_block_slider",
"place_in_slider"
] | 2 | false | 0 | 485 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 40}, {"sub_idx": 1, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 85}, {"sub_idx": 2, "type": "attempt", "subtask": "push_into_drawer", "status": "fail", "steps": 360}] |
heldout | flower_alone | 53 | 700,053 | 0 | [
"turn_on_led",
"open_drawer",
"rotate_pink_block_left",
"lift_blue_block_table",
"place_in_drawer"
] | 5 | true | 0 | 370 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 40}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 67}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_pink_block_left", "status": "success", "steps": 170}, {"sub_idx": 3, "type":... |
heldout | flower_alone | 57 | 700,057 | 0 | [
"push_blue_block_left",
"push_into_drawer",
"lift_blue_block_drawer",
"place_in_slider",
"move_slider_right"
] | 1 | false | 0 | 425 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 65}, {"sub_idx": 1, "type": "attempt", "subtask": "push_into_drawer", "status": "fail", "steps": 360}] |
heldout | flower_alone | 58 | 700,058 | 0 | [
"move_slider_right",
"turn_off_lightbulb",
"close_drawer",
"rotate_blue_block_right",
"lift_blue_block_table"
] | 4 | false | 0 | 697 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 62}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 64}, {"sub_idx": 2, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 95}, {"sub_idx": 3, "type... |
heldout | flower_alone | 59 | 700,059 | 0 | [
"turn_off_lightbulb",
"push_blue_block_left",
"open_drawer",
"move_slider_left",
"lift_blue_block_table"
] | 5 | true | 0 | 334 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 55}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 72}, {"sub_idx": 2, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 63}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 61 | 700,061 | 0 | [
"push_blue_block_right",
"move_slider_right",
"turn_on_lightbulb",
"lift_red_block_table",
"stack_block"
] | 5 | true | 0 | 614 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 185}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 98}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 58}, {"sub_idx":... |
heldout | flower_alone | 64 | 700,064 | 0 | [
"open_drawer",
"push_pink_block_right",
"lift_red_block_table",
"place_in_slider",
"move_slider_right"
] | 5 | true | 0 | 454 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 59}, {"sub_idx": 1, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 233}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 59}, {"sub_idx": 3,... |
heldout | flower_alone | 65 | 700,065 | 0 | [
"open_drawer",
"turn_on_led",
"move_slider_right",
"lift_pink_block_table",
"place_in_drawer"
] | 5 | true | 0 | 263 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 56}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 67}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 57}, {"sub_idx": 3, "type": "atte... |
heldout | flower_alone | 68 | 700,068 | 0 | [
"push_pink_block_left",
"lift_blue_block_table",
"place_in_drawer",
"close_drawer",
"move_slider_right"
] | 5 | true | 0 | 315 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 65}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 62}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 44}, {"sub_idx":... |
heldout | flower_alone | 70 | 700,070 | 0 | [
"close_drawer",
"rotate_blue_block_right",
"turn_off_led",
"move_slider_right",
"lift_pink_block_slider"
] | 5 | true | 0 | 355 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 62}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 106}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 54}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 71 | 700,071 | 0 | [
"open_drawer",
"push_red_block_left",
"turn_off_lightbulb",
"lift_pink_block_slider",
"place_in_slider"
] | 5 | true | 0 | 332 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 57}, {"sub_idx": 1, "type": "attempt", "subtask": "push_red_block_left", "status": "success", "steps": 93}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 66}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 72 | 700,072 | 0 | [
"move_slider_right",
"push_pink_block_left",
"lift_blue_block_slider",
"place_in_drawer",
"turn_off_led"
] | 2 | false | 0 | 502 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 80}, {"sub_idx": 1, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 62}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "fail", "steps": 360}] |
heldout | flower_alone | 74 | 700,074 | 0 | [
"turn_off_lightbulb",
"move_slider_right",
"rotate_blue_block_right",
"lift_blue_block_table",
"stack_block"
] | 5 | true | 0 | 334 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 57}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 80}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 97}, {"sub_idx... |
heldout | flower_alone | 75 | 700,075 | 0 | [
"move_slider_right",
"rotate_pink_block_right",
"lift_red_block_table",
"stack_block",
"open_drawer"
] | 5 | true | 0 | 321 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 63}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 96}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 64}, {"sub_i... |
heldout | flower_alone | 78 | 700,078 | 0 | [
"lift_blue_block_slider",
"place_in_slider",
"close_drawer",
"push_pink_block_right",
"move_slider_left"
] | 5 | true | 0 | 469 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 70}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 11}, {"sub_idx": 2, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 90}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 80 | 700,080 | 0 | [
"push_pink_block_left",
"turn_on_lightbulb",
"lift_pink_block_table",
"place_in_drawer",
"lift_red_block_slider"
] | 5 | true | 0 | 290 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 70}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 47}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 61}, {"sub_idx... |
heldout | flower_alone | 81 | 700,081 | 0 | [
"open_drawer",
"move_slider_right",
"lift_red_block_slider",
"place_in_drawer",
"turn_off_lightbulb"
] | 5 | true | 0 | 354 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 55}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 94}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 76}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 82 | 700,082 | 0 | [
"lift_red_block_slider",
"place_in_drawer",
"turn_on_led",
"close_drawer",
"lift_blue_block_table"
] | 5 | true | 0 | 356 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 73}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 56}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 43}, {"sub_idx": 3, "type... |
heldout | flower_alone | 83 | 700,083 | 0 | [
"push_into_drawer",
"lift_red_block_slider",
"place_in_slider",
"move_slider_right",
"turn_off_led"
] | 5 | true | 0 | 294 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_into_drawer", "status": "success", "steps": 87}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 71}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 12}, {"sub_idx": 3, ... |
heldout | flower_alone | 85 | 700,085 | 0 | [
"lift_pink_block_slider",
"place_in_slider",
"open_drawer",
"turn_on_lightbulb",
"lift_blue_block_table"
] | 5 | true | 0 | 275 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 58}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 11}, {"sub_idx": 2, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 73}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 86 | 700,086 | 0 | [
"move_slider_right",
"turn_on_led",
"lift_pink_block_slider",
"place_in_drawer",
"lift_pink_block_drawer"
] | 5 | true | 0 | 269 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 64}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 51}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 60}, {"sub_idx": 3, "t... |
heldout | flower_alone | 87 | 700,087 | 0 | [
"push_pink_block_right",
"close_drawer",
"lift_red_block_slider",
"stack_block",
"move_slider_right"
] | 5 | true | 0 | 529 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 218}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 79}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 87}, {"sub_idx": ... |
heldout | flower_alone | 88 | 700,088 | 0 | [
"push_blue_block_left",
"close_drawer",
"turn_off_lightbulb",
"lift_blue_block_table",
"place_in_slider"
] | 5 | true | 0 | 327 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 62}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 75}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 77}, {"sub_idx": 3, "t... |
heldout | flower_alone | 91 | 700,091 | 0 | [
"turn_on_led",
"rotate_blue_block_right",
"lift_blue_block_table",
"stack_block",
"lift_red_block_slider"
] | 3 | false | 0 | 567 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 36}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 91}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 80}, {"sub_idx": ... |
heldout | flower_alone | 92 | 700,092 | 0 | [
"open_drawer",
"turn_off_led",
"push_pink_block_right",
"move_slider_right",
"push_into_drawer"
] | 5 | true | 0 | 479 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 58}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 68}, {"sub_idx": 2, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 172}, {"sub_idx": 3, "type":... |
heldout | flower_alone | 94 | 700,094 | 0 | [
"rotate_pink_block_right",
"move_slider_left",
"close_drawer",
"turn_off_led",
"lift_pink_block_table"
] | 1 | false | 0 | 597 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 237}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 95 | 700,095 | 0 | [
"close_drawer",
"move_slider_right",
"turn_on_led",
"rotate_red_block_left",
"lift_red_block_table"
] | 5 | true | 0 | 505 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 59}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 94}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 35}, {"sub_idx": 3, "type": "att... |
heldout | flower_alone | 96 | 700,096 | 0 | [
"lift_pink_block_table",
"stack_block",
"turn_on_lightbulb",
"unstack_block",
"push_pink_block_left"
] | 1 | false | 0 | 409 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 49}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "fail", "steps": 360}] |
heldout | flower_alone | 97 | 700,097 | 0 | [
"close_drawer",
"move_slider_left",
"rotate_pink_block_left",
"turn_on_led",
"lift_red_block_slider"
] | 5 | true | 0 | 359 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 54}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 83}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_pink_block_left", "status": "success", "steps": 79}, {"sub_idx": 3, "t... |
heldout | flower_alone | 98 | 700,098 | 0 | [
"move_slider_left",
"push_red_block_right",
"push_into_drawer",
"turn_on_led",
"close_drawer"
] | 2 | false | 0 | 465 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 44}, {"sub_idx": 1, "type": "attempt", "subtask": "push_red_block_right", "status": "success", "steps": 61}, {"sub_idx": 2, "type": "attempt", "subtask": "push_into_drawer", "status": "fail", "steps": 360}] |
heldout | flower_alone | 100 | 700,100 | 0 | [
"open_drawer",
"rotate_red_block_left",
"move_slider_left",
"turn_off_lightbulb",
"lift_red_block_table"
] | 5 | true | 0 | 340 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 59}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_red_block_left", "status": "success", "steps": 89}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 63}, {"sub_idx": 3, "typ... |
heldout | flower_alone | 103 | 700,103 | 0 | [
"push_red_block_right",
"lift_pink_block_table",
"stack_block",
"move_slider_right",
"lift_blue_block_slider"
] | 5 | true | 0 | 319 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_red_block_right", "status": "success", "steps": 93}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_table", "status": "success", "steps": 57}, {"sub_idx": 2, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 17}, {"sub_idx": 3, ... |
heldout | flower_alone | 105 | 700,105 | 0 | [
"turn_off_led",
"rotate_pink_block_left",
"lift_red_block_slider",
"place_in_drawer",
"close_drawer"
] | 5 | true | 0 | 354 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 41}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_pink_block_left", "status": "success", "steps": 84}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 103}, {"sub_idx":... |
heldout | flower_alone | 108 | 700,108 | 0 | [
"turn_off_lightbulb",
"push_red_block_right",
"lift_blue_block_slider",
"place_in_drawer",
"move_slider_right"
] | 5 | true | 0 | 447 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 58}, {"sub_idx": 1, "type": "attempt", "subtask": "push_red_block_right", "status": "success", "steps": 152}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 83}, {"sub_... |
heldout | flower_alone | 111 | 700,111 | 0 | [
"move_slider_left",
"push_blue_block_right",
"lift_pink_block_slider",
"place_in_drawer",
"turn_on_lightbulb"
] | 5 | true | 0 | 368 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 57}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 102}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 98}, {"sub_i... |
heldout | flower_alone | 112 | 700,112 | 0 | [
"push_red_block_right",
"move_slider_right",
"lift_blue_block_table",
"place_in_slider",
"close_drawer"
] | 3 | false | 0 | 597 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_red_block_right", "status": "success", "steps": 53}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 84}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 100}, {"sub_id... |
heldout | flower_alone | 113 | 700,113 | 0 | [
"rotate_blue_block_right",
"open_drawer",
"turn_on_lightbulb",
"move_slider_left",
"lift_blue_block_table"
] | 5 | true | 0 | 374 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 95}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 70}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 71}, {"sub_idx": 3, "... |
heldout | flower_alone | 115 | 700,115 | 0 | [
"push_pink_block_left",
"close_drawer",
"lift_blue_block_table",
"place_in_slider",
"turn_on_led"
] | 3 | false | 0 | 585 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 51}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 62}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 112}, {"sub_idx": 3... |
heldout | flower_alone | 119 | 700,119 | 0 | [
"move_slider_right",
"push_blue_block_right",
"turn_off_lightbulb",
"push_into_drawer",
"lift_red_block_slider"
] | 5 | true | 0 | 427 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 68}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_right", "status": "success", "steps": 107}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 51}, {"sub_idx"... |
heldout | flower_alone | 121 | 700,121 | 0 | [
"turn_off_led",
"open_drawer",
"rotate_red_block_left",
"lift_red_block_table",
"stack_block"
] | 5 | true | 0 | 319 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 35}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 63}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_red_block_left", "status": "success", "steps": 99}, {"sub_idx": 3, "type": ... |
heldout | flower_alone | 122 | 700,122 | 0 | [
"rotate_pink_block_left",
"close_drawer",
"turn_on_led",
"move_slider_left",
"lift_red_block_slider"
] | 5 | true | 0 | 420 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_left", "status": "success", "steps": 74}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 74}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 97}, {"sub_idx": 3, "type":... |
heldout | flower_alone | 123 | 700,123 | 0 | [
"move_slider_right",
"lift_red_block_table",
"place_in_drawer",
"rotate_blue_block_left",
"turn_off_led"
] | 5 | true | 0 | 310 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 83}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 47}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 32}, {"sub_idx": 3, ... |
heldout | flower_alone | 124 | 700,124 | 0 | [
"open_drawer",
"push_blue_block_left",
"move_slider_left",
"turn_on_lightbulb",
"lift_pink_block_slider"
] | 2 | false | 0 | 555 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 54}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 141}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 125 | 700,125 | 0 | [
"push_pink_block_left",
"move_slider_left",
"lift_pink_block_table",
"place_in_slider",
"turn_off_lightbulb"
] | 1 | false | 0 | 429 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_left", "status": "success", "steps": 69}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 128 | 700,128 | 0 | [
"turn_off_led",
"lift_blue_block_slider",
"stack_block",
"move_slider_right",
"unstack_block"
] | 5 | true | 0 | 264 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 38}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 64}, {"sub_idx": 2, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 35}, {"sub_idx": 3, "type":... |
heldout | flower_alone | 129 | 700,129 | 0 | [
"lift_blue_block_slider",
"stack_block",
"unstack_block",
"move_slider_left",
"rotate_blue_block_left"
] | 5 | true | 0 | 304 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 72}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 54}, {"sub_idx": 2, "type": "attempt", "subtask": "unstack_block", "status": "success", "steps": 39}, {"sub_idx": 3, "type"... |
heldout | flower_alone | 132 | 700,132 | 0 | [
"close_drawer",
"push_red_block_left",
"lift_pink_block_slider",
"stack_block",
"turn_on_lightbulb"
] | 5 | true | 0 | 429 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 50}, {"sub_idx": 1, "type": "attempt", "subtask": "push_red_block_left", "status": "success", "steps": 161}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 100}, {"sub_idx": ... |
heldout | flower_alone | 133 | 700,133 | 0 | [
"lift_blue_block_slider",
"place_in_drawer",
"turn_off_led",
"push_red_block_left",
"move_slider_right"
] | 5 | true | 0 | 295 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 52}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 51}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 45}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 134 | 700,134 | 0 | [
"push_into_drawer",
"lift_pink_block_slider",
"place_in_drawer",
"close_drawer",
"turn_on_lightbulb"
] | 5 | true | 0 | 309 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_into_drawer", "status": "success", "steps": 74}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_pink_block_slider", "status": "success", "steps": 63}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 42}, {"sub_idx": 3,... |
heldout | flower_alone | 135 | 700,135 | 0 | [
"lift_blue_block_slider",
"place_in_slider",
"turn_off_led",
"move_slider_left",
"rotate_pink_block_left"
] | 3 | false | 0 | 487 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 74}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 11}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 42}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 138 | 700,138 | 0 | [
"rotate_red_block_left",
"open_drawer",
"lift_blue_block_table",
"place_in_slider",
"move_slider_left"
] | 3 | false | 0 | 599 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_red_block_left", "status": "success", "steps": 86}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 82}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 71}, {"sub_idx": 3,... |
heldout | flower_alone | 141 | 700,141 | 0 | [
"turn_on_lightbulb",
"move_slider_left",
"push_blue_block_left",
"push_into_drawer",
"lift_red_block_slider"
] | 1 | false | 0 | 404 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 44}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 144 | 700,144 | 0 | [
"rotate_pink_block_right",
"close_drawer",
"move_slider_right",
"lift_red_block_slider",
"stack_block"
] | 5 | true | 0 | 470 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 144}, {"sub_idx": 1, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 78}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 91}, {"sub_idx": 3,... |
heldout | flower_alone | 145 | 700,145 | 0 | [
"move_slider_right",
"lift_blue_block_slider",
"place_in_slider",
"rotate_red_block_left",
"lift_red_block_table"
] | 3 | false | 0 | 510 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 65}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 78}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 7}, {"sub_idx": 3,... |
heldout | flower_alone | 146 | 700,146 | 0 | [
"push_pink_block_right",
"turn_on_led",
"move_slider_left",
"push_into_drawer",
"lift_red_block_slider"
] | 5 | true | 0 | 639 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 192}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 60}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 123}, {"sub_idx": 3, "t... |
heldout | flower_alone | 149 | 700,149 | 0 | [
"turn_on_lightbulb",
"open_drawer",
"push_into_drawer",
"lift_red_block_drawer",
"place_in_slider"
] | 4 | false | 0 | 599 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 50}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 69}, {"sub_idx": 2, "type": "attempt", "subtask": "push_into_drawer", "status": "success", "steps": 82}, {"sub_idx": 3, "type": ... |
heldout | flower_alone | 151 | 700,151 | 0 | [
"open_drawer",
"turn_on_led",
"rotate_red_block_left",
"lift_blue_block_slider",
"place_in_drawer"
] | 5 | true | 0 | 364 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 55}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 71}, {"sub_idx": 2, "type": "attempt", "subtask": "rotate_red_block_left", "status": "success", "steps": 94}, {"sub_idx": 3, "type": "... |
heldout | flower_alone | 152 | 700,152 | 0 | [
"turn_on_led",
"rotate_red_block_left",
"open_drawer",
"lift_blue_block_table",
"place_in_drawer"
] | 5 | true | 0 | 325 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 45}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_red_block_left", "status": "success", "steps": 92}, {"sub_idx": 2, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 81}, {"sub_idx": 3, "type": "... |
heldout | flower_alone | 153 | 700,153 | 0 | [
"turn_on_lightbulb",
"rotate_pink_block_right",
"lift_blue_block_slider",
"place_in_drawer",
"close_drawer"
] | 5 | true | 0 | 368 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_on_lightbulb", "status": "success", "steps": 49}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_pink_block_right", "status": "success", "steps": 128}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 71}, {"su... |
heldout | flower_alone | 154 | 700,154 | 0 | [
"turn_off_led",
"push_into_drawer",
"lift_blue_block_drawer",
"place_in_drawer",
"lift_red_block_slider"
] | 5 | true | 0 | 241 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 34}, {"sub_idx": 1, "type": "attempt", "subtask": "push_into_drawer", "status": "success", "steps": 84}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_drawer", "status": "success", "steps": 42}, {"sub_idx": 3, "t... |
heldout | flower_alone | 157 | 700,157 | 0 | [
"lift_blue_block_slider",
"place_in_drawer",
"move_slider_left",
"rotate_red_block_right",
"lift_blue_block_drawer"
] | 5 | true | 0 | 420 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 70}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 54}, {"sub_idx": 2, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 105}, {"sub_idx": 3... |
heldout | flower_alone | 158 | 700,158 | 0 | [
"push_pink_block_right",
"move_slider_left",
"turn_off_led",
"push_into_drawer",
"lift_pink_block_drawer"
] | 5 | true | 0 | 272 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 45}, {"sub_idx": 1, "type": "attempt", "subtask": "move_slider_left", "status": "success", "steps": 52}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 40}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 159 | 700,159 | 0 | [
"close_drawer",
"rotate_red_block_right",
"lift_red_block_table",
"stack_block",
"turn_on_led"
] | 5 | true | 0 | 314 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 53}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_red_block_right", "status": "success", "steps": 108}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 56}, {"sub_idx": ... |
heldout | flower_alone | 160 | 700,160 | 0 | [
"lift_red_block_table",
"stack_block",
"unstack_block",
"rotate_red_block_right",
"close_drawer"
] | 3 | false | 0 | 493 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_red_block_table", "status": "success", "steps": 51}, {"sub_idx": 1, "type": "attempt", "subtask": "stack_block", "status": "success", "steps": 28}, {"sub_idx": 2, "type": "attempt", "subtask": "unstack_block", "status": "success", "steps": 54}, {"sub_idx": 3, "type": ... |
heldout | flower_alone | 161 | 700,161 | 0 | [
"move_slider_right",
"rotate_blue_block_right",
"lift_blue_block_table",
"place_in_drawer",
"lift_red_block_slider"
] | 5 | true | 0 | 388 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "move_slider_right", "status": "success", "steps": 60}, {"sub_idx": 1, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 88}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_table", "status": "success", "steps": 111}, {"sub... |
heldout | flower_alone | 163 | 700,163 | 0 | [
"lift_blue_block_slider",
"place_in_drawer",
"close_drawer",
"turn_off_lightbulb",
"move_slider_left"
] | 5 | true | 0 | 413 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 79}, {"sub_idx": 1, "type": "attempt", "subtask": "place_in_drawer", "status": "success", "steps": 58}, {"sub_idx": 2, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 66}, {"sub_idx": 3, "ty... |
heldout | flower_alone | 164 | 700,164 | 0 | [
"rotate_blue_block_right",
"open_drawer",
"turn_off_led",
"move_slider_right",
"lift_blue_block_table"
] | 5 | true | 0 | 495 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "rotate_blue_block_right", "status": "success", "steps": 82}, {"sub_idx": 1, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 79}, {"sub_idx": 2, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 82}, {"sub_idx": 3, "type"... |
heldout | flower_alone | 165 | 700,165 | 0 | [
"close_drawer",
"turn_on_led",
"push_blue_block_left",
"lift_blue_block_table",
"place_in_slider"
] | 5 | true | 0 | 365 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "close_drawer", "status": "success", "steps": 63}, {"sub_idx": 1, "type": "attempt", "subtask": "turn_on_led", "status": "success", "steps": 62}, {"sub_idx": 2, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 79}, {"sub_idx": 3, "type": "... |
heldout | flower_alone | 166 | 700,166 | 0 | [
"open_drawer",
"push_pink_block_right",
"lift_blue_block_slider",
"place_in_slider",
"move_slider_left"
] | 5 | true | 0 | 493 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "open_drawer", "status": "success", "steps": 47}, {"sub_idx": 1, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 235}, {"sub_idx": 2, "type": "attempt", "subtask": "lift_blue_block_slider", "status": "success", "steps": 102}, {"sub_idx":... |
heldout | flower_alone | 170 | 700,170 | 0 | [
"turn_off_lightbulb",
"push_blue_block_left",
"move_slider_right",
"close_drawer",
"lift_blue_block_table"
] | 1 | false | 0 | 414 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_lightbulb", "status": "success", "steps": 54}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_left", "status": "fail", "steps": 360}] |
heldout | flower_alone | 174 | 700,174 | 0 | [
"push_pink_block_right",
"lift_red_block_slider",
"place_in_slider",
"lift_pink_block_table",
"stack_block"
] | 4 | false | 0 | 641 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "push_pink_block_right", "status": "success", "steps": 146}, {"sub_idx": 1, "type": "attempt", "subtask": "lift_red_block_slider", "status": "success", "steps": 67}, {"sub_idx": 2, "type": "attempt", "subtask": "place_in_slider", "status": "success", "steps": 3}, {"sub_idx"... |
heldout | flower_alone | 175 | 700,175 | 0 | [
"turn_off_led",
"push_blue_block_left",
"push_into_drawer",
"move_slider_right",
"lift_blue_block_drawer"
] | 2 | false | 0 | 471 | 0 | [{"sub_idx": 0, "type": "attempt", "subtask": "turn_off_led", "status": "success", "steps": 35}, {"sub_idx": 1, "type": "attempt", "subtask": "push_blue_block_left", "status": "success", "steps": 76}, {"sub_idx": 2, "type": "attempt", "subtask": "push_into_drawer", "status": "fail", "steps": 360}] |
FLOWER on CALVIN ABC→D, with blind retries and with a training-free decision layer
Per-chain evaluation records for a frozen FLOWER VLA policy (mbreuss/flower_calvin_abc,
arXiv 2509.04996) on 600 held-out chains of the CALVIN long-horizon benchmark, in three arms that all
see the same chains with the same seeds.
This dataset contains evaluation results and one replay video only. It contains no model weights and no benchmark data.
- Code and full write-up: https://github.com/nvdm59/frozen-policy-decision-layers
Headline numbers (600 held-out chains, one pass)
| Arm | Avg. instructions completed (of 5) | Chains completing 5/5 | Environment steps |
|---|---|---|---|
flower_alone |
4.487 | 461 | 238,038 |
flower_plus_blind_retries (2 extra attempts per chain) |
4.565 | 482 | 323,089 |
flower_plus_layer (same budget) |
4.732 | 523 | 265,764 |
Layer vs blind retries on full-chain success: 55 wins / 14 losses, exact McNemar p = 6.9e-7.
What the layer does
Before each instruction it checks that instruction's precondition on the simulator state. If the precondition is violated (typically: the robot is still holding the block it should have released), it runs a short scripted recovery and then gives the instruction. It has the same budget of extra attempts as the blind-retry arm.
Read this before quoting the numbers
- Simulation only, privileged state. The layer reads contact and joint state from the simulator.
- 600 of the 1,000 standard chains. The split (seed 2026: development 200 / validation 200 /
held-out 600) is in
splits.json. This is not a leaderboard number. FLOWER's published score on the full protocol is 4.54. - 114 held-out chains had been run with the policy alone before the split existed (listed in
splits.json), during the baseline reproduction and before the layer existed. - Same software renderer for all arms, so arms are comparable with each other.
Columns
| Column | Meaning |
|---|---|
arm |
flower_alone, flower_plus_blind_retries, flower_plus_layer |
chain_idx, seed |
index in the standard 1,000-chain protocol; seed used (700000 + chain_idx) |
instructions |
the five subtasks of the chain |
instructions_completed, full_chain_success |
0–5, and whether all five were completed |
extra_attempt_budget, budget_left |
extra attempts allowed per chain and how many were unused |
env_steps |
environment steps used, including retries and recoveries |
n_layer_interventions |
recoveries performed by the layer in this chain |
events_json |
JSON list of every attempt and recovery |
- Downloads last month
- 73
