SpaMEM

Data release for SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments (ECCV 2026). Paper

Each episode is recorded in a ProcTHOR house while a resident agent moves objects through a sequence of actions. Every action step is captured from 8 headings (0 to 315 degrees) at two camera heights. Each snapshot has depth, instance, meta, rgb, semantic files, and object-level annotations (receptacle groups, 2D boxes, 3D positions, camera distances) with question targets Q1_sort, Q2_rel, Q3_type_count, Q4_count.

Release	Houses	Episodes	Snapshots	Files	Size
v1.0	104	104	53,664	271,778	19.5 GB

Versions

v1.0 is the version used in the ECCV 2026 paper. Pin it for reproducible results (revision="v1.0"); later updates will be published under new tags and v1.0 will not change.

Quick start

pip install huggingface_hub
hf download mill-ct-liao/SpaMEM spamem.py --repo-type dataset --revision v1.0 --local-dir .

python spamem.py --list                                   # all episodes
python spamem.py --episode episode_resident_sim_1771159768410 --check   # download one episode and check it
python spamem.py --all --root ./SpaMEM                    # everything

import spamem

ep_dir = spamem.download_episode("episode_resident_sim_1771159768410", root="./SpaMEM")
ep = spamem.Episode(ep_dir)
for frame in ep.frames():
    frame["action"], frame["step"], frame["snapshot_label"]
    frame["paths"]["rgb"]            # pathlib.Path inside the extracted episode
    frame["topology_clusters"]        # objects, boxes, 3D positions, question targets

spamem.py only needs huggingface_hub. It checks every archive against the SHA-256 in manifest.csv before extracting it.

Layout

README.md
spamem.py                                             # loader
manifest.csv                                          # one row per episode: actions, snapshots, files, bytes, sha256
house_train_<id>/
    episode_resident_sim_<ts>.tar.gz                  # the episode
    episode_resident_sim_<ts>.spamem_full_data.json   # its annotation, also inside the archive

An extracted episode:

episode_resident_sim_<ts>/
    spamem_full_data.json
    action0/ ... actionN/
        <step>/
            rgb/ depth/ semantic/ instance/ meta/

The file_paths stored in spamem_full_data.json were written on the recording machine (they start with dynamic_procthor/...). Resolve them relative to the extracted episode folder; Episode.resolve() and Episode.frames() do this for you.

Without the loader:

hf download mill-ct-liao/SpaMEM --repo-type dataset --revision v1.0 --local-dir SpaMEM
cd SpaMEM && sha256sum -c <(tail -n +2 manifest.csv | awk -F, '{print $9"  "$3}')
for f in */*.tar.gz; do tar -xzf "$f" -C "$(dirname "$f")"; done

License

The data is released under CC BY 4.0. ProcTHOR and AI2-THOR assets keep their original licenses.

Citation

@inproceedings{liao2026spamem,
  title     = {SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception--Memory Integration in Embodied Environments},
  author    = {Liao, Chih-Ting and Xiao, Xi and Meng, Chunlei and Chen, Zhangquan and Qiao, Yitong and Zhou, Weilin and Wang, Tianyang and Zheng, Xu and Cao, Xin},
  booktitle = {Computer Vision -- ECCV 2026},
  series    = {Lecture Notes in Computer Science},
  volume    = {17052},
  pages     = {425--442},
  year      = {2026},
  publisher = {Springer, Cham},
  doi       = {10.1007/978-3-032-37235-2_25}
}