---
pretty_name: AgroMind
license: cc-by-sa-4.0
task_categories:
- image-text-to-text
- visual-question-answering
size_categories:
- 10K<n<100K
source_datasets:
- OAM-TCD Dataset
- OilPalmUAV Dataset
- Agriculture Vision Challenge Dataset
- PhenoBench Dataset
- Maize Tassel Identification Dataset
- IP102 Dataset
- ACFR Orchard Fruit Dataset
- Global Parcel Dataset
- 2018 AI Challenge Dataset
- CropHarvest Dataset
configs:
- config_name: default
  data_files:
  - split: train
    path: train-0000-of-0001.parquet
dataset_info:
  features:
  - name: images
    list:
    - name: bytes
      dtype: 'null'
    - name: path
      dtype: string
  - name: id
    dtype: string
  - name: messages
    list:
    - name: role
      dtype: string
    - name: content
      list:
      - name: type
        dtype: string
      - name: text
        dtype: string
  - name: raw_metadata
    dtype: string
  splits:
  - name: train
    num_examples: 28482
tags:
- vlm
- vision-language
- agriculture
- vqa
- agromind
---

# AgroMind

![AgroMind sample images](static/AgroMind.jpg)

AgroMind is a benchmark that tests how well large multimodal models understand agricultural remote sensing scenes. It has 28,482 question-answer pairs over 20,850 images taken by UAVs, satellites and ground cameras. The images come from nine public datasets and one private global parcel dataset, and cover 106 regions around the world. By sensor, there are 7,158 UAV, 12,268 satellite and 9,056 camera QA pairs.

The questions cover 4 dimensions and 13 task types, from crop identification and health monitoring to environmental analysis. The authors tested 20 open-source and 4 closed-source models. Models struggled most with spatial reasoning and fine-grained recognition, and several leading models did better than humans.

- Paper: [Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind](https://arxiv.org/abs/2505.12207)
- Original dataset: [AgroMind/AgroMind](https://huggingface.co/datasets/AgroMind/AgroMind)
- Code: [rssysu/AgroMind](https://github.com/rssysu/AgroMind)

## Tasks

| Dimension | Task | Rows |
|---|---|---|
| Spatial Perception | Spatial Localization (SL) | 2,036 |
| | Spatial Relationship (SR) | 1,286 |
| | Boundary Detection (BD) | 1,820 |
| Object Understanding | Object Classification (OC) | 3,477 |
| | Pest/Disease Diagnostics (PDD) | 2,300 |
| | Growth Status Recognition (GSR) | 1,200 |
| Scene Understanding | Scene Comparison (SC) | 1,657 |
| | Counting (CO) | 4,231 |
| | Area Statistics (AS) | 3,176 |
| Scene Reasoning | Visual Prompt Reasoning (VPR) | 2,364 |
| | Anomaly Reasoning (AR) | 1,521 |
| | Climate Type Reasoning (CTR) | 1,073 |
| | Planning (PL) | 2,341 |

## Structure

```
train-0000-of-0001.parquet
images/
  image-train-000-of-NNN.zip
  path_to_shard.parquet
```

Each row has:

- `images`: list of `{"bytes": null, "path": <sha256>}`. The path is the SHA256 of the image bytes and points into a zip shard through `images/path_to_shard.parquet`.
- `id`: `agromind_<task>_<counter>`, for example `agromind_pdd_66`.
- `messages`: one user turn (images, then question text) and one assistant turn (answer).
- `raw_metadata`: JSON string with `task`, `type_id`, `item_id`, `level1_id`, `level2_id`, `level3_id` and `raw_image_paths`. Some rows also have `image_id` or `isMultiple`.

Example:

```json
{
  "id": "agromind_ar_1",
  "messages": [
    {"role": "user", "content": [
      {"type": "image", "text": null},
      {"type": "text", "text": "which anomalous area is in this image of farmland?\nA. cloud_shadow\nB. double_plant\nC. planter_skip\nD. standing_water\nE. waterway\nF. weed_cluster\nG. No anomaly"}
    ]},
    {"role": "assistant", "content": [{"type": "text", "text": "F. weed_cluster"}]}
  ],
  "raw_metadata": "{\"task\": \"AR\", \"type_id\": 9, \"item_id\": 125, \"level1_id\": 4, \"level2_id\": 2, \"level3_id\": 1, \"raw_image_paths\": [\"./Agriculture/images/7268V1Z27_1995-6302-2507-6814.jpg\"]}"
}
```

### Changes from the original data

- The 13 QA files are merged into one `train` split. The file name is kept as `task` in `raw_metadata`.
- Options are added to the question as lines like `A. text`.
- When the options are images (`type_id` 4 and 5), the images are given in A to D order and the options read `A. Image 1`, `B. Image 2` and so on.
- Answers combine the letter and the option text, for example `B. Citrus`. Questions with more than one correct answer list each answer on its own line.
- Number options and bounding box answers are turned into strings. Bounding box values are unchanged. They were already normalized to 0-1 as `[xmin, ymin, xmax, ymax]`.
- `raw_metadata` keeps the source fields except `question`, `answer`, `options`, `image_path`, `image1` and `image2`, which are already in `messages`. All image paths of a row are kept in `raw_image_paths`.
- Images are stored once, deduplicated by SHA256.

## Usage

```python
from agml import loadImageTextToTextDataset

dataset = loadImageTextToTextDataset("Project-AgML/AgroMind")

sample = dataset["train"][0]
sample["images"]    # list of PIL images
sample["messages"]
```

## Citation

```bibtex
@misc{li2025largemultimodalmodelsunderstand,
      title={Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind},
      author={Qingmei Li and Yang Zhang and Zurong Mai and Yuhang Chen and Shuohong Lou and Henglian Huang and Jiarui Zhang and Zhiwei Zhang and Yibin Wen and Weijia Li and Haohuan Fu and Jianxi Huang and Juepeng Zheng},
      year={2025},
      eprint={2505.12207},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.12207},
}
```

This dataset is indexed and structured on https://project-agml.github.io/ as part of Project AgML @ UC Davis. This dataset was reformatted from its original format to match HuggingFace's Imagefolder standards but requires an external module (agml) that processes and returns a HF Dataset object faster than HF module functions.