# Dataset Card: Heat-Assisted Detection and Ranging (HADAR) Inputs

## Overview

This bundle supports ground-based long-wave infrared (LWIR) hyperspectral imaging for heat-assisted detection and ranging. It combines five scenes from the [DARPA IH Dataset](https://www.kitware.com/ihdataset/) with **per-scene** atmospheric products: a zenith (vertically downward) downwelling spectral radiance and a matching spectral attenuation coefficient.

The atmospheric files are **not** a shared U.S. Standard Atmosphere (USSA) snapshot. For each capture, attenuation and zenith downwelling were computed with [libRadtran](http://www.libradtran.org/) (`uvspec`, DISORT, thermal emission) from a layered temperature and gas-density profile at the **nearest analysis time and nearest grid location** to that scene, plus high-resolution HITRAN line-by-line optical depths (H₂O, CO₂, O₃, NO₂). Both products use the same 6–14 µm grid at 4 nm spacing.

The ENVI cubes are too large for GitHub. The image build fetches this same layout from the frozen Hugging Face dataset [dccc2025/tmp_dataset_hadar](https://huggingface.co/datasets/dccc2025/tmp_dataset_hadar) and checks SHA256. That fetch is pinned to a commit (not a moving `main` branch) and is not a live scrape of Kitware.

## Directory Layout

```text
inputs/
├── DATASET_CARD.md
├── attenuation/
│   ├── IHTest_202104_Path15_Step2_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path17_Step1_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path17_Step3_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA.npz
│   └── IHTest_202104_Path27_Step25_LWHSI1_collect0_DistStA.npz
├── downwelling_radiance/
│   ├── IHTest_202104_Path15_Step2_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path17_Step1_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path17_Step3_LWHSI1_collect0_DistStA.npz
│   ├── IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA.npz
│   └── IHTest_202104_Path27_Step25_LWHSI1_collect0_DistStA.npz
└── original_datasets/
    ├── IHTest_202104_Path15_Step2_LWHSI1_collect0_DistStA.{bsq,hdr}
    ├── IHTest_202104_Path17_Step1_LWHSI1_collect0_DistStA.{bsq,hdr}
    ├── IHTest_202104_Path17_Step3_LWHSI1_collect0_DistStA.{bsq,hdr}
    ├── IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA.{bsq,hdr}
    └── IHTest_202104_Path27_Step25_LWHSI1_collect0_DistStA.{bsq,hdr}
```

Each scene stem has **one attenuation file and one downwelling file** with the same name. Pair them by stem; do not mix products across scenes.

## 1. Hyperspectral Scenes (`original_datasets/`)

Five ENVI-format LWIR hyperspectral cubes (`.bsq` + `.hdr`) from the DARPA IH Dataset. Each pair is a band-sequential cube plus acquisition metadata. Headers record wavelength (µm) and a near-surface air `temperature` (kelvin). They do **not** record a radiance scale factor.

| Scene stem | Acquisition time (UTC) | Site | Lines | Samples | Bands |
|------------|------------------------|------|------:|--------:|------:|
| `IHTest_202104_Path15_Step2_LWHSI1_collect0_DistStA` | 2021-04-16T17:53:52 | Fort A.P. Hill, VA | 260 | 1600 | 256 |
| `IHTest_202104_Path17_Step1_LWHSI1_collect0_DistStA` | 2021-04-16T21:43:22 | Fort A.P. Hill, VA | 260 | 1600 | 256 |
| `IHTest_202104_Path17_Step3_LWHSI1_collect0_DistStA` | 2021-04-16T21:52:54 | Fort A.P. Hill, VA | 260 | 1600 | 256 |
| `IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA` | 2021-04-20T23:13:02 | Fort A.P. Hill, VA | 260 | 1600 | 256 |
| `IHTest_202104_Path27_Step25_LWHSI1_collect0_DistStA` | 2021-04-21T03:24:47 | Fort A.P. Hill, VA | 260 | 1600 | 256 |

- **Format:** ENVI BSQ, 32-bit float (`data type = 4`).
- **Wavelengths:** Stored in each `.hdr` file; units are micrometers (µm). **All five cubes are offset by 120 nm toward longer wavelengths in the header calibration.** Subtract 0.12 µm from every listed band center before use. Example: a header range of 8–13.17 µm corresponds to a corrected range of 7.88–13.05 µm. Apply this shift when aligning cubes to Planck functions or to the atmosphere files.
- **Distribution:** `Distribution Statement A` (see scene headers).

### Cube radiance units

The `.bsq` arrays are **not** already in
\(\mathrm{W\,m^{-2}\,sr^{-1}\,\mu m^{-1}}\). They are stored a factor of
\(10^{2}\) high relative to physical spectral radiance. Convert with

\[
L_{\lambda}\;[\mathrm{W\,m^{-2}\,sr^{-1}\,\mu m^{-1}}]
= 10^{-2}\,L_{\mathrm{bsq}}.
\]

Use this scaled \(L_{\lambda}\) in any comparison to Planck \(B_\lambda(T)\)
or to the bundled zenith downwelling (which is already in physical units).
Do not treat header `temperature` as the surface temperature map; it is
local air temperature.

### Source Citation

```bibtex
@InProceedings{Yellin_2024_WACV,
    author    = {Yellin, Florence and McCloskey, Scott and Hill, Cole and Smith, Eric and Clipp, Brian},
    title     = {Concurrent Band Selection and Traversability Estimation from Long-Wave Hyperspectral Imagery in Off-Road Settings},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    month     = {January},
    year      = {2024}
}
```

**Dataset homepage:** https://www.kitware.com/ihdataset/

## 2. How the atmosphere files were computed

For each scene, a vertical profile of temperature and molecular number density was taken at the nearest available analysis time and the nearest grid point to the capture site (Fort A.P. Hill, Virginia). Layer optical depths on 6–14 µm were built from HITRAN high-resolution line parameters for H₂O, CO₂, O₃, and NO₂. libRadtran then produced:

1. **Spectral attenuation** of the lowest (near-surface) layer, stored as α in dB m⁻¹.
2. **Zenith downwelling spectral radiance** only: DISORT thermal calculation, surface observer, viewing cosine `umu = −1` (zenith angle 0°). There is **no** multi-angle downwelling basis.

**Spectral grid (both products, all scenes):** 6.000–14.000 µm, Δλ = 4 nm, 2001 samples. This grid is **not** the camera wavelength axis in the `.hdr` files.

## 3. Spectral Attenuation (`attenuation/<scene_stem>.npz`)

One file per scene. Use the file whose stem matches the cube.

| Key | Shape | Description |
|-----|-------|-------------|
| `attenuation` | `(2001, 1)` | Spectral attenuation coefficient α, in **dB m⁻¹** |
| `lambda` | `(1, 2001)` | Wavelength grid, **6–14 µm** at **4 nm** |

Approximate α ranges on this 4 nm grid (water-band peaks near 6–8 µm are much larger than the LWIR window):

| Scene stem | α min (dB m⁻¹) | α max (dB m⁻¹) |
|------------|----------------:|----------------:|
| Path15 Step2 LWHSI1 | 3.02×10⁻⁴ | 44.9 |
| Path17 Step1 LWHSI1 | 3.49×10⁻⁴ | 50.3 |
| Path17 Step3 LWHSI1 | 3.49×10⁻⁴ | 50.3 |
| Path26 Step2 LWHSI1 | 5.59×10⁻⁴ | 71.0 |
| Path27 Step25 LWHSI1 | 5.22×10⁻⁴ | 66.5 |

Load example:

```python
import numpy as np

stem = "IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA"
data = np.load(f"attenuation/{stem}.npz")
alpha = data["attenuation"]   # (2001, 1), dB/m
lam_att = data["lambda"]      # (1, 2001), µm
```

Horizontal-path transmittance at range \(d\) (metres) is \(\tau(d)=10^{-\alpha d/10}\). Resample `alpha` onto the **corrected** scene wavelength grid before applying this formula.

## 4. Zenith Downwelling Radiance (`downwelling_radiance/<scene_stem>.npz`)

One file per scene, already in physical radiance units. A single column: **zenith only** (vertical downward / zenith angle 0°). This is not an angle-resolved sky basis.

| Key | Shape | Description |
|-----|-------|-------------|
| `lambda` | `(1, 2001)` | Same 6–14 µm, 4 nm grid as the matching attenuation file |
| `downwelling_res` | `(2001, 1)` | Zenith downwelling spectral radiance, **W m⁻² sr⁻¹ µm⁻¹** |
| `zenith_angle_deg` | `(1,)` | `[0]` |

Load example:

```python
import numpy as np

stem = "IHTest_202104_Path26_Step2_LWHSI1_collect0_DistStA"
data = np.load(f"downwelling_radiance/{stem}.npz")
lam = data["lambda"]                    # (1, 2001), µm
I_down = data["downwelling_res"]        # (2001, 1), W·m⁻²·sr⁻¹·µm⁻¹
zenith = data["zenith_angle_deg"]       # [0]
```

## Units and Conventions

| Quantity | Symbol / key | Units |
|----------|--------------|-------|
| Wavelength | `lambda` | µm |
| Attenuation coefficient | `attenuation` (α) | dB m⁻¹ |
| Zenith downwelling spectral radiance | `downwelling_res` | W m⁻² sr⁻¹ µm⁻¹ |
| Hyperspectral cube | ENVI `.bsq` | stored \(10^{2}\) high; \(L=10^{-2}L_{\mathrm{bsq}}\) in W m⁻² sr⁻¹ µm⁻¹ |
| Header `temperature` | air temperature | K |

## Notes for Use

1. **Header wavelength offset.** Every cube’s `.hdr` wavelengths are 120 nm too long. Correct with \(\lambda_{\mathrm{true}}=\lambda_{\mathrm{hdr}}-0.12\,\mu\mathrm{m}\) before any spectral work. If a header lists 8–13.17 µm, the physical band is 7.88–13.05 µm.
2. **Wavelength alignment.** After that 120 nm correction, all five LWHSI1 cubes have 256 bands. The atmosphere `lambda` grid is 6–14 µm at 4 nm and does **not** match the camera bands. Interpolate or band-select attenuation and downwelling onto the corrected scene wavelength grid before combining them with a cube.
3. **Paired products.** Each scene has its own matching attenuation and zenith-downwelling files, computed from the nearest analysis time and nearest grid location to that capture. Load both `.npz` files from the same scene stem; they share one `lambda` axis. This per-scene pairing is more accurate than a shared standard atmosphere.
4. **Zenith only.** `downwelling_res` has shape `(2001, 1)`. There are no additional zenith-angle columns. This vector is not the delivered single-band texture map `X`.