diff --git a/.github/workflows/lint_sqlfluff.yml b/.github/workflows/lint_sqlfluff.yml index 0986785b..d6e1c3f0 100644 --- a/.github/workflows/lint_sqlfluff.yml +++ b/.github/workflows/lint_sqlfluff.yml @@ -1,11 +1,17 @@ name: SQLFluff +# Lints changed mimic-iv/concepts/*.sql on pull requests and posts GitHub +# annotations. Fork PRs skip Annotate — GITHUB_TOKEN cannot create check runs +# from forks (Resource not accessible by integration / 403). on: - pull_request jobs: lint-mimic-iv: runs-on: ubuntu-latest + permissions: + contents: read + checks: write steps: - name: checkout uses: actions/checkout@v7 @@ -35,8 +41,14 @@ jobs: shell: bash run: sqlfluff lint --format github-annotation --annotation-level failure --nofail ${{ steps.get_files_to_lint.outputs.lintees }} > annotations.json - name: Annotate + # Same-repo PRs only: fork tokens get 403 creating check runs, and + # ignore-unauthorized-error does not treat that as unauthorized. + if: > + steps.get_files_to_lint.outputs.lintees != '' && + github.event.pull_request.head.repo.full_name == github.repository uses: yuzutech/annotations-action@v0.6.0 with: repo-token: "${{ secrets.GITHUB_TOKEN }}" title: "SQLFluff Lint" - input: "./annotations.json" \ No newline at end of file + input: "./annotations.json" + ignore-unauthorized-error: true diff --git a/README.md b/README.md index 53fe9f9f..756f7726 100644 --- a/README.md +++ b/README.md @@ -114,3 +114,7 @@ By committing your code to the [MIMIC Code Repository](https://github.com/mit-lc ### Coding style Please refer to the [style guide](https://github.com/MIT-LCP/mimic-code/blob/main/styleguide.md) for guidelines on formatting your code for the repository. + +### MIMIC-IV timing notes + +- [Admission deathtime vs transfers](mimic-iv/docs/DEATHTIME.md) diff --git a/mimic-iv/README.md b/mimic-iv/README.md index 919b196c..adda4a6d 100644 --- a/mimic-iv/README.md +++ b/mimic-iv/README.md @@ -105,4 +105,8 @@ mimic_utils convert_folder mimic-iv/concepts mimic-iv/concepts_duckdb --destinat # To PostgreSQL: mimic_utils convert_folder mimic-iv/concepts mimic-iv/concepts_postgres --destination_dialect postgres -``` \ No newline at end of file +``` + +## Mortality timestamps + +`admissions.deathtime` can precede later `transfers` rows — see [docs/DEATHTIME.md](docs/DEATHTIME.md). diff --git a/mimic-iv/concepts/documentation/README.md b/mimic-iv/concepts/documentation/README.md new file mode 100644 index 00000000..15576507 --- /dev/null +++ b/mimic-iv/concepts/documentation/README.md @@ -0,0 +1,4 @@ +# Documentation helper queries + +Ad-hoc SQL that illustrates known data quirks. These are **not** part of the +official derived concept build — run them against a local or BigQuery copy. diff --git a/mimic-iv/concepts/documentation/icu_outtime_after_deathtime.sql b/mimic-iv/concepts/documentation/icu_outtime_after_deathtime.sql new file mode 100644 index 00000000..4fadf65a --- /dev/null +++ b/mimic-iv/concepts/documentation/icu_outtime_after_deathtime.sql @@ -0,0 +1,13 @@ +-- ICU stays whose outtime is after the admission deathtime. + +SELECT + COUNT(*) AS icustays_out_after_death, + COUNT(*) FILTER ( + WHERE ie.outtime > a.deathtime + AND ie.outtime <= a.deathtime + INTERVAL '48 hours' + ) AS within_48h +FROM icu.icustays AS ie +INNER JOIN hosp.admissions AS a + ON ie.hadm_id = a.hadm_id +WHERE a.deathtime IS NOT NULL + AND ie.outtime > a.deathtime; diff --git a/mimic-iv/concepts/documentation/transfers_after_deathtime.sql b/mimic-iv/concepts/documentation/transfers_after_deathtime.sql new file mode 100644 index 00000000..26482544 --- /dev/null +++ b/mimic-iv/concepts/documentation/transfers_after_deathtime.sql @@ -0,0 +1,11 @@ +-- Count hospital transfers whose intime falls after admissions.deathtime. +-- Illustrates the #1945 ADT-lag pattern (not a stay_id join failure). + +SELECT + COUNT(*) AS transfer_rows_after_deathtime, + COUNT(DISTINCT t.hadm_id) AS admissions_affected +FROM hosp.transfers AS t +INNER JOIN hosp.admissions AS a + ON t.hadm_id = a.hadm_id +WHERE a.deathtime IS NOT NULL + AND t.intime > a.deathtime; diff --git a/mimic-iv/docs/DEATHTIME.md b/mimic-iv/docs/DEATHTIME.md new file mode 100644 index 00000000..751e6717 --- /dev/null +++ b/mimic-iv/docs/DEATHTIME.md @@ -0,0 +1,61 @@ +# Interpreting `admissions.deathtime` + +Hospital admissions can show a `deathtime` while `transfers` (and ICU +`icustays`) still list care-unit activity **after** that timestamp. That is a +known pattern in MIMIC-IV, not usually a join bug. + +See [issue #1945](https://github.com/MIT-LCP/mimic-code/issues/1945). + +## What `deathtime` is + +| Column | Table | Meaning | +|--------|-------|---------| +| `deathtime` | `hosp.admissions` | Time of death recorded for **that hospital admission**, when death occurred in-hospital | +| `hospital_expire_flag` | `hosp.admissions` | 1 if the patient died during the admission | +| `dod` | `hosp.patients` | Date of death at the **patient** level (coarser; may come from hospital or Social Security sources depending on version/docs) | + +`deathtime` is an admission-scoped clinical/administrative death time. It is +**not** forced to equal the last `transfers.outtime` or `icustays.outtime`. + +## Why transfers can continue after `deathtime` + +Common explanations (often several at once): + +1. **ADT lag.** Transfer / location rows are written when the bed-management + system updates, which can trail the documented time of death. +2. **Post-mortem logistics.** Patients may still appear to "move" (e.g. ICU → + PACU / morgue-related locations) in the transfer chain after death is + recorded. +3. **Clock disagreement.** Death documentation and ADT systems are not always + on one synchronized clock in the released tables. +4. **Stay construction.** Derived ICU stays use transfer/careunit rules; they + are not clipped to `deathtime` in the PhysioNet loaders. + +So for `hadm_id` examples like the one in #1945, prefer treating `deathtime` +as the mortality timestamp for that admission, and treat later transfer rows as +ADT/administrative trail — unless your study specifically needs location-based +definitions of "end of care." + +## Practical guidance + +- For **in-hospital mortality**, use `hospital_expire_flag` / `deathtime` on + `admissions` (and `patients.dod` when you need patient-level death date). +- Do **not** require `deathtime >= max(transfers.outtime)` as a data-quality + filter without documenting that you are excluding real deaths with late ADT. +- If you need "alive in ICU until outtime," define that rule explicitly (e.g. + ICU outtime, or `least(outtime, deathtime)`), and report sensitivity to the + choice. +- This repository will **not** rewrite transfer out-times from `deathtime` in + the CSV loaders. + +## Related notes + +- ICU charting after outtime: `mimic-iv/docs/CHARTEVENTS_TIMING.md` +- ED intime vs pyxis: `mimic-iv-ed/docs/TIMING_NOTES.md` + +## What this repository will not change + +MIMIC-Code does **not** clip `transfers` or `icustays` to `admissions.deathtime` +in the PhysioNet loaders. Aligning ADT trails with mortality timestamps belongs +in analysis code or an explicitly documented derived concept. +