Workshop code: lab_llm

The take-home code for the LLM APIs for Research Analysis workshop.

The workshop site does the live teaching. This repo is the polished result: one small reusable package (lab_llm) plus a runnable example per module.

Layout

lab_llm/                the reusable package (install once, use everywhere)
  calls.py              call_llm(), the reusable one-call helper
  conversations.py      stored and stateless multi-turn helpers
  jobs.py               sequential or multiprocess job execution
  records.py            durable responses and resume bookkeeping
  ratings.py            easy transcript x item rating batches
  runs.py               standard batch arguments, plans, and manifests
  inputs.py             prompt templates, transcripts, and item banks
  progress.py           elapsed time, ETA, and token-cost estimates
  structured.py         versioned output types and validation rules
  privacy.py            local PII detection + stable pseudonyms
  files.py              persistent or temporary Files API uploads
  tools.py              readable hosted-tool configurations
  config.py             API key + model, loaded from the environment
  errors.py             small package-specific exception types
examples/                runnable examples from the workshop
  01_first_call/        one call: raw SDK + lab_llm
  02_examples_gallery/  response-object field examples
  03_tiny_chat_loop/    multi-turn chat: raw SDK + lab_llm
  04_stateless_conversation/  local history; API response storage off
  05_files/              Files API upload and response input
  06_web/                hosted web-search example
  07_code_interpreter/   hosted Python example
  08_sequential_ratings/ transcript x item ratings; one or more workers
  09_check_results/      classify saved rating outcomes
  10_structured_outputs/ simple JSON Schema and typed output
  11_complex_structured_outputs/ nested evidence and justifications
  12_mood_diary_workflow/ extract, synthesize, score, audit, report
  13_tool_calling/       raw Responses API function-tool loop
  14_research_agent/     multi-turn study investigator + hosted Python
  15_local_deidentification/ preview and locally filter research text
data/                    shared sample transcripts, item banks, and instructions
  mood_diaries/          eight synthetic, dated diary entries
  model_pricing.csv      saved OpenAI token-price snapshot for long runs
  synthetic_rating_results.jsonl  five example rating outcomes
scripts/                 setup / run / uninstall (macOS + Windows)

Each module contains runnable examples. Files with nicer_example in the name repeat the raw SDK calls through lab_llm. The core is written once. Later examples build on it.

Two ways to use this repo

• Build it from scratch. The workshop site and the raw example.py files use the plain OpenAI SDK, so you can see exactly what happens and write your own scripts.
• Adopt the package. lab_llm wraps those same calls with conveniences worth reusing: it loads your .env, fails closed on incomplete or failed responses (LLMResponseError), and honors optional timeout/retry settings. The nicer_example.py and gallery files show it in use.

Neither is more correct. Use the raw calls to learn; adopt lab_llm when you would rather not rewrite the plumbing every time.

Setup

You do not need Python installed. Setup downloads a private Python and all dependencies inside this folder. Nothing is installed system-wide. Deleting the folder removes the project files.

macOS / Linux

./scripts/setup.sh          # installs a private Python + packages, all in this folder
# open .env and paste your OpenAI key
./scripts/run.sh examples/01_first_call/example.py

Windows (PowerShell)

.\scripts\setup.ps1         # installs a private Python + packages, all in this folder
# open .env and paste your OpenAI key
.\scripts\run.ps1 examples\01_first_call\example.py

Your key lives in .env (gitignored) and is read from the environment. It never lives in code.

The helpers keep the complete OpenAI response. Reply text and token usage stay easy to reach. Conversation uses one durable conversation ID. StatelessConversation keeps the complete history locally and sends it again with every turn. Both reuse their instructions across turns.

PromptTemplate validates named placeholders before a run. TranscriptBank loads one text file per transcript. ItemBank loads uniquely identified items with numeric bounds or exact user-defined text responses from CSV. Each stays iterable, so the transcript x item loop remains ordinary Python.

call_llm() fails closed when a response is incomplete or failed. It raises LLMResponseError with the full response attached. OpenAI SDK exceptions are left unchanged, so callers can still catch specific authentication, rate-limit, connection, and API errors.

run_jobs() runs independent calls sequentially by default. Set workers to use multiple processes. Workers make API calls; the parent alone writes each returned attempt to JSONL. Reusing the output path skips completed jobs and retries failed ones. Pass explicit TokenPricing to add live elapsed time, ETA, usage cost, and projected final cost. OpenAI responses contain token counts, not a dollar charge; the saved rate card makes the estimate auditable.

run_rating_batch() is the short path for transcript x item studies. Define the Pydantic result and list every study path. The helper supplies the standard command line, preflight, resume, validation, CSV, and run summary. Use run_jobs() directly when a study needs a different job shape.

Jobs may also carry an explicit Responses API output_format. The ratings example uses strict Structured Outputs, validates item-specific ranges or exact text choices again locally, supports a zero-call --dry-run, and saves summary.json.

OutputContract versions a Pydantic output type. It produces the Responses API JSON Schema and parses JSON into that Python type. Pass it to run_jobs() to validate each completed response before the parent process saves it. Parsed output is saved as plain JSON. With the lower-level runner, research-specific checks remain in the calling code.

Optional local de-identification

OpenAI Privacy Filter is an optional, open-weight model that runs before an API request. Unfiltered text stays on the researcher's machine; only the filtered copy is sent to the configured LLM endpoint. The standard workshop setup installs its pinned official runtime package. If this environment was created before local de-identification was added, update it once with:

.bin/uv pip install --python .venv/bin/python -e ".[agents,privacy]"

The first use downloads the model checkpoint to ~/.opf/privacy_filter. For an offline or controlled environment, download and approve the checkpoint first, set OPF_CHECKPOINT to that local directory, and then disable network access. CPU execution is the workshop default; pass device="cuda" on a supported GPU. The optional checkpoint is outside this project, so the workshop uninstall scripts do not remove it.

Reuse one Deidentifier across related text so the same person receives the same process-local placeholder:

from lab_llm import Deidentifier, call_llm

privacy = Deidentifier(device="cpu")
preview = privacy.deidentify("Interview with Maya Chen on 2026-04-12.")
print(preview.preview())                 # local review; reveals detected PII

result = call_llm(
    "Interview with Maya Chen on 2026-04-12.",
    instructions="Summarize the interview.",
    deidentifier=privacy,
)
print(result.deidentification.to_dict()) # counts only; no original PII

The same deidentifier= option is supported by Conversation, StatelessConversation, run_jobs, run_rating_batch, upload_file, and temporary_file. Filter selected CSV/JSON-style columns with deidentify_records(). For raw SDK calls with message lists or local function tool output, pass the input through deidentify_responses_input() before client.responses.create().

Filtered uploads support UTF-8 text files. Extract text from PDF, Word, images, audio, and other binary formats before filtering; passing an already-uploaded file ID cannot retroactively protect that file. Direct OpenAI() calls also bypass lab_llm, so they must use deidentify_responses_input() explicitly.

Privacy Filter reduces exposure; it does not prove that data is anonymous or compliant with a particular policy. Its taxonomy can miss study-specific IDs, rare attributes, and combinations of details that permit re-identification. Evaluate it on representative in-domain data, tune the operating point when needed, retain an appropriate review path, and document which labels are filtered. The default filters all eight released label categories. Selecting a subset can preserve analytically important fields, but should be a deliberate, reviewed policy choice.

Optional .env settings:

OPENAI_TIMEOUT=60       # seconds
OPENAI_MAX_RETRIES=2    # automatic retries after the first attempt

Leave either setting unset to use the OpenAI SDK default.

Run a specific module

./scripts/run.sh examples/03_tiny_chat_loop/example.py      # macOS / Linux
.\scripts\run.ps1 examples\03_tiny_chat_loop\example.py     # Windows

Remove the local install

./scripts/uninstall.sh      # macOS / Linux
.\scripts\uninstall.ps1     # Windows

This deletes the private Python, the environment—including the installed Privacy Filter runtime—caches, and .env. Source code and run outputs stay. Privacy Filter checkpoints outside the project stay because they may be shared with other projects. Deleting the whole folder removes the remaining project-local files.

Prefer your own Python?

If you already have Python 3.10+ and would rather manage it yourself:

python3 -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install -e ".[agents,privacy]"
cp .env.example .env           # then add your OpenAI key
python examples/01_first_call/example.py

Tests

python -m unittest discover -s tests -v

The tests use fake clients. They do not make paid API calls.

Requirements

• No Python required. setup installs a private one. (Or use your own, 3.10+.)
• An OpenAI API key (OPENAI_API_KEY)