|
| 1 | +--- |
| 2 | +outline: deep |
| 3 | +title: Process |
| 4 | +description: Run shell commands and manage subprocesses with a fluent, testable API in Fastapi Startkit. |
| 5 | +keywords: process, subprocess, shell, commands, async, pipes, pools, testing, fastapi startkit |
| 6 | +--- |
| 7 | + |
| 8 | +# Process |
| 9 | + |
| 10 | +Fastapi Startkit ships a `Process` facade that wraps Python's `subprocess` module behind a clean, fluent interface. It lets you run shell commands synchronously, stream output asynchronously, build pipelines, and run pools of concurrent processes — all with first-class support for test fakes so no real processes are spawned during your test suite. |
| 11 | + |
| 12 | +## Basic Usage |
| 13 | + |
| 14 | +Import `Process` and call `run()` with any shell command. It returns a `ProcessResult`: |
| 15 | + |
| 16 | +```python |
| 17 | +from fastapi_startkit.process import Process |
| 18 | + |
| 19 | +result = Process.run('ls -la') |
| 20 | + |
| 21 | +print(result.output()) # stdout as a string |
| 22 | +print(result.exit_code()) # integer exit code |
| 23 | +``` |
| 24 | + |
| 25 | +### Handling Success and Failure |
| 26 | + |
| 27 | +```python |
| 28 | +result = Process.run('git status') |
| 29 | + |
| 30 | +if result.successful(): |
| 31 | + print("Command succeeded") |
| 32 | + |
| 33 | +if result.failed(): |
| 34 | + print("Command failed:", result.error_output()) |
| 35 | +``` |
| 36 | + |
| 37 | +## ProcessResult API |
| 38 | + |
| 39 | +Every `run()` call returns a `ProcessResult` with the following methods: |
| 40 | + |
| 41 | +| Method | Description | |
| 42 | +|---|---| |
| 43 | +| `output()` | Returns captured stdout as a string | |
| 44 | +| `error_output()` | Returns captured stderr as a string | |
| 45 | +| `exit_code()` | Returns the integer exit code | |
| 46 | +| `successful()` | `True` if exit code is `0` | |
| 47 | +| `failed()` | `True` if exit code is non-zero | |
| 48 | +| `throw()` | Raises `ProcessFailedException` if the process failed, otherwise returns `self` | |
| 49 | +| `throw_if(condition)` | Raises `ProcessFailedException` if `condition` is truthy | |
| 50 | +| `command()` | Returns the original command string | |
| 51 | + |
| 52 | +`throw()` is useful for asserting success in a chain: |
| 53 | + |
| 54 | +```python |
| 55 | +result = Process.run('python migrate.py').throw() |
| 56 | +# raises ProcessFailedException if migration fails |
| 57 | +``` |
| 58 | + |
| 59 | +## Fluent Builder Options |
| 60 | + |
| 61 | +Any method on the `Process` facade that is not `run()` or `start()` returns a `PendingProcess` builder. Chain options before calling `run()`: |
| 62 | + |
| 63 | +### `timeout(seconds)` |
| 64 | + |
| 65 | +Sets how long to wait before killing the process and raising `ProcessTimedOutException`. The default is 60 seconds. |
| 66 | + |
| 67 | +```python |
| 68 | +result = Process.timeout(10).run('bash slow_script.sh') |
| 69 | +``` |
| 70 | + |
| 71 | +### `forever()` |
| 72 | + |
| 73 | +Disables the timeout entirely. |
| 74 | + |
| 75 | +```python |
| 76 | +result = Process.forever().run('bash long_import.sh') |
| 77 | +``` |
| 78 | + |
| 79 | +### `quietly()` |
| 80 | + |
| 81 | +Discards all stdout and stderr output. Useful when you only care about the exit code. |
| 82 | + |
| 83 | +```python |
| 84 | +result = Process.quietly().run('npm install') |
| 85 | +``` |
| 86 | + |
| 87 | +### `tty()` |
| 88 | + |
| 89 | +Passes stdin, stdout, and stderr directly through to the terminal. Output is not captured. |
| 90 | + |
| 91 | +```python |
| 92 | +Process.tty().run('vim file.txt') |
| 93 | +``` |
| 94 | + |
| 95 | +### `env(variables)` |
| 96 | + |
| 97 | +Merges additional environment variables into the process environment. |
| 98 | + |
| 99 | +```python |
| 100 | +result = Process.env({'APP_ENV': 'production', 'DEBUG': '0'}).run('python app.py') |
| 101 | +``` |
| 102 | + |
| 103 | +### `path(directory)` |
| 104 | + |
| 105 | +Sets the working directory for the process. |
| 106 | + |
| 107 | +```python |
| 108 | +result = Process.path('/var/www/app').run('git pull origin main') |
| 109 | +``` |
| 110 | + |
| 111 | +### `input(data)` |
| 112 | + |
| 113 | +Pipes a string into the process's stdin. |
| 114 | + |
| 115 | +```python |
| 116 | +result = Process.input('yes\n').run('apt-get install -y some-package') |
| 117 | +``` |
| 118 | + |
| 119 | +### Combining Options |
| 120 | + |
| 121 | +Options chain together — each returns the same `PendingProcess`: |
| 122 | + |
| 123 | +```python |
| 124 | +result = ( |
| 125 | + Process |
| 126 | + .timeout(30) |
| 127 | + .path('/var/www/app') |
| 128 | + .env({'APP_ENV': 'staging'}) |
| 129 | + .quietly() |
| 130 | + .run('bash deploy.sh') |
| 131 | +) |
| 132 | +``` |
| 133 | + |
| 134 | +## Async Execution |
| 135 | + |
| 136 | +Use `Process.start()` to launch a process without blocking. It returns an `InvokedProcess` that streams output as lines arrive via a callback: |
| 137 | + |
| 138 | +```python |
| 139 | +import time |
| 140 | + |
| 141 | +def on_output(kind, line): |
| 142 | + # kind is 'stdout' or 'stderr' |
| 143 | + print(f"[{kind}] {line}", end='') |
| 144 | + |
| 145 | +invoked = Process.start('bash build.sh', callback=on_output) |
| 146 | + |
| 147 | +while invoked.running(): |
| 148 | + invoked.ensure_not_timed_out() |
| 149 | + time.sleep(0.5) |
| 150 | + |
| 151 | +result = invoked.wait() |
| 152 | +print("Exit code:", result.exit_code()) |
| 153 | +``` |
| 154 | + |
| 155 | +### `InvokedProcess` API |
| 156 | + |
| 157 | +| Method | Description | |
| 158 | +|---|---| |
| 159 | +| `running()` | Returns `True` while the process is still executing | |
| 160 | +| `pid()` | Returns the process ID | |
| 161 | +| `wait()` | Blocks until the process finishes and returns a `ProcessResult` | |
| 162 | +| `kill()` | Kills the process immediately | |
| 163 | +| `signal(sig)` | Sends a signal (e.g. `signal.SIGTERM`) to the process | |
| 164 | +| `ensure_not_timed_out()` | Raises `ProcessTimedOutException` if the configured timeout has elapsed | |
| 165 | + |
| 166 | +## Piping |
| 167 | + |
| 168 | +`Process.pipe()` builds a shell pipeline from multiple commands. Pass a callback that receives a `Pipe` builder and calls `.command()` for each stage: |
| 169 | + |
| 170 | +```python |
| 171 | +result = Process.pipe(lambda p: ( |
| 172 | + p.command('cat access.log'), |
| 173 | + p.command('grep "ERROR"'), |
| 174 | + p.command('wc -l'), |
| 175 | +)) |
| 176 | + |
| 177 | +print(result.output()) # count of ERROR lines |
| 178 | +``` |
| 179 | + |
| 180 | +The commands are joined with `|` and executed as a single shell command. All fluent builder options apply before calling `pipe()`: |
| 181 | + |
| 182 | +```python |
| 183 | +result = Process.path('/var/www/app').timeout(15).pipe(lambda p: ( |
| 184 | + p.command('find . -name "*.py"'), |
| 185 | + p.command('xargs wc -l'), |
| 186 | +)) |
| 187 | +``` |
| 188 | + |
| 189 | +## Pools (Concurrent Processes) |
| 190 | + |
| 191 | +`Process.pool()` runs multiple commands concurrently. Pass a callback that adds commands to a `Pool`, then call `.start()` to launch all of them and `.wait()` to collect results: |
| 192 | + |
| 193 | +```python |
| 194 | +pool = Process.pool(lambda p: ( |
| 195 | + p.command('bash job1.sh'), |
| 196 | + p.command('bash job2.sh'), |
| 197 | + p.command('bash job3.sh'), |
| 198 | +)).start() |
| 199 | + |
| 200 | +results = pool.wait() |
| 201 | + |
| 202 | +for i, result in enumerate(results): |
| 203 | + print(f"Job {i}: exit {result.exit_code()}") |
| 204 | + |
| 205 | +if results.successful(): |
| 206 | + print("All jobs completed successfully") |
| 207 | +``` |
| 208 | + |
| 209 | +Each process in a pool runs in parallel. `PoolResults` supports indexing, iteration, `len()`, `successful()`, and `failed()`. |
| 210 | + |
| 211 | +### Streaming Pool Output |
| 212 | + |
| 213 | +Pass a callback to `start()` to receive output lines as they arrive. The callback receives `(kind, line, index)` where `index` identifies which command produced the output: |
| 214 | + |
| 215 | +```python |
| 216 | +def on_output(kind, line, index): |
| 217 | + print(f"[job {index}] {line}", end='') |
| 218 | + |
| 219 | +pool = Process.pool(lambda p: ( |
| 220 | + p.command('bash job1.sh'), |
| 221 | + p.command('bash job2.sh'), |
| 222 | +)).start(callback=on_output) |
| 223 | + |
| 224 | +results = pool.wait() |
| 225 | +``` |
| 226 | + |
| 227 | +### Pool Working Directories |
| 228 | + |
| 229 | +Set a per-command working directory with `.path()` inside the pool callback: |
| 230 | + |
| 231 | +```python |
| 232 | +pool = Process.pool(lambda p: ( |
| 233 | + p.path('/srv/service-a').command('npm test'), |
| 234 | + p.path('/srv/service-b').command('npm test'), |
| 235 | +)).start() |
| 236 | + |
| 237 | +results = pool.wait() |
| 238 | +``` |
| 239 | + |
| 240 | +## Testing with Fakes |
| 241 | + |
| 242 | +Call `Process.fake()` at the start of a test to intercept all process calls. No real subprocesses are spawned. |
| 243 | + |
| 244 | +```python |
| 245 | +from fastapi_startkit.process import Process |
| 246 | + |
| 247 | +def test_deploy_script(): |
| 248 | + fake = Process.fake() |
| 249 | + |
| 250 | + # ... call code that internally runs Process.run('bash deploy.sh') |
| 251 | + |
| 252 | + fake.assert_ran('bash deploy.sh') |
| 253 | + |
| 254 | + Process.reset_fake() |
| 255 | +``` |
| 256 | + |
| 257 | +Always call `Process.reset_fake()` in teardown so subsequent tests run real processes. |
| 258 | + |
| 259 | +### Defining Fake Output |
| 260 | + |
| 261 | +Supply a dictionary mapping command strings to `FakeProcessDescription` objects built with `Process.describe()`: |
| 262 | + |
| 263 | +```python |
| 264 | +fake = Process.fake({ |
| 265 | + 'bash deploy.sh': Process.describe().output('Deployed successfully').exit_code(0), |
| 266 | + 'bash rollback.sh': Process.describe().error_output('Rollback failed').exit_code(1), |
| 267 | +}) |
| 268 | + |
| 269 | +result = Process.run('bash deploy.sh') |
| 270 | +assert result.output() == 'Deployed successfully' |
| 271 | +assert result.successful() |
| 272 | + |
| 273 | +result = Process.run('bash rollback.sh') |
| 274 | +assert result.failed() |
| 275 | +``` |
| 276 | + |
| 277 | +Use `'*'` as a wildcard to match any command not explicitly listed: |
| 278 | + |
| 279 | +```python |
| 280 | +fake = Process.fake({'*': Process.describe().exit_code(0)}) |
| 281 | +``` |
| 282 | + |
| 283 | +### `FakeProcessDescription` API |
| 284 | + |
| 285 | +| Method | Description | |
| 286 | +|---|---| |
| 287 | +| `output(text)` | Adds a stdout line to the fake result | |
| 288 | +| `error_output(text)` | Adds a stderr line to the fake result | |
| 289 | +| `exit_code(code)` | Sets the exit code (default `0`) | |
| 290 | + |
| 291 | +### Assertions |
| 292 | + |
| 293 | +`ProcessFake` (returned by `Process.fake()`) provides several assertion helpers: |
| 294 | + |
| 295 | +```python |
| 296 | +fake.assert_ran('bash deploy.sh') # assert command was run |
| 297 | +fake.assert_not_ran('bash rollback.sh') # assert command was NOT run |
| 298 | +fake.assert_ran_times('bash deploy.sh', 2) # assert exact run count |
| 299 | +fake.assert_nothing_ran() # assert no commands ran at all |
| 300 | +``` |
| 301 | + |
| 302 | +Pass a callable to `assert_ran()` for custom inspection: |
| 303 | + |
| 304 | +```python |
| 305 | +fake.assert_ran(lambda pending, result: result.exit_code() == 0) |
| 306 | +``` |
| 307 | + |
| 308 | +### Using a Fixture |
| 309 | + |
| 310 | +For pytest, a fixture makes teardown automatic: |
| 311 | + |
| 312 | +```python |
| 313 | +import pytest |
| 314 | +from fastapi_startkit.process import Process |
| 315 | + |
| 316 | +@pytest.fixture(autouse=True) |
| 317 | +def fake_process(): |
| 318 | + fake = Process.fake() |
| 319 | + yield fake |
| 320 | + Process.reset_fake() |
| 321 | + |
| 322 | +def test_my_command(fake_process): |
| 323 | + Process.run('echo hello') |
| 324 | + fake_process.assert_ran('echo hello') |
| 325 | +``` |
| 326 | + |
| 327 | +## Exception Handling |
| 328 | + |
| 329 | +### `ProcessFailedException` |
| 330 | + |
| 331 | +Raised by `result.throw()` when the process exits with a non-zero code. It exposes the original `ProcessResult` via `.result`: |
| 332 | + |
| 333 | +```python |
| 334 | +from fastapi_startkit.process.exception import ProcessFailedException |
| 335 | + |
| 336 | +try: |
| 337 | + Process.run('bash risky.sh').throw() |
| 338 | +except ProcessFailedException as e: |
| 339 | + print(e.result.error_output()) |
| 340 | + print("Exit code:", e.result.exit_code()) |
| 341 | +``` |
| 342 | + |
| 343 | +### `ProcessTimedOutException` |
| 344 | + |
| 345 | +Raised when a process exceeds its configured timeout. It exposes `.command` — the command string that timed out: |
| 346 | + |
| 347 | +```python |
| 348 | +from fastapi_startkit.process.exception import ProcessTimedOutException |
| 349 | + |
| 350 | +try: |
| 351 | + Process.timeout(5).run('sleep 60') |
| 352 | +except ProcessTimedOutException as e: |
| 353 | + print("Timed out:", e.command) |
| 354 | +``` |
0 commit comments