Skip to content

Commit 0ee0e03

Browse files
committed
updated README file with more detail.
1 parent 3c89fad commit 0ee0e03

4 files changed

Lines changed: 104 additions & 95 deletions

File tree

Article.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Data Platform APIs HTTP REST Application using Httpx
2+
3+
- Version: 1.0
4+
- Last update: Mar 2026
5+
- Environment: Python + JupyterLab + Data Platform Account
6+
- Prerequisite: Data Platform access/entitlements
7+
8+
## Overview
9+
10+
The [Requests](https://requests.readthedocs.io/en/latest/) library is widely regarded as *the de facto* standard HTTP client for Python applications. Many Python developers first learn REST API calls through Requests — including through our [Data Platform APIs Tutorials](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/tutorials) (or you can try RDP HTTP operations with the [built-in http.client](https://docs.python.org/3/library/http.client.html) if you enjoy a challenge.).
11+
12+
That said, there are other Python HTTP libraries worth considering — [HTTPX](https://www.python-httpx.org/), [Aiohttp](https://docs.aiohttp.org/en/stable/), [Urllib3](https://urllib3.readthedocs.io/en/stable/), [Grequests](https://pypi.org/project/grequests/), [PycURL](http://pycurl.io/docs/latest/index.html), and more — each offering different trade-offs in performance and features that may better suit your requirements.
13+
14+
I was drawn to HTTPX because it provides a **requests-compatible API** while also supporting **asynchronous operations** out of the box. That combination made migrating from Requests to HTTPX straightforward, with the added benefit of async support when needed.
15+
16+
This project demonstrates how to use [`httpx`](https://www.python-httpx.org/) to authenticate and retrieve data from [LSEG Data Platform APIs](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis) via HTTP endpoints — covering both synchronous and asynchronous patterns for comparison.
17+
18+
**Note**: A basic knowledge of Python [built-in asyncio](https://docs.python.org/3/library/asyncio.html) library is required to understand example codes.
19+
20+
## Included Notebook
21+
22+
### `src/sync_call_nb.ipynb` — Synchronous, step-by-step Jupyter notebook
23+
24+
Interactive notebook version of the synchronous workflow. Each logical step is a separate cell with a markdown explanation above it, making it easy to run and inspect results incrementally.
25+
26+
Demonstrates:
27+
- `POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
28+
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data with corporate-action adjustments for 10 RICs
29+
- `POST /auth/oauth2/v1/revoke` — session token revocation using HTTP Basic Auth
30+
- Shared `httpx.Client` inside a `with` block for clean connection-pool teardown
31+
- Wall-clock timing across the full workflow
32+
33+
Notebook structure:
34+
1. Imports
35+
2. Constants (endpoint paths, RIC list)
36+
3. Credentials loaded from `src/.env`
37+
4. Helper functions (`post_authentication`, `post_auth_revoke`, `get_historical_interday_summaries`)
38+
5. Main execution block — authenticate, fetch data sequentially, revoke token
39+
6. Elapsed time output
40+
41+
### `src/async_call_nb.ipynb` — Async, concurrent Jupyter notebook (`asyncio.gather`)
42+
43+
Interactive notebook version of the async concurrent workflow using `httpx.AsyncClient` and `asyncio.gather()`. Jupyter's native top-level `await` support means no `asyncio.run()` wrapper is needed.
44+
45+
Demonstrates:
46+
- `POST /auth/oauth2/v1/token` — async OAuth 2.0 Password Grant authentication
47+
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data fetched concurrently for 10 RICs
48+
- `asyncio.Semaphore` — caps concurrent in-flight requests (default: 3) to respect server rate limits
49+
- `asyncio.gather(return_exceptions=True)` — all RIC coroutines run simultaneously; one failure does not cancel the rest
50+
- Per-result error inspection: `httpx.HTTPStatusError`, `httpx.RequestError`, generic `Exception`
51+
- `async with httpx.AsyncClient` — shared connection pool, closed cleanly on exit
52+
- Wall-clock timing across the full workflow
53+
54+
Notebook structure:
55+
1. Imports
56+
2. Constants (endpoint paths, RIC list)
57+
3. Credentials loaded from `src/.env`
58+
4. Helper functions (`post_authentication`, `post_auth_revoke`, `get_historical_interday_summaries`)
59+
5. Main execution block — authenticate, gather concurrent RIC fetches, per-result error handling
60+
6. Elapsed time output
61+
62+
## Included Scripts
63+
64+
### `src/example_async_gather.py` — Async with `asyncio.gather()` and `Semaphore`
65+
66+
Async script that fires all RIC requests concurrently via `asyncio.gather()`, with an `asyncio.Semaphore` to cap the number of in-flight requests and avoid hitting server rate limits.
67+
68+
Demonstrates:
69+
- `POST /auth/oauth2/v1/token` — async authentication
70+
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — concurrent fetches for 10 RICs
71+
- `asyncio.Semaphore` — limits concurrent requests (default: 3)
72+
- `return_exceptions=True` — prevents one failure from cancelling the rest; each result is inspected individually
73+
- Per-result error handling: `httpx.HTTPStatusError`, `httpx.RequestError`, generic `Exception`
74+
75+
### `src/example_client.py` — Synchronous with shared client
76+
77+
Synchronous (blocking) script using a single shared `httpx.Client` instance for connection pooling and consistent configuration across all requests.
78+
79+
Demonstrates:
80+
- `POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
81+
- `GET /data/pricing/chains/v1/` — chain constituent lookup
82+
- `POST /data/historical-pricing/v1/views/events` — historical trade events for multiple RICs (commented out, ready to enable)
83+
- Refresh token flow (`grant_type=refresh_token`) — commented out, ready to enable
84+
- `POST /auth/oauth2/v1/revoke` — session revocation — commented out, ready to enable
85+
- Environment validation with a `_require_env()` helper that fails fast on missing credentials
86+
87+
## Security Notes
88+
89+
- All examples use `verify=False` to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` or supply a proper CA bundle for production use.
90+
- Do not log or print access tokens in production applications.

README.md

Lines changed: 14 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@ if __name__ == "__main__":
115115

116116
If the HTTP request takes 60 seconds, the program idles for those 60 seconds before executing the next line. For a single request this is fine, but it becomes a bottleneck when you need to fetch data for many symbols or endpoints.
117117

118+
![synchronous](images/synchronous_simple.png)
119+
118120
On the other hand, **Asynchronous** code allows multiple tasks to run concurrently in a non-blocking manner. While one task is waiting for I/O (such as a network response), the event loop can hand control to another task (execute next line of codes) instead of sitting idle. The example below uses `asyncio.create_task()` to launch a fetch in the background and immediately continues to the next line — without waiting for the response:
119121

120122
```python
@@ -140,113 +142,36 @@ if __name__ == "__main__":
140142

141143
![asynchronous code result](images/02_httpx_async.png)
142144

145+
![asynchronous](images/asynchronous_simple.png)
146+
143147
The real payoff of async comes when you have **many requests to make**. With `asyncio.gather()`, you can fire all of them concurrently so the total wall-clock time is roughly that of the single slowest response — instead of the sum of all response times. That is exactly the pattern used in `example_async_gather.py` and `async_call_nb.ipynb` examples for fetching multiple RICs.
144148

145149
## Prerequisites
146150

147-
- Python 3.11+ (required for `asyncio.TaskGroup` and `except*`)
151+
- Python 3.11+
148152
- LSEG Data Platform credentials with Historical Pricing permission:
149153
- Machine ID
150154
- Password
151155
- AppKey
152156

153-
If you do not have access yet, contact your LSEG representative or account manager.
157+
Please your LSEG representative or account manager for the Data Platform Access
154158

155159
## Project Structure
156160

157161
```
158162
├── requirements.txt # Pinned dependencies
163+
├── README.md # Project README file
164+
├── LICENSE.md # Project LICENSE file
165+
├── Article.md # Project Implementation detail (article) file
159166
src/
160167
├── .env.example # Environment variable template
161-
├── sync_call_nb.ipynb # Jupyter notebook — synchronous, shared httpx.Client
168+
├── sync_call_nb.ipynb # Jupyter notebook — synchronous, shared httpx.Client
162169
├── async_call_nb.ipynb # Jupyter notebook — async, asyncio.gather() with Semaphore
163-
├── example_sync_httpx.py # Synchronous — direct httpx module calls (no shared client)
164-
├── example_client.py # Synchronous — shared httpx.Client
165-
└── example_async_gather.py # Async — asyncio.gather() with Semaphore
170+
├── example_client.py # Synchronous — shared httpx.Client console app
171+
└── example_async_gather.py # Async — asyncio.gather() with Semaphore console app
166172
```
167173

168-
## Included Notebook
169-
170-
### `src/sync_call_nb.ipynb` — Synchronous, step-by-step Jupyter notebook
171-
172-
Interactive notebook version of the synchronous workflow. Each logical step is a separate cell with a markdown explanation above it, making it easy to run and inspect results incrementally.
173-
174-
Demonstrates:
175-
- `POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
176-
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data with corporate-action adjustments for 10 RICs
177-
- `POST /auth/oauth2/v1/revoke` — session token revocation using HTTP Basic Auth
178-
- Shared `httpx.Client` inside a `with` block for clean connection-pool teardown
179-
- Wall-clock timing across the full workflow
180-
181-
Notebook structure:
182-
1. Imports
183-
2. Constants (endpoint paths, RIC list)
184-
3. Credentials loaded from `src/.env`
185-
4. Helper functions (`post_authentication`, `post_auth_revoke`, `get_historical_interday_summaries`)
186-
5. Main execution block — authenticate, fetch data sequentially, revoke token
187-
6. Elapsed time output
188-
189-
### `src/async_call_nb.ipynb` — Async, concurrent Jupyter notebook (`asyncio.gather`)
190-
191-
Interactive notebook version of the async concurrent workflow using `httpx.AsyncClient` and `asyncio.gather()`. Jupyter's native top-level `await` support means no `asyncio.run()` wrapper is needed.
192-
193-
Demonstrates:
194-
- `POST /auth/oauth2/v1/token` — async OAuth 2.0 Password Grant authentication
195-
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data fetched concurrently for 10 RICs
196-
- `asyncio.Semaphore` — caps concurrent in-flight requests (default: 3) to respect server rate limits
197-
- `asyncio.gather(return_exceptions=True)` — all RIC coroutines run simultaneously; one failure does not cancel the rest
198-
- Per-result error inspection: `httpx.HTTPStatusError`, `httpx.RequestError`, generic `Exception`
199-
- `async with httpx.AsyncClient` — shared connection pool, closed cleanly on exit
200-
- Wall-clock timing across the full workflow
201-
202-
Notebook structure:
203-
1. Imports
204-
2. Constants (endpoint paths, RIC list)
205-
3. Credentials loaded from `src/.env`
206-
4. Helper functions (`post_authentication`, `post_auth_revoke`, `get_historical_interday_summaries`)
207-
5. Main execution block — authenticate, gather concurrent RIC fetches, per-result error handling
208-
6. Elapsed time output
209-
210-
## Included Scripts
211-
212-
### `src/example_async_gather.py` — Async with `asyncio.gather()` and `Semaphore`
213-
214-
Async script that fires all RIC requests concurrently via `asyncio.gather()`, with an `asyncio.Semaphore` to cap the number of in-flight requests and avoid hitting server rate limits.
215-
216-
Demonstrates:
217-
- `POST /auth/oauth2/v1/token` — async authentication
218-
- `GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — concurrent fetches for 10 RICs
219-
- `asyncio.Semaphore` — limits concurrent requests (default: 3)
220-
- `return_exceptions=True` — prevents one failure from cancelling the rest; each result is inspected individually
221-
- Per-result error handling: `httpx.HTTPStatusError`, `httpx.RequestError`, generic `Exception`
222-
223-
### `src/example_client.py` — Synchronous with shared client
224-
225-
Synchronous (blocking) script using a single shared `httpx.Client` instance for connection pooling and consistent configuration across all requests.
226-
227-
Demonstrates:
228-
- `POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
229-
- `GET /data/pricing/chains/v1/` — chain constituent lookup
230-
- `POST /data/historical-pricing/v1/views/events` — historical trade events for multiple RICs (commented out, ready to enable)
231-
- Refresh token flow (`grant_type=refresh_token`) — commented out, ready to enable
232-
- `POST /auth/oauth2/v1/revoke` — session revocation — commented out, ready to enable
233-
- Environment validation with a `_require_env()` helper that fails fast on missing credentials
234-
235-
### `src/example_sync_httpx.py` — Synchronous, direct `httpx` calls
236-
237-
Simplest synchronous example. Each function calls `httpx.get()` / `httpx.post()` directly — no shared client or connection pool. Good as a minimal reference or quick script.
238-
239-
Demonstrates:
240-
- `POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
241-
- `GET /data/pricing/chains/v1/` — chain constituent lookup for a single RIC
242-
- `POST /data/historical-pricing/v1/views/events` — historical trade events for multiple RICs
243-
- Refresh token flow (`grant_type=refresh_token`)
244-
- `POST /auth/oauth2/v1/revoke` — session revocation using HTTP Basic Auth
245-
- Per-call `verify=False` passed directly to each `httpx` function
246-
247-
248-
249-
## Setup
174+
## Project Setup
250175

251176
1. Create and activate a virtual environment.
252177

@@ -261,7 +186,7 @@ python -m venv .venv
261186
pip install -r requirements.txt
262187
```
263188

264-
3. Create your environment file by creating `src/.env` with the following content:
189+
3. Create your environment file by creating `src/.env` with the following content (see `src/.env.example` file):
265190

266191
```dotenv
267192
RDP_BASE_URL=https://api.refinitiv.com
@@ -279,12 +204,6 @@ jupyter lab src/simple_call_nb.ipynb
279204
280205
# Jupyter notebook (async — asyncio.gather)
281206
jupyter lab src/async_call_nb.ipynb
282-
283-
# Synchronous
284-
python .\src\example_client.py
285-
286-
# Async — concurrent via asyncio.gather()
287-
python .\src\example_async_gather.py
288207
```
289208

290209
Each script prints the authenticated request URLs and JSON responses. Timing is printed on exit for the async scripts.

images/asynchronous_simple.png

88.5 KB
Loading

images/synchronous_simple.png

85.7 KB
Loading

0 commit comments

Comments
 (0)