You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -17,74 +17,281 @@ This project demonstrates how to use [`httpx`](https://www.python-httpx.org/) to
17
17
18
18
**Note**: A basic knowledge of Python [built-in asyncio](https://docs.python.org/3/library/asyncio.html) library is required to understand example codes.
[LSEG Data Platform](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis) (RDP APIs, also known as Delivery Platform in LSEG Real-Time) provides simple web based API access to a broad range of LSEG content.
23
23
24
-
Interactive notebook version of the synchronous workflow. Each logical step is a separate cell with a markdown explanation above it, making it easy to run and inspect results incrementally.
24
+
RDP APIs give developers seamless and holistic access to all of the LSEG content such as Historical Pricing, Environmental Social and Governance (ESG), News, Research, etc, and commingled with their content, enriching, integrating, and distributing the data through a single interface, delivered wherever they need it. The RDP APIs delivery mechanisms are the following:
25
+
* Request - Response: RESTful web service (HTTP GET, POST, PUT or DELETE)
26
+
* Alert: delivery is a mechanism to receive asynchronous updates (alerts) to a subscription.
27
+
* Bulks: deliver substantial payloads, like the end-of-day pricing data for the whole venue.
28
+
* Streaming: deliver real-time delivery of messages.
25
29
26
-
Demonstrates:
27
-
-`POST /auth/oauth2/v1/token` — OAuth 2.0 Password Grant authentication
28
-
-`GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data with corporate-action adjustments for 10 RICs
29
-
-`POST /auth/oauth2/v1/revoke` — session token revocation using HTTP Basic Auth
30
-
- Shared `httpx.Client` inside a `with` block for clean connection-pool teardown
31
-
- Wall-clock timing across the full workflow
30
+
This example project is focusing on the Request-Response: RESTful web service delivery method only.
-[RDP APIs: Introduction to the Request-Response API](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/tutorials#introduction-to-the-request-response-api) page.
Interactive notebook version of the async concurrent workflow using `httpx.AsyncClient`and `asyncio.gather()`. Jupyter's native top-level `await` support means no `asyncio.run()` wrapper is needed.
39
+
[HTTPX](https://www.python-httpx.org/) is a full featured modern HTTP client for Python 3. It provides a set of synchronous and modern asynchronous APIs with [HTTP/2](https://httpwg.org/specs/rfc7540.html) supported. It is largely [compatible with the Requests library](https://www.python-httpx.org/compatibility/), so any Python developers can migrate their existing [Requests](https://requests.readthedocs.io/en/latest/) library code to the HTTPX easily.
44
40
45
-
Demonstrates:
46
-
-`POST /auth/oauth2/v1/token` — async OAuth 2.0 Password Grant authentication
47
-
-`GET /data/historical-pricing/v1/views/interday-summaries/{ric}` — daily OHLCV data fetched concurrently for 10 RICs
48
-
-`asyncio.Semaphore` — caps concurrent in-flight requests (default: 3) to respect server rate limits
49
-
-`asyncio.gather(return_exceptions=True)` — all RIC coroutines run simultaneously; one failure does not cancel the rest
r = httpx.get('https://httpbin.org/get', params=params)
47
+
r.raise_for_status()
48
+
print(r.json())
61
49
62
-
## Included Scripts
50
+
# HTTP Post
51
+
data = {'integer': 123, 'boolean': True, 'list': ['a', 'b', 'c']}
52
+
r = httpx.post('https://httpbin.org/post', json=data)
53
+
r.raise_for_status()
54
+
print(r.json())
55
+
```
63
56
64
-
### `src/example_async_gather.py` — Async with `asyncio.gather()`and `Semaphore`
57
+
For synchronous use, HTTPX also provides [`httpx.Client`](https://www.python-httpx.org/advanced/clients/) object which is the equivalent of `requests.Session()`— it maintains a shared connection pool across multiple requests:
65
58
66
-
Async script that fires all RIC requests concurrently via `asyncio.gather()`, with an `asyncio.Semaphore` to cap the number of in-flight requests and avoid hitting server rate limits.
### `src/example_client.py` — Synchronous with shared client
64
+
with httpx.Client(base_url='http://httpbin.org') as client:
65
+
r = client.get('/get')
66
+
r.raise_for_status()
67
+
print(r.status_code)
68
+
```
76
69
77
-
Synchronous (blocking) script using a single shared `httpx.Client` instance for connection pooling and consistent configuration across all requests.
70
+
For asynchronous use, [`httpx.AsyncClient`](https://www.python-httpx.org/api/#asyncclient) works with [asyncio](https://docs.python.org/3/library/asyncio.html), [Trio](https://trio.readthedocs.io/en/stable/), and [AnyIO](https://anyio.readthedocs.io/en/stable/). I am demonstrating with asyncio in this project.:
## What are Synchronous and Asynchronous Execution Models?
87
+
88
+
**Synchronous** code runs tasks one at a time in a strict sequence — each task must finish before the next one starts. The application pauses and waits at every blocking call. For example, the `httpx.get()` function call below (equivalent to `requests.get()`) blocks the entire program until the HTTP response arrives:
89
+
90
+
```python
91
+
import httpx
92
+
93
+
deffetch(url):
94
+
"""Fetch the content of the URL synchronously."""
95
+
r = httpx.get(url, verify=False)
96
+
print("Fetched:", url, "status:", r.status_code)
97
+
return r.text
98
+
99
+
defmain():
100
+
""" Main function."""
101
+
fetch("https://example.org")
102
+
print("This line prints ONLY after the request is done!")
If the HTTP request takes 60 seconds, the program idles for those 60 seconds before executing the next line. For a single request this is fine, but it becomes a bottleneck when you need to fetch data for many symbols or endpoints.
111
+
112
+
113
+
114
+
On the other hand, **Asynchronous** code allows multiple tasks to run concurrently in a non-blocking manner. While one task is waiting for I/O (such as a network response), the event loop can hand control to another task (execute next line of codes) instead of sitting idle. The example below uses `asyncio.create_task()` to launch a fetch in the background and immediately continues to the next line — without waiting for the response:
115
+
116
+
```python
117
+
import asyncio
118
+
import httpx
119
+
120
+
asyncdeffetch(url):
121
+
"""Fetch the content of the URL asynchronously."""
122
+
asyncwith httpx.AsyncClient(verify=False) as client:
123
+
r =await client.get(url)
124
+
print("Fetched:", url, "status:", r.status_code)
125
+
return r.text
126
+
127
+
asyncdefmain():
128
+
""" Main function."""
129
+
asyncio.create_task(fetch("https://example.org"))
130
+
print("Task launched and not awaited!")
131
+
# Sleep to allow the fetch task to complete before the program exits.
The real payoff of async comes when you have **many requests to make**. With `asyncio.gather()`, you can fire all of them concurrently so the total wall-clock time is roughly that of the single slowest response — instead of the sum of all response times. That is exactly the pattern used in `example_async_gather.py` and `async_call_nb.ipynb` examples for fetching multiple RICs.
142
+
143
+
## Throttling and Rate Limits
144
+
145
+
The Data Platform API request limits (throttles) to effectively manage and protect its service and ensure fair usage across the non-streaming content.
146
+
147
+
An application would receive an error from the API call if an application reached or exceeds a limit (especially with the Asynchronous HTTP calls). You required to make some necessary adjustments to rectify the interaction with the API and retry the respective API call.
148
+
149
+
Two different server errors on API request limits are:
150
+
151
+
|**HTTP Status**|**Detail**|
152
+
| --- | --- |
153
+
|**429**|**Error Message**: too many attempts |
154
+
||**Description**: A per account limit where the number of requests per second is limited for each account accessing the platform. If this limit is reached, applications will receive a standard HTTP error (HTTP 429 too many requests). |
155
+
||**Suggestion**: Please reduce the number of requests per second and retry. |
156
+
157
+
Please find more detail regarding the Data Platform HTTP error status messages from the [RDP API General Guidelines](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/documentation) document page.
158
+
159
+
The Historical Pricing endpoint rate limits information is available on the **Reference** tab of the [Data Platform API Playground](https://apidocs.refinitiv.com/Apps/ApiDocs) page. The current rate limits (**As of Mar 2026**) is as follows:
- Environment validation with a `_require_env()` helper that fails fast on missing credentials
86
163
87
164
## Security Notes
88
165
89
166
- All examples use `verify=False` to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` or supply a proper CA bundle for production use.
90
-
- Do not log or print access tokens in production applications.
167
+
- Do not log or print access tokens in production applications.
168
+
169
+
## Code Walkthrough
170
+
171
+
Now we come to the code walkthrough. This article focuses primarily on the asynchronous code. Synchronous equivalents are shown in select places for comparison.
172
+
173
+
The examples use the following Python libraries for demonstration in Jupyter Notebook files.
174
+
175
+
| Library | Purpose |
176
+
| --- | --- |
177
+
|`asyncio`| Python's built-in async event loop and concurrency primitives |
178
+
|`os`| Read environment variables |
179
+
|`time`| Wall-clock timing via `time.perf_counter()`|
180
+
|`httpx`| Async HTTP client |
181
+
|`IPython.display`| Render formatted Markdown output in the notebook |
182
+
|`dotenv`| Load credentials from `src/.env`|
183
+
184
+
### Data Platform Authentication
185
+
186
+
Let's start with the authentication. The first step of any application workflow is to log in to the RDP Auth Service.
187
+
188
+
The required credentials are:
189
+
190
+
-**Username**: The machine ID associated with your account.
191
+
-**Password**: The password for the machine ID.
192
+
-**Client ID (AppKey)**: A unique identifier for your app, generated via the App Key Generator. Keep it private.
193
+
-**Grant Type `password`**: Used for the initial authentication request with a username/password combination.
194
+
195
+
I strongly suggest reading the [Data Platform: Authorization - All about tokens](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/tutorials#authorization-all-about-tokens) tutorial for a deeper understanding of RDP authentication.
196
+
197
+
The authentication function uses Python's [`async`](https://docs.python.org/3/reference/compound_stmts.html#async-def)/[`await`](https://docs.python.org/3/reference/expressions.html#await) syntax so the HTTP request can be suspended and resumed when the network response arrives — without blocking other tasks. The `client` parameter is a shared `httpx.AsyncClient` instance passed in from the caller, so the same underlying TCP connection pool is reused across all requests rather than opening a new connection each time.
The `raise_for_status()` call handles any non-[HTTP 200 OK](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/200) response — such as 4xx or 5xx errors — by raising an exception that propagates back to the caller.
223
+
224
+
Moving on to the main code. The `async with` block opens a shared `httpx.AsyncClient` and guarantees its connection pool is closed cleanly when the block exits, whether it completes normally or raises an exception. Inside the block, `post_authentication_async()`*is awaited* to obtain the Bearer token before any data requests are made.
225
+
226
+
```python
227
+
# Main Code
228
+
asyncwith httpx.AsyncClient(
229
+
verify=False,
230
+
base_url=base_url,
231
+
timeout=10.0,
232
+
follow_redirects=True,
233
+
) as client:
234
+
# --- Authentication (must complete before any data requests) ---
# --- Exception handlers ordered from most-specific to least-specific ---
242
+
except httpx.HTTPStatusError as e:
243
+
# Server returned a 4xx or 5xx status code.
244
+
print(f"HTTP error during request: {e.request.url}{e.response.status_code} - {e.response.text}")
245
+
except httpx.TimeoutException as e:
246
+
# Request exceeded the configured timeout (must precede RequestError
247
+
# because TimeoutException is a subclass of RequestError).
248
+
print(f"Timeout error: {e}")
249
+
except httpx.RequestError as e:
250
+
# Network-level failure: DNS, connection refused, SSL error, etc.
251
+
print(f"Network error: {e}")
252
+
exceptExceptionas e:
253
+
# Catch-all for unexpected errors (e.g. JSON decode, assertion).
254
+
print(f"Unexpected error: {e}")
255
+
```
256
+
257
+
### Where is asyncio.run(main())?
258
+
259
+
You might wonder why the main code does not call `asyncio.run(main())`. The reason is that Jupyter natively supports top-level `await`, so no `asyncio.run()` wrapper is needed.
260
+
261
+
### Comparing to Synchronous Code
262
+
263
+
For a single HTTP request, the synchronous equivalent is *almost* identical. The only real differences are the absence of `async`/`await` and the use of `httpx.Client` instead of `httpx.AsyncClient`. The code runs line by line — each statement blocks and waits for the network response before moving on.
0 commit comments