You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Article.md
+20-27Lines changed: 20 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
# Concurrent LSEG Data Platform API Calls with Python Asyncio and HTTPX
2
2
3
3
- Version: 1.0
4
-
- Last update: Mar 2026
4
+
- Last update: Apr 2026
5
5
- Environment: Python + JupyterLab + Data Platform Account
6
6
- Prerequisite: Data Platform access/entitlements
7
7
@@ -69,7 +69,7 @@ with httpx.Client(base_url='http://httpbin.org') as client:
69
69
print(r.status_code)
70
70
```
71
71
72
-
For asynchronous use, [`httpx.AsyncClient`](https://www.python-httpx.org/api/#asyncclient)works with [asyncio](https://docs.python.org/3/library/asyncio.html), [Trio](https://trio.readthedocs.io/en/stable/), and [AnyIO](https://anyio.readthedocs.io/en/stable/). I am demonstrating with asyncio in this project.:
72
+
For asynchronous use, [`httpx.AsyncClient`](https://www.python-httpx.org/api/#asyncclient)object to work with [asyncio](https://docs.python.org/3/library/asyncio.html), [Trio](https://trio.readthedocs.io/en/stable/), and [AnyIO](https://anyio.readthedocs.io/en/stable/). I am demonstrating with asyncio in this project.:
73
73
74
74
Example:
75
75
@@ -117,7 +117,7 @@ If the HTTP request takes 60 seconds, the program idles for those 60 seconds bef
117
117
118
118

119
119
120
-
On the other hand, **Asynchronous** code allows multiple tasks to run concurrently in a non-blocking manner. While one task is waiting for I/O (such as a network response), the event loop can hand control to another task (execute next line of codes) instead of sitting idle. The example below uses `asyncio.create_task()` to launch a fetch in the background and immediately continues to the next line — without waiting for the response:
120
+
On the other hand, **Asynchronous** code allows multiple tasks to run concurrently in a non-blocking manner. While one task is waiting for I/O (such as a network response), the event loop can hand control to another task (execute next line of codes) instead of sitting idle. The example below uses `asyncio.create_task()`method to launch a fetch in the background and immediately continues to the next line — without waiting for the response:
121
121
122
122
```python
123
123
import asyncio
@@ -174,7 +174,7 @@ The Historical Pricing endpoint rate limits information is available on the **Re
174
174
175
175
## Security Notes
176
176
177
-
- All examples use `verify=False` to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` or supply a proper CA bundle for production use.
177
+
- All examples use `verify=False`parameter to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` parameterparameter or supply a proper CA bundle for production use.
178
178
- Do not log or print access tokens in production applications.
179
179
180
180
## Code Walkthrough
@@ -187,10 +187,10 @@ The examples use the following Python libraries for demonstration in Jupyter Not
187
187
| --- | --- |
188
188
|`asyncio`| Python's built-in async event loop and concurrency primitives |
189
189
|`os`| Read environment variables |
190
-
|`time`| Wall-clock timing via `time.perf_counter()`|
190
+
|`time`| Wall-clock timing via `time.perf_counter()`statement |
191
191
|`httpx`| Async HTTP client |
192
192
|`IPython.display`| Render formatted Markdown output in the notebook |
193
-
|`dotenv`| Load credentials from `src/.env`|
193
+
|`dotenv`| Load credentials from `src/.env`file |
194
194
195
195
### Data Platform Authentication
196
196
@@ -201,7 +201,7 @@ The required credentials are:
201
201
-**Username**: The machine ID associated with your account.
202
202
-**Password**: The password for the machine ID.
203
203
-**Client ID (AppKey)**: A unique identifier for your app, generated via the App Key Generator. Keep it private.
204
-
-**Grant Type `password`**: Used for the initial authentication request with a username/password combination.
204
+
-**Grant Type `password`: Used for the initial authentication request with a username/password combination.
205
205
206
206
I strongly suggest reading the [Data Platform: Authorization - All about tokens](https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis/tutorials#authorization-all-about-tokens) tutorial for a deeper understanding of RDP authentication.
The `raise_for_status()` call handles any non-[HTTP 200 OK](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/200) response — such as 4xx or 5xx errors — by raising an exception that propagates back to the caller.
233
+
The `raise_for_status()`statement call handles any non-[HTTP 200 OK](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/200) response — such as 4xx or 5xx errors — by raising an exception that propagates back to the caller.
234
234
235
235
The `post_authentication_async` method sends a single HTTP POST request to the RDP authentication endpoint asynchronously and retrieves the access token for use in subsequent data requests. Nothing too exciting here — just a straightforward async HTTP call.
236
236
@@ -239,7 +239,7 @@ You maybe noticed that the code just define a function is like a normal HTTP POS
239
239
- define a function name with `async def` syntax
240
240
- call the `AsyncClient.post()` statement with `await` syntax
241
241
242
-
Moving on to the main code. The `async with` block opens a shared `httpx.AsyncClient` and guarantees its connection pool is closed cleanly when the block exits, whether it completes normally or raises an exception. Inside the block, `post_authentication_async()`*is awaited* to obtain the Bearer token before any data requests are made.
242
+
Moving on to the main code. The `async with` block opens a shared `httpx.AsyncClient`instance and guarantees its connection pool is closed cleanly when the block exits, whether it completes normally or raises an exception. Inside the block, `post_authentication_async()`method **is awaited** to obtain the Bearer token before any data requests are made.
243
243
244
244
```python
245
245
# Main Code
@@ -291,7 +291,7 @@ if __name__ == "__main__":
291
291
292
292
### Comparing to Synchronous Code
293
293
294
-
For a single HTTP request, the synchronous equivalent is *almost* identical. The only real differences are the absence of `async`/`await` and the use of `httpx.Client` instead of `httpx.AsyncClient`. The code runs line by line — each statement blocks and waits for the network response before moving on.
294
+
For a single HTTP request, the synchronous equivalent is *almost* identical. The only real differences are the absence of `async`/`await` and the use of `httpx.Client`instance instead of `httpx.AsyncClient`. The code runs line by line — each statement blocks and waits for the network response before moving on.
The `get_historical_interday_summaries_async` method sends a single HTTP GET request to the RDP endpoint asynchronously and retrieves historical data from the Data Platform. Like the `post_authentication_async` method above, it defines a method with `async def` syntax and calls `AsyncClient.get()` with `await` syntax.
405
405
406
-
You may have noticed the `semaphore` parameter. This [`asyncio.Semaphore`](https://docs.python.org/3/library/asyncio-sync.html#asyncio.Semaphore) object limits the number of concurrent coroutines that can access a specific resource or block of code simultaneously (throttle). To call `AsyncClient.get()` with a semaphore, use the following code:
406
+
You may have noticed the `semaphore` parameter. This [`asyncio.Semaphore`](https://docs.python.org/3/library/asyncio-sync.html#asyncio.Semaphore) object limits the number of concurrent coroutines that can access a specific resource or block of code simultaneously (throttle). To call `AsyncClient.get()`statement with a semaphore, use the following code:
407
407
408
408
```python
409
409
# Acquire the semaphore slot before sending the request so that at most
@@ -508,7 +508,7 @@ tasks_history = [
508
508
]
509
509
```
510
510
511
-
That brings us to [`asyncio.gather`](https://docs.python.org/3/library/asyncio-task.html#asyncio.gather), which runs all the coroutines in `tasks_history`*concurrently*. The `return_exceptions=True` parameter ensures that any exception from a coroutine is returned as a result value rather than being raised immediately — so a single failure does not cancel the remaining tasks. All coroutines tasks in the group are allowed to complete regardless of errors.
511
+
That brings us to [`asyncio.gather`](https://docs.python.org/3/library/asyncio-task.html#asyncio.gather) method, which runs all the coroutines in `tasks_history`**concurrently**. The `return_exceptions=True` parameter ensures that any exception from a coroutine is returned as a result value rather than being raised immediately — so a single failure does not cancel the remaining tasks. All coroutines tasks in the group are allowed to complete regardless of errors.
512
512
513
513
```python
514
514
# gather() runs all tasks concurrently. return_exceptions=True
@@ -532,7 +532,7 @@ for ric, result in result_data_dict.items():
532
532
print(f"Historical interday summaries for '{ric}': {result}\n\n")
533
533
```
534
534
535
-
For the `ASML.AS` RIC that my account does not have permission for, the Data Platform Historical Pricing endpoint returns *HTTP 200 (OK)* — but with a permission-denied payload in the response body. Because it is not a 4xx/5xx error, it passes the exception checks and ends up in `result_data_dict` as a regular result.
535
+
For the `ASML.AS` RIC that my account does not have permission for, the Data Platform Historical Pricing endpoint returns *HTTP 200 (OK)* — but with a permission-denied payload in the response body. Because it is not a 4xx/5xx error, it passes the exception checks and ends up in `result_data_dict`variable as a regular result.
536
536
537
537
```json
538
538
[
@@ -568,7 +568,7 @@ Finished all Historical Pricing requests:
568
568
569
569
### Comparing to Synchronous Code
570
570
571
-
The synchronous `get_historical_interday_summaries` method is *almost* identical to its async counterpart. The only real differences are the absence of `async`/`await` and the `semaphore` parameter, and the use of `httpx.Client` instead of `httpx.AsyncClient`. The code runs line by line — each call blocks and waits for the network response before moving on.
571
+
The synchronous `get_historical_interday_summaries` method is *almost* identical to its async counterpart. The only real differences are the absence of `async`/`await`syntax and the `semaphore` parameter, and the use of `httpx.Client` instance instead of `httpx.AsyncClient`. The code runs line by line — each call blocks and waits for the network response before moving on.
The synchronous main code is much simpler — it just loops over the `HISTORICAL_RICS` list and calls `get_historical_interday_summaries` for each RIC one at a time, waiting for each response before moving to the next.
591
+
The synchronous main code is much simpler — it just loops over the `HISTORICAL_RICS` list and calls `get_historical_interday_summaries`method for each RIC one at a time, waiting for each response before moving to the next.
592
592
593
593
```python
594
594
# Main code, use httpx.Client.
@@ -635,7 +635,7 @@ Finished all Historical Pricing requests:
635
635
636
636
**Note**: I intentionally waited for a period of time to minimize the impact of any potential network or endpoint data caching (if any) before comparing the results of the asynchronous and synchronous notebook examples.
637
637
638
-
That’s all I have to say about multiple RDP data request using asynchronous exceution model.
638
+
That’s all I have to say about multiple RDP data request using asynchronous execution model.
639
639
640
640
### Bonus: Working with the Historical Data
641
641
@@ -720,21 +720,14 @@ That said, async code does introduce more moving parts — coroutines, semaphore
720
720
721
721
## Next Step
722
722
723
-
The `asyncio.gather` is not the only option that Python Asyncio offers for developers. If you prefer to manual manage and run coroutine, the [`asyncio.create_task`](https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task)migth suite your need. This low-level method schedules a coroutine to run as a background task immediately, without waiting.
723
+
The `asyncio.gather` is not the only option that Python Asyncio offers for developers. If you prefer to manual manage and run coroutine, the [`asyncio.create_task`](https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task)might suite your need. This low-level method schedules a coroutine to run as a background task immediately, without waiting.
724
724
725
-
Introduced in Python version 3.11, [`asyncio.TaskGroup`](https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup) provides a modern, cleaner, and fail-safe way for structured concurrency I/O bound operations. It is suitable for I/O piplines that any failure should abort the whole operation.
725
+
Introduced in Python version 3.11, [`asyncio.TaskGroup`](https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup) provides a modern, cleaner, and fail-safe way for structured concurrency I/O bound operations. It is suitable for I/O pipelines that any failure should abort the whole operation.
726
726
727
-
There are much more Data Platform endpoints, HTTPX and Asyncio features that you can explore to find the things that suite your technical and bussiness needs. For further reading, I recommend the following resources:
727
+
There are much more Data Platform endpoints, HTTPX and Asyncio features that you can explore to find the things that suite your technical and business needs. For further reading, I recommend the following resources:
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -130,7 +130,7 @@ Both notebooks cover the same RDP API workflow — authentication, fetching hist
130
130
131
131
## Security Notes
132
132
133
-
- All examples use `verify=False` to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` or supply a proper CA bundle for production use.
133
+
- All examples use `verify=False`parameter to disable TLS certificate verification. This is intended for local/dev environments only (e.g. where a TLS-inspecting proxy such as ZScaler is in use). Remove `verify=False` parameter or supply a proper CA bundle for production use.
134
134
- Do not log or print access tokens in production applications.
0 commit comments