Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/workflows/rspec.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: RSpec
on:
pull_request:
types: [synchronize, opened, reopened]

permissions:
contents: read

concurrency:
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}

jobs:
rspec:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit

- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Set up Ruby
uses: ruby/setup-ruby@84684c07c1965536eb4802c8daf1a77968df0cb1 # v1
with:
ruby-version: .ruby-version

- name: Ruby gem cache
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: ${{ github.workspace }}/vendor/bundle
key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
restore-keys: |
${{ runner.os }}-gems-

- name: Bundle Setup
run: bundle config path ${{ github.workspace }}/vendor/bundle

- name: Bundle Install
run: bundle install --jobs 4 --retry 3

- name: Run RSpec
run: bundle exec rspec
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Configure dynamic authentication to LLM providers using HashiCorp vault
title: Configure dynamic authentication to LLM providers using HashiCorp Vault
permalink: /ai-gateway/v1/how-to/configure-hashicorp-vault-as-a-vault-for-llm-providers/
description: "Use HashiCorp Vault to securely store and reference API keys for OpenAI, Mistral, and other LLM providers in {{site.ai_gateway}}."
content_type: how_to
Expand Down Expand Up @@ -46,7 +46,7 @@ tags:
tldr:
q: How can I access HashiCorp Vault secrets in {{site.base_gateway}}?
a: |
Store secrets using `vault kv put secret/openai key="OPENAI_API_KEY"` to HashiCorp Vault. Then configure a Vault entity in {{site.base_gateway}} with the host, token, and mount path. Inside the Gateway container, run `kong vault get {vault://hashicorp-vault/openai/key}` to confirm access. Next Use the `{vault://...}` syntax in a plugin field to [dynamically authenticate to LLM providers](/ai-gateway/v1/how-to/use-semantic-load-balancing-with-dynamic-vault-authentication/) such as OpenAI and Mistral.
Store secrets using `vault kv put secret/openai key="Bearer OPENAI_API_KEY"` to HashiCorp Vault. Then configure a Vault entity in {{site.base_gateway}} with the host, token, and mount path. Inside the Gateway container, run `kong vault get {vault://hashicorp-vault/openai/key}` to confirm access. Next, use the `{vault://...}` syntax in a plugin field to [dynamically authenticate to LLM providers](/ai-gateway/v1/how-to/use-semantic-load-balancing-with-dynamic-vault-authentication/) such as OpenAI and Mistral.

tools:
- deck
Expand Down Expand Up @@ -87,14 +87,17 @@ major_version:

## Create secrets in HashiCorp Vault

LLM providers such as OpenAI and {{ site.mistral }} expect the `Authorization` header to be `Bearer <api-key>`. Store this full header value, including the `Bearer ` prefix, as the secret. This way, any plugin
field that references the secret directly with `{vault://...}` sends a correctly formatted header, with no other configuration needed.

Replace the placeholder with your OpenAI API key and run:

{% validation custom-command %}
command: |
curl -X POST http://localhost:8200/v1/secret/data/openai \
-H "X-Vault-Token: $VAULT_TOKEN" \
-H "Content-Type: application/json" \
--data '{"data": {"key": "'$DECK_OPENAI_API_KEY'" }}'
--data '{"data": {"key": "Bearer '$DECK_OPENAI_API_KEY'" }}'
expected:
return_code: 0
render_output: false
Expand All @@ -107,19 +110,19 @@ command: |
curl -X POST http://localhost:8200/v1/secret/data/mistral \
-H "X-Vault-Token: $VAULT_TOKEN" \
-H "Content-Type: application/json" \
--data '{"data": {"key": "'$DECK_MISTRAL_API_KEY'" }}'
--data '{"data": {"key": "Bearer '$DECK_MISTRAL_API_KEY'" }}'
expected:
return_code: 0
render_output: false
{% endvalidation %}

Both secrets will be stored under their respective paths (`secret/openai` and `secret/mistral`) in the key field.
Both secrets will be stored under their respective paths (`secret/openai` and `secret/mistral`) in the key field, with the `Bearer ` prefix included.

## Create decK environment variables

We'll use decK environment variables for the `host` and `token` in the {{site.base_gateway}} Vault configuration. This is because these values typically vary between environments.

In this tutorial, we're using `host.docker.internal` as our host instead of the `localhost` variable that HashiCorp Vault uses by default. This is because if you used the quick-start script {{site.base_gateway}} is running in a Docker container and uses a different `localhost`.
In this tutorial, we're using `host.docker.internal` as our host instead of the `localhost` variable that HashiCorp Vault uses by default. This is because if you used the quick-start script, {{site.base_gateway}} is running in a Docker container and uses a different `localhost`.

Because we are running HashiCorp Vault in dev mode, we are using `root` for our `token` value.

Expand Down Expand Up @@ -168,14 +171,13 @@ To validate that the secret was stored correctly in HashiCorp Vault, you can cal

{% validation vault-secret %}
secret: '{vault://hashicorp-vault/mistral/key}'
value: $DECK_MISTRAL_API_KEY
value: Bearer $DECK_MISTRAL_API_KEY
{% endvalidation %}


{% validation vault-secret %}
secret: '{vault://hashicorp-vault/openai/key}'
value: $DECK_OPENAI_API_KEY
value: Bearer $DECK_OPENAI_API_KEY
{% endvalidation %}


If the vault was configured correctly, this command should return the value of the secrets for OpenAI and {{ site.mistral }}. You can use `{vault://hashicorp-vault/openai/key}` and `{vault://hashicorp-vault/mistral/key}` to reference the secret in any referenceable field.
If the vault was configured correctly, this command should return the value of the secrets for OpenAI and {{ site.mistral }}, prefixed with `Bearer `. You can use `{vault://hashicorp-vault/openai/key}` and `{vault://hashicorp-vault/mistral/key}` to reference the secret in any referenceable field. Because the stored value already includes the `Bearer ` prefix, use the bare `{vault://...}` reference as the field's value. Kong only resolves a vault reference when it's the field's entire value, not when it's embedded alongside other text.
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ entities:
description: CATCHALL
variables:
redis_host:
value: $DECK_REDIS_HOST
value: $REDIS_HOST
{% endentity_examples %}


Expand All @@ -155,12 +155,14 @@ These prompts are routed to **OpenAI GPT-3.5-Turbo**, since it performs well on
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: How can I build a REST API using Flask?
status_code: 200
{% endvalidation %}
<!-- vale on -->

Expand All @@ -169,12 +171,14 @@ You can also try a question regarding debugging JavaScript code:
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: How can you effectively debug asynchronous code in JavaScript to identify where a Promise or callback might be failing?
status_code: 200
{% endvalidation %}
<!-- vale on -->

Expand All @@ -185,12 +189,14 @@ These prompts should match the **OpenAI GPT-4o** target, which is designated for
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: What is the derivative of sin(x)?
status_code: 200
{% endvalidation %}
<!-- vale on -->

Expand All @@ -199,12 +205,14 @@ You can also try asking a question related to theorems:
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: Explain me Gödel`s incompleteness theorem.
status_code: 200
{% endvalidation %}
<!-- vale on -->

Expand All @@ -215,12 +223,14 @@ These general-purpose or unmatched prompts are routed to **{{ site.mistral }} Ti
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: What is Wulfila Bible?
status_code: 200
{% endvalidation %}
<!-- vale on -->

Expand All @@ -229,11 +239,13 @@ You can also try another general question:
<!-- vale off -->
{% validation request-check %}
url: /anything
method: POST
headers:
- 'Content-Type: application/json'
body:
messages:
- role: user
content: Who was Edward Gibbon and what he is famous for?
status_code: 200
{% endvalidation %}
<!-- vale on -->
5 changes: 3 additions & 2 deletions app/_how-tos/gateway/configure-oidc-with-kong-oauth2.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,8 +181,9 @@ extract_body:
- name: 'access_token'
variable: ACCESS_TOKEN
status_code: 200
capture: ACCESS_TOKEN
jq: ".access_token"
capture:
- variable: ACCESS_TOKEN
jq: ".access_token"
{% endvalidation%}
<!-- vale on -->

Expand Down
4 changes: 0 additions & 4 deletions app/_includes/how-tos/validations/qwen/snippet.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
```sh
{% if include.config.base_url -%}
export OPENAI_BASE_URL={{include.config.base_url}}

{% endif -%}
{{include.config.base_command}}
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
{%- elsif capture_size > 1 -%}
_response=$({{ curl_cmd }})
{%- else -%}
{{ curl_cmd }}{% if count > 1 %}
{{ curl_cmd }}{% if count > 1 %} \
; done{% endif -%}
{%- endif %}
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
</div>
{% endif %}

This request returns a `{{config.status_code}}` error with the message `{{config.message}}`.
{% include how-tos/validations/unauthorized-check/message.md %}
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@

{%- capture on_prem_snippet -%}{% include how-tos/validations/unauthorized-check/snippet.md url=config.on_prem_url headers=config.headers %}{%- endcapture -%}

{% include works_on_wrapper.md on_prem_content=on_prem_snippet konnect_content=konnect_snippet %}
{% include works_on_wrapper.md on_prem_content=on_prem_snippet konnect_content=konnect_snippet %}

{% include how-tos/validations/unauthorized-check/message.md %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{%- if config.status_code and config.message -%}
This request returns a `{{ config.status_code }}` error with the message `{{ config.message }}`.
{%- elsif config.status_code -%}
This request returns a `{{ config.status_code }}` error.
{%- elsif config.message -%}
This request returns an error with the message `{{ config.message }}`.
{%- endif -%}
Loading
Loading