Skip to content

Memory Spikes #168

@alterEgo123

Description

@alterEgo123

Hello, I'm running Overpass API in Kubernetes with an EFS volume using the official image wiktorn/overpass-api:latest.
I use these environment variables:

env:
- name: OVERPASS_META
  value: "yes"
- name: OVERPASS_MODE
  value: "clone"
- name: OVERPASS_PLANET_URL
  value: "https://planet.openstreetmap.org/planet/planet-latest.osm.bz2"
- name: OVERPASS_DIFF_URL
  value: "https://planet.openstreetmap.org/replication/minute/"
- name: OVERPASS_RULES_LOAD
  value: "10"
- name: OVERPASS_ALLOW_DUPLICATE_QUERIES
  value: "yes"

And I set a 25GB, 2.5 cpu limits, and 500GB of disk space.

I use this locustfile to test it, and even with a single user, the server reaches the memory limit quickly, restarting in ungraceful pod restart.


from locust import HttpUser, task, between
import random
import base64

USERNAME = "admin"
PASSWORD = "admin"


QUERIES = [
    # small bbox (most common)
    """
    [out:json][timeout:25];
    node["amenity"="restaurant"](36.85,10.15,36.86,10.16);
    out;
    """,

    # medium bbox
    """
    [out:json][timeout:25];
    node["amenity"="restaurant"](36.84,10.14,36.87,10.18);
    out;
    """,

    # heavier query (less frequent)
    """
    [out:json][timeout:25];
    node["shop"](36.8,10.1,36.9,10.2);
    out;
    """
]

class OverpassUser(HttpUser):
    wait_time = between(2, 10)  # realistic think time

    def on_start(self):
        credentials = f"{USERNAME}:{PASSWORD}"
        token = base64.b64encode(credentials.encode()).decode()
        self.headers = {
            "Authorization": f"Basic {token}",
            "Content-Type": "application/x-www-form-urlencoded"
        }

    @task
    def query_overpass(self):
        query = random.choice(QUERIES)
        self.client.post(
            "/api/interpreter",
            data={"data": query},
            headers=self.headers
        )

This is what the resource consumption looks like:

Image

The pod restart causes this issue runtime error: open64: 2 No such file or directory /db/db//osm3s_osm_base Unix_Socket::7 but I solve it with chmod og+rx /db && exec /docker-entrypoint.sh /app/docker-entrypoint.sh in the entrypoint.

Is there a way to prevent the Overpass API server from reaching the limit and crashing?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions