Magnet 🧲

Magnet is a high-performance backend system built with Rust (server/) and controlled via a Python SDK (client/). The entire infrastructure is containerized and orchestrated via Docker Compose, including an APISIX gateway, PostgreSQL database, and Redis cache.

---

📋 Prerequisites

• Docker installed.
• Docker Compose installed.

---

🛠️ 1. Environment Setup

Before starting the stack, you must define the environment variables required by the containers.

1. Copy the provided example environment file:

cp .env.example .env

2. Open .env and adjust the passwords, ports, and directory paths to match your environment.

---

🚀 2. Running the Stack

To build the Rust backend and start all services in the background, run:

docker compose up -d --build

Note: The --build flag ensures that any changes you made to the Rust code in ./server are compiled into the new container.

Check the Status

Because Magnet depends on Postgres and Redis, it will wait until they are fully healthy before starting. You can watch the boot process by viewing the logs:

docker compose logs -f magnet

To see the health status of all containers:

docker compose ps

---

🛑 3. Stopping and Managing Data

Stop the Server (Keep Data)

To shut down the containers but keep your database and drive files safe:

docker compose down

💥 Nuke the Server (Destroy All Data)

If you want to completely wipe the database, wipe the Redis cache, and delete all uploaded Drive files to start fresh:

docker compose down -v

(The -v flag deletes all persistent Docker volumes attached to this project).

---

🌐 4. Direct API Access (Without the Python SDK)

If you prefer to bypass the Python SDK, you can interact directly with the Magnet HTTP API via the APISIX gateway (default port 8000).

Authentication: The API uses cookie-based sessions. You must save the cookie from your login request (-c cookies.txt) and pass it to subsequent requests (-b cookies.txt).

General & Auth Routes

# Public Ping (No auth required)
curl -X GET http://localhost:8000/ping

# Login (Saves session to cookies.txt)
curl -X POST http://localhost:8000/login \
     -H "Content-Type: application/json" \
     -d '{"username": "user123", "password": "password123"}' \
     -c cookies.txt

# Whoami (Check current logged-in user)
curl -X GET http://localhost:8000/whoami -b cookies.txt

# Logout (Invalidates session)
curl -X GET http://localhost:8000/logout -b cookies.txt

# Signup (Requires a code generated by an Admin)
curl -X POST http://localhost:8000/signup \
     -H "Content-Type: application/json" \
     -d '{"code": "ADMIN_CODE", "username": "new_user", "password": "password123"}'

Admin Routes (Requires Admin Cookie)

# Admin Ping (Verifies Admin RBAC)
curl -X GET http://localhost:8000/admin/ping -b cookies.txt

# Generate a new User Signup Code (Valid for 48 hours)
curl -X GET http://localhost:8000/admin/signup_code -b cookies.txt

Drive Routes (Requires Cookie)

⚠️ Important Note on File Transfers: Because the API communicates strictly via JSON, raw binary file uploads and downloads are not supported directly. All file contents must be Base64 encoded.

# Get Root Folder Contents
curl -X GET http://localhost:8000/drive -b cookies.txt

# Get Specific Folder Contents OR Download a File
curl -X GET http://localhost:8000/drive/<item_id> -b cookies.txt

# Upload a File to Root
# Note: "content" must be a Base64 encoded string.
curl -X POST http://localhost:8000/drive \
     -b cookies.txt -H "Content-Type: application/json" \
     -d '{"name": "photo.jpg", "content": "iVBORw0KGgo..."}'

# Create an Empty Folder in Root
# Note: Omit the "content" field or set to null.
curl -X POST http://localhost:8000/drive \
     -b cookies.txt -H "Content-Type: application/json" \
     -d '{"name": "My Documents"}'

# Upload/Create inside a specific parent folder
curl -X POST http://localhost:8000/drive/<parent_folder_id> \
     -b cookies.txt -H "Content-Type: application/json" \
     -d '{"name": "secret.txt", "content": "SGVsbG8="}'

# Delete a File or Folder (Cascades to children)
curl -X DELETE http://localhost:8000/drive/<item_id> -b cookies.txt

---

🏗️ Architecture & Configuration Notes

Port Reference

All backend services are shielded from the host machine to ensure traffic only flows through the API gateway.

Service	Local Port	Description
APISIX	${APISIX_PORT} (e.g., 8000)	The main entry point. All client requests should hit this port.
Postgres	Hidden	Database storage. Shielded and only communicates internally.
Magnet	Hidden	The Rust backend. Shielded by APISIX and only communicates internally.
Redis	Hidden	Ephemeral cache and session storage.

APISIX Gateway Features

To optimize performance and protect the backend infrastructure, the APISIX gateway is configured with the following built-in plugins:

• limit-conn: Protects the server from being overwhelmed by limiting the number of concurrent connections per client/IP.

• gzip: Automatically compresses outgoing HTTP responses to reduce bandwidth usage and improve client-side load times.

Enabling Redis Persistence (RDB/AOF)

By default, Redis is configured strictly as a volatile cache with a 256MB memory limit (--save "" and --appendonly no). It will evict the oldest keys if it gets full, and all data is lost if the container restarts.

If you want Redis data (like active sessions or signup codes) to survive container restarts, you must edit the docker-compose.yml file:

1. Add a volume map under the redis service to persist the data to disk:

volumes:
  - redis_data:/data

2. Update the command section to enable your preferred persistence strategy:

• To enable RDB (Snapshotting): Change --save "" to --save 60 1 (This saves a snapshot every 60 seconds if at least 1 key changed).
• To enable AOF (Append Only File): Change --appendonly no to --appendonly yes. (More durable, logs every write operation).

3. Add redis_data: to the top-level volumes: block at the bottom of the compose file.