diff --git a/content/about/images/market-growth.png b/content/about/images/market-growth.png deleted file mode 100644 index 75172f78..00000000 Binary files a/content/about/images/market-growth.png and /dev/null differ diff --git a/content/about/images/vision-values.png b/content/about/images/vision-values.png deleted file mode 100644 index 04ebf537..00000000 Binary files a/content/about/images/vision-values.png and /dev/null differ diff --git a/content/about/index.md b/content/about/index.md deleted file mode 100644 index d65b64fe..00000000 --- a/content/about/index.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: "About Us" -description: "RustFS is a cutting-edge open-source infrastructure solution for distributed object storage. It is a project driven by a global community of talented…" ---- - -**RustFS is dedicated to leading the industry in data security and cost-efficiency.** - -RustFS is a cutting-edge open-source infrastructure solution for distributed object storage. It is a project driven by a global community of talented storage architects and open-source enthusiasts. Our core values are: Integrity, Focus, and Simplicity. - -Our vision is to provide secure, multilingual, and cost-effective distributed object storage solutions to the world. - -RustFS continuously ensures data security and reduces storage costs across diverse sectors, including artificial intelligence, big data, media streaming, cloud computing, encrypted storage, industrial IoT, cloud-native applications, and data backup. - -We are committed to building the future of global storage infrastructure. - -## Our Vision and Values - -### Vision - -Secure and cost-effective data solutions for all humanity - -### Values - -Integrity, Focus, Simplicity - -![Vision and Values](./images/vision-values.png) - -## RustFS Provides Secure and Reliable Distributed Storage Solutions Globally - -![Global Data Storage Market Growth](./images/market-growth.png) - -> According to Fortune Business Insights, the global data storage market is expected to grow from $218.33 billion in 2024 to $774 billion in 2032, with a compound annual growth rate of 17.1% during this period. - -## Contact Us - -For general inquiries, investment opportunities, or strategic partnerships: - -📧 **Email**: diff --git a/content/features/ai/images/ai-performance.png b/content/features/ai/images/ai-performance.png deleted file mode 100644 index f116256a..00000000 Binary files a/content/features/ai/images/ai-performance.png and /dev/null differ diff --git a/content/features/ai/images/multi-engine-1.svg b/content/features/ai/images/multi-engine-1.svg deleted file mode 100644 index a7d3e2d5..00000000 --- a/content/features/ai/images/multi-engine-1.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/content/features/ai/images/multi-engine-2.svg b/content/features/ai/images/multi-engine-2.svg deleted file mode 100644 index 57f0c9d2..00000000 --- a/content/features/ai/images/multi-engine-2.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/content/features/ai/index.md b/content/features/ai/index.md deleted file mode 100644 index a3fd450a..00000000 --- a/content/features/ai/index.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: "AI Revolution Powered by GPUs and High-Performance Object Storage" -description: "The High-Performance Object Storage for AI" ---- - -The High-Performance Object Storage for AI - -## AI Storage Delivers Performance at Scale - -![AI Performance](images/ai-performance.png) - -RustFS accelerates AI/ML workloads by leveraging its distributed architecture and object storage capabilities. During model training, RustFS's distributed setup allows parallel data access and I/O operations, reducing latency and speeding up training times. For model inference, RustFS's high-throughput data access ensures rapid retrieval and deployment of data stored for AI models, enabling predictions with minimal latency. Crucially, RustFS scales linearly from 100 TB to 100 PB and beyond. This optimizes end-to-end AI workflows, enhancing model development and serving, resulting in more efficient AI workloads and faster response times for applications. - -## Core of the AI Ecosystem - -RustFS sets the standard for S3-compatible object storage for AI workloads. This ubiquity means the entire AI/ML ecosystem integrates with RustFS. RustFS is widely supported across the AI/ML ecosystem. - -![AI Ecosystem Support](images/multi-engine-1.svg) - -![AI Ecosystem Support](images/multi-engine-2.svg) - -## Scale Required for Training and Inference - -Enterprises continuously collect and store AI data that applications and large language models can use to retrain models for improved accuracy. RustFS's scalability allows organizations to scale their storage capacity on demand, ensuring smooth data access and high-performance computing essential for the success of AI/ML applications. - -## Fault-Tolerant AI Storage - -RustFS allows organizations to store large amounts of data, including training datasets, models, and intermediate results in a fault-tolerant manner. This resilience is crucial for ML and AI storage as it ensures data remains accessible even in cases of hardware failures or system crashes. With RustFS's distributed architecture and data replication capabilities, AI/ML workflows can run seamlessly and continue to provide accurate insights and predictions, enhancing the overall reliability of AI-driven applications. - -## High-Availability Storage for AI Workloads - -RustFS's active-active replication capabilities support simultaneous access across multiple geographically distributed clusters. This is important for AI/ML as it enhances data availability and performance. AI/ML workloads often involve globally collaborative teams and require low-latency access to data stored for AI model training and inference - ensuring data can be accessed from the nearest cluster location, reducing latency. Additionally, it provides failover capabilities, ensuring uninterrupted access to data even during cluster failures, which is crucial for maintaining reliability and continuity of AI/ML processes. - -## Storage Solutions for Large Language Models - -RustFS can seamlessly integrate with Large Language Models (LLMs) as a reliable and scalable storage solution for the massive data required by such models. Organizations can use RustFS storage for pre-trained LLMs, fine-tuning datasets, and other artifacts. This ensures easy access and retrieval during model training and model serving. RustFS's distributed nature allows parallel data access, reducing data transfer bottlenecks and accelerating LLM training and inference, enabling data scientists and developers to fully leverage the potential of large language models for natural language processing tasks. - -## Contextual Storage for Retrieval Augmented Generation - -RustFS can serve as a high-performance object storage backend for AI models for Retrieval Augmented Generation (RAG) and data. In RAG setups, RustFS can store corpora used to create domain-specific responses from Large Language Models (LLMs). AI-powered applications can access the corpus and the result is more contextually relevant and accurate responses for natural language generation tasks, improving the overall quality of generated content. - -## Cloud as Operating Model - Starting with S3 - -RustFS adheres to cloud operating models - containerization, orchestration, automation, APIs, and S3 compatibility. This allows cross-cloud and cross-storage type data storage and access by providing a unified interface for storing and accessing data. Since most AI/ML frameworks and applications are designed to work with S3 APIs, strong S3 compatibility is crucial. RustFS's S3 compatibility is continuously exercised by a broad developer and application community. This compatibility ensures AI workloads can access and leverage data stored in RustFS object storage regardless of the underlying cloud infrastructure, facilitating flexible and agnostic data management and processing approaches across different cloud environments. - -## Edge AI Storage - -At the edge, network latency, data loss, and software bloat degrade performance. RustFS is a fast object storage server with a binary under 100 MB that can be deployed on any hardware. Additionally, bucket event notifications make it easy to build systems that can immediately run inference on newly introduced data. Whether it's airborne object detection on high-altitude drones or traffic trajectory prediction for autonomous vehicles, RustFS's AI storage enables mission-critical applications to store and use their data in a fast, fault-tolerant, and simple manner. - -## Lifecycle Management for ML/AI Workloads - -Modern AI/ML workloads require sophisticated lifecycle management. RustFS's lifecycle management capabilities automate data management tasks, optimizing storage efficiency and reducing operational overhead. With lifecycle policies, organizations can automatically move infrequently accessed AI data to lower-cost storage tiers, freeing up valuable resources for more critical and active workloads. These capabilities ensure AI/ML practitioners can focus on model training and development while RustFS intelligently manages data, enhancing overall workflow performance and cost-effectiveness. Additionally, lifecycle management layers ensure AI/ML datasets comply with regulatory requirements by enforcing retention and deletion policy requirements. - -## Object Retention for AI/ML Workflows - -Few workloads depend more on when things happened than AI/ML. This is addressed through advanced object retention capabilities that ensure the integrity and compliance of stored data over time. By implementing retention policies, RustFS can help organizations maintain data consistency for AI/ML models and datasets, preventing accidental or unauthorized deletions or modifications. This feature is crucial for data governance, regulatory compliance, and reproducibility of AI/ML experiments, as it guarantees that critical data remains accessible and immutable for specific durations, supporting precise model training and analysis. - -## Data Protection for Core AI Datasets - -RustFS protects data through a number of different capabilities. It supports erasure coding and site replication, ensuring data redundancy and fault tolerance to prevent hardware failures or data corruption. RustFS also allows data encryption at rest and in transit, protecting data from unauthorized access. Additionally, RustFS's support for Identity and Access Management (IAM) enables organizations to control access to their data stored for AI workloads, ensuring only authorized users or applications can access and modify data. These comprehensive data protection mechanisms provided by RustFS help maintain the integrity, availability, and confidentiality of AI datasets throughout their lifecycle. diff --git a/content/features/aliyun/images/sec1-1.png b/content/features/aliyun/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/aliyun/images/sec1-1.png and /dev/null differ diff --git a/content/features/aliyun/index.md b/content/features/aliyun/index.md deleted file mode 100644 index cd184001..00000000 --- a/content/features/aliyun/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for Alibaba Cloud Kubernetes Service (ACK)" -description: "RustFS provides high-performance object storage for Alibaba Cloud ACK with hybrid cloud capabilities and enterprise features." ---- - -Alibaba Cloud Container Service for Kubernetes (ACK) is a managed service for running Kubernetes on Alibaba Cloud without needing to install, operate, and maintain your own Kubernetes control plane or nodes. - -Three reasons customers run RustFS on ACK: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on ACK gives you control over the software stack and the flexibility to avoid cloud lock-in. - -RustFS runs on all major Kubernetes platforms and deploys on ACK with the official Helm chart, making it easier to run your own large-scale, multi-tenant object storage as a service. Unlike a managed cloud storage service, RustFS lets applications scale across multi-cloud and hybrid cloud infrastructure without expensive software rewrites or proprietary integrations. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on ACK, you need: - -- An ACK cluster (ACK managed or ACK dedicated) with worker nodes sized for your storage workload -- A block-storage `StorageClass` backed by the Alibaba Cloud CSI driver (for example, ESSD cloud disks) for RustFS persistent volumes -- A load balancer for external access, typically an Alibaba Cloud CLB/NLB provisioned through a `LoadBalancer` service or an ingress controller -- `kubectl` and Helm configured against your cluster - -## Deploy RustFS on ACK - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering, external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/aws-elastic/images/sec1-1.png b/content/features/aws-elastic/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/aws-elastic/images/sec1-1.png and /dev/null differ diff --git a/content/features/aws-elastic/index.md b/content/features/aws-elastic/index.md deleted file mode 100644 index e3a51ae8..00000000 --- a/content/features/aws-elastic/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for Amazon Elastic Kubernetes Service (EKS)" -description: "RustFS provides high-performance object storage for Amazon EKS with enterprise-grade features and multi-cloud capabilities." ---- - -Amazon Elastic Kubernetes Service (EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane or nodes. - -Three reasons customers run RustFS on EKS: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on EKS gives you control over the software stack and the flexibility to avoid cloud lock-in. - -RustFS deploys on EKS with the official Helm chart and works with the EKS toolchain, making it easier to operate your own large-scale, multi-tenant object storage as a service. Because RustFS is S3-compatible from the start, applications built for the S3 API run against RustFS on EKS without changes — and can later move to any other cloud or on-premises environment. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on EKS, you need: - -- An EKS cluster with worker nodes sized for your storage workload -- A block-storage `StorageClass` backed by the Amazon EBS CSI driver (for example, `gp3`) for RustFS persistent volumes -- A load balancer for external access, typically an NLB/ALB provisioned through the AWS Load Balancer Controller or an ingress controller such as NGINX -- `kubectl` and Helm configured against your cluster - -## Deploy RustFS on EKS - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering (including transition to cold tiers such as S3 Glacier), external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/baremetal/images/.gitkeep b/content/features/baremetal/images/.gitkeep deleted file mode 100644 index 41f3ca0d..00000000 --- a/content/features/baremetal/images/.gitkeep +++ /dev/null @@ -1,26 +0,0 @@ -# 图片文件夹 - -此目录用于存放 baremetal 相关的图片文件。 - -## 如何获取图片 - -如果您需要添加原网站的图片到文档中，可以： - -1. 访问 https://rustfs.com/features/baremetal/ -2. 使用浏览器开发者工具查看页面源码 -3. 找到图片的实际 URL 地址 -4. 下载图片并保存到此目录 -5. 在 markdown 文档中添加图片引用 - -## 原始图片位置 - -根据网页内容，以下位置应该有图片： -- WORM 功能部分 -- 身份管理部分 -- 连续复制部分 -- 全球联合部分 - -## 注意 - -当前文档已移除了图片引用，改为纯文字版本，确保文档可以正常显示。 -如需要添加图片，请按上述步骤获取后再更新 markdown 文件。 diff --git a/content/features/baremetal/index.md b/content/features/baremetal/index.md deleted file mode 100644 index e67413a8..00000000 --- a/content/features/baremetal/index.md +++ /dev/null @@ -1,233 +0,0 @@ ---- -title: "Bare Metal and Virtualized Deployment" -description: "Open source, S3-compatible, and enterprise-hardened." ---- - -Open source, S3-compatible, and enterprise-hardened. - -RustFS is a high-performance distributed object storage system. It is software-defined, runs on industry-standard hardware, and is 100% open source (Apache V2.0). - -RustFS is designed for private/hybrid cloud object storage. Its single-layer architecture achieves all necessary functionality without compromising performance. RustFS is high-performance, scalable, and lightweight. - -RustFS supports traditional use cases (secondary storage, disaster recovery, archiving) and modern workloads (machine learning, analytics, cloud-native applications). - -## Core Features - -### Erasure Coding - -RustFS uses inline erasure coding to protect data while providing high performance. RustFS uses Reed-Solomon codes to stripe objects into data and parity blocks with user-configurable redundancy levels. - -With maximum parity of N/2, RustFS can ensure uninterrupted read and write operations using only ((N/2)+1) operational drives. For example, in a 12-drive setup (6 data + 6 parity), RustFS can reliably write new objects or rebuild existing objects with only 7 drives remaining. - -```mermaid -flowchart TD - EX(["export-xl"]) - EX --> D1[("Disk1")] - EX --> D2[("Disk2")] - EX --> D3[("Disk3")] - EX --> D4[("Disk4")] - D1 --> B1["MyBucket"] - D2 --> B2["MyBucket"] - D3 --> B3["MyBucket"] - D4 --> B4["MyBucket"] - B1 --> O1["MyObject"] - B2 --> O2["MyObject"] - B3 --> O3["MyObject"] - B4 --> O4["MyObject"] - O1 --> F1["xl.json part.1"] - O2 --> F2["xl.json part.1"] - O3 --> F3["xl.json part.1"] - O4 --> F4["xl.json part.1"] - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class EX accent - class D1,D2,D3,D4 store - class B1,B2,B3,B4 server - class O1,O2,O3,O4 svc - class F1,F2,F3,F4 muted -``` - -### Bitrot Protection - -Bitrot (silent data corruption) is a serious problem for disk drives. RustFS uses HighwayHash to detect and repair corrupted data. By calculating hashes on READ and verifying them on WRITE, it ensures end-to-end integrity. The implementation achieves hash speeds exceeding 10 GB/s on a single core. - -```mermaid -flowchart TD - H["Object · erasure-coded across 16 drives"] - S["Tolerates 8 disk failures"] - H --> S - subgraph DATA["Data Block"] - direction LR - d1["1 checksum"] - d2["2 checksum"] - d3["3 checksum"] - de["..."] - d8["8 checksum"] - d1 -.- d2 -.- d3 -.- de -.- d8 - end - subgraph PARITY["Parity Block"] - direction LR - p1["1P checksum"] - p2["2P checksum"] - p3["3P checksum"] - pe["..."] - p8["8P checksum"] - p1 -.- p2 -.- p3 -.- pe -.- p8 - end - S --> DATA - S --> PARITY - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - class H svc - class d1,d2,d3,d8,de muted - class p1,p2,p3,p8,pe server -``` - -### Server-Side Encryption - -RustFS supports multiple server-side encryption schemes to protect data at rest. RustFS ensures confidentiality, integrity, and authenticity with negligible performance overhead. Supported algorithms include AES-256-GCM, ChaCha20-Poly1305, and AES-CBC. - -Encrypted objects are tamper-proof using AEAD server-side encryption. RustFS is compatible with common key management solutions (e.g., HashiCorp Vault) and uses KMS to support SSE-S3. - -If a client requests SSE-S3 or auto-encryption is enabled, the RustFS server encrypts each object with a unique object key protected by a master key managed by KMS. - -```mermaid -flowchart LR - D1["Data · SSE-S3"] - D2["Data · SSE-C"] - R(["RustFS"]) - KMS[("KMS")] - B1["My Bucket"] - B2["My Bucket"] - O1["Object"] - O2["Object"] - O3["Object"] - O4["Object"] - O5["Object"] - O6["Object"] - D1 --> R - D2 --> R - R --> KMS - R --> B1 - R --> B2 - B1 --> O1 - B1 --> O2 - B1 --> O3 - B2 --> O4 - B2 --> O5 - B2 --> O6 - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class D1,D2 server - class R accent - class KMS store - class B1,B2 svc - class O1,O2,O3,O4,O5,O6 svc -``` - -### WORM (Write Once Read Many) - -#### Identity Management - -RustFS supports advanced identity management standards and integrates with OpenID Connect providers and major external IDP vendors. Access is centralized, and passwords are temporary and rotated. Access policies are fine-grained and highly configurable. - -#### Continuous Replication - -The challenge with traditional replication methods is that they don't scale effectively beyond a few hundred TiB. That said, everyone needs a replication strategy to support disaster recovery, and that strategy needs to span geographic locations, data centers, and clouds. - -RustFS's continuous replication is designed for large-scale, cross-data center deployments. By leveraging Lambda compute notifications and object metadata, it can efficiently and quickly calculate increments. Lambda notifications ensure immediate propagation of changes rather than traditional batch modes. - -Continuous replication means that in case of failure, data loss remains minimal - even in the face of highly dynamic datasets. Finally, like other RustFS features, continuous replication is multi-vendor, meaning your backup location can be anywhere from NAS to public cloud. - -#### Global Federation - -Modern enterprise data is everywhere. RustFS allows these disparate instances to be combined to form a unified global namespace. Specifically, any number of RustFS servers can be combined into a distributed mode set, and multiple distributed mode sets can be combined into a RustFS server federation. Each RustFS server federation provides unified administration and namespace. - -RustFS federated servers support unlimited numbers of distributed mode sets. The impact of this approach is that object storage can scale massively for large enterprises with geographically dispersed locations while retaining the ability to accommodate various applications (Splunk, Teradata, Spark, Hive, Presto, TensorFlow, H20) from a single console. - -#### Multi-Cloud Gateway - -All enterprises are adopting multi-cloud strategies. This includes private clouds as well. Therefore, your bare metal virtualized containers and public cloud services (including non-S3 providers like Google, Microsoft, and Alibaba) must look the same. While modern applications are highly portable, the data supporting these applications is not. - -Providing access to this data regardless of where it resides is the primary challenge RustFS solves. RustFS runs on bare metal, network-attached storage, and every public cloud. More importantly, RustFS ensures that from an application and management perspective, the view of that data looks exactly the same through the Amazon S3 API. - -RustFS can go further, making your existing storage infrastructure Amazon S3 compatible. The implications are profound. Now organizations can truly unify their data infrastructure - from file to block, all data appears as objects accessible through the Amazon S3 API without migration. - -When WORM is enabled, RustFS disables all APIs that might alter object data and metadata. This means data becomes tamper-proof once written. This has practical applications in many different regulatory requirements. - -```mermaid -flowchart LR - APP[Applications] --> S3API(["S3 API"]) - - subgraph DIST["Distributed RustFS"] - direction TB - subgraph N1["Node 1"] - direction LR - S3a[S3] - subgraph OL1["Object Layer"] - direction TB - C1[Cache] - K1[Compression] - E1[Encryption] - B1["Erasure Code · Bitrot"] - end - SL1["Storage Layer"] - J1[("JBOD / FS disks")] - S3a -->|Object API| OL1 - OL1 -->|Storage API| SL1 - SL1 <--> J1 - end - subgraph N2["Node 2"] - direction LR - S3b[S3] - subgraph OL2["Object Layer"] - direction TB - C2[Cache] - K2[Compression] - E2[Encryption] - B2["Erasure Code · Bitrot"] - end - SL2["Storage Layer"] - J2[("JBOD / FS disks")] - S3b -->|Object API| OL2 - OL2 -->|Storage API| SL2 - SL2 <--> J2 - end - NN["Node n ..."] - N1 <-->|Internal RESTful API| N2 - N2 <-->|Internal RESTful API| NN - end - - S3API --> N1 - S3API --> N2 - S3API --> NN - - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class APP,NN muted - class S3API accent - class S3a,S3b,SL1,SL2 server - class C1,K1,E1,B1,C2,K2,E2,B2 svc - class J1,J2 store -``` - -## System Architecture - -RustFS is designed to be cloud-native and can run as lightweight containers managed by external orchestration services like Kubernetes. The application is compiled into a single static binary (~100 MB) that efficiently uses CPU and memory resources even under high load. As a result, you can co-host a large number of tenants on shared hardware. - -RustFS runs on commodity servers with locally attached drives (JBOD/JBOF). All servers in the cluster are functionally equal (completely symmetric architecture). There are no name nodes or metadata servers. - -RustFS writes data and metadata together as objects, requiring no metadata database. Additionally, RustFS performs all functions (erasure coding, bitrot checking, encryption) as inline, strictly consistent operations. The result is that RustFS has extraordinary resilience. - -Each RustFS cluster is a collection of distributed RustFS servers with one process per node. RustFS runs as a single process in user space and uses lightweight coroutines to achieve high concurrency. Drives are grouped into erasure sets (16 drives per set by default) and objects are placed on these sets using deterministic hashing algorithms. - -RustFS is designed for large-scale, multi-data center cloud storage services. Each tenant runs their own RustFS cluster, completely isolated from other tenants, enabling them to protect against any disruptions from upgrades, updates, and security events. Each tenant scales independently by federating clusters across geographic locations. diff --git a/content/features/cloud-native/images/multi-cloud-architecture.png b/content/features/cloud-native/images/multi-cloud-architecture.png deleted file mode 100644 index 4833d675..00000000 Binary files a/content/features/cloud-native/images/multi-cloud-architecture.png and /dev/null differ diff --git a/content/features/cloud-native/index.md b/content/features/cloud-native/index.md deleted file mode 100644 index f64d84aa..00000000 --- a/content/features/cloud-native/index.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: "Hybrid/Multi-Cloud Object Storage" -description: "Hybrid/multi-cloud architecture enables consistent performance, security, and economics across different environments." ---- - -Hybrid/multi-cloud architecture enables consistent performance, security, and economics. - -## Multi-Cloud Storage Strategies - -### Public Cloud - -Public cloud providers include AWS, Azure, GCP, IBM, Alibaba, Tencent, and government clouds. Hybrid/multi-cloud storage software must run wherever the application stack runs. RustFS provides consistent storage across public cloud providers, avoiding the need to rewrite applications when expanding to new clouds. - -### Private Cloud - -Kubernetes is the primary software architecture for modern private clouds (VMware Tanzu, RedHat OpenShift, Rancher, etc.). Multi-cloud Kubernetes requires software-defined, cloud-native object storage. - -### Edge - -Edge computing moves computation to where data is generated. Edge storage solutions must be lightweight, powerful, cloud-native, and resilient. - -## Multi-Cloud Architecture with RustFS - -![Multi-Cloud Architecture](images/multi-cloud-architecture.png) - -## Properties of Hybrid/Multi-Cloud Storage - -Multi-cloud storage adopts public cloud patterns. New applications are typically written for the AWS S3 API. To scale and perform like cloud-native technologies, applications should be compatible with the S3 API and refactored into microservices. - -### Kubernetes-Native - -Kubernetes-native storage must be deployable and manageable through standard Kubernetes primitives — declarative configuration, services, and persistent volumes. RustFS is designed for Kubernetes and ships an official Helm chart for deployment, upgrades, and scaling. The lightweight RustFS binary allows multiple deployments to be densely co-located in separate namespaces without exhausting resources. - -### Consistent - -Hybrid/multi-cloud storage must be consistent in API compatibility, performance, security, and compliance. RustFS enables non-disruptive updates across public, private, and edge environments, maintaining a consistent experience. RustFS abstracts differences in key management, identity management, access policies, and hardware/OS. - -### Performance - -Object storage must deliver performance at scale for workloads ranging from mobile/web applications to AI/ML. RustFS delivers high throughput on both NVMe and HDD hardware; for representative figures, see [RustFS vs other storage products](/concepts/comparison). - -### Scalable - -Many people think scale only refers to how large a system can become. However, this thinking ignores the importance of operational efficiency as environments evolve. Multi-cloud object storage solutions must scale efficiently and transparently regardless of underlying environment, with minimal human interaction and maximum automation. This can only be achieved through API-driven platforms built on simple architectures. - -RustFS's relentless focus on simplicity means large-scale, multi-petabyte data infrastructure can be managed with minimal human resources. This is a function of APIs and automation, creating an environment on which scalable multi-cloud storage can be built. - -### Software-Defined - -The only way to succeed in multi-cloud is with software-defined storage. The reason is straightforward. Hardware appliances don't run on public clouds or Kubernetes. Public cloud storage service offerings aren't designed to run on other public clouds, private clouds, or Kubernetes platforms. Even if they did, bandwidth costs would exceed storage costs because they weren't developed for cross-network replication. Admittedly, software-defined storage can run on public clouds, private clouds, and edge. - -RustFS was born in software and is portable across various operating systems and hardware architectures, running on AWS, GCP, Azure, and on-premises infrastructure alike. diff --git a/content/features/cold-archiving/index.md b/content/features/cold-archiving/index.md deleted file mode 100644 index 00e53c25..00000000 --- a/content/features/cold-archiving/index.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -title: "Object Storage Cold Archiving Solution" -description: "Built for long-term data storage, constructing secure, intelligent, and sustainable cold data infrastructure" ---- - -Built for long-term data storage, constructing secure, intelligent, and sustainable cold data infrastructure. - -## Core Pain Points - -### Long-Term Storage Challenges - -**Pain Point**: Data needs to be stored for decades, facing multiple risks including media aging, technology obsolescence, and regulatory changes. - -**Technical Challenges**: - -- Limited hardware lifespan (tape typically 10-30 years) -- Old data formats cannot adapt to new systems -- High compliance audit costs - -**RustFS Solution**: - -- Lightweight, agentless architecture: data is written continuously to standard S3 buckets, so it stays accessible through a stable, widely supported protocol -- Media-agnostic storage: objects can move between media generations without application changes -- Audit support: audit logging and retention policies help you demonstrate compliance - -### Offline Media Disaster Recovery - -**Pain Point**: Offline storage is affected by the physical environment and human operational errors, and traditional archive solutions carry data loss risks of their own. - -**Technical Challenges**: - -- Physical damage risk to tape libraries -- High network latency for cross-regional replication -- Long restore times for cold data (hours to days) - -**RustFS Solution**: - -- Tiered media strategy: combine object storage with low-cost archive media for disaster recovery copies -- Cold data direct read: data on warm tiers can be read without a restore/rehydration step -- Metadata synchronization: replication keeps copies consistent across sites - -### Air-Gap and Ransomware Protection - -**Pain Point**: Long-term data is susceptible to malware and ransomware, potentially rendering archives unusable. - -**Technical Challenges**: - -- High cost of implementing an air gap -- Risk of silent data corruption accumulating over time -- Risk of metadata index loss - -**RustFS Solution**: - -- Immutable storage: object locking (WORM) prevents tampering with archived objects -- Self-healing verification: periodic checksum verification with erasure coding repairs silent errors automatically -- Offline and isolated copies: replication to physically isolated or offline media limits the blast radius of an attack - -## Solutions - -### Tiered Storage Engine - -#### Intelligent Tiering - -Automatically divides storage tiers based on access frequency (hot→warm→cold→deep cold), migrating data to low-cost media (such as HDD, tape, or optical) by policy. - -#### Cross-Platform Compatibility - -S3-compatible access connects public cloud and private deployments seamlessly. - -### Long-Term Data Management Technology - -#### Media-Agnostic Design - -A logical abstraction layer shields hardware differences, supporting smooth migration between media generations. - -#### Self-Healing Data Inspection - -Periodic checksum and erasure coding verification repairs silent errors automatically. - -### Secure and Trusted System - -#### Air Gap Support - -Physical isolation and offline media implement a "data vault" that resists network attacks. - -#### Tamper-Evident Auditing - -Object locking and audit logs keep operation history verifiable. - -### Green Energy Practices - -#### Low-Power Storage - -Cold tiers on spun-down disks or offline media consume far less energy than always-on storage. - -#### Hot-Cold Collaborative Scheduling - -Access-pattern-driven tiering keeps only the data you actually use on powered media. - -## Core Advantage Comparison - -| Dimension | Traditional Solution | RustFS Solution | Value Gain | -|-----------|---------------------|-----------------|------------| -| **Lifespan** | Depends on regular media migration | Media-agnostic design plus logical redundancy | Lower migration costs, less technology obsolescence risk | -| **Energy Consumption** | Always-on tape library standby | Intelligent tiering with low-power cold tiers | Lower total cost of ownership | -| **Recovery Speed** | Deep archive restore takes days | Cold data direct read on warm tiers | Faster emergency retrieval | -| **Compliance** | Manual audit with human error risk | Object locking, retention policies, audit logs | Easier certification and review | - -## Industry Scenario Empowerment - -### Financial Compliance Archiving - -Audio/video records are automatically classified and retained to meet banking regulators' multi-year retention requirements. - -### Supercomputing Center Cold Backup - -PB-scale scientific research data protected with erasure coding and compression for dense, durable storage. - -### Media Asset Library - -4K/8K original film archives combine archive media with object storage for fast retrieval of copyrighted material. diff --git a/content/features/commvault/index.md b/content/features/commvault/index.md deleted file mode 100644 index 601b4ce8..00000000 --- a/content/features/commvault/index.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -title: "High-Performance Object Storage for Commvault Backup, Recovery and Replication" -description: "Simple. Scalable. Fast. Ransomware-resistant. In other words, exactly what you want." ---- - -**Simple. Scalable. Fast. Ransomware-resistant. In other words, exactly what you want.** - -## Core Advantages - -### 🔒 Simple = Secure - -The world is already complex enough. Commvault and RustFS simplify backup and recovery to protect your data. It works for a range of data sources from VMs to Office 365. - -### 📈 Simple Delivery at Scale - -RustFS object storage seamlessly scales to EB and beyond through its server pool approach. This ensures Commvault can focus on its core mission while leaving the rest (from hardware heterogeneity to erasure coding and bitrot protection) to RustFS. This means enterprises can scale their backups and protect as much data as possible. - -### ⚡ Fast Backup is One Thing, Fast Recovery is Another - -Regardless of size, backups and recoveries need to be fast. RustFS and Commvault can read/write at speeds exceeding **325 GiB/s** in a single 32-node cluster, enabling backup and recovery from object storage at speeds once thought impossible. When your business depends on fast recovery, there's no better solution in the market. - -### ⚛️ Atomic Power - -Because RustFS atomically writes metadata together with object data, no external metadata database is required (Cassandra in most cases). This eliminates performance penalties associated with small objects. RustFS can provide performance within Commvault's recommended object size ranges, helping with fast deletion and deduplication. - -### 🔐 Inline and Strictly Consistent - -Data in RustFS is always readable and consistent because all I/O is synchronously committed with inline erasure coding, bitrot hashing, and encryption. The S3 service provided by RustFS can flexibly handle any interruptions or restarts during busy transactions. There is no cached or staged data in asynchronous I/O. This guarantees the success of all backup operations. - -### 🔧 Hardware Independent - -Like Commvault, RustFS is software-defined and hardware-independent. This approach provides Commvault customers with tremendous savings and flexibility when designing systems to accommodate various different backup use cases. - -## Solution Overview - -RustFS and Commvault provide various software-defined optimized backup solutions. We work together to add high-performance object storage as endpoints in backup environments, disaggregating compute and storage while providing excellent performance, scalability, and economics. A single RustFS cluster can serve as a Commvault endpoint for anything in VMs, Oracle, SAP, and MS Office. - -## Main Application Scenarios - -### 🖥️ Commvault Backups for VMware ESXi Using RustFS - -Use Commvault to seamlessly backup virtual infrastructure to object storage, providing you with the flexibility of nearly unlimited object storage capacity. You can control costs and security, thereby controlling how data is accessed. - -### 📧 Commvault Backups for Office 365 Using RustFS - -Use Commvault to seamlessly backup Office 365 data to object storage, providing you with the flexibility of nearly unlimited object storage capacity. You can control costs and security, thereby controlling how data is accessed. - -### 💼 Commvault Backups for SAP HANA Using RustFS - -With RustFS, Commvault backup solutions for SAP HANA are faster and more secure. - -### 🗄️ Commvault Backups for Oracle Using RustFS - -Backing up Oracle workloads requires performance, resilience, and security. Optimize this mission-critical backup using RustFS object storage. diff --git a/content/features/data-lake/images/data-lake-architecture.png b/content/features/data-lake/images/data-lake-architecture.png deleted file mode 100644 index ef36c5a0..00000000 Binary files a/content/features/data-lake/images/data-lake-architecture.png and /dev/null differ diff --git a/content/features/data-lake/images/multi-engine-1.svg b/content/features/data-lake/images/multi-engine-1.svg deleted file mode 100644 index a7d3e2d5..00000000 --- a/content/features/data-lake/images/multi-engine-1.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/content/features/data-lake/images/multi-engine-2.svg b/content/features/data-lake/images/multi-engine-2.svg deleted file mode 100644 index 57f0c9d2..00000000 --- a/content/features/data-lake/images/multi-engine-2.svg +++ /dev/null @@ -1 +0,0 @@ - \ No newline at end of file diff --git a/content/features/data-lake/images/performance.png b/content/features/data-lake/images/performance.png deleted file mode 100644 index 2d777c28..00000000 Binary files a/content/features/data-lake/images/performance.png and /dev/null differ diff --git a/content/features/data-lake/images/table-formats.png b/content/features/data-lake/images/table-formats.png deleted file mode 100644 index ec126556..00000000 Binary files a/content/features/data-lake/images/table-formats.png and /dev/null differ diff --git a/content/features/data-lake/index.md b/content/features/data-lake/index.md deleted file mode 100644 index 685ca4c2..00000000 --- a/content/features/data-lake/index.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -title: "RustFS for Modern Data Lakes" -description: "Modern data lakes and lakehouse architectures rely on object storage. RustFS provides a unified storage solution for modern data lakes/lakehouses that can…" ---- - -Modern data lakes and lakehouse architectures rely on object storage. RustFS provides a unified storage solution for modern data lakes/lakehouses that can run anywhere: private cloud, public cloud, colos, bare metal, and edge. - -![Data Lake Architecture](images/data-lake-architecture.png) - -## Open Table Format Ready - -![Table Formats](images/table-formats.png) - -Modern data lakes are multi-engine. They require central table storage, portable metadata, access control, and persistent structure. RustFS supports all major table formats, including Iceberg, Hudi, and Delta Lake. - -## Cloud Native - -RustFS operates on cloud principles: containerization, orchestration, microservices, APIs, infrastructure as code, and automation. The cloud-native ecosystem integrates seamlessly with RustFS, including Spark, Presto/Trino, Snowflake, Dremio, NiFi, Kafka, Prometheus, OpenObserve, Istio, Linkerd, Hashicorp Vault, and Keycloak. - -## Multi-Engine - -RustFS supports all S3-compatible query engines. - -![Multi-Engine Support](images/multi-engine-1.svg) - -![Multi-Engine Support](images/multi-engine-2.svg) - -## Performance - -Modern data lakes require high performance. RustFS delivers high throughput on commodity hardware, improving the efficiency of query engines (Spark, Presto, Trino, Snowflake, SQL Server, Teradata) and AI/ML platforms (MLflow, Kubeflow) compared to legacy Hadoop systems. For representative throughput figures, see [RustFS vs other storage products](/concepts/comparison). - -## Lightweight - -The RustFS server binary is < 100 MB. It is robust enough for data centers and lightweight enough for the edge. Enterprises can access data anywhere with the same S3 API. RustFS edge locations and replication capabilities allow data capture and filtering at the edge before aggregation. - -## Decomposition - -Modern data lakes separate compute and storage. High-speed query processing engines outsource storage to high-throughput object storage like RustFS. By keeping subsets of data in memory and leveraging features like predicate pushdown (S3 Select) and external tables, query engines gain flexibility. - -## Open Source - -Open source is a key driver for data lake adoption. RustFS is open source under the Apache-2.0 license, ensuring freedom from lock-in. - -## Rapid Growth - -Data is constantly being generated, which means it must be constantly ingested - without causing indigestion. RustFS is built for this world and works out of the box with Kafka, Flink, RabbitMQ, and numerous other solutions. The result is that the data lake/lakehouse becomes a single source of truth that can seamlessly scale to exabytes and beyond. - -## Simplicity - -Simplicity is hard. It requires work, discipline, and most importantly, commitment. RustFS prioritizes simplicity in design and operation and is a philosophical commitment that makes our software easy to deploy, use, upgrade, and scale. Modern data lakes don't have to be complex. There are a few parts, and we're committed to ensuring RustFS is the easiest to adopt and deploy. - -## ELT or ETL - It Just Works - -RustFS doesn't just work with every data streaming protocol, but every data pipeline - every data streaming protocol and data pipeline works with RustFS. Every vendor has been extensively tested, and typically, data pipelines have resilience and performance. - -## Resilience - -RustFS protects data using inline erasure coding for each object, which is far more efficient than the HDFS replication alternatives that were never adopted. Additionally, RustFS's bitrot detection ensures it never reads corrupted data - capturing and repairing corrupted data dynamically for objects. RustFS also supports cross-region, active-active replication. Finally, RustFS supports a complete object locking framework providing legal hold and retention (with governance and compliance modes). - -## Software Defined - -The successor to Hadoop HDFS is not a hardware appliance but software running on commodity hardware. This is the essence of RustFS - software. Like Hadoop HDFS, RustFS is designed to take full advantage of commodity servers. By leveraging NVMe drives and 100 GbE networks, RustFS can shrink data centers, thereby improving operational efficiency and manageability. Companies building alternative data lakes can substantially reduce their hardware footprint while improving performance and reducing the staff required to manage it. - -## Security - -RustFS supports multiple sophisticated server-side encryption schemes to protect data wherever it resides, whether in flight or at rest. RustFS's approach ensures confidentiality, integrity, and authenticity with negligible performance overhead. Server-side and client-side encryption support using AES-256-GCM, ChaCha20-Poly1305, and AES-CBC ensures application compatibility. Additionally, RustFS supports industry-leading key management systems (KMS). diff --git a/content/features/distributed/index.md b/content/features/distributed/index.md deleted file mode 100644 index 52be33f2..00000000 --- a/content/features/distributed/index.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: "Infrastructure for Large-Scale Data" -description: "RustFS is designed for scaling - technical scale, operational scale, and economic scale." ---- - -RustFS is engineered for scalability across all dimensions: technical, operational, and economic. - -RustFS is designed to be cloud-native and can run as lightweight containers managed by external orchestration services like Kubernetes. The entire application is compiled into a single static binary (~100 MB) that efficiently uses CPU and memory resources even under high load. As a result, you can co-host large numbers of tenants on shared hardware. - -```mermaid -flowchart LR - APP[Applications] --> S3API(["S3 API"]) - - subgraph DIST["Distributed RustFS"] - direction TB - subgraph N1["Node 1"] - direction LR - S3a[S3] - subgraph OL1["Object Layer"] - direction TB - C1[Cache] - K1[Compression] - E1[Encryption] - B1["Erasure Code · Bitrot"] - end - SL1["Storage Layer"] - J1[("JBOD / FS disks")] - S3a -->|Object API| OL1 - OL1 -->|Storage API| SL1 - SL1 <--> J1 - end - subgraph N2["Node 2"] - direction LR - S3b[S3] - subgraph OL2["Object Layer"] - direction TB - C2[Cache] - K2[Compression] - E2[Encryption] - B2["Erasure Code · Bitrot"] - end - SL2["Storage Layer"] - J2[("JBOD / FS disks")] - S3b -->|Object API| OL2 - OL2 -->|Storage API| SL2 - SL2 <--> J2 - end - NN["Node n ..."] - N1 <-->|Internal RESTful API| N2 - N2 <-->|Internal RESTful API| NN - end - - S3API --> N1 - S3API --> N2 - S3API --> NN - - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class APP,NN muted - class S3API accent - class S3a,S3b,SL1,SL2 server - class C1,K1,E1,B1,C2,K2,E2,B2 svc - class J1,J2 store -``` - -RustFS can run anywhere and on any cloud, but typically runs on commodity servers with locally attached drives (JBOD/JBOF). All servers in the cluster are functionally equal (fully symmetric architecture). There are no name nodes or metadata servers. - -RustFS atomically writes data and metadata, eliminating the need for a separate metadata database. Additionally, RustFS performs all functionality (erasure coding, bitrot checking, encryption) as inline, strictly consistent operations. This results in extraordinary resilience. - -Each RustFS cluster is a collection of distributed RustFS servers, with one process per node. RustFS runs as a single process in user space and uses lightweight coroutines to achieve high concurrency. Drives are grouped into erasure sets (see the [erasure code calculator](https://rustfs.com/erasure-code-calculator/)), and objects are placed on these sets using a deterministic hashing algorithm. - -RustFS is designed for large-scale, multi-datacenter cloud storage services. Each tenant runs their own RustFS cluster, completely isolated from other tenants, enabling them to protect themselves from any disruption due to upgrades, updates, and security events. Each tenant scales independently by federating clusters across geographies. diff --git a/content/features/domestic/index.md b/content/features/domestic/index.md deleted file mode 100644 index a8d31c9d..00000000 --- a/content/features/domestic/index.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -title: "Sovereign Cloud & Compliance Solutions" -description: "We provide complete storage compliance and innovation solutions from hardware and operating systems to classified and encryption systems." ---- - -We provide complete storage compliance and innovation solutions from hardware and operating systems to classified and encryption systems. - -## Pain Points in Sovereign Storage Implementation - -### Traditional Solution Defects - -- Limited adaptation rate for localized chips -- Performance degradation in secure environments -- High expansion costs for centralized architectures - -### Our Technical Implementation - -- Complete adaptation of major localized chip architectures (ARM, MIPS, etc.) -- Self-developed RDMA acceleration protocol with IOPS performance reaching 32.4 million and latency <20μs -- Distributed metadata cluster design supporting EB-level expansion in single clusters - -## Why Choose Us - -### Full-Stack Adaptation - -Supports major localized chip platforms -Hardware performance loss rate <5% -Full compliance with industry certifications - -### Zero-Decay Performance Guarantee - -Self-developed RDMA acceleration protocol -Single node 16.2 million IOPS -Latency <20μs - -### Enterprise-Grade Security System - -Full support for advanced encryption standards -Encryption throughput ≥8 GB/s -Secure supply chain structure - -### Deep Ecosystem Compatibility - -Adapts to various secure operating systems -200+ ISV joint certifications -Extensive application verifications - -## Solutions - -### Government & Public Sector Acceleration Engine - -**Covers full process of government document management and supervision** -**Cross-agency system mutual recognition mechanism** - -- ✓ Provincial government cloud completed 8 system migrations in 3 months -- ✓ Document circulation efficiency improved by 210% -- ✓ 100% pass rate for high-level security certifications - -### Financial-Grade Distributed Core System - -**Million-level TPS distributed main engine (supports advanced security algorithms)** -**Intelligent batch pipeline (automated container 40+ types of batch operations)** - -- ✓ Joint-stock bank system latency reduced from 68ms to 9ms -- ✓ Significant savings in software licensing fees -- ✓ Passed rigorous financial industry security audits - -### Large Enterprise Cross-Platform Cloud Brain Solution - -**Heterogeneous resource scheduling one-click smooth (x86/ARM/MIPS adaptation management)** -**Technology stack seamless migration toolchain (Oracle one-stop migration efficiency improved 6x)** - -- ✓ Heterogeneous management IT operations cost reduced by 37% -- ✓ Centralized R&D system downtime cycle shortened by 82% -- ✓ Supports future deep integration and expansion of distributed computing - -## Core Function Comparison Table - -| Function Dimension | Traditional Storage Solutions | RustFS Solutions | Advantage Improvement | -|---------|------------|--------------|----------| -| Protocol Support | Mainly FC/iSCSI | ✓ Full protocol support (NVMe-oF/S3/NFSv4) | Protocol compatibility improved by 200% ↑ | -| Data Protection | Dual controller + RAID | ✓ Cross-zone 3 replicas + erasure coding | RTO shortened from hours to minutes | -| Performance Benchmark | Single node IOPS ≤500k | ✓ Single node 16.2M IOPS, throughput 144 GB/s | Single machine performance improved 32x ↑ | - -## Service Guarantee System - -### Adaptation Verification - -Time cycle: 5-7 working days - -Hardware compatibility test report -Performance tuning solutions - -### Deployment Implementation - -Time cycle: 3 days/PB level - -Dual-active architecture setup -Data lossless migration - -### Continuous Operations - -Service level: 7x24 monitoring - -Annual health checks -SLA 99.99% guarantee diff --git a/content/features/encryption/images/s5i-1.png b/content/features/encryption/images/s5i-1.png deleted file mode 100644 index db2ea300..00000000 Binary files a/content/features/encryption/images/s5i-1.png and /dev/null differ diff --git a/content/features/encryption/images/s5i-2.png b/content/features/encryption/images/s5i-2.png deleted file mode 100644 index 61905754..00000000 Binary files a/content/features/encryption/images/s5i-2.png and /dev/null differ diff --git a/content/features/encryption/images/s5i-3.png b/content/features/encryption/images/s5i-3.png deleted file mode 100644 index f5782f16..00000000 Binary files a/content/features/encryption/images/s5i-3.png and /dev/null differ diff --git a/content/features/encryption/images/s5i-4.png b/content/features/encryption/images/s5i-4.png deleted file mode 100644 index 0959a232..00000000 Binary files a/content/features/encryption/images/s5i-4.png and /dev/null differ diff --git a/content/features/encryption/images/s5i-5.png b/content/features/encryption/images/s5i-5.png deleted file mode 100644 index ddc13573..00000000 Binary files a/content/features/encryption/images/s5i-5.png and /dev/null differ diff --git a/content/features/encryption/images/s5i-6.png b/content/features/encryption/images/s5i-6.png deleted file mode 100644 index db50277d..00000000 Binary files a/content/features/encryption/images/s5i-6.png and /dev/null differ diff --git a/content/features/encryption/index.md b/content/features/encryption/index.md deleted file mode 100644 index 08a2c668..00000000 --- a/content/features/encryption/index.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -title: "Data Encryption" -description: "Server-side encryption in RustFS with SSE-S3 and SSE-C, designed to minimize performance overhead." ---- - -In the object storage field, robust encryption is a fundamental requirement for enterprise storage. RustFS provides more functionality through the highest level of encryption and extensive optimizations, minimizing the performance overhead typically associated with storage encryption operations. - -```mermaid -flowchart LR - subgraph DATA["Data"] - SSES3["SSE-S3"] - SSEC["SSE-C"] - end - R(["RustFS"]) - KMS[("KMS")] - subgraph B1["My Bucket"] - OBJ1["Object"] - end - subgraph META1["Object Metadata"] - M1A["Random IV"] - M1B["Sealed Object Key"] - M1C["KMS Key ID"] - M1D["Sealed KMS Data Key"] - end - SSES3 --> R - SSEC --> R - KMS --- R - R --> OBJ1 - OBJ1 --> META1 - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class R server - class KMS store - class SSES3,SSEC muted - class OBJ1 svc - class M1A,M1B,M1C,M1D accent -``` - -RustFS encrypts data both when stored on disk and when transmitted over the network. RustFS's state-of-the-art encryption scheme supports fine-grained object-level encryption using modern industry-standard encryption algorithms such as AES-256-GCM, ChaCha20-Poly1305, and AES-CBC. RustFS is fully compatible with S3 encryption semantics and also extends S3 by supporting non-AWS key management services such as Hashicorp Vault, Gemalto KeySecure, and Google Secrets Manager. - -## Network Encryption - -When data is transmitted between object storage and applications, it may bounce between any number of unknown and/or untrusted networks. Encrypting data while it's transmitted over the network (also known as "in transit") successfully mitigates man-in-the-middle attacks and ensures data remains secure regardless of the routing path taken. - -RustFS supports Transport Layer Security (TLS) v1.2+ between all components in the cluster. This approach ensures there are no weak links in encrypted traffic between or within clusters. TLS is a ubiquitous encryption framework: it's the same encryption protocol used by banks, e-commerce websites, and other enterprise-level systems that rely on data storage encryption. - -RustFS's TLS implementation is optimized at the CPU instruction level with negligible performance overhead. It only requires specifying TLS private keys and public certificates for each RustFS server in the cluster. For Kubernetes environments, the RustFS Kubernetes Operator integrates/automatically generates and assigns TLS certificates during tenant deployment. RustFS supports multiple TLS certificates, where each certificate corresponds to a specific domain name. RustFS uses Server Name Indication (SNI) to determine which certificate to serve for any given request. - -## Object Encryption - -Data stored on disk relies entirely on the security of the disk and extends to the host system to ensure data security. RustFS server-side object encryption automatically encrypts data before it's stored on disk (encryption at rest). This approach guarantees that no data is written to unencrypted disks. This baseline security layer ensures the confidentiality, integrity, and authenticity of data at rest. RustFS supports both client-driven and automatic bucket default object encryption for maximum flexibility in data encryption. - -RustFS server-side encryption is compatible with Amazon AWS-S3 semantics (SSE-S3). RustFS extends baseline support for AWS KMS to include common enterprise KMS systems such as Hashicorp Vault and Thales Ciphertrust (formerly Gemalto KeySecure). RustFS also supports client-driven encryption (SSE-C), where applications can specify the data key used to encrypt objects. For both SSE-S3 and SSE-C, the RustFS server performs all encryption operations, including key rotation and object re-encryption. - -Through automatic server-side encryption, RustFS encrypts each object with a unique key and applies multiple layers of additional encryption using dynamic encryption keys and keys derived from external KMS or client-provided keys. This secure and sophisticated approach is performed within RustFS without the need to handle multiple independent kernel and userspace encryption utilities. - -RustFS uses Authenticated Encryption with Associated Data (AEAD) schemes to encrypt/decrypt objects when objects are written to or read from object storage. RustFS AEAD encryption supports industry-standard encryption protocols such as AES-256-GCM and ChaCha20-Poly1305 to protect object data. RustFS's CPU-level optimizations (such as SIMD acceleration) ensure negligible performance overhead for encryption/decryption operations. Organizations can run automatic bucket-level encryption at any time rather than being forced to make suboptimal security choices. - -## RustFS Key Encryption Service - -RustFS provides built-in options for key encryption. RustFS's Key Encryption Service (KES) is a stateless distributed key management system for high-performance applications. It's designed to run in Kubernetes and distribute encryption keys to applications. KES is a required component for RustFS server-side object encryption (SSE-S3). - -KES supports encryption operations on RustFS clusters and is a key mechanism for ensuring scalable and high-performance encryption operations. KES acts as an intermediary between RustFS clusters and external KMS, generating encryption keys as needed and performing encryption operations without being limited by KMS constraints. Therefore, there's still a central KMS that protects master keys and serves as the root of trust in the infrastructure. KES simplifies deployment and management by eliminating the need to bootstrap KMS for each set of applications. Instead, applications can request data encryption keys (DEKs) from KES servers or ask KES servers to decrypt encrypted DEKs. - -Since KES servers are completely stateless, they can be automatically scaled, such as through Kubernetes Horizontal Pod Autoscaler. Additionally, since KES independently handles the vast majority of application requests, the load on the central KMS doesn't increase significantly. - -For Kubernetes environments, the RustFS Kubernetes Operator supports deploying and configuring KES for each tenant, enabling SSE-S3 as part of each tenant deployment. - -```mermaid -flowchart LR - Client["RustFS Server · KES Client"] - KES["RustFS KES Server"] - ExtKMS["External KMS"] - Client <-->|TLS| KES - KES <-->|TLS| ExtKMS - subgraph FLOW["Key Flow"] - App["Application"] - NewKey["Create / Fetch DEK"] - App --> NewKey - end - subgraph AUTH["Authentication"] - Certs["Key / Cert pairs"] - Identity["Identity = Hash of Cert"] - Certs --> Identity - end - App -.->|API| KES - Identity -.-> KES - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - class KES server - class ExtKMS store - class Client,App svc - class NewKey,Certs,Identity muted -``` - -## Supported External Key Management Systems - -| ![AWS KMS](images/s5i-1.png) | ![HashiCorp Vault](images/s5i-2.png) | ![Google Secret Manager](images/s5i-3.png) | -|-------------------------------|----------------------------------------|-------------------------------------------| -| ![Azure Key Vault](images/s5i-4.png) | ![Thales CipherTrust](images/s5i-5.png) | ![Fortanix](images/s5i-6.png) | diff --git a/content/features/hdfs/index.md b/content/features/hdfs/index.md deleted file mode 100644 index 92f8bf56..00000000 --- a/content/features/hdfs/index.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -title: "HDFS Replacement" -description: "RustFS provides a modern, high-performance alternative to traditional Hadoop HDFS." ---- - -## Challenges with HDFS - -Traditional HDFS architecture faces many challenges: - -### Operational Complexity - -- **NameNode Single Point of Failure**: NameNode remains a system bottleneck. -- **Complex Cluster Management**: Requires professional Hadoop operations teams. -- **Difficult Configuration**: Involves numerous parameters requiring deep expertise. - -### Performance Bottlenecks - -- **Small File Problem**: Large numbers of small files consume excessive NameNode memory. -- **Metadata Limitations**: NameNode memory becomes a scaling bottleneck. -- **Network Overhead**: Data replication creates significant network traffic. - -### Cost Considerations - -- **High Hardware Costs**: Requires large numbers of servers and storage devices. -- **High Personnel Costs**: Requires professional operations and development teams. -- **Energy Costs**: Power and cooling costs for large-scale clusters. - -## RustFS Advantages - -RustFS provides comprehensive solutions for HDFS pain points: - -### Architectural Advantages - -- **Decentralized Design**: Eliminates single points of failure and improves reliability. -- **Cloud-Native Architecture**: Supports containerized deployment and elastic scaling. -- **Multi-Protocol Support**: Supports HDFS, S3, NFS, and other protocols. - -### Performance Advantages - -- **High Concurrency**: Rust language's zero-cost abstractions and memory safety. -- **Intelligent Caching**: Multi-level caching strategies improve data access speed. -- **Optimized Data Layout**: Reduces network transmission and improves I/O efficiency. - -### Operational Advantages - -- **Simplified Deployment**: One-click deployment with automated operations. -- **Intelligent Monitoring**: Real-time monitoring and alerting systems. -- **Elastic Scaling**: Automatically adjusts resources based on load. -## Technical Comparison - -| Feature | HDFS | RustFS | -|------|------|---------| -| **Architecture Pattern** | Master-slave architecture (NameNode/DataNode) | Decentralized peer-to-peer architecture | -| **Single Point of Failure** | NameNode has single point risk | No single point of failure | -| **Scalability** | Limited by NameNode memory | Linear scaling | -| **Protocol Support** | HDFS protocol | HDFS, S3, NFS multi-protocol | -| **Small File Handling** | Poor performance | Optimized handling | -| **Deployment Complexity** | Complex configuration and tuning | Simplified deployment | -| **Operational Costs** | Requires professional teams | Automated operations | -| **Cloud Native** | Limited support | Native support | - -## Migration Strategies - -RustFS provides multiple migration strategies to ensure smooth transition from HDFS: - -### Offline Migration - -Use DistCP tools for batch data migration: - -- **Plan Migration Windows**: Choose business off-peak periods for data migration -- **Batch Migration**: Migrate large datasets in batches to reduce risk -- **Data Validation**: Ensure integrity and consistency of migrated data - -### Online Migration - -Achieve zero-downtime migration through dual-write mechanisms: - -- **Dual-Write Mode**: Applications write to both HDFS and RustFS simultaneously -- **Gradual Switching**: Read traffic gradually switches from HDFS to RustFS -- **Data Synchronization**: Real-time synchronization of historical data to RustFS - -### Hybrid Deployment - -Support hybrid deployment of HDFS and RustFS: - -- **Unified Interface**: Manage both systems through unified data access layer -- **Intelligent Routing**: Route to most suitable storage system based on data characteristics -- **Progressive Migration**: New data writes to RustFS, old data remains in HDFS - -## Modern Architecture - -### S3 Compatibility - -RustFS provides complete S3 API compatibility, supporting: - -- **Standard S3 Operations**: Basic operations like PUT, GET, DELETE, LIST -- **Multipart Upload**: Support for sharded upload of large files -- **Pre-signed URLs**: Secure temporary access authorization -- **Version Control**: Object version management and historical tracking - -### Security Architecture - -Comprehensive security assurance mechanisms: - -- **End-to-End Encryption**: Full encryption of data transmission and storage -- **Access Control**: Role-based fine-grained permission management -- **Audit Logs**: Complete operational auditing and logging -- **Compliance Certification**: Meets various industry compliance requirements - -### Auto-Scaling - -Intelligent resource management: - -- **Dynamic Scaling**: Automatically add/remove nodes based on load -- **Load Balancing**: Intelligently distribute requests and data -- **Resource Optimization**: Automatically optimize resource usage efficiency -- **Cost Control**: Pay-as-you-use, reduce total cost of ownership - -### Monitoring and Operations - -Complete monitoring and operations system: - -- **Real-time Monitoring**: Real-time monitoring of system performance and health -- **Intelligent Alerting**: Timely notification and handling of anomalies -- **Performance Analysis**: Deep performance analysis and optimization recommendations -- **Automated Operations**: Reduce manual intervention, improve operational efficiency - -## Cost Analysis - -### TCO Comparison - -| Cost Item | HDFS | RustFS | Savings Ratio | -|----------|------|---------|----------| -| **Hardware Costs** | High | Medium | 30-40% | -| **Operational Costs** | High | Low | 50-60% | -| **Personnel Costs** | High | Low | 40-50% | -| **Energy Costs** | High | Medium | 20-30% | -| **Total TCO** | Baseline | | **40-50%** | - -### Return on Investment - -- **Fast Deployment**: Reduced from weeks to hours -- **Simplified Operations**: 60% reduction in operational workload -- **Performance Improvement**: 2-3x performance improvement -- **Cost Savings**: 40-50% reduction in total cost of ownership - -### Migration Value - -RustFS is not just an alternative to HDFS, but an important step in enterprise data architecture modernization: - -1. **Technical Debt Cleanup**: Break free from legacy technology stack constraints -2. **Cloud-Native Transformation**: Support enterprise cloud-native strategies -3. **Cost Optimization**: Significantly reduce storage and operational costs -4. **Innovation-Driven**: Provide better infrastructure for AI and big data applications - -By choosing RustFS as an alternative to HDFS, enterprises can not only solve current technical challenges but also lay a solid foundation for future digital transformation. diff --git a/content/features/huaweicloud/images/sec1-1.png b/content/features/huaweicloud/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/huaweicloud/images/sec1-1.png and /dev/null differ diff --git a/content/features/huaweicloud/index.md b/content/features/huaweicloud/index.md deleted file mode 100644 index 258e03d3..00000000 --- a/content/features/huaweicloud/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for Huawei Cloud CCE Kubernetes Service" -description: "RustFS provides high-performance object storage for Huawei Cloud CCE with enterprise-grade features and multi-cloud capabilities." ---- - -Huawei Cloud Cloud Container Engine (CCE) is a fully managed Kubernetes service that provides high-performance, highly available container clusters and integrates with Huawei Cloud infrastructure and services. - -Three reasons customers run RustFS on CCE: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on CCE gives you flexible control over the software stack and avoids cloud lock-in. - -RustFS deploys on CCE with the official Helm chart, making it easier to operate your own large-scale, multi-tenant object storage as a service. Because RustFS is S3-compatible from the start, applications built for the S3 API run against RustFS on CCE without changes. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on CCE, you need: - -- A CCE cluster with worker nodes sized for your storage workload -- A block-storage `StorageClass` backed by the Huawei Cloud CSI driver (for example, EVS disks) for RustFS persistent volumes -- A load balancer for external access, typically a Huawei Cloud ELB provisioned through a `LoadBalancer` service or an ingress controller such as NGINX -- `kubectl` and Helm configured against your cluster - -## Deploy RustFS on CCE - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering, external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/industry/images/cold-backup-solution.png b/content/features/industry/images/cold-backup-solution.png deleted file mode 100644 index f10bd735..00000000 Binary files a/content/features/industry/images/cold-backup-solution.png and /dev/null differ diff --git a/content/features/industry/images/multi-cloud-solution.png b/content/features/industry/images/multi-cloud-solution.png deleted file mode 100644 index bee80e5b..00000000 Binary files a/content/features/industry/images/multi-cloud-solution.png and /dev/null differ diff --git a/content/features/industry/images/ssd-hdd-solution.png b/content/features/industry/images/ssd-hdd-solution.png deleted file mode 100644 index 8ae0c7aa..00000000 Binary files a/content/features/industry/images/ssd-hdd-solution.png and /dev/null differ diff --git a/content/features/industry/images/tech-value-pyramid.svg b/content/features/industry/images/tech-value-pyramid.svg deleted file mode 100644 index ab611659..00000000 --- a/content/features/industry/images/tech-value-pyramid.svg +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - Technology Foundation - - - - Scenario Capabilities - - - - Quantifiable Benefits - - - - Strategic Value - - - - - - - Strategic value - Industry 4.0 digital foundation · Flexible - manufacturing data hub · Digital twin core - - - - - Quantifiable benefits - Zero production-line data loss (RTO < 15 min) - Storage cost −55% (TCO) · Process +70% (MES/SCADA) - - - - - Scenario capabilities - Million-sensor data lake (TB/s writes) - Global supply-chain sync (< 1 s) · Frame-level AI QC - - - - - Technology foundation - ∞ Infinite scaling · Eleven-nines reliability - Military-grade erasure coding · 2B files located in 1 s - - diff --git a/content/features/industry/index.md b/content/features/industry/index.md deleted file mode 100644 index c6606e85..00000000 --- a/content/features/industry/index.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -title: "Industrial Production Solutions" -description: "Storage, quality inspection, tracking and long-term preservation of massive data in industrial production, reducing costs and increasing efficiency" ---- - -Storage, quality inspection, tracking and long-term preservation of massive data in industrial production, reducing costs and increasing efficiency - -## Four Core Pain Points in Industrial Production - -| Pain Point | Specific Scenarios/Challenges | User Requirements | -|------------|------------------------------|-------------------| -| **Massive Data Storage and Scalability** | Industrial production generates PB-level data from sensors and equipment, traditional storage is difficult to expand and costly. | Elastic storage capacity expansion, support dynamic growth, reduce hardware investment and maintenance costs. | -| **Real-time Processing and Low Latency** | Real-time monitoring, predictive maintenance scenarios require millisecond-level data read/write, traditional storage has high latency affecting decision efficiency. | High concurrent read/write capability, support real-time data analysis and edge computing, reduce response time. | -| **Data Security and Compliance** | Industrial data involves core process parameters, must meet GDPR, ISO 27001 regulations, prevent leakage and tampering. | End-to-end encryption, fine-grained permission control, audit logs, ensure data lifecycle compliance. | -| **Multi-source Heterogeneous Data Integration** | Industrial environments have multiple protocols/formats like S3, NFS, databases, scattered storage leads to complex management and low utilization. | Unified storage platform compatible with multi-protocol access, centralized data management and seamless cross-system calls. | - -## Solutions - -### SSD and HDD Tiered Storage Cost Reduction - -![SSD and HDD Tiered Storage Solution](./images/ssd-hdd-solution.png) - -SSDs provide fast read/write speeds suitable for applications requiring high I/O performance, while HDDs are lower cost and suitable for large-capacity storage. By storing frequently accessed data on SSDs and infrequently accessed data on HDDs, costs can be reduced without sacrificing performance. - -#### Core Advantages of Tiered Storage - -- **No Performance Compromise**: Achieve SSD acceleration for business needs -- **Cost Cut in Half**: HDD usage for 70% performance data -- **Automated Operations**: AI predicts data lifecycle -- **Elastic Scaling**: On-demand expansion + comprehensive cloud access -- **Risk Distribution**: Media backup + data mirroring -- **Green Low Carbon**: Energy saving + low carbon utilization - -#### Use SSD for performance, HDD for cost reduction, optimizing resources where they matter most for storage spending through intelligent tiering - -#### SSD+HDD Tiered Storage vs Single Storage Solution Cost Comparison - -| Comparison Item | Pure SSD Solution | Pure HDD Solution | Tiered Storage Solution | -|-----------------|-------------------|-------------------|------------------------| -| **Storage Media Cost** | High | Low | Mixed (SSD only stores hot data) | -| **Performance** | Sub-millisecond latency | Millisecond-level latency | Hot data on SSD, cold data read on demand | -| **Energy Consumption** | High (always-on flash) | High (all spindles active) | Lower (small SSD tier plus HDD sleep) | -| **Capacity Expansion Cost** | Full expansion required | Performance bottleneck | Tier-by-tier expansion (e.g., HDD tier only) | -| **Total Cost of Ownership** | Highest | Low, but slow | Balanced: near-HDD cost with near-SSD hot-data performance | -| **Applicable Scenarios** | Real-time trading, high-frequency read/write | Archive, backup | Most enterprise mixed workloads (database/file services) | - -### Cold Backup Storage Cost Reduction - -![Cold Backup Storage Solution](./images/cold-backup-solution.png) - -Compared to traditional tape storage, Blu-ray discs have lower storage costs, especially for large-scale storage. The cost-effectiveness of Blu-ray technology makes it an ideal choice for large-scale data archiving. - -Blu-ray storage devices consume far less energy during operation than hard disk drives (HDDs) or solid-state drives (SSDs), meaning lower energy costs. - -#### Core Advantages of Cold Backup Storage - -- **Lower Cost**: Optical media cost per GB is a fraction of hard-disk solutions -- **Long-term Reliability**: No need for regular data migration -- **Compliance Security**: Enterprise-grade encryption protection - -Cold backup storage substantially reduces low-frequency industrial data archiving costs through intelligent tiering and elastic scaling, balancing security compliance with efficient resource utilization. - -#### Media Comparison - -| Media | Relative Cost | Energy Consumption | Typical Lifespan | -|-------|------------|-------------------|----------| -| **Blu-ray Storage** | Low | Lowest | 50+ years | -| **Tape** | Medium | Low | ~30 years | -| **HDD Series** | High | Highest | ~5 years | - -### Multi-Cloud Transformation Cost Reduction - -![Multi-Cloud Transformation Solution](./images/multi-cloud-solution.png) - -Cloud storage achieves cost reduction and efficiency improvement through integrated dynamic scheduling of data resources, allocating hot and cold data storage networks on demand, calculating based on each cloud vendor's solution, utilizing standardized interfaces to select optimal paths nearby, completing combined reserved/elastic instance cost optimization. - -Simultaneously supports industrial IoT data, service images, and other unstructured data across cloud and edge computing, reducing storage costs while preserving business continuity. - -#### Core Advantages of Multi-Cloud Transformation - -- **Cross-Cloud Scheduling**: Critical business data accelerated on elastic SSD tiers -- **Cost Reduction Through Tiering**: HDD carries the bulk of low-frequency data -- **Lifecycle Automation**: Access-pattern-driven policies move data to the right tier - -### Technology Value Pyramid - -![Technology Value Pyramid](./images/tech-value-pyramid.svg) - -Based on highly reliable, horizontally scalable distributed object storage technology, protect the industrial data chain end to end, support AI quality inspection and real-time global supply chain collaboration, and help manufacturing enterprises move toward agile Industry 4.0 operations. diff --git a/content/features/integration/images/backup-1.png b/content/features/integration/images/backup-1.png deleted file mode 100644 index 07ccfd23..00000000 Binary files a/content/features/integration/images/backup-1.png and /dev/null differ diff --git a/content/features/integration/images/backup-2.png b/content/features/integration/images/backup-2.png deleted file mode 100644 index c07bed9f..00000000 Binary files a/content/features/integration/images/backup-2.png and /dev/null differ diff --git a/content/features/integration/images/backup-3.png b/content/features/integration/images/backup-3.png deleted file mode 100644 index f97582d0..00000000 Binary files a/content/features/integration/images/backup-3.png and /dev/null differ diff --git a/content/features/integration/images/backup-4.png b/content/features/integration/images/backup-4.png deleted file mode 100644 index 2b60ff6d..00000000 Binary files a/content/features/integration/images/backup-4.png and /dev/null differ diff --git a/content/features/integration/images/federation-1.png b/content/features/integration/images/federation-1.png deleted file mode 100644 index 6b02f0e3..00000000 Binary files a/content/features/integration/images/federation-1.png and /dev/null differ diff --git a/content/features/integration/images/federation-2.png b/content/features/integration/images/federation-2.png deleted file mode 100644 index e8469952..00000000 Binary files a/content/features/integration/images/federation-2.png and /dev/null differ diff --git a/content/features/integration/images/hybrid-1.png b/content/features/integration/images/hybrid-1.png deleted file mode 100644 index 1b93e7a2..00000000 Binary files a/content/features/integration/images/hybrid-1.png and /dev/null differ diff --git a/content/features/integration/images/hybrid-2.png b/content/features/integration/images/hybrid-2.png deleted file mode 100644 index 18f02fd0..00000000 Binary files a/content/features/integration/images/hybrid-2.png and /dev/null differ diff --git a/content/features/integration/images/hybrid-3.png b/content/features/integration/images/hybrid-3.png deleted file mode 100644 index 361e2a15..00000000 Binary files a/content/features/integration/images/hybrid-3.png and /dev/null differ diff --git a/content/features/integration/images/hybrid-4.png b/content/features/integration/images/hybrid-4.png deleted file mode 100644 index b8596037..00000000 Binary files a/content/features/integration/images/hybrid-4.png and /dev/null differ diff --git a/content/features/integration/images/identity-1.png b/content/features/integration/images/identity-1.png deleted file mode 100644 index e0f32ff2..00000000 Binary files a/content/features/integration/images/identity-1.png and /dev/null differ diff --git a/content/features/integration/images/identity-2.png b/content/features/integration/images/identity-2.png deleted file mode 100644 index e2b3a78c..00000000 Binary files a/content/features/integration/images/identity-2.png and /dev/null differ diff --git a/content/features/integration/images/identity-3.png b/content/features/integration/images/identity-3.png deleted file mode 100644 index f9031d1e..00000000 Binary files a/content/features/integration/images/identity-3.png and /dev/null differ diff --git a/content/features/integration/images/kms-1.png b/content/features/integration/images/kms-1.png deleted file mode 100644 index 44295408..00000000 Binary files a/content/features/integration/images/kms-1.png and /dev/null differ diff --git a/content/features/integration/images/kms-2.png b/content/features/integration/images/kms-2.png deleted file mode 100644 index ddc13573..00000000 Binary files a/content/features/integration/images/kms-2.png and /dev/null differ diff --git a/content/features/integration/images/loadbalancer-1.png b/content/features/integration/images/loadbalancer-1.png deleted file mode 100644 index d1bccbc1..00000000 Binary files a/content/features/integration/images/loadbalancer-1.png and /dev/null differ diff --git a/content/features/integration/images/loadbalancer-2.png b/content/features/integration/images/loadbalancer-2.png deleted file mode 100644 index e42d3081..00000000 Binary files a/content/features/integration/images/loadbalancer-2.png and /dev/null differ diff --git a/content/features/integration/images/loadbalancer-3.png b/content/features/integration/images/loadbalancer-3.png deleted file mode 100644 index d1d25268..00000000 Binary files a/content/features/integration/images/loadbalancer-3.png and /dev/null differ diff --git a/content/features/integration/images/loadbalancer-4.png b/content/features/integration/images/loadbalancer-4.png deleted file mode 100644 index 4f587c47..00000000 Binary files a/content/features/integration/images/loadbalancer-4.png and /dev/null differ diff --git a/content/features/integration/images/ml-1.png b/content/features/integration/images/ml-1.png deleted file mode 100644 index 6ad18018..00000000 Binary files a/content/features/integration/images/ml-1.png and /dev/null differ diff --git a/content/features/integration/images/ml-2.png b/content/features/integration/images/ml-2.png deleted file mode 100644 index b8596037..00000000 Binary files a/content/features/integration/images/ml-2.png and /dev/null differ diff --git a/content/features/integration/images/ml-3.png b/content/features/integration/images/ml-3.png deleted file mode 100644 index 690c687a..00000000 Binary files a/content/features/integration/images/ml-3.png and /dev/null differ diff --git a/content/features/integration/images/ml-4.png b/content/features/integration/images/ml-4.png deleted file mode 100644 index 1c2c3cb4..00000000 Binary files a/content/features/integration/images/ml-4.png and /dev/null differ diff --git a/content/features/integration/images/ml-5.png b/content/features/integration/images/ml-5.png deleted file mode 100644 index f225a8e0..00000000 Binary files a/content/features/integration/images/ml-5.png and /dev/null differ diff --git a/content/features/integration/images/ml-6.png b/content/features/integration/images/ml-6.png deleted file mode 100644 index a4fb0ab4..00000000 Binary files a/content/features/integration/images/ml-6.png and /dev/null differ diff --git a/content/features/integration/images/monitoring-1.png b/content/features/integration/images/monitoring-1.png deleted file mode 100644 index 5ad08d23..00000000 Binary files a/content/features/integration/images/monitoring-1.png and /dev/null differ diff --git a/content/features/integration/images/monitoring-2.png b/content/features/integration/images/monitoring-2.png deleted file mode 100644 index 6e6e4004..00000000 Binary files a/content/features/integration/images/monitoring-2.png and /dev/null differ diff --git a/content/features/integration/images/monitoring-3.png b/content/features/integration/images/monitoring-3.png deleted file mode 100644 index a1c12a69..00000000 Binary files a/content/features/integration/images/monitoring-3.png and /dev/null differ diff --git a/content/features/integration/images/monitoring-4.png b/content/features/integration/images/monitoring-4.png deleted file mode 100644 index 23e9161d..00000000 Binary files a/content/features/integration/images/monitoring-4.png and /dev/null differ diff --git a/content/features/integration/images/notification-1.png b/content/features/integration/images/notification-1.png deleted file mode 100644 index 11ebe18c..00000000 Binary files a/content/features/integration/images/notification-1.png and /dev/null differ diff --git a/content/features/integration/images/notification-2.png b/content/features/integration/images/notification-2.png deleted file mode 100644 index 92a3251a..00000000 Binary files a/content/features/integration/images/notification-2.png and /dev/null differ diff --git a/content/features/integration/images/notification-3.png b/content/features/integration/images/notification-3.png deleted file mode 100644 index 6610c936..00000000 Binary files a/content/features/integration/images/notification-3.png and /dev/null differ diff --git a/content/features/integration/images/notification-4.png b/content/features/integration/images/notification-4.png deleted file mode 100644 index 5620f5a0..00000000 Binary files a/content/features/integration/images/notification-4.png and /dev/null differ diff --git a/content/features/integration/images/orchestrator-1.png b/content/features/integration/images/orchestrator-1.png deleted file mode 100644 index e561729a..00000000 Binary files a/content/features/integration/images/orchestrator-1.png and /dev/null differ diff --git a/content/features/integration/images/orchestrator-2.png b/content/features/integration/images/orchestrator-2.png deleted file mode 100644 index 0755e7e4..00000000 Binary files a/content/features/integration/images/orchestrator-2.png and /dev/null differ diff --git a/content/features/integration/images/orchestrator-3.png b/content/features/integration/images/orchestrator-3.png deleted file mode 100644 index 99641401..00000000 Binary files a/content/features/integration/images/orchestrator-3.png and /dev/null differ diff --git a/content/features/integration/index.md b/content/features/integration/index.md deleted file mode 100644 index 841150ae..00000000 --- a/content/features/integration/index.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -title: "Integrating with RustFS" -description: "Modern data stacks are connected data stacks. Browse our extensive integration list with links to relevant documentation." ---- - -Modern data stacks are connected data stacks. Browse our extensive integration list with links to relevant documentation. - -## Integration Type Overview - -- 👥 [External Identity Providers](#external-identity-providers) - Single sign-on identity management -- 🔐 [External Key Management](#external-key-management) - Centralized encryption key management -- 📊 [Monitoring and Alerting](#monitoring-and-alerting) - Continuous event monitoring -- 🔔 [Notification Targets](#notification-targets) - Event notification services -- 🌐 [Federation](#federation) - Cross-datacenter authentication -- ⚙️ [Orchestrators](#orchestrators) - Cloud-native orchestration platforms -- ⚖️ [Load Balancers](#load-balancers) - Traffic distribution and management -- ☁️ [Hybrid Cloud](#hybrid-cloud) - Multi-cloud environment support -- 🤖 [Machine Learning and Big Data](#machine-learning-and-big-data) - AI/ML framework integration -- 💾 [Backup](#backup) - Data backup solutions - ---- - -## External Identity Providers - -Trusted identity providers are key components of single sign-on. RustFS supports application and user identities through the following integrations. - -| | | | -|---|---|---| -| ![Identity Provider 1](./images/identity-1.png) | ![Identity Provider 2](./images/identity-2.png) | ![Identity Provider 3](./images/identity-3.png) | - -## External Key Management - -Key Management Service (KMS) enables you to easily create and manage encryption keys and centrally control their usage across your organization. - -| | | -|---|---| -| ![Key Management 1](./images/kms-1.png) | ![Key Management 2](./images/kms-2.png) | - -## Monitoring and Alerting - -Containers and microservices require continuous event monitoring and alerting. Keep a close eye on any cloud-native application or infrastructure through these integrations. - -| | | | | -|---|---|---|---| -| ![Monitoring 1](./images/monitoring-1.png) | ![Monitoring 2](./images/monitoring-2.png) | ![Monitoring 3](./images/monitoring-3.png) | ![Monitoring 4](./images/monitoring-4.png) | - -## Notification Targets - -Event notifications are central to any system's operational acuity. RustFS logs all object operations for lambda computing, object search, analytics, and security auditing. - -| | | | | -|---|---|---|---| -| ![Notification 1](./images/notification-1.png) | ![Notification 2](./images/notification-2.png) | ![Notification 3](./images/notification-3.png) | ![Notification 4](./images/notification-4.png) | - -## Federation - -When distributed deployments span datacenters and geographic locations, central federated authentication services are needed. RustFS integrates with the following. - -| | | -|---|---| -| ![Federation 1](./images/federation-1.png) | ![Federation 2](./images/federation-2.png) | - -## Orchestrators - -RustFS supports modern cloud-native orchestration platforms for fully automated deployment and management of physical resources (CPU, network, and drives). - -| | | | -|---|---|---| -| ![Orchestrator 1](./images/orchestrator-1.png) | ![Orchestrator 2](./images/orchestrator-2.png) | ![Orchestrator 3](./images/orchestrator-3.png) | - -## Load Balancers - -For public-facing infrastructure, load balancers provide the following services: routing, service discovery, SSL termination, and traffic shaping. RustFS integrates with the following. - -| | | | | -|---|---|---|---| -| ![Load Balancer 1](./images/loadbalancer-1.png) | ![Load Balancer 2](./images/loadbalancer-2.png) | ![Load Balancer 3](./images/loadbalancer-3.png) | ![Load Balancer 4](./images/loadbalancer-4.png) | - -## Hybrid Cloud - -RustFS makes existing infrastructure from on-premises deployments to public clouds look like Amazon S3. Additionally, it adds caching CDN functionality in front of public clouds to save bandwidth while providing high performance. - -| | | | | -|---|---|---|---| -| ![Hybrid Cloud 1](./images/hybrid-1.png) | ![Hybrid Cloud 2](./images/hybrid-2.png) | ![Hybrid Cloud 3](./images/hybrid-3.png) | ![Hybrid Cloud 4](./images/hybrid-4.png) | - -## Machine Learning and Big Data - -Modern enterprises are data-driven. RustFS has native integrations with leading analytics and machine learning frameworks. - -| | | | -|---|---|---| -| ![Machine Learning 1](./images/ml-1.png) | ![Machine Learning 2](./images/ml-2.png) | ![Machine Learning 3](./images/ml-3.png) | -| ![Machine Learning 4](./images/ml-4.png) | ![Machine Learning 5](./images/ml-5.png) | ![Machine Learning 6](./images/ml-6.png) | - -## Backup - -Object storage using the AWS S3 API has become the ubiquitous backup target for every modern backup application. RustFS integrates with S3-compatible systems including the following leading vendors (the list is long). - -| | | | | -|---|---|---|---| -| ![Backup 1](./images/backup-1.png) | ![Backup 2](./images/backup-2.png) | ![Backup 3](./images/backup-3.png) | ![Backup 4](./images/backup-4.png) | diff --git a/content/features/kubernetes-common/index.md b/content/features/kubernetes-common/index.md deleted file mode 100644 index bda6e19b..00000000 --- a/content/features/kubernetes-common/index.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: "RustFS on Kubernetes: Common Capabilities" -description: "Capabilities that RustFS provides on every Kubernetes platform, including storage tiering, load balancing, encryption, identity, TLS, monitoring, and logging." ---- - -RustFS runs on any CNCF-conformant Kubernetes distribution and is deployed with the official RustFS Helm chart. This page describes the capabilities that are common to every Kubernetes platform. For platform-specific prerequisites, see the individual platform pages: [Alibaba Cloud ACK](/features/aliyun), [Amazon EKS](/features/aws-elastic), [Huawei Cloud CCE](/features/huaweicloud), [Tencent Cloud TKE](/features/qcloud), [Red Hat OpenShift](/features/openshift), and [VMware Tanzu](/features/tanzu). - -## Deployment Model - -RustFS is a single lightweight binary distributed as a container image. On Kubernetes you deploy it with the RustFS Helm chart, which manages pods, services, persistent volume claims, and configuration. RustFS does not require an Operator or custom resource definitions (CRDs); standard Kubernetes primitives and Helm releases are all you need. See the [cloud-native installation guide](/installation/cloud-native) for deployment steps. - -## Storage Classes and Tiering - -A key requirement for running RustFS at scale is the ability to use different storage classes (NVMe, HDD, public cloud object storage) for different data temperatures, so you can manage cost and performance simultaneously. - -RustFS supports transitioning aging objects from fast NVMe tiers to more cost-effective HDD tiers, and to cold public cloud storage tiers. When tiering, RustFS provides a unified namespace across tiers: movement is transparent to applications and triggered by policies you define. - -Because RustFS encrypts objects at the source, you keep control of your data even when a cold tier lives in a public cloud. - -## External Load Balancing - -All RustFS communication is HTTP-based (S3 RESTful API), so any standard Kubernetes ingress controller or platform load balancer can front a RustFS deployment. NGINX Ingress is a common choice; managed platforms also provide native load balancer integrations. Expose the RustFS service through your platform's ingress or a `LoadBalancer` service. - -## Encryption and Key Management - -For production environments, we recommend enabling encryption on all buckets. RustFS uses AEAD ciphers (AES-256-GCM, ChaCha20-Poly1305) to protect data integrity and confidentiality with low performance overhead. - -RustFS ships a built-in KMS subsystem, so no separate key-management component needs to be deployed alongside it. The KMS supports the following backends: - -- `local` — keys stored locally, suitable for development and evaluation -- `vault` — HashiCorp Vault KV backend -- `vault-transit` — HashiCorp Vault Transit engine - -For production we recommend a Vault-backed configuration so that master keys live outside the storage system. See [encryption](/features/encryption) for details on server-side encryption modes. - -## Identity and Access Management - -You can manage single sign-on (SSO) through external OpenID Connect-compatible identity providers such as Keycloak or Okta. External IdPs let administrators manage user and application identities centrally, while RustFS provides AWS IAM-style users, groups, policies, and access keys on top. An IAM layer that is independent of the underlying infrastructure gives you the same access model on every platform. - -## TLS and Certificates - -Traffic between applications and RustFS, including inter-node traffic, can be encrypted with TLS. On Kubernetes, you can issue and renew certificates with [cert-manager](https://cert-manager.io/) or bring your own certificates, and mount them into RustFS pods as Kubernetes secrets. Running separate RustFS deployments in separate namespaces with their own certificates keeps workloads isolated from each other. - -## Monitoring and Alerting - -RustFS observability is built on OpenTelemetry: the server exports metrics, logs, and traces through an OTLP endpoint. Deploy an OpenTelemetry Collector in your cluster to receive this telemetry, then forward it to Prometheus, Grafana, Jaeger, or any other OTLP-compatible backend you already run. Alerting thresholds can be defined in your monitoring stack and routed to notification platforms such as PagerDuty. - -## Logging and Auditing - -Enabling RustFS auditing generates logs for every operation on the object storage cluster. In addition to audit logs, RustFS logs server errors for troubleshooting. Logs can be shipped to Elastic Stack or another log analysis platform through your cluster's standard log collection pipeline. diff --git a/content/features/lifecycle/index.md b/content/features/lifecycle/index.md deleted file mode 100644 index f5fef3f6..00000000 --- a/content/features/lifecycle/index.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -title: "Data Lifecycle Management and Tiering" -description: "Data growth requires efficient lifecycle management for access, security, and economics. RustFS provides features to protect data within and between clouds,…" ---- - -Data growth requires efficient lifecycle management for access, security, and economics. RustFS provides features to protect data within and between clouds, including versioning, object locking, and lifecycle management. - -## Object Expiration - -Data Retention: RustFS lifecycle management tools allow you to define how long data remains on disk before deletion. Define retention periods as a specific date or number of days. - -Lifecycle management rules are created per bucket and can be constructed using any combination of object and tag filters. Omitting filters applies the expiration rule to the entire bucket. - -RustFS object expiration rules also apply to versioned buckets. For example, you can specify expiration rules only for non-current versions to minimize storage costs. - -Bucket expiration rules comply with RustFS WORM locking and legal holds. Objects in a locked state remain on disk until the lock expires or is explicitly released. - -RustFS object expiration lifecycle management rules are compatible with AWS Lifecycle Management. RustFS supports importing existing rules in JSON format. - -## Policy-Based Object Tiering - -RustFS can be programmatically configured for object storage tiering. Objects transition from one state or class to another based on variables like time and frequency of access. Tiering allows users to optimize storage costs or functionality. - -## Cross-Media Tiering - -RustFS abstracts the underlying media to optimize for performance and cost. For example, performance workloads might use NVMe or SSD, while older data is tiered to HDD. - -```mermaid -flowchart LR - subgraph Cloud[Storage Cluster] - HOT[RUSTFS Hot Tier] - WARM[RUSTFS Warm Tier] - NVME[(NVMe SSDs)] - SAS[("SAS/SATA HDDs")] - HOT -->|ILM Transition| WARM - HOT --- NVME - WARM --- SAS - end - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class HOT server - class WARM accent - class NVME,SAS store -``` - -## Hybrid Cloud Tiering - -Public cloud storage can serve as a tier for private clouds. Performance-oriented workloads run on private cloud media. As data ages, enterprises can use public cloud cold storage to optimize costs. - -RustFS runs on both private and public clouds. Using replication, RustFS moves data to public cloud options and protects it. The public cloud serves as a storage tier. - -```mermaid -flowchart LR - subgraph Private[Private Cloud Storage] - HOT[RUSTFS Hot Tier] - end - subgraph Public[Public Cloud Storage] - subgraph Cold["Warm / Cold Tier"] - S3[Amazon S3] - GCS[Google Cloud Storage] - AZ[Azure Blob Storage] - end - end - HOT -->|ILM Transition| Cold - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - class HOT server - class S3,GCS,AZ svc -``` - -## In Public Clouds - -RustFS typically serves as the primary application storage tier in public clouds. RustFS determines which data belongs where based on management parameters. - -RustFS combines different storage tiering layers and determines appropriate media to provide better economics without compromising performance. Applications address objects through RustFS, while RustFS transparently applies policies to move objects between tiers. - -```mermaid -flowchart LR - subgraph Public["Public Cloud Storage"] - HOT["RustFS Hot Tier"] - subgraph Cold["Warm / Cold Tier"] - S3["Amazon S3"] - GCS["Google Cloud Storage"] - AZ["Azure Blob Storage"] - end - BS[("Block Storage")] - OS[("Object Storage")] - HOT -->|ILM Transition| Cold - HOT --- BS - Cold --- OS - end - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - class HOT server - class S3,GCS,AZ svc - class BS,OS store -``` diff --git a/content/features/logging/index.md b/content/features/logging/index.md deleted file mode 100644 index 33ef37a1..00000000 --- a/content/features/logging/index.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -title: "Logging and Auditing" -description: "RustFS observability: OpenTelemetry metrics, audit logging, and event notifications to external targets." ---- - -Metrics and logging are crucial for system health. RustFS provides robust monitoring and observability through detailed storage performance monitoring, metrics, and logging. - -## Features - -### Monitoring Metrics - -Provides complete system monitoring and performance metrics collection. - -### Logging - -Records detailed log information for every operation, supporting audit trails. - -## Metrics Monitoring - -RustFS collects a wide range of fine-grained hardware and software metrics and exports them over OTLP (the OpenTelemetry Protocol, configured via `RUSTFS_OBS_ENDPOINT`). Deploy an OpenTelemetry Collector to forward these metrics to Prometheus, Grafana, or any OTLP-compatible backend. The upstream [`docker-compose.yml`](https://github.com/rustfs/rustfs/blob/main/docker-compose.yml) ships a ready-made observability profile with OpenTelemetry Collector, Prometheus, Grafana, and Jaeger. - -RustFS also provides health check endpoints (`/health` and `/health/ready`) for probing node and cluster liveness. - -## Audit Logs - -Audit logging generates logs for every cluster operation. Each operation generates an audit log containing a unique ID and detailed information about the client, object, bucket, and metadata. RustFS writes audit data to configured targets such as HTTP/HTTPS webhook endpoints and Kafka (`RUSTFS_AUDIT_*` environment variables). - -RustFS event notifications provide additional logging support: bucket and object events can be pushed automatically to third-party systems (for example RabbitMQ via AMQP, Kafka, or a webhook) for event-driven processing. - -## Architecture - -RustFS does not natively expose metrics via Prometheus-compatible HTTP(S) endpoints for direct scraping. To integrate with Prometheus, deploy an OpenTelemetry Collector to gather metrics from RustFS and forward them to your Prometheus backend. - -```mermaid -flowchart TD - RUSTFS["RustFS Object Storage"] - OTEL["OpenTelemetry Collector"] - subgraph PROM["Prometheus"] - AM[Alertmanager] - QA["Query API"] - RRW["Remote Read/Write"] - WH[Webhooks] - end - AR["Alert Response"] - VA["Visualization / Analytics"] - AB["Archival / Backup"] - RUSTFS -->|OTLP| OTEL - OTEL --> PROM - PROM --> AR - PROM --> VA - PROM --> AB - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class RUSTFS server - class OTEL,AM,QA,RRW,WH svc - class AR,VA,AB accent -``` - -RustFS event notifications automatically push bucket and object events to supported target services. Administrators can define bucket-level notification rules. - -```mermaid -flowchart LR - subgraph Clients["Client Applications"] - IOT["Internet of Things"] - WEB["Web Applications"] - BK["Backup / Archival"] - BS["Block Storage"] - end - TENANT["RustFS Cluster"] - subgraph Targets["Notification Targets"] - WH[Webhooks] - KAFKA[Kafka] - AMQP[AMQP] - MQTT[MQTT] - RED[Redis] - NATS[NATS] - PULSAR[Pulsar] - MYSQL[MySQL] - PG[PostgreSQL] - end - Clients <-->|S3 operations| TENANT - TENANT -->|events| Targets - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - class IOT,WEB,BK,BS server - class TENANT svc - class WH,KAFKA,AMQP,MQTT,RED,NATS,PULSAR,MYSQL,PG accent -``` - -## Requirements - -### For Metrics - -Deploy an OpenTelemetry Collector and point `RUSTFS_OBS_ENDPOINT` at it; visualize with Prometheus and Grafana (see the upstream observability compose profile). - -### For Audit Logs - -Configure one or more audit targets (webhook, Kafka) via `RUSTFS_AUDIT_*` environment variables. - -### For Event Notifications - -Configure bucket notification rules toward the supported targets: webhook, Kafka, AMQP, MQTT, Redis, NATS, Pulsar, MySQL, or PostgreSQL. diff --git a/content/features/openshift/images/sec1-1.png b/content/features/openshift/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/openshift/images/sec1-1.png and /dev/null differ diff --git a/content/features/openshift/index.md b/content/features/openshift/index.md deleted file mode 100644 index 297f1160..00000000 --- a/content/features/openshift/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for Red Hat OpenShift Container Platform" -description: "RustFS provides high-performance object storage for Red Hat OpenShift with enterprise-grade features and multi-cloud capabilities." ---- - -Red Hat OpenShift is an enterprise Kubernetes container platform with full-stack automated operations that can manage hybrid cloud, multi-cloud, and edge deployments. OpenShift includes an enterprise Linux operating system, container runtime, networking, monitoring, registry, and authentication and authorization solutions. - -Three reasons customers run RustFS on OpenShift: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on OpenShift gives you flexible control over the software stack and avoids cloud lock-in. - -RustFS deploys on OpenShift with the official Helm chart and works alongside the OpenShift toolchain, making it easier to operate your own large-scale, multi-tenant object storage as a service. Because RustFS is S3-compatible from the start, applications built for the S3 API run against RustFS on OpenShift without changes. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on OpenShift, you need: - -- An OpenShift cluster with worker nodes sized for your storage workload -- A block-storage `StorageClass` (for example, one provided by your platform's CSI driver or local storage) for RustFS persistent volumes -- External access through OpenShift Routes, an ingress controller, or a `LoadBalancer` service -- The `oc`/`kubectl` CLI and Helm configured against your cluster, with any security context constraints (SCCs) your policies require - -## Deploy RustFS on OpenShift - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering, external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/qcloud/images/sec1-1.png b/content/features/qcloud/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/qcloud/images/sec1-1.png and /dev/null differ diff --git a/content/features/qcloud/index.md b/content/features/qcloud/index.md deleted file mode 100644 index d7a0e017..00000000 --- a/content/features/qcloud/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for Tencent Cloud TKE Kubernetes Service" -description: "RustFS provides high-performance object storage for Tencent Cloud TKE with enterprise-grade features and multi-cloud capabilities." ---- - -Tencent Kubernetes Engine (TKE) is a fully managed Kubernetes service that provides high-performance, highly available container clusters and integrates with Tencent Cloud infrastructure and services. - -Three reasons customers run RustFS on TKE: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on TKE gives you flexible control over the software stack and avoids cloud lock-in. - -RustFS deploys on TKE with the official Helm chart, making it easier to operate your own large-scale, multi-tenant object storage as a service. Because RustFS is S3-compatible from the start, applications built for the S3 API run against RustFS on TKE without changes. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on TKE, you need: - -- A TKE cluster with worker nodes sized for your storage workload -- A block-storage `StorageClass` backed by the Tencent Cloud CSI driver (for example, CBS disks) for RustFS persistent volumes -- A load balancer for external access, typically a Tencent Cloud CLB provisioned through a `LoadBalancer` service or an ingress controller such as NGINX -- `kubectl` and Helm configured against your cluster - -## Deploy RustFS on TKE - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering, external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/quantitative-trading/index.md b/content/features/quantitative-trading/index.md deleted file mode 100644 index a56d6d36..00000000 --- a/content/features/quantitative-trading/index.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: "Quantitative Trading File Storage Solutions" -description: "Storage architecture designed for high-frequency trading and quantitative strategy backtesting, with high-throughput order flow processing and low-latency access to tick-level data." ---- - -Storage architecture designed for high-frequency trading and quantitative strategy backtesting, supporting high-throughput order flow processing and low-latency access to tick-level market data. - -## Industry Challenges and Pain Points - -| Category | Traditional Solution Defects | Quantitative Requirements | -|------|-------------|----------| -| **Data Management** | Single-protocol storage (S3 only or POSIX only) | Unified access across protocols and tools | -| **Performance** | Limited IOPS on small-file random reads | High IOPS with sub-millisecond latency for tick data | -| **Storage Cost** | Cold data kept on expensive hot storage | Intelligent tiering that moves cold data to low-cost media | - -## Why Choose RustFS - -### Fast Response - -- Distributed, parallel I/O keeps latency low and throughput high for market-data reads -- Backtesting jobs read historical data in parallel instead of queueing behind a single storage head - -### Massive File Support - -- Object storage semantics handle very large numbers of small files without a central metadata bottleneck -- Metadata is stored with the objects, so listing and retrieval scale with the cluster - -### Elastic Scaling - -- Supports hybrid deployment: hot data on local SSD, cold data tiered to cheaper media or the cloud -- Capacity scales linearly by adding nodes - -### Financial Security - -- Enterprise-grade encryption (AES-256-GCM, ChaCha20-Poly1305) with low performance overhead -- Multi-region replication for disaster recovery - -For representative performance figures, see [RustFS vs other storage products](/concepts/comparison). - -## Scenario-Based Solutions - -### High-Frequency Strategy Development - -Strategy code in C++ or Python reads raw trading data directly over the S3 API, and parallel reads shorten large backtests from days to hours compared with single-head storage. - -### AI Factor Mining - -Feature datasets map naturally to S3 object paths, so TensorFlow/PyTorch pipelines can stream training data straight from RustFS and run many factor computations in parallel. - -### Regulatory Compliance Storage - -Object locking provides WORM (Write Once Read Many) semantics for non-tamperable trading records, and audit logging records operations for regulatory review. - -## Industry Compliance and Security - -### Encryption - -Server-side encryption with strong ciphers (AES-256-GCM, ChaCha20-Poly1305) protects data at rest. - -### Cross-Regional Synchronization - -Replication across sites supports off-site disaster recovery requirements such as SEC 17a-4-style retention policies. - -### Audit Interface - -Audit logs can be shipped to analysis platforms such as Splunk or Elastic. - -## Deployment - -RustFS is delivered as software you can run on your own hardware or in the cloud. See the [installation guides](/installation/linux/quick-start) to get started. diff --git a/content/features/replication/images/s6-1.png b/content/features/replication/images/s6-1.png deleted file mode 100644 index 41fe8f58..00000000 Binary files a/content/features/replication/images/s6-1.png and /dev/null differ diff --git a/content/features/replication/index.md b/content/features/replication/index.md deleted file mode 100644 index e6e29772..00000000 --- a/content/features/replication/index.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: "Multi-Site, Active-Active Replication for Object Storage" -description: "Active replication ensures data availability. RustFS supports active-active replication. It operates at the bucket level." ---- - -## Active Replication for Object Storage - -![Object Storage Replication](images/s6-1.png) - -Active replication ensures data availability. RustFS supports active-active replication. It operates at the bucket level. - -RustFS supports synchronous and near-synchronous replication, depending on architectural choices and data change rates. Replication aims for strict consistency within data centers and eventual consistency between data centers. - -## Resilience Features - -- **Encrypted/Unencrypted Objects**: Replicates objects and metadata. -- **Object Versions**: Preserves version history. -- **Object Tags**: Replicates tags. -- **S3 Object Lock**: Maintains retention information. - -## Core Features - -### Identical Bucket Naming - -Enables transparent failover to remote sites without interruption. - -### Object Lock Replication - -Ensures data integrity and compliance requirements are maintained during replication. - -### Near-Synchronous Replication - -Updates objects immediately after mutation. - -### Notifications - -Pushes replication failure events for operations teams. - -## Implementation Considerations - -Key factors include: - -### Infrastructure - -RustFS recommends using the same hardware at both ends of the replication endpoints to simplify troubleshooting. - -### Bandwidth - -Bandwidth is critical for synchronization. If bandwidth is insufficient to handle peaks, changes will queue to the remote site. - -### Latency - -After bandwidth, latency is the most important consideration when designing an active-active model. Latency represents the round-trip time (RTT) between two RustFS clusters. The goal is to reduce latency to the smallest possible number within the budget constraints imposed by bandwidth. RustFS recommends RTT thresholds not exceeding 20 milliseconds for Ethernet links and networks, with packet loss rates not exceeding 0.01%. - -### Architecture - -Currently, RustFS only recommends replication across two data centers. Replication across multiple data centers is possible, however, the complexity involved and the trade-offs required make this quite difficult. - -## Large-Scale Deployment Architecture - -RustFS supports very large deployments in each data center, including source and target, with the above considerations determining scale. - -```mermaid -flowchart TB - WAN(["WAN · 10 Gbps"]) - subgraph DC1["Data Center 1"] - LB1["Spine / Leaf Switches"] - KES1["RustFS KES Encryption"] - REP1["RustFS Replication"] - subgraph C1["Object Storage Cluster"] - A1[("Storage")] - A2[("Storage")] - A3[("Storage")] - A4[("Storage")] - end - LB1 -->|100 Gbps| C1 - KES1 -->|Secret Keys| REP1 - REP1 --- C1 - end - subgraph DC2["Data Center 2"] - LB2["Spine / Leaf Switches"] - KES2["RustFS KES Encryption"] - REP2["RustFS Replication"] - subgraph C2["Object Storage Cluster"] - B1[("Storage")] - B2[("Storage")] - B3[("Storage")] - B4[("Storage")] - end - LB2 -->|100 Gbps| C2 - KES2 -->|Secret Keys| REP2 - REP2 --- C2 - end - WAN --> LB1 - WAN --> LB2 - REP1 <-->|Async Replication| REP2 - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - class WAN,LB1,LB2 muted - class KES1,KES2 accent - class REP1,REP2 svc - class A1,A2,A3,A4,B1,B2,B3,B4 store -``` - -## Frequently Asked Questions - -### What happens when the replication target fails? - -If the target goes down, the source will cache changes and begin synchronizing after the replication target recovers. There may be some delay in reaching full synchronization, depending on the duration, number of changes, bandwidth, and latency. - -### What are the parameters for immutability? - -Immutability is supported. Key concepts can be found in this article. In active-active replication mode, immutability can only be guaranteed when objects are versioned. Versioning cannot be disabled on the source. If versioning is suspended on the target, RustFS will begin failing replication. - -### What other impacts are there if versioning is suspended or there's a mismatch? - -In these cases, replication may fail. For example, if you try to disable versioning on the source bucket, an error will be returned. You must first remove the replication configuration before you can disable versioning on the source bucket. Additionally, if versioning is disabled on the target bucket, replication will fail. - -### How is it handled if object locking is not enabled on both ends? - -Object locking must be enabled on both source and target. There's an edge case where after setting up bucket replication, the target bucket can be deleted and recreated but without object locking enabled, and replication may fail. If object locking settings are not configured on both ends, inconsistent situations may occur. In this case, RustFS will fail silently. diff --git a/content/features/s3-compatibility/images/s1-4.png b/content/features/s3-compatibility/images/s1-4.png deleted file mode 100644 index 454caa05..00000000 Binary files a/content/features/s3-compatibility/images/s1-4.png and /dev/null differ diff --git a/content/features/s3-compatibility/index.md b/content/features/s3-compatibility/index.md deleted file mode 100644 index 3530dd72..00000000 --- a/content/features/s3-compatibility/index.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: "S3 Compatibility" -description: "The RustFS S3 compatibility matrix: which S3 APIs are implemented, which are planned, and which are intentionally out of scope, backed by the executable Ceph s3tests suites." ---- - -RustFS provides broad S3 API compatibility for supported features. It does not claim complete coverage of every standard or vendor-specific S3 behavior. This page reproduces the upstream [S3 compatibility matrix](https://github.com/rustfs/rustfs/blob/main/docs/architecture/s3-compatibility-matrix.md), which ties the compatibility claim to the executable Ceph s3tests lists under `scripts/s3-tests/` in the RustFS repository. - -Legend: ✅ implemented and gated by tests · ❌ planned but not yet implemented · ⊘ intentionally excluded (out of scope). - -## Test list sources - -| List | Purpose | Count | Source | -| --- | --- | ---: | --- | -| Implemented tests | Standard S3 tests expected to pass; used by the default local s3tests run. | 452 | `scripts/s3-tests/implemented_tests.txt` | -| Lifecycle behavior tests | Expiration behavior cases gated by the dedicated lifecycle lane (debug-accelerated day + scanner enabled). | 5 | `scripts/s3-tests/lifecycle_behavior_tests.txt` | -| Unimplemented tests | Standard S3 features planned but not yet implemented. | 17 | `scripts/s3-tests/unimplemented_tests.txt` | -| Excluded tests | Vendor-specific or intentionally unsupported behavior excluded from compatibility gating. | 273 | `scripts/s3-tests/excluded_tests.txt` | - -Counts ignore blank lines and comments. - -## Supported coverage - -The implemented test list currently covers the common object-storage surface: - -### Bucket APIs - -| Area | Status | -| --- | --- | -| Bucket create / delete / list / head | ✅ | -| Bucket and object tagging | ✅ | -| Bucket policy put / get / delete | ✅ | -| Public access block put / get / delete | ✅ | - -### Object APIs - -| Area | Status | -| --- | --- | -| Object put / get / delete / copy / head | ✅ | -| ListObjects / ListObjectsV2 with prefix, delimiter, marker, max-keys | ✅ | -| Presigned GET and PUT URLs | ✅ | -| Range and conditional reads | ✅ | -| User metadata | ✅ | -| SSE-C and selected SSE-KMS edge cases | ✅ | -| Selected versioning, object-lock, checksum, CORS, raw request, and conditional write behavior | ✅ | - -### Multipart APIs - -| Area | Status | -| --- | --- | -| Multipart upload create / upload / complete / abort | ✅ | -| Selected multipart copy, checksum, and object-attribute behavior | ✅ | - -## Planned standard coverage - -These are standard S3 areas that remain planned work and are not yet complete: - -| Area | Status | Evidence | -| --- | --- | --- | -| Bucket access logging | ❌ | `unimplemented_tests.txt` | -| POST Object form upload checksum handling | ❌ | `unimplemented_tests.txt` | -| Bucket ownership controls | ❌ | `unimplemented_tests.txt` | -| IAM-account or multi-storage-class dependent cases | ❌ | `unimplemented_tests.txt` | -| Tenanted bucket policy edge cases | ❌ (needs investigation) | `unimplemented_tests.txt` | -| Multipart upload listing and part lookup compatibility edge cases | ⊘ (not part of default gate) | `excluded_tests.txt` | - -## Intentional exclusions - -The excluded list contains tests that do not block the RustFS compatibility gate. They fall into two classes: - -- vendor-specific or non-portable behavior not required for RustFS S3 compatibility; -- intentionally unsupported product behavior, such as ACL authorization. - -## Lifecycle behavior lane - -The lifecycle behavior lane runs real Days-based expiration cases. It requires `RUSTFS_ILM_DEBUG_DAY_SECS` (the Ceph `lc_debug_interval` equivalent) and an enabled background scanner, and runs separately from the default single-server gate because a global debug day would also shrink the `x-amz-expiration` header asserted by other lifecycle header tests. - -:::note -For the authoritative, always-current test lists and the update rule that moves entries from unimplemented to implemented, see the upstream matrix: [docs/architecture/s3-compatibility-matrix.md](https://github.com/rustfs/rustfs/blob/main/docs/architecture/s3-compatibility-matrix.md). -::: diff --git a/content/features/small-file/index.md b/content/features/small-file/index.md deleted file mode 100644 index 86c2f3b9..00000000 --- a/content/features/small-file/index.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: "Small File Optimization" -description: "Memory Object Storage for High Performance" ---- - -> Memory Object Storage for High Performance - -Use server DRAM for distributed shared memory pools for workloads requiring massive IOPS and throughput performance. - -## Background - -Small file optimization improves IOPS and throughput. In modern architectures, this is critical for AI/ML workloads. Without caching, I/O can become a bottleneck for GPUs. - -Caching accelerates access to training, validation, and test datasets. - -## Features - -### Dedicated Object Cache - -RustFS small file optimization is designed for caching file objects. -If an object is not found in the cache, RustFS retrieves it, caches it for future requests, and returns it to the caller. - -### Consistent Hashing - -RustFS uses consistent hashing algorithms to distribute cached object data across a cluster of cache nodes. Consistent hashing ensures objects can be easily found based on the object's key. This creates a one-to-one relationship between the object's key and the node holding the cached object. It ensures balanced data distribution and minimizes reshuffling when nodes are added or removed. - -### Rolling Cache - -RustFS uses rolling cache for memory management. It keeps the total cache size within specified limits. If adding new objects would exceed the limit, objects are removed based on timestamps (LRU). - -### Automatic Version Updates - -RustFS automatically updates the cache with new object versions when they are updated in storage. - -### Seamless API Integration - -Small file optimization is a seamlessly integrated extension of RustFS. Developers use the same APIs. If the requested object is in cache, RustFS fetches it from cache. If not, it fetches from storage and caches it. diff --git a/content/features/sql-server/index.md b/content/features/sql-server/index.md deleted file mode 100644 index c7e2018c..00000000 --- a/content/features/sql-server/index.md +++ /dev/null @@ -1,507 +0,0 @@ ---- -title: "Running SQL Server 2022 Anywhere" -description: "Leverage the power of RustFS to run SQL Server 2022 on any cloud (public, private, or edge) using external table functions and PolyBase." ---- - -Leverage the power of RustFS to run SQL Server 2022 on any cloud (public, private, or edge) using external table functions and PolyBase. - -## Any to Any, All the Time - -Use SQL Server 2022 data cloud to query and analyze multiple data sources residing on RustFS. Now enterprises can query data residing on RustFS from any SQL Server instance (in public cloud, private cloud, or even streaming edge instances). - -### Supported Deployment Environments - -RustFS integration with SQL Server 2022 supports the following deployment environments: - -- **AWS**: Amazon Web Services cloud environment -- **GCP**: Google Cloud Platform -- **Azure**: Microsoft Azure cloud platform -- **Tanzu**: VMware Tanzu container platform -- **OpenShift**: Red Hat OpenShift container platform -- **HPE Ezmeral**: HPE's container platform -- **SUSE Rancher**: SUSE's Kubernetes management platform -- **Traditional Bare Metal Deployment**: On-premises data center environments - -### Unified Data Access - -Through RustFS's unified S3-compatible interface, SQL Server 2022 can: - -- Access data across multiple cloud environments -- Eliminate data silos -- Provide consistent query experience -- Reduce data integration complexity - -## Connect to Data, Don't Move It - -Using external tables, enterprises can enjoy the full functionality of SQL Server without incurring the cost or coordination challenges of moving data. - -### PolyBase Feature Advantages - -PolyBase functionality allows users to query data directly from SQL Server and most other database installations using Transact-SQL: - -#### Supported Data Sources - -- **SQL Server**: On-premises and cloud instances -- **Oracle**: Enterprise-grade relational database -- **Teradata**: Big data analytics platform -- **MongoDB**: NoSQL document database -- **S3 API**: Access object storage through RustFS - -#### Core Advantages - -1. **Zero Data Movement**: Direct querying of remote data sources -2. **Unified Query Language**: Use familiar T-SQL syntax -3. **Real-time Data Access**: No need to pre-load data -4. **Reduced Storage Costs**: Avoid duplicate data storage - -### Data Silo Integration - -RustFS provides unique capabilities for accessing all hyperscale cloud environments. The combination of SQL Server 2022 and RustFS enables enterprises to: - -- Access data scattered across different systems -- Gain comprehensive insights from data silos -- Achieve unified data view -- Simplify complex data integration scenarios - -## Massive Scale Performance - -Massive scale performance solutions for all enterprise data. - -### Performance Characteristics - -With this new capability, enterprises can use SQL Server 2022 for all organizational data: - -#### Unlimited Data Scale - -- **Location Agnostic**: Data can be located anywhere -- **Unlimited Scale**: Support for multi-petabyte data storage -- **Fast Queries**: High-speed queries for massive datasets -- **Concurrent Processing**: Support for multi-user concurrent access - -#### Performance Optimization - -With RustFS's industry-leading performance characteristics: - -1. **High Throughput**: Optimized data transfer speeds -2. **Low Latency**: Fast response to query requests -3. **Intelligent Caching**: Improve performance for frequently accessed data -4. **Load Balancing**: Automatic query load distribution - -### Resource Utilization Enhancement - -This means higher utilization: - -- **SQL Server Utilization**: More fully utilize existing SQL Server investments -- **RustFS Instance Utilization**: Maximize storage resource value -- **Enterprise Data Utilization**: Unlock the full value of data - -## Backup and Recovery - -Backup and restore like you've always dreamed of. - -### Core Use Cases - -One of the core use cases for SQL Server 2022 and RustFS is backup and restore: - -#### Diverse Configuration Support - -- **Multiple Architectures**: Support different deployment architectures -- **Flexible Configuration**: Adapt to various business needs -- **Scalability**: Scale with business growth - -#### Fast Recovery Capabilities - -RustFS's industry-leading throughput characteristics: - -1. **Time Compression**: Reduce weeks of recovery time to hours -2. **High Availability**: Ensure business continuity -3. **Data Integrity**: Guarantee backup data integrity -4. **Automated Processes**: Reduce manual intervention - -### Backup Strategy Optimization - -Effective backup strategies include: - -- **Incremental Backup**: Only backup changed data -- **Differential Backup**: Changes based on last full backup -- **Full Backup**: Regular complete data backup -- **Instant Recovery**: Fast recovery of critical business data - -## Secure and Available - -To ensure the right data is available to the right users, fine-grained access control must be implemented on these multi-cloud data lakes. - -### Identity Authentication and Authorization - -#### Third-Party IDP Integration - -RustFS can integrate with third-party identity providers (IDPs): - -- **Unified Identity Management**: Centralized user identity management -- **Single Sign-On (SSO)**: Simplified user access experience -- **Multi-Factor Authentication (MFA)**: Enhanced security -- **Role Mapping**: Automatic assignment of appropriate permissions - -#### Access Control Mechanisms - -Ensure access to object storage is limited to those who need it: - -1. **Principle of Least Privilege**: Only grant necessary permissions -2. **Regular Permission Reviews**: Ensure permission timeliness -3. **Access Logging**: Complete audit trails -4. **Anomaly Detection**: Identify abnormal access behavior - -### Policy-Based Access Control (PBAC) - -#### Fine-Grained Permission Management - -RustFS's sophisticated PBAC functionality ensures: - -- **Resource-Level Control**: Precise permissions to specific resources -- **Dynamic Permission Assignment**: Adjust permissions based on context -- **Policy Inheritance**: Simplify permission management -- **Compliance Support**: Meet regulatory requirements - -#### Security Assurance - -- **Data Encryption**: Encryption protection during transmission and storage -- **Network Isolation**: Secure network communication -- **Threat Detection**: Real-time security threat monitoring -- **Incident Response**: Rapid response to security incidents - -## Resilience - -SQL Server is one of the most widely used analytics tools in enterprises, making it a mission-critical application. - -### Disaster Recovery Capabilities - -#### Continuous Data Replication - -SQL Server 2022 allows continuous data replication to and from the cloud: - -- **Real-time Synchronization**: Ensure data is up-to-date -- **Bidirectional Replication**: Support active-active deployment -- **Conflict Resolution**: Automatically handle data conflicts -- **Failover**: Quick switch to backup systems - -#### Tiered Storage Strategy - -The combination with RustFS allows: - -1. **Fast Storage Tier**: NVMe high-speed storage -2. **Warm Storage Tier**: Balance performance and cost -3. **Cold Storage Tier**: Long-term archival storage -4. **Automatic Tiering**: Intelligent data movement - -### Data Processing Capabilities - -#### Multiple Processing Methods - -Enterprises can read, write, and process big data using multiple methods: - -- **Transact-SQL**: Traditional SQL query language -- **Spark Libraries**: Big data processing framework -- **Hybrid Analytics**: Combine relational and non-relational data -- **Real-time Processing**: Stream data processing capabilities - -#### High Availability Architecture - -- **Multi-site Deployment**: Cross-regional data distribution -- **Active-Active Replication**: Provide highest availability -- **Strict Consistency**: Ensure data consistency -- **Cloud Disaster Recovery**: Resist complete cloud failures - -## Streaming Edge - -By adding external table functionality, enterprises can now set up streaming pipelines to save data on RustFS - in the cloud or on-premises. - -### Real-time Data Processing - -#### Streaming Data Pipelines - -- **Real-time Data Ingestion**: Continuously receive streaming data -- **Data Preprocessing**: Clean and transform data -- **Storage Optimization**: Efficient data storage -- **Query Optimization**: Query optimization for streaming data - -#### Real-time Query Capabilities - -SQL Server can be configured to execute queries on this data in real-time: - -1. **Eliminate Batch Imports**: No need to wait for batch processing -2. **Instant Insights**: Real-time business insights -3. **Reduced Latency**: Minimize data processing delays -4. **Enhanced Experience**: Add new dimensions to SQL Server - -### Edge Computing Advantages - -#### Edge Deployment Characteristics - -- **Low Latency Processing**: Process data close to source -- **Bandwidth Optimization**: Reduce data transmission -- **Offline Capabilities**: Support intermittent connectivity -- **Local Intelligence**: Edge intelligent decision-making - -#### Application Scenarios - -- **IoT Data Processing**: Internet of Things device data -- **Real-time Monitoring**: System status monitoring -- **Predictive Maintenance**: Equipment failure prediction -- **Smart Manufacturing**: Production process optimization - -## Cloud as Operating Model - -Cloud operating model starting from S3. - -### Cloud Operations Characteristics - -RustFS adheres to the cloud operating model: - -#### Core Technology Stack - -- **Containerization**: Containerized application deployment -- **Orchestration**: Kubernetes container orchestration -- **Automation**: Automated operations management -- **API-Driven**: Complete API interface -- **S3 Compatibility**: Standard S3 API support - -#### Unified Interface Advantages - -Provides unified interface across clouds and storage types: - -1. **Simplified Development**: Unified development interface -2. **Reduced Learning Costs**: Standardized operation methods -3. **Improved Portability**: Cross-cloud application migration -4. **Reduced Lock-in**: Avoid vendor lock-in - -### AI/ML Framework Compatibility - -#### Broad Framework Support - -Since most AI/ML frameworks and applications are designed to use S3 API: - -- **TensorFlow**: Google's machine learning framework -- **PyTorch**: Facebook's deep learning framework -- **Scikit-learn**: Python machine learning library -- **Apache Spark**: Big data processing engine - -#### Developer Validation - -With over 1.3 billion Docker pulls: - -- **Most Developer Validated**: Extensive developer community -- **24/7/365 Validation**: Continuous compatibility validation -- **Best Compatibility**: Industry-best compatibility record -- **Production Ready**: Large-scale production validated - -### Data Management Flexibility - -This compatibility ensures: - -- **AI Workload Access**: Seamless access to stored data -- **Cloud Infrastructure Agnostic**: Independent of specific cloud environments -- **Flexible Data Approaches**: Adapt to different data processing needs -- **Cross-Cloud Environment Processing**: Support multi-cloud data processing - -## Edge AI Storage - -At the edge, network latency, data loss, and software bloat degrade performance. - -### Edge Optimization Features - -#### Performance Advantages - -RustFS is the world's fastest object storage: - -- **Less than 100 MB**: Extremely small binary files -- **Any Hardware**: Can be deployed on any hardware -- **High Performance**: Optimized edge performance -- **Low Resource Consumption**: Minimal system requirements - -#### Intelligent Features - -RustFS's advanced features: - -1. **Bucket Notifications**: Storage bucket event notifications -2. **Object Lambda**: Object processing functions -3. **Real-time Inference**: Instant data processing -4. **Automatic Triggers**: Event-based automatic processing - -### Edge Application Scenarios - -#### Mission-Critical Applications - -- **Airborne Object Detection**: High-altitude drone applications -- **Traffic Trajectory Prediction**: Autonomous vehicles -- **Industrial Control**: Real-time industrial control systems -- **Security Monitoring**: Real-time security monitoring - -#### Technical Characteristics - -RustFS's AI storage features: - -- **Fast Response**: Millisecond response times -- **Fault Tolerance**: High reliability design -- **Simple Deployment**: Simplified deployment process -- **Edge Optimization**: Optimized for edge scenarios - -## Lifecycle Management for ML/AI Workloads - -Modern AI/ML workloads require complex lifecycle management. - -### Automated Data Management - -#### Core Functions - -RustFS's lifecycle management capabilities: - -- **Automated Tasks**: Automatically execute data management tasks -- **Storage Optimization**: Optimize storage efficiency -- **Reduced Overhead**: Lower operational overhead -- **Intelligent Tiering**: Automatic data tiering - -#### Cost Optimization Strategies - -With lifecycle policies: - -1. **Automatic Migration**: Migrate infrequently accessed data to low-cost storage -2. **Resource Release**: Free resources for active workloads -3. **Storage Tiering**: Multi-tier storage architecture -4. **Cost Control**: Effective storage cost control - -### ML/AI Specialized Functions - -#### Developer Experience - -These features ensure AI/ML practitioners can: - -- **Focus on Core**: Focus on model training and development -- **Automatic Management**: RustFS intelligently manages data -- **Performance Enhancement**: Improve overall workflow performance -- **Cost Effectiveness**: Achieve maximum cost effectiveness - -#### Compliance Support - -Lifecycle management layer: - -- **Enforce Policies**: Enforce retention and deletion policies -- **Regulatory Compliance**: Ensure compliance with regulations -- **Audit Trails**: Complete operation records -- **Automated Compliance**: Automated compliance processes - -## Object Retention for AI/ML Workflows - -Compared to AI/ML, fewer workloads depend more on when things happen. - -### Advanced Object Retention - -#### Core Guarantees - -Addressed through advanced object retention features: - -- **Data Integrity**: Ensure integrity of stored data -- **Compliance Requirements**: Meet regulatory compliance requirements -- **Time Sensitivity**: Handle time-related business needs -- **Data Consistency**: Maintain data consistency - -#### Retention Policy Implementation - -By implementing retention policies, RustFS can help organizations: - -1. **Model Consistency**: Maintain data consistency for AI/ML models and datasets -2. **Prevent Accidental Deletion**: Avoid accidental or unauthorized deletion -3. **Prevent Modification**: Protect data from unauthorized modification -4. **Version Control**: Maintain data version history - -### Data Governance Advantages - -#### Governance Framework - -This feature is crucial for: - -- **Data Governance**: Establish comprehensive data governance framework -- **Regulatory Compliance**: Meet various regulatory requirements -- **Experiment Reproducibility**: Ensure AI/ML experiment reproducibility -- **Data Lineage**: Complete data lineage tracking - -#### Guarantee Mechanisms - -Guarantee critical data: - -- **Specific Duration**: Remain accessible for specified time -- **Data Immutability**: Ensure data is not modified -- **Precise Training**: Support precise model training -- **Reliable Analysis**: Provide reliable data analysis foundation - -## Data Protection for Core AI Datasets - -RustFS provides comprehensive data protection through different feature quantities. - -### Data Redundancy and Fault Tolerance - -#### Protection Mechanisms - -- **Erasure Coding**: Efficient data redundancy mechanism -- **Site Replication**: Cross-site data replication -- **Data Redundancy**: Ensure redundant data storage -- **Fault Tolerance**: Prevent hardware failures or data corruption - -#### Failure Recovery - -Automatically handle various failure scenarios: - -1. **Hardware Failures**: Automatic detection and recovery -2. **Data Corruption**: Real-time detection and repair -3. **Network Failures**: Automatic recovery from network interruptions -4. **Site Failures**: Cross-site failover - -### Data Encryption Protection - -#### Encryption Mechanisms - -RustFS supports multi-level data encryption: - -- **Encryption at Rest**: Encryption protection for stored data -- **Encryption in Transit**: Encryption during data transmission -- **Key Management**: Secure key management mechanisms -- **Compliance Encryption**: Encryption standards meeting compliance requirements - -#### Access Control - -- **Unauthorized Access Protection**: Prevent unauthorized data access -- **Authentication**: Enforce authentication mechanisms -- **Permission Control**: Fine-grained permission control -- **Access Monitoring**: Real-time access behavior monitoring - -### Identity and Access Management (IAM) - -#### IAM Support - -RustFS's IAM support enables organizations to: - -- **Access Control**: Control access to AI storage data -- **User Management**: Unified user management -- **Application Authorization**: Application access control -- **Permission Assignment**: Flexible permission assignment mechanisms - -#### Security Assurance - -Ensure only authorized users or applications can: - -1. **Access Data**: Secure data access -2. **Modify Data**: Controlled data modification -3. **Delete Data**: Secure data deletion -4. **Manage Permissions**: Permission management operations - -### Full Lifecycle Protection - -#### Comprehensive Protection Mechanisms - -RustFS provides comprehensive data protection mechanisms: - -- **Integrity Protection**: Maintain AI dataset integrity -- **Availability Assurance**: Ensure high data availability -- **Confidentiality Protection**: Protect data confidentiality -- **Lifecycle Coverage**: Cover entire data lifecycle - -Through deep integration of SQL Server 2022 with RustFS, enterprises can build a powerful, secure, high-performance modern data platform supporting comprehensive needs from traditional relational data processing to the latest AI/ML workloads. diff --git a/content/features/tanzu/images/sec1-1.png b/content/features/tanzu/images/sec1-1.png deleted file mode 100644 index 07116ed9..00000000 Binary files a/content/features/tanzu/images/sec1-1.png and /dev/null differ diff --git a/content/features/tanzu/index.md b/content/features/tanzu/index.md deleted file mode 100644 index 17e3f98f..00000000 --- a/content/features/tanzu/index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: "RustFS for VMware Tanzu Container Platform" -description: "RustFS provides high-performance object storage for VMware Tanzu with enterprise-grade features and multi-cloud capabilities." ---- - -VMware Tanzu is an enterprise Kubernetes container platform for building, running, and managing containerized applications across vSphere, public clouds, and edge environments. - -Three reasons customers run RustFS on Tanzu: - -- RustFS serves as a consistent storage layer in hybrid cloud or multi-cloud deployment scenarios. -- RustFS is a Kubernetes-native, high-performance product that delivers predictable performance across public cloud, private cloud, and edge environments. -- Running RustFS on Tanzu gives you control over the software stack and the flexibility to avoid cloud lock-in. - -RustFS deploys on Tanzu Kubernetes clusters with the official Helm chart, making it easier to operate your own large-scale, multi-tenant object storage as a service. Because RustFS is S3-compatible from the start, applications built for the S3 API run against RustFS on Tanzu without changes. - -![RustFS Architecture Diagram](images/sec1-1.png) - -## Prerequisites - -Before deploying RustFS on Tanzu, you need: - -- A Tanzu Kubernetes cluster with worker nodes sized for your storage workload -- A block-storage `StorageClass` backed by the vSphere CSI driver (or your infrastructure's CSI driver) for RustFS persistent volumes -- A load balancer for external access, for example NSX Advanced Load Balancer or an ingress controller such as NGINX -- `kubectl` and Helm configured against your cluster - -## Deploy RustFS on Tanzu - -RustFS is deployed with its official Helm chart; no Operator or CRDs are required. Follow the [cloud-native installation guide](/installation/cloud-native) for the deployment steps. - -## Common Capabilities - -Storage tiering, external load balancing, encryption and built-in KMS, identity management, TLS certificates, OpenTelemetry-based monitoring, and audit logging work the same on every Kubernetes platform. See [RustFS on Kubernetes: Common Capabilities](/features/kubernetes-common). diff --git a/content/features/veeam/index.md b/content/features/veeam/index.md deleted file mode 100644 index fbf3c5e7..00000000 --- a/content/features/veeam/index.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -title: "High-Performance Object Storage for Veeam Backup and Replication" -description: "Using RustFS to extend your v12 instances and significantly improve Veeam storage capacity and performance." ---- - -Use RustFS to extend your v12 instances and significantly improve Veeam storage capacity and performance. - -## RustFS Partners with Veeam to Add High-Performance Private Cloud Object Storage to S3 Endpoint Portfolio - -Veeam Backup and Replication provides various software-defined optimized backup solutions. We work together to add high-performance object storage as endpoints, disaggregating compute and storage in backup environments while providing excellent performance, scalability, and economics. A single RustFS instance can serve as a Veeam endpoint for virtual machines, Oracle, SAP, and MS Office. - -## Main Application Scenarios - -### 🖥️ Veeam Backups for VMware ESXi Using RustFS - -Use Veeam to seamlessly backup virtual infrastructure to object storage, providing you with the flexibility of nearly unlimited object storage capacity. You can control costs and security, thereby controlling how data is accessed. - -### 📧 Veeam Backups for Office 365 Using RustFS - -Use Veeam to seamlessly backup virtual infrastructure to object storage, providing you with the flexibility of nearly unlimited object storage capacity. You can control costs and security, thereby controlling how data is accessed. - -### 💼 Veeam Backups for SAP HANA Using RustFS - -With RustFS, Veeam backup solutions for SAP HANA are faster and more secure. - -### 🗄️ Veeam Backups for Oracle Using RustFS - -Backing up Oracle workloads requires performance, resilience, and security. Optimize this mission-critical backup using RustFS object storage. - ---- - -## Veeam and RustFS are Natural Partners - -Both Veeam and RustFS provide best-in-class software solutions for their respective technologies. From VMs to Office 365, large-scale performance is the metric for end-to-end solutions. RustFS object storage provides a highly scalable, high-performance object storage solution, making it a strong choice for Veeam customers. - -## Core Advantages - -### ⚡ Fast Backup is One Thing, Fast Recovery is Another - -Regardless of size, backups and recoveries need to be fast. RustFS delivers high read/write throughput that scales with cluster size, enabling backup and recovery directly from object storage at speeds that keep pace with tight recovery objectives. For representative throughput figures, see [RustFS vs other storage products](/concepts/comparison). - -### 🗃️ Metadata Advantages - -Because RustFS atomically writes metadata together with object data, Veeam backups don't require external metadata databases (Cassandra in most cases). This eliminates performance penalties associated with small objects. RustFS can provide performance within Veeam's recommended object size ranges, helping with fast deletion and deduplication. - -### 🔒 Inline and Strictly Consistent - -Data in RustFS is always readable and consistent because all I/O is synchronously committed with inline erasure coding, bitrot hashing, and encryption. The S3 service provided by RustFS can flexibly handle any interruptions or restarts during busy transactions. There is no cached or staged data in asynchronous I/O. This guarantees the success of all backup operations. - -### 🔧 Hardware Independent - -Like Veeam, RustFS is software-defined and hardware-independent. This approach provides Veeam customers with tremendous savings and flexibility when designing systems to accommodate various different backup use cases. - -### 🚀 RustFS and Veeam: Backup and Recovery from Object Storage - -RustFS and Veeam make a powerful combination! Deploying RustFS object storage with Veeam brings multiple advantages. These include advantages related to software-defined solutions, performance characteristics of fast backup and recovery, and the resilience and flexibility of object storage that writes metadata atomically. diff --git a/content/features/versioning/images/architecture.png b/content/features/versioning/images/architecture.png deleted file mode 100644 index e728e41f..00000000 Binary files a/content/features/versioning/images/architecture.png and /dev/null differ diff --git a/content/features/versioning/index.md b/content/features/versioning/index.md deleted file mode 100644 index 7567124e..00000000 --- a/content/features/versioning/index.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -title: "Bucket and Object Versioning" -description: "Object-level versioning improves data protection. Versioning serves as the foundation for object locking, immutability, tiering, and lifecycle management." ---- - -## RustFS Object Storage Provides AWS S3 Versioning Compatibility - -Object-level versioning improves data protection. Versioning serves as the foundation for object locking, immutability, tiering, and lifecycle management. - -RustFS implements S3-compatible versioning. RustFS assigns a unique ID to each version of an object. Applications can specify a version ID to access a point-in-time snapshot. - -Versioning allows users to preserve multiple variants of an object in the same bucket, enabling retrieval and restoration of every version. - -Versioning is enabled at the bucket level. Once enabled, RustFS automatically creates a unique version ID for objects. - -Versioning prevents accidental overwrites and deletions. When a versioned object is deleted, a delete marker is created. The object can be restored by deleting the delete marker. - -If a versioned object is overwritten, RustFS creates a new version. Old versions can be restored as needed. - -## RustFS Supports Object Versioning with Three Different Bucket States - -```mermaid -stateDiagram-v2 - direction LR - state "Versioning Not Enabled" as Unversioned - state "Versioning Enabled" as Enabled - state "Versioning Suspended" as Suspended - - [*] --> Unversioned - Unversioned --> Enabled: Enable - Enabled --> Suspended: Suspend - Suspended --> Enabled: Re-enable -``` - -Versioning can be suspended but not disabled once enabled. Versioning is a global setting in the bucket. - -Users with appropriate permissions can suspend versioning to stop accumulating object versions. - -Manage versioning via Console, CLI (`mc`), or SDK. - -Versioning increases bucket size and may create object dependencies. Mitigate these factors through lifecycle management. - -## Features - -- **Bucket Replication** (Active-Active, Active-Passive) -- **`mc undo`**: Rollback PUT/DELETE objects. -- **Object Lock** -- **Continuous Data Protection (CDP)** -- **`mc rewind`**: View buckets or objects at any point in time. - -## Architecture - -```mermaid -flowchart LR - CLOUD(["Internet Cloud"]) --> GLB["Global Load Balancer"] - GLB --> S1 - GLB --> S2 - GLB --> SN - subgraph S1["Site 1 · US-WEST"] - A1["Zone 1 · Erasure Sets 1-n"] - A2["Zone 2 · Erasure Sets 1-n"] - A3["Zone n · Erasure Sets 1-n"] - end - subgraph S2["Site 2 · US-EAST"] - B1["Zone 1 · Erasure Sets 1-n"] - B2["Zone 2 · Erasure Sets 1-n"] - B3["Zone n · Erasure Sets 1-n"] - end - subgraph SN["Site n · EU-CENTRAL"] - D1["Zone 1 · Erasure Sets 1-n"] - D2["Zone 2 · Erasure Sets 1-n"] - D3["Zone n · Erasure Sets 1-n"] - end - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef accent fill:#fae8ff,stroke:#c026d3,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - class CLOUD svc - class GLB accent - class A1,A2,A3,B1,B2,B3,D1,D2,D3 store -``` - -### System Requirements - -> Versioning requires: Erasure coding and at least four disks. - -### Versioning States - -RustFS supports three different bucket versioning states: - -1. **🔴 Unversioned** - Default state, no versioning performed -2. **🟢 Enabled** - Full versioning functionality, assigns unique ID to each object version -3. **🟡 Suspended** - Stops accumulating new versions but retains existing versions - -### Key Features - -- 🆔 **Unique Version ID** - Each object version has a unique identifier -- 🔄 **Point-in-Time Recovery** - Can access any historical version of an object -- 🛡️ **Delete Protection** - Uses delete markers to prevent accidental deletion -- 📊 **Lifecycle Management** - Automatically manages version count and storage costs -- 🔐 **Permission Control** - Fine-grained access permission management diff --git a/content/features/video/images/solution.png b/content/features/video/images/solution.png deleted file mode 100644 index 3807049c..00000000 Binary files a/content/features/video/images/solution.png and /dev/null differ diff --git a/content/features/video/index.md b/content/features/video/index.md deleted file mode 100644 index 325690fc..00000000 --- a/content/features/video/index.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -title: "Cost-Effective Video Storage Solutions" -description: "Achieve dramatic cost reductions in video storage through object storage and hybrid cloud approaches." ---- - -Achieve dramatic cost reductions in video storage through object storage and hybrid cloud approaches. - -## Core Pain Points of Video Storage - -### Challenges with Traditional Solutions - -- Linear storage architecture causes read/write speeds to decline as capacity increases. -- Cold footage occupies high-performance storage long-term, inflating cost. -- Single-replica storage plus periodic backup leaves recovery gaps. -- Storage expansion requires downtime maintenance and lacks intelligent management tools. - -### Business Impact - -- Slow key-frame retrieval delays emergency response. -- Storage costs grow year over year while most footage is rarely accessed. -- Hardware failures lead to long data recovery cycles and risk of critical evidence loss. -- Manual operations drive up per-terabyte cost and reduce availability. - -## Core Cost Reduction Capabilities - -### Lower Storage Costs - -- Intelligent hot-cold separation: footage not accessed for a configurable period (for example, 30 days) transitions automatically to cheaper archive tiers. -- Erasure coding stores data with far less overhead than multi-replica schemes. -- Capacity scales horizontally, so you grow the cluster instead of over-provisioning up front. - -### Fast Data Access - -- High-concurrency ingest supports large camera fleets writing simultaneously. -- Low-latency reads keep playback and key-frame retrieval responsive. - -### Enterprise-Grade Data Protection - -- Erasure coding plus optional cross-site replication protects against disk, node, and site failures. -- Object locking (WORM) preserves critical footage for evidentiary retention requirements. -- Versioning allows recovery of overwritten or deleted objects within your retention window. - -### Seamless Integration - -- Standard S3 API works with surveillance platforms and media asset systems that support object storage. -- SDKs and RESTful APIs for custom integration. -- Migration tooling for moving existing data from NAS/SAN systems. - -### Observability - -- Monitoring of storage health, capacity, and access patterns through OpenTelemetry-based metrics. -- Capacity trends help you plan expansion before hitting bottlenecks. - -## Solutions - -Video feeds can be uploaded to storage through three methods: - -### Hybrid Cloud Tiered Storage - -Applicable scenarios: Large campuses, smart cities (1000+ cameras). - -#### Core Capabilities - -- Intelligent tiering: hot data stored locally on SSD, full data automatically synced to a cloud or archive tier. -- Reduced bandwidth usage: only necessary data leaves the local site. -- Disaster recovery: replication between local and remote deployments. - -### Direct Cloud Storage - -Applicable scenarios: Shops, communities, homes (50-200 cameras). - -#### Core Advantages - -- Rapid deployment with standard S3 endpoints. -- Event-based clips can be generated by the surveillance platform and stored as objects. -- Minimal on-site maintenance. - -### Server Relay Storage - -Applicable scenarios: Educational campuses, cross-regional enterprises. - -#### Key Technologies - -- Edge preprocessing: frame extraction and filtering before upload saves bandwidth. -- Tiered archiving: original footage retained short-term, low-bitrate copies retained longer, driven by lifecycle policies. - -```mermaid -flowchart LR - subgraph Ingest["Camera Ingest"] - C1["Cameras → Surveillance Platform"] - C2["Cameras → VPN"] - C3["Cameras → HTTPS"] - end - GW["Hybrid Array / S3 Gateway"] - NET(["Dedicated Line · VPN · HTTPS"]) - subgraph OSS["RustFS Object Storage"] - PROC["Transcode · Capture · Playback"] - STD["Standard Storage"] - IA["Infrequent Storage"] - ARC["Archive Storage"] - PROC --- STD - PROC --- IA - PROC --- ARC - end - C1 --> GW - GW --> NET - C2 --> NET - C3 --> NET - NET -->|Record / Playback| OSS - classDef muted fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#1e293b; - classDef server fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b; - classDef svc fill:#eef2ff,stroke:#6366f1,stroke-width:2px,color:#1e293b; - classDef store fill:#dcfce7,stroke:#22c55e,stroke-width:2px,color:#1e293b; - class C1,C2,C3 muted - class GW server - class NET svc - class PROC svc - class STD,IA,ARC store -``` - -## Why Choose RustFS - -### Controllable Costs - -Elastic expansion with lifecycle policies keeps cold footage on the cheapest suitable media. - -### Fast Response - -High-throughput distributed architecture keeps ingest and playback responsive as camera counts grow. For representative performance figures, see [RustFS vs other storage products](/concepts/comparison). - -### Encrypted Upload and Storage - -Server-side encryption protects footage against leakage and unauthorized distribution, and helps platforms meet privacy protection regulations. - -### Tamper Protection - -Object locking and versioning protect original footage from tampering and accidental deletion, preserving its evidentiary value. diff --git a/content/features/worm/index.md b/content/features/worm/index.md deleted file mode 100644 index f27f8af7..00000000 --- a/content/features/worm/index.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: "Object Immutability" -description: "Retention rules enforce WORM protection. Retention policies specify the retention period set on object versions, either explicitly or through bucket default…" ---- - -## Object Retention - -Retention rules enforce WORM protection. Retention policies specify the retention period set on object versions, either explicitly or through bucket default settings. Default lock configurations apply to subsequently created objects. - -When using bucket default settings, a duration in days or years defines the protection period. New objects inherit this duration. - -Retention periods can be explicitly set for object versions. Explicit retention periods specify a "retain until date". - -After the retention period expires, the object version can be deleted unless a legal hold is active. - -Explicit retention mode settings override default bucket settings. - -Retention periods can be extended by submitting a new lock request. - -## Governance Mode - -Governance mode prevents objects from being deleted by standard users. Users with special permissions (e.g., `s3:BypassGovernanceRetention`) can modify retention settings or delete objects. - -## Compliance Mode - -Compliance mode ensures that no one (including the root user) can delete objects during the retention period. - -## Legal Hold - -Legal hold provides indefinite WORM protection without an expiration date. It can only be removed by authorized users. - -When objects have retention or legal hold policies, they continue to be versioned. Replication operations do not transfer retention and legal hold settings. - -## RustFS Data Immutability Meets Cohasset Certification Standards - -RustFS meets Cohasset Associates standards for object locking, retention, and legal hold, including SEC Rule 17a-4(f), FINRA Rule 4511, and CFTC Regulation 1.31. - -Download the Cohasset Associates report for details on configuring RustFS to meet regulatory requirements. diff --git a/content/index.mdx b/content/index.mdx index ec11ad29..e956dbf3 100644 --- a/content/index.mdx +++ b/content/index.mdx @@ -27,8 +27,8 @@ RustFS is a high-performance, distributed object storage system written in Rust, How it works and how it measures up against alternatives. - - The API compatibility matrix — what works out of the box. + + The boundaries you should know before committing. diff --git a/content/meta.json b/content/meta.json index c093d285..ee1b30ab 100644 --- a/content/meta.json +++ b/content/meta.json @@ -49,52 +49,14 @@ "developer/examples", "[MinIO Client (mc)](/developer/mc)", "[MCP Server](/developer/mcp)", + "[Open Source License](/developer/license)", "---Reference---", "[Environment Variables](/reference/environment-variables)", "[CLI](/reference/cli)", "[Ports & Health Endpoints](/reference/ports)", "[Metrics](/reference/metrics)", - "[S3 Compatibility](/features/s3-compatibility)", "[Usage Limits](/concepts/limit)", - "[Glossary](/concepts/glossary)", - - "---Features---", - "[Distributed Architecture](/features/distributed)", - "[Versioning](/features/versioning)", - "[Object Immutability (WORM)](/features/worm)", - "[Replication](/features/replication)", - "[Data Encryption](/features/encryption)", - "[Lifecycle Management](/features/lifecycle)", - "[Logging & Auditing](/features/logging)", - "[Small File Optimization](/features/small-file)", - - "---Solutions---", - "[Modern Data Lake](/features/data-lake)", - "[AI & Machine Learning](/features/ai)", - "[Hybrid & Multi-Cloud](/features/cloud-native)", - "[Big Data Storage-Compute Separation](/features/hdfs)", - "[SQL Server 2022](/features/sql-server)", - "[Quantitative Trading](/features/quantitative-trading)", - "[Manufacturing](/features/industry)", - "[Cold Archiving](/features/cold-archiving)", - "[Video Storage](/features/video)", - "[Sovereign Cloud & Compliance](/features/domestic)", - "[Ecosystem Integrations](/features/integration)", - "[Kubernetes · Common Capabilities](/features/kubernetes-common)", - "[Kubernetes · Alibaba Cloud ACK](/features/aliyun)", - "[Kubernetes · AWS EKS](/features/aws-elastic)", - "[Kubernetes · Huawei Cloud CCE](/features/huaweicloud)", - "[Kubernetes · Tencent Cloud TKE](/features/qcloud)", - "[Kubernetes · Red Hat OpenShift](/features/openshift)", - "[Kubernetes · VMware Tanzu](/features/tanzu)", - "[Backup · Veeam](/features/veeam)", - "[Backup · Commvault](/features/commvault)", - "[Bare Metal & Virtualization](/features/baremetal)", - - "---About---", - "[About Us](/about)", - "[Trademark](/trademark)", - "[Open Source License](/developer/license)" + "[Glossary](/concepts/glossary)" ] } diff --git a/content/trademark/images/logo-black-bg.svg b/content/trademark/images/logo-black-bg.svg deleted file mode 100644 index 9b67231c..00000000 --- a/content/trademark/images/logo-black-bg.svg +++ /dev/null @@ -1,13 +0,0 @@ - - - logo 2备份 - - - - - - - - - - \ No newline at end of file diff --git a/content/trademark/images/logo-gray-bg.svg b/content/trademark/images/logo-gray-bg.svg deleted file mode 100644 index 15a19ac6..00000000 --- a/content/trademark/images/logo-gray-bg.svg +++ /dev/null @@ -1,13 +0,0 @@ - - - logo 2备份 2 - - - - - - - - - - \ No newline at end of file diff --git a/content/trademark/images/logo-pink-bg.svg b/content/trademark/images/logo-pink-bg.svg deleted file mode 100644 index 4524cb6f..00000000 --- a/content/trademark/images/logo-pink-bg.svg +++ /dev/null @@ -1,13 +0,0 @@ - - - logo 2备份 3 - - - - - - - - - - \ No newline at end of file diff --git a/content/trademark/images/logo-white-bg.svg b/content/trademark/images/logo-white-bg.svg deleted file mode 100644 index 02b5d0a0..00000000 --- a/content/trademark/images/logo-white-bg.svg +++ /dev/null @@ -1,11 +0,0 @@ - - - logo 2 - - - - - - - - \ No newline at end of file diff --git a/content/trademark/images/trademark-guidelines.svg b/content/trademark/images/trademark-guidelines.svg deleted file mode 100644 index 0600e37a..00000000 --- a/content/trademark/images/trademark-guidelines.svg +++ /dev/null @@ -1,65 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - RustFS 官方配色 - 我们的二次色彩可用于增加布局的趣味性。这些颜色 - - 的色调或色调是允许的。 - rustfs.com/html/trademark.html - - diff --git a/content/trademark/index.md b/content/trademark/index.md deleted file mode 100644 index 00c10dfe..00000000 --- a/content/trademark/index.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -title: "Trademark Download and Usage" -description: "All images on this page are provided for your use only when representing RustFS in your product architecture diagrams or support lists. When using the…" ---- - -All images on this page are provided for your use only when representing RustFS in your product architecture diagrams or support lists. When using the images, please indicate that the image source is the official RustFS website. Images may be proportionally resized but cannot be modified in any other way. - -RustFS® and the rustfs logo® are registered trademarks of RustFS. No one may use these images for any other purpose without written consent from RustFS. - -For any other matters not covered here, please contact us via email at . - -## RustFS Logo - -### White Background Version - -![RustFS Logo - White Background](./images/logo-white-bg.svg) - -### Different Background Versions - -| Black Background | Gray Background | Pink Background | -|------------------|-----------------|-----------------| -| ![RustFS Logo - Black Background](./images/logo-black-bg.svg) | ![RustFS Logo - Gray Background](./images/logo-gray-bg.svg) | ![RustFS Logo - Pink Background](./images/logo-pink-bg.svg) | - -### Download Links - -🔗 [Download All](https://rustfs.com/images/trademark/logo.zip) - -## Trademark Usage Guidelines - -![Trademark Usage Guidelines](./images/trademark-guidelines.svg) - -## Trademark Usage Policy Document - -### Version Information - -v1.0 - -### Introduction - -RustFS is a high-performance distributed object storage system. It is software-defined, runs on industry-standard hardware, and is 100% commercially friendly open source under the Apache Version 2.0 open source license. - -This document outlines the RustFS project's policy regarding the use of its trademarks. Any use of RustFS trademarks must comply with this policy. For the purposes of this policy, "trademarks" refers to RustFS's word marks, service marks, logos, trade dress, product names, services, business and company names. - -RustFS trademarks include RustFS and its derivative sub-products. - -As firm believers in the spirit of free software and respected community members, we want users, distributors, and other community members to be able to widely use and improve our code, which is distributed under open source licenses. While our code is open source, RustFS wants to ensure that its trademarks remain reliable indicators of the quality and source that users expect from us. Strict enforcement of our trademark rights is also very important to protect our users from those who use trademarks fraudulently. This means that while you have considerable freedom to redistribute and modify our software, you must comply with trademark law and this policy, even for open source software. Balancing these two competing interests is not easy. We rely on our users, customers, and community to help us achieve this balance. - -### General Guidelines - -The fundamental basis of RustFS's trademark policy is the general law of trademarks. - -RustFS is designed to be used and extended, and RustFS recognizes that community members may need a way to identify RustFS-based products, but you must ensure that consumers are not confused about whether they are official (meaning approved by RustFS). - -Your use of RustFS trademarks must always avoid confusion. People should always know who they are dealing with and where the software they download comes from. Websites and software not made or officially authorized by RustFS should not directly or indirectly suggest that they are made or officially authorized by RustFS. - -If you have any doubts or need clarification, please email - -### What You Can Do - -You may distribute the unmodified official binaries downloaded from to anyone in any way, subject to the relevant terms of applicable law and licenses, without obtaining any further permission from RustFS. However, you may not remove or alter any RustFS trademarks. In your websites or other materials, you may truthfully state that the software you provide is an unmodified version of RustFS, keeping in mind the general guidelines regarding the use of RustFS trademarks detailed in this policy document. We recommend that if you choose to provide website visitors with the opportunity to download RustFS binaries, you can help ensure faster and more reliable downloads by linking to our website for downloads. - -You may use RustFS trademarks in marketing and other promotional materials. This includes indicating that an individual or organization is shipping or selling RustFS products. Of course, any use of RustFS trademarks must comply with the fundamental requirement that its use must not cause confusion. - -**RustFS-related services:** If you provide RustFS-related services, you may use RustFS trademarks when describing and promoting your services, as long as you do not violate the general guidelines for using RustFS trademarks or do anything that might mislead customers into believing that RustFS has any direct relationship with your organization. - -**Logos and merchandise:** You may make T-shirts, desktop wallpapers, or baseball caps with RustFS logos, but only for yourself and your friends (i.e., people from whom you will not receive any valuable return). - -You may use RustFS trademarks to truthfully reference and/or link to unmodified RustFS programs, products, services, and technologies. - -### What You Cannot Do - -1. You cannot put the RustFS logo on any product you commercially produce. -2. You may not modify RustFS trademarks, abbreviate them, or combine them with any other symbols, words, or images, or incorporate them into slogans or catchphrases. -3. You may not create modified versions of RustFS Logos for any purpose. -4. You may not use RustFS trademarks in a way that falsely suggests that RustFS is associated with, sponsors, endorses, or approves your product or service. -5. You may not use RustFS trademarks for any form of commercial use unless such use is limited to truthful and descriptive references. -6. You cannot use RustFS trademarks in the names and titles of social media accounts. - -You may modify RustFS software under open source license terms, but you may not redistribute your modifications under any RustFS trademarks. For example, your product or website should not say "Based on RustFS." Instead, to be completely accurate, you should describe it as "Based on RustFS technology" or "Contains RustFS source code." You must also change the name of the product and binaries to reduce the likelihood that users of the modified software will be misled into believing it is native RustFS or affiliated with us. - -### What Permissions You Need - -If you plan to use RustFS trademarks as website icons, you need to request permission. - -**Domain names:** If you want to include all or part of RustFS trademarks in domain names, you must obtain written permission from RustFS. Almost any use of RustFS trademarks in domain names can confuse consumers and therefore violates the general requirement that the use of RustFS trademarks must not cause confusion. - -### How to Use Our Trademarks - -1. **Correct form** - RustFS trademarks should be used in their accurate form - neither abbreviated nor combined with any other words. -2. **Accompanying symbols** - The first or most prominent mention of RustFS trademarks should be accompanied by a symbol indicating whether the trademark is a registered trademark ("®") or an unregistered trademark ("™"); -3. **Attribution statement** - The following statement should appear somewhere near the use of RustFS trademarks (at least on the same page): "[Trademark] is a ["registered," if applicable] trademark of RustFS Corporation"; -4. **Distinguishable** - Trademarks should be distinguished from surrounding text, which can be distinguished by capitalization, italics, bold, or underlining. - -You may not alter any RustFS logos except to scale them. This means you may not add decorative elements, change colors, change proportions, distort it, add elements, or combine it with other logos. However, when the context requires the use of black and white graphics and the logo is colored, you may reproduce the logo in a way that produces a black and white image. - -### Questions - -RustFS has tried to make its trademark policy as comprehensive as possible. If you are considering using RustFS trademarks not covered by this policy and are unsure whether such use violates RustFS guidelines, please contact us at . - -If RustFS determines at any time in its sole discretion that your use of any of our trademarks violates this policy, we may revoke your license to use it, and you must immediately stop all use of that trademark. - -This policy may be updated from time to time. Refer to this page for all updates.