🛡️ DataGuard Engine v1.2.0 — Clean Architecture Edition

DataGuard Engine is an anonymization and data loss prevention (DLP) engine featuring Streaming I/O, Zero-Garbage architecture, Virtual Threads (Project Loom), and strict Clean Architecture / SOLID compliance. It operates in two modes: PII file sanitization and DLP auditing for Git pre-commit hooks and CI/CD pipelines.

v1.2.0 Highlights: Pure Core engine (zero System.err side effects), RuleProvider SPI enforced via OCP, LicenseValidator SPI wired into CLI gate, DlpAuditResult domain records, DRY-compliant single source of truth for all regex patterns, and a 24-test Red Team bypass suite.

Learning Project: This project was built to learn, practice and deepen skills in DevSecOps, Java 21 (Virtual Threads / Project Loom), Clean Architecture, SOLID principles, high-performance file processing, secure coding practices, JUnit 5 testing, Git-based security automation, and AI-assisted software engineering. It also serves as a hands-on exploration of how modern AI coding agents can be leveraged to design, generate, review, and strengthen automated test suites, including adversarial (Red Team) security testing.

📦 Project Structure

src/
├── main/java/com/devdiego/
│   ├── Main.java                                  # CLI Boundary — owns all presentation (System.out/err)
│   └── dataguard/
│       ├── config/
│       │   ├── RuleProvider.java                  # SPI interface for rule catalogs (OCP)
│       │   ├── StandardPiiRuleProvider.java       # 4 PII rules: EMAIL, CREDIT_CARD, IPV4, CELLPHONE
│       │   └── DlpAuditRuleProvider.java          # 4 DLP rules: AWS, GitHub, PrivateKey, JWT
│       ├── core/
│       │   ├── DataGuardEngine.java               # Pure engine: sanitizeStream / auditDLP / auditConcurrent
│       │   └── license/
│       │       ├── LicenseValidator.java          # SPI interface for license validation (OCP)
│       │       └── CommunityLicenseValidator.java  # Community Edition implementation (MIT)
│       └── model/
│           ├── Rule.java                          # Immutable record (name, pattern, replacementTag)
│           ├── DlpViolation.java                   # Immutable record (ruleName, file, lineNumber)
│           └── DlpAuditResult.java                 # Immutable record (leakDetected, violations)
└── test/java/com/devdiego/dataguard/core/
    ├── DataGuardEngineTest.java                   # 26 functional tests (JUnit 5)
    └── DlpBypassTest.java                         # 24 Red Team adversarial bypass tests

⚙️ Architecture — v1.2.0

Clean Architecture Layers

┌─────────────────────────────────────────────────────────────┐
│                    Presentation Layer                        │
│  Main.java (CLI) — owns System.out, System.err, System.exit │
├─────────────────────────────────────────────────────────────┤
│                    Core / Domain Layer                       │
│  DataGuardEngine — pure functions, returns domain records    │
│  RuleProvider (SPI)        LicenseValidator (SPI)             │
├─────────────────────────────────────────────────────────────┤
│                    Model / Domain Layer                       │
│  Rule · DlpViolation · DlpAuditResult (immutable records)     │
├─────────────────────────────────────────────────────────────┤
│                    Infrastructure / Config Layer             │
│  StandardPiiRuleProvider · DlpAuditRuleProvider               │
│  CommunityLicenseValidator                                   │
└─────────────────────────────────────────────────────────────┘

Dependency Rule: Dependencies point only inward. The Core layer has zero references to System.err, System.out, or any presentation concern.

Components

Component	Responsibility
`Rule` (record)	Defines a compiled regex pattern, its name, and replacement tag. Immutable, thread-safe, validated on construction.
`RuleProvider` (SPI)	Interface for pluggable rule catalogs. Enforces Open/Closed Principle — new providers can be added without modifying the engine.
`StandardPiiRuleProvider`	Provider with 4 PII rules: EMAIL, CREDIT_CARD, IPV4, CELLPHONE. Used by the sanitization pipeline.
`DlpAuditRuleProvider`	Provider with 4 DLP rules: AWS_ACCESS_KEY, GITHUB_TOKEN, PRIVATE_KEY, JWT_TOKEN. Used by the DLP audit pipeline.
`DlpViolation` (record)	Immutable record capturing a single violation (ruleName, file, lineNumber).
`DlpAuditResult` (record)	Immutable aggregate result with leakDetected flag and violations list. Returned by the engine — no System.err side effects.
`LicenseValidator` (SPI)	Interface for pluggable license validation. Enforces OCP — swap implementations without modifying the CLI.
`CommunityLicenseValidator`	Community Edition (MIT) implementation. Always validates `true`.
`DataGuardEngine`	Pure core with three modes: streaming sanitization, DLP auditing with short-circuit, and concurrent auditing with Virtual Threads. Accepts `RuleProvider` SPI. Zero presentation side effects.
`Main`	CLI with subcommands: `--dlp-audit <file>` (Git hook) and `--sanitize <src> <dst>` (file cleanup). Gates execution behind `LicenseValidator`. Owns all presentation (System.out/System.err).

Mode 1: Streaming Sanitization (`sanitizeStream`)

Original line → [EMAIL] → [CREDIT_CARD] → [IPV4] → [CELLPHONE] → Clean line

Chained pipeline that recycles StringBuilder and Matcher. Each rule receives the output of the previous one. Inline counting without a second pass. Rules sourced from the injected RuleProvider SPI.

Mode 2: DLP Audit (`auditDLP`) — Short-Circuit

Line 1 → [AWS_ACCESS_KEY] → [GITHUB_TOKEN] → [PRIVATE_KEY] → [JWT_TOKEN] → clean
Line 2 → [AWS_ACCESS_KEY] → [GITHUB_TOKEN] → [PRIVATE_KEY] → [JWT_TOKEN] → clean
Line 3 → [AWS_ACCESS_KEY] → MATCH! → return DlpAuditResult (closes FD, reads no further)

On first match, immediately aborts without reading the rest of the file. Returns a DlpAuditResult with the first DlpViolation. Ideal for CI/CD where time is critical.

Mode 3: Concurrent Auditing (`auditMultipleFilesConcurrently`) — Java 21

File 1 ──→ [Virtual Thread 1] ──→ auditDLP() ──→ DlpAuditResult
File 2 ──→ [Virtual Thread 2] ──→ auditDLP() ──→ DlpAuditResult
File 3 ──→ [Virtual Thread 3] ──→ auditDLP() ──→ DlpAuditResult
   ...
File N ──→ [Virtual Thread N] ──→ auditDLP() ──→ DlpAuditResult
                                              ↓
                         Aggregated DlpAuditResult (all violations)

Uses Executors.newVirtualThreadPerTaskExecutor() from Project Loom. Each Virtual Thread weighs ~KB, enabling auditing of thousands of files simultaneously without saturating the OS. All DlpViolation objects from virtual threads are aggregated into a thread-safe Collections.synchronizedList and returned as a unified DlpAuditResult.

Performance Comparison

Technique	Naïve approach	DataGuard Engine v1.2.0
Read	`Files.readString` (all in RAM)	`BufferedReader` (streaming O(1))
Regex	`String.replaceAll` (millions of Strings)	`Matcher.appendReplacement` (recycled buffers)
Memory	`OutOfMemoryError` with >1 GB	~10–15 MB constant
Count	Second pass over the file	Inline counting in a single pass
DLP	Always reads entire file	Short-circuit: returns on first match
Concurrency	Platform Threads (~1 MB each)	Virtual Threads (~KB each, Project Loom)
Architecture	Core writes to `System.err`	Pure Core — returns `DlpAuditResult` domain records
Extensibility	Static factory (`RuleConfigurator`)	`RuleProvider` SPI (OCP-compliant)

🚀 Usage Guide

Requirements

Java 21 or higher
Apache Maven 3.8 or higher

1. Clone and build

git clone https://github.com/your-user/java-anonimo-engine.git
cd java-anonimo-engine
mvn clean compile

2. Run tests

mvn clean test
# Tests run: 50, Failures: 0, Errors: 0, Skipped: 0
# BUILD SUCCESS

3. Sanitization Mode (clean PII from files)

java -jar dataguard-engine.jar --sanitize logs.txt clean_logs.txt

Output:

License: Community Edition (MIT License - Free for all uses)
[DataGuard Engine] Sanitizing file stream...
Sanitization complete. Audit metrics:
 -> EMAIL: 2 redactions
 -> IPV4: 2 redactions
 -> CREDIT_CARD: 1 redactions
 -> CELLPHONE: 0 redactions

4. DLP Audit Mode (Git pre-commit hook / CI/CD)

# Audit a file for secrets
java -jar dataguard-engine.jar --dlp-audit config.env

If a secret is found (exit code 1):

License: Community Edition (MIT License - Free for all uses)
🛡️ [DataGuard Engine v1.2.0] Starting DLP Audit on: config.env
🚨 [DLP VIOLATION] Rule: AWS_ACCESS_KEY | File: config.env | Line: 3
X / COMMIT REJECTED: Secrets or sensitive data detected in code.

If clean (exit code 0):

License: Community Edition (MIT License - Free for all uses)
🛡️ [DataGuard Engine v1.2.0] Starting DLP Audit on: clean_service.java
OK / DLP Audit Passed. Zero leaks detected.

Exit codes:

Code	Meaning
0	Operation successful (audit passed / sanitize complete)
1	DLP violation detected or I/O error
2	License validation failed

Git Hook Integration

# .git/hooks/pre-commit
#!/bin/bash
for file in $(git diff --cached --name-only); do
    java -jar dataguard.jar --dlp-audit "$file" || exit 1
done

⚠️ Security Note: Pre-commit hooks are advisory and can be bypassed with git commit --no-verify. For production enforcement, add a server-side pre-receive hook as a backstop. See the Red Team Analysis section for known bypass vectors.

🔧 Rule Catalogs

Standard PII Rules (`StandardPiiRuleProvider`)

Rule	Detects	Tag
`EMAIL`	`user@domain.com`	`[EMAIL_REDACTED]`
`CREDIT_CARD`	`1234-5678-9012-3456`	`[CARD_REDACTED]`
`IPV4`	`192.168.1.50`	`[IP_REDACTED]`
`CELLPHONE`	`+57 3001234567`	`[CELLPHONE_REDACTED]`

DLP Secrets Rules (`DlpAuditRuleProvider`)

Rule	Detects	Example
`AWS_ACCESS_KEY`	`AKIA...` (20 chars)	`AKIA_FAKE_FAKE_FAKE_`
`GITHUB_TOKEN`	`ghp_...` / `github_pat_...`	`ghp_abc123...`
`PRIVATE_KEY`	`-----BEGIN RSA PRIVATE KEY-----`	Private key headers
`JWT_TOKEN`	`eyJ...` (base64)	`eyJhbGciOiJIUzI1NiJ9.eyJzdWIi...`

🧩 Extensibility

Custom RuleProvider (OCP)

Create your own RuleProvider implementation to add custom rule catalogs without modifying the engine (Open/Closed Principle):

public class MyCustomRuleProvider implements RuleProvider {
    private static final List<Rule> RULES = List.of(
        new Rule("DNI", Pattern.compile("\\b\\d{8}[A-Z]\\b"), "[DNI_REDACTED]"),
        new Rule("API_KEY", Pattern.compile("sk-[a-zA-Z0-9]{32}"), "[API_KEY_BLOCKED]")
    );

    @Override public List<Rule> getRules() { return RULES; }
    @Override public String getProviderName() { return "Custom Rule Provider"; }
}

// Sanitization
Map<String, Long> metrics = DataGuardEngine.sanitizeStream(source, target, new MyCustomRuleProvider());

// DLP Audit — returns DlpAuditResult (pure data, no System.err)
DlpAuditResult result = DataGuardEngine.auditDLP(file, new MyCustomRuleProvider());
if (result.leakDetected()) {
    result.violations().forEach(v -> System.err.printf("🚨 %s | %s | line %d%n",
        v.ruleName(), v.file().getFileName(), v.lineNumber()));
}

// Concurrent DLP (thousands of files)
DlpAuditResult result = DataGuardEngine.auditMultipleFilesConcurrently(fileList, new MyCustomRuleProvider());

Custom LicenseValidator (OCP)

public class EnterpriseLicenseValidator implements LicenseValidator {
    @Override
    public boolean validate() throws SecurityException {
        // Check license server, verify signature, etc.
        return verifyEnterpriseLicense();
    }

    @Override
    public String getLicenseType() {
        return "Enterprise Edition (Commercial License)";
    }
}

// In Main.java — swap one line:
LicenseValidator license = new EnterpriseLicenseValidator();

🧪 Testing

Functional Test Suite — 26 tests

Test suite using JUnit 5 and @TempDir:

Test Group	Tests	What it validates
DLP Audit — Single File Detection	6	AWS key, GitHub token, private key, JWT, clean files, empty files
DLP Audit — Short-Circuit Optimization	3	Stop at line 1, stop at line 50, no false positives on near-miss patterns
DLP Audit — Concurrent (Virtual Threads)	3	Leak detection across files, all-clean, 100 files without crash
Sanitization — PII Redaction Pipeline	5	All PII types, accurate counts, clean line preservation, empty file, 10K-line file
Custom Rules & Edge Cases	5	Custom Stripe key, custom DNI, null/blank name rejection, null pattern rejection, default tag
Enterprise Scenario — CI/CD Simulation	2	Pre-commit hook (20 staged files), CI/CD log sanitization (500 lines)
License Validator SPI	2	Community license validates, non-blank license type

Red Team Bypass Suite — 24 tests

Adversarial tests proving regex bypass vectors and pre-commit hook weaknesses:

Test Group	Tests	What it validates
AWS Access Key — Bypass Vectors	6	Lowercase prefix, split lines, runtime assembly, base64, 15-char edge, zero-width space
GitHub Token — Bypass Vectors	4	35-char body, hyphen in body, hex encoding, runtime assembly
JWT Token — Bypass Vectors	7	Space instead of dot, split lines, URL-encoded dots, zero-width space, hex, URL-encoded `ey`, control test
Pre-Commit Hook — Filesystem Bypass	6	`--no-verify`, symlink, word-splitting, binary null bytes, UTF-16, OOM long line
ReDoS — Catastrophic Backtracking	1	Alternating `A.` pattern × 10,000

mvn clean test
# Tests run: 50, Failures: 0, Errors: 0, Skipped: 0
# BUILD SUCCESS

🔴 Red Team Security Analysis

The DlpBypassTest.java suite empirically demonstrates 24 bypass vectors against the current DLP regex patterns and pre-commit hook. Key findings:

Regex Bypasses

Pattern	Bypass Examples	Severity
`AKIA[0-9A-Z]{16}`	Lowercase `akia`, split across lines, base64-encoded, zero-width space	🔴 Critical
`(ghp\|github_pat)_[a-zA-Z0-9]{36,82}`	35-char body, hyphen in body, hex-encoded, runtime assembly	🔴 Critical
`ey[A-Za-z0-9-_=]+\.[A-Za-z0-9-_=]+...`	URL-encoded dots, zero-width space in `ey`, hex-encoded, space instead of dot	🔴 Critical

Pre-Commit Hook Bypasses

Vector	Mechanism	Severity
`git commit --no-verify`	Hooks are advisory, not enforced	🔴 Critical
Symlink to clean file	Engine follows symlink; real secret never staged	🔴 Critical
UTF-16 encoded file	BufferedReader (UTF-8) reads garbled chars	🔴 Critical
Binary file with null bytes	Null bytes break ASCII sequence	🔴 High

Recommended Mitigations

Server-side pre-receive hook — backstop for --no-verify bypass
Multi-line sliding window — detect secrets split across lines
Encoding detection — detect UTF-16/UTF-32 BOMs
Case-insensitive regex — add Pattern.CASE_INSENSITIVE for AWS keys
Flexible quantifiers — {16} → {15,20}, {36,82} → {30,90}
Max line length cap — prevent OOM on minified files
Symlink detection — flag staged symlinks

🐳 Docker

# STAGE 1: Build & Compile
FROM eclipse-temurin:21-jdk-alpine AS builder
WORKDIR /build
COPY pom.xml .
RUN apk add --no-cache maven && mvn dependency:go-offline -B
COPY src ./src
RUN mvn clean package -DskipTests

# STAGE 2: Production Runtime
FROM eclipse-temurin:21-jre-alpine AS runtime
WORKDIR /app
RUN addgroup -S dataguard && adduser -S dataguard -G dataguard
USER dataguard
COPY --from=builder /build/target/java-anonimo-engine-1.0-SNAPSHOT.jar ./dataguard-engine.jar
VOLUME ["/data", "/etc/dataguard"]
ENV JAVA_OPTS="-XX:+UseG1GC -XX:MaxRAMPercentage=75.0"
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -jar dataguard-engine.jar $0 $@"]
CMD ["--help"]

# Build and run
docker build -t dataguard-engine .
docker run --rm -v $(pwd):/data dataguard-engine --dlp-audit /data/config.env

Navigation Menu

🛡️ DataGuard Engine v1.2.0 — Clean Architecture Edition

📦 Project Structure

⚙️ Architecture — v1.2.0

Clean Architecture Layers

Components

Mode 1: Streaming Sanitization (`sanitizeStream`)

Mode 2: DLP Audit (`auditDLP`) — Short-Circuit

Mode 3: Concurrent Auditing (`auditMultipleFilesConcurrently`) — Java 21

Performance Comparison

🚀 Usage Guide

Requirements

1. Clone and build

2. Run tests

3. Sanitization Mode (clean PII from files)

4. DLP Audit Mode (Git pre-commit hook / CI/CD)

Git Hook Integration

🔧 Rule Catalogs

Standard PII Rules (`StandardPiiRuleProvider`)

DLP Secrets Rules (`DlpAuditRuleProvider`)

🧩 Extensibility

Custom RuleProvider (OCP)

Custom LicenseValidator (OCP)

🧪 Testing

Functional Test Suite — 26 tests

Red Team Bypass Suite — 24 tests

🔴 Red Team Security Analysis

Regex Bypasses

Pre-Commit Hook Bypasses

Recommended Mitigations

🐳 Docker

📄 License

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

🛡️ DataGuard Engine v1.2.0 — Clean Architecture Edition

📦 Project Structure

⚙️ Architecture — v1.2.0

Clean Architecture Layers

Components

Mode 1: Streaming Sanitization (sanitizeStream)

Mode 2: DLP Audit (auditDLP) — Short-Circuit

Mode 3: Concurrent Auditing (auditMultipleFilesConcurrently) — Java 21

Performance Comparison

🚀 Usage Guide

Requirements

1. Clone and build

2. Run tests

3. Sanitization Mode (clean PII from files)

4. DLP Audit Mode (Git pre-commit hook / CI/CD)

Git Hook Integration

🔧 Rule Catalogs

Standard PII Rules (StandardPiiRuleProvider)

DLP Secrets Rules (DlpAuditRuleProvider)

🧩 Extensibility

Custom RuleProvider (OCP)

Custom LicenseValidator (OCP)

🧪 Testing

Functional Test Suite — 26 tests

Red Team Bypass Suite — 24 tests

🔴 Red Team Security Analysis

Regex Bypasses

Pre-Commit Hook Bypasses

Recommended Mitigations

🐳 Docker

📄 License

Mode 1: Streaming Sanitization (`sanitizeStream`)

Mode 2: DLP Audit (`auditDLP`) — Short-Circuit

Mode 3: Concurrent Auditing (`auditMultipleFilesConcurrently`) — Java 21

Standard PII Rules (`StandardPiiRuleProvider`)

DLP Secrets Rules (`DlpAuditRuleProvider`)