Laravel Batch Upload & Export

High-performance, asynchronous CSV batch import and export system powered by Laravel Bus Batching, Queue Workers, and Real-Time Progress Polling.

✨ Features

• Chunked CSV Import — Efficiently processes massive CSV files by splitting data into micro-chunks and dispatching them as batched background jobs.
• Real-Time Progress Polling — Live frontend progress bar that queries /batch-status/{batchId} to monitor batch completion percentage, remaining jobs, and failure states.
• Asynchronous Queue Processing — Prevents HTTP timeouts by offloading database insertions to background queue workers (php artisan queue:work).
• CSV Data Export — Fast dataset exporter using CsvExportService to stream records into downloadable CSV files.
• Secure File Storage — Utilizes Laravel's private storage disk to securely buffer uploaded files prior to job processing.
• Clean Service Architecture — Decoupled controller logic using dedicated CsvImportService, CsvExportService, and ImportSaleCSVProcess job classes.

📊 Batch Processing Flow

The import pipeline isolates long-running CSV parsing from the web server thread using Laravel's queue driver and job batching:

sequenceDiagram
    autonumber
    actor User
    participant Browser
    participant Controller as CsvController
    participant Storage
    participant Service as CsvImportService
    participant Queue as Laravel Queue / Bus
    participant DB as MySQL Database

    User->>Browser: Upload CSV File
    Browser->>Controller: POST /csv/import
    Controller->>Storage: Save file to private storage
    Controller->>Service: Trigger import(filePath)
    Service->>Queue: Split CSV into chunks & Bus::batch(jobs)
    Queue-->>Controller: Return Batch ID
    Controller-->>Browser: Render importing-file view with Batch ID
    
    loop Every 1-2 Seconds
        Browser->>Controller: GET /batch-status/{batchId}
        Controller->>Queue: Bus::findBatch(batchId)
        Queue-->>Controller: Return % progress & completion status
        Controller-->>Browser: Return JSON status update
        Browser->>Browser: Update UI Progress Bar
    end

    par Background Processing
        Queue->>DB: Worker executes ImportSaleCSVProcess jobs
    end

Loading

🛠 Architecture & Endpoints

Method	Endpoint	Controller Action	Purpose
GET	/	ViewController@home	Renders dashboard / landing page
GET	/csv/import	ViewController@import	Renders CSV upload form interface
POST	/csv/import	CsvController@import	Validates CSV, stores file, dispatches batch, returns status page
GET	/csv/export	CsvController@export	Streams database records into a downloadable CSV file
GET	/batch-status/{batchId}	CsvController@getBatchStatus	Returns JSON batch progress (percent, finished, failed)

Prerequisites

• PHP 8.2+ and Composer
• MySQL 8.0+ (or PostgreSQL/SQLite)
• Redis or Database Queue Connection (required for background batch processing)

Quick Start

# 1. Clone the repo
git clone https://github.com/ttncode/laravel-batch-upload.git
cd laravel-batch-upload

# 2. Install PHP and NPM dependencies
composer install
npm install && npm run build

# 3. Create your environment file
cp .env.example .env
# Edit .env and configure your DB and QUEUE_CONNECTION (database or redis)

# 4. Generate app key & create database tables
php artisan key:generate
php artisan migrate

# 5. Create job batching table (if not already published)
php artisan queue:batches-table
php artisan migrate

# 6. Start queue worker (in a separate terminal)
php artisan queue:work

# 7. Start local server
php artisan serve

Open http://localhost:8000/csv/import in your browser to upload sample CSV files.

Environment Variables

Configure your .env file with the following database and queue parameters:

Variable	Required	Default	Description
APP_ENV	Yes	local	Application environment
DB_CONNECTION	Yes	mysql	Primary database driver
DB_HOST	Yes	127.0.0.1	Database host
DB_PORT	Yes	3306	Database port
DB_DATABASE	Yes	laravel_batch_upload	Database name
QUEUE_CONNECTION	Yes	database	Queue driver (database or redis recommended)

Development & Testing

# Run feature and unit tests using Pest / PHPUnit
php artisan test

# Run queue listener in debug mode
php artisan queue:listen --verbose

# Run code style checks
./vendor/bin/pint

Tech Stack

Laravel 11 · PHP 8.2 · Laravel Bus Batching · Blade · Tailwind CSS · Vite · MySQL