Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 92 additions & 61 deletions README-NuGet.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,132 @@
## MiniExcel
[![NuGet Version](https://img.shields.io/nuget/v/MiniExcel.svg)](https://www.nuget.org/packages/MiniExcel) 
[![NuGet Downloads](https://img.shields.io/nuget/dt/MiniExcel.svg)](https://www.nuget.org/packages/MiniExcel) 
[![GitHub Stars](https://img.shields.io/github/stars/mini-software/MiniExcel?logo=github)](https://github.com/mini-software/MiniExcel) 
[![Gitee Stars](https://gitee.com/dotnetchina/MiniExcel/badge/star.svg)](https://gitee.com/dotnetchina/MiniExcel) 
[![.NET Version](https://img.shields.io/badge/.NET-%3E%3D%204.5-red.svg)](https://www.nuget.org/packages/MiniExcel) 
[![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mini-software/MiniExcel)

This project is part of the [.NET Foundation](https://dotnetfoundation.org/projects/project-detail/miniexcel) and operates under their code of conduct.

---

### Introduction
MiniExcel is a simple and efficient Excel processing tool for .NET, specifically designed to minimize memory usage.

MiniExcel is simple and efficient to avoid OOM's .NET processing Excel tool.
At present, most popular frameworks need to load all the data from an Excel document into memory to facilitate operations, but this may cause memory consumption problems. MiniExcel's approach is different: the data is processed row by row in a streaming manner, reducing the original consumption from potentially hundreds of megabytes to just a few megabytes, effectively preventing out-of-memory(OOM) issues.

At present, most popular frameworks need to load all the data into the memory to facilitate operation, but it will cause memory consumption problems. MiniExcel tries to use algorithm from a stream to reduce the original 1000 MB occupation to a few MB to avoid OOM(out of memory).
```mermaid
flowchart LR
A1(["Excel analysis<br>process"]) --> A2{{"Unzipping<br>XLSX file"}} --> A3{{"Parsing<br>OpenXML"}} --> A4{{"Model<br>conversion"}} --> A5(["Output"])

![image](https://user-images.githubusercontent.com/12729184/113086657-ab8bd000-9214-11eb-9563-c970ac1ee35e.png)
B1(["Other Excel<br>Frameworks"]) --> B2{{"Memory"}} --> B3{{"Memory"}} --> B4{{"Workbooks &<br>Worksheets"}} --> B5(["All rows at<br>the same time"])

C1(["MiniExcel"]) --> C2{{"Stream"}} --> C3{{"Stream"}} --> C4{{"POCO or dynamic"}} --> C5(["Deferred execution<br>row by row"])

### Features
classDef analysis fill:#D0E8FF,stroke:#1E88E5,color:#0D47A1,font-weight:bold;
classDef others fill:#FCE4EC,stroke:#EC407A,color:#880E4F,font-weight:bold;
classDef miniexcel fill:#E8F5E9,stroke:#388E3C,color:#1B5E20,font-weight:bold;

- Low memory consumption, avoid OOM (out of memory) and full GC
- Supports real time operation of each row of data
- Supports LINQ deferred execution, it can do low-consumption, fast paging and other complex queries
- Lightweight, without Microsoft Office installed, no COM+, DLL size is less than 400KB
- Easy API style to read/write/fill excel
class A1,A2,A3,A4,A5 analysis;
class B1,B2,B3,B4,B5 others;
class C1,C2,C3,C4,C5 miniexcel;
```

### Get Started
### Features

- [Import/Query Excel](#getstart1)
- Minimizes memory consumption, preventing out-of-memory (OOM) errors and avoiding full garbage collections
- Enables real-time, row-level data operations for better performance on large datasets
- Supports LINQ with deferred execution, allowing for fast, memory-efficient paging and complex queries
- Lightweight, without the need for Microsoft Office or COM+ components, and a size under 800KB
- Simple and intuitive API to import, export, and template Excel worksheets

- [Export/Create Excel](#getstart2)
### Quickstart

- [Excel Template](#getstart3)
#### Importing

- [Excel Column Name/Index/Ignore Attribute](#getstart4)
You can query worksheets and map the data either to strongly typed classes or dynamic objects:

- [Examples](#getstart5)
```csharp
public class UserAccount
{
public Guid ID { get; set; }
public string Name { get; set; }
public DateTime DateOfBirth { get; set; }
public int Age { get; set; }
public bool Vip { get; set; }
public decimal Points { get; set; }
}

var userRows = MiniExcel.Query<UserAccount>(path);

// or simply

### Installation
var rows = MiniExcel.Query(path);
```

You can install the package [from NuGet](https://www.nuget.org/packages/MiniExcel)
#### Exporting

### Release Notes
There are multiple ways to export data to an Excel document:

Please Check [Release Notes](docs)
```csharp
// From strongly typed objects

### TODO
var values = new[]
{
new { Name = "MiniExcel", Value = 1 },
new { Name = "Github", Value = 2 }
};
MiniExcel.SaveAs(yourPath, values);

Please Check [TODO](https://github.com/mini-software/MiniExcel/projects/1?fullscreen=true)

### Performance
// From anonymous objects

The code for the benchmarks can be found in [MiniExcel.Benchmarks](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/MiniExcel.Benchmarks).
To run all the benchmarks use:
public class TestType
{
public string Name { get; set; }
public int Value { get; set; }
}

```bash
dotnet run -project .\benchmarks\MiniExcel.Benchmarks -c Release -f net9.0 -filter * --join
```
TestType[] values =
[
new TestType { Name = "MiniExcel", Value = 1 },
new TestType { Name = "Github", Value = 2 }
];
MiniExcel.SaveAs(yourPath, values);

Hardware and settings used are the following:
```
BenchmarkDotNet v0.15.0, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
.NET SDK 9.0.300
[Host] : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2
ShortRun : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX2
```

#### Import/Query Excel
//From a IEnumerable<IDictionary<string, object>>

The file used to test performance is [**Test1,000,000x10.xlsx**](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/MiniExcel.Benchmarks/Test1%2C000%2C000x10.xlsx), a 32MB document containing 1,000,000 rows * 10 columns whose cells are filled with the string "HelloWorld".
List<Dictionary<string, object>>() dicts =
[
new Dictionary<string, object> { { "Name", "MiniExcel" }, { "Value", 1 } },
new Dictionary<string, object> { { "Name", "Github" }, { "Value", 2 } }
];
MiniExcel.SaveAs(yourPath, dicts);

| Method | Mean | StdDev | Error | Gen0 | Gen1 | Gen2 | Allocated |
|--------------------------------------|-----------------:|---------------:|-----------------:|------------:|------------:|----------:|--------------:|
| &#39;MiniExcel QueryFirst&#39; | 63.70 μs | 0.337 μs | 6.144 μs | 2.9297 | 2.7669 | - | 49.67 KB |
| &#39;ExcelDataReader QueryFirst&#39; | 5,010,679.51 μs | 53,245.186 μs | 971,390.400 μs | 105000.0000 | 333.3333 | - | 1717272.56 KB |
| &#39;MiniExcel Query&#39; | 9,172,286.91 μs | 12,805.326 μs | 233,616.824 μs | 448500.0000 | 4666.6667 | - | 7327883.36 KB |
| &#39;ExcelDataReader Query&#39; | 10,609,617.09 μs | 29,055.953 μs | 530,088.745 μs | 275666.6667 | 68666.6667 | - | 4504691.87 KB |
| &#39;Epplus QueryFirst&#39; | 13,770,656.24 μs | 45,909.809 μs | 837,565.827 μs | 174333.3333 | 88833.3333 | 4333.3333 | 3700587.76 KB |
| &#39;Epplus Query&#39; | 19,257,306.83 μs | 63,117.956 μs | 1,151,506.486 μs | 452333.3333 | 90500.0000 | 5333.3333 | 8223933.16 KB |
| &#39;ClosedXml Query&#39; | 31,070,263.83 μs | 342,973.671 μs | 6,257,116.502 μs | 401666.6667 | 104166.6667 | 3333.3333 | 6822559.68 KB |
| &#39;ClosedXml QueryFirst&#39; | 31,141,877.48 μs | 21,006.538 μs | 383,237.459 μs | 402166.6667 | 104833.3333 | 3833.3333 | 6738357.8 KB |
| &#39;OpenXmlSDK QueryFirst&#39; | 31,750,686.63 μs | 263,328.569 μs | 4,804,093.357 μs | 374666.6667 | 374500.0000 | 3166.6667 | 6069266.96 KB |
| &#39;OpenXmlSDK Query&#39; | 32,919,119.46 μs | 411,395.682 μs | 7,505,388.691 μs | 374666.6667 | 374500.0000 | 3166.6667 | 6078467.83 KB |

// Directly from a IDataReader

#### Export/Create Excel
using var connection = yourConnectionProvider.GetConnection();
connection.Open();

Logic: create a total of 10,000,000 "HelloWorld" cells Excel document
using var cmd = connection.CreateCommand();
cmd.CommandText = """
SELECT 'MiniExcel' AS "Name", 1 AS "Value"
UNION ALL
SELECT 'Github', 2
""";

| Method | Mean | StdDev | Error | Gen0 | Gen1 | Gen2 | Allocated |
|----------------------------------------------|---------:|---------:|---------:|------------:|------------:|----------:|----------:|
| &#39;MiniExcel Create Xlsx&#39; | 4.427 s | 0.0056 s | 0.1023 s | 251666.6667 | 1833.3333 | 1666.6667 | 3.92 GB |
| &#39;OpenXmlSdk Create Xlsx by DOM mode&#39; | 22.729 s | 0.1226 s | 2.2374 s | 307000.0000 | 306833.3333 | 3833.3333 | 6.22 GB |
| &#39;ClosedXml Create Xlsx&#39; | 22.851 s | 0.0190 s | 0.3473 s | 195500.0000 | 54500.0000 | 4166.6667 | 4.48 GB |
| &#39;Epplus Create Xlsx&#39; | 23.027 s | 0.0088 s | 0.1596 s | 89000.0000 | 17500.0000 | 6000.0000 | 2.51 GB |
using var reader = cmd.ExecuteReader();
MiniExcel.SaveAs(yourPath, reader);

Warning: these results may be outdated. You can find the benchmarks for the latest release [here](https://github.com/mini-software/MiniExcel/tree/master/benchmarks/results).

// From a DataTable

### Documents
var table = new DataTable();
table.Columns.Add("Name", typeof(string));
table.Columns.Add("Value", typeof(int));
table.Rows.Add("MiniExcel", 1);
table.Rows.Add("Github", 2);

https://github.com/mini-software/MiniExcel
MiniExcel.SaveAs(path, table);
```
8 changes: 4 additions & 4 deletions src/MiniExcel/Csv/CsvWriter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,12 @@ private async Task<int> WriteValuesAsync(StreamWriter writer, object values, str
cancellationToken.ThrowIfCancellationRequested();

IMiniExcelWriteAdapter writeAdapter = null;
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
IAsyncMiniExcelWriteAdapter asyncWriteAdapter = null;
#endif
try
{
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
if (!MiniExcelWriteAdapterFactory.TryGetAsyncWriteAdapter(values, _configuration, out asyncWriteAdapter))
{
writeAdapter = MiniExcelWriteAdapterFactory.GetWriteAdapter(values, _configuration);
Expand Down Expand Up @@ -156,7 +156,7 @@ private async Task<int> WriteValuesAsync(StreamWriter writer, object values, str
rowsWritten++;
}
}
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
else
{
await foreach (var row in asyncWriteAdapter.GetRowsAsync(props, cancellationToken))
Expand All @@ -182,7 +182,7 @@ private async Task<int> WriteValuesAsync(StreamWriter writer, object values, str
}
finally
{
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
if (asyncWriteAdapter is IAsyncDisposable asyncDisposable)
{
await asyncDisposable.DisposeAsync().ConfigureAwait(false);
Expand Down
24 changes: 11 additions & 13 deletions src/MiniExcel/MiniExcelLibs.csproj
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net45;netstandard2.0;net8.0;net9.0;net10.0</TargetFrameworks>
<Version>1.44.1</Version>
<TargetFrameworks>net45;net461;netstandard2.0;net8.0;net9.0;net10.0</TargetFrameworks>
<Version>1.45.0</Version>
<LangVersion>14</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<RootNamespace>MiniExcelLibs</RootNamespace>
Expand All @@ -13,11 +13,7 @@
<Title>MiniExcel</Title>
<Product>MiniExcel</Product>
<PackageTags>excel;xlsx;csv;micro-helper;mini;openxml;helper;</PackageTags>
<Description>Fast, Low-Memory, Easy Excel .NET helper to import/export/template spreadsheet
Github : https://github.com/mini-software/MiniExcel
Gitee : https://gitee.com/dotnetchina/MiniExcel
Issues : https://github.com/mini-software/MiniExcel/issues
Todo : https://github.com/mini-software/MiniExcel/projects/1?fullscreen=true</Description>
<Description>Lightweight, fast and simple .NET processing tool for importing, exporting and templating spreadsheets.</Description>
<Authors>Wei Lin, Michele Bastione, PING-HSIU SHIH, Amos(izanhzh), eynarhaji, Mini-Software team</Authors>
<PackageId>MiniExcel</PackageId>
<Copyright>Wei Lin, 2021 onwards</Copyright>
Expand All @@ -43,13 +39,15 @@ Todo : https://github.com/mini-software/MiniExcel/projects/1?fullscreen=true</De
<None Include="..\..\README-NuGet.md" Link="README.md" Pack="true" PackagePath="README.md" />
</ItemGroup>

<ItemGroup Condition=" '$(TargetFramework)' == 'net45'">
<Reference Include="System.IO.Compression" />
</ItemGroup>
<ItemGroup Condition="'$(GITHUB_ACTIONS)' == 'true'">
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All" />
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="10.0.300" PrivateAssets="All" />
Comment thread
michelebastione marked this conversation as resolved.
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == 'net45' OR '$(TargetFramework)' == 'net461'">
<PackageReference Include="System.IO.Compression" Version="4.3.0" />
</ItemGroup>
Comment thread
michelebastione marked this conversation as resolved.
<ItemGroup Condition="'$(TargetFramework)' == 'netstandard2.0'">
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.0" />

<ItemGroup Condition="'$(TargetFramework)' == 'netstandard2.0' OR '$(TargetFramework)' == 'net461'">
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.9" />
Comment thread
michelebastione marked this conversation as resolved.
</ItemGroup>
</Project>
8 changes: 4 additions & 4 deletions src/MiniExcel/OpenXml/ExcelOpenXmlSheetWriter.Async.cs
Original file line number Diff line number Diff line change
Expand Up @@ -171,12 +171,12 @@ private async Task<int> WriteValuesAsync(MiniExcelAsyncStreamWriter writer, obje
cancellationToken.ThrowIfCancellationRequested();

IMiniExcelWriteAdapter writeAdapter = null;
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
IAsyncMiniExcelWriteAdapter asyncWriteAdapter = null;
#endif
try
{
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
if (!MiniExcelWriteAdapterFactory.TryGetAsyncWriteAdapter(values, _configuration, out asyncWriteAdapter))
{
writeAdapter = MiniExcelWriteAdapterFactory.GetWriteAdapter(values, _configuration);
Expand Down Expand Up @@ -261,7 +261,7 @@ await WriteCellAsync(writer, currentRowIndex, cellValue.CellIndex, cellValue.Val
await writer.WriteAsync(WorksheetXml.EndRow);
}
}
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
else
{
await foreach (var row in asyncWriteAdapter.GetRowsAsync(props, cancellationToken))
Expand Down Expand Up @@ -309,7 +309,7 @@ await OverWriteColumnWidthPlaceholdersAsync(writer, columnWidthsPlaceholderPosit
}
finally
{
#if NETSTANDARD2_0_OR_GREATER || NET
#if !NET45
if (asyncWriteAdapter is IAsyncDisposable asyncDisposable)
{
await asyncDisposable.DisposeAsync().ConfigureAwait(false);
Expand Down
Loading
Loading