Skip to content

Add cifar10 inference with resnet20 model #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
code-perspective wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add cifar10 inference with resnet20 model #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@ Each workload represents a real-world use case with clear specifications, refere

- [**Fetch-by-Similarity**](./fetch-by-similarity/index.html). Private database queries using cosine similarity search over encrypted data.

- [**ML Inference**](./ml-inference/index.html). Privacy-preserving machine learning inference on encrypted inputs. Currently features MNIST digit classification, with other models to come.
- [**MNIST Inference**](./ml-inference/index.html). Privacy-preserving machine learning inference on encrypted inputs for MNIST digit classification.

- [**CIFAR10 Inference**](./cifar10-inference/index.html). Privacy-preserving machine learning inference on encrypted inputs. for CIFAR-10 image classification, with more models to come.

- [**Zn Multiplication**](./Zn-multiplication/index.html). Multiplication of two encrypted 64-bit encrypted integers. Other bit-sizes for the multiplicands will be added.

Expand Down
95 changes: 95 additions & 0 deletions cifar10-inference/Large.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
<!DOCTYPE html>
<html>
<head>
<title>FHE Benchmarking Results - CIFAR10 Inference (Large)</title>
<link rel="stylesheet" href="https://cdn.datatables.net/1.13.6/css/jquery.dataTables.min.css">
<link rel="stylesheet" href="https://cdn.datatables.net/buttons/2.4.2/css/buttons.dataTables.min.css">
<script src="https://code.jquery.com/jquery-3.7.0.min.js"></script>
<script src="https://cdn.datatables.net/1.13.6/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/dataTables.buttons.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/buttons.colVis.min.js"></script>
<style>
body { font-family: Arial, sans-serif; padding: 8px; }
#resultsTable { border-collapse: collapse; }
#resultsTable th, #resultsTable td { padding: 4px 8px; }
#resultsTable thead tr:first-child th { background-color: #2c3e50; color: white; font-weight: bold; text-align: center; }
#resultsTable thead tr:nth-child(2) th { background-color: #34495e; color: white; font-size: 0.9em; }
.submitter-header { background-color: #3498db !important; }
.model-header { background-color: #3498db !important; }
.bandwidth-header { background-color: #27ae60 !important; }
.quality-header { background-color: #103c88 !important; }
.harness-timing-header { background-color: #e67e22 !important; }
.server-timing-header { background-color: #9b59b6 !important; }
#resultsTable tbody tr:nth-child(odd) { background-color: #ecf0f1; }
#resultsTable tbody tr:nth-child(even) { background-color: #ffffff; }
.divider { width: 3px; padding: 0 !important; background-color: #000 !important; }
.dt-buttons { margin-bottom: 10px; }
</style>
</head>
<body>
<h1>FHE Benchmarking Results - CIFAR10 Inference (Large)</h1>

<table id="resultsTable" class="display">
<thead>
<tr>
<th colspan="5">Submitter</th> <th class="divider"></th> <th colspan="3">Bandwidth</th> <th class="divider"></th> <th colspan="1">Quality</th> <th class="divider"></th> <th colspan="9">Timing (harness)</th> <th class="divider"></th> <th colspan="9">Timing (server)</th>
</tr>
<tr><th id="col0" class="submitter-header" title="Submitter Name">Name</th>
<th id="col1" class="submitter-header" title="Submission Date">Date</th>
<th id="col2" class="submitter-header" title="Platform">Env</th>
<th id="col3" class="submitter-header" title="Remote/Local">R/L</th>
<th id="col4" class="model-header" title="Model name">Model</th>
<th id="col5" class="divider"></th>
<th id="col6" class="bandwidth-header" title="Size of keys">Keys</th>
<th id="col7" class="bandwidth-header" title="Encrypted Input size">Input</th>
<th id="col8" class="bandwidth-header" title="Encrypted results size">Result</th>
<th id="col9" class="divider"></th>
<th id="col10" class="quality-header" title="Result Accuracy (if applicable)">Accuracy</th>
<th id="col11" class="divider"></th>
<th id="col12" class="harness-timing-header" title="Total Time">Total</th>
<th id="col13" class="harness-timing-header" title="Dataset generation">DB Gen</th>
<th id="col14" class="harness-timing-header" title="Key Generation">Keygen</th>
<th id="col15" class="harness-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col16" class="harness-timing-header" title="Input generation">Input Gen</th>
<th id="col17" class="harness-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col18" class="harness-timing-header" title="Input encryption">Input Enc</th>
<th id="col19" class="harness-timing-header" title="Encrypted computation">Compute</th>
<th id="col20" class="harness-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col21" class="divider"></th>
<th id="col22" class="server-timing-header" title="Total Time">Total</th>
<th id="col23" class="server-timing-header" title="Dataset generation">DB Gen</th>
<th id="col24" class="server-timing-header" title="Key Generation">Keygen</th>
<th id="col25" class="server-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col26" class="server-timing-header" title="Input generation">Input Gen</th>
<th id="col27" class="server-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col28" class="server-timing-header" title="Input encryption">Input Enc</th>
<th id="col29" class="server-timing-header" title="Encrypted computation">Compute</th>
<th id="col30" class="server-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col31" class="server-timing-header" title="Server-reported I/O time">I/O</th>
</tr>
</thead>
<tbody>

</tbody>
</table>

<script>
$(document).ready(function() {
$('#resultsTable').DataTable({
dom: 'Bfrtip',
buttons: [{
extend: 'colvis',
columns: ':not(.no-toggle)'
}],
columnDefs: [
{ targets: [13, 17, 23, 24, 25, 26, 27, 28, 30], visible: false },
{ targets: [0, 1, 2, 3, 5, 7, 9, 11, 12, 21], className: 'no-toggle' },
{ targets: [5, 9, 11, 21], orderable: false, searchable: false, className: 'divider' }
],
pageLength: 25,
order: [[1, 'desc']]
});
});
</script>
</body>
</html>
95 changes: 95 additions & 0 deletions cifar10-inference/Medium.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
<!DOCTYPE html>
<html>
<head>
<title>FHE Benchmarking Results - CIFAR10 Inference (Medium)</title>
<link rel="stylesheet" href="https://cdn.datatables.net/1.13.6/css/jquery.dataTables.min.css">
<link rel="stylesheet" href="https://cdn.datatables.net/buttons/2.4.2/css/buttons.dataTables.min.css">
<script src="https://code.jquery.com/jquery-3.7.0.min.js"></script>
<script src="https://cdn.datatables.net/1.13.6/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/dataTables.buttons.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/buttons.colVis.min.js"></script>
<style>
body { font-family: Arial, sans-serif; padding: 8px; }
#resultsTable { border-collapse: collapse; }
#resultsTable th, #resultsTable td { padding: 4px 8px; }
#resultsTable thead tr:first-child th { background-color: #2c3e50; color: white; font-weight: bold; text-align: center; }
#resultsTable thead tr:nth-child(2) th { background-color: #34495e; color: white; font-size: 0.9em; }
.submitter-header { background-color: #3498db !important; }
.model-header { background-color: #3498db !important; }
.bandwidth-header { background-color: #27ae60 !important; }
.quality-header { background-color: #103c88 !important; }
.harness-timing-header { background-color: #e67e22 !important; }
.server-timing-header { background-color: #9b59b6 !important; }
#resultsTable tbody tr:nth-child(odd) { background-color: #ecf0f1; }
#resultsTable tbody tr:nth-child(even) { background-color: #ffffff; }
.divider { width: 3px; padding: 0 !important; background-color: #000 !important; }
.dt-buttons { margin-bottom: 10px; }
</style>
</head>
<body>
<h1>FHE Benchmarking Results - CIFAR10 Inference (Medium)</h1>

<table id="resultsTable" class="display">
<thead>
<tr>
<th colspan="5">Submitter</th> <th class="divider"></th> <th colspan="3">Bandwidth</th> <th class="divider"></th> <th colspan="1">Quality</th> <th class="divider"></th> <th colspan="9">Timing (harness)</th> <th class="divider"></th> <th colspan="9">Timing (server)</th>
</tr>
<tr><th id="col0" class="submitter-header" title="Submitter Name">Name</th>
<th id="col1" class="submitter-header" title="Submission Date">Date</th>
<th id="col2" class="submitter-header" title="Platform">Env</th>
<th id="col3" class="submitter-header" title="Remote/Local">R/L</th>
<th id="col4" class="model-header" title="Model name">Model</th>
<th id="col5" class="divider"></th>
<th id="col6" class="bandwidth-header" title="Size of keys">Keys</th>
<th id="col7" class="bandwidth-header" title="Encrypted Input size">Input</th>
<th id="col8" class="bandwidth-header" title="Encrypted results size">Result</th>
<th id="col9" class="divider"></th>
<th id="col10" class="quality-header" title="Result Accuracy (if applicable)">Accuracy</th>
<th id="col11" class="divider"></th>
<th id="col12" class="harness-timing-header" title="Total Time">Total</th>
<th id="col13" class="harness-timing-header" title="Dataset generation">DB Gen</th>
<th id="col14" class="harness-timing-header" title="Key Generation">Keygen</th>
<th id="col15" class="harness-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col16" class="harness-timing-header" title="Input generation">Input Gen</th>
<th id="col17" class="harness-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col18" class="harness-timing-header" title="Input encryption">Input Enc</th>
<th id="col19" class="harness-timing-header" title="Encrypted computation">Compute</th>
<th id="col20" class="harness-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col21" class="divider"></th>
<th id="col22" class="server-timing-header" title="Total Time">Total</th>
<th id="col23" class="server-timing-header" title="Dataset generation">DB Gen</th>
<th id="col24" class="server-timing-header" title="Key Generation">Keygen</th>
<th id="col25" class="server-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col26" class="server-timing-header" title="Input generation">Input Gen</th>
<th id="col27" class="server-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col28" class="server-timing-header" title="Input encryption">Input Enc</th>
<th id="col29" class="server-timing-header" title="Encrypted computation">Compute</th>
<th id="col30" class="server-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col31" class="server-timing-header" title="Server-reported I/O time">I/O</th>
</tr>
</thead>
<tbody>

</tbody>
</table>

<script>
$(document).ready(function() {
$('#resultsTable').DataTable({
dom: 'Bfrtip',
buttons: [{
extend: 'colvis',
columns: ':not(.no-toggle)'
}],
columnDefs: [
{ targets: [13, 17, 23, 24, 25, 26, 27, 28, 30], visible: false },
{ targets: [0, 1, 2, 3, 5, 7, 9, 11, 12, 21], className: 'no-toggle' },
{ targets: [5, 9, 11, 21], orderable: false, searchable: false, className: 'divider' }
],
pageLength: 25,
order: [[1, 'desc']]
});
});
</script>
</body>
</html>
95 changes: 95 additions & 0 deletions cifar10-inference/Single.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
<!DOCTYPE html>
<html>
<head>
<title>FHE Benchmarking Results - CIFAR10 Inference (Single)</title>
<link rel="stylesheet" href="https://cdn.datatables.net/1.13.6/css/jquery.dataTables.min.css">
<link rel="stylesheet" href="https://cdn.datatables.net/buttons/2.4.2/css/buttons.dataTables.min.css">
<script src="https://code.jquery.com/jquery-3.7.0.min.js"></script>
<script src="https://cdn.datatables.net/1.13.6/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/dataTables.buttons.min.js"></script>
<script src="https://cdn.datatables.net/buttons/2.4.2/js/buttons.colVis.min.js"></script>
<style>
body { font-family: Arial, sans-serif; padding: 8px; }
#resultsTable { border-collapse: collapse; }
#resultsTable th, #resultsTable td { padding: 4px 8px; }
#resultsTable thead tr:first-child th { background-color: #2c3e50; color: white; font-weight: bold; text-align: center; }
#resultsTable thead tr:nth-child(2) th { background-color: #34495e; color: white; font-size: 0.9em; }
.submitter-header { background-color: #3498db !important; }
.model-header { background-color: #3498db !important; }
.bandwidth-header { background-color: #27ae60 !important; }
.quality-header { background-color: #103c88 !important; }
.harness-timing-header { background-color: #e67e22 !important; }
.server-timing-header { background-color: #9b59b6 !important; }
#resultsTable tbody tr:nth-child(odd) { background-color: #ecf0f1; }
#resultsTable tbody tr:nth-child(even) { background-color: #ffffff; }
.divider { width: 3px; padding: 0 !important; background-color: #000 !important; }
.dt-buttons { margin-bottom: 10px; }
</style>
</head>
<body>
<h1>FHE Benchmarking Results - CIFAR10 Inference (Single)</h1>

<table id="resultsTable" class="display">
<thead>
<tr>
<th colspan="5">Submitter</th> <th class="divider"></th> <th colspan="3">Bandwidth</th> <th class="divider"></th> <th colspan="9">Timing (harness)</th> <th class="divider"></th> <th colspan="9">Timing (server)</th>
</tr>
<tr><th id="col0" class="submitter-header" title="Submitter Name">Name</th>
<th id="col1" class="submitter-header" title="Submission Date">Date</th>
<th id="col2" class="submitter-header" title="Platform">Env</th>
<th id="col3" class="submitter-header" title="Remote/Local">R/L</th>
<th id="col4" class="model-header" title="Model name">Model</th>
<th id="col5" class="divider"></th>
<th id="col6" class="bandwidth-header" title="Size of keys">Keys</th>
<th id="col7" class="bandwidth-header" title="Encrypted Input size">Input</th>
<th id="col8" class="bandwidth-header" title="Encrypted results size">Result</th>
<th id="col9" class="divider"></th>
<th id="col10" class="harness-timing-header" title="Total Time">Total</th>
<th id="col11" class="harness-timing-header" title="Dataset generation">DB Gen</th>
<th id="col12" class="harness-timing-header" title="Key Generation">Keygen</th>
<th id="col13" class="harness-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col14" class="harness-timing-header" title="Input generation">Input Gen</th>
<th id="col15" class="harness-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col16" class="harness-timing-header" title="Input encryption">Input Enc</th>
<th id="col17" class="harness-timing-header" title="Encrypted computation">Compute</th>
<th id="col18" class="harness-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col19" class="divider"></th>
<th id="col20" class="server-timing-header" title="Total Time">Total</th>
<th id="col21" class="server-timing-header" title="Dataset generation">DB Gen</th>
<th id="col22" class="server-timing-header" title="Key Generation">Keygen</th>
<th id="col23" class="server-timing-header" title="Encrypted model preprocessing">Model Prep</th>
<th id="col24" class="server-timing-header" title="Input generation">Input Gen</th>
<th id="col25" class="server-timing-header" title="Input preprocessing">Input Prep</th>
<th id="col26" class="server-timing-header" title="Input encryption">Input Enc</th>
<th id="col27" class="server-timing-header" title="Encrypted computation">Compute</th>
<th id="col28" class="server-timing-header" title="Result decryption and postprocessing">Decrypt</th>
<th id="col29" class="server-timing-header" title="Server-reported I/O time">I/O</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://github.com/fhe-benchmarking/ml-inference">Reference</a></td><td>2026&#8209;08&#8209;05 03:38:16</td><td>CPU</td><td>L</td><td>resnet20</td><td class="divider"></td><td>50.4G</td><td>28M</td><td>8M</td><td class="divider"></td><td>33.52m</td><td>14.34s</td><td>1.0766m</td><td>6ms</td><td>2.9962s</td><td>2.7333ms</td><td>533.5ms</td><td>32.14m</td><td>370.2ms</td><td class="divider"></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td>
</tr>
</tbody>
</table>

<script>
$(document).ready(function() {
$('#resultsTable').DataTable({
dom: 'Bfrtip',
buttons: [{
extend: 'colvis',
columns: ':not(.no-toggle)'
}],
columnDefs: [
{ targets: [11, 15, 21, 22, 23, 24, 25, 26, 28], visible: false },
{ targets: [0, 1, 2, 3, 5, 7, 9, 10, 19], className: 'no-toggle' },
{ targets: [5, 9, 19], orderable: false, searchable: false, className: 'divider' }
],
pageLength: 25,
order: [[1, 'desc']]
});
});
</script>
</body>
</html>
Loading