Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
8e5c35a
Testing for pep8
synthetic-corpus Feb 28, 2026
63d7772
not pep8 valid
synthetic-corpus Feb 28, 2026
0ae5e48
should not be able to commit
synthetic-corpus Feb 28, 2026
d03b74e
pep8 compliant?
synthetic-corpus Feb 28, 2026
ba2758a
checked for pycodestyle again.
synthetic-corpus Feb 28, 2026
62a4e20
mostly known working code for s3_acces
synthetic-corpus Feb 28, 2026
27b68e0
new version of flake8?
synthetic-corpus Feb 28, 2026
e672eb2
added helper tool to load data from s3 to a local ec2 instance ebs dr…
synthetic-corpus Feb 28, 2026
2f4c04c
installed the initializer
synthetic-corpus Feb 28, 2026
924b3cb
splicer now should resize down to smaller tiles.
synthetic-corpus Feb 28, 2026
feae3a0
hardcoded the out folder
synthetic-corpus Feb 28, 2026
0a45fc2
slightly redundant write functionality to s3 now provided.
synthetic-corpus Feb 28, 2026
0b03b9d
save of a mosaic now goes to a static location.
synthetic-corpus Feb 28, 2026
41e4981
removing mount check for now.
synthetic-corpus Feb 28, 2026
394ba37
corrected a breaking typo.
synthetic-corpus Feb 28, 2026
ec79112
updated .gitignore
synthetic-corpus Feb 28, 2026
ca8b09f
made a dangerous idea for Image. Disabled some protections.
synthetic-corpus Feb 28, 2026
a15dc71
minimized logging for my own sanity.
synthetic-corpus Feb 28, 2026
73619b1
ensures smaller sources files. Logging is no longer excessive or silent.
synthetic-corpus Feb 28, 2026
014a185
m brain transplated a processer. Now uses sci-kit learn tools.
synthetic-corpus Feb 28, 2026
bbd09d5
updating requirements for scikit
synthetic-corpus Mar 1, 2026
e28a052
adding anti clumping.
synthetic-corpus Mar 1, 2026
1fb90fb
set the splicer for multithreading.
synthetic-corpus Mar 1, 2026
02aef15
added some Quality of Life to the v2 console too.
synthetic-corpus Mar 1, 2026
24d22be
hopefully no more redundant tile processing.
synthetic-corpus Mar 1, 2026
9762cc4
more finely tuned default penalty.
synthetic-corpus Mar 1, 2026
f015004
enforcing color channel uniformity at download from s3
synthetic-corpus Mar 1, 2026
d070c37
creating a test script to verify that dependancie run okay on hardware.
synthetic-corpus Mar 1, 2026
8ab0322
updated dependanices for python3.14
synthetic-corpus Mar 1, 2026
e733402
downgraded requirements to python 3.9
synthetic-corpus Mar 1, 2026
a67addc
making code resilient to what I think must be bad input.
synthetic-corpus Mar 1, 2026
39c32d4
fixing win size
synthetic-corpus Mar 1, 2026
6db8ea1
Merge pull request #1 from synthetic-corpus/optimize
synthetic-corpus Mar 1, 2026
3a3985d
a slightly more organized version of this tool
synthetic-corpus Mar 7, 2026
6cfea61
sorted mosaic-v2 into its own folder.
synthetic-corpus Mar 7, 2026
4cbfef7
updated splicer
synthetic-corpus Mar 7, 2026
046d195
configurered to pull from tools.
synthetic-corpus Mar 7, 2026
043b8d6
set splicer to actually process videos this time.
synthetic-corpus Mar 7, 2026
1ef2e02
Made sure Mosaic v2 can use suffixes for files.
synthetic-corpus Mar 7, 2026
b09ae71
import class not module.
synthetic-corpus Mar 7, 2026
0ccc029
fixed all imports
synthetic-corpus Mar 7, 2026
eb9135c
fixed mosaic invocation
synthetic-corpus Mar 7, 2026
48f2134
cannot run Tile Processor.
synthetic-corpus Mar 7, 2026
cad587d
no errors, but not processing correctly.
synthetic-corpus Mar 7, 2026
dc5e3f1
I think this will work now.
synthetic-corpus Mar 7, 2026
9a3f9cd
Tile Processer still not working.
synthetic-corpus Mar 7, 2026
0a87230
adding an easy downloader tool.
synthetic-corpus Mar 7, 2026
e8621b1
expanded on Mosaic Class so that assemble is method. Legacy funcation…
synthetic-corpus Mar 8, 2026
5fe0991
added option for s3 data tool to get a smaller sample of videos, pics…
synthetic-corpus Mar 8, 2026
95b29ea
also added option to grab only three sources for mosaics.
synthetic-corpus Mar 8, 2026
7f565d2
have set up TileFitterSciKiet and Mosaic Classes to use methods now.
synthetic-corpus Mar 8, 2026
5d13201
removed an extra suffix argument
synthetic-corpus Mar 8, 2026
c4fb4e5
removed a deadlock.
synthetic-corpus Mar 8, 2026
df4b265
updated functionality of progress counter.
synthetic-corpus Mar 8, 2026
a7c9069
added a stat tracker to get vibes on matching.
synthetic-corpus Mar 8, 2026
7b267bc
optimizing for caching
synthetic-corpus Mar 8, 2026
cb2c143
Merge pull request #2 from synthetic-corpus/refactor
synthetic-corpus Mar 9, 2026
c58e065
Refactor (#3)
synthetic-corpus Mar 22, 2026
327ea0f
Optimize memory (#4)
synthetic-corpus Apr 3, 2026
c973de4
Optimize memory (#5)
synthetic-corpus May 6, 2026
9ed7cbb
added a process timer class
synthetic-corpus May 6, 2026
7ba7713
Implemented process timer
synthetic-corpus May 6, 2026
6995656
Created funcationality to process tiles outside of compose.
synthetic-corpus May 7, 2026
16c07fe
updated files for legacy processing.
synthetic-corpus May 14, 2026
d1ff668
added a new arguement to get more frames from a video
synthetic-corpus May 14, 2026
1d6b1d4
reduced splicer log output so it is not completely loco
synthetic-corpus May 16, 2026
f11e176
created a script so that running this code is less of a headache.
synthetic-corpus May 16, 2026
21a49a9
Upload results now too
synthetic-corpus May 16, 2026
76e6706
will create root folder manually
synthetic-corpus May 16, 2026
1a80116
fixed splicer problem.
synthetic-corpus May 16, 2026
fc41d6a
Modified the original Mosiac.py. Still not sure if it will ... work.
synthetic-corpus May 16, 2026
2964448
running another experiment
synthetic-corpus May 17, 2026
0d346c9
finished a readme.md
synthetic-corpus Jun 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/pep8-enforcement.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: PEP 8 Compliance Check

on:
workflow_dispatch:
push:
branches:
- '**' # Run on pushes to any branch

jobs:
pep8_check:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0 # IMPORTANT: Fetch all history for git diff to work correctly

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x' # Specify your Python version (e.g., '3.9', '3.10', '3.11')

- name: Install flake8
run: pip install flake8

- name: Get changed Python files
id: changed-files
run: |
# For push events, compare with the previous commit
# Handle cases where commits might not exist or be accessible
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# Manual trigger - check all Python files
CHANGED_PYTHON_FILES=$(find . -name "*.py" -type f | tr '\n' ' ')
elif [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ]; then
# First commit - check all Python files
CHANGED_PYTHON_FILES=$(find . -name "*.py" -type f | tr '\n' ' ')
else
# Try git diff, fallback to all files if it fails
CHANGED_PYTHON_FILES=$(git diff --name-only --diff-filter=ACM ${{ github.event.before }} ${{ github.sha }} 2>/dev/null | grep '\.py$' | tr '\n' ' ' || find . -name "*.py" -type f | tr '\n' ' ')
fi

# Ensure we have a value even if empty
if [ -z "$CHANGED_PYTHON_FILES" ]; then
CHANGED_PYTHON_FILES=""
fi

echo "changed_python_files=${CHANGED_PYTHON_FILES}" >> $GITHUB_OUTPUT
echo "Event type: ${{ github.event_name }}"
echo "Before commit: ${{ github.event.before }}"
echo "Current commit: ${{ github.sha }}"
echo "Found Python files: '$CHANGED_PYTHON_FILES'"
shell: bash

- name: Run flake8 on changed Python files
if: success() && steps.changed-files.outputs.changed_python_files
run: |
echo "Checking the following files for PEP 8 compliance:"
echo "${{ steps.changed-files.outputs.changed_python_files }}"
echo ""
# Use xargs to properly handle multiple files
echo "${{ steps.changed-files.outputs.changed_python_files }}" | xargs flake8
shell: bash

- name: Report success if no Python files were changed (Optional, for clarity)
# This step only runs if the 'Run flake8' step was skipped due to no Python files being found.
# It provides a clear message in the GitHub Actions UI.
if: success() && !steps.changed-files.outputs.changed_python_files
run: echo "No .py files changed in this push. PEP 8 compliance check passed (skipped)."
226 changes: 224 additions & 2 deletions .gitignore
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,4 +1,226 @@
.idea
ENV
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# UV
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
#uv.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# Abstra
# Abstra is an AI-powered process automation framework.
# Ignore directories containing user credentials, local state, and settings.
# Learn more at https://abstra.io/docs
.abstra/

# Visual Studio Code
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# you could uncomment the following to ignore the enitre vscode folder
# .vscode/

# Ruff stuff:
.ruff_cache/

# PyPI configuration file
.pypirc

# Cursor
# Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
# exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
# refer to https://docs.cursor.com/context/ignore-files
.cursorignore
.cursorindexingignore

# avoid mac stuff
.DS_Store

# Ignore all image files
*.jpg
*.jpeg
*.png

# Terraform Stuff to Ignore
# Local .terraform directories
**/.terraform/*
**/.venv/

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log
crash.*.log
# other stuff
.vscode
.venv
__pycache__
secrets.txt
test-media/
test-results/
logs/

# fiddle and experimentation files
fiddle/
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
repos:
- repo: https://github.com/PyCQA/flake8
rev: 7.3.0
hooks:
- id: flake8
Empty file modified LICENSE
100644 → 100755
Empty file.
Loading