Add education demo seed and EduProject metadata - #127
Open
Oluwajuwon-O wants to merge 7 commits into
Open
Conversation
20 tasks
Oluwajuwon-O
marked this pull request as ready for review
August 5, 2026 13:10
Oluwajuwon-O
marked this pull request as draft
August 5, 2026 13:10
information_schema.tables includes both object types; issuing only DROP VIEW fails on base tables. Clear views first, then BASE TABLEs.
Pass source_schema through the Airflow sync path, namespace target and test schemas as data_<conn>_<schema>, and add EduProject config pointing at data_dot_data_education alongside ScanProject1's data_dot_data_public.
When date_field is set, save_object appends instead of replacing and deletes overlapping target rows by id_field (chunked) before write.
Oluwajuwon-O
force-pushed
the
feature/education_demo_seed_data
branch
from
August 8, 2026 07:45
7637754 to
9699d19
Compare
Oluwajuwon-O
marked this pull request as ready for review
August 8, 2026 07:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses the database bootstrap and demo data enhancements described in the technical documentation:
Lack of Domain-Agnostic Demonstration
Historically, DOT only included a single demo project (
ScanProject1) which was tightly coupled to healthcare terminology and data structures. This made it difficult to demonstrate DOT's capabilities as a generic, domain-agnostic data quality framework.Resolution
db/dot/5-education_demo_data.sqlto initialize a neweducationschema in the source database and populate it with synthetic tables (students,courses,enrollments,excel_grades).db/dot/6-eduproject_metadata.sqlto register the newEduProjectin the DOT metadata tables. This script configures a suite of data quality tests against the new education entities.Note: This PR depends on Isolate target schemas by source schema #125 and Incremental append loads with ID-based overlap deletion #126.
In the Technical Documentation, this supports the EduProject / education-schema examples in Sections 2.2 and 3.2 (bootstrap/demo data context in Section 9).
In the Technical Documentation, this is addressed in Sections 9.
Asana Task
Deployment Readiness*
Testing
Describe or check:
Deployment Notes
Describe or check:
Rollback Plan
Describe or check:
git revert)Reviewer Guidance / Questions*
Screenshots / Testing Evidence*
SOC 2 Change Management Checklist
./vendor/bin/pest)Provide justification if you are submitting a PR with any boxes checked other than the first.
Reminder for Reviewers: By approving this PR you are confirming that you have reviewed the code for correctness, security, and compliance with our engineering and SOC 2 standards. Do not approve PRs where SOC 2 checklist items are checked without documented justification.
*Optional