-
Notifications
You must be signed in to change notification settings - Fork 44
[10주차] Batch 활용 주간 월간 랭킹 시스템 - 최호석 #418
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ghtjr410
wants to merge
21
commits into
Loopers-dev-lab:ghtjr410
Choose a base branch
from
ghtjr410:volume-10
base: ghtjr410
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
f11e96b
feat: 랭킹 배치 anchorDate 파라미터 리스너와 롤링 윈도우 계산기 추가
ghtjr410 69a51f3
feat: 랭킹 배치 스테이징 테이블과 Step 0 초기화 Tasklet 추가
ghtjr410 c4f692b
feat: 랭킹 배치 Step 1 - View 메트릭 cursor 스트리밍 집계 추가
ghtjr410 bccd9b0
feat: 랭킹 배치 Step 2/3 - Like/Order 메트릭 스테이징 적재 추가
ghtjr410 ebe14ce
feat: 랭킹 배치 Step 4a/4b - MV 사전 DELETE Tasklet 추가
ghtjr410 8ca5d0c
feat: 랭킹 배치 Step 5 - 전체 상품 score 계산을 2차 스테이징에 적재
ghtjr410 97e453a
feat: 랭킹 배치 Step 5b/7/6 - Promote/Audit/Redis 로 파이프라인 완성
ghtjr410 f018ade
feat: commerce-api 의 랭킹 MV 조회 어댑터 추가
ghtjr410 066ae51
feat: 랭킹 조회 API 를 LAST_7D/LAST_30D 롤링 윈도우로 교체
ghtjr410 aef63c9
test: 랭킹 배치 재시작 시나리오 4종 추가 + chunk Reader saveState 비활성화
ghtjr410 d716afc
test: 랭킹 배치 시드 생성기와 분포 검증 추가
ghtjr410 b4072ca
test: 랭킹 배치 선형성/스파이크 측정 벤치마크와 실행 스크립트 추가
ghtjr410 d440609
fix: streaming aggregator lookahead 를 ExecutionContext 에 직렬화하여 chunk-…
ghtjr410 cfe03d1
test: Scenario 4 - Hot product 긴 bucket 체인의 chunk 경계 무절단 검증
ghtjr410 bf1fa78
fix: weight_group 을 Job 시작 시점에 ExecutionContext 스냅샷으로 동결
ghtjr410 b2a74b8
fix: Step 7 audit 실패 시 오염 MV 격리 + API 전일 anchor 자동 fallback
ghtjr410 9772899
refactor: Step 4a/4b 를 단일 PurgeMvTasklet 으로 통합
ghtjr410 acc517f
style: 배치 테스트 13개 파일을 프로젝트 test-patterns 컨벤션으로 교체
ghtjr410 d33a90e
refactor: MV 교체를 DELETE+INSERT 단일 TX 원자 교체로 변경
ghtjr410 c236b24
chore: Step 번호를 0~7 순차로 재정렬
ghtjr410 776b6a5
refactor: audit 검증 Step 제거, 실행 이력 기록을 promote Step으로 통합
ghtjr410 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
13 changes: 12 additions & 1 deletion
13
apps/commerce-api/src/main/java/com/loopers/domain/ranking/RankingPeriod.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,16 @@ | ||
| package com.loopers.domain.ranking; | ||
|
|
||
| /** | ||
| * 랭킹 조회 기간. | ||
| * | ||
| * <p>WEEKLY/MONTHLY 캘린더 경계는 "월요일 오전/매월 1일 오전에 표본이 1일치" 라는 빈약성 | ||
| * 문제가 있고, 실무에선 이커머스 랭킹을 롤링 N일 (오늘 제외) 로 구현하는 것이 일반적이다 | ||
| * (설계.md 프롤로그 + 데빈/케브/앨런 멘토링 결론). 본 API 는 배치가 만드는 롤링 MV 와 | ||
| * 일관되게 LAST_7D / LAST_30D 로 노출한다.</p> | ||
| */ | ||
| public enum RankingPeriod { | ||
| REALTIME, DAILY, WEEKLY, MONTHLY | ||
| REALTIME, | ||
| DAILY, | ||
| LAST_7D, | ||
| LAST_30D | ||
| } |
39 changes: 39 additions & 0 deletions
39
apps/commerce-api/src/main/java/com/loopers/domain/ranking/mv/MvProductRankId.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| package com.loopers.domain.ranking.mv; | ||
|
|
||
| import java.io.Serializable; | ||
| import java.time.LocalDate; | ||
| import java.util.Objects; | ||
|
|
||
| /** | ||
| * mv_product_rank_last_7d / mv_product_rank_last_30d 공통 PK. | ||
| * commerce-batch 가 생성·적재하는 MV 를 commerce-api 가 읽기 위한 스키마 미러. | ||
| */ | ||
| public class MvProductRankId implements Serializable { | ||
|
|
||
| private LocalDate anchorDate; | ||
| private String weightGroup; | ||
| private Long productId; | ||
|
|
||
| public MvProductRankId() { | ||
| } | ||
|
|
||
| public MvProductRankId(LocalDate anchorDate, String weightGroup, Long productId) { | ||
| this.anchorDate = anchorDate; | ||
| this.weightGroup = weightGroup; | ||
| this.productId = productId; | ||
| } | ||
|
|
||
| @Override | ||
| public boolean equals(Object o) { | ||
| if (this == o) return true; | ||
| if (!(o instanceof MvProductRankId that)) return false; | ||
| return Objects.equals(anchorDate, that.anchorDate) | ||
| && Objects.equals(weightGroup, that.weightGroup) | ||
| && Objects.equals(productId, that.productId); | ||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() { | ||
| return Objects.hash(anchorDate, weightGroup, productId); | ||
| } | ||
| } |
72 changes: 72 additions & 0 deletions
72
apps/commerce-api/src/main/java/com/loopers/domain/ranking/mv/MvProductRankLast30d.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| package com.loopers.domain.ranking.mv; | ||
|
|
||
| import jakarta.persistence.Column; | ||
| import jakarta.persistence.Entity; | ||
| import jakarta.persistence.Id; | ||
| import jakarta.persistence.IdClass; | ||
| import jakarta.persistence.Index; | ||
| import jakarta.persistence.Table; | ||
| import lombok.Getter; | ||
|
|
||
| import java.time.LocalDate; | ||
| import java.time.LocalDateTime; | ||
|
|
||
| @Entity | ||
| @Table( | ||
| name = "mv_product_rank_last_30d", | ||
| indexes = @Index( | ||
| name = "idx_last_30d_rank", | ||
| columnList = "anchor_date, weight_group, rank_position" | ||
| ) | ||
| ) | ||
| @IdClass(MvProductRankId.class) | ||
| @Getter | ||
| public class MvProductRankLast30d { | ||
|
|
||
| @Id | ||
| @Column(name = "anchor_date", nullable = false) | ||
| private LocalDate anchorDate; | ||
|
|
||
| @Id | ||
| @Column(name = "weight_group", length = 32, nullable = false) | ||
| private String weightGroup; | ||
|
|
||
| @Id | ||
| @Column(name = "product_id", nullable = false) | ||
| private Long productId; | ||
|
|
||
| @Column(name = "view_count", nullable = false) | ||
| private long viewCount; | ||
|
|
||
| @Column(name = "like_count", nullable = false) | ||
| private long likeCount; | ||
|
|
||
| @Column(name = "sales_amount", nullable = false) | ||
| private long salesAmount; | ||
|
|
||
| @Column(name = "score", nullable = false) | ||
| private double score; | ||
|
|
||
| @Column(name = "rank_position", nullable = false) | ||
| private int rankPosition; | ||
|
|
||
| @Column(name = "created_at", nullable = false) | ||
| private LocalDateTime createdAt; | ||
|
|
||
| protected MvProductRankLast30d() { | ||
| } | ||
|
|
||
| public MvProductRankLast30d(LocalDate anchorDate, String weightGroup, Long productId, | ||
| long viewCount, long likeCount, long salesAmount, | ||
| double score, int rankPosition, LocalDateTime createdAt) { | ||
| this.anchorDate = anchorDate; | ||
| this.weightGroup = weightGroup; | ||
| this.productId = productId; | ||
| this.viewCount = viewCount; | ||
| this.likeCount = likeCount; | ||
| this.salesAmount = salesAmount; | ||
| this.score = score; | ||
| this.rankPosition = rankPosition; | ||
| this.createdAt = createdAt; | ||
| } | ||
| } |
76 changes: 76 additions & 0 deletions
76
apps/commerce-api/src/main/java/com/loopers/domain/ranking/mv/MvProductRankLast7d.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| package com.loopers.domain.ranking.mv; | ||
|
|
||
| import jakarta.persistence.Column; | ||
| import jakarta.persistence.Entity; | ||
| import jakarta.persistence.Id; | ||
| import jakarta.persistence.IdClass; | ||
| import jakarta.persistence.Index; | ||
| import jakarta.persistence.Table; | ||
| import lombok.Getter; | ||
|
|
||
| import java.time.LocalDate; | ||
| import java.time.LocalDateTime; | ||
|
|
||
| /** | ||
| * 롤링 7일 랭킹 확정 MV 의 commerce-api 측 읽기 모델. | ||
| * commerce-batch 가 쓰기 소유자이며, 여기서는 조회만 한다. | ||
| */ | ||
| @Entity | ||
| @Table( | ||
| name = "mv_product_rank_last_7d", | ||
| indexes = @Index( | ||
| name = "idx_last_7d_rank", | ||
| columnList = "anchor_date, weight_group, rank_position" | ||
| ) | ||
| ) | ||
| @IdClass(MvProductRankId.class) | ||
| @Getter | ||
| public class MvProductRankLast7d { | ||
|
|
||
| @Id | ||
| @Column(name = "anchor_date", nullable = false) | ||
| private LocalDate anchorDate; | ||
|
|
||
| @Id | ||
| @Column(name = "weight_group", length = 32, nullable = false) | ||
| private String weightGroup; | ||
|
|
||
| @Id | ||
| @Column(name = "product_id", nullable = false) | ||
| private Long productId; | ||
|
|
||
| @Column(name = "view_count", nullable = false) | ||
| private long viewCount; | ||
|
|
||
| @Column(name = "like_count", nullable = false) | ||
| private long likeCount; | ||
|
|
||
| @Column(name = "sales_amount", nullable = false) | ||
| private long salesAmount; | ||
|
|
||
| @Column(name = "score", nullable = false) | ||
| private double score; | ||
|
|
||
| @Column(name = "rank_position", nullable = false) | ||
| private int rankPosition; | ||
|
|
||
| @Column(name = "created_at", nullable = false) | ||
| private LocalDateTime createdAt; | ||
|
|
||
| protected MvProductRankLast7d() { | ||
| } | ||
|
|
||
| public MvProductRankLast7d(LocalDate anchorDate, String weightGroup, Long productId, | ||
| long viewCount, long likeCount, long salesAmount, | ||
| double score, int rankPosition, LocalDateTime createdAt) { | ||
| this.anchorDate = anchorDate; | ||
| this.weightGroup = weightGroup; | ||
| this.productId = productId; | ||
| this.viewCount = viewCount; | ||
| this.likeCount = likeCount; | ||
| this.salesAmount = salesAmount; | ||
| this.score = score; | ||
| this.rankPosition = rankPosition; | ||
| this.createdAt = createdAt; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MV 조회에 타임아웃/실패 처리 정책이 없어 cascading 장애로 번질 수 있다.
fallbackFromMv는 최대 3 회 DB 조회를 수행한다. 각 호출의 JDBC 타임아웃 / 커넥션 풀 상한 / 지연 허용치에 대한 방어가 없고, MV 테이블이 락이나 대형 트랜잭션에 물리면 3 회가 누적되어 API 스레드가 오래 점유된다. Redis 장애가 났을 때 모든 요청이 이 경로로 몰리면 MV 조회 큐잉으로 DB 커넥션 풀이 바닥나는 연쇄 장애가 발생한다.또한
catch (Exception e)로 모든 예외를List.of()로 바꿔 삼키는 구조 (120-123) 는 다음을 의미한다:권고다:
MvRankingQueryRepository구현체에 statement timeout /@Transactional(timeout = N)설정 (예: 1 초).CoreException) 또는 Micrometer 카운터 + 구조화 로그 (mv_fallback_errortag) 를 방출해 운영 가시성 확보.추가 테스트로는
MvRankingQueryRepositoryMockito stub 으로SQLTimeoutException을 던졌을 때:error인지,List.of()인지확인하는 케이스를 포함하길 권고한다.
코딩 가이드라인 (
외부 호출에는 타임아웃/재시도/서킷브레이커 고려 여부를 점검하고, 실패 시 대체 흐름을 제안한다) 에 따라 제안한다.🤖 Prompt for AI Agents