Hive 29574 merge join poc#6456
Conversation
asf-ci-hive
added
tests pending
tests passed
and removed
tests passed
tests pending
labels
|
abstractdog
left a comment
abstractdog
left a comment
There was a problem hiding this comment.
nice patch @illiabarbashov-sketch so far, let some comments
| skewedKeyFlagged = new boolean[maxAlias]; | ||
| } | ||
|
|
||
| public boolean isActive() { |
There was a problem hiding this comment.
can be private, or package-protected @VisibleForTesting if unit tests need it
There was a problem hiding this comment.
updated
| SET hive.vectorized.execution.enabled=false; | ||
| set hive.mapred.mode=nonstrict; | ||
| set hive.explain.user=false; | ||
| set hive.cbo.enable=false; |
There was a problem hiding this comment.
cbo should be enabled, we can simply delete this
There was a problem hiding this comment.
updated
| SET hive.vectorized.execution.enabled=false; | ||
| set hive.mapred.mode=nonstrict; | ||
| set hive.explain.user=false; | ||
| set hive.cbo.enable=false; |
There was a problem hiding this comment.
cbo should be enabled, we can simply delete this
There was a problem hiding this comment.
updated
| @@ -0,0 +1,35 @@ | |||
| SET hive.vectorized.execution.enabled=false; | |||
There was a problem hiding this comment.
vectorization should be enabled, we can simple delete this
There was a problem hiding this comment.
updated
| @@ -0,0 +1,20 @@ | |||
| SET hive.vectorized.execution.enabled=false; | |||
There was a problem hiding this comment.
vectorization should be enabled, we can simple delete this
There was a problem hiding this comment.
updated
| set hive.explain.user=false; | ||
| set hive.cbo.enable=true; | ||
| set hive.auto.convert.join=false; | ||
| set hive.optimize.ppd=false; |
There was a problem hiding this comment.
is hive.optimize.ppd=false needed?
There was a problem hiding this comment.
updated
| @@ -0,0 +1,35 @@ | |||
| SET hive.vectorized.execution.enabled=true; | |||
| set hive.mapred.mode=nonstrict; | |||
There was a problem hiding this comment.
is hive.mapred.mode=nonstrict needed?
There was a problem hiding this comment.
removed
| @@ -0,0 +1,20 @@ | |||
| SET hive.vectorized.execution.enabled=true; | |||
| set hive.mapred.mode=nonstrict; | |||
There was a problem hiding this comment.
is hive.mapred.mode=nonstrict needed?
There was a problem hiding this comment.
removed
| HIVE_JOIN_CACHE_SIZE("hive.join.cache.size", 25000, | ||
| "How many rows in the joining tables (except the streaming table) should be cached in memory."), | ||
| HIVE_MERGE_JOIN_SKEW_THRESHOLD("hive.merge.join.skew.threshold", -1L, | ||
| "Maximum number of rows allowed per join key in a single Tez sort-merge join task before a " |
There was a problem hiding this comment.
we can remove "Tez"
even "Tez" is the only execution engine we're currently supporting, this feature is theoretically orthogonal to that, so this is rather "Maximum number of rows allowed per join key in a single sort-merge reducer join task before..."
There was a problem hiding this comment.
removed
| "Maximum number of rows allowed per join key in a single Tez sort-merge join task before a " | ||
| + "skew event is reported."), | ||
| HIVE_MERGE_JOIN_SKEW_ABORT("hive.merge.join.skew.abort", false, | ||
| "When set to true and the row count is equal to hive.merge.join.skew.threshold, the Tez task will be aborted."), |
There was a problem hiding this comment.
maybe remove "Tez" from here too
There was a problem hiding this comment.
removed
asf-ci-hive
added
tests pending
tests unstable
tests failed
and removed
tests passed
tests pending
tests unstable
tests failed
labels
illiabarbashov-sketch
force-pushed
the
HIVE-29574_merge_join_poc
branch
from
d9bc666 to
f97785a
Compare
|
3 participants
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
No
How was this patch tested?