Skip to content

Add Presto cost profiling support and default parameters - #789

Open
JunWang222 wants to merge 23 commits into
apache:mainfrom
JunWang222:feature/presto-cost-profiling
Open

Add Presto cost profiling support and default parameters#789
JunWang222 wants to merge 23 commits into
apache:mainfrom
JunWang222:feature/presto-cost-profiling

Conversation

@JunWang222

Copy link
Copy Markdown
Contributor

This PR adds Presto cost profiling support, including:

  • A Presto profiling pilot for running operator cost profiling locally
  • Shared profiling documentation that explains how users can rerun profiling on their own machine
  • Presto-specific setup notes and command examples
  • Default Presto cost parameters for users who want to try Wayang with Presto

The default parameters are intended as starting values. For accurate optimization on a specific machine or deployment, users should follow the profiling guide and regenerate parameters locally.

@zkaoudi zkaoudi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor request, plus conflicts to be resolved


// Instantiate Wayang and activate the Spark backend.
// Instantiate Wayang and activate the Java/Flink backends.
WayangContext wayangContext = makeContext(FLINK);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here it has to be BOTH

…-profiling

# Conflicts:
#	guides/cost-profiling.md
#	wayang-profiler/src/main/java/org/apache/wayang/profiler/log/GeneticOptimizerApp.java
#	wayang-tests-integration/src/test/java/org/apache/wayang/tests/FlinkIntegrationIT.java
Comment thread presto-setup/README.md Outdated
The Presto cost-profiling branch is named `feature/presto-cost-profiling`:

```bash
git checkout feature/presto-cost-profiling

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. No need to checkout

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants