Add Spark CBO config tips for boosting motif finding performance#845
Conversation
|
@SemyonSinchenko would you take a look ? :) |
There was a problem hiding this comment.
Pull request overview
This PR improves GraphFrames motif-finding documentation by adding practical Spark Cost-Based Optimizer (CBO) configuration guidance to help users speed up motif queries (which compile down to multi-join plans).
Changes:
- Expanded
GraphFrame.find()Python docstring with a motif DSL explanation and join/CBO performance tips. - Added a “Performance” section to the motif-finding user guide describing CBO + join reordering and the DP threshold.
- Added an analogous “Performance” section to the Scala
GraphFrame.findScaladoc.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| python/graphframes/graphframe.py | Enhances Python API docstring for motif finding, including CBO/join reordering tips. |
| docs/src/04-user-guide/04-motif-finding.md | Adds a new Performance section documenting Spark CBO settings for motif-finding joins. |
| core/src/main/scala/org/graphframes/GraphFrame.scala | Updates Scala Scaladoc for find with the same CBO/join reorder performance guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #845 +/- ##
==========================================
+ Coverage 80.75% 80.78% +0.03%
==========================================
Files 78 78
Lines 4421 4428 +7
Branches 543 536 -7
==========================================
+ Hits 3570 3577 +7
Misses 851 851 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@WeichenXu123 Thanks for the contribution! Could you please apply |
|
@WeichenXu123 Thanks for this contrition! I will trigger docs re-build. |
What changes were proposed in this pull request?
Add Spark CBO config tips for boosting motif finding performance
Why are the changes needed?
Doc and guide updates.