Skip to content

Conversation

@dnhatn
Copy link
Member

@dnhatn dnhatn commented Jan 2, 2025

Currently, we only account for attributes after the TopN operation when estimating its row size. This results in invalid estimates for the final TopN on the coordinator and the node-level reduction TopN. This change corrects the estimates and removes the workaround, paving the way for enabling node-level reduction in ES|QL.

Closes #106956

@dnhatn dnhatn force-pushed the row-size-estimate branch 2 times, most recently from 2a4debf to 1a44abe Compare January 2, 2025 21:14
@elasticsearchmachine
Copy link
Collaborator

Hi @dnhatn, I've created a changelog YAML for you.

@dnhatn dnhatn requested a review from nik9000 January 2, 2025 22:23
@dnhatn dnhatn requested a review from astefan January 2, 2025 22:23
@dnhatn dnhatn marked this pull request as ready for review January 2, 2025 22:23
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 2, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@nik9000
Copy link
Member

nik9000 commented Jan 3, 2025

Oh boy! I remember estimating row sizes! Thanks for fixing this!

@dnhatn dnhatn added the auto-backport Automatically create backport pull requests when merged label Jan 3, 2025
@dnhatn
Copy link
Member Author

dnhatn commented Jan 3, 2025

Thanks Nik!

@dnhatn dnhatn merged commit ce2c5d9 into elastic:main Jan 3, 2025
16 checks passed
@dnhatn dnhatn deleted the row-size-estimate branch January 3, 2025 21:43
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

dnhatn added a commit to dnhatn/elasticsearch that referenced this pull request Jan 3, 2025
Currently, we only account for attributes after the TopN operation when 
estimating its row size. This results in invalid estimates for the final
TopN on the coordinator and the node-level reduction TopN. This change
corrects the estimates and removes the workaround, paving the way for
enabling node-level reduction in ES|QL.

Closes elastic#106956
elasticsearchmachine pushed a commit that referenced this pull request Jan 3, 2025
Currently, we only account for attributes after the TopN operation when 
estimating its row size. This results in invalid estimates for the final
TopN on the coordinator and the node-level reduction TopN. This change
corrects the estimates and removes the workaround, paving the way for
enabling node-level reduction in ES|QL.

Closes #106956
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ESQL: a more accurate node-level reduction estimated row size value

3 participants