Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • lava lava
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 138
    • Issues 138
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 30
    • Merge requests 30
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • lava
  • lavalava
  • Issues
  • #531

Closed
Open
Created Mar 21, 2022 by Paweł Wieczorek@pawieczContributor

RFC: Change default counting mechanism for pagination

This is split off from #517.

Context: Default Django pagination handling requires applying access control filters and counting the entire dataset just to determine the exact number of pages to serve. Changing default counting mechanism for pagination (or even pagination mechanism) could improve perfomance but each approach comes with its drawbacks.

"A Dumb Paginator" (always returning constant instead of counting dataset) drops counting overhead entirely, but also drops precision and breaks the UI (while small datasets might not be affected by the pagination). Estimating returned constant is another point for consideration (instead of using e.g. sys.maxsize).

Time-limited paginator covers the case of dropping precision even for small datasets. For the worst case scenario (counting exceeds time limit) page "count" will be returned after period defined by the lab admin. Choice of the constant still applies.

Keyset pagination introduction could also result in significant performance boost by no longer requiring counting pages and filtering directly by the ordered field, e.g. TestJob ID or date fields.

COUNT approximation or COUNT caching complexity seems to be too much effort compared to possible gains (especially taking into account access control filters that would have to applied as well).

Please let me know if the LAVA team would be open to allow adding time-limited paginator and keyset pagination to the main codebase as a part of REST API v0.3 (on limited subset of endpoints for start).

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking