Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • lava lava
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 138
    • Issues 138
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 28
    • Merge requests 28
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • lava
  • lavalava
  • Issues
  • #517

Closed
Open
Created Nov 11, 2021 by Paweł Wieczorek@pawieczContributor

Database performance drops

Context: with >2M rows in lava_scheduler_app_testjob table and ~2k jobs in queue (to be scheduled) database performance drops significantly.

Analysis of highest-cost queries with pg_stat_statements showed that most of their execution time is spent scanning TestJob entries to filter the ones which can be accessed by the user who made a query. Database is also queried twice due to pagination handling in Django - first to count the items, second to get the data.

Mitigation methods suggested by the Django/PostgreSQL communities involve:

  • timing out initial count query [1]
  • row number estimation [1][2]
  • switching to keyset pagination [2]

[1] https://hakibenita.com/optimizing-the-django-admin-paginator [2] https://pganalyze.com/blog/pagination-django-postgres

Are there mitigation methods recommended by the LAVA team?

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking