Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
lava
lava
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 112
    • Issues 112
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 25
    • Merge Requests 25
  • Requirements
    • Requirements
    • List
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • lava
  • lavalava
  • Issues
  • #452

Closed
Open
Opened Nov 06, 2020 by Paul Sokolovsky@pfalconContributor

"Internal Server Error" when trying to view jobs in "stuck" state

I've posted a multinode job (to local Dockerized LAVA 2020.09 setup), which somehow didn't start well, and 2 subjobs stuck for a while in Scheduled/Scheduling state. When clicking on job details for these jobs, I was getting pages like:

500 Internal Server Error
Reverse for 'lava.scheduler.job.detail' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)$']

Oops, something has gone wrong!

And backtraces in the log like:

lava-server_1                | ERROR:django.request:Internal Server Error: /scheduler/job/3.1
lava-server_1                | Traceback (most recent call last):
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 41, in inner
lava-server_1                |     response = get_response(request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
lava-server_1                |     response = self.process_exception_by_middleware(e, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 185, in _get_response
lava-server_1                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lava-server_1                |   File "/usr/lib/python3/dist-packages/lava_scheduler_app/views.py", line 1642, in job_detail
lava-server_1                |     return render(request, "lava_scheduler_app/job.html", data)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/shortcuts.py", line 30, in render
lava-server_1                |     content = loader.render_to_string(template_name, context, request, using=using)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 68, in render_to_string
lava-server_1                |     return template.render(context, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 66, in render
lava-server_1                |     return self.template.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 207, in render
lava-server_1                |     return self._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 216, in render
lava-server_1                |     nodelist.append(node.render_annotated(context))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 458, in render
lava-server_1                |     url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/base.py", line 91, in reverse
lava-server_1                |     return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/resolvers.py", line 497, in _reverse_with_prefix
lava-server_1                |     raise NoReverseMatch(msg)
lava-server_1                | django.urls.exceptions.NoReverseMatch: Reverse for 'lava.scheduler.job.detail' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)$']

and

lava-server_1                | ERROR:django.request:Internal Server Error: /scheduler/job/3
lava-server_1                | Traceback (most recent call last):
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 41, in inner
lava-server_1                |     response = get_response(request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
lava-server_1                |     response = self.process_exception_by_middleware(e, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 185, in _get_response
lava-server_1                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lava-server_1                |   File "/usr/lib/python3/dist-packages/lava_scheduler_app/views.py", line 1642, in job_detail
lava-server_1                |     return render(request, "lava_scheduler_app/job.html", data)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/shortcuts.py", line 30, in render
lava-server_1                |     content = loader.render_to_string(template_name, context, request, using=using)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 68, in render_to_string
lava-server_1                |     return template.render(context, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 66, in render
lava-server_1                |     return self.template.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 207, in render
lava-server_1                |     return self._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 458, in render
lava-server_1                |     url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/base.py", line 91, in reverse
lava-server_1                |     return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/resolvers.py", line 497, in _reverse_with_prefix
lava-server_1                |     raise NoReverseMatch(msg)
lava-server_1                | django.urls.exceptions.NoReverseMatch: Reverse for 'lava.scheduler.job.cancel' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)/cancel$']
To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
2021.01
Milestone
2021.01 (Past due)
Assign milestone
Time tracking
None
Due date
None
Reference: lava/lava#452