Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • lava lava
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 96
    • Issues 96
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 64
    • Merge requests 64
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • lava
  • lavalava
  • Issues
  • #452

Closed
Open
Created Nov 06, 2020 by Paul Sokolovsky@pfalconContributor

"Internal Server Error" when trying to view jobs in "stuck" state

I've posted a multinode job (to local Dockerized LAVA 2020.09 setup), which somehow didn't start well, and 2 subjobs stuck for a while in Scheduled/Scheduling state. When clicking on job details for these jobs, I was getting pages like:

500 Internal Server Error
Reverse for 'lava.scheduler.job.detail' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)$']

Oops, something has gone wrong!

And backtraces in the log like:

lava-server_1                | ERROR:django.request:Internal Server Error: /scheduler/job/3.1
lava-server_1                | Traceback (most recent call last):
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 41, in inner
lava-server_1                |     response = get_response(request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
lava-server_1                |     response = self.process_exception_by_middleware(e, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 185, in _get_response
lava-server_1                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lava-server_1                |   File "/usr/lib/python3/dist-packages/lava_scheduler_app/views.py", line 1642, in job_detail
lava-server_1                |     return render(request, "lava_scheduler_app/job.html", data)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/shortcuts.py", line 30, in render
lava-server_1                |     content = loader.render_to_string(template_name, context, request, using=using)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 68, in render_to_string
lava-server_1                |     return template.render(context, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 66, in render
lava-server_1                |     return self.template.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 207, in render
lava-server_1                |     return self._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 216, in render
lava-server_1                |     nodelist.append(node.render_annotated(context))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 458, in render
lava-server_1                |     url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/base.py", line 91, in reverse
lava-server_1                |     return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/resolvers.py", line 497, in _reverse_with_prefix
lava-server_1                |     raise NoReverseMatch(msg)
lava-server_1                | django.urls.exceptions.NoReverseMatch: Reverse for 'lava.scheduler.job.detail' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)$']

and

lava-server_1                | ERROR:django.request:Internal Server Error: /scheduler/job/3
lava-server_1                | Traceback (most recent call last):
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 41, in inner
lava-server_1                |     response = get_response(request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 187, in _get_response
lava-server_1                |     response = self.process_exception_by_middleware(e, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 185, in _get_response
lava-server_1                |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lava-server_1                |   File "/usr/lib/python3/dist-packages/lava_scheduler_app/views.py", line 1642, in job_detail
lava-server_1                |     return render(request, "lava_scheduler_app/job.html", data)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/shortcuts.py", line 30, in render
lava-server_1                |     content = loader.render_to_string(template_name, context, request, using=using)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader.py", line 68, in render_to_string
lava-server_1                |     return template.render(context, request)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 66, in render
lava-server_1                |     return self.template.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 207, in render
lava-server_1                |     return self._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 177, in render
lava-server_1                |     return compiled_parent._render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 199, in _render
lava-server_1                |     return self.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 72, in render
lava-server_1                |     result = block.nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 322, in render
lava-server_1                |     return nodelist.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 990, in render
lava-server_1                |     bit = node.render_annotated(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/base.py", line 957, in render_annotated
lava-server_1                |     return self.render(context)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 458, in render
lava-server_1                |     url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/base.py", line 91, in reverse
lava-server_1                |     return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
lava-server_1                |   File "/usr/lib/python3/dist-packages/django/urls/resolvers.py", line 497, in _reverse_with_prefix
lava-server_1                |     raise NoReverseMatch(msg)
lava-server_1                | django.urls.exceptions.NoReverseMatch: Reverse for 'lava.scheduler.job.cancel' with arguments '('',)' not found. 1 pattern(s) tried: ['scheduler/job/(?P<pk>[0-9]+|[0-9]+\\.[0-9]+)/cancel$']
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking