multinode lava-wait, etc.: Default timeout is too short, no visible way to adjust
We're using multinode synchronization support for "interactive" tests, as added in LAVA 2020.05. These are networking test jobs, with a following plan: 2 devices, a Zephyr board and a virtual docker device, serving as a host. A server-like application is deployed and booted on a board, after boot it issues "lava-send booted". The host lava-wait's for "booted", and once received, starts to perform networking tests against board. The board in the meantime "lava-wait done". Once host completes, it issues "lava-send done", and the test process for both devices is finished.
So, what I found that all LAVA requests are actually issues (against a multinode coordinator) with a timeout, and this timeout for actual commands is 30s (it seems that timeout for initial/final connection is 90s). 30s is clearly too small a timeout for lava-wait, lava-wait-all, etc.
The problem is demonstrated by https://validation.linaro.org/scheduler/job/2063026.1#L71 .