RFC: lava_dispatcher: Rework ShellLogger to preserve all characters
The rework has following properties:
1). Add support for \r\n and \r line separators. Some software uses \r as line separators and before that LAVA would simply log entire output as one big line if the software used \r. LAVA would also stall all output until it could read the new line.
2). Make logs preserve unprintable characters such as new lines, carriage return and ANSI terminal escape sequences. This allows for better compatibility with other software that relies on those characters such as Gitlab. The unprintable characters are stripped from output before the logs are displayed in the LAVA's web UI.
Unit test will check various logger inputs with different new line separators and fragmented inputs. (for example, if \r\n gets split between the write calls)
We at Collabora had issues with how LAVA would process the logs. For example, LAVA would strip all terminal control characters which made Gitlab's collapsible sections impossible:
https://docs.gitlab.com/ee/ci/jobs/index.html#custom-collapsible-sections
This rework allows to 100% reconstructable logs.
The unprintable characters are now filtered before display on the web UI rather when saving logs.
This is a breaking change because previously to approximately reconstruct
the logs you would do "\n".join(x for x in logs) now the correct way would be
"".join(x for x in logs) meaning now reading the logs would already provide
the necessary newlines.
This supersedes the #561.