tap2junit/build/python nightly failed with unicode problem

[sam-github]

00:21:52   File "/usr/local/lib/python2.7/site-packages/yaml/reader.py", line 144, in check_printable
00:21:52     'unicode', "special characters are not allowed")
00:21:52 yaml.reader.ReaderError: unacceptable character #x001b: special characters are not allowed

Nightly build from 2 days ago:

• https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1795/
  OS failure:
• https://ci.nodejs.org/job/node-test-commit-osx/31139/nodes=osx1011/console

00:21:51 + tap2junit -i test.tap -o test.xml
00:21:52 Traceback (most recent call last):
00:21:52   File "/usr/local/bin/tap2junit", line 10, in <module>
00:21:52     sys.exit(main())
00:21:52   File "/usr/local/lib/python2.7/site-packages/tap2junit/__main__.py", line 51, in main
00:21:52     convert(args.input, args.output)
00:21:52   File "/usr/local/lib/python2.7/site-packages/tap2junit/__main__.py", line 42, in convert
00:21:52     result = parse(input_file, data)
00:21:52   File "/usr/local/lib/python2.7/site-packages/tap2junit/__main__.py", line 34, in parse
00:21:52     tap_parser.parse(data)
00:21:52   File "/usr/local/lib/python2.7/site-packages/tap2junit/tap13.py", line 147, in parse
00:21:52     self._parse(StringIO.StringIO(source))
00:21:52   File "/usr/local/lib/python2.7/site-packages/tap2junit/tap13.py", line 78, in _parse
00:21:52     self.tests[-1].yaml = yamlish.load(self.tests[-1].yaml_buffer)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yamlish.py", line 222, in load
00:21:52     out = load('\n'.join(inobj), ignore_wrong_characters)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yamlish.py", line 203, in load
00:21:52     out = yaml.load(source, Loader=_YamlishLoader)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 69, in load
00:21:52     loader = Loader(stream)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yamlish.py", line 145, in __init__
00:21:52     yaml.loader.SafeLoader.__init__(self, stream)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yaml/loader.py", line 24, in __init__
00:21:52     Reader.__init__(self, stream)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yaml/reader.py", line 74, in __init__
00:21:52     self.check_printable(stream)
00:21:52   File "/usr/local/lib/python2.7/site-packages/yaml/reader.py", line 144, in check_printable
00:21:52     'unicode', "special characters are not allowed")
00:21:52 yaml.reader.ReaderError: unacceptable character #x001b: special characters are not allowed
00:21:52   in "<unicode string>", position 114

[richardlau]

FWIW the failing test is #31094 and has also been observed failing on freebsd, https://ci.nodejs.org/job/node-test-commit-freebsd/30729/nodes=freebsd11-x64/console where it also caused the tap2junit failure reported in this issue.

[shackijj]

Looked at last four failed jobs.

https://ci.nodejs.org/job/node-daily-master/1932/
https://ci.nodejs.org/job/node-daily-master/1931/
https://ci.nodejs.org/job/node-daily-master/1930/
https://ci.nodejs.org/job/node-daily-master/1929/

It seems that the unicode related error is no longer causes problem.

pummel/test-net-throttle fails in the jobs I mentioned earlier. It's written in the README that pummel directory doesn't run on CI.

Unfortunately, I can't see the last success job and don't have access to CI configuration. Could anyone tell if tests in this job should pass with no errors? I tried to launch the same command that CI does. There were 25 errors.

[shackijj]

@richardlau @sam-github Could anyone please answer my question above?

I made a couple of pull requests during the last two month. Every second job that were launched through my PRs failed because of flaky tests or other not related to my code reasons. Therefore, I think that this issue is very important. Daily builds allow us to monitor stability of master branch.

Ready to help here, but I don't have access to CI configuration because I'm not a colloborator.

I checked the last two failed jobs. Just to make sure that the builds still fail.

https://ci.nodejs.org/job/node-test-commit-linux/nodes=alpine-latest-x64/35728/

13:00:30 Please use either Python 2.6 or 2.7
13:00:30 make: *** [Makefile:475: build-ci] Error 1
13:00:30 Build step 'Execute shell' marked build as failure

https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/14966

10:16:40 not ok 66 pummel/test-crypto-timing-safe-equal-benchmarks
10:16:40   ---
10:16:40   duration_ms: 6.527
10:16:40   severity: fail
10:16:40   exitcode: 1
10:16:40   stack: |-
10:16:40     assert.js:385
10:16:40         throw err;

[richardlau]

I think there is a Unicode issue with tap2junit but it only manifests if a test fails that happens to output certain Unicode characters.

Otherwise yes unfortunately we do have flaky tests and not many people appear to be monitoring the daily builds on a regular basis. There is an OpenJS World Collaborator Summit session on CI Stability this Thursday for folks to get together to brainstorm ideas.
cc @Trott @AshCripps

[shackijj]

I'm looking at this job.
It seems that it failed because of this and that. I don't see the unicode issue there.

@richardlau please correct me if i'm wrong.

[richardlau]

Correct, you're not seeing the Unicode issue.

https://ci.nodejs.org/job/node-test-commit-linux/35719/nodes=alpine-latest-x64/console is probably related to the alpine changes in nodejs/build#2364 (cc @rvagg)

[rvagg]

the alpine problem should be gone now with nodejs/build#2364 merged - it's excluded from 10 and 12.x now