aix: select clang for Node.26+ builds

[abmusse]

Selects clang complier for Node.js 26+ builds

This is expected to land after we merge: nodejs/node#62656

[targos]

Does it need to be tested before we merge it?

[abmusse]

Yes Lets run a test build first to make sure things are working

[richardlau]

Test build: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/20/nodes=test-ibm-aix72-ppc64_be-2/console

[richardlau]

Does it need to be tested before we merge it?

Yes Lets run a test build first to make sure things are working

I've applied the Ansible change onto the machines and kicked off the test build above (which has selected clang and is building).

[richardlau]

Test build: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/20/nodes=test-ibm-aix72-ppc64_be-2/console

This built and failed one test (parallel/test-fs-rm).
I'm rerunning to see if this is a flake: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/21/nodes=test-ibm-aix72-ppc64_be-2/console

[targos]

It failed a test, but that's probably unrelated to the toolchain.

[richardlau]

Test build: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/20/nodes=test-ibm-aix72-ppc64_be-2/console

This built and failed one test (parallel/test-fs-rm). I'm rerunning to see if this is a flake: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/21/nodes=test-ibm-aix72-ppc64_be-2/console

The rerun failed the same test. If I log into the machine and run the test with the build in the workspace, it is consistently failing 😞.

bash-5.3$ ./tools/test.py --temp-dir /home/iojs/build/ parallel/test-fs-rm
=== release test-fs-rm ===
Path: parallel/test-fs-rm
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
hint:
hint: Disable this message with "git config set advice.defaultBranchName false"
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
hint:
hint: Disable this message with "git config set advice.defaultBranchName false"
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
hint:
hint: Disable this message with "git config set advice.defaultBranchName false"
/home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js:563
            throw err;
            ^

AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected

  Comparison {
+   code: '',
-   code: 'EACCES',
    name: 'Error'
  }

    at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js:553:18)
    at Module._compile (node:internal/modules/cjs/loader:1829:14)
    at Object..js (node:internal/modules/cjs/loader:1969:10)
    at Module.load (node:internal/modules/cjs/loader:1552:32)
    at Module._load (node:internal/modules/cjs/loader:1354:12)
    at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)
    at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
    at node:internal/main/run_main_module:33:47 {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: Error: , Unknown error: File exists '/home/iojs/build/.tmp.0/rm-11/fs-LcfMea'
      at Object.rmSync (node:fs:1206:18)
      at assert.throws.code.code (/home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js:554:16)
      at getActual (node:assert:580:5)
      at assert.throws (node:assert:728:24)
      at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js:553:18)
      at Module._compile (node:internal/modules/cjs/loader:1829:14)
      at Object..js (node:internal/modules/cjs/loader:1969:10)
      at Module.load (node:internal/modules/cjs/loader:1552:32)
      at Module._load (node:internal/modules/cjs/loader:1354:12)
      at wrapModuleLoad (node:internal/modules/cjs/loader:255:19) {
    errno: -4094,
    code: '',
    path: '/home/iojs/build/.tmp.0/rm-11/fs-LcfMea',
    syscall: 'rm'
  },
  expected: { code: 'EACCES', name: 'Error' },
  operator: 'throws',
  diff: 'simple'
}

Node.js v26.0.0-pre
Command: out/Release/node --expose-internals /home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js


[00:11|% 100|+   0|-   1]: Done

Failed tests:
out/Release/node --expose-internals /home/iojs/build/workspace/node-test-commit-aix-abmusse/nodes/test-ibm-aix72-ppc64_be-2/test/parallel/test-fs-rm.js
bash-5.3$

The Error: , Unknown error: File exists '/home/iojs/build/.tmp.0/rm-11/fs-LcfMea' message makes me wonder if we're getting a different error with clang instead of the expected EACCES, which would be odd (shouldn't it be linking to the same runtime libraries?).

[richardlau]

I'm also not sure ccache is working as expected -- the rerun took around the same amount of time (~2 hours) and while it was running I observed (ccache -s) the ccache misses increasing (which at least indicates the build is running through ccache) but the hits remaining the same.

[richardlau]

Re. the failing parallel/test-fs-rm test, I spoke to @abmusse last night and he continued debugging after I had stopped. It looks like there is a difference in behaviour with libc++ on AIX that might be a bug.

I've put together a potential workaround patch based on the notes he left me (we'll discuss when he's online) and am running that through CI to see if that addresses the test failure: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/22/nodes=test-ibm-aix72-ppc64_be-2/console
If it does, I'll open a PR that we can fast track to unblock this (which can then unblock the V8 update) (or maybe I'll optimistically open a PR so we can have CIs running on the other platforms while we wait for the above test with clang).

Separately we'll report/see if we can address in libc++ as a proper long term solution but that's not likely to be a quick turnaround.

[richardlau]

I've put together a potential workaround patch based on the notes he left me (we'll discuss when he's online) and am running that through CI to see if that addresses the test failure: https://ci.nodejs.org/job/node-test-commit-aix-abmusse/22/nodes=test-ibm-aix72-ppc64_be-2/console

This passed 🎉. I've opened nodejs/node#62788.