Skip to content

citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

New issue
New issue
Closed
Closed
citgm/stress jobs do not select compiler for AIX on v10.x/master#1254
Labels
ci-public

Description

@richardlau
richardlau
opened on Apr 27, 2018

Refs: nodejs/node#20137 (comment)

The stress job appears not to be using the compiler selector script added by #1240 so it's using the wrong level of gcc on AIX for master/v10.x.

e.g., https://ci.nodejs.org/job/node-stress-single-test/1825/nodes=aix61-ppc64/consoleFull

16:38:06 WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=g++)

Activity

  1. addaleax commented on Apr 28, 2018

    @addaleax
    addaleax
    on Apr 28, 2018
    Member

    This apparently goes for CITGM as well: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1390/nodes=rhel72-s390x/console

  2. addaleax
    added 2 commits that reference this issue on Apr 28, 2018

    src: avoid `std::make_unique`

    7a6b63d

    src: avoid `std::make_unique`

    283a967

  3. mhdawson commented on Apr 30, 2018

    @mhdawson
    mhdawson
    on Apr 30, 2018
    Member

    Fixing the stress job.

    @gdams can you update the CITGM jobs, they need to use

    rm -rf build
git clone https://github.com/nodejs/build.git
. ./build/jenkins/scripts/select-compiler.sh

    For all platforms (selection is currently done on PPC, s390 and AIX, but will be on others in the future). In addition for AIX if there is a CC= definition it should be removed as that is now set by the selection script.

  4. mhdawson commented on May 1, 2018

    @mhdawson
    mhdawson
    on May 1, 2018
    Member

    Stress job fixed.

  5. MylesBorins
    added a commit that references this issue on May 4, 2018

    src: avoid `std::make_unique`

    58be6ef

  6. richardlau commented on May 22, 2018

    @richardlau
    richardlau
    on May 22, 2018
    MemberAuthor

    Ping @gdams re. the CitGM jobs ^

    Note that this is affecting the v10.2.0 PR: nodejs/node#20724 (comment)

  7. maclover7
    added
    ci-public
    on May 22, 2018

  8. mhdawson commented on May 23, 2018

    @mhdawson
    mhdawson
    on May 23, 2018
    Member

    @gdams can you prioritize updating the Citgm jobs?

  9. gdams commented on May 23, 2018

    @gdams
    gdams
    on May 23, 2018
    Member

    I have added this script to the citgm-smoker job

  10. MylesBorins commented on May 23, 2018

    @MylesBorins
    MylesBorins
    on May 23, 2018
    Contributor

    thanks all! Can this be considered fixed now?

  11. mhdawson commented on May 23, 2018

    @mhdawson
    mhdawson
    on May 23, 2018
    Member

    @gdams did you test with a citgm run? I think we want to do that and then we can close.

    The key thing is to make sure that the job selects the right compiler based on the Node.js version. If the job builds node.js from source then it should do the right thing. For jobs that use existing binaries it needs an extra step that I added to the job for the tests for the node-addon-api.

  12. richardlau commented on May 23, 2018

    @richardlau
    richardlau
    on May 23, 2018
    MemberAuthor

    I think the CitGM jobs may need more work. From the latest v10.2.0 CitGM runs:

    rhel72-s390x 
    + . ./build/jenkins/scripts/select-compiler.sh
++ '[' '' '!=' DONT ']'
++ case $NODE_NAME in
++ SELECT_ARCH=S390X
++ '[' S390X = PPC64LE ']'
++ '[' S390X = S390X ']'
++ export COMPILER_LEVEL=
++ COMPILER_LEVEL=
+++ python tools/getnodeversion.py
++ NODE_VERSION=10.2.0
+++ echo 10.2.0
+++ cut -d . -f 1
++ NODE_MAJOR_VERSION=10
++ echo 'Setting compiler for Node version 10 on s390x'
Setting compiler for Node version 10 on s390x
++ '[' 10 -gt 9 ']'
++ export PATH=/data/gcc-4.9/bin:/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ PATH=/data/gcc-4.9/bin:/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ export LD_LIBRARY_PATH=/data/gcc-4.9/lib64:
++ LD_LIBRARY_PATH=/data/gcc-4.9/lib64:
++ export COMPILER_LEVEL=-4.9
++ COMPILER_LEVEL=-4.9
++ export 'CC=ccache gcc-4.9'
++ CC='ccache gcc-4.9'
++ export 'CXX=ccache g++-4.9'
++ CXX='ccache g++-4.9'
++ export LINK=g++-4.9
++ LINK=g++-4.9
++ echo 'Compiler set to -4.9'
Compiler set to -4.9
...
+ node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)

    https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/nodes=rhel72-s390x/consoleFull

    aix61-ppc64 
    + . ./build/jenkins/scripts/select-compiler.sh
++ '[' '' '!=' DONT ']'
++ case $NODE_NAME in
++ SELECT_ARCH=AIXPPC
++ '[' AIXPPC = PPC64LE ']'
++ '[' AIXPPC = S390X ']'
++ '[' AIXPPC = AIXPPC ']'
+++ python tools/getnodeversion.py
++ NODE_VERSION=10.2.0
+++ echo 10.2.0
+++ cut -d . -f 1
++ NODE_MAJOR_VERSION=10
++ echo 'Setting compiler for Node version 10 on AIX'
Setting compiler for Node version 10 on AIX
++ '[' 10 -gt 9 ']'
++ export LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
++ LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
++ export PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
++ PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
+++ which gcc
+++ which g++
+++ which g++
++ export 'CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc' 'CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++' 'CXX_host=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ CC='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc'
++ CXX='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ CXX_host='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ echo 'Compiler set to 6.3'
Compiler set to 6.3
+ CONFIG_FLAGS='--with-intl=small-icu --download=all --download-path=/home/iojs/node-icu/'
+ mkdir -p /home/iojs/node-icu/
+ [[ 10.2.0 =~ ^0\.10 ]]
+ rm -rf /home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker
+ mkdir /home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker
++ uname -s
+ '[' AIX = AIX ']'
+ MAKE=gmake
+ unset LIBPATH
+ echo

+ python ./configure --prefix=/home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker --with-intl=small-icu --download=all --download-path=/home/iojs/node-icu/
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
	0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
	0509-152   Member libmpc.so.3 is not found in archive 
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1 because of the following errors:
	0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
	0509-152   Member libmpc.so.3 is not found in archive 
WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++)
WARNING: C compiler too old, need gcc 4.2 or clang 3.2 (CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc)
Traceback (most recent call last):
  File "./configure", line 1475, in <module>
    configure_node(output)
  File "./configure", line 886, in configure_node
    host_arch = host_arch_win() if os.name == 'nt' else host_arch_cc()
  File "./configure", line 798, in host_arch_cc
    k = cc_macros('gcc')
  File "./configure", line 755, in cc_macros
    p.stdin.write('\n')
IOError: [Errno 32] Broken pipe

    https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/nodes=aix61-ppc64/console

    We shouldn't be attempting to build Node.js 10 on AIX 6.1 as the minimum supported level is 7.1 TL04: https://github.com/nodejs/node/blob/v10.0.0/BUILDING.md#supported-platforms
    (aix61-ppc64 is the only AIX axis in https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/).

  13. mhdawson commented on May 23, 2018

    @mhdawson
    mhdawson
    on May 23, 2018
    Member

    @richardlau, while in the perfect world you are correct for Node 10.x the reality is that we don't have the hw at this point and building on 6.1 is what we need to do. We made the minimum level 7.1 to that we'd have the flexibility of moving up to that but we are not planning to build the infrastructure in the near future so we SHOULD continue to build 10.x on AIX 6.1 for now.

  14. richardlau commented on May 23, 2018

    @richardlau
    richardlau
    on May 23, 2018
    MemberAuthor

    @mhdawson ack. The CitGM job is failing on AIX with the compiler script though.

  15. mhdawson commented on May 23, 2018

    @mhdawson
    mhdawson
    on May 23, 2018
    Member

    @gdams the problem with the runs shown for 9.x and 10.x above is that previously CC was set in the configure step. When I updated the other jobs I had to remove that as CC is now set in the compiler selection script. I'm guessing that if the original set of CC is still there it is overwriting what the script had put in place.

  16. richardlau
    changed the title [-]Stress job doesn't select compiler for AIX on v10.x/master[/-] [+]citgm/stress jobs do not select compiler for AIX on v10.x/master[/+] on May 24, 2018

  17. mhdawson commented on May 29, 2018

    @mhdawson
    mhdawson
    on May 29, 2018
    Member

    @gdams do you have a timeline for when you might be able to look at this?

  18. mhdawson commented on May 29, 2018

    @mhdawson
    mhdawson
    on May 29, 2018
    Member

    One problem was that the LIBPATH was being unset after the selection, moved it to be after and we'll see if that helps. @gdams I only changed this job https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/ so others probably need the same change.

  19. mhdawson commented on May 29, 2018

    @mhdawson
    mhdawson
    on May 29, 2018
    Member

    Looks to have run much further, but still compiling.

  20. gdams commented on May 29, 2018

    @gdams
    gdams
    on May 29, 2018
    Member

    so I can see the following failure on the aix jobs:

    + . ./build/jenkins/scripts/select-compiler.sh
+ [  != DONT ]
+ SELECT_ARCH=AIXPPC
+ [ AIXPPC = PPC64LE ]
+ [ AIXPPC = S390X ]
+ [ AIXPPC = AIXPPC ]
+ + python tools/getnodeversion.py
python: can't open file 'tools/getnodeversion.py': [Errno 2] No such file or directory
NODE_VERSION=

    I'll take a look at what's going wrong

  21. mhdawson commented on May 29, 2018

    @mhdawson
    mhdawson
    on May 29, 2018
    Member

    I fixed it up and seems to be ok now, had to do with the current directory when the selection was being done.

  22. mhdawson commented on May 29, 2018

    @mhdawson
    mhdawson
    on May 29, 2018
    Member

    It now ran ok I believe but like other platforms there are failures. https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1445/nodes=aix61-ppc64/console

    @gdams can you see if these match the other platforms. If so I think we can close this issue.

  23. gdams commented on May 30, 2018

    @gdams
    gdams
    on May 30, 2018
    Member

    I'm still seeing this failure on s390x:

    + node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)

    https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/nodes=rhel72-s390x/1446/console

  24. richardlau commented on Nov 29, 2018

    @richardlau
    richardlau
    on Nov 29, 2018
    MemberAuthor

    I'm still seeing this failure on s390x:

    + node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)

    https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/nodes=rhel72-s390x/1446/console

    This was fixed by #1548.

  25. richardlau
    closed this as completedon Nov 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-public

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions