checkout@v3 using node20 instead of node16

[jcooke-impinj]

Running on rhel7 with self-hosted runners and starting today we're running into issues where @V3 is using node20 instead of node16.
Happening on runner versions 2.316.1 and 2.317.0.
Oddly some of our Repos don't run into this issue when executing checkout@v3 on the same runners.

Run actions/checkout@v3
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version GLIBC_2.28' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version GLIBC_2.25' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)

[jcooke-impinj]

Our team found that setting these before starting the runners get's things going again. Still unsure what's causing this.

export ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
export ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

[adpaco-aws]

This doesn't seem to be specific to actions/checkout@v3 since we tried to upgrade to actions/checkout@v4 (here) and got the same failure (here).

[biswalc]

I am having the same issue while running below versions, on ubuntu 20.04 VM:

actions/[email protected]
actions/checkout@v3
actions/checkout@v4
actions/[email protected]

Error:

/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

Issue started today, i.e. 3 Jul 10am PDT.

[junghyun-apr]

Hello! Our development team is also experiencing the same issue. We are using AWS instances as self-hosted.
We temporarily resolved the issue by simply changing the node16 directory to node20 in the same path. 🥲

AWS AMI :

amzn2-ami-kernel-5.10-hvm-2.0.20240223.0-x86_64-gp2

Git Action Error :

/home/ec2-user/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/ec2-user/actions-runner/externals/node20/bin/node)
/home/ec2-user/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /home/ec2-user/actions-runner/externals/node20/bin/node)

[GuyAv46]

Following this blog post: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Adding this to the envs that require node16 fixed the issue

ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true

[Flamefire]

Our team found that setting these before starting the runners get's things going again. Still unsure what's causing this.

export ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
export ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

I tried setting this inside a workflow running on GH hosted runners and it didn't work.
So it looks like this only works when the runner itself was started with those set, i.e. you cannot set them for a running runner which includes the GH hosted ones.

This doesn't seem to be specific to actions/checkout@v3 since we tried to upgrade to actions/checkout@v4 (here) and got the same failure (here).

v4 always used Node 20

I am having the same issue while running below versions, on ubuntu 20.04 VM:

actions/[email protected]
actions/checkout@v3
actions/checkout@v4
actions/[email protected]

See above

[david-batranu]

Fixed by renaming the node20 folder and symlinking node16 to node20:

cd $RUNNER_HOME/actions-runner/externals
mv node20 node20.bak
ln -s node16 node20

[eshogo]

I tried modifying /home/runner/actions-runner/.env and env.sh and it works fine.

% cat .env
ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

and adding their environment variables into 'varCheckList' array in env.sh.
(not needed?)

varCheckList=(
    'LANG'
    'JAVA_HOME'
    'ANT_HOME'
    'M2_HOME'
    'ANDROID_HOME'
    'ANDROID_SDK_ROOT'
    'GRADLE_HOME'
    'NVM_BIN'
    'NVM_PATH'
    'LD_LIBRARY_PATH'
    'PERL5LIB'
    'ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION'
    'ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION'
    )

Then restart runner service

% systemctl restart github-actions-runner.service

[david-batranu]

Looks like this is happening because of: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/

[n1ngu]

Following this blog post: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ Adding this to the envs that require node16 fixed the issue

ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true

Quoting the post paragraph about that workaround: "This will only work until we upgrade the runner removing Node16 later in the spring"

We are walking a tightrope.

Just wanted to raise some awareness about that. Plan to actually upgrade your github actions in the short term or move away from them. E.g. I think I'll quit my job. 😄

[biswalc]

I am having the same issue while running below versions, on ubuntu 20.04 VM:

actions/[email protected]
actions/checkout@v3
actions/checkout@v4
actions/[email protected]

Error:

/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

Issue started today, i.e. 3 Jul 10am PDT.

My glibc version:

ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

So why do I get this error? I am using [email protected]. I do not understand what I have to do to upgrade.

[ayizhi]

Here's a critical issue: Firstly, you shouldn't upgrade such fundamental environments arbitrarily. Even if you do upgrade, you must ensure the environment is correct. Upgrading to Node 20 and causing a glib error is absolutely unacceptable, especially in a product from a company like Microsoft.

[FelixSchwarz]

I wish there was an alternative version of the checkout action, maybe written in rust so we could just deploy a static binary. Then we would just compile that against a really old version of glibc. This would also simplify running GitHub Actions with nektos/act which does not map node into the container.

Update: I found checkout-action in GitHub's market place and added support for private repos. Not sure if that is a permanent solution but at least our CI jobs are passing again...

[hasufell]

This means you now can't run any docker image that has a glibc older than 2.27. That includes:

• debian:9
• fedora:27
• ubuntu:18.04

What a huge f*ckup. I'm baffled by the amount of incompetence. What's wrong with providing a statically linked node binary? Do you need help with redistribution github?

[sawwn23]

Fixed by renaming the node20 folder and symlinking node16 to node20:

cd $RUNNER_HOME/actions-runner/externals
mv node20 node20.bak
ln -s node16 node20

Thanks for your workaround solution

[eli-schwartz]

Upgrading the version of node can be done independently of upgrading the version of glibc. Building the node binary redistributables used by actions should ideally be done using either one of:

• building it using an old glibc at compile time, so that it works well on both old and new containers

• building node statically linked, so that it doesn't care which glibc version is available in the container

The former has very little to do with dropping support for EOL stacks, and the latter has nothing to do with dropping support for EOL stacks.

Therefore IMHO this breaking change was totally unnecessary.

Oh also, by the way: it appears this has actually been proposed in actions/runner#3128 back in February.