logo
57
174
WeChat Login

npm install安装依赖时频繁遇到网络超时#967

Resolved
created 2025-06-13
Edit

描述一下体验不好的点

似乎与特定节点有关系,amd64节点不管使用哪个源都会报ERR_SOCKET_TIMEOUT,但在arm64节点则一切正常。

#935 也反馈了相同的问题,希望能尽早解决

[docker build admin] #8 [builder 3/8] RUN npm install -g pnpm
[docker build admin] #8 1.257 
[docker build admin] #8 1.257 added 1 package in 937ms
[docker build admin] #8 1.257 
[docker build admin] #8 1.257 1 package is looking for funding
[docker build admin] #8 1.257   run `npm fund` for details
[docker build admin] #8 1.258 npm notice
[docker build admin] #8 1.258 npm notice New major version of npm available! 10.8.2 -> 11.4.2
[docker build admin] #8 1.258 npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.4.2
[docker build admin] #8 1.258 npm notice To update run: npm install -g npm@11.4.2
[docker build admin] #8 1.258 npm notice
[docker build admin] #8 DONE 1.3s
[docker build admin] 
[docker build admin] #10 [builder 4/8] COPY package.json pnpm-lock.yaml ./
[docker build admin] #10 DONE 0.0s
[docker build admin] 
[docker build admin] #11 [builder 5/8] RUN pnpm install --frozen-lockfile
[docker build admin] #11 0.604 Lockfile is up to date, resolution step is skipped
[docker build admin] #11 0.690 Progress: resolved 1, reused 0, downloaded 0, added 0
[docker build admin] #11 114.8 Packages: +558
[docker build admin] #11 114.8 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[docker build admin] #11 204.1 Progress: resolved 558, reused 0, downloaded 0, added 0
[docker build admin] #11 208.7  WARN  GET https://registry.npmjs.org/@headlessui/react/-/react-2.2.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 208.9  WARN  GET https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.7.6.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 209.0  WARN  GET https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 209.0  WARN  GET https://registry.npmjs.org/esbuild/-/esbuild-0.19.12.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 209.4  WARN  GET https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.19.12.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 209.5  WARN  GET https://registry.npmjs.org/esbuild-register/-/esbuild-register-3.5.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 210.0  WARN  GET https://registry.npmjs.org/drizzle-orm/-/drizzle-orm-0.31.4.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 210.1  WARN  GET https://registry.npmjs.org/mysql2/-/mysql2-3.11.4.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.0  WARN  GET https://registry.npmjs.org/aws-ssl-profiles/-/aws-ssl-profiles-1.1.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.0  WARN  GET https://registry.npmjs.org/denque/-/denque-2.1.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.0  WARN  GET https://registry.npmjs.org/generate-function/-/generate-function-2.3.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.0  WARN  GET https://registry.npmjs.org/is-property/-/is-property-1.0.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/long/-/long-5.2.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/lru.min/-/lru.min-1.1.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/named-placeholders/-/named-placeholders-1.1.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/lru-cache/-/lru-cache-7.18.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/seq-queue/-/seq-queue-0.0.5.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/sqlstring/-/sqlstring-2.3.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/drizzle-zod/-/drizzle-zod-0.5.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/zod/-/zod-3.23.8.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/i18next/-/i18next-24.2.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/typescript/-/typescript-5.5.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/lucide-react/-/lucide-react-0.400.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/next-auth/-/next-auth-5.0.0-beta.25.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/@auth/core/-/core-0.37.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/@panva/hkdf/-/hkdf-1.2.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/@types/cookie/-/cookie-0.6.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/cookie/-/cookie-0.7.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/jose/-/jose-5.9.6.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/oauth4webapi/-/oauth4webapi-3.1.4.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/preact/-/preact-10.11.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/preact-render-to-string/-/preact-render-to-string-5.2.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/pretty-format/-/pretty-format-3.8.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/@floating-ui/react/-/react-0.26.28.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/react-day-picker/-/react-day-picker-8.10.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/react-i18next/-/react-i18next-15.4.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/html-parse-stringify/-/html-parse-stringify-3.0.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/void-elements/-/void-elements-3.1.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/react-syntax-highlighter/-/react-syntax-highlighter-15.6.1.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/highlight.js/-/highlight.js-10.7.3.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/highlightjs-vue/-/highlightjs-vue-1.0.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/lowlight/-/lowlight-1.20.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/fault/-/fault-1.0.4.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/format/-/format-0.2.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/prismjs/-/prismjs-1.29.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/refractor/-/refractor-3.6.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/hastscript/-/hastscript-6.0.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.1  WARN  GET https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-1.0.8.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 335.5  WARN  GET https://registry.npmjs.org/space-separated-tokens/-/space-separated-tokens-1.1.5.tgz error (ETIMEDOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] #11 337.0 Progress: resolved 558, reused 0, downloaded 1, added 0
[docker build admin] #11 339.7 Progress: resolved 558, reused 0, downloaded 46, added 3
[docker build admin] #11 343.2 Progress: resolved 558, reused 0, downloaded 108, added 11
[docker build admin] #11 347.3 Progress: resolved 558, reused 0, downloaded 203, added 19
[docker build admin] #11 ERROR: process "/bin/sh -c pnpm install --frozen-lockfile" did not complete successfully: exit code: 137
[docker build admin] ------
[docker build admin]  > [builder 5/8] RUN pnpm install --frozen-lockfile:
[docker build admin] 335.1  WARN  GET https://registry.npmjs.org/format/-/format-0.2.2.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 335.1  WARN  GET https://registry.npmjs.org/prismjs/-/prismjs-1.29.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 335.1  WARN  GET https://registry.npmjs.org/refractor/-/refractor-3.6.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 335.1  WARN  GET https://registry.npmjs.org/hastscript/-/hastscript-6.0.0.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 335.1  WARN  GET https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-1.0.8.tgz error (ERR_SOCKET_TIMEOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 335.5  WARN  GET https://registry.npmjs.org/space-separated-tokens/-/space-separated-tokens-1.1.5.tgz error (ETIMEDOUT). Will retry in 10 seconds. 2 retries left.
[docker build admin] 337.0 Progress: resolved 558, reused 0, downloaded 1, added 0
[docker build admin] 339.7 Progress: resolved 558, reused 0, downloaded 46, added 3
[docker build admin] 343.2 Progress: resolved 558, reused 0, downloaded 108, added 11
[docker build admin] 347.3 Progress: resolved 558, reused 0, downloaded 203, added 19
[docker build admin] ------
[docker build admin] Dockerfile:7
[docker build admin] --------------------
[docker build admin]    5 |     
[docker build admin]    6 |     COPY package.json pnpm-lock.yaml ./
[docker build admin]    7 | >>> RUN pnpm install --frozen-lockfile
[docker build admin]    8 |     RUN npx update-browserslist-db@latest
[docker build admin]    9 |     
[docker build admin] --------------------
[docker build admin] ERROR: failed to solve: process "/bin/sh -c pnpm install --frozen-lockfile" did not complete successfully: exit code: 137

优化建议?如有

added labels
体验优化
Developer

哪个仓库
发一下复现的方式,

Developer

可以用这两个命令测试下.
nslookup registry.npmjs.org
curl -v https://registry.npmjs.org/refractor/-/refractor-3.6.0.tgz -O

如果是198的ip就是正常的, 如果是其他ip,检查下是不是自己改过host用的不是默认的dns
/tmp/iOALog/.new_Clipboard/Clipboard_Screenshot_1749799809.pnga6c92fa8-c813-4926-aeb9-bd59a8003522.png

Developer

我在远程开发和 ci 里跑,看着下载是正常加速的
https://cnb.cool/tian-yu/tmp/test-new-model/-/build/logs/cnb-alo-1itk2ni4q

/tmp/iOALog/.new_Clipboard/Clipboard_Screenshot_1749799986.pngd5efff39-5ed5-4b38-9410-fbfae90ebc14.png

Creator

哪个仓库
发一下复现的方式,

https://cnb.cool/maplez/example

Creator

可以用这两个命令测试下.
nslookup registry.npmjs.org
curl -v https://registry.npmjs.org/refractor/-/refractor-3.6.0.tgz -O

如果是198的ip就是正常的, 如果是其他ip,检查下是不是自己改过host用的不是默认的dns
/tmp/iOALog/.new_Clipboard/Clipboard_Screenshot_1749799809.pnga6c92fa8-c813-4926-aeb9-bd59a8003522.png

我在云开发环境和ci执行这两个命令也没问题,但实际构建还是一堆 ERR_SOCKET_TIMEOUT

Developer

能复现, 还得再看看. 看是 npmjs.org 本身慢导致的还是其他情况

Developer

做了若干测试

  1. 跑 ci 构建,较慢, 且出现较多连接超时, 是目前待优化的点 https://cnb.cool/tian-yu/ai/fork/test-npm-build/-/build/logs/cnb-4ha-1itrob8sh
  2. 相同时间点, 跟上面在同一台机器的宿主机上直接 docker build 正常,速度比较快 https://cnb.cool/tian-yu/ai/fork/test-npm-build/-/blob/new-branch/build.log , 另外查了下当时时间点机器负载不高,就2~3条ci流水线在跑
  3. 在远程开发里的prepare阶段, 运行docker build指令, 正常. https://cnb.cool/tian-yu/ai/fork/test-npm-build/-/build/logs/cnb-ma8-1itrlkl7q
  4. 看arm64节点和ci节点使用的是相同的加速通道, 且加速已生效,未见其他域名请求或302
  5. 相同代码, pnpm install换成 npm install --legacy-peer-deps (忽略依赖冲突), ci 构建正常.
  6. 在 ci 构建中直接执行 shell命令, 有些慢, 但未超时, https://cnb.cool/tian-yu/ai/fork/test-npm-build/-/build/logs/cnb-vbo-1itrr3flc
    见pipeline2
added priority
P0
Developer

临时解法:先给一个临时解法,加一个 PNPM_WORKERS 的变量,限制一下 worker 数量。

FROM node:20.18-alpine AS builder
WORKDIR /app

RUN npm install -g pnpm

COPY package.json pnpm-lock.yaml ./

# 填写您需要的核数
ENV MY_CPU=2

# 使用 PNPM_WORKERS 限制 pnpm worker 数量
RUN PNPM_WORKERS=$(( $(/usr/bin/nproc) - $MY_CPU )) pnpm install --frozen-lockfile

RUN npx update-browserslist-db@latest

原因

pnpm 使用了一个 nodejs 的新版 API(os.availableParallelism) , 获取可用 CPU 数量,但是这个 nodejs 的 API 在容器环境有 bug , 获取到了母机的 CPU 数量,导致启动了 300 多个 worker,直接干 OOM 了,pnpm 和 nodejs 仓库的相关 Issue 讨论:

pnpm 源码是这样配置 CPU 核数的,配置的是需要“空出来”多少核,也就是总核数减去你需要用多少,故产生上述的临时方案。
43a6b083-db33-425d-bbb1-eb237dda324b.png

有更佳的方案继续补充。

Developer

对, 排查下来是 pnpm 在大核心宿主机上, 对虚拟化的兼容和性能问题. 看代码, 在docker中申请使用 cpu=2 核时, 仍按照宿主机总核数384去创建WORKER数. 造成的性能下降.

指定 PNPM_WORKERS 能规避. 其中 PNPM_WORKERS 计算方式是: PNPM_WORKERS = 宿主机总核数-申请CPU数, 即 PNPM_WORKERS=384-2=382。 (也可以参照上条评论自动计算)

代码里加下就好了:
RUN pnpm install --frozen-lockfile
改成
RUN PNPM_WORKERS=382 pnpm install --frozen-lockfile

版本是 node20.18 和 pnpm v10.12.1

构建日志 https://cnb.cool/tian-yu/ai/fork/test-npm-build/-/build/logs/cnb-pgg-1iu315bik

Resolved ISSUE
Developer

可以试试这个设置, 完整的优化需要等 libuv升级-》node 升级 -〉支持 os.availableParallelism正确返回docker中限制的cpu值 或者pnpm优化.

目前解决方式是上面的 PNPM_WORKERS=xxx 的设值方案. issue先关闭了

ci: 修复依赖安装慢的问题
4d9b5b58-fb89-48f5-91e1-07843700f28c.png

也可以升级pnpm 到10.6.4以上版本

Assignee
None yet
Label
体验优化
Priority
P0
Time period
-
Property
Add custom properties to record and label key information
Participant