Hi, DEV Community! I've been working in the software testing field for more than eight years. Apart from web services testing, I maintain CI/CD Pip...
For further actions, you may consider blocking this person and/or reporting abuse
Instead of caching node_modules, consider caching node's caching directory instead.
The difference is caching downloaded tar.gz files instead of thousands of small files. Despite gitlab's efforts, their caching mechanism sucks big time for for a large amount of small files.
Sorry, what do you mean under "node's caching directory"? Which one is that?
And what tar.gz files do you mean?
This is pure CI gold. Thank you!
Why do you create a hidden job while you only extend it in 1 job? This could all be included in the setup job right? And right now the cache.policy is always overwritten to pull-push. Or am I missing something?
Should we add
dependenciesin other jobs too?node_modules can be huge in real world, and then unsuitable for artifacts which are limited in size. Worth knowing, it is also uploaded to central Gitlab, which can be a bottleneck for a large Gitlab instance with lots of runners uploading to it.
Other than that thank you, I learned that npm ci is slow due to node_modules deletion 🙏
If you compare the time on the clean system, I bet
npm ciwould be faster thannpm install. Cause it just downloads full tree of dependencies from package-lock.json.npm installwill check which deps can be updated and build new dependency tree.Yes, this! My project's node_modules is 2GB and is too big for artifacts. What is the recommended solution to deal with that? I've had to include
npm cion every step to get my pipeline to work at all.You should use cache. This is why cache exists, and can be shared even across pipelines.
But cache has to be configured on your runners, or you will experience missing cache each time your jobs switch runners (which should not be a problem, npm will handle it)
Thank you so much for your effort, but I still didn't get why we need to add artifacts. You described the problem artifacts solves like so:
Isn't this why we use cache at the first place? to not install packages again? We already had added the cache at this point, so why do we need to add artifacts, too?
We need an article about GitHub Actions!
Thanks! Just a doubt, don't you need to specify the cache location to the npm ci command? Something like npm ci --cache ${npm_config_cache} --prefer-offline ?
The variables section has
npm_config_cachewhich will be used by npm automatically.Please let us know what's the (5)
It looks like that is just for descriibing the "lines of the code" that are being talked about instead of using actual "line numbers" (since they aren't visible)