3 Deployment Pipeline Regressions Fixed with `printf` and `distDir` Isolation!

I had a pretty rough time dealing with several regressions in my deployment pipeline. It was chaos, especially with unexpected errors popping up one after another during the build process.

Attempts and Pitfalls

Initially, I found a YAML parsing error caused by nested heredocs. I thought a simple change to use printf would fix it.

# Old code (problematic)
cat <<EOF
key:
  nested_key: |
    $(some_command)
EOF

# Attempted code (using printf)
printf "key:\n  nested_key: |\n    %s\n" "$(some_command)"

But that wasn't the end of it. There was another regression causing 500 errors due to the absence of the .next directory during the build, and yet another issue arose from RSC manifest paths.

Causes

Nested heredocs could be interpreted in unexpected ways by the shell during parsing, leading to YAML syntax errors. The missing .next directory was either not being properly generated during the build process or previous build artifacts were not retained. The RSC manifest path problem was related to how Next.js manages build artifacts; issues arose with path references when frontend and server builds were separated.

Solutions

First, I cleanly handled the nested heredoc by using printf.

printf "key:\n  nested_key: |\n    %s\n" "$(some_command)"

I resolved the .next directory issue by isolating the distDir setting. This ensures that each build creates an independent .next directory, preventing interference from previous builds.

// next.config.js
module.exports = {
  experimental: {
    outputFileTracing: true,
  },
  distDir: '.next-build', // Use a unique directory for each build
};

Finally, I addressed the RSC manifest path problem by switching to perform frontend builds directly on the server. This unifies the build environment and reduces path-related issues.

Results

YAML parsing errors in the deployment pipeline have been completely eliminated.
The frequency of 500 errors during builds has significantly decreased.
Stability has improved with the resolution of RSC manifest path regressions.

Takeaways — To Avoid the Same Pitfalls

[ ] When using heredocs in deployment pipelines, be mindful of shell interpretation and consider replacing them with printf.
[ ] Verify that intermediate artifacts like the .next directory are properly managed during builds and explore methods like distDir isolation.
[ ] Consider unifying the build environment to resolve dependency and path issues between frontend and server builds.
[ ] Always keep in mind the potential for regressions at each stage and invest time in root cause analysis when problems arise.