High Memory Usage During Content Sync with Astro 5.1.7 and Starlight 0.31

[justin5267]

What version of starlight are you using?

0.31.1

What version of astro are you using?

5.1.7

What package manager are you using?

npm

What operating system are you using?

windows

What browser are you using?

edge

Describe the Bug

Summary

After upgrading to Astro 5.1.7 and Starlight 0.31, the development server fails to start due to memory issues during the "content sync" phase.

Details

Setup:

• 800 Markdown (.md) files organized in a deeply nested folder structure:
  src/content/docs/
  ├── 1.文件夹/
  │ ├── A.文件夹/
  │ │ ├── A1.文件夹/
  │ │ │ ├── file1.md
  │ │ │ ├── file2.md
  │ │ │ └── ...
  │ │ └── fileX.md
  │ └── B.文件夹/
  │ │ ├── B1.文件夹/
  │ │ │ ├── file1.md

• Each Markdown (.md) file has an average size of 400KB.

• Files include Chinese filenames and use slug in their frontmatter for URL generation.

• Memory allocation set to 16GB for Node.js.

Observed Behavior:

• During the "content sync" phase, Node.js memory usage gradually increases until it exceeds 16GB, at which point the process crashes with a memory error.

• Reducing the number of Markdown files to around 200 allows the development server to start, but it takes 161258ms.

Expected Behavior:

Under Astro 4.16.7 and Starlight 0.29.3, the same setup (800 Markdown files) allows the development server to start in 619ms without memory issues.

Link to Minimal Reproducible Example

n/a

Participation

• I am willing to submit a pull request for this issue.

[delucis]

Thank you for the issue @justin5267!

Would you be able to share a reproduction at can look at? Although your description is quite detailed (thank you), it would take quite a bit of time for us to try to build a new project matching that description and there's no guarantee it would be exactly the same. As an example, Astro's Docs has thousands of content files and does not have the same issue, so I guess it's not just volume of content in this case

If you could share a reproduction, then we can take a look.