The high volume of XHR requests is expected for an application like ours.
We render the most critical part of each page server-side — a skeleton — then hydrate it with content once JavaScript executes. This gives us both good performance and rich interactivity. As bots move between routes, they need those client-side JSON documents to build a complete picture of each page: the same view you see in your browser.
Google processes JavaScript web apps in three phases:
- Crawling
- Rendering
- Indexing
Below is a crawl stats snapshot from one of our white-labels for reference. Your domain may show fewer requests, but the proportions should look similar.
References: https://support.google.com/websearch/answer/10622781https://support.google.com/websearch/answer/9351707