How we passed our Core Web Vitals — LCP — Skip Server Side rendering for bots

TL:DR — We stopped doing Server Side rendering (SSR) for bots as it was causing timeouts on our servers.

The presenting complaint

We started seeing lots of timeouts and an increase in our server response time anytime we did a “botify” crawl. Botify help us with our SEO and they send a bot to hit all our pages for reasons (I’m not close to this area :))

SSR was the culprit

We’d been doing these botify crawls for months and it had only started happening recently.

The culprit turned out to be the introduction of Server Side rendering of the Listing Carousel on PDP.

The solution — only SSR the carousel for humans!

We made a very small change by adding an if-else to our carousel code.

BEFORE
<%= react_component "ListingCarousel", prerender: true %>
AFTER
<% if browser.bot? %>
  <%= react_component "ListingCarousel" %>
<% else %>
  <%= react_component "ListingCarousel", prerender: true %>
<% end %>

Impact
The timeouts and slower response times stopped. Bots make up about 50–60% of our traffic so this was a lot of work we saved our server from having to do.