SSR a "Create React App" the hard way

Posted at October 20, 2018

I’ve been building out a web app using Create React App, developing and deploying is a breeze. But now I need a server side rendering solution for my specific sitation. All the solutions around the web I have found are great, but they don’t address the issue I have. So in this post I’ll share my own “from scratch” solution. The problem I am solving is:

Google has trouble with my single page app written in javascript, I want to create a static HTML render of all pages inside my app. I want to serve these static pages to googlebot (and other crawlers). Since my app displays a lot of dynamic information (pulled in from an API, updated hourly) I want to be able to easily rerender the complete UI with minimal effort (and compilation steps).

So this is NOT a SSR solution that prerenders an initial page for a user, instead it statically “compiles” a large number of pages in my app whenever the API data changes into plain HTML files which are served through a webserver (to crawlers, not to users). I’ve build my solution for the create-react-app version I am using, which is react-scripts 2.0.3 and uses react 16.4.1. If you are using another version everything below might not work.

Note that my approach is NOT recommended, it requires you to put monkey patch components in your app and forces you to break promises (out of spec compliance). As soon as React updates their lifecycle hooks (again) you will need to come and fix this or you might break everything below. Here by dragons.

My codebase I want to SSR

I had to tackle some issues as they came up with the libraries I used in my codebase.

  • I’m using React Router as a router.
  • Right now I’m not using any advanced state management solution (like Redux or Mobx), instead I’m using a simple module to fetch & cache REST API endpoints to components that need them (I will move to Redux when I implement realtime websocket communication).
  • There are some components relying on browser specific properties (like window size for responsive components, a lot of charts attach mouse event handles).
  • Everything else is super plain (like the twitter bootstrap library reactstrap).
  • I serve all static files through nginx. I want to serve my normal react assets (bundle) as well as the SSR generated html files through nginx.

SSR a “create-react-app” the hard way

The react team has created an awesome library to render react code on the server (in node), it’s called ReactDOMServer and recent versions can actually render react code faster than the normal react dom library. Mainly because it doesn’t come with a virtual dom (since it only renders things once, doesn’t need to diff later) and it ignores all your lifecycle hooks. Despite how awesome this library is, we need to overcome a few hurdles if we want to use this app, we need to:

  • Compile our react codebase so node can run all the components natively (using babel)
  • Figure out how to easily render a single page managed by react-router (without running in a browser and having an actual browser history)
  • Load API data our frontend usually pulls from an external API so we can render our app via SSR with this data preloaded.
  • Monkey patching react lifecycle hooks since they don’t work in SSR:
    • If your components rely on componentDidMount you’ll need to figure something out since ReactDOMServer won’t ever “mount” your components.
    • Since your components are not mounted, you cannot use methods like this.setState since they require components to be mounted.
  • Make our components render completely in the initial tick (render call), since even resolving a promise happens in the next tick which is too late for ReactDOMServer.
  • Finally: Serve the statically generated HTML to crawlers while we serve the dynamic react app to all normal users, but it’s important to serve these pages over the same URL your visitors can hit.

Compile our react codebase

In create-react-app you write different javascript than you normally would in nodejs. For example you import and export all your react modules using ES6 modules (import and export) while in node you use different modules (with require and module.exports). While there are ways to run ES6 modules in nodejs I recommend transpiling the codebase down to something nodejs can work with. If you try to run an ES6 module in nodejs you’ll see this error:

/Users/pear/projects/gekko-react-ui/src/full.js:1
(function (exports, require, module, __filename, __dirname) { import React from 'react';
                                                                     ^^^^^

SyntaxError: Unexpected identifier

So first we transpile the complete codebase with this command:

NODE_ENV=production babel src/ --presets=@babel/preset-env,@babel/react --plugins=@babel/plugin-proposal-class-properties,ignore-html-and-css-imports --out-dir nodesrc

That will tell babel to transpile everything inside your src directory and put the results in a new directory nodesrc.

SSR a single page when using react-router

React Router actually has a component you can use instead of <BrowserRouter> called <StaticRouter>. In order to allow for both SSR and normal rendering without duplicating everything I changed my App.js that looked like this:

const App = () => {
  return <BrowserRouter>
    <Explainer />
    <Header />
    <ScrollToTop>
      <Switch>
        <Route exact path="/" component={Dashboard} />
        <Route path="/bot/:id" component={BotResultPage} />
        <Route path='/strategies/:id' component={StrategyDetails} />
        <Route exact path='/strategies' component={StrategyList} />
        <Route exact path='/create-bot' component={CreateBot} />
        <Route path='/news/:id' component={NewsPage} />
        <Route exact path='/news' component={NewsList} />
        <Route component={NotFound} />
      </Switch>
    </ScrollToTop>
  </BrowserRouter>
}

Into this:

const App = () => {
  return <>
    <Explainer />
    <Header />
    <ScrollToTop>
      <Switch>
        <Route exact path="/" component={Dashboard} />
        <Route path="/bot/:id" component={BotResultPage} />
        <Route path='/strategies/:id' component={StrategyDetails} />
        <Route exact path='/strategies' component={StrategyList} />
        <Route exact path='/create-bot' component={CreateBot} />
        <Route path='/news/:id' component={NewsPage} />
        <Route exact path='/news' component={NewsList} />
        <Route component={NotFound} />
      </Switch>
    </ScrollToTop>
  </>
}

And I create two entry files, one for web and one for SSR. The one for web looks like:

// web.js
import React from 'react';
import { BrowserRouter } from "react-router-dom";
import { App } from './App';

const Page = () => {
  return <BrowserRouter>
    <App />
  </BrowserRouter>
}

export default Page;

And I can now create another simple entry file to render for SSR:

  // server.js
  import React from 'react';
  import { StaticRouter } from "react-router-dom";
  import { App } from './App';

  const Wrapper = ({ url, context = {}}) => {
    return <StaticRouter
      location={url}
      context={context}
    >
      <App />
    </StaticRouter>
  }

I can create a <Wrapper> component, pass it a url I want to render and react-router will only render components matching that exact route!

Load API data

When our frontend loads it will display a loading spinner while it goes and fetches required API data from the backend. Once the data comes back (and the fetch promise resolves) we pass updated data to our components and render the full pages. Unfortunately this approach doesn’t work with ReactDOMServer since it will render the spinner and ignore the fetch completely. Instead we need to fetch the required API data in the node process we will use to SSR all our pages (before we try render the pages).

This was the API module that fetches external data in our frontend:

// api.js
const cache = {};

const get = url => {

  if(cache[url]) {
    console.log(`cache hit for "${url}"`);
    return new Promise(resolve => resolve(cache[url]));
  }


  console.log(`cache miss for "${url}", fetching..`);
  return fetch(url)
    .then(resp => resp.json())
    .then(resp => {
      cache[url] = resp;
      return resp;
    });
}

export { get };

To make sure we always hit the cache we can simply populate the cache object before we render our components that call the get method:

const populateCache = (key, value) => {
  cache[key] = value;
}

export { get, populateCache };

Now our node script that will SSR can load all API endpoints before rendering (from disk, from another backend, etc) and populate the cache:

import { populateCache } from './api';

const apiFiles = [
  'backtest-MACD-basic-profit.json',
  'gekkos.json',
  'news.json',
  'strategies.json'
]

apiFiles.forEach(f => {
  const content = fs.readFileSync('../public/' + f, 'utf8');
  populateCache('/' + f, JSON.parse(content));
});

Monkey patching react lifecycle hooks

A lot of my components rely on componentDidMount, I am using this hook to attach DOM events for interactive charts and such. componentDidMount does not get invoked by ReactDOMServer! This doesn’t matter for the browser events we want to attach (since we dont care about interactive charts when serving our web page to a crawler). However my components that fetch API endpoints also start fetching in componentDidMount. While we now have made sure that the API data is available directly by prepoluating the cache, the following HOC won’t work when SSRing:

// apifetcher.js
import React, { Component } from 'react';

import { get } from '../../api';

const ApiFetcher = (Wrapped, url) => class extends Component {
  state = {
    isLoading: true,
    error: false,
    data: false
  }

  componentDidMount() {
    this.setState({isLoading: true, error: false});
    get(url)
      .then(data => {
        this.setState({data, isLoading: false, error: false})
      })
      .catch(error => this.setState({error}))
  }

  render() {
    return <Wrapped {...this.state} {...this.props} />
  }
}


export default ApiFetcher;

That’s an helper HOC I use to give dynamic API data to components. When ran in a browser the following would happen:

  1. Component would instantiate
  2. Initial state would be set to isLoading true and data is false.
  3. componentDidMount is fired by React.
  4. get would be invoked to load API data from the url.
    • if it’s in the cache go to step 6.
  5. Load the data using fetch (inside api.js).
  6. Resolve the promise with the data and pass it back to the component.
  7. set the component’s state with this data.

But when we render our components using ReactDOMServer the function in step 3 never gets called! The solution I found (NOTE: this will break whenever React updates lifecycle methods again) is to create a simple component that manually invokes componentDidMount of a child component on render (since SSR only calls render once anyway). The other problem we have is that step 7 is NOT synchronous, so using SSR you cannot use this.setState at all. My component solves both of these problems:

// SSRInitialStateComponent.js
import { Component } from 'react';

import { isServer } from '../../globals';
import _ from 'lodash';

class SSRInitialStateComponent extends Component {
  constructor() {
    super();

    // PLEASE make sure you only actually run this in SSR mode
    if(isServer) {
      // monkey patch this setState so that:
      // - it runs sync
      // - it directly mutates `this.state`
      // - runs outside the react lifecycle which
      //   does not exist in SSR.
      this.setState = updates => {
        this.state = _.assign(this.state, updates);
      }
    }
  }

  // components who need to set non initial state
  // should call this in render (in SSR render is only
  // ever invoked once).
  forceState() {
    if(isServer && this.componentDidMount) {
      this.componentDidMount();
    }
  }
}

export default SSRInitialStateComponent;

We now only have to do two things to our components that need to use componentDidMount: extend SSRInitialStateComponent instead of Component:

const ApiFetcher = (Wrapped, url) => class extends SSRInitialStateComponent

In the render method call this.forceState (this is a noop if the app is currently being rendered by browser React):

render() {
  this.forceState();

  return <Wrapped {...this.state} {...this.props} />
}

Make our codebase fully synchronous

If we go back to the steps above there is one final problem: even though our components rely on API data that’s in cache (since we manually populated it), it’s still resolves using a promise. And promises never resolves synchronous (but in the next tick, even if you fire resolve straight away). While this is for good reason, ReactDOMServer can’t work with promises because of it (since it requires everything to be fully sync). The solution to this is break all promises you use (while rendering SSR) using a library like synchronous-promise. I manually hacked my way with a promise mock since I only use them in one place currently:

// api.js
const get = url => {

  if(isServer) {
    const resp = cache[url];

    const ret = {
      then: fn => {

        let next;

        if(resp) {
          next = fn(resp);
        }

        // allow .then() chaining
        if(next && next.then) {
          return next;
        }

        return ret;
      }
    };

    return ret;
  }

  if(cache[url]) {
    console.log(`cache hit for "${url}"`);
    return new Promise(resolve => resolve(cache[url]));
  }

  console.log(`cache miss for "${url}", fetching..`);
  return fetch(url)
    .then(resp => resp.json())
    .then(resp => {
      cache[url] = resp;
      return resp;
    });
}

I did this instead of the Promise mock since I was too lazy to figure out how to make sure that library was not used in the app as rendered by web React, and on top of that how to make sure the lib wasn’t included in the clientside bundle.

The SSR render script

Tying it all together I use a few more tricks to make sure I can render the full app, here is the main render script:

import React from 'react';
import ReactDOMServer from 'react-dom/server';
import fs from 'fs';
import { StaticRouter } from "react-router-dom";

import { App, routes } from './App';
import { populateCache } from './api';
import { getTitle } from './globals';

import walkSync from 'walk-sync';

import _ from 'lodash';

const writeTo = '../build/static-render';

const Wrapper = ({ url, context = {}}) => {
  return <StaticRouter
    location={url}
    context={context}
  >
    <App />
  </StaticRouter>
}

const apiFiles = [
  'backtest-MACD-basic-profit.json',
  'gekkos.json',
  'news.json',
  'strategies.json'
]

console.log(new Date, 'start');

apiFiles.forEach(f => {
  const content = fs.readFileSync('../public/' + f, 'utf8');
  populateCache('/' + f, JSON.parse(content));
});

console.log(new Date, 'loaded api data');

let index = fs.readFileSync('../build/index.html', 'utf8')

// strip the bundle + webpack inline js
const indexParts = index.split('<script');
index = indexParts[0] + '<script' + indexParts[1] + '</body></html>';

const dirs = [];

routes.forEach(r => {
  const parts = r.split('/');
  parts.shift();
  if(parts.length > 1 && parts[0] && !dirs.includes(parts[0])) {
    dirs.push(parts[0]);
  }
})

if(!fs.existsSync(writeTo)) {
  fs.mkdirSync(writeTo);
  console.log(new Date, 'written dir', writeTo);
}

_.each(dirs, d => {
  const path = writeTo + '/' + d;
  if(!fs.existsSync(path)) {
    fs.mkdirSync(path);
    console.log(new Date, 'written dir', path);
  }
});

// this renders each route as a static page
routes.forEach(r => {
  const reactPage = ReactDOMServer.renderToStaticMarkup(<Wrapper url={r} />);
  const fullPage = index
    .replace('<div id="root"></div>', '<div id="root">' + reactPage + '</div>')
    .replace('<title>Gekko Plus</title>', '<title>' + getTitle() + '</title>')

  console.log(getTitle());

  if(r === '/') {
    r = '/index';
  }

  fs.writeFileSync(writeTo + r + '.html', fullPage, 'utf8');
  console.log(new Date, 'written', writeTo + r + '.html');
});

Serving the static files through nginx

Great! Everything worked and we can run one simple file to statically compile our complete react codebase into flat HTML files! The last thing we need to do is serve these files to Google and make sure we serve our default app. For that I used this nginx config file as a basis which upstreams requests coming from crawlers.

If nginx detects a crawler (based on the user_agent header), the request upstreamed to a tiny node server that serves the files but does some correction (serves the news.html when Google requests /news, etc.). Here are the initial logs of some crawlers fetching the statically rendered version of my create-react-app:

2018-10-20T04:56:55.215Z '66.249.64.218' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/news/gekko-plus-strategy-competition'
2018-10-20T05:25:57.586Z '66.249.64.216' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/'
2018-10-20T05:25:57.732Z '66.249.64.216' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/'
2018-10-20T11:09:04.065Z '66.249.64.216' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/survey'
2018-10-20T15:41:40.699Z '66.249.64.218' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/lab'
2018-10-20T18:52:27.408Z '66.249.64.216' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/'
2018-10-20T18:52:28.132Z '66.249.64.218' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/'
2018-10-20T18:55:49.705Z '66.249.64.216' 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/'
2018-10-20T19:23:12.863Z '31.13.115.8' 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' '/'
2018-10-20T19:23:13.157Z '31.13.127.5' 'facebookexternalhit/1.1' '/'
2018-10-20T19:23:21.691Z '66.249.64.220' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/news'
2018-10-20T19:28:20.099Z '31.13.115.8' 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' '/'
2018-10-20T19:57:29.881Z '31.13.127.4' 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' '/'
2018-10-20T19:57:30.046Z '31.13.127.2' 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' '/'
2018-10-20T20:05:25.932Z '149.154.167.167' 'TelegramBot (like TwitterBot)' '/news/gekko-plus-upcoming-strategy-competition'
2018-10-20T20:14:44.827Z '66.249.64.216' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/create-bot'
2018-10-20T20:18:26.497Z '66.249.64.216' 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' '/bot/backtest-MACD-basic-profit'
2018-10-20T20:44:14.888Z '149.154.167.164' 'TelegramBot (like TwitterBot)' '/strategies/macd'
2018-10-20T21:17:21.114Z '149.154.167.170' 'TelegramBot (like TwitterBot)' '/'
2018-10-20T21:58:33.704Z '54.234.37.47' 'Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)' '/'
2018-10-21T06:52:11.132Z '199.59.150.183' 'Twitterbot/1.0' '/news/gekko-plus-upcoming-strategy-competition'
2018-10-21T06:52:11.136Z '199.59.150.182' 'Twitterbot/1.0' '/news/gekko-plus-upcoming-strategy-competition'
2018-10-21T07:16:56.239Z '217.19.25.198' 'Twitterbot/1.0' '/news/gekko-plus-upcoming-strategy-competition'

That’s it! The current app can be found here: app.gekkoplus.com.

If you have questions or suggestions feel free to reach out on twitter: @mikevanrossum.

« Full blog

contact me