Introduction

I have by now worked on various XM Cloud projects and some of them are now continuously growing and as such the code base and the amount of teams working on the code growing. because we needed a strategy of managing and structuring code sufficiently I dove into leveraging monorepo' s in XM Cloud, more specifically leverage Turborepo in our XM Cloud frontend repo. In this blogpost i will outline what you need to do to incorporate Turborepo based upon the sxastarter example from Sitecore.

For anyone who just wants to skip ahead to the working solution. it can be found here : https://github.com/dhudepol/xmcloud-turborepo

Monorepo & Turborepo

Before starting with the explanation on how to get to a monorepo setup with XM Cloud, lets dive into what a monorepo is and why you would use it.

A monorepo is a single repository containing multiple distinct projects, with well-defined relationships. This means that it a pattern for setting up your code repository so that you have a clear setup for apps leveraging packages without having to setup multiple repo' s to arrange this.

As to why you would want this ? :

No overhead to create new projects: Use the existing CI setup, and no need to publish versioned packages if all consumers are in the same repo.
One version of everything: No need to worry about incompatibilities because of projects depending on conflicting versions of third party libraries.
Atomic commits across projects: Everything works together at every commit. There's no such thing as a breaking change when you fix everything in the same commit.
Developer mobility: Get a consistent way of building and testing applications written using different tools and technologies. Developers can confidently contribute to other teams’ applications and verify that their changes are safe.
Structure for larger applications: When the amount of code increases and the amount of teams increase, a monorepo provides the structure to do this in a maintainable way

So now that we now what a monorepo is, what is turborepo then? Turborepo is a tool that allows running monorepo applications whilst enabling benefits like build caching, concurrent build agents and performance improvements. Turborepo is also owned by vercel which drove me to chose turborepo rather then other tools.

Steps to bringing Turborepo & monorepo to XM Cloud.

To start out with setting up a Turborepo with XM Cloud we need to start by grabbing a copy of the sxastarter application of Sitecore. This can be found here : https://github.com/sitecorelabs/xmcloud-foundation-head-dev. to create a local copy you need to run git clone:

1git clone https://github.com/sitecorelabs/xmcloud-foundation-head-staging.git sxa-starter

next we need to follow the standard steps in creating a turborepo project, for now create it outside of the sxa-starter project. Start by running the command below:

1npx create-turbo@latest

After running the command a number of questions will be asked. the one key question is which package manager we want to use. Choose NPM here.

When the wizard is done your repository should look like below:

1 - apps/web: Next.js with TypeScript
2 - apps/docs: Next.js with TypeScript
3 - packages/ui: Shared React component library
4 - packages/eslint-config-custom: Shared configuration (ESLint)
5 - packages/tsconfig: Shared TypeScript `tsconfig.json`

For more information on how this structure works you can take a look at the official Turborepo documentation: https://turbo.build/repo/docs/getting-started/create-new

Now that we have the two basic solutions : 1 turborepo and 1 sxstarter we can start combining the two. The first step to do this is to move the turborepo into the sxastarter. This means we need to move it to the directory "src" inside the sxastarter. after this the sxastarter should like like:

Now, we need to move the "src/sxastarter" folder into "src/turborepo/apps" .After moving the folder we need to rename "src/turborepo" to "src/sxastarter" to make it compatible with the starterkit from Sitecore. After these actions the repository should look like below:

before we close of we need to run the npm install command from the root "src/sxastarter":

1npm install

Now we have got the repository setup done and can start doing the configuration adjustments to make everything play together.

Configuration adjustments

To make everything work together we need to adjust two files "src/sxastarter/package.json" and "src/sxastarter/turbo.json". Let us start with the package.json. here we need to add atleast 2 scripts "start:connected" and "start:production". The resulting package.json should be :

1{
2  "private": true,
3  "scripts": {
4    "build": "turbo run build",
5    "dev": "turbo run dev",
6    "lint": "turbo run lint",
7    "start:connected": "turbo run start:connected",
8    "start:production": "turbo run start:production",
9    "next:start": "turbo run next:start",
10    "format": "prettier --write \"**/*.{ts,tsx,md}\""
11  },
12  "devDependencies": {
13    "eslint": "^8.48.0",
14    "prettier": "^3.0.3",
15    "tsconfig": "*",
16    "turbo": "latest"
17  },
18  "name": "example-turbo",
19  "packageManager": "npm@10.1.0",
20  "workspaces": [
21    "apps/*",
22    "packages/*"
23  ]
24}
25

Now lets adjust turbo.json. we will need to add configuration for the 2 new scripts. The file should look like:

1{
2  "$schema": "https://turbo.build/schema.json",
3  "globalDependencies": ["**/.env.*local"],
4  "pipeline": {
5    "build": {
6      "dependsOn": ["^build"],
7      "outputs": [".next/**", "!.next/cache/**"]
8    },
9    "lint": {},
10    "start:connected":{
11      "cache": false,
12      "persistent": true
13    },
14    "start:production":{
15      "cache": false,
16      "persistent": true
17    },
18    "next:start":{
19      "cache": false,
20      "persistent": true
21    },
22    "dev": {
23      "cache": false,
24      "persistent": true
25    }
26  }
27}
28

the configuration for cache is set to false as this is running a server which cannot work with the turborepo cache and persistant is set to true as the command is expected to run continuously.

Fixing a issue with ssas and fontawesome

The sxastarter template from Sitecore uses a alias for fontawesome which is reliant on path. As the fotnawesome npm package is being installed in a different location in a monorepo we need to adjust this. To do this open the file "src/sxastarter/apps/sxastarter/src/lib/next-config/plugins/ssas.js". Here we need to add "../../" to the fontawesome path, resulting in:

1const path = require('path');
2const SassAlias = require('sass-alias');
3
4/**
5 * @param {import('next').NextConfig} nextConfig
6 */
7 const sassPlugin = (nextConfig = {}) => {
8  return Object.assign({}, nextConfig, {
9      sassOptions: {
10        importer: new SassAlias({
11          '@sass': path.join(__dirname, '../../../assets', 'sass'),
12          '@fontawesome': path.join(__dirname, '../../../../../../node_modules', 'font-awesome'),
13        }).getImporter(),
14      },
15    });
16};
17
18module.exports = sassPlugin;
19

Configuring the app for the first run

What is left now is to setup the correct settings to run the application. the settings sit in "src/sxastarter/apps/sxastarter/.env". If you want more information on how to set it up please read my other blog post: Setting up your local dev machine

Starting it up for the first time

Now we can run it for the first time. Start up the docker environment first and then go to the root of the turborepo "src/sxastarter" and run :

1npm run start:connected

if everything went well you should now see turbo starting the sxastarter app :

1> start:connected
2> turbo run start:connected
3
4• Packages in scope: content-components, eslint-config-custom, sxastarter, tsconfig, ui
5• Running start:connected in 5 packages
6• Remote caching disabled
7sxastarter:start:connected: cache bypass, force executing 32039b5669889c6f
8sxastarter:start:connected: 
9sxastarter:start:connected: > sxastarter@21.5.0 start:connected
10sxastarter:start:connected: > npm-run-all --serial bootstrap --parallel next:dev start:watch-components
11sxastarter:start:connected:
12sxastarter:start:connected: 
13sxastarter:start:connected: > sxastarter@21.5.0 bootstrap
14sxastarter:start:connected: > ts-node --project tsconfig.scripts.json scripts/bootstrap.ts
15....
16sxastarter:start:connected: - event compiled client and server successfully in 348 ms (18 modules)

And when opening localhost:3000 you should see the sxastarter site:

Further enhancements to the XM Cloud turborepo

Allthough we have now setup XM Cloud to work with turborepo it is not really setup to use anything that turborepo brings. Let' s move the Promo components to a package to illustrate how we would be able to use it.

To do this we can start by copying the "ui" package located at "src/sxastarter/packages/ui" and renaming it to "content-components". Then we adjust the package.json located at "src/sxastarter/packages/content-components/package.json" to reflect the right dependencies and name:

1{
2  "name": "content-components",
3  "version": "0.0.0",
4  "main": "./index.tsx",
5  "types": "./index.tsx",
6  "license": "MIT",
7  "scripts": {
8    "lint": "eslint .",
9    "generate:component": "turbo gen react-component"
10  },
11  "dependencies": {
12    "@sitecore-jss/sitecore-jss-nextjs": "~21.5.0"
13  },
14  "devDependencies": {
15    "@turbo/gen": "^1.10.12",
16    "@types/node": "^20.5.2",
17    "@types/react": "^18.2.0",
18    "@types/react-dom": "^18.2.0",
19    "eslint-config-custom": "*",
20    "react": "^18.2.0",
21    "tsconfig": "*",
22    "typescript": "^4.5.2"
23  }
24}
25

we have changed:

Name to be content-components
Added "@sitecore-jss/sitecore-jss-nextjs": "~21.5.0" as a dependency

Now that we have a basic package we can go ahead and take the following actions:

Delete the "Card.tsx" in the "content-components" package
Move the "Promo.tsx" file from our "sxastarter" app to the "content-components" package, this file can be found at "src/sxastarter/apps/sxastarter/src/components/Promo.tsx".
Replace the ".eslintrc" file in the "content-components" package with the ".eslintrc" file in "src/sxastarter/apps/sxastarter"

Now that all files are in place we can adjust the "index.tsx" file in the "content components" package to export the Promo component:

1// component exports
2export * as Promo from './Promo';
3

Why do we use "export * as Promo" ? this is because of how JSS works with sxa variants. we need the export to be an object containing multiple actual exports.

Changing the sxastarter app to expose the Promo component to Sitecore

Now we are ready to adjust the sxastarter app so that we can actually use the Promo component.

We will start with adjusting the package.json of sxastarter in "src/sxastarter/apps/sxastarter":

1{
2  "name": "sxastarter",
3  "description": "Application utilizing Sitecore JavaScript Services and Next.js",
4  "version": "21.5.0",
5  "private": true,
6  "config": {
7    "appName": "sxastarter",
8    "rootPlaceholders": [
9      "jss-main"
10    ],
11    "sitecoreConfigPath": "/App_Config/Include/zzz",
12    "graphQLEndpointPath": "/sitecore/api/graph/edge",
13    "language": "en",
14    "templates": [
15      "nextjs",
16      "nextjs-sxa",
17      "nextjs-personalize",
18      "nextjs-multisite"
19    ]
20  },
21  "engines": {
22    "node": ">=12",
23    "npm": ">=6"
24  },
25  "author": {
26    "name": "Sitecore Corporation",
27    "url": "https://jss.sitecore.com"
28  },
29  "repository": {
30    "type": "git",
31    "url": "git+https://github.com/sitecore/jss.git"
32  },
33  "bugs": {
34    "url": "https://github.com/sitecore/jss/issues"
35  },
36  "license": "Apache-2.0",
37  "dependencies": {
38    "@sitecore-feaas/clientside": "^0.3.17",
39    "@sitecore-jss/sitecore-jss-nextjs": "~21.5.0",
40    "@sitecore/engage": "^1.4.1",
41    "bootstrap": "^5.1.3",
42    "font-awesome": "^4.7.0",
43    "graphql": "~15.8.0",
44    "graphql-tag": "^2.12.6",
45    "next": "^13.4.16",
46    "next-localization": "^0.12.0",
47    "react": "^18.2.0",
48    "react-dom": "^18.2.0",
49    "sass": "^1.52.3",
50    "sass-alias": "^1.0.5",
51    "content-components": "*",
52    "ui": "*"
53  },
54  "devDependencies": {
55    "@graphql-codegen/cli": "^1.21.8",
56    "@graphql-codegen/import-types-preset": "^2.2.6",
57    "@graphql-codegen/plugin-helpers": "^3.1.2",
58    "@graphql-codegen/typed-document-node": "^2.3.12",
59    "@graphql-codegen/typescript": "^2.8.7",
60    "@graphql-codegen/typescript-operations": "^2.5.12",
61    "@graphql-codegen/typescript-resolvers": "^2.7.12",
62    "@graphql-typed-document-node/core": "^3.1.1",
63    "@sitecore-jss/sitecore-jss-cli": "~21.5.0",
64    "@sitecore-jss/sitecore-jss-dev-tools": "~21.5.0",
65    "@types/node": "^18.11.18",
66    "@types/react": "^18.0.12",
67    "@types/react-dom": "^18.0.5",
68    "@typescript-eslint/eslint-plugin": "^5.49.0",
69    "@typescript-eslint/parser": "^5.49.0",
70    "chalk": "~4.1.2",
71    "chokidar": "~3.5.3",
72    "constant-case": "^3.0.4",
73    "cross-env": "~7.0.3",
74    "dotenv": "^16.0.3",
75    "eslint": "^8.32.0",
76    "eslint-config-next": "^13.1.5",
77    "eslint-config-prettier": "^8.6.0",
78    "eslint-plugin-prettier": "^4.2.1",
79    "eslint-plugin-react": "^7.32.1",
80    "eslint-plugin-yaml": "^0.5.0",
81    "graphql-let": "^0.18.6",
82    "npm-run-all": "~4.1.5",
83    "prettier": "^2.8.3",
84    "ts-node": "^10.9.1",
85    "tsconfig-paths": "^4.1.2",
86    "typescript": "~4.9.4",
87    "yaml-loader": "^0.8.0"
88  },
89  "scripts": {
90    "bootstrap": "ts-node --project tsconfig.scripts.json scripts/bootstrap.ts",
91    "build": "npm-run-all --serial bootstrap next:build",
92    "graphql:update": "ts-node --project tsconfig.scripts.json ./scripts/fetch-graphql-introspection-data.ts",
93    "install-pre-push-hook": "ts-node --project tsconfig.scripts.json ./scripts/install-pre-push-hook.ts",
94    "jss": "jss",
95    "lint": "eslint ./src/**/*.tsx ./src/**/*.ts ./scripts/**/*.ts",
96    "next:build": "next build",
97    "next:dev": "cross-env NODE_OPTIONS='--inspect' next dev",
98    "next:start": "next start",
99    "scaffold": "ts-node --project tsconfig.scripts.json scripts/scaffold-component/index.ts",
100    "start:connected": "npm-run-all --serial bootstrap --parallel next:dev start:watch-components",
101    "start:production": "npm-run-all --serial bootstrap next:build next:start",
102    "start:watch-components": "ts-node --project tsconfig.scripts.json scripts/generate-component-builder/index.ts --watch"
103  }
104}
105

What we have changed is that we have added "content-components" on line 51

Now to finalise run npm install in the root of the turborepo application "src/sxastarter". this will make sure the new package is actually registered as a package for our sxastarter app.

Now we need to make another adjustment: we need to tell nextjs to transpile our "content-components" package. We do this by adjusting the next.config.js file in the "src/sxastarter/apps/sxastarter" folder. We need to add the following code snippet:

1const nextConfig = {
2  transpilePackages: ['content-components'],
3
4.....
5}

Now the final step to take it to make our sxastarter jss app expose the promo component andits variants in the componentbuilder we can do this by adjusting the "packages.ts" file in "src/sxastarter/apps/sxastarter/scripts/generate-component-builder/plugins". We need to make it look like:

1import { ComponentBuilderPlugin, ComponentBuilderPluginConfig } from '..';
2
3/**
4 * Provides custom packages configuration
5 */
6class PackagesPlugin implements ComponentBuilderPlugin {
7  order = 0;
8
9  exec(config: ComponentBuilderPluginConfig) {
10    /**
11     * You can specify components which you want to import from external/internal packages
12     * in format:
13     *  {
14     *    name: 'package name',
15     *    components: [
16     *      {
17     *        componentName: 'component name', // component rendering name,
18     *        moduleName: 'module name' // component name to import from the package
19     *      }
20     *    ]
21     *  }
22     */
23    config.packages = [];
24
25    config.packages.push({
26      name: 'content-components',
27      components: [
28        {
29          componentName: 'Promo',
30          moduleName: 'Promo',
31        },
32      ],
33    });
34
35    return config;
36  }
37}
38
39export const packagesPlugin = new PackagesPlugin();
40

What we have done here is add an entry for our Promo component into the packages file of the components configuration. We told jss that we have a React component called "Promo" which should be mapped against the Sitecore component called "Promo".

Now we are all set and can run our new setup. Do this by running:

1npm run start:connected

After your application has started you can open the localhost:3000 site to see exactly the same screen as earlier. But now with the promo component coming from the "content-components" package. Everything will still work including fast-reload when adjusting the Promo component.

if you now open the file "src/sxastarter/apps/sxastarter/src/temp/componentBuilder.ts" you will see our promo component has been added from our "content-components" package:

1/* eslint-disable */
2// Do not edit this file, it is auto-generated at build time!
3// See scripts/generate-component-builder/index.ts to modify the generation of this file.
4
5
6import { ComponentBuilder } from '@sitecore-jss/sitecore-jss-nextjs';
7
8import { Promo } from 'content-components';
9import { BYOCWrapper, FEaaSWrapper } from '@sitecore-jss/sitecore-jss-nextjs';
10
11import * as CdpPageView from 'src/components/CdpPageView';
12import * as ColumnSplitter from 'src/components/ColumnSplitter';
13import * as Container from 'src/components/Container';
14import * as ContentBlock from 'src/components/ContentBlock';
15import * as Image from 'src/components/Image';
16import * as LinkList from 'src/components/LinkList';
17import * as Navigation from 'src/components/Navigation';
18import * as PageContent from 'src/components/PageContent';
19import * as PartialDesignDynamicPlaceholder from 'src/components/PartialDesignDynamicPlaceholder';
20import * as RichText from 'src/components/RichText';
21import * as RowSplitter from 'src/components/RowSplitter';
22import * as Title from 'src/components/Title';
23
24const components = new Map();
25components.set('Promo', Promo);
26components.set('BYOCWrapper', BYOCWrapper);
27components.set('FEaaSWrapper', FEaaSWrapper);
28
29components.set('CdpPageView', CdpPageView);
30components.set('ColumnSplitter', ColumnSplitter);
31components.set('Container', Container);
32components.set('ContentBlock', ContentBlock);
33components.set('Image', Image);
34components.set('LinkList', LinkList);
35components.set('Navigation', Navigation);
36components.set('PageContent', PageContent);
37components.set('PartialDesignDynamicPlaceholder', PartialDesignDynamicPlaceholder);
38components.set('RichText', RichText);
39components.set('RowSplitter', RowSplitter);
40components.set('Title', Title);
41
42export const componentBuilder = new ComponentBuilder({ components });
43
44export const moduleFactory = componentBuilder.getModuleFactory();
45

Wrap up

I hope you have been able to setup a monorepo with turborepo following my blog post. If you want to see the end result please visit https://github.com/dhudepol/xmcloud-turborepo and grab a copy

XM Cloud: leveraging Turborepo with XM Cloud