This article is a part of the series "Creating a blog with Remix"
How to configure Remix and mdx-bundler for a serverless platform - barrel files approach, Remix v1
In the previous post I outlined how to make use of mdx-bundler to access your frontmatter metadata. This approach requires file system access as the files are parsed at run time.
In this post, we'll we'll talk about how to achieve the same effect for use in a serverless platform, such a Netlify or AWS Lambda.
The solution involves a build time step that extracts the frontmatter to JSON files. These JSON files can then be read at runtime.
This solution is for Remix V1
This solution is for Remix V1. The reason is because V2 made changes to Remix's routing and got rid of nested routing.
Where in V1 we can organise our routing like:
/app
/routes
/posts
blogpost1.mdx
blogpost2.mdx
/drafts
draftpost1.mdx
posts.tsx
some_other_page.tsx
Remix V2 got rid of the ability to use file/folder nesting to determine the URL, the equivalent solution would look like:
/app
/routes
posts.blogpost1.mdx
posts.blogpost2.mdx
drafts.draftpost1.mdx
posts.tsx
some_other_page.tsx
For potentially hundreds of files this would get quite messy.
(However, I concede that this is potentially something I could live with).
Also note that I don't want to do a 'store the blog posts in a database' solution. This is something that is recommended in Remix's documentation but I don't agree with the reasoning. In my opinion two of the major advantages of using a Github-hosted-MDX-files solution is that you provide a convenient way of allow viewers to submit edit requests, and it provides a handy version history.
This solution uses barrel files
A barrel file is an index file that looks like this:
export {default as adding_msw_bundler_to_remix_app_2} from './adding_msw_bundler_to_remix_app_2.json';
export {default as adding_msw_bundler_to_remix_app} from './adding_msw_bundler_to_remix_app.json';
//etc
The reason we need to use barrel files is we can't do dynamic imports with Remix (though the change to using Vite might have changed this).
Maybe Remix isn't the right tool for the job.
This is all feeling a bit hacky. When I started this blog it felt 50/50 the race between NextJS and Remix, whereas in 2024 it seems like NextJS is really pulling ahead. If I were to be starting this blog today, I'd be using NextJS or perhaps Astro.
That said, this is going pretty smoothly, for what it's worth.
Starting Point - We have frontmatter in our posts
For example, here is the frontmatter for this post:
1---
2meta:
3 title: How to configure Remix and mdx-bundler for a serverless platform - barrel files approach, Remix v1
4 description: If using Remix on a serverless platform such as Netlify we can use a build time compilation and barrel files to access frontmatter metadata.
5 dateCreated: 2024-03-22
6
7series:
8 name: remix_blog
9 part: 3
10---
11 Note that this frontmatter is entirely custom, we're going to need to write handlers for it later.
Starting point - We have multiple folders of blog posts
My file structure looks like this:
/app
/routes
/posts
blogpost1.mdx
blogpost2.mdx
/drafts
draftpost1.mdx
/test
testpost1.mdx
posts.tsx
drafts.tsx
test.tsx
some_other_page.tsx
This allows me to write draft posts and see how they look, but selectively only publish the posts posts.
Create a validation schema for our custom frontmatter
Here I'm using Zod to create a validation schema:
Lines 1 to 33 in 24c6f3e
1import {z} from "zod";
2
3const metaSchema = z.object({
4 title: z.string(),
5 description: z.string(),
6 dateCreated: z.date().or(z.string()),
7 });
8
9 const seriesSchema = z.object({
10 name: z.string(),
11 part: z.number(),
12 description: z.string().optional()
13 });
14
15export const frontMatterSchema = z.object({
16 meta: metaSchema,
17 series: seriesSchema.optional(),
18})
19
20export type FrontMatter = z.infer<typeof frontMatterSchema>;
21
22export type FrontMatterPlusSlug = {
23 slug: string;
24 frontmatter: FrontMatter;
25}
26
27
28export type EnrichedFrontMatterPlusSlug = FrontMatterPlusSlug & {
29 seriesFrontmatter: Array<FrontMatterPlusSlug> | null;
30}
31
32
33What I've done is I've said that title, description and date created are always going to exist. I can continue extending this schema in the future, and adding optional properties.
A build time tool to generate JSON and barrel files from our frontmatter
Lines 1 to 109 in 24c6f3e
1import fm from "front-matter";
2
3import fs from 'fs';
4import fsAsync from "fs/promises"
5import path from 'path';
6import { frontMatterSchema } from "./frontmatterTypings";
7
8async function _writeFile(filePath:string, slug: string, output: unknown) {
9 await fsAsync.writeFile(filePath, JSON.stringify({
10 slug,
11 frontmatter: output
12 }, null, 2));
13}
14
15export type WriteFileFn = typeof _writeFile;
16export type AppendIndexFileFn = typeof _appendIndexFile;
17export type CreateSubfolderFn = typeof _generateSubfolder;
18
19async function _appendIndexFile(filePath:string, fileName: string) {
20 await fsAsync.appendFile(filePath, `export {default as ${fileName}} from './${fileName}.json';\n`);
21}
22
23async function _generateSubfolder(subPath: string) : Promise<string> {
24 const basePath = path.join(process.cwd(), "app", "generated", "frontmatter", subPath);
25 await fsAsync.mkdir(basePath, { recursive: true });
26 return basePath;
27}
28
29export async function extractFrontMatter(folderPath: string, writeFile : WriteFileFn = _writeFile, appendIndexFile : AppendIndexFileFn = _appendIndexFile, generateSubfolder : CreateSubfolderFn = _generateSubfolder) {
30
31 const endToken = ".mdx";
32 function getParts(inputString: string): [string, string] {
33 const startIndex = inputString.indexOf(folderPath);
34 const endIndex = inputString.indexOf(endToken);
35 if (startIndex !== -1 && endIndex !== -1) {
36 const extractedPart = inputString.substring(folderPath.length + 1, endIndex);
37 const segments = extractedPart.split('/');
38 if (segments.length !== 2) {
39 throw new Error("Expected an array of length 2")
40 }
41
42 return segments as [string, string];
43 } else {
44 throw new Error("String format is not as expected");
45 }
46 }
47
48 async function processFile(inputString: string) {
49 const file = await fsAsync.readFile(inputString);
50 const fileText = file.toString();
51 const [subPath, fileName] = getParts(inputString);
52
53 const output = fm(fileText);
54
55 const basePath = await generateSubfolder(subPath);
56
57 try {
58 frontMatterSchema.parse(output.attributes);
59 }
60 catch (e) {
61 throw new Error(`
62 Error parsing file : '${inputString}'.
63 Failed Zod validation with: ${e instanceof Error && e.message}
64
65 The object was: ${JSON.stringify(output.attributes)}
66 `)
67 }
68
69 await writeFile(path.join(basePath, fileName + ".json"), `${subPath}/${fileName}`, output.attributes);
70 await appendIndexFile(path.join(basePath, 'index.js'), fileName);
71
72}
73
74 async function findMdxFiles(folderPath: string) {
75 const promises = [] as Array<Promise<unknown>>;
76 function traverseDir(currentPath: string) {
77 const files = fs.readdirSync(currentPath);
78
79 files.forEach((file) => {
80 const filePath = path.join(currentPath, file);
81 const fileStats = fs.statSync(filePath);
82 if (fileStats.isDirectory()) {
83 traverseDir(filePath);
84 } else if (file.endsWith('.mdx')) {
85 // If it's an .mdx file, log it
86 promises.push(processFile(filePath));
87 }
88 });
89
90
91 }
92
93 traverseDir(folderPath);
94
95 await Promise.all(promises);
96 }
97
98
99 await findMdxFiles(folderPath)
100}
101
102As an aside, this code was largely generated with assistance of ChatGPT, and then I tweaked it as I want.
Now looking at it, it looks kind of gross.
But right now, I don't care enough. The code is tested, and my philosophy is that 'good code is easy to delete', the interface is fine, we can rewrite this later if we need.
Essentially the process here is:
- Recursively traverse our routes structure, finding the .mdx files
- Use the front-matter package to parse the MDX files, extracting the front-matter.
- Validate that the frontmatter matches our Zod schema
- For each MDX file, create a json file in
app/generated/frontmattermatching the mdx file path - Once all JSON files have been created, create a index.js barrel file in each folder.
I have a separate file that runs this code:
Lines 1 to 16 in 24c6f3e
1import path from 'path';
2import { extractFrontMatter } from './extractFrontMatter';
3
4
5const folderPath = path.join(process.cwd(), 'app', 'routes'); // Update the path accordingly
6console.info("Being extracting frontmatter...")
7extractFrontMatter(folderPath).then(() => {
8 console.info("Front matter extraction complete!")
9}).catch(err => {
10 console.error("Error extracting frontmatter:")
11 throw err;
12});
13
14
15
16And we have a script in our package.json to calls this:
Lines 12 to 13 in 24c6f3e
12 "generate:mdxjson": "rm -rf app/generated/frontmatter && tsx utils/extractFrontMatter.bin.ts",
13 "test": "jest --watchAll"Example Layout Route
Here's what my posts.tsx layout route is looking like:
Lines 1 to 29 in 73fc879
1import { Outlet, useLocation } from "@remix-run/react"
2import { EditWithGithub } from "~/components/EditWithGithub/EditWithGithub"
3import PostComments from "~/components/PostComments/PostComments";
4import { createLoaderFunction, createMetaFunction } from "~/utils/blogPosts";
5import { FrontmatterBox } from "~/components/FrontmatterBox/FrontmatterBox";
6import { IndexRoute } from "~/components/IndexRoute";
7
8
9
10export const loader = createLoaderFunction("posts");
11export const meta = createMetaFunction("posts");
12
13
14
15export default () => {
16 const params = useLocation();
17 const isIndexRoute = params.pathname.slice(1) === "posts";
18
19 if (isIndexRoute){
20 return <IndexRoute folder ="posts"/>
21 }
22 return <>
23 <FrontmatterBox>
24 <Outlet />
25 </FrontmatterBox>
26 <EditWithGithub postName={params.pathname} />
27 <PostComments />
28 </>
29}This is what gives a consistent look feel across all the posts.
The key things we need to implement are:
- FrontmatterBox - This is blog post specific frame, it contains that series box, as well as can contain things like author, date published, etc.
- createLoaderFunction - the Remix loader function, this will need examine the frontmatter jsons.
- createMetaFunction - the Remix meta function, this will need to examine the frontmatter jsons.
createLoaderFunction
This function creates our Remix loader function.
Lines 1 to 84 in 73fc879
1import type { LoaderFunction, MetaFunction } from "@remix-run/node";
2import * as allDraftMetaData from "../generated/frontmatter/drafts";
3import * as allPostMetaData from "../generated/frontmatter/posts";
4import * as allTestMetaData from "../generated/frontmatter/test";
5import type { EnrichedFrontMatterPlusSlug, FrontMatter, FrontMatterPlusSlug } from "utils/frontmatterTypings";
6
7export type BlogPostFolders = "drafts" | "posts" | "test";
8
9
10export const allMetadata = {
11 "drafts": allDraftMetaData as Record<string, FrontMatterPlusSlug>,
12 "posts": allPostMetaData as Record<string, FrontMatterPlusSlug>,
13 "test": allTestMetaData as Record<string, FrontMatterPlusSlug>,
14}
15
16
17const DEFAULT_METADATA = {}
18
19export function getFolderAndFilenameFromSlug(slug: string): {
20 folder: BlogPostFolders,
21 filename: string;
22
23} {
24
25 const [, folder, fName] = slug.split(/[?/#]/);
26
27 if (!folder) {
28 throw new Error("Expected folder to exist");
29 }
30 if (!fName) {
31 throw new Error("Expect fName to exist");
32 }
33
34 return {
35 folder: folder as BlogPostFolders,
36 filename: fName,
37 }
38}
39
40/**
41 * Although it is currently unnecessary for this function to be async
42 * In future we may be retrieving the frontmatter from a database or async import
43 * So let's handle for it being async now
44 * @param slug
45 * @returns
46 */
47export async function getFrontmatterFromSlug(slug: string): Promise<EnrichedFrontMatterPlusSlug> {
48 const { folder, filename } = getFolderAndFilenameFromSlug(slug);
49
50 const data = allMetadata[folder][filename];
51 if (!data) {
52 throw new Error(`Frontmatter did not exist for slug: '${slug}`)
53 }
54
55 let seriesFrontmatter : Array<FrontMatterPlusSlug> | null = null;
56 if ('series' in data.frontmatter) {
57 seriesFrontmatter = (Object.values(allMetadata[folder]).filter((v) => {
58 return v.frontmatter.series?.name === data.frontmatter.series?.name
59 }) ).sort((a, b) => {
60 return (a.frontmatter.series?.part ?? 0) - (b.frontmatter.series?.part ?? 0)
61 });
62 }
63
64 return { ...data, seriesFrontmatter } as EnrichedFrontMatterPlusSlug;
65}
66
67
68export function createLoaderFunction(folder: BlogPostFolders): LoaderFunction {
69 return async (loaderArgs) => {
70
71 // Unfortunately we don't have access to the path via loader args, so we have to manually extract
72 // it from the request.
73 const url = loaderArgs.request.url;
74
75 const path = new URL(url).pathname;
76
77 if(path.slice(1) === folder) {
78 return null;
79 }
80
81
82 return getFrontmatterFromSlug(path)
83 }
84}The logic here is a little hardcoded - it won't work for deeper levels of nesting, but essentially we use the location URL to determine the folder and filename to retrieve the metadata json from the correct barrel file.
Note for the series metadata, I also collate all of the metadata for that series in with response payload. That's how I create the series contents on a given page.
createMetaFunction
This function creates our Remix meta function.
Lines 87 to 115 in e3162ff
87function mergeFrontmatterAndDefaultMetadata(frontmatter: FrontMatterPlusSlug | null) {
88
89 if (!frontmatter) {
90 return DEFAULT_METADATA;
91 }
92
93 return {
94 ...DEFAULT_METADATA,
95 title: frontmatter.frontmatter.meta?.title,
96 description: frontmatter.frontmatter?.meta?.description,
97 "twitter:title": frontmatter.frontmatter.meta?.title,
98 "twitter:description": frontmatter.frontmatter?.meta?.description,
99 }
100}
101
102export function createMetaFunction(folder: BlogPostFolders): MetaFunction {
103 return (metaInput) => {
104 const loaderResult = metaInput.data as FrontMatterPlusSlug | null;
105 return mergeFrontmatterAndDefaultMetadata(loaderResult ?? null);
106 }
107
108}
109
110
111export async function getAllPostFrontmatter() : Promise<Array<FrontMatterPlusSlug>> {
112 return Object.values(allPostMetaData as Record<string, FrontMatterPlusSlug>).sort((a,b) => {
113 return new Date(b.frontmatter.meta?.dateCreated ?? 0).valueOf() - new Date(a.frontmatter.meta?.dateCreated ?? 0).valueOf();
114 });
115}Importantly, it does things like transforming the title and description into the correct format twitter meta tags.
FrontmatterBox
This is component is responsible for transforming any of metadata into whatever content we want for that post, series contents, 'next post in series', etc.
1import type { PropsWithChildren} from "react";
2import React, { useEffect, useState } from "react";
3import { Link,useLocation } from "@remix-run/react";
4import { getFrontmatterFromSlug } from "~/utils/blogPosts";
5import type { EnrichedFrontMatterPlusSlug } from "utils/frontmatterTypings";
6
7type FrontmatterBoxProps = {
8 frontmatter: EnrichedFrontMatterPlusSlug| null;
9}
10
11
12function SeriesBox(props: FrontmatterBoxProps) {
13
14 if(!props.frontmatter?.seriesFrontmatter){
15 return null;
16 }
17
18 if(!props.frontmatter.frontmatter.series){
19 return null;
20 }
21
22 const firstSeriesItem = props.frontmatter?.seriesFrontmatter[0];
23
24 return <div className="series-box">
25 <p>
26 This article is a part of the series "<i className="series-description-text">{firstSeriesItem.frontmatter.series?.description ?? firstSeriesItem.frontmatter.series?.name}</i>"
27 </p>
28 <ul>
29 {props.frontmatter?.seriesFrontmatter?.map((v) => {
30 return <li key={v.slug}>
31 <Link to={`/${v.slug}`} className={v.slug === props.frontmatter?.slug ? "current" : ""} >
32 {v.frontmatter.meta?.title ?? v.slug}
33 </Link>
34 </li>
35 })}
36 </ul>
37 </div>
38
39}
40
41
42function partIsNumber(part: number | undefined) : part is number {
43 return typeof part === 'number';
44}
45
46function NextBox(props: FrontmatterBoxProps) {
47 const part = props.frontmatter?.frontmatter.series?.part;
48 if( !partIsNumber(part)){
49 return null;
50 }
51
52 const nextInSeries = part; // nb. the series are 1 indexed, but the array here is 0 indexed.
53
54 if (props.frontmatter?.seriesFrontmatter && props.frontmatter.seriesFrontmatter[nextInSeries]) {
55
56 const nextPost = props.frontmatter.seriesFrontmatter[nextInSeries];
57
58 return <div className ="next-post">
59 <Link to = {`/${nextPost.slug}`}><strong>Next:</strong> {nextPost.frontmatter.meta.title}</Link>
60 </div>
61 }
62 return null;
63}
64
65export function FrontmatterBox(props: PropsWithChildren<{}>) {
66
67 const location = useLocation();
68 const [pathName, setPathname] = useState(null as null | string);
69 const [value, setValue] = useState<null | EnrichedFrontMatterPlusSlug>(null);
70
71 useEffect(() => {
72 if(pathName !== location.pathname){
73 setPathname(location.pathname);
74 getFrontmatterFromSlug(location.pathname).then((v) => {
75 setValue(v);
76 });
77 }
78 }, [pathName, location.pathname]);
79
80 if(!value){
81 return props.children;
82 }
83 return <><div>
84 <SeriesBox frontmatter={value}/>
85 </div>
86 {props.children}
87
88 <NextBox frontmatter={value}/>
89
90 <>
91 <br/>
92 <br/>
93 Questions? Comments? Criticisms? <a href="#comments">Get in the comments! 👇</a>
94 </>
95 </>
96}Creating a table of contents
We can create a table of contents on our home page using the getAllPostFrontmatter function.
Lines 27 to 29 in e3162ff
27export async function loader() {
28 return await getAllPostFrontmatter();
29}Conclusions and my experience
Now that I've got this going, it works pretty well.
One thing that isn't work for me - is that I can't put images in the metadata, which is important for social sharing. However, a similar barrel file approach might work.
The rigmarole of doing this suggests that Remix might not be the right tool for the job. Astro is something worth considering, also I might look at NextJS.
For one - a blog a good use case for static site generation, which Remix doesn't support.
This solution doesn't support an automated created/updated fields - those have to maintained manually. Perhaps some git commands in the build time JSON generation could do it.
Note that the barrel files are only ever used by Remix loader and meta functions, ie. on the server, so we don't need to worry about them being needlessly pulled in the frontend. If we had thousands of posts this might start posing a problem.
Questions? Comments? Criticisms? Get in the comments! 👇
Spotted an error? Edit this page with Github