Black Sheep Code
rss_feed

I've migrated this blog to use NextJS. There's two main reasons I did this:

Core Principles

  1. I want to keep the same codebase - retaining my git history, I don't want to create a brand new codebase.

  2. Try not to change everything. While I might also want to considering moving to Vercel - bundling all those little changes into the one migration makes a large task larger and becomes death by a thousands cuts.

Some examples of things that I might consider changing, but am avoiding in the first iteration of the migration are:

The Plan Of Attack

warning

In practise it was not nearly as smooth as this blog post suggests. There were missteps in the wrong direction and course reversals that I'm leaving out for the sake brevity.

  1. Create a new project, scaffolded with create-next-app, get all of the blog functionality working generally.
  2. While I'm doing that, also create a preparation pull request to the existing codebase that will:
  1. Copy paste the new project over. Fix anything that's broken.
  2. Get the build deploying on Netlify.
  3. Get the Cypress tests working.
  4. Get Sentry working.

TL;DR Conclusions

  1. The actual migration of core functionality was fine. In fact if anything several things were much easier to do using NextJS - for example where dynamic imports work fine under NextJS whereas they didn't work for me with Remix.

  2. The fact that I had tests for some of my build-time scripts made it so much easier to modify those scripts, using ChatGPT, and use the tests to check the functionality.

  3. Cypress has been a nightmare.

The Execution

Phase 1: Create a new application using create-next-app - reproduce the functionality there.

I scaffold a new application using create-next-app at this repository.

1.1 File structure for the blog posts

Difficulty: 🟠 - Not hard once I got my head around it, but there were some false starts.

The existing file structure looked like this:


app/ <--  nb. this is Remix's default 'src' directory, it has nothing to do with Next's app router. 
   routes/ 
      posts/ 
         a_blog_post.mdx
         another_post.mdx
      test/ 
         test_post.mdx
      drafts/ 
         draft_post.mdx
          

(Test posts are ones I run cypress tests against, and draft posts are where I put my posts while I'm working on them, and then I move them to posts when I 'publish' them)

Now I could (and in fact I did go down this path until I realised it wouldn't work for me) go down the path of doing Next's file based MDX routing and have a structure like this:

src/ 
   app/ 
      posts/
        a_blog_post/ 
            page.mdx
        another_post/ 
            page.mdx
      test/
        test_post/ 
            page.mdx
      drafts/
        draft_post/ 
            page.mdx

but:

File based MDX routing doesn't work for me

This all comes down to the frontmatter metadata I have included in my MDX files.

I make use of Frontmatter to provide metadata in this posts like the title and description, but also to create the blog series sections, create the list of blog posts, etc.

The difficulty is, each blog post needs to be wrapped in what I'm calling a BlogPostFrame - this contains the standard layout, the 'edit with github', the comments section, as well as that series box at the top, if the post is part of the series.

That sounds like a NextJS layout component right?

Maybe we do something like this?

src/ 
   app/ 
      posts/
        layout.tsx <-- Put the BlogPostFrame here 
        a_blog_post/ 
            page.mdx
        another_post/ 
            page.mdx

The problem is - this layout.tsx has no way of knowing what page we're on, or what the metadata is supposed to be.

My Approach: Compile the MDX myself + Use dynamic routes

The folder structure I use looks like this:

src/ 
   app/ 
      posts/
        [slug]/
           page.mdx
      test/
        [slug]/
           page.mdx
      drafts/
        [slug]/
           page.mdx
   routes/ <-- This is the existing Remix routes folder, but there is no file based routing occurring here any more, these files will be compiled at build time
      posts/ 
         a_blog_post.mdx
         another_post.mdx
      test/ 
         test_post.mdx
      drafts/ 
         draft_post.mdx

I then have build-time scripts that run to compile the mdx files into a structure that looks like this:

src/
   app/
       generated/
           mdx/ 
               posts/
                  a_blog_post.mjs   <-- MDX Compiled into plain React JavaScript
                  another_post.mjs
               test/
                  test_post.mjs
               drafts/
                  draft_post.mjs
           frontmatter/
               posts/
                  a_blog_post.json <--  Frontmatter extracted to JSON format
                  another_post.json
               test/
                  test_post.json
               drafts/
                  draft_post.json

The whole process of separately generating frontmatter JSONs may look unnecessary - the remark-mdx-frontmatter package can export the frontmatter as part of the same compiled MDX file.

However, my existing tooling also included:

  • Validating the metadata is of a consistent format. I want a build time error if the metadata is not the right shape.

  • Additional munging relating to post series - when it's a post series I need to also collect a reference too all other posts in the series, and my existing tooling was already doing this for me.

The generation of the frontmatter json files is something I did in my Remix solution, you can read about this here.

The plain rendering of my page then just uses dynamic import to retrieve the javascript it needs and renders that:

The metadata rendering is done in a similar fashion, I export a NextJS generateMetatdata function. This function is what causes our meta tags etc to put in the HTML head. The generateMetadata function asynchronously fetches the metadata based on the slug.

1.2 Change import aliases

Difficulty: 🟢 - Easy.

NextJS by default uses @/ aliases, whereas my Remix config had ~/ as an alias, so use find replace to update these.

Difficulty: 🟢 - Easy.

I copy paste over my app/components folder, which has various miscellaneous components, like the info boxes.

The linter gives me a warning about maybe I should use NextJS <Image/> tags - but in the philosophy of 'try not to change everything' that can be a job for another day.

The main things to be done where:

  1. Links - Change Remix links to NextJS links
  2. Images - Remix image imports resolve as a string, whereas NextJS image imports resolve as a {src: string, height: number, width: number} and so my use of images are changed to: <img src ={image.src}/>.

1.4 Migrate components - add "use client" directive

Difficulty: 🟢 - Easy.

Because I'm using NextJS's app router any of my components that have a useEffect etc in them, need to be prefixed with "use client".

Difficulty: 🟢 - Easy with a twist.

I maintain a package react-github-permalink to display the codeblocks here. Pages with the codeblocks error with:

Error: Super expression must either be null or a function

which appears to relate to client components running in a RSC.

To fix, I updated react-github-permalink to add a "use client" directive everywhere.

The plan is to eventually have react-github-permalink behave as a RSC but for now client component will do.

One little gnarl I ran into is this error:

"Unsupported Server Component type: undefined"

According to this Stack Overflow answer not only do we need to add the "use client" directive to the components, but if doing an export * from we also need to add the directive to that.

1.6 Add the CSS in

Difficulty: 🟢 - Easy.

At this point I'm just checking that the pages are rendering without error.

I copy my CSS file (I'm writing just vanilla CSS) and that all works fine.

1.7 Populate my root layout

Difficulty: 🟢 - Easy.

Here I take the logic from my Remix root.tsx and put it in src/app/layout.tsx

The Remix style LinksFunction objects

are converted to plain HTML rendering:

1.8 Generate sitemap.xml, rss.xml

Difficulty: 🟢 - Easy.

In Remix these files can be handled with a [sitemap.xml].tsx file - which is pretty convenient.

NextJS is a bit simpler - as part of the build process - I generate the files and put them in my public folder directly.

End of Phase 1

Status: I have a standalone project separate from my main blog that is running and appears to be functional.

Phase 2: Preparation pull request for migration

As I encountered errors while creating the new application - I wrote them down in a list, and created cypress tests for them.

I also made sure that Cypress was testing core pieces of functionality like that the 404 pages were working, the rss.xml was working etc.

I renamed the root level app folder of Remix to src. It appears the in Remix you can configure what the root directory is, but in NextJs you can't.

I otherwise moved some other files about as part of a tidy up.

End of Phase 2

Status: No major changes to the main blog. Some extra tests are running. I've renamed the root folder.

The pull request for this change is here.

Phase 3: Copy the files

In this phase I copy over the changes I made in my temporary repo, over to the main repo.

3.1 Update the package.json

This is a fairly manual process, I need to remove all the Remix dependencies and add the NextJS dependencies.

I also add a convenience generate:all script which run all of my build steps except the main application build.

3.2 Copy over the main files

This is basically updates to everything in my src folder and utils folder (which is a badly named folder, it contains all of the build time scripts).

3.3 Remix specific files

Removes remix.config.js, entry.client.tsx, entry.server.tsx, root.tsx etc.

3.4 Add NextJS specific files

Replaces the eslint config with the NextJS provided one.

Replaces the tsconfig with the NextJS provided one.

At time of writing I'm noticing that I neglected to copy across next.config.mjs and it appears to work fine 🤷‍♂️

3.5 Replace jest with vitest

The reason I did this was that I didn't want to keep babel around just so I can run my jest tests.

I feel like I had a better reason than this, but if I did, I didn't write it down in my notes.

Updating the tests themselves was pretty simple - it's just a matter of replacing references to jest with vitest, and some minor changes to how vitest types it's fn function.

End of Phase 3

Status: I can run the blog locally.

The PR for this change is here.

At time of writing, the PR has 87 file changes, these can be broken down roughly as follows :

Phase 4: Get it running on Netlify

Turns out this is pretty simple.

I remove my server.ts and make the netlify.toml be:

I use Netlify's branch previews to verify that it's working.

End of Phase 4

Status: Branch previews are deploying properly, but the Cypress tests are failing

Phase 5: Get the Cypress tests working

At this point lots of my cypress tests are failing, so we'll go through an fix them.

It is this part of the migration that has given me the most problems. I actually thought I'd stumbled on a solution to many of these problems, but a while later the problems were still apparent - this is one of the difficulties - the issues I'm facing are not reliably reproducible.

At time of writing I have not solved these problems, but I don't want to delay publishing this post, so the plan is to document them here and maybe write a second post about Cypress later.

5.1 Assertions on static resources

Difficulty: 🟢 Easy.

One expected error is my tests that look like:

That file path can obviously be expected to differ in the way the Remix and Next render it - so I go through and change those. Straight forward.

Difficulty: 🔴 - Difficult.

This test is failing on line 65 - the link is clicked but the page doesn't redirect and the subsequent content isn't found.

This test fails consistently, but we can't reproduce the error manually.

This very helpful Stack Overflow helps us shed some light. The link element is found, and we click it, but by the time we click it the element is detached from the DOM and the click doesn't register.

As a workaround - we can add {force:true} to click command.

This is a thoroughly unsatisfying resolution though.

5.2 Mysterious Minified React error #329 / Unknown root exit status error.

Difficulty: 🔴 - Difficult.

In this test the Github Permalinks fail to render.

We see Minified React error #329, (Unknown root exit status) in the console.

This test fails consistently, but we can't reproduce the error manually.

This appears to be something to do with NextJS, but it's unclear what.

I don't have a resolution for this - I'm skipping those tests.

Note minified React errors only seem to occur when visiting the application via Cypress. This is an issue that I've encountered before with my tests against Remix as well , so that they exist doesn't especially concern me.

But it does concern me is that in NextJS these errors cause functional problems - in Remix the functionality of the application would continue to work.

5.3 iframe not found

Difficulty: 🔴 - Difficult.

In this test the Utterances comment section iframe fails to appear.

This test fails sporadically. We are seeing a lot of minified React errors, and it could be something to do with that #329 error above.

It may be that this iframe is the offender for all of the issues. This comment in the above Cypress issue suggests wrapping scripts in a suspense boundary.

I wrapped one around my PostComments component and for a bit these issues did seem to be resolved, though they resurfaced.

End of phase 5

Status: most of the Cypress tests are passing, and others are erroring and I'm ignoring them. We're almost ready to go!

My debugging of Cypress tests consisted of repeatedly running the test to see if we encountered the errors.

Ultimately I found three sources of errors:

I have not yet been able to create a reliable reproduction of these, so maybe I'm off on a wild goose chase here.

Phase 6: Get Sentry working

I added Sentry using their wizard.

Conclusions

Migrating to NextJS is not a particularly difficult task.

Where some pause should be given, is that adjacent tooling, such as Storybook, React Testing Library, Cypress, may not have caught up to the new RSC paradigm - where there may be a mature ecosystem of tooling for conventional SPAs you may find yourself on the bleeding (read: unreliable) edge for RSCs.



Questions? Comments? Criticisms? Get in the comments! 👇

Spotted an error? Edit this page with Github