Watch and Learn | Migrating My Site From Directus + Next.js To Astro

I have this website.

It’s for this Youtube channel. The only thing this site does is display videos that are on my Youtube channel and Courses which are essentially playlists.

A pretty simple website.

So I asked myself: “why the hell am I running Directus CMS on the backend, NextJS on the frontend, paying for Digital Ocean Droplet 17.69 dollars a month, and on top of that I have to maintain all that infrastructure myself?”

In this video I’m going to show you how I transformed this basically static site to an actual static site with Astro and Cloudflare. And reduced the cost to only 5 bucks a month.

The site how it is now

Lets take a quick look at how the site operates now.

On the frontend it features latest videos, when you click on a video it takes you to the video details page. Clicking on a tag takes you to the page that will display all the videos with that tag.

There are also Courses, those are just glorified playlists, and a way to categorise videos.

And I also have one page where I display all the videos, and one page where I display all the courses.

Very simple.

On the backed I’m using Directus, and it’s great. You know I have a lot of love for Directus on this channel. However as I mentioned in the intro, I think it’s an overkill. At least for me, if this was a site built for a client, I wouldn’t change it. Clients usually need a simple way to add content to the site, and Directus serves that purpose very well.

But since I’m building this for myself, I don’t mind editing Markdown files to add new content to the site and then deploying it online via Git to Cloudflare.

Steps needed to make the transfer

To transfer this site to a more simpler stack I needed a plan, a few things that needed to be resolved to make this work.

First, I needed to define a stack. These are my requirements:

It needs to be completely static
No server side rendering
No database connections
Ability to reuse code

So I decided to use Astro. It’s simple, it has the ability to create fully static sites, and it supports React components, so I can reuse components from the old site.

Next I needed to be able to transfer all of the data from the old site to the new site. In Astro you can edit and create your content using Markdown files. So I decided to go that route. More on that later.

I needed to decide where I will host this site. I heard a lot of people praising Cloudflare Pages, and how it’s practically free to host static sites there, I decided to check them out.

On the old site all media assets lived on the Digital Oceans Droplet in Directus. Which is cool, but since now I wouldn’t have a CMS, I would need a place to store the images.

For that I decided to use Cloudflare Images, which have a very generous plan for hosting thousands of images and what’s more important they also have image transformation features, which I needed so that I can serve optimized images, and this is very similar to what you get from Directus, because it also has image transformation features that you can use through frontend code as a URL parameters.

This is the only thing I’m currently paying for and it costs 5 dollars a month.

Transferring all the videos and courses

When the time came to transfer videos and courses I was afraid that it’s going to be a problem and take up a lot of time. But it was actually pretty simple.

Directus is a headless CMS that provides you with a REST API to get the data from the database and then you can display that data on your frontend, but you can also use that API to transfer data to the format you need. Like I’m doing right here:

import path from 'path'  
import fs from 'fs'  
  
import { createDirectus, readItems, rest } from '@directus/sdk'  
  
const client = createDirectus('http://watch-learn.com/backend').with(rest())  
  
async function readData(items, queryObject) {  
  return await client.request(  
    readItems(items, queryObject)  
  )}  
  
const videos = await readData('videos', {  
  fields: ['*.*', 'courses.courses_id.title,courses.courses_id.slug', 'tags.tags_id.slug'],  
  limit: 500,  
  sort: '-date_created'  
})  
  
videos.forEach((video) => {  
  const filePath = path.join('./src/content/videos', `${video.slug}.md`)  
  const content = `---  
id: "${video.id}"  
date_created: ${video.date_created}  
title: "${video.title}"  
featured_image: "${video.featured_image.filename_disk}"  
youtube: "${video.youtube}"  
introtext: "${video.introtext.replace(/"/g, '\'')}"  
episode_number: "${video.episode_number}"  
duration: "${video.duration}"  
download: "${video.download}"  
course_slug: "${video?.courses[0]?.courses_id.slug || 'uncategorized'}"  
course_title: "${video?.courses[0]?.courses_id.title || 'uncategorized'}"  
${video.tags !== null && video.tags.length > 0 ? 'tags:' : ''} ${video.tags.filter(item => item.tags_id?.slug).map(item => '- ' + item.tags_id.slug).join('\n')}  
---  
  
${video.body}  
`  
  fs.writeFileSync(filePath, content)  
})

This code is just using Directus SDK to get all the videos. And then for each video I define content that is going to be used to create a Markdown file and save it to ./src/content/videos which is a place where the content lives in Astro.

Most of this code is just defining frontmatter for each video, like: date_created , featured_image , id of the youtube video and so on.

Later on I can use that frontmatter to sort videos by date, get videos by tag and of course display the video on video details page.

Luckily the body of each video is already written in Markdown because Directus comes with a Markdown editor. So I didn’t have to do anything on that front.

When the transfer finished I ended up with a bunch of files that look like this:

---  
id: "1022"  
date_created: 2022-09-06T00:00:00  
title: "Let's Checkout... Astro"  
featured_image: "8c9ee5f2-ae29-4365-89e6-1ccafe7d32de.jpg"  
youtube: "m08eXKqkpKE"  
introtext: "In this video we are going to checkout Astro - all-in-one web framework for building fast, content-focused websites."  
episode_number: "21"  
duration: "16:10"  
download: ""  
course_slug: "frontend-tips"  
course_title: "Frontend Tips"  
tags:   
- javascript  
- static-site-generator  
- astro  
---  
  
Although Astro just recently got its first stable release, there has been a lot of talk about it in the last year.

...

Next I needed to transfer courses. To do this I used the same steps like for transferring videos, just the code was a bit different.

import path from 'path'  
import fs from 'fs'  
  
import { createDirectus, readItems, rest } from '@directus/sdk'  
  
const client = createDirectus('http://watch-learn.com/backend').with(rest())  
  
async function readData(items, queryObject) {  
  return await client.request(readItems(items, queryObject))  
}  
  
const videos = await readData('courses', {  
  fields: ['*.*', 'videos.videos_id.slug', 'videos.videos_id.title'],  
  limit: 500,  
  sort: '-date_created'  
})  
  
videos.forEach((course) => {  
  const filePath = path.join('./src/content/courses', `${course.slug}.md`)  
  const content = `---  
id: "${course.id}"  
date_created: ${course.date_created}  
title: "${course.title}"  
${course.description !== null ? 'description: ' + '"' + course.description.replace(/<p>|<\/p>/g, '') + '"' : ''}  
image: "${course.image.filename_disk}"  
videos: ${course.videos  
  .filter((item) => item.videos_id?.slug)  
  .map((item) => '- ' + item.videos_id.slug)  
  .join('\n')}  
---  
  
${course.body}  
`  
  fs.writeFileSync(filePath, content)  
})

When I ran that code, I got a bunch of files that look like this:

---  
id: "2"  
date_created: 2017-05-06T00:00:00  
title: "Frontend Tips"  
description: "In this series I will show you some tips and tricks that I find interesting and that can posibly help you in your frontend development."  
image: "5820d240d2b71799504397.jpg"  
videos:   
- frontend-tips-using-laravel-elixir-with-wordpress  
- introduction-css-grids  
- introduction-pattern-lab  
- smashing-magazine-netlify-whats-big-deal  
- css-grid-layouts-one-third-two-thirds  
- css-grid-layouts-grid-template-areas-mediumcom-example  
- css-grid-layouts-gridify-my-site  
- frontend-tips-organising-javascript  
- one-language-every-programmer-should-know  
- wp-rest-api-custom-endpoints  
- wp-rest-api-custom-post-types-and-fields  
- wp-rest-api-custom-filters  
- wp-rest-api-add-posts-frontend  
- lets-checkout-blitzjs  
- lets-checkout-prisma-nextjs  
- infinite-scroll-and-filters-react-query  
- lets-checkout-directus-9  
- let-s-checkout-wunder-graph  
- let-s-checkout-remix-and-compare-it-to-next-js  
- let-s-checkout-app-write  
- let-s-checkout-astro  
---

Courses Markdown files don’t have a body, but they do have the slugs of all the videos that are in that course in the frontmatter. Which will be very helpful when displaying Courses page, that needs to show all the videos that are on that page.

Although tags are their own entity on the old site, on this site I didn’t need to transfer them because I’m going to infer them from the videos. Since when creating a static site you need to generate every page that is going to exist, I can just go through all the videos with Astro and create all tags and their pages.

Transferring images

Most problems I had when transferring the data was with images. And that is because when you upload an image to Cloudflare Images they don’t keep their own filename but are instead renamed to an ID that Cloudflare Images assign to them.

This is a big problem for me because all my videos depend on those image filenames that I got from Directus.

Luckily I found a solution by reading the documentation and Cloudflare forums. So I created this shell script:

#!/bin/bash  
  
# Set your Bearer token

TOKEN="THE_TOKEN"  
  
# Iterate over all files in the current directory  
for file in *; do  
  # Skip directories  
  if [ -d "$file" ]; then  
    continue  
  fi  
  
  # Get the absolute path of the file  
  absolute_path=$(realpath "$file")  
  
  # POST the file  
  curl --request POST https://api.cloudflare.com/client/v4/accounts/CLIENT_ID/images/v1 \  
    --header "Authorization: Bearer $TOKEN" \  
    --form "file=@$absolute_path" \  
    --form "id=$file"  
done

This code uploads images using Cloudflare API, and more importantly it sets the ID of the image to it’s original filename. So now those images have IDs that I can use on the frontend.

Defining Collections

Astro uses collections for content. In my case I only have two collections: Videos and Courses.

To define collections I needed to create a config.ts file inside the content folder. And in that file I defined my collections. Like this:

import { defineCollection, reference, z } from 'astro:content'  
  
const videos = defineCollection({  
  type: 'content',  
  schema: z.object({  
    id: z.string(),  
    date_created: z.date(),  
    title: z.string(),  
    featured_image: z.string(),  
    youtube: z.string(),  
    introtext: z.string(),  
    episode_number: z.string(),  
    duration: z.string(),  
    download: z.string(),  
    course_slug: z.string(),  
    course_title: z.string(),  
    tags: z.array(z.string()).optional()  
  })  
})  
  
const courses = defineCollection({  
  type: 'content',  
  schema: z.object({  
    id: z.string(),  
    date_created: z.date(),  
    title: z.string(),  
    description: z.string().optional(),  
    image: z.string(),  
    videos: z.array(reference('videos'))  
  })  
})  
  
export const collections = {  
  videos: videos,  
  courses: courses  
}

This is just a simple config file where you define schemas of your collections so that Astro knows what to expect.

Dynamic Routes For The Content

Just like NextJS Astro uses page based routing, meaning it will create routes based on the structure of src/pages folder. This is where I added all my pages.

Of course Astro supports dynamic routes, so that I don’t have to create every page separately. I just create a dynamic route and then Astro takes care of the rest.

For example, since I wanted all my URLs to be like on the old site, so that I don’t loose SEO rating on Google, I needed to create dynamic route for my videos.

Directory structure looks like this: [category]/[slug].astro

[category] is going to be course slug, and [slug].astro is the slug of the video that I want to display.

I’m using angled brackets here because category and slug are dynamic parts of the route.

An then in [slug].astro I get the video that I wanna display like this:

---  
IMPORTS
  
export async function getStaticPaths() {  
  const allVideos = await getCollection('videos')  
  return allVideos.map((video) => {  
    return {  
      params: {  
        category: video.data.course_slug,  
        slug: video.slug  
      }  
    }
    })  
}  
  
const { category, slug } = Astro.params  
  
const imageUrl = 'https://imagedelivery.net/SECRET-ID/'  
const video = await getEntry('videos', slug)  
const { Content } = await video.render()  
  
const date = new Date(video.data.date_created)  
const dateFormatted = new Intl.DateTimeFormat('hr').format(date)  
  
const relatedVideos = await getCollection('videos', ({ data }) => {  
  return data.course_slug === category  
})  
---

CONTENT

The most important part here is actually getStaticPaths, in that exported function I go through all the Videos and define category param and slug param so that Astro knows which pages to create.

So now when I build the app, Astro generates dist folder with all Courses as folders, and then in those folders there is a folder for each video with index.html file in it. and that file is the actual video that you can see on the site.

One more interesting thing about dynamic routes is how it handles pagination. For pagination I needed to define special dynamic route called [page].astro. It’s very similar to normal dynamic route, except here we return paginate function from getStaticPaths

For example look at the all/[page].astro route. On this route I’m displaying all videos.

---  
IMPORTS  
  
export async function getStaticPaths({ paginate }) {  
  const paginatedVideos = await getCollection('videos')  
  const sortedVideos = sortAndLimit(paginatedVideos, 1000)  
  return paginate(sortedVideos, { pageSize: 18 })  
}  
  
const { page } = Astro.props  
---

CONTENT

Here I’m getting all the Videos, sorting them, and then passing them into paginate function. Also notice page variable that I’m getting from Astro.props

That variable contains all the information about pagination like start, end, currentPage, total number of pages and so on.

So I can pass that data to my pagination component to render pagination for pages that need it:

<PaginationMain 
	start={page.start} 
	end={page.end} 
	total={page.total} 
	currentPage={page.currentPage}  
    size={page.size} 
    lastPage={page.lastPage} 
    url={page.url} 
    prefix="/all" 
/>

Styling

For styling I’m using Tailwind, since that is what I was using on the old site. I also added shadcn/ui to my stack. But I’m only using three components from it. I’m usually not a fan of component frameworks, but since shadcn/ui gives me the ability to change the code of each component I decided to use it, mostly because of the nice styling and because it provides accessibility through Radix UI, which is nice to have for the critical components like buttons and pagination.

Hosting and Deploying

Hosting and deploying was the easiest part of the migration, because Cloudflare Pages make the process very smooth.

I just connected Cloudflare Pages to my Github repo, choose a Framework Preset, which was Astro, and clicked Save and Deploy button. And that was it. Now every time I want to update my site I just push the changes to the main branch of my Github repo, Cloudflare receives those changes and builds all those files you saw before, and my site is updated.

Editing

Of course editing and adding new content is a bit easier and straight forward in Directus, since it’s a full blown CMS. But doing everything through Markdown files is not so bad. I just need to setup the frontmatter, upload an image to Cloudflare Images and get the id for the video thumbnail, and then I can just write the content like I would in Directus, since there the body of the article is also Markdown.

Conclusion

Of course there were hiccups here and there, but those were mostly because I didn’t work with Astro on anything serious, I just gave it a quick glance when I first heard about it, but Astro has great documentation and community so I was able to fix problems that arose very quickly.

I would recommend Astro if you need a simple way to create static sites that have a lot of content. I didn’t fully test Astros SSR capabilities, I only connected it to Directus and that worked really nice, but later I decided to go full static generation so I didn’t spend to much time on that.

Migrating My Site From Directus + Next.js To Astro