Build a Modern Static Website using NPM & Shell Scripts

Wait, no framework?

Open Sky has worked with many static website development frameworks and technologies over our 24 year history. These include content management systems like wordpress, drupal, and joomla... as well as component front-ends like bootstrap, foundation, and materialize. And while they have their place for certain circumstances, we generally find them constraining, painful, and more expensive than pursuing custom builds. Developers just end-up banging their head against them trying to get them to do what they want. The problem is fundamentally these are not coding systems, but wizarding systems. Moreover they incur significant technical debt over time. For these reasons, coupled with the advent of html5, css3, flexbox, and linting tools - we find these frameworks unnecessary. In this article we'll review our flexible way to build a modern static website using npm & shell scripts.

Our Requirements

Open Sky Software's website (the one you're reading) is in it's third version. Released in June of 2020, it's the culmination of our development team sitting down one day and cherry-picking the best techniques from our custom web application development process and trimming it back to a simple, lean, flexible build framework for a modern static website. Our requirements included:

• Automated build & watch process
• HTML fragments with parameter passing
• Lean, semantic HTML5 with linting
• CSS/JS compiling & linting
• Image optimization
• Cache busting
• Production minification & optimization
• Sitemap generation
• Easy deployment process
• Simple server side rendering (Apache only)

Wait again, hasn't this been done?

Indeed it has... just not the way we like it. And per Open Sky's Golden Rules one should not be afraid to reinvent the wheel if your wheel works better. There are indeed great lightweight solutions out there like staticboilerplate.com - and we almost went with them. We just wanted something even more lightweight, plus a couple bells and whistles of our own. All that said, it was just too simple and fun not to roll our own anyway!

The Recipe

The site build process is all command-line driven using the npm-run-script (aka, npm run). The build commands compile the site from our /src directory into our /dist directory, and once there - they are further operated on with shell scripts to optimize images and cache-bust resources. We use an IDE for editing the source files, configs, and scripts. The following is the general recipe behind of the process.

Step 1
Install the pre-requisites for your operating system such as node.‍js and webp (for image conversion & compression). You won't need Apache locally as for development we just open the html files directly from our /dist directory in a browser.

Step 2
Setup your source code folder the way you like. We use a pretty basic tree with directories for things like /fonts, /html, /images, /scripts, /styles, etc.

Step 3
Add these dependencies to your node package json file:

• less for css less features
• less-plugin-clean-css for css compression
• less-plugin-autoprefix for browser-specific prefixes
• lesshint for linting your css
• jshint for linting your javascript
• html-includes for fragments with parameter passing
• htmllint-cli for linting your html
• sitemap-generator-cli for generating your sitemap.xml
• nodemon to watch changes and hot swap'em
• concurrently to run multiple npm commands
• uglifyjs-folder to minimize your javascript
• rimraf for deleting and cleaning-up

Step 4
Create a variety of build commands within the scripts tag in your package json file for individual tasks, and bundle them together to create comprehensive build commands. Here's a few examples:

"scripts": {
...
	"clean:fonts": "npm run rimraf dist/fonts/*",
	"build:fonts": "npm run clean:fonts && cp -R src/fonts/* ./dist/fonts/",
	"watch:fonts": "nodemon -q -w src/fonts/ --ext ttf --exec \"npm run build:fonts\"",
...

"scripts": {
...
	"clean:html": "npm run rimraf dist/**/*.htm*",
	"build:html": "npm run clean:html -s && npm run lint:html && html-includes --minify --src src/html/ --dest dist/",
	"watch:html": "nodemon -q -w src/html/ --ext html --exec \" npm run build:html\"",
...

"scripts": {
...
	"dev": "concurrently -k \"npm run watch:html -s\" \"npm run watch:fonts -s\"",
...
}

With the above laid out properly you can run command line commands such as 'npm run dev' and node will go off and clean your directories, build your fonts, css and html files (including linting, packing your html-includes, and minifying your html), and set a watch on them so you can change the source files in real-time and those changes get hot-swaped back into your /dist directory with each and every save. It's highly efficient. Iterate on the above to add your css, javascript, images, etc.

Step 5
Add some shell scripts to the mix which you can call from npm, run by hand, and integrate into your production deployment scripts. Here's one that finds all the lossless png source images and converts them to lossy, compressed webp images, and does it in parallel to speed up processing time:

echo "Convert and compress images to webp..."

find dist/images/ -type f -name '*.‍png' | parallel -eta cwebp -quiet -q 60 {} -o {.}.‍webp

And here's one that greps the html files for any instances of webp images and uses sed to add a ?version=datetimestamp to the end of them for cache busting:

echo "Cache busting images"

grep -il --exclude-dir={js,fonts,images,css} "\.‍webp" dist/**/*.htm | xargs sed -i "s/\.‍webp/\.‍webp?version=$(date +%s)/g"

Summary

By using a mix of npm & shell scripts we're able to automate our static website build in a slick fashion. And it's all very flexible and easy to change, maintain, and update. The most powerful piece (and yet the weakest link) in our build is the html-include npm module. It's incredibly powerful becuase it allows you to create html fragments and inject custom parameters (text) into those fragments. This lends itself to very rich SEO optimizations within the site. At the same time, it's one library that you quickly grow dependent upon so replacement could prove sticky. Only time will tell. For now we're very pleased with the results.