Christopher's Blog

Fast Integration Tests with MySQL

Tue, 23 Mar 2021 00:00:00 +0000

We have a collection of nearly 700 integration tests (and growing!) in over 130 suites that we run against a live MySQL database. These tests are mostly performing GraphQL mutations, and the tests basically just assert that the mutation does what it’s meant to do.

As time goes on and we add more and more tests, the test suite was becoming slower and slower…

Resetting the DB between tests

To ensure tests aren’t too fragile, early on we added a beforeAll routine that reset the database between each suite.

About

Sat, 08 Aug 2020 00:00:00 +0000

I’m Christopher Nadeau. I’m the CTO over at Deskpro. I like writing and shipping software, and building businesses.

Connect

GitHub: @chroder
Twitter: @chroder
Email: chris at nadeau dot tv

Secure home server with Unraid and Cloudflare

Sat, 08 Aug 2020 00:00:00 +0000

I built a home server earlier this year to serve as a NAS and home media center. I’m trying to make a more concerted effort to take control over my own data and rely less on cloud services.

I chose Unraid as the underlying operating system. If you’re building a home server, I really recommend checking it out. Out of the options I tried, Unraid was by far the easiest to get up and running with.

Quickie: Color console output with parallel command

Thu, 18 Jul 2019 16:20:00 +0000

I’ve got a Lerna monorepos with a few microservices. I needed to run the dev version a few of these microservices so I decided to use parallel to run them all at the same time in my terminal:

parallel --tagstring '{/}' --linebuffer 'cd {} && npm run dev' ::: packages/service-a packages/service-b packages/service-c

This worked, but I noticed that the color output was stripped when doing this. This is really annoying. The color really helps during development to spot errors.

SPF, DKIM, DMARC: What they do and what they don't

Fri, 14 Jun 2019 15:54:31 +0000

Email is such a mess of technologies and standards, it’s hard to understand how everything works together. When it comes to SPF, DKIM, DMARC, there are some misconceptions.

The major misconception is around what SPF and DKIM actually do. Many people think that SPF and DKIM are somehow related to controlling who can send messages From your domain, but that’s not totally true.

The first thing to understand is that there are two email addresses in an email:

Configure a Hyper Key on OSX

Tue, 28 Aug 2018 12:00:00 +0000

If you’re anything like me, you like keyboard shortcuts. I like to set up custom shortcuts and macros to automate everything I can. The problem is that the most convenient keyboard combinations are already in use by the OS or by user applications.

That’s where the Hyper Key (also sometimes called a Super Key) comes in. Basically: Get rid of the useless caps-lock key and turn it into a new modifier key that isn’t used by anything else. For example, maybe you want HYPER+V to mean “Paste without formatting”.

Nginx proxy_pass DNS Cache

Fri, 31 Mar 2017 10:00:12 +0000

We use nginx to proxy to an Amazon S3 bucket to serve static content to customers. Yesterday, I noticed a high failure rate through the proxy but I couldn’t figure out why.

After some debugging, I finally discovered that the IP address nginx was hitting was different from the one that DNS was returning. It turns out that nginx resolves hostnames only once on load, meaning whatever IP address it got on load would stick around until reload.

pt-online-schema-change error "Error creating new table ... duplicate key"

Thu, 16 Jun 2016 16:11:04 +0000

pt-online-schema-change is a tool in the Percona Toolkit which allows you to make large table alters without locking the database.

Newer versions of MySQL have Online DDL which reduce the need for this tool. But if you’re like us, pt-online-schema-change is still very valuable because we use Galera. Running a DDL query on a Galera cluster blocks the entire cluster (even other databases in the same cluster).

I’ve successfully used pt-online-schema-change in the past with great success. This week I tried to use it again to perform some expensive queries and I ran into an error message like this:

Quickie: Prevent Mac apps from quitting when last window is closed

Thu, 03 Mar 2016 10:29:18 +0000

There are a few apps on OSX that quit/terminate once their window is closed. This is typical on Windows of course, but it’s a bit random on OSX. Some (most?) apps stay open, and some close. I find it really bloody annoying. I’ll quit my apps when I’m ready to quit them, thank you very much.

The biggest sore spot with me is 1password which I use dozens of times a day, and every time I CMD+W to close the window (muscle memory!), it quits the app. Re-opening it takes about 2 seconds. It doesn’t seem like much, but it’s annoying when I use it so often.

Quickie: Quickly rename MySQL database

Wed, 02 Mar 2016 17:42:43 +0000

There’s no way to rename a MySQL database. Typical advice is to dump/restore the database, but this is obviously very slow if you have a huge database.

I found a solution today. Create your database, then execute this one-line:

mysql -uUSER -pPASS OLD_DB -sNe 'SHOW TABLES' | while read t; do mysql -u username -ppassword -sNe "rename table OLD_DB.$t to NEW_DB.$t"; done

Obviously replace the params as necessary.

Or here’s the same as a slightly easier to read bash script:

SSH with a Bastion Host

Sat, 27 Feb 2016 13:33:16 +0000

A bastion host is a server that sits on a public network whose sole purpose is to provide access to an inner private network.

For example, if you use AWS and have instances on a private VPC subnet, then the only way you can gain SSH access to them is to use a bastion host as a kind of proxy.

You > Bastion > Server

For the best security, you typically have SSH configured for key-only authentication. This presents a problem: Your bastion server needs the keys to any internal server. Having a single bastion server with the keys to the castle is a pretty big risk.

AWS VPC Network Structure

Sat, 27 Feb 2016 13:32:41 +0000

I recently started playing around with AWS. One of the really cool things about AWS is the Virtual Private Cloud (VPC) features. By creating a VPC, you create a private network for yourself complete with NAT gateways, custom route tables, firewalls and more.

VPC Components

A VPC is made up of several main components:

CIDR Block

When you create your VPC, you are asked to define a CIDR block, which is the range of IP addresses you want your VPC to use. For example, 10.0.0.0/16 means you can use 10.0.0.0 - 10.0.255.255 (65,534 addresses).

Setting up a new MySQL slave database via steaming xtrabackup

Wed, 27 Jan 2016 18:16:45 +0000

The goal today is to build a new MySQL slave database off an existing database. We’ll use Percona’s xtrabackup to stream a hot backup over the wire to a new machine, while keeping the existing server online.

Prep

Install MySQL on New Server

Install MySQL in the way you normally would (e.g., apt-get etc).

The only very important thing you need to do is edit my.cnf and set a server-id for the server. This can be any integer. I tend to use the current date and time just because it’s generally pretty unique (like 201601271730).

Using NgModelController with Custom Directives

Sat, 01 Feb 2014 15:38:00 +0000

Creating directives with AngularJS is fairly straightforward. But most directives also need to interact with a model which represents their state. You could bake in your own custom model handling, but you can also plug right in to AngularJS’s own NgModelController – the same ng-model that is used for things like input boxes and select menus.

Example directive: `<time-duration />`

As a simple example, let’s build a directive where the user can input a duration using one of many possible units of time.