A common way to use AWS CloudFront is to use it as the front end for an Application Load Balancer. You would normally set it up as so:

CloudFront is doing the perimeter work, including caching and WAF, which it then passes to the Origin - the ALB - which distributes it to the back end, in this case, a set of containers.

However, there is nothing stopping someone scanning the AWS IP space, connecting to every IP on port 80 and 443 to see what they can find. If they find your server, they would have bypassed the WAF, circumventing SQL Injection and Cross Site Scripting protection amongst other features. This is usually something you want to avoid.

We recently implemented one of the techniques below, with logging, and it was surprising how much of this happens in the “background noise” of operating a service on the internet. The best way to stop someone breaking in is to not let them get to the front door in the first place, so let's try to stop that.

Option 0: If you like it put a hostname on it

The ALB rules allow you to specify a hostname, so you could set that to your domain name, and blackhole anything which doesn’t match. This covers about 80% of cases, as all they have is the IP and a port, but doesn’t stop someone with some basic knowledge of your infrastructure from bypassing the WAF. This is really a good idea regardless.

locals {
  environment = "test"
  
  domains = [
    "foo.com",
    "bar.com,
  ]
  
  route_priority = 10
  vpc_id         = "vpc-123456"
}

resource "aws_lb" "alb" {
  name = "${local.environment}-alb"

  ...
  
  load_balancer_type = "application"
  
  ...
}

resource "aws_lb_listener" "https_listener" {
  load_balancer_arn = aws_lb.alb.arn
  port              = "443"
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-2015-05"

  certificate_arn = var.certificate_arn

  # return a 403 by default
  default_action {
    type = "fixed-response"

    fixed_response {
      content_type = "text/plain"
      status_code  = "403"
    }
  }
}

resource "aws_lb_target_group" "target_group" {
  name     = "${local.environment}-service-name-tg"
  protocol = "HTTPS"
  port     = 4430

  health_check {
    ...
  }

  vpc_id = local.vpc_id
}

resource "aws_lb_listener_rule" "application_rule" {
  count        = length(local.domains)
  listener_arn = aws_lb_listener.https_listener.arn
  priority     = local.route_priority + count.index

  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.target_group.arn
  }

  #you can only have one host header per rule, so make multiple rules?
  condition {
    field  = "host-header"
    values = [element(local.domains, count.index)]
  }
}

Option 1: CloudFront IP list

The official AWS way to do this is, of course, with a Lambda.

AWS publish a list of CIDR ranges for their services - EC2, CloudFront and others - and they send an SNS message when this list is updated. You can trigger a lambda from that SNS message, and update the Security Group on the ALB to only allow traffic from those ranges.

This works, however it has a couple of problems.

• If the SNS message is dropped, you may have new or changed IP ranges which are sending you traffic, but you’re not allowing traffic from. AWS add new CloudFront POP’s on a regular basis, so this list is changing fairly often.
• The list is fairly long, and a security group can only have 50 entries, so you have to manage splitting it over a number of security groups.

AWS have handled the second one in their sample lambda, but this has a number of moving parts which would be difficult to diagnose in the event of a problem.

I’d also recommend you add your own IP’s to this list, in case you need to bypass CloudFront for some reason, or put them in another security group.

Option 2: Injected header and WAF

CloudFront also has the option to inject a header with a fixed value before calling the origin. This header can be anything - prefixing it with x- is recommended - and the value can be anything. This makes it easy, for example, to inject a header with

x-my-token: a1b2c3d4

You can then pick that up on the ALB side with a valid WAF which blocks all traffic, allowing only traffic with that header.

You do need to plan to keep the header field in sync, but as it’s not a real secret, you can put it into Terraform, or manually switch them out to rotate it.

Note that the value may end up in your logs, if you are logging headers. This can be more a volume/size issue than anything - I’ve had to avoid calling it x-cloudfront-token: 7c9c2d3a-8033-4295-9b28-af8b0683353c , which makes sense as it’s readable and a UUID is fairly unique, but it also adds 60-odd bytes (or 120 if everything is UTF-16 encoded) per request. It’s not massive, but it’s redundant and unnecessary.

I’d strongly suggest you set this up by hand - add the header in CloudFront, verify that it’s being sent correctly, then connect the WAF, possibly in COUNT mode rather than BLOCK, until you’re confident that it’s working.

locals {
  environment        = "test"
  origin_domain_name = "loadbalancer_origin.foo.com"
  
  cloudfront_header_name = "x-cloudfront-token"
  cloudfront_token       = "7c9c2d3a-8033-4295-9b28-af8b0683353c"
}

resource "aws_cloudfront_distribution" "cloudfront" {
  
  ...
  
  enabled = true
  
  ...
  
  origin {
    domain_name = local.origin_domain_name
    
    
    ...
    
    custom_header {
      name  = local.cloudfront_header_name
      value = local.cloudfront_token
    }
  }
  ...
}

resource "aws_alb" "alb" {
  ...
}

resource "aws_wafregional_web_acl" "alb_waf" {
  metric_name = "albwaf"
  name        = "${local.environment}-web-alb-waf"

  default_action {
    type = "BLOCK"
  }

  rule {
    priority = 100
    rule_id  = aws_wafregional_rule.header_match_rule.id

    action {
      type = "ALLOW"
    }
  }
}

# bind it to the ALB
resource "aws_wafregional_web_acl_association" "web_waf_binding" {
  resource_arn = aws_alb.web_alb.arn
  web_acl_id   = aws_wafregional_web_acl.web_alb_waf.id
}

resource "aws_wafregional_rule" "header_match_rule" {
  depends_on  = ["aws_wafregional_byte_match_set.header_match_set"]
  name        = "${local.environment}_web_alb_header_match_rule"
  metric_name = "albheadermatchrule"

  predicate {
    negated = false
    type    = "ByteMatch"
    data_id = aws_wafregional_byte_match_set.header_match_set.id
  }
}


resource "aws_wafregional_byte_match_set" "header_match_set" {
  name  = "${local.environment}-web-alb_header_match_set"

  byte_match_tuples {
    text_transformation   = "NONE"
    target_string         = local.cloudfront_token
    positional_constraint = "EXACTLY"

    field_to_match {
      type = "HEADER"
      data = local.cloudfront_header_name
    }
  }
}

If you need to bypass this with CURL, you can use the header option to set the value

curl -k -H 'x-cloudfront-token: 7c9c2d3a-8033-4295-9b28-af8b0683353c' https://origin.server.com/foo/bar

This is the option we went with, and it's been working rather well. Disabling it if needed is a matter of just disconnecting the WAF-ALB association.

Option 3: Injected header and ALB rules

The last option combines a few of the above options. It’s a little less flexible, if you are using ALB routing rules for other things, but it’s a lot cheaper (WAF costs per request, tho not a lot) and it has one fewer moving part - no WAF involved.

This involves setting the same header in CloudFront as in Option 2, but checking it in the Target Group routing rule, so you are matching on both the host name and the presence of the header value.

You need to have a fall thru for this - we’d normally just return 403 or 444 if nothing matches - but it’s fairly simple to implement and reason about.

You would setup CloudFront in the same way as Option 2. The ALB no longer needs a WAF, you'd just add more conditions to your listener rule

At present, Terraform doesn't support this - they only have host-header and path-pattern - but I'd assume that it's coming (or I need to write some Go code)

There are a couple of ways you can stop people bypassing your CloudFront protections, all of which are easy to implement and reason about, and very robust in production.

They will keep your back-end more secure, and make sure that all visitors to your sites pass thru the correct checks and filters.

AWS CloudFront is an extremely powerful service, which gives you a global Content Delivery Network (CDN) with over 100 points of presence, as well as robust DDOS protection and mitigation, edge caching, TLS termination, HTTP to HTTPS redirection, content streaming, and routing rules. Even with caching turned off, this is a service that you want to be fronting your website.

While it has a few flaws, like the time it takes to roll out a change, overall it's a solid, reliable and essential part of hosting a modern website. It can also be extended with Lambda@Edge, which allows you to run a lambda function on every request, which can modify parts of the result before sending it to the user.

One thing that CloudFront is missing, that a lot of people need, is IP whitelisting. This is useful if you want to lock your site down to a specific set of IP addresses - eg before a site launches - or in reverse, and more commonly, block a range of IPs from accessing your site.

Normally you could use a security group to handle the whitelisting - don’t allow from 0.0.0.0/0, just put in the list of CIDRs you want to allow and you’re done. CloudFront, however, doesn’t support security groups, so this option isn’t available.

Enter the AWS Web Application Firewall - WAF.

A WAF is normally used to inspect traffic for attacks like SQL Injection or Cross Site Scripting (XSS), and block them. Like Marmite, WAFs are universally loved (by PCI) or hated (by pretty much every developer and administrator who's had one forced on them). They are, however, very useful when done well.

As well as SQLi and XSS, the AWS WAF also has

• IP matching - a rule which matches if the source address is on a list of addresses
• String and byte matching in various fields, using lists, partial matches and regular expressions
• Custom WAF rules from 3rd party providers like Fortinet or AlertLogic
• AWS Shield, which provides DDOS mitigation (not proactive - that’s Shield Advanced, which is rather pricey, unless you need it, then it’s exceptionally cheap)

The WAF consists of a number of pieces

• The WAF ACL (Access Control List): a data structure which defines which rules your WAF contains, how it reports metrics and other configuration information. This is the root of the WAF structure and the entity that you attach to things which want to use the WAF.
• A WAF ACL has one or more WAF Rules attached to it. If a rule matches, its action is applied (block, allow, count) and processing stops. Multiple rules in an ACL are applied in an OR manner. If no rules match, the default action applies.
• A rule is made up of a list of predicates which are applied in an AND manner - for the rule to fire, you have to have (eg) an ip in the IP list predicate and also a specific string in a header field (string match predicate).
  
  Predicates can also be negated (match if it’s not on the list). Each predicate has a matching type -  Byte, Geographic, IP, Regex, Size constraint, SQL Injection and Cross Site Scripting.
• Each WAF Rule has one or more WAF Sets attached, eg a IPSet, StringSet, GeoMatchSet. This is a list of items which make up the predicate, and could be a list of IP addresses, a set of strings and fields to match, where to look for XSS and SQLi, areas of the world for a geographic match, or other configuration.
• AWS WAF also supports Managed Rules, which can be bought in the AWS Marketplace. These do not have sets, and you can just include them, rather than providing any configuration.

So a WAF ACL looks something like:

• If the IP is in the list, ALLOW (Rule, priority 1)
• If the string is not in the list, BLOCK (Rule, priority 2)
• If nothing above matched, COUNT (default action)

If the user is blocked, they will receive a 403 error from CloudFront, which you can customize. This is different to a security group rule on an ALB, which will just ignore traffic that doesn't match.

Using CloudFront and WAF to pinhole a service

You can use CloudFront and WAF to easily pinhole a service to your IP address. You need to setup the following in Terraform

locals {
  # the name of the ACL, can have _ etc in it
  acl_id = "foo"

  # the name of the CloudWatch metric - must be a-z only
  metric_name = "foo"
  
  # the list of IPs we want to whitelist
  cidr_whitelist = [
  	"202.14.100.0/24",
    "8.8.8.8/32",
    "1.1.1.1/32",
  ]
}

resource "aws_waf_web_acl" "waf_acl" {
  name        = "${local.acl_id}_waf_acl"
  metric_name = "${local.metric_name}wafacl"

  default_action {
    type = "BLOCK"
  }

  rules {
    priority = 10
    rule_id  = aws_waf_rule.ip_whitelist.id

    action {
      type = "ALLOW"
    }
  }

  depends_on = [
    "aws_waf_rule.ip_whitelist",
    "aws_waf_ipset.ip_whitelist"
  ]
}

resource "aws_waf_rule" "ip_whitelist" {

  name        = "${local. acl_id}_ip_whitelist_rule"
  metric_name = "${local.metric_name}ipwhitelist"
  
  depends_on = ["aws_waf_ipset.ip_whitelist"]
  
  predicates {
    data_id = aws_waf_ipset.ip_whitelist.id
    negated = false
    type    = "IPMatch"
  }

}

resource "aws_waf_ipset" "ip_whitelist" {
  name = "${local. acl_id}_match_ip_whitelist"
  
  # dynamic below generates this from the list
  #
  # ip_set_descriptors {
  #   type = "IPV4"
  #   value = "8.8.8.8/32"
  # }

  dynamic "ip_set_descriptors" {
    for_each = toset(local.cidr_whitelist)

    content {
      type  = "IPV4"
      value = ip_set_descriptors.key
    }
  }
}

Once you have that, you can then connect it to the CloudFront distribution

resource "aws_cloudfront_distribution" "cloudfront" {
  ...
  web_acl_id = aws_waf_web_acl.waf_acl.id
  ...
}

If you have more than one CloudFront distribution you want to use for this, you can attach the same ACL to multiple distributions, which does drop the cost a bit ($1/month). However, the management can be easier if it's one ACL per distribution, depending on how you structure your Terraform code.

You can also do this to an ALB or API Gateway, but you need to use the aws_regional_waf_acl structure (and other regional resources). The concept is identical.

While the black box nature of the SQLi and XSS rules can be frustrating at times, the other features of the AWS WAF are very useful, and can help you secure your site from unwanted attention.

It’s becoming more common to split build and deployment infrastructure into distinct entities, and the account boundary in AWS is a good line to split things.

Some of the tools don’t handle this that well, tho. CodePipeline is definitely one of them.

AWS have documentation on how to setup CodePipeline using remote resources, but I found the document to have too many coded steps (which makes it specific to their situation) and lacking in general concepts. I’m going to try to address the latter.

Basic structure

This assumes you are building and deploying the following way:

• You have 2 accounts (could apply to more, but at minimum 2). One is for build / test and the other is for deployment / execution. I’m calling these build and test in here.
• You are storing code in Git somewhere - Github in this case; building it in CodeBuild; and deploying it (in test) to a basic ECS Cluster and service. No Blue / Green, just redeploy the service with the new container and let ECS handle the deployment.
• CodePipeline is orchestrating things for you

Environment Setup

CodePipeline needs a few things to get going, and as far as I can tell, they are difficult to do in the console, so I ended up using Terraform for it. I’m sure you could do them in the console and CLI, as the document above shows, I just found it easier to do it in Terraform. Plus, infrastructure as code. It's a good thing.

First, it needs somewhere to store the artefacts. This is a normal S3 bucket in the build account. The test account will need access to this, so you need to make sure that the bucket policy includes the root user in test having access to the bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::TEST_ACCOUNT_ID:root"
                ]
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObjectVersion",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:GetBucketAcl"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME/*",
                "arn:aws:s3:::BUCKET_NAME"
            ]
        },

Next, you will need to make a custom KMS key and alias. Again, the root user of test needs access to this as a user, as does the role that CodeBuild and CodePipeline and running as. It doesn’t matter who can administrate it.

{
    "Sid": "",
    "Effect": "Allow",
    "Principal": {
        "AWS": [
            "arn:aws:iam::TEST_ACCOUNT_ID:root",
            "arn:aws:iam::BUILD_ACCOUNT_ID:role/CODE_PIPELINE_ROLE",
            "arn:aws:iam::BUILD_ACCOUNT_ID:role/CODE_BUILD_ROLE"
        ]
    },
    "Action": [
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:Encrypt",
        "kms:DescribeKey",
        "kms:Decrypt"
    ],
    "Resource": "*"
},
{
    "Sid": "",
    "Effect": "Allow",
    "Principal": {
        "AWS": [
            "arn:aws:iam::TEST_ACCOUNT_ID:root",
            "arn:aws:iam::BUILD_ACCOUNT_ID:role/CODE_PIPELINE_ROLE",
            "arn:aws:iam::BUILD_ACCOUNT_ID:role/CODE_BUILD_ROLE"
        ]
    },
    "Action": [
        "kms:RevokeGrant",
        "kms:ListGrants",
        "kms:CreateGrant"
    ],
    "Resource": "*",
    "Condition": {
        "Bool": {
            "kms:GrantIsForAWSResource": "true"
        }
    }
}

You can also set it as the default key to use for encryption in the S3 bucket, tho that’s optional. CodePipeline and CodeBuild will upload with an encryption key regardless of the bucket setup.

Finally, you’ll need an ECR repository to store the container, and that will need to allow the test account root user to pull from it.

Build Setup

You will need a CodeBuild project which will do the container build, and push it to ECR.

This has CODEPIPELINE as the input and output artefact types, and it also must include a reference to the KMS key you created, otherwise CodeBuild will use the default aws/s3 key when it uploads artefacts, otherwise the test-deployment role in test will not be able to get to the artefacts. You can't share the default key between accounts.

Your build project also needs to output a file called imagedefinitions.json. The end of it is likely to look like this

version: 0.2
phases:
  ...
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker push $IMAGE_REPO_NAME:$IMAGE_TAG
      - echo finished
      - printf '[{"name":"%s","imageUri":"%s"}]' "app" "$IMAGE_REPO_NAME:$IMAGE_TAG" | tee imagedefinitions.json
artifacts:
  files: imagedefinitions.json

This contains the name of the container in the Task Definition (app in this case) and the URL of the new image in ECR (eg 123456789012.dkr.ecr.us-west-2.amazonaws.com/my_thing:latest)

Remote Account Setup

In the remote account, you will need a role (test-deployment) which can be assumed from the build account, and can manipulate ECS, EC2, and load balancing. It’ll also need access to the S3 bucket and the KMS key (mostly for decryption).

Pipeline setup

You can now setup your CodePipeline. You will need to use the bucket you made above, and be sure to set the encryption key for use there.

The rest of the pipeline is pretty much as-is, except the deploy step - this also needs the role setting, to be the test-deployment role you made in Remote Account Setup. Your CodePipeline role will need to be able to assume that role.

// Trust relationship in test-deployment
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::BUILD_ACCOUNT_ID:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

// assume role section in the codepipeline role
{
    "Sid": "",
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": [
        "arn:aws:iam::TEST_ACCOUNT_ID:role/*"
    ]
},
{
    "Sid": "",
    "Effect": "Allow",
    "Action": "iam:PassRole",
    "Resource": "*"
}

You may want to further limit which roles the CodePipeline role can assume (not role/*) and which roles can be passed.

Execution

The pipeline executes like this:

• CodePipeline is watching the Git repo (or gets a Webhook), gets the source down, zips it and puts it into S3, encrypted with the KMS key.
• CodeBuild is started, and pointed at the artefacts. CodeBuild gets the artefacts from S3, runs the build, and puts the output artefacts into S3, again using the KMS key.
• CodePipeline assumes the provided role into the test account, and makes a new Task Definition with the imageUri from imagedefinitions.json, and then deploys that into ECS.
• ECS handles the load balancer, adding the new containers in and removing the old ones once the new ones are healthy.

Things that tripped me up

• The KMS key in CodeBuild. I had assumed that CodePipeline, which had been told about the key, received the artefacts and put them into S3, but it’s CodeBuild which does that, so it needs to be aware of which KMS key to use. It defaults to aws/s3 which can't be used outside of the current account.
• The role which is being assumed in test needs to not just deploy a service, but also manipulate the other services that you need to get a container out - elastic load balancing, EC2, and anything else you need.

Once setup, this works nicely and could be easily extended to have further steps to wait for intervention, then promote to prod after it’s been on test and is accepted.

I don’t believe you can do much of this in the console - you have to use something like terraform (fairly easy) or the CLI (harder, but not impossible). CodePipeline has been described to me as a set of Lego - all the bits are there, you just have to do a LOAD of work to wire it up, especially if you want to do things which are outside of the flows that the setup wizard does. Even editing the pipeline after it's created hides a lot of the configuration - I think AWS could do a lot of work here, even if it’s just exposing the json behind the pipeline, as they do with IAM (json vs the visual editor).

I've been using Bear for writing and note taking for a while. It's a Markdown editor, but it also syncs nicely (iCloud) between my Macs, iPhone and iPad. It works great for writing blog posts, too.

From the Bear blog

This got us thinking: what if we could make it a little easier for Bear users to publish their writing? Once again, our Marketing & Research Division sprung into action to find where all the hip kids publish essays online.

With today’s update to WordPress for iOS, you can now publish Bear notes to WordPress blogs. It’s really simple to use:
First, make sure you update Bear and WordPress
In Bear, tap the share sheet at the top right of a note
Tap WordPress in the top row of options (learn how to enable app extensions on iOS)
The WordPress app will open. If you have multiple blogs, a screen will appear so you can pick one as your post’s destination
A draft screen will appear with your post filled in, complete with proper formatting of all headings, links, text styles, lists, and even photos

In theory, I should be able to write a small app which just logs into Ghost, pulls down the posts, allows them to be exported as TextBundle files (into Bear) or allows them to be shared to as TextBundles. They even have source code for it.

This would make it SO much easier, I think. Much better than trying to recreate the full editing environment, and keep it up to date with Ghost.

Side project, found.

Getting there

As is often the case, getting there is half the fun. Or half the un-fun in this case.

Before we even left - about 4 weeks before - a huge storm went thru the west coast, dumping between 600 and 1000mm of rain in 24 hours. This took out one of the bridges on the main road - the only road - just outside of Franz Joseph. We managed to book around this, but that would have cut off about 30% of the trip, and added about 6 hours more driving to an already driving-heavy trip.

Thankfully, the army and other engineers rebuilt a bailey bridge in about 2 weeks, so the week before we were meant to leave, it was all back up and running, so we were back to our original plans and bookings.

We’d opted to rebook our flights to get down 5 hours earlier, as our original plans involved staying in Christchurch the first night. However, on the day, AirNZ was having none of this, and while we got to the airport for our 10am flight, we didn’t manage to leave until 1pm. Still, 2 extra hours was great, as we got thru Arthurs Pass in daylight, and to our first nights accomodation just after dark.

Once we were in Christchurch, we remembered that it was Good Friday, so all but essential shops were shut. With no groceries for the first day, we were able to get some takeaways and gas and get on our way, feeling only a bit sheepish.

Not off to a good start, but it could have been a lot worse. We could have been “not on holiday”.

Unwinding at the Woodpecker Hut

Now, to say that the two of us needed a break is an understatement. Burnt out and exhausted would be closer to the truth. Our first stop was the Woodpecker Hut near Punakaiki. We’d intentionally left the first full day empty of things to do, so after waking up (and totally forgetting to bring coffee), we headed into Greymouth, which is about a 40min drive south, to get supplies.

Stocked up on wine, coffee, chocolate and chips, we headed back to fire up the outdoor, wood-fired hot tub and relax with the local Te Chickens (Weka)

On the second day, after a quick morning soak in the tub, we headed north to Westport, via Cape Foulwind, inland to Inagahua, with a stop at the old ghost town at Waiuta, then up to Arthurs Pass for a proper look around.

Arthurs Pass really turned on the weather - or rather, turned on the rain. By the time we got up there, it was chucking it down and misty, which set the tone quite nicely. The Kea, however, didn’t mind, and we had a nice, if fairly short and wet, walk up to the Devils Punchbowl in the rain.

Sadly, our time at Woodpecker Hut was done, and we fired up the hot tub one last time and enjoyed the roar of the southern ocean.

Okarito

Our next stop was a couple of hours south in Okarito, which is an old port town and large wetland, and home to, amongst other things, Rowi Kiwi, which are a rare subspecies of Kiwi. There are only about 600 of them left, and they only exist within the Okarito area. It’s also home to Kotuku (Great White Heron), and a lot of others.

We arrived in Okarito right on sunset, and it was spectacular.

The next day, we headed south to Lake Matheson, and down as far as Bruce Bay, which had a great ice cream shack right on the beach.

On the last day, we headed out on a boat trip into the Okarito Lagoon. It’s normally a bird watching trip (I think we dropped the average age down by about 20 years), but it wasn’t the right time of year for most of the birds. Still, we saw a few Kotuku, some of the other wetland dwellers, and when we were parked up having morning tea, a group of 4 Kea landed in the trees above us and proceeded to eat the berries, calling and laughing to each other. Magic.

From there we headed south again....

Haast and Makarora

Next proper stop was Haast, and lunch at Hard Antler in Haast, which is as hard case and kiwi as you might expect. We drove out to Jackson Bay, which is as far south as you can go on the west coast without a proper four-wheel drive. It very much feels like the end of the road there, with a couple of houses and an outpost of the local crayfish company. We did find a pod of dolphins (hectors we think, but not sure) swimming and hunting around the rocks on the way out, and Leonie got a couple of good photos of them playing.

From Haast, we went over the Haast Pass to Makarora. The Haast Pass is quite low compared to Arthurs - you don’t need to climb much to get over it. There are some spectacular views up there tho, and the river it follows is a beautiful light blue.

Tekapo

Makarora is a fair way up the river valley at the top of Wanaka, so it was quite cold when we woke up. We headed off to the Blue Pools, which would be lovely for a swim on a warmer day, then down to Wanaka for an amazing Burrito from Burrito Craft and some gas, and headed over the Lindis Pass to Tekapo.

This area reminds me a lot of the Scottish Highlands, except the hills are a bit taller, to the agreement of a travelling Scot I was talking to on the top of the pass. Like the Highlands, it’s stunning country to drive thru.

We ended the day at Tekapo, which is a Dark Sky Preserve.

The following morning, we headed up to Mt Cook, which is about an hours drive from Tekapo. As we have had the whole trip, the weather has been perfect, usually clear blue skies, and today was no exception. Mt Cook was out and cloud-free, and the view from Now-lake-formally-glacier Tasman, down the valley, was stunning.

Lake Tasman is now a lake, with a glacier at it's head, about 5 or 6km long. In 1990, the whole area was a fully formed glacier. It's receeding at about 0.5-1km/year at the moment. The othor major glaciers - Fox, Franz Joseph - are receeding at a similar rate, and you can no longer drive or walk to them.

These will have gone from "normal" to gone in my lifetime.

Just let that sink in.

Thats pretty much the end of our holiday. I'm writing this in Tekapo, and we fly back to Auckland tomorrow, and work on Monday. It's been a wonderful, and well-earned, break, and like all holidays, isn't nearly long enough. We could have spent the whole week up around Woodpecker Hut, or the whole week around Okarito or Haast. We'll just have to come back.

Observations

If you think that cellphone cameras are not valid, primary-use cameras, you are just wrong. I saw a mix of large screen phones (maybe 95%), DSLRs (4%) and a few compact or micro 4/3's. Phone photography is what people are using. They are the current point and shoot.

The number of people doing things which have no purpose except Instagram or YouTube is disturbing and disappointing.

So many people getting to places which have amazing natural beauty, then seemingly ignoring it, after taking a couple of pictures, and just huddling around discussing which picture they should add to their Instagram Story. Then leaving. Or walking around looking at the back of a phone or gopro, rather than at the mountain they were filming. And then there were the drones (tho there are also a lot of no-drone signs, basically all of the national parks).

Lake Matheson was the most obvious example of this, but it was in a lot of the places we went.  (I could understand enough German to know roughly what they were saying)

Previously, I could never understand the Japanese, who’d flood out of a tour bus, take 100 photos (or video), then get back on the bus, until I talked to a Japanese friend who explained that they get around 2 weeks per year when they can take a holiday that they choose - what we'd call annual leave - and they have to cram in as much as they can. They live off the photos and video for months and years after. (They get more holidays, it's just public holidays where they don’t get to pick the days).

I’m not sure there is going to be as reasonable an explanation for the Instabook crowd.

There’s a lot of people down the west coast, and it’s not summer anymore. We (NZ) need to spend a LOT more money on resources - toilets at camping areas and tourist spots, roads, track maintenance - otherwise the flood of people is going to seriously damage the already heaving environment. Even more so in places like Queenstown and Wanaka. If only the government could borrow at historically low rates...

The previous blog post, about the West Coast, was done end-to-end on my iPad with an external keyboard. This is something I’ve been trying to work out - how I can move more of my day to day stuff to the iPad, rather than refresh my ageing 2012 retina MacBook Pro.

The main goal here is two-fold - to improve my writing and communication skills, and also to see how far I can go with an iPad, rather than with macOS.

At present, I use the iPad for about 95% of non-development stuff - reading, writing, general internet use - and it’s the device I take with me when I travel.

I use my personal laptop (or my work laptop) for the other 5%, and any development or infrastructure stuff I need to do.

While Bear is fantastic to write Markdown in, Ghost, the software I use to publish this blog, doesn’t play nicely with the iOS browser. It works, mostly, but has some warnings and a few minor bugs.

Some people have tried to do an iOS app for Ghost - Phantom - but it’s stagnated for the past 12 months. I plan to have a look at that when I get back.

So this was an experiment to see if it would work, and so far, it did. I managed to write all the words, and take, pick and edit the photos, all on either my iPhone 8 or iPad. It’s not perfect, but it’s a good start. A better iPhone wouldn’t have changed much, tho a newer iPad (or a pro) might have.

Another option would be to move back to Wordpress, but I don’t think I want to do that. It’s “ok”, and I run a few Wordpress instances already for others, but I like the ethos of Ghost, even if this area is lacking a bit. There are other options like Jekyll or Github Pages, which could also work with some infrastructure.

The next big bit I want to be able to do is infrastructure or development tasks - doing Terraform, changing lambdas etc. I have Pythonista for python stuff, and a Git client, but neither of them is very good for what I want, and I’m lacking a decent code editor. I can easily SSH into a remote EC2 instance to do the actual execution of whatever I’m doing, so I don’t need to run it on the iPad. I can just as easily VPN into home and SSH into my Ubuntu-running Mac Mini. Terminals are a nicely solved problem with the likes of Prompt or even iSH, which is a full Alpine install, inside an iOS sandbox.

I’m really hoping that AWS Cloud9 might work, in Web View rather than in Safari. I’m yet to try that out, but it already does all the editing, syncing with the remote EC2 instance, and machine management.

Having that on the iPad in some sensible way would be amazing.

A wise couple, Alice and Ross, once told me:

If there is a river, or even the sea, go jump in it

We don't have rivers on Waiheke. But we do have lots of ocean  

A while back, Github announced Github Actions, which is a containerised platform for running things when you do something in Github, like make a PR, comment, etc.

On the surface, it looks like a very nice wrapper over the top of their existing web-hook apis, with container execution infrastructure for free. Oh, and a library of existing stuff you can use.

I signed up for the beta, and eventually got in, only to find that it was private repos only - and I couldn't make them (I can now). Today, I came across @jessfraz (Jessie Frazelle) blog post on The Life of a Github Action, which is a nice, small, understandable introduction to making a small Action. I decided to use my ferry ride to look into it again. But....

Eventually, the docs came back up (Thanks Jess and Brendan)

My goal for this is to scratch a small itch we have at work. We tag PRs with ready for review when they are ready, and a webhook look at that, and posts it off into Slack for us, someone sees it, reviews it, and then does the PR approval flow to approve it, and finally, marks it as ready to land.

We used to just do the last bit, before approvals arrived. Now we have to do both. It's only one more click, but it's also difficult to do on mobile, and we have a few bits which trigger off that flag being set.

So, I started hacking on it. The actual function is quite simple, and I've included the code below (until I make the repo public), but here's some of the issues I hit, which might be in the docs.... which I couldn't read.

First, you can have actions locally. I thought I had to refer to another public repository, or initially, a docker registry. But no, you can have it in the same repo, which is awesome.

action "tag on approve" {
    uses = "./tag-on-review/" <--- this is local to my project
    secrets = ["GITHUB_TOKEN"]
}

.github/
  main.workflow

tag-on-review/
  Dockerfile
  tag-on-review.sh

I was using a PR as a test, but committing to master for the workflow and container. It took a few goes to realise that the workflow which was being executed is the one inside the branch!

So Protip: Use another repo for your workflow actions, not the local one. Then you can tie it to a specific branch with uses = "nicwise/my-action@master" and not have this problem. Downside is, the repo has to be public. I really hope I can have a mono-repo for these, I don't want a repo with just one shell script and a Dockerfile.

There is a bit in the workflow which talks about the GITHUB_TOKEN, which is marked as a secret. Actions provides secret management (awesome!) so I was expecting to have to set this - but this one appears to be a special case, and it's just injected in. Nice.

action "tag on approve" {
    uses = "./tag-on-review/"
    secrets = ["GITHUB_TOKEN"] <-- special unicorn. 
# If you make your own, you need to add them into the UI in Settings -> Secrets
}

The content of the event is provided for you. You don't get a URL, it's just a local file - all very concourse-y, which is good. You can then just run jq over it to get stuff out, if you are using bash. Otherwise, pick your preferred Json parser.

action=$(jq --raw-output .action "$GITHUB_EVENT_PATH")
pr_url=$(jq --raw-output .pull_request.url "$GITHUB_EVENT_PATH")

Another snag I hit was that I was testing the approvals action, and I can't approve my own work, so I had to make another account to approve it. Would be nice to have a flag saying "can approve own work", even if it's off by default, and has a big warning. Maybe it is there if I paid Github some money.

Overall, it was really easy, and now I need to think of other things to use Actions for.

One of the main missing bits I'd like to know - it might be there - is how I'd apply this to all (or a wide subset) of our repos without putting a main.workflow in each one, and having to maintain it - could I apply it at the org level somehow? Sort of like how webhooks are also done at an org level.

This is also quite many-repo specific. We tend to use a mono-repo pattern (one repo with many services, in their own folders). We could have a load of .workflow files, and just check the path in a starting step, but still, it would be nice to be able to iterate down N levels (N = 2 for us - services/NAME/.github would do) and have changes below there, trigger just those workflows.

The code for this is on github nicwise/actions-test. Most of it is bastardised off Jess' blog post, so I can't really claim much originality on it.

John William Wise was born the middle child of 3, in a small village called Nunthorpe, North Yorkshire, in July 1936, 3 years before the end of World War II. His father - also John William - was a farm labourer, and his mother was a post office worker in the nearby village.

Dad died on November 26th, 2018. This is the eulogy I wrote for the funeral gathering, and some of the photos we "found".

He went to a small 3 classroom school, situated by the local parish church, where he was a choirboy, tho I can’t say we heard any proof of angelic singing around the house.

In his early life, everything and everyone was close and familiar, across all aspects of life. But at the same time, everyone knew their place, and what their likely path in life was going to be. The nearest suburban area was Middlesbrough, where the iron and steel industry dominated the employment landscape. Auckland Harbour Bridge was one of the projects that used Middlesbrough steel.

Parts of Nunthorpe housed the captains of the local industry, and one of John’s youthful tasks was to walk across the fields to take milk, in a can, to Lady Dorman, who often received only half of what was expected - not because he drunk it - he hated milk, unless it was ice cream or cheese - but because he would swing the can round as he walked.

As well as his love of sweet things, he loved simple foods, and his other trick was to eat out the middle of the loaf of bread he was sent to buy, made easier if the fresh bread was tucked into his jacket to keep warm.

He enjoyed a simple, close-knit community, where the whole village would turn out on summer days to watch the village cricket matches. At 13 he passed a scholarship to Eston Technical Collage, which involved 2 long bus trips, but came with an exposure to the wider world. He was head boy in his last year, his school report showing great improvement - except for divinity studies. He never was one for that.

The best part of being the head boy was being in charge of the tuck shop. I’m pretty sure he would have enjoyed that.

In keeping with with his early upbringing, he chose an apprenticeship as an agricultural fitter, and as the firm he worked for was a fair distance from his home, he acquired his first vehicle, a 350 BSA motorbike.

He and mum met when he was 20, in a park. He and his village mates had travelled from their country village to a pub on the edge of Middlesbrough, where the innocent youth of the Methodist Church had gone for an evening walk after attending the service.

It wasn’t meant to be a pick up job - and there definitely wasn’t Tinder back then - one of the village woman also attended the church Mum went to, and had a thing for the vicar, who tended to the 4 local methodist churches, including Mum’s local church.

There were further liaisons with these two groups, including bus trips to Christmas pantos and summer outdoor theatre in seaside resorts.

It was there that Mum and Dad became a pair, which resulted in 62 years of union, and 56 years of marriage.

After a few years of courtship, they agreed to make the trip to New Zealand and a lifetime of adventure. They chose New Zealand because Mum’s uncle had settled here 40 years before, in 1922, and Mum’s brothers Eric and Harry had both served time in New Zealand waters as members of the Merchant Navy, and it came highly recommended.

And what an adventure it became, with so many places to explore.

They managed to cover almost every part of New Zealand, except Stewart Island, with several long trips in the South Island, and even when I popped in, it didn’t stop them - they took me on my first tramp to the Kitekite Falls in Piha at 6 weeks old.

Sure, I grumbled “how much further” and “are we there yet”, but Dad helped me on many journeys with his made up stories and it was always worthwhile to get to the top of the mountain or the hut at the end of the day. Sadly, I can’t remember the exact words, but there was one about Bee Pee - something about how bee’s make cars go - and a story about how Waikaremoana got its name, which was about a local chief complaining that he had to carry his daughter, Moana.

And then there was being carried down from Ketetahi Hut, in exchange for agreeing to carry him up when he’s 80. It’s a coupe of years late, but we’re going to do that just before Christmas.

Most of you are aware of all of these adventures, having being a part of our lives since. The long list of places I’ve been in this country, mostly with my parents, are something I’m proud of, and still places I love going.

For me, Dad was always there and always supportive, be it cricket, badminton, or being picked up from the various sports and, er, recreations that a teenager gets up to. There was never any question about getting a lift back from a party, even if I’d had a bit much to drink, and dropping of a few friends on the way.

I’d spend school holiday summers helping him with the lawns, paid with the accumulated change in his cap on the dashboard, and we’d have Bundaberg ginger beer, sandwiches from the bakery, and Earnest Adams slice for lunch.

Oh, and the odd secret ice cream. Don’t tell Mum.

Mum and Dad were opposites, but complementary, and Dad was forever supportive of her schemes - a dibbler here, a wooden bowl there, a set of tea light candle holders or placemats, and a threaded carpet or 3.

Just as he’d been mentored growing up, he was happy to pass his knowledge of gardening and motor mowers on to others. He was always generous with his time.

But as time went on, his body started to fail him. About a month ago, the blade on the mower gave up, and what would normally have been an hour long job, took all day, and exacerbated the physical problems he had. The last straw came 2 days before he died, when another part came off while mum was mowing the lawn, and he had to reluctantly concede that it had to go into the repair shop - something he’d done his whole life now eluded him.

He had and cooked a good breakfast on Sunday - eggs, bacon, black pudding, which was surprising as he’d not been eating a lot - but after Mum came back from a walk he was not well, having developed a fever, and decided to go to Middlemore of his own accord, where he died that night.

There is a lot I’ve learned writing this - about his early life and their life together. Things I didn’t know and things I hadn’t taken the time to learn. He taught me a lot, directly and indirectly, and while he’ll always be with me, I’ll always miss him, too.

Ka kite anō au i a koe. I’ll see you again.

I was looking at Tomas Blog a while ago, when he mentioned that he had bought a couple of the Sonoff WIFI switches to do some outdoor LED lighting.

Sadly, the current default firmware that comes with the Sonoff switches no longer allows OTA firmware upgrades, so you have to do it with a serial adapter. I got mine from SparkFun, which took AGES to get here, but works perfectly. Just make sure it's 3.3v not 5 (most can do both). Takes about the same amount of time as OTA.

They sounded like they were quite good, and fairly cheap, too. I already have a few Wemo switches, which are very good so far, as well as a few other bits, but I specifically wanted something to control the spa, which has a simple time-based on/off timer.

Problem is, it's running a heater and a pump, so the draw is rather more than the 2.4kw that a 10A Wemo can handle - it needs at least 16A.

Doing a bit of a google around for something else, I found that Sonoff have a 16A switch - the TH16 - which is what I need for the spa. Conveniently, it also comes with a waterproof thermometer. So I picked one up from AliExpress for $11 + a waterproof temperature sensor and temperature+humidity sensor. Not bad for $20.

Last week, they turned up - turns out Ali Express is quite quick, sometimes. I grabbed a spare, old extension lead, stripped it back, and wired it up - the TG16 doesn't have a normal plug, just terminals which you can connect wires to.

Once it was wired up, I plugged it in, grabbed their app, and gave it a try. It worked pretty well out of the box with usual flow - SSID, password, username and password.....

.... wait a moment. It appears these will only work with an internet connection, and they have to talk via the ITEAD/Sonoff cloud service. So yeah. Someone else, out there, has direct access to turning things on and off. And if they go away, all my devices stop working.

While there does appear to be a homebridge plugin which will use it, I don't really like this model very much. Plus their app is a bit naff.

I had a look around at other stuff - I can run a local server, sort of. But they can also be reflashed with an open source firmware called Tasmota, which supports HTTP, MQTT, all the sensors, and doesn't need to communicate out at all.

Sounds good.

Only problem is, normally you have to break them open and plug in a serial port thing. Rather beyond my electronics skill level.

Then I came across SonOTA, which is a python app which does the various steps to do a local over-the-air (OTA) firmware upgrade - adding the device to the local wifi, loading the updated firmware, rebooting etc. Once it's done (about 5 mins or so), you get the basic, but very functional, Tasmota interface.

Note that for one of the switches, the very basic one, iTEAD have prevented OTA upgrades. Not good :(

The steps are pretty simple. Turn off your local firewall (as they listen to a few ports during the process), make sure you have only one network connection, then run python3 sonota.py and follow the prompts.

After a few reboots, and a check of the router to work out the IP, I'm into the new interface. It has a load of settings, but the main ones I want are the sensor (Configuration -> Module -> Set it as a TH16, and add the correct type of sensor in the gpio14 dropdown), and the MQTT configuration.

Next step is a MQTT broker. This is kind of like a message queue, but designed for IoT stuff. It works over HTTP and websockets, so the clients can be very low powered and uncomplicated, and have the broker handle most of the rest of the communications, polling etc. It's also an open standard, so lots of systems support it, including AWS.

I googled around a bit, again, and found Mosquitto, which is from Eclipse. They have a basic Docker container, which I modified to use 1.4.15 and also to set the directories and permissions in the startup script.

Once that was up, I could see in the logs that the switch was talking to server. Now for the other end of the connection.

Side note: I didn't bother setting up a username+password in mosquitto. I'll most likely do it in the future, but the machine isn't on the internet, and never will be, so until I get all of this sorted out, I'm ok with no username + password. But it can and does support authentication, and you should use it.

The last step is to get the homebridge side setup. There are a set of plugins which supports the MQTT interface, and they work quite nicely:

• Switch
• Temperature
• Humidity

The configuration is pretty much normal homebridge. Replace your server name, add a username+password if you have one, and rename spaswitch with the topic name of your device. The Name can be anything, that's what shows up in HomeKit.

...
"accessories":[
{
    "accessory": "mqtt-switch-tasmota",

    "name": "Spa",

    "url": "mqtt://(servernamehere)",
    "username": "",
    "password": "",

    "topics": {
        "statusGet": "stat/spaswitch/POWER",
        "statusSet": "cmnd/spaswitch/POWER"
    },
    "manufacturer": "ITEAD",
    "model": "Sonoff TH"
},
{
    "accessory": "mqtt-temperature-tasmota",
    
    "name": "Spa Temperature",

    "url": "mqtt://(servernamehere)",
    "username": "",
    "password": "",
    
    "topic": "tele/spaswitch/SENSOR",
    
    "activityTopic": "tele/spaswitch/LWT",
    "activityParameter": "Online",
    
    "startCmd": "cmnd/spaswitch/TelePeriod",
    "startParameter": "120",
    
    "manufacturer": "ITEAD",
    "model": "Sonoff TH"
},
{
    "accessory": "mqtt-humidity-tasmota",
    
    "name": "Spa Humidity",

    "url": "mqtt://(servernamehere)",
    "username": "",
    "password": "",
    
    "topic": "tele/spaswitch/SENSOR",
    
    "activityTopic": "tele/spaswitch/LWT",
    "activityParameter": "Online",
    
    "startCmd": "cmnd/sonoff/TelePeriod",
    "startParameter": "120",
    
    "manufacturer": "ITEAD",
    "model": "Sonoff TH"
}
],
...

There are a few other plugins for other bits from the same guy, too. Nicely done Jaromir.

So now I have the switch and temperature wired up, I just need to wire it into the spa control, which is just an old-skool clock at present, and run the sensor over the edge. It'll also need a splash proof box, which I have spare from another project.

I'm going to wait for that bit until I have more time, incase it goes wrong and I need to wire the old one back in.

I have a few other uses for a temperature meter and switch - keeping the studio above "very cold" for a start. So I'll most likely be getting a few more TM16's for a few bits - water pump controller, heater controller. Or maybe a POW R2, which looks like a slightly cutdown version of the TM16 with a power meter on it.

For my sister-in-laws birthday, we all flew down to Wellington, then caught the train up to Auckland on the Northern Explorer.

We go year-about, having one year on the island, and one year doing something else. Last time, we walked the Routeburn Track.

This time it was a return to Lake Waikaremoana, in Te Urewera.

I've not been there for a few years, but it is, hands down, my favourate place on the planet^[1].

It just feels like.... home. It always has. I keep coming back to this place for some reason that I can't quite explain, to walk the track, visit the lakes, or just get away from everything and reset.

I've come here has a child. As an adult trying to escape a job I desperatly needed a break from. On our honeymoon. On a final camper trip before going to the UK. On a "do we want to move back to NZ" trip.

I always keep coming back. I hope I always will.

It's my chill out space. My stop and reconnect space. My "remember why the fuck I'm doing all of this" space. It was the part of NZ I missed the most while we were away.

I've walked this track many times. I think I did it first at about 4 years old, and I've done it maybe 5 or 6 times since. Back then it used to cost 50c for kids, and $1 for adults, in the honestly box in each hut. Now it's a "great walk", $35 for an adult, and don't even start without a booking cos it's booked solid.

I'm ok with that.

And I love seeing the diversity of people walking it. Chinese/Asian New Zealanders, Pakeha, European tourists (German, Swiss, American, Canadian), Kiwis - hell, we met 3 groups of people from Waiheke!

This is the first time I've done it this way around (Onepoto -> Hopuruahine^[2]). We always did it the other way for some reason, which might have more to do with my parents desire to cut a day off the end than anything.

Except one time, when we got to the top of the bluff to find the water tanks all frozen and we had to turn back. It was no different this time - water was still scarce, but at least it wasn't frozen. It was good and bad to get the bluff done on the first day - good in that it's done, bad in that I have 4 days of food in my pack, not zero.

Compared to the walk up the hill, the walk down is an easy 2.5-3 hour jaunt. But throw in some mist (The Tūhoe have the name "children of the mist" for a reason) and it's a stunning morning walk, ending in a swim at the hut.

Day 3 starts with a walk to the Korokoro Falls (half an hour from the track, without your pack). It's very worth the walk tho.

And finally to the Marauiti Hut, which I swear hasn't changed in 25 years.

The last day is a nice, flat walk to the Whanganui Hut, and the water taxi back to the car.

This hut, Waiharuru, didn't exist when I did it last (around 1992 or so). It was a tiny hut called Te Puna, which was only ever used by "boaties" because of it's location. This is, by far, the fanciest hut on the track.

Leonie has written some about it, too:

• Waikaremoana Day 1
• Waikaremoana Day 2
• Waikaremoana Day 3
• Waikaremoana Day 4