# Installing Rancher on K3s without Public Internet Access

## What is this guide? 

This is for those wanting to setup Rancher on your K3s cluster that is running on a network where users from the public internet can't access it. In my instance, that means running my cluster on a separate subnet of my home network. I opted to keep this cluster internal only because I don't run anything I would want to access outside my home at the moment. If you are okay with opening holes in your firewall, enabling port forwarding, etc. to get you cluster accessible to the outside world then this guide may not be for you.

Here is a quick guide to setting up [Rancher](https://rancher.com/docs/rancher/v2.6/en/) on the same cluster as your [K3s](https://rancher.com/docs/k3s/latest/en/) installation.

Before we begin, I want to quickly shoutout the guides from [NetworkChuck](https://www.youtube.com/watch?v=X9fSMGkjtug) and GitHub User [kopwei](https://gist.github.com/kopwei/47dfd853261f36943aee80cc7fa5e1aa) that I used as a starting point.

## Setting Up K3s
### 1 - Set up your SD Cards
  - Install PiOS lite using the [Raspberry Pi Imager](https://www.raspberrypi.org/downloads.../)

![CleanShot 2022-01-01 at 01.48.26.gif](https://cdn.hashnode.com/res/hashnode/image/upload/v1641019734551/a2w8XMygr.gif)

### 2 - Init your PiOS
  - Put SD cards in your RPi
  - Turn On
  - Wait ~5-10 minutes
  - Turn Off
  - Take out SD cards

### 3 - Enable the needed settings. 

Take your initialized RPi SD cards and for each one, insert into your reader and edit the following files on the boot partition.

> Replace the stuff in angle brackets with your values


#### `cmdline.txt`

```
cgroup_memory=1 cgroup_enable=memory ip=<static-ip-request>::<dhcp-server-address>:<subnet mask|255.255.255.0>:<hostname>:eth0:off
```

#### `config.txt`

In the `all` key of this config file add the arm64 flag. It should look like the below.

```
[all]
arm_64bit=1
```

### 4 - Put SD card back into your RPi and boot up.

I've had the hostname not change despite changing it with the line in the `cmdline.txt` file. If that happens you can edit it by using the `sudo raspi-config` tool.

## Enable Legacy IP Tables

The latest version of Raspberry PiOS doesn't have legacy IP tables installed. You can install it by running

```bash
sudo apt install iptables
```

Once you have that installed you can run this:

```bash
sudo iptables -F;
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy;
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy;
sudo reboot
```

## Install K3s

Enter sudo mode

```bash
sudo su -
```

Run on your master node

```bash
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s 
```

Get your access token by following the instruction in the output of your master node install step.

Run this to set up your worker nodes

```bash
curl -sfL https://get.k3s.io | K3S_TOKEN="<your-token>" K3S_URL="https://<your-master-node-ip>:6443" K3S_NODE_NAME="<node-name>" sh -
```

On the master node, run `kubectl get nodes` to check for your node status and watch them join.

W00t! You should have your k3s installed and running.

First off, I recommend getting your kubectl config object and loading it into your `~/.kube/config` so you can run `helm` and `kubectl` commands from your local machine. You can do that by copying your `/etc/rancher/k3s/k3s.yaml` and editing the `server` key in the file to be the IP or address as accessed from your local machine. It should probably be what you're using to SSH into your master node. 
Example below

```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <cert-data>
    server: https://127.0.0.1:6443 <-- EDIT THIS
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```

## Install Cert-Manager

Follow the [Cert-Manager](https://cert-manager.io/docs/installation/helm/) install instructions for in-depth instructions. I'm just going to share what I did.

Add helm repo

```bash
helm repo add jetstack https://charts.jetstack.io
helm repo update
```

Install

```bash
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.6.1 \
--set installCRDs=true
```

Use `kubectl get pods -n cert-manager` to verify the pods are running before moving on

## Install Rancher

Again, you can follow the [instructions](https://rancher.com/docs/rancher/v2.6/en/installation/install-rancher-on-k8s/) on the Rancher website if you want more in-depth discussion on what is going on. I'm just going to share what I did.

Add helm repo

```bash
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
helm repo update
```

Create namespace

```bash
kubectl create ns cattle-system
```

Install

```bash
helm install rancher rancher-stable/rancher \
--namespace cattle-system \
--set hostname=<your hostname> \
--set bootstrapPassword=admin \
--set ingress.tls.source=letsEncrypt \
--set letsEncrypt.email=<your email> \
--set rancherImageTag=v2.6.2-linux-arm64
```

From here, this is where you have to start playing with things...

If you have [kubens](https://github.com/ahmetb/kubectx) I'd recommend switching context now. If not, you'll want to make sure you're adding the `-n cattle-system` flag to your `kubectl` commands in the rest of this post.

```bash
kubens cattle-system
```

Check for an issuer

```bash
kubectl get issuer
```

You should see

![CleanShot 2022-01-01 at 02.31.28@2x.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1641022319041/brqispw2P.png)

You'll want to edit that Issuer to use the `dns01` challenge. Here is an example of what I did for the DigitalOcean networking.

### Create secret in the namespace

Secret file

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: digitalocean-dns <- This can be whatever you want to name it
data:
  access-token: "<base64 encoded secret>"
```

Save the above into a file and then create it on the cluster

```bash
kubectl create -f <path/to/secret-file.yaml>
```

Edit Issuer

I used `kubectl edit issuer rancher`

```yaml
apiVersion: v1
items:
- apiVersion: cert-manager.io/v1
  kind: Issuer
  metadata:
    annotations:
      meta.helm.sh/release-name: rancher
      meta.helm.sh/release-namespace: cattle-system
    creationTimestamp: "2021-12-31T03:56:49Z"
    generation: 2
    labels:
      app: rancher
      app.kubernetes.io/managed-by: Helm
      chart: rancher-2.6.3
      heritage: Helm
      release: rancher
    name: rancher
    namespace: cattle-system
    resourceVersion: "45795"
    uid: <uid>
  spec:
    acme:
      email: [redacted]
      preferredChain: ""
      privateKeySecretRef:
        name: letsencrypt-production
      server: https://acme-v02.api.letsencrypt.org/directory
      solvers:
      - dns01:   <== REMOVE http01 and replace with dns01 solver
          digitalocean:
            tokenSecretRef:
              key: access-token
              name: digitalocean-dns
```

From there, you should only need to find the name of the certificate using `kubectl get certificate` and then delete it using `kubectl delete certificate <name>`. Once you delete it, cert-manager will create a new certificate and certificate request and you'll be able to access your rancher portal by using the hostname you used the path `/dashboard/?setup=<bootstrap password used>`

## Conclusion

So, what did we do?

We edited the rancher certificate issuer to use an [ACME DNS01](https://cert-manager.io/docs/configuration/acme/dns01/) solver instead of the default [HTTP01](https://cert-manager.io/docs/configuration/acme/http01/) solver. Rancher _only_ runs with HTTPS mode so we have to find a way to prove to Lets Encrypt that we control the domain we claim to control. I have DigitalOcean configured to handle the external DNS for my domain. Because of that, I was able to pass an access token to Cert Manager so they can create the TXT records needed to prove I own the domain. You should be able to use any of the setup method described in the DNS01 section of the ACME Cert-Manager docs.