Well that’s a learning; in Dockerfiles you can have steps that look like:
RUN --mount=type=cache,id=cargo,target=/cargo do whatever
RUN --mount=type=cache for example from the docs…
And it’ll only mount the cache folder for that step.. turns out if you do that on a mac, the underlying filesystem of the cache thing is a mac filesystem, so shit gets BROKEN AND WEIRD when the container’s Linux and the programs expect Linux filesystem things.
This broke both SUSE’s zypper and sccache-based rust builds for reasons I can’t quite explain.. fun.
I build some VM’s in orbstack, running arm/amd64 debian, one of each:
orb create -a arm64 debian:13 \
--user-data ./kanidm-builder-userdata.sh \
kanidm-arm64-build
orb create -a amd64 debian:13 \
--user-data ./kanidm-builder-userdata.sh \
kanidm-amd64-build
Then created the buildx config:
```shell
docker buildx create \
--name multiarch --driver docker-container \
--platform linux/amd64 tcp://kanidm-amd64-build.orb.local:2375 \
&& docker buildx create \
--name multiarch \
--append \
--platform linux/arm64 \
tcp://kanidm-arm64-build.orb.local:2375 \
&& docker buildx use multiarch \
&& docker buildx inspect --bootstrap
… and away we go!
The userdata is:
#!/bin/bash
set -e
# install things
sudo apt-get update
sudo apt-get install -y docker.io ripgrep jq ufw docker-buildx
# configure docker to listen
sudo mkdir -p /etc/systemd/system/docker.service.d/
cat > /etc/systemd/system/docker.service.d/override.conf <<EOF
[Service]
ExecStart=
ExecStart=/usr/sbin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -H tcp://0.0.0.0:2375 $DOCKER_OPTS
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker.service
# firewall things
sudo ufw allow ssh
sudo ufw allow 2375/tcp
echo y | sudo ufw enable
echo "Done!"