Funkin/build/Dockerfile

189 lines
4.6 KiB
Text
Raw Normal View History

feat: ci parallelization, linux builds, ci refactoring (#484) * first linux build attempt * linux deps! * hxcodec dependencies * build timeouts * reup hxcpp cache every time by default gh actions will not update caches on cache hit. since the hxcpp cache grows with compiles, that's not what we want here. since we *do* need the files newly compiled. * ci speed test * group runners by purpose * REFACTOR CI. IT WAS NEEDED * smol changies * second attempt at libc * fix any format issues * it's 1:50am * migrate away from gacts/run-and-post-run@v1 * apt does not have a stable cli interface. use with caution in scripts * first attempt at libffi6 * second attempt at libffi6 * fuck that * sigh * html5 also needs new libc * make sure rerunning ln -s doesn't fail the build * desperate attempt * arc attempt * arc, ii * ci * apt-GET * who needs safeguards anyway * clean ci build * debug time * lots of connectivity debugging * :pleading_face: * natesales/q * i'm not very smart * debug hard? * whose traceroute? * pls * ... * we go even newer * merge moment * haxelib maybe * debug info * :pleading_face: * lower mtu runner? * libffi my beloved * no multiline env? * smol buggy * non docker-aware vars * i love bash * builds hopefully go nyoom * forgor native dep * [skip ci] meow * convenient typo * sigh * [skip ci] waow * [skip ci] docker is hard * i don't understand docker tbh * debuggering * docking * small amount of dumb but still doesn't explain why curl doesn't curl * just vsc things * ca certs issue? * please this has to be the one * find -type d fail, laugh at this user * too eepy * im not that smart am i * attempt to run containerized from docker base image * [skip-ci] some more docking * might as well try * :pleaading_face: * ? * ! * idea * sigh * i give in * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa * :pleading_face: * hopefully fix hxcpp cache * [skip ci] a little dx never hurt anyone * try removing things until it breaks * welp that was easy * not-docker-friendly paths in non-docker env, etc. you get the point * more sane default cache size * finishing touches * welp * mounted volume * no systemd, got it * more failproof dockerfile * does this not have ossh * haxelib master * hopefully final docker build * bob the builder * docking :3 * image cleanup * github moment * okay mayb * gotta set it up first * i have an idea * hope we ready * :pleading_face: * fuck * sigh * trigger build on new image * no comment * global hxcpp, first idea * yikes * hxcpp oopsy * code dupe * more code dupe * lint * increase hxcpp cache size on native builds as well * buttons :3 * oops * forgor to export env variable * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2024-04-14 17:49:41 -04:00
FROM ubuntu:mantic
ARG haxe_version=4.3.4
ARG haxelib_version=4.1.0
ARG neko_version=2.3.0
# prepare runner
ENV GITHUB_HOME="/github/home"
RUN <<EOF
mkdir -p "$GITHUB_HOME"
mkdir -p /opt
mkdir -p /usr/share/hxcpp
mkdir -p /usr/local/bin
chmod -R 777 /opt
chmod -R 777 /usr/share
chmod -R 777 /usr/local/bin
EOF
# Prepare Ubuntu
# https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/configure-environment.sh
# https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/configure-system.sh
RUN <<EOF
echo 'vm.max_map_count=262144' | tee -a /etc/sysctl.conf
echo 'fs.inotify.max_user_watches=655360' | tee -a /etc/sysctl.conf
echo 'fs.inotify.max_user_instances=1280' | tee -a /etc/sysctl.conf
EOF
ENV DEBIAN_FRONTEND="noninteractive"
2024-04-24 15:44:24 -04:00
ENV GIT_TERMINAL_PROMPT="0"
feat: ci parallelization, linux builds, ci refactoring (#484) * first linux build attempt * linux deps! * hxcodec dependencies * build timeouts * reup hxcpp cache every time by default gh actions will not update caches on cache hit. since the hxcpp cache grows with compiles, that's not what we want here. since we *do* need the files newly compiled. * ci speed test * group runners by purpose * REFACTOR CI. IT WAS NEEDED * smol changies * second attempt at libc * fix any format issues * it's 1:50am * migrate away from gacts/run-and-post-run@v1 * apt does not have a stable cli interface. use with caution in scripts * first attempt at libffi6 * second attempt at libffi6 * fuck that * sigh * html5 also needs new libc * make sure rerunning ln -s doesn't fail the build * desperate attempt * arc attempt * arc, ii * ci * apt-GET * who needs safeguards anyway * clean ci build * debug time * lots of connectivity debugging * :pleading_face: * natesales/q * i'm not very smart * debug hard? * whose traceroute? * pls * ... * we go even newer * merge moment * haxelib maybe * debug info * :pleading_face: * lower mtu runner? * libffi my beloved * no multiline env? * smol buggy * non docker-aware vars * i love bash * builds hopefully go nyoom * forgor native dep * [skip ci] meow * convenient typo * sigh * [skip ci] waow * [skip ci] docker is hard * i don't understand docker tbh * debuggering * docking * small amount of dumb but still doesn't explain why curl doesn't curl * just vsc things * ca certs issue? * please this has to be the one * find -type d fail, laugh at this user * too eepy * im not that smart am i * attempt to run containerized from docker base image * [skip-ci] some more docking * might as well try * :pleaading_face: * ? * ! * idea * sigh * i give in * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa * :pleading_face: * hopefully fix hxcpp cache * [skip ci] a little dx never hurt anyone * try removing things until it breaks * welp that was easy * not-docker-friendly paths in non-docker env, etc. you get the point * more sane default cache size * finishing touches * welp * mounted volume * no systemd, got it * more failproof dockerfile * does this not have ossh * haxelib master * hopefully final docker build * bob the builder * docking :3 * image cleanup * github moment * okay mayb * gotta set it up first * i have an idea * hope we ready * :pleading_face: * fuck * sigh * trigger build on new image * no comment * global hxcpp, first idea * yikes * hxcpp oopsy * code dupe * more code dupe * lint * increase hxcpp cache size on native builds as well * buttons :3 * oops * forgor to export env variable * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2024-04-14 17:49:41 -04:00
# Prepare APT
RUN <<EOF
cat <<EOC >> /etc/apt/apt.conf.d/10apt-autoremove
APT::Get::AutomaticRemove "0";
APT::Get::HideAutoRemove "1";
EOC
echo <<EOC >> /etc/apt/apt.conf.d/80retries
"APT::Acquire::Retries \"10\";"
EOC
echo <<EOC >> /etc/apt/apt.conf.d/90assumeyes
"APT::Get::Assume-Yes \"true\";"
EOC
EOF
# Prepare apt-fast
RUN <<EOF
apt-get update
apt-get install -y --no-install-recommends software-properties-common
add-apt-repository -y ppa:apt-fast/stable
apt-get -y install apt-fast
echo debconf apt-fast/maxdownloads string 8 | debconf-set-selections
echo debconf apt-fast/dlflag boolean true | debconf-set-selections
echo debconf apt-fast/aptmanager string apt-get | debconf-set-selections
EOF
# Base packages
# https://github.com/actions/runner-images/blob/main/images/ubuntu/toolsets/toolset-2204.json#L114
RUN <<EOF
apt-fast install -y --no-install-recommends \
ca-certificates \
bzip2 curl g++ gcc make jq tar unzip wget \
sudo git openssh-client
EOF
# Prepare git
RUN <<EOF
cat <<EOC >> /etc/gitconfig
[safe]
directory = *
2024-04-24 15:40:45 -04:00
[credential]
helper = cache
feat: ci parallelization, linux builds, ci refactoring (#484) * first linux build attempt * linux deps! * hxcodec dependencies * build timeouts * reup hxcpp cache every time by default gh actions will not update caches on cache hit. since the hxcpp cache grows with compiles, that's not what we want here. since we *do* need the files newly compiled. * ci speed test * group runners by purpose * REFACTOR CI. IT WAS NEEDED * smol changies * second attempt at libc * fix any format issues * it's 1:50am * migrate away from gacts/run-and-post-run@v1 * apt does not have a stable cli interface. use with caution in scripts * first attempt at libffi6 * second attempt at libffi6 * fuck that * sigh * html5 also needs new libc * make sure rerunning ln -s doesn't fail the build * desperate attempt * arc attempt * arc, ii * ci * apt-GET * who needs safeguards anyway * clean ci build * debug time * lots of connectivity debugging * :pleading_face: * natesales/q * i'm not very smart * debug hard? * whose traceroute? * pls * ... * we go even newer * merge moment * haxelib maybe * debug info * :pleading_face: * lower mtu runner? * libffi my beloved * no multiline env? * smol buggy * non docker-aware vars * i love bash * builds hopefully go nyoom * forgor native dep * [skip ci] meow * convenient typo * sigh * [skip ci] waow * [skip ci] docker is hard * i don't understand docker tbh * debuggering * docking * small amount of dumb but still doesn't explain why curl doesn't curl * just vsc things * ca certs issue? * please this has to be the one * find -type d fail, laugh at this user * too eepy * im not that smart am i * attempt to run containerized from docker base image * [skip-ci] some more docking * might as well try * :pleaading_face: * ? * ! * idea * sigh * i give in * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa * :pleading_face: * hopefully fix hxcpp cache * [skip ci] a little dx never hurt anyone * try removing things until it breaks * welp that was easy * not-docker-friendly paths in non-docker env, etc. you get the point * more sane default cache size * finishing touches * welp * mounted volume * no systemd, got it * more failproof dockerfile * does this not have ossh * haxelib master * hopefully final docker build * bob the builder * docking :3 * image cleanup * github moment * okay mayb * gotta set it up first * i have an idea * hope we ready * :pleading_face: * fuck * sigh * trigger build on new image * no comment * global hxcpp, first idea * yikes * hxcpp oopsy * code dupe * more code dupe * lint * increase hxcpp cache size on native builds as well * buttons :3 * oops * forgor to export env variable * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2024-04-14 17:49:41 -04:00
EOC
ssh-keyscan -t rsa,ecdsa,ed25519 github.com >> /etc/ssh/ssh_known_hosts
ssh-keyscan -t rsa,ecdsa,ed25519 ravy.dev >> /etc/ssh/ssh_known_hosts
EOF
# Haxe native dependencies
RUN <<EOF
apt-fast install -y --no-install-recommends \
libc6-dev libffi-dev \
libx11-dev libxi-dev libxext-dev libxinerama-dev libxrandr-dev \
libgl-dev libgl1-mesa-dev \
libasound2-dev \
libvlc-dev libvlccore-dev
EOF
# Janky libffi.6 fix
RUN <<EOF
ln -s \
/usr/lib/x86_64-linux-gnu/libffi.so.8 \
/usr/lib/x86_64-linux-gnu/libffi.so.6 \
|| true
EOF
# neko
# https://github.com/HaxeFoundation/neko/releases/download/v2-3-0/neko-2.3.0-linux64.tar.gz
RUN <<EOF
neko_url=$(curl https://api.github.com/repos/HaxeFoundation/neko/releases -sfL \
| jq '.[] | select(.name == "'"$neko_version"'")' \
| jq '.assets[] | select(.name | endswith("linux64.tar.gz"))' \
| jq -r '.browser_download_url')
curl -sfL "$neko_url" | tar -xz -C /usr/local
EOF
RUN <<EOF
neko_path="$(find /usr/local -maxdepth 1 -type d -name 'neko*')"
ln -s "$neko_path" /usr/local/neko
EOF
ENV NEKOPATH="/usr/local/neko"
ENV LD_LIBRARY_PATH="$NEKOPATH:$LD_LIBRARY_PATH"
ENV PATH="$NEKOPATH:$PATH"
# haxe
# https://github.com/HaxeFoundation/haxe/releases/download/4.0.5/haxe-4.0.5-linux64.tar.gz
RUN <<EOF
haxe_url=$(curl https://api.github.com/repos/HaxeFoundation/haxe/releases -sfL \
| jq '.[] | select(.name == "'"$haxe_version"'")' \
| jq '.assets[] | select(.name | endswith("linux64.tar.gz"))' \
| jq -r '.browser_download_url')
curl -sfL "$haxe_url" | tar -xz -C /usr/local
EOF
RUN <<EOF
haxe_path="$(find /usr/local -maxdepth 1 -type d -name 'haxe*')"
ln -s "$haxe_path" /usr/local/haxe
EOF
ENV HAXEPATH="/usr/local/haxe"
ENV HAXE_STD_PATH="$HAXEPATH/std"
ENV PATH="$HAXEPATH:$PATH"
# haxelib
RUN <<EOF
HOME=/etc haxelib setup "$HAXEPATH/lib"
haxelib --global --never install haxelib $haxelib_version
haxelib --global --never git haxelib https://github.com/HaxeFoundation/haxelib.git master
haxelib --global --never install hmm
EOF
# hxcpp
ENV HXCPP_COMPILE_CACHE="/usr/share/hxcpp"
ENV HXCPP_CACHE_MB="4096"
# Clean up
# https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/cleanup.sh
RUN <<EOF
rm -r /var/cache/apt/apt-fast
apt-get clean
if [ -d /var/lib/apt/lists ]; then
rm -rf /var/lib/apt/lists/*
fi
if [ -d /tmp ]; then
rm -rf /tmp/*
fi
if [ -d /root/.cache ]; then
rm -rf /root/.cache
fi
if command -v journalctl; then
journalctl --rotate
journalctl --vacuum-time=1s
fi
if [ -d /var/log ]; then
find /var/log -type f -regex ".*\.gz$" -delete
find /var/log -type f -regex ".*\.[0-9]$" -delete
find /var/log/ -type f -exec cp /dev/null {} \;
fi
if [ -f /usr/local/bin/invoke_tests ]; then
rm -rf /usr/local/bin/invoke_tests
fi
EOF
# Print debug info
RUN <<EOF
echo "/root"
ls -la /root
cat /root/.haxelib && echo
id
env
EOF