This was a right pain and I do love to blog about things that cause me trouble and strife! I created a new Gitlab project and noticed that you can create from template. How exciting! So I selected Android and was delighted to spot a docker file and a yml file had been generated for CI/CD so it integrates nicely with Gitlab pipelines. Amazing! I can run my unit tests and do a build on every commit, this is fantastic.
Gitlab pipelines are a great little tool that builds and runs your project and can be configured as needed to do all sorts of clever things. What's more there's no hardware config involved, it uses docker so you just commit the docker image and let some hardware somewhere in the world build it for you. So simple.
Naturally this builds nicely from the template but the first thing I did with this file was upgrade the compile and target versions in Android's gradle file to the latest version so my app would be compatible with the Google Play Store.
Fail!
Warning: License for package Android SDK Platform 33 not accepted.
FAILURE: Build failed with an exception.
What went wrong:
A problem occurred configuring project ':app'.
Failed to install the following Android SDK packages as some licences have not been accepted.
platforms;android-33 Android SDK Platform 33
To build this project, accept the SDK license agreements and install the missing components using the Android Studio SDK
Manager.
What's happening here is Gitlab pipeline is creating a docker image from our new dockerfile and running the Android build using this docker image. However the docker image is out of date so when it looks for Android SDK 33 it can't find it and tries to install it, but this fails because it can't find an accepted license for that SDK. Not a big deal, I've played with sdkmanager before we can easily update this docker image.
So looking at our newly generated docker file I can see this is badly out of date. It's using the old android-sdk tool which has been replaced by sdkmanager and it's using SDK 28 and some old build tools. Also it's running off jdk 8 which I'm sure we can improve on.
I won't go through every step I took as this would be a really long post, but I'll post some of the things I learnt along the way.
Firstly playing with Docker was really fun, but it was also quite frustrating. Constantly building and re-building took a while. What I found faster was to install Docker desktop and run it locally. This also means you can mount your docker image and run bash to debug your environment.
Also I learnt that a docker file is not docker compose. A docker file is just a dockerfile and you use this to create an image using the docker build command. Docker compose is the yml or yaml file. This allows you to create multi-container applications. Think of this a bit like config for the docker container that is running your docker image.
Another thing I learnt was that Android sdkmanager is fussy and you need to be quite precise with paths and directories or the toys and the pram part ways. The license problem mentioned above ended up being really frustrating. What I eventually found was I had to make sure the sdk wasn't in the root folder and just giving /sdk write permissions wasn't enough. I had to explicitly grant the /sdl/licenses folder write permissions. After many many runs that straightened all that out.
Below is my new dockerfile for Gitlab. You'll notice the following big differences from the stock file I got from Gitlab:
- It now uses JDK11
- It uses SDK 33
- It uses sdkmanager
- It updates sdkmanager as it goes
- It should (hopefully) be able to install the latest SDK and accept the license at build time
# This Dockerfile creates a static build image for CI
#!/bin/sh
FROM openjdk:11-jdk
# Just matched `app/build.gradle`
ENV ANDROID_COMPILE_SDK "33"
# Just matched `app/build.gradle`
ENV ANDROID_BUILD_TOOLS "30.0.0"
# Version from https://developer.android.com/studio/releases/sdk-tools
ENV ANDROID_SDK_TOOLS "8512546_latest"
ENV ANDROID_HOME /opt/sdk
ENV ANDROID_SDK_PATH /opt/sdk
# install OS packages
RUN apt-get --quiet update --yes
RUN apt-get --quiet install --yes wget apt-utils tar unzip lib32stdc++6 lib32z1 build-essential ruby ruby-dev
# We use this for xxd hex->binary
RUN apt-get --quiet install --yes vim-common
# create sdk directory, install Android SDK
# https://dl.google.com/android/repository/commandlinetools-linux-8512546_latest.zip
RUN mkdir -p /opt/sdk
RUN wget --quiet --output-document=android-sdk.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}.zip && \
mv android-sdk.zip /opt/sdk && \
cd /opt/sdk/ && \
unzip android-sdk.zip
#Sort out the mess - https://stackoverflow.com/a/65262939
RUN mkdir /opt/sdk/cmdline-tools/tools && \
mv /opt/sdk/cmdline-tools/bin /opt/sdk/cmdline-tools/tools/ && \
mv /opt/sdk/cmdline-tools/lib /opt/sdk/cmdline-tools/tools/ && \
mv /opt/sdk/cmdline-tools/NOTICE.txt /opt/sdk/cmdline-tools/tools/ && \
mv /opt/sdk/cmdline-tools/source.properties /opt/sdk/cmdline-tools/tools/ && \
export PATH="${PATH}:/opt/sdk/cmdline-tools/tools/bin:${ANDROID_HOME}"
RUN chmod -R 777 /opt/sdk
RUN /opt/sdk/cmdline-tools/tools/bin/sdkmanager --licenses
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager --update
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "platforms;android-${ANDROID_COMPILE_SDK}"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "platform-tools"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS}"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "extras;m2repository;com;android;support;constraint;constraint-layout;1.0.2"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "extras;android;m2repository"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "extras;google;google_play_services"
RUN echo y | /opt/sdk/cmdline-tools/tools/bin/sdkmanager "patcher;v4" "tools"
RUN chmod -R 777 /opt/sdk
RUN chmod -R 777 /opt/sdk/licenses
RUN yes | /opt/sdk/cmdline-tools/tools/bin/sdkmanager --licenses
RUN chmod -R 777 /opt/sdk
# install FastLane
COPY Gemfile.lock .
COPY Gemfile .
RUN gem install bundler -v 1.16.6
RUN bundle install
