This is a weird one but it took up most of my day, so I thought I'd post it here for posterity. This article describes the problem very well:
https://dev.to/dbottillo/rxjava-a-story-about-delay-and-schedulers-j48
.delay(200, TimeUnit.MILLISECONDS)
In short calling .delay on a rxjava observable causes very strange results. The reason is because delay forces a change to the subscribeOn type. Why it does this is a mystery to me. However luckily the solution is very simple.
.delay(200, TimeUnit.MILLISECONDS, AndroidSchedulers.mainThread())