Deem interview question

Very complex augorithm question