Dailyhunt interview question

design a crawler