Within a week after applying online, I've got email with instruction for a non-paid trial project.
I submitted the project at weekend and then next weekend I was informed that I was not selected.
Thanks to them that atleast they were transparent and provided their review.
However I do not agree 100% to their review, which was something like below;
When dealing with data that's structured hierarchically it is best if the call structure follows, because that makes understanding and maintenance easier. Single methods/functions with complex conditionals are difficult to understand and debug. The number of items in the job's output is below the known total, and field coverage is 100%, which is against requirements; taking more time to review the results would have helped with that.
I've spent 10-12 hours on learning Scrapy framework, writing code, execution of spider for a test website etc. I've added my assumptions where I thought it could be further improved both in code and in separate file, yet if you are judging me for things that I've already mentioned.
1) "Number of items in the job's output is below the known total"
The test website had no total number of records anywhere, you've to parse complete website to know the total.
2) "When dealing with data that's structured hierarchically it is best if the call structure follows"
Totally agreed, that is why I mentioned my assumptions. Plus there is no "best practices" section, within limited time all I could do is to google, which I did and realized that its not recommended to pass response object to pipeline due to possible memory leaks and I clearly mentioned in both code (inline comments) and assumptions file.
3) "field coverage is 100%"
I dont know what this meant. However I am sure it was not mentioned in the requirements section.
4) "taking more time to review the results would have helped with that."
After spending 10~12 hours, and you having no total records on a sample/test website, you expect person to invest more hours (each job execution takes around 30 minutes)?
I still think that I am best suitable for this job and already mentioned to my point of contact, however I am sure I am not going to get this job, but I do like their efforts they've put in whole platform, 10 on 10 for this.
