Aimleap

India (Remote)

Posted on Jun 06, 2024

Web Scraping Expert (Scrapy & Python) WFH

Experience: 3 to 5 years
Location: Work from Home/Bangalore/India
Salary: 3 Lacs to 7 Lacs INR PA
No of positions: 4
Employment time: Full time
Educational qualification: CS/IT/Engineering
Industry: IT-ITES
Notice Period: Immediate

• Experienced in developing and maintaining robust web scraping pipelines using Scrapy, a high-performance Python framework
• Proficient in Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures
• Exposure in data cleaning and manipulation libraries (Pandas, NumPy) and data extraction methods (CSS selectors, XPath)

RESPONSIBILITIES:

• Design, develop, and maintain web scraping projects using Scrapy, including:
  Crawling websites to extract specific data points
  Implementing efficient crawling strategies to handle pagination, dynamic content, and complex website structures
  Employing data extraction techniques with CSS selectors and XPath
  Processing and cleaning scraped data using Python libraries (e.g., Pandas, NumPy)
  Storing extracted data in appropriate formats (e.g., CSV, JSON, databases)
• Collaborate with data engineers and analysts to identify data needs and define scraping requirements
• Write well-documented, maintainable, and efficient Scrapy code
• Integrate scraping pipelines with other Python frameworks and tools as needed
• Stay up-to-date with the latest web scraping trends and best practices
• Troubleshoot and address challenges related to rate limiting, authentication, and website changes
• Implement caching strategies for efficient data retrieval and reduced website load
• Consider potential ethical and legal implications of web scraping

QUALIFICATIONS:

• Proven experience with web scraping techniques and tools (Scrapy preferred)
• Strong proficiency in Python programming
• Knowledge of HTML and CSS for navigating website structures
• Experience with data extraction methods (CSS selectors, XPath)
• Familiarity with data cleaning and manipulation libraries (Pandas, NumPy)
• Understanding of distributed crawling concepts for large-scale data extraction
• Excellent problem-solving and analytical skills
• Ability to work independently and as part of a team
• Strong communication and documentation skills

AIMLEAP is an ISO 9001:2015 and ISO/IEC 27001:2013 certified global technology consulting and service provider offering Digital IT, AI-augmented Data Solutions, Automation, and Research & Analytics Services.

AIMLEAP has been recognized as ‘The Great Place to Work®’. With focus on AI and automation-first approach, our services include end-to-end IT application management, Mobile App Development, Data Management, Data Mining Services, Web Data Scraping, Self-serving BI reporting solutions, Digital Marketing, and Analytics solutions.

We started in 2012 and successfully delivered projects in IT & digital transformation, automation driven data solutions, and digital marketing for more than 750 fast-growing companies in the USA, Europe, New Zealand, Australia, Canada; and more.

– An ISO 9001:2015 and ISO/IEC 27001:2013 certified
– Served 750+ customers
– 12+ Years of industry experience
– 98% Client Retention
– Great Place to Work® Certified
– Global Delivery Centers in the USA, Canada, India & Australia.