ABOUT THE EMPLOYER
We are a leading technology company specializing in web data extraction and analysis. Our innovative solutions empower businesses with valuable insights derived from web data. We are seeking a skilled Web Crawler Developer to join our dynamic team and contribute to the development and maintenance of our web crawling infrastructure.
WHAT WILL YOU WORK ON?
- Develop new web crawlers in Go to extract data from various sources.
- Perform regular maintenance and updates to existing crawlers to ensure optimal performance and reliability.
- Implement strategies to efficiently handle proxies and prevent IP blocks while crawling websites.
- Collaborate with cross-functional teams to integrate web data into existing systems and applications.
- Provide support and troubleshooting assistance related to web crawling activities.
- Work with legacy PHP code as needed to integrate with existing systems.
WHAT SKILLS AND EXPERIENCE ARE WE LOOKING FOR?
- Proficient in Go programming language.
- Strong understanding of web scraping techniques and best practices.
- Experience working with proxies and IP rotation to prevent blocking.
- Familiarity with PHP/Python for working with legacy codebases.
- Excellent problem-solving skills and attention to detail.
- Effective communication skills and ability to work collaboratively in a remote team environment.
- Minimum of 4 hours overlap with PST timezone (9 AM to 5 PM).
ARE THERE ANY OTHER REQUIREMENTS?
- Experience with burpsuite/OWASP ZAP.
- Experience with ajax/reverse engineering.
- Knowledge of web development technologies such as HTML, CSS, and JavaScript.
