Scraping a Million Pokemon Battles: Distributed Systems By Example
I love Pokemon. However, I don’t love how some players make the community less welcoming towards beginners by hiding their strategies. So I did what any defiant engineer would. I signed up for a free AWS account and began (responsibly) scraping millions of their unauthenticated Pokemon battles. We’ll journey together through this passion project of mine and draw on specific examples to better understand the trade-offs of working with distributed systems or microservice architectures …