Recently I had a use case where we need to log in to a third-party site and fetch protected CSV files.
Just the one read to share for the month of July. It was quite a long read but enjoyable and enlightening.
When using Google Chrome in Headless mode to scrape websites it can often be faster to prototype interaction and data extraction commands via Developer tools. Here is a quick tutorial showing how to get started with talking to your NodeJS script.
Here at Sumo we use Amazon ECS containers for a variety of different micro-tasks because being standard Docker containers they are very easy to develop and test with locally then simply use Docker push to run in the Amazon Cloud.