We’ve built a web platform that simplifies the analysis of ad reports for DemandHelm – an ad tech company founded by former ad experts from Amazon and Facebook.
Check them out www.demandhelm.com
The platform had to support multiple data sources like Amazon Advertising, Target, Walmart, and others. Therefore, it needed a scalable data pipeline that could handle a massive volume of incoming data. Plus, we had to top it off with a polished UI that’d make it easy to review it.
The most challenging part was deciding how to process and store data. Since we’re working with semi-structured data from different sources – we used Amazon S3 as a data lake. And we choose RDS with MySQL-compatible Aurora for final data reporting.
One of the crucial data processing moments is the transformation layer, so we used AWS DataPipeline to implement the ETL because of the fault-tolerant architecture.
A NodeJS based Frontend API app is hosted on a distributed cluster of EC2 and managed by ElasticBeanstalk. Amazon AutoScaling provides seamless scaling based on current resource usage and deletes excessive compute units to reduce costs and handle traffic spikes.
We’ve designed and built a frontend app with React and Redux. Considering the amount of analytical data, we’ve focused on a flexible framework for analytical views to make working with data faster. The frontend app supports server-side rendering, so it’s deployed on EC2 and hosted by CloudFront at all edge locations.