Talent Array is a high performing multi-threaded data harvesting platform, whereby data from heterogenous sources in varying formats had to be scrubbed, normalized into a data lake, before being transformed and validated to downstream multi-purpose data warehouse and data marts. Real time visualization dashboards were built on top of the data warehouse/data marts to highlight key KPIs and enable dynamic trend analysis. Technologies: Java, Spring Integration, In-house ETL, RabbitMQ, Kafka, SQL Server, ELK, Grafana, Redash, HtmlUnit, Selenium, Jsoup
