Cross-Culture Reports

PayPal has partnered with a local survey platform to conduct market research for their Southern African division. Due to stringent policies within the region, all survey data is to be stored within the platform’s data centers, each contained within each country’s borders.

While most of the survey data is pre-quantified, a decent chunk is pure text data, along with different languages in the region. To centralize the analytics, a translation module has been utilized to ensure that analysts can derive a cohesive analysis of the region.

As a consulting engineer, you have been tasked to look at the ETL pipeline connecting PayPal’s data marts with the survey platform’s data warehouses. Within this layer also comes another layer of ETL pipelines, connecting transactional data stores with the survey platform’s data warehouse, as well as the pipeline normalizing this data through translation modules.

How would you ensure the data quality across these different ETL platforms?