About the vacancy

Our client is a US-based healthcare company that building a solution to process drugs reviews information. The Big Data Engineer will work on the ETL framework aimed at building configurable, extendable, easy to use ETL processes. The framework is a Spark application, which is configured from a set of metadata tables stored in Apache Cassandra to load data from different source servers, saving obtained files into HDFS, further obtaining data from them, transforming and saving into Cassandra target tables. The framework also allowed loading data to CRM. Orchestration of all these processes was implemented with a help of Apache Oozie.

The specialist will develop and support ETL platform based on the Apache Spark and Hadoop ecosystem.

Must have

• Basic theoretical knowledge in Big Data and related technologies: RDBMS, NoSQL, Consistency (ACID, BASE), OLAP and OLTP, massively parallel processing, data warehousing
• Novice level in Scala
• Intermediate level in data platforms (RDBMS). Understanding of relational model, basic databases concepts, and components
• Experience with one relational database at least
• Novice level in Big Data Platform (Apache Spark)

Would be a plus

• Experience with Hadoop Ecosystem