we offer you an extraordinary chance to learn, to develop and to be part of an exciting experience and team.
Remote | Long term
✓ Must have experience in developing Spark applications using Pyspark and Spark SQL in both Azure databricks & Hadoop platforms.
✓ Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming.
✓ Experience in creating a automated framework using Pyspark for data profiling/data quality tasks and generate reports/files.
✓ Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure Databricks
✓ Good experience in reading the data from ADLS Gen2/Hadoop distributed file system using Azure databricks.
✓ Good understanding of Big Data Hadoop platform and HDFS.
✓ Worked on various file formats like Parquet, ORC & AVRO file.
✓ Nice to have retail domain experience