Data Engineer – Qatar Airways , Qatar

About Your Job:
In this role, you will create reliable architectures for building highly scalable data pipelines to collect a large amount of data from different sources and transform it into a usable format for analysis. You will design, implement, maintain a full suite of real-time and batch jobs that fuels our cutting edge data analytics platform to provide real-time intelligence to our businesses.

Key Responsibilities:
· Design, construct, install, test and maintain highly scalable and optimized data pipelines with state-of-the-art monitoring and logging practices.
· Bring together large, complex and sparse data sets to meet functional and non-functional business requirements and use a variety of languages, tools and frameworks to marry data.
· Design and implement data tools for analytics and data scientist team members to help them in building, optimizing and tuning of use cases.
· Build data warehouse solutions.
· Develop routines for cleansing and harmonization of data from a variety of data sources.
· Defines data catalogues, metadata to provide search ability and governance (including Records Management) for structured and unstructured data.
· Identifies and manages reference data.
· Design and develop business intelligence dashboards using data visualization tools.
· Develop REST APIs using data lake store as the source for application consumption.
· Conduct and own Root Cause Analysis (RCA) of reported incidents in operational systems through code, log and configuration reviews and ensure timely code, configuration or infrastructure fixes.
· Build solutions which are scalable, resilient and sustainable to address business requirements.
· Tackle challenges and solve complex problems on a daily basis.

Qualifications
About you:
The applicant should have a Bachelor’s Degree or equivalent (Degree in engineering, computer applications, commerce, or business administration). You must have minimum 4 years of data engineering experience. Should have excellent verbal and written communications skills. Also possess good analytical, interpersonal skills and a proven team player.

REQUIRED
2+ years hands-on data engineering experience in working with big data using technologies like Hadoop/Hive, Hyperscale PostgreSQL, Java/Scala, Spark, Kafka, SQL and NoSQL, Python, azure cloud-based data engineering solutions (ex: Azure Data Factory, Azure Data Lake Store, Azure Databricks, Azure HDInsight).
Hands-on experience on data tools – striim, streamsets, NiFi.
Hands-on experience in data modeling, data visualization, and pipeline design & development.
Hands on experience with data warehouse platforms (ex: Snowflake, Azure Data Lake Analytics).
Strong technical knowledge of performance tuning and query optimization on large data sets.
Very good command of English language.

PREFFERED
Experience with elasticsearch.
Experience with cloud-based data-warehousing system Snowflake.
Experience with ETL tool Informatica.
Experience with data virtualization, semantic layer tool dremio.
Experience with visualization tools – Tableau, Power BI.
Knowledge of airline domain.
Knowledge of agile/lean development methodologies.

Click here to Apply Online

Enter your email address to receive latest Gulf Jobs Updates: