Job Title: Data Engineer Location: Bank Muscat Head Office Department: Enterprise Project Management & Innovation Job Overview: The data engineer is responsible for designing, developing, and maintaining the infrastructure and systems required for data storage, processing, and analysis for AI initiatives. They play a crucial role in building and managing the data pipelines that enable efficient and reliable data integration, transformation, and delivery for all Al projects. Key Responsibilities: • Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and load it into the appropriate data storage systems • Collaborates with AI developers, data scientists and analysts to optimize models and algorithms for data quality, security, and governance • Integrates data from different sources, including databases, data warehouses, APIs, and external systems • Ensures data consistency and integrity during the integration process, performing data validation and cleaning as needed • Transforms raw data into a usable format by applying data cleansing, aggregation, filtering, and enrichment techniques • Optimizes data pipelines and data processing workflows for performance, scalability, and efficiency • Monitors and tunes data systems, identifies and resolves performance bottlenecks, and implements caching and indexing strategies to enhance query performance • Implements data quality checks and validations within data pipelines to ensure the accuracy, consistency, and completeness of data • Supports the governance of data and algorithms used for analysis, analytical applications, and automated decision making Qualifications: • A bachelor's degree in computer science, data science, software engineering, information systems, or related quantitative field; master's degree preferred • At least six years of work experience in data management disciplines, including data integration, modeling, optimization and data quality, or other areas directly relevant to data engineering responsibilities and tasks • Proven project experience developing and maintaining data pipelines for AI projects Technical Skills: • Expert knowledge in Apache technologies such as Kafka, Airflow, and Spark to build scalable and efficient data pipelines • Ability to design, build, and deploy data solutions that capture, explore, transform, and utilize data to support AI, ML, and Bl • Strong ability in programming languages such as Java, Python, and C/C++ • Ability in data science languages/tools such as SQL, R, SAS, or Excel • Proficiency in the design and implementation of modern data architectures and concepts such as cloud services (AWS, Azure, GCP) and modern data warehouse tools (Snowflake, Databricks) • Experience with database technologies such as SQL, NoSQL, Oracle, Hadoop, or Teradata • Ability to collaborate within and across teams of different technical knowledge to support delivery and educate end users on data products • Expert problem-solving skills, including debugging skills, allowing the determination of sources of issues in unfamiliar code or systems, and the ability to recognize and solve repetitive problems Key Competencies: • Excellent business acumen and interpersonal skills; able to work across business lines at a senior level to influence and effect change to achieve common goals. • Ability to describe business use cases/outcomes, data sources and management concepts, and analytical approaches/options • Ability to translate among the languages used by executive, business, IT, and quant stakeholders.
اشترك مجاناً واختر تخصصاتك، وسنرسل لك الوظائف المناسبة مباشرة على واتساب
اشترك عبر واتساب مجاناً