Data Engineer

5+ Years
Open to Work From Home
With Business Certificates
Established in 2007
51-200 Employees
Home-based
Total vacancies for this job title: 1
Posted On: September 26, 2022
Job ID: 411762

Details

About the Client

The client is a team who do things a little differently: they’re a global fintech that was born digital, nimble, connected and united in their purpose to truly support every one of their customers’ quests for trading mastery

 

DATA ENGINEER


Skills & Experience

  • Solid experience and knowledge in both functional and object oriented languages;

  • Experienced in delivering software by using test-driven development practices;

  • A deep understanding of data engineering and distributed computing problems, patterns, systems and tools;

  • Practical experience with AWS Cloud and infrastructure-as-code;

  • Knowledgeable in the alternative data formats and data types available and their advantages and disadvantages;

  • Solid understanding of differing enterprise data architectures and relevant concepts such as data lakes, data marts and constituent technology offerings;

Specific Languages and Technologies

 

Your practical experience should mean a strong ability in most of these languages and technologies.

 

l. Programming Languages:

  • Java

  • Scala

  • Golang (a plus)

  • Python

ll. AWS Services:

  • Lambda

  • Redshift

  • Glue

  • EC2, ECS and EKS

  • IAM

  • S3

  • Data Migration Service

lll. Other:

  • Data formats: e.g. Apache Parquet, ORC,

  • Avro and protobuf

  • Data processing and data lake technologies such as Apache Hadoop

  • DBMS’s: MySQL, PostgreSQL, no-sql and graph DB’s

  • Modeling: ERD, data flow and state diagrams

lV. Data Streaming and Batch:

  • Apache Kafka and related technologies such as Kafka Streaming and Kafka

  • Connect

  • Apache Spark

  • Airflow

V. DevOps / Infrastructure-as-code:

  • Terraform

  • Ansible

  • Docker

  • CI Tools such as Buildkite, Jenkins or CircleCI

Duties and Responsibilities:

 

Scope

 

The Data Engineer will work across our different data platforms. These platforms are responsible for recording, collating and aggregating trading and user behavior, and then surfacing this information to both internal stakeholders and third-party services.

The role reports to the Data Team Lead.

 

Key Responsibilities

  • Maintain and improve existing data infrastructure and data processing systems;

  • Liaise with project stakeholders to provide advice, document and create solutions and communicate delivery milestones;

  • Provide technical leadership and guidance to team members and the wider development team group;

  • Ensure a high level of quality of delivered software through coding standards and continuous integration monitoring systems;

  • Ensure an appropriate level of data source performance and reliability for internal customers;

  • Lead initiatives targeted at improving the team’s software development practices;

  • Work with the Data Team Lead on Data Engineering team goals aligned with the wider Corporate and IT group’s goals;

  • Help team members set and achieve their personal professional development goals through regular meetings and reviews;