Data Engineer

☞ Dome Recruitment Group

View: 105

Update day: 11-06-2024

Location: Crawley South East

Category: IT - Software

Industry: Staffing Recruiting

Job type: Full-time

Loading ...

Job content

Data Engineer

Our client requires a Data Engineer to design and develop data systems to support the rapid growth in their business. This will involve working with different types of data and systems, identifying areas for improvement, and designing new data processing pipelines and systems to provide a robust, scalable data platform for the future.

Responsibilities:

  • Design robust data processing systems to replace existing manual or inefficient processes.
  • Construct ETL pipelines and develop stored procedures to create efficient data applications and workflows using industry best practices.
  • Identify ways to improve data reliability and quality, such as automating manual steps to increase the quality and speed of data processing where possible.
  • Deploy sophisticated analytics programs, machine learning and statistical methods so that they can integrate with insight platform.
  • Create functions and data views that allow other application and business users to easily interact with large datasets and processing systems.
  • Document all designs, specifications and interfaces so that users can understand them clearly.
  • Ensure environments are stable and available.
  • Write production quality code including good unit test coverage.
  • Keep up to date with advances in Data Engineering, including new applications and ways of working, so that the best industry-standard tools are always used in place of bespoke solutions.

Skills / Ability

  • Experience working on data pipelines at scale using the latest technologies.
  • Desirable to have demonstrable continuous professional development in programming languages such as Python, C#, or Java.
  • Good knowledge of the relevant technologies e.g. SQL, Oracle, PostgreSQL, Python, ETL pipelines, Airflow, Hadoop, Parquet.
  • Comfortable working in Linux and docker deployment environments.
  • In-depth understanding of data modelling, access and storage.
  • Strong problem-solving and analytical abilities.
  • Ability to present solutions and limitations to non-IT business experts.
Loading ...
Loading ...

Deadline: 26-07-2024

Click to apply for free candidate

Apply

Report job
Loading ...
Loading ...

SIMILAR JOBS

Loading ...
Loading ...