Mid-Level Data Engineer

☞ Pimento Connection

View: 107

Update day: 11-06-2024

Location: Wembley South East Enfield Brentford

Category: High Technology Mechanical / Technical

Industry: Information Technology

Job type: Permanent, full-time

Salary: £50,000 - £65,000 p.a

Loading ...

Job description

Mid-Level Data Engineer

Security Clearance Essential

Location: Remote working with 1-2 days per month in Central London

The Mid-Level Engineer will be part of a small technical team including other data engineers and an infrastructure lead to consolidate and scale the technical product offering that our customer has already built.

Within the secure-by-design AWS infrastructure, the Engineer will be comfortable participating in daily stand-ups and working closely with the Lead Engineer to ensure the best-in-class national security product continues to evolve and maintain the precision and quality our government customers expect.

This is an opportunity to be part of a small but ambitious team looking to scale rapidly throughout 2021 and 2022, with excellent career progression for the right candidate.

Candidates will:

- Be Security Cleared

- Have experience working in the private sector, building technical products for government customers in the field of open-source intelligence data solutions

- Have an in-depth understanding of the open-source information (social media) environment in particular the opportunities and challenges presented by such an environment

- Be excited, motivated and have knowledge of analytics and products in the national security space

- Comfortable working for Defence and Government customers in areas such as Disinformation, Terrorism and Extremism



Responsibilities

- Setup and maintain dedicated data pipelines including twitter and telegram

- Automate monitoring and evaluation of data quality

- Work closely with senior stakeholders to outline project scope and requirements

- With the Lead Engineer, be able to rapidly and professionally advise on feasibility of technical requirements in terms of data availability and appropriateness.

- Scale projects to ensure data is available as required

- Scope out new data collection opportunities and manage access

- Scope requirements and target architecture for project propositions

- Support the rest of the tech team in debugging, blockers etc

- Participate in daily standups and be able to prioritise tasks autonomously

- Architect data solutions for new projects and new technology

- Comfortable with being ’hands-on’ as well as manage the more strategic / technical aspects of code

- Have some experience in working with data scientists to productionise data science and AI models

- Professional attitude at all times

Essential

  • Security Cleared to SC

Technical

  • SQL
  • Python
  • Selenium
  • AWS
  • Data warehousing
  • Serverless functions (AWS Lambda)
  • Docker
  • AWS Glue (PySpark)

Desirable

  • Terraform
  • CI/CD pipelining
Loading ...
Loading ...

Deadline: 26-07-2024

Click to apply for free candidate

Apply

Report job
Loading ...
Loading ...

SIMILAR JOBS

Loading ...
Loading ...