Databricks and Scala Engineer geospatial

Job content

This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven track record as a senior/self-starting data engineerin implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies, specifically SPARK performance uningoptimisation and Databricks , to play an important role in developing and delivering early proofs of concept and production implementation. Your main responsibilities will be: Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using Databricks and Spark/Scala Streaming and Batch processes in Databricks Providing technical guidance for complex geospatial problems and spark dataframes Developing scalable and re-usable frameworks for ingestion and transformation of large data sets Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times Working with other members of the project team to support delivery of additional project components (Reporting tools, API interfaces, Search) Evaluating the performance and applicability of multiple tools against customer requirements Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints. SPARK performance uningoptimisation Qualifications Direct experience of building data piplines using Azure Data Factory and Databricks Spark using Scala Fluent in Scala, Python, Java Experience working with structured and unstructured data including imaging & geospatial data. Experience of working with relational databases: (SQL Server, PostgreSQL) Additional Information At Version 1, we believe in providing our employees with a comprehensive benefits package that prioritizes their well-being, professional growth, and financial stability. One of our standout advantages is the ability to work remote or with a hybrid schedule along with business travel, allowing our employees to strike a balance between work and life. We also offer a range of tech-related benefits, including an innovative Tech Scheme, Octopus Car Lease, and MyGym Discounts to help keep our team members up-to-date with the latest technology. We prioritize the health and safety of our employees, providing private medical and life insurance coverage, as well as free eye tests and contributions towards glasses. Our team members can also stay ahead of the curve with incentivized certifications and accreditations, including AWS, Microsoft, Oracle, and Red Hat. Our employee-designed Profit Share scheme divides a portion of our company’s profits each quarter among all full-time employees. We are dedicated to helping our employees reach their full potential, offering Pathways Career Development Quarterly, a program designed to support professional growth.