Senior Data Engineer
Are you a passionate software developer with a keen eye for detail and engineering elegance? Do you love solving problems, innovating and developing ideas? Are you proud of past work where you have delivered high quality, performant, and maintainable software solutions?
Operations Research/ Decision Analytics Team (OR/DA) are looking to expand our team with a talented and experienced Senior Data Engineer to work on building automated data pipelines for the Operations Research and Decision Analytics function. We are part of a centralised business group which runs and delivers the platforms, tools, infrastructures and workflows that are used by the Arm product development teams.
The OR/DA team works with data to add business value by applying a mixture of Data Engineering, Data Visualisation and Data Science disciplines to a variety of complex engineering problems. We extract information from large data sets to enable data driven decisions and improved efficiency and effectiveness of use of engineering platforms and infrastructure
You will be an experienced software engineer with experience of Spark, Python, databases and/ or equivalent data engineering tools, and keen to pursue the problems of extracting knowledge from data and driven to understand the core 'stories' that data can tell. You will help drive efficiency and effectiveness improvements by providing the means to make data driven decisions.
* You will take ownership for the development and deployment of our ETL data engineering process, that allows relevant data to be centrally collected, aggregated, and processed
* You will work with engineering and IT functions to ensure sufficient relevant data is being gathered to provide the best information for engineering platforms like HPC, Storage, EDA, Emulation, FPGA's..
* You will have continuous communication with the organisation for changing priorities and improvements needed for development of the service in the area of data engineering
Your tasks will typically include
* ETL Code Development Data source exploration, extract, transform and load engineering data from various sources for visualisation
* Build and deploy ETL workflows for continuous data delivery to the dashboards making sure it is robust and scalable
* Follow software engineering standards Testing analytics products for robustness Source Control, Code Testing, Verify Code Logic and Data integrity, Debugging and Review of Code and Data anomaly.
* Enhance data collection and data processing methodologies.
* Collaborate with the other team members who use various statistical and Machine Learning models to build predictive analytical products for our Engineering and Platform Operation teams
Required Skills And Qualifications
* A good university degree in computer science/software engineering or equivalent life experience and knowledge.
* You will consider yourself to have extensive experience and a track history of delivering complex software projects on schedule and within budget
* Sound understanding of software development process and practices such as coding standards, code reviews and version control systems for software build and test solutions such as Jenkins, TeamCity, Bamboo
We would be even more impressed if you have had experience/ passionate about the following
* Exposure to large scale clustered compute environments (LSF) and Storage Architecture with some exposure to cloud services
* Knowledge of SQL, NoSQL, Exasol, Linux, Docker, Mesos, etc.
* NoSQL distributed databases, such as Scylla, Cassandra
* Stream processing framework for distributed, always-available data streaming applications like Kafka, Flink
* Data analysis and visualisation using Hadoop, Elasticsearch, Kibana, Tableau D3.js, etc.
* An understanding of virtualisation and cloud technologies such as Docker, Vagrant, OpenStack, Kubernetes etc.
* Knowledge of working with cloud services like AWS (S3, Kinesis, EMR, Redshift, Glue, Lambda etc..)
* Knowledge of data preparation techniques for machine learning
Additionally, you will make the biggest impact in this role if you are
* Highly inventive
* Passionate about making a difference
* Ability to work independently and in teams; working in collaboration with engineers/ scientists and business partners.
* Excellent communication skills; able to rationalise your approach and influence
Arm has a responsibility to ensure that all employees are eligible to live and work in the UK.