Overview

This occupation is found in a wide range of public and private sector organisations who work with large data sets including Government departments, NHS, financial and professional services, IT companies, retail and sales and education providers. The purpose of the occupation is to build systems that collect, manage, and convert data into usable information for data scientists, data analysts and business intelligence analysts to interpret. A data engineer’s main aim is to make data accessible and valid so that an organisation can use it to evaluate and optimise their performance. The role of the data engineer is pivotal to any organisation; it ensures that data pipelines are established to support data scientists and other business stakeholders.

A data engineer will build and implement data flows to connect operational systems, and re-engineer manual data flows to enable scalable and repeatable use. They integrate, support and manage the build of data streaming systems, writing extract transform and load scripts that perform in line with business requirements.

They are responsible for providing high quality, transparent data that enables effective governance and smart business decisions. They will analyse the performance indicators of the data systems that provide clean, regular, and accurate data. A data engineer will understand how data and an organisation’s data architecture is essential to business outcomes.

A data engineer will be able to gather requirements for data solutions, and they demonstrate and articulate data solutions to stakeholders in a way that can be easily understood. Data engineering encompasses a range of activities from collecting data to employing various data processing frameworks, including but not limited to ETL (Extract, Transform, Load), and collaborating with data scientists and other data-centric roles. Data engineers may work in an office or work remotely depending on the sector they work in and location of the employer. In their daily work, an employee in this occupation will work autonomously or collaboratively with clients, in the business and or data team. A data engineer will work with data analysts, Data scientists and data architects and liaise with other teams and internal and external stakeholders to ensure their data requirements are captured and managed to the specified standard. They will also work closely with machine learning engineer (Ops), software engineers, software developers and technology teams.

An employee in this occupation will be responsible for completing their own work to specification, , ensuring they meet set deadlines. A data engineer contributes towards, engineering designs, plans, execution and evaluation working to time, cost and quality targets. They deliver to the product roadmap and are responsible for meeting quality requirements and working in accordance with health and safety and environmental considerations. They will work according to organisational procedures and policies, to maintain security and compliance.

Foundation Apprenticeship Assessment

Each assessment organisation is responsible for designing its own assessments in line with the apprenticeship standard. The Academy for Project Management has developed an assessment model that allows apprentices to demonstrate competence against the required assessment outcomes. Comprehensive guidance and resources are provided to support both apprentices and training providers.

The assessment for the Data Engineer includes:

• At least a project

And the choice of a:

• written assessment
• presentation
• question and answer
• professional discussion
• portfolio
• project

The standard also includes Employability Skills and Behaviours (EBs), which must be demonstrated consistently by the apprentice throughout the programme. Employers are required to formally confirm this before completion.

Assessment Timing

Components of the assessment may be scheduled at appropriate points during the apprenticeship. The Academy for Project Management works closely with training providers to ensure assessments are timed effectively, allowing apprentices to demonstrate their skills and knowledge at key stages in their development.

Additional information on the apprenticeship standard and the assessment process can be viewed here.

Becoming a Centre

Training providers interested in delivering elements of the assessment internally can explore the Centre model with us. Guidance is available on how the model operates and the steps required to become an approved Centre. For more information, please contact administration@academy4pm.com