G7 Lead Psychometrician

Job content

You’ll create psychometrically robust assessments and the psychometric aspects of integration with our main candidate management systems. This will include everything from developing new content and conducting subject matter expert reviews to content trialling and implementing test products for your customers.

Managing the day-to-day availability and integrity of the test player and analytical function, you’ll work alongside a multi-disciplinary digital team in an agile project management environment and contribute to developing best professional practice.

The Lead Psychometrician will lead the building and testing of prototype expert test delivery systems and the psychometric aspects of integration with our main candidate management systems.

They will develop new content, carry out subject matter expert reviews and content trialling and lead the assembly and implementation of test products with our departmental customers – using a combination of surveys, item trialling, workshops and one to one user research.

They will research and analyse new innovations and products, carrying out in-depth statistical evaluations. The Lead Psychometrician will act as a subject matter expert and represent the service line professionally with a wide range of customers and stakeholders.

They will manage the day to day availability and integrity of the test player and analytical function to ensure delivery is according to agreed standards. The Lead Psychometrician will work very closely with a multi-disciplinary digital team operating an agile project management approach, which is based in Bristol. Occasional travel within the UK will be a requirement of the role.

The Lead Psychometrician will be a member of a wider team of professional psychologists and will be expected to contribute to the development of professional practice and work collaboratively with colleagues on a wider portfolio of activities when the need arises.

The Lead Psychometrician reports to the Principal Psychologist responsible for online tests and assessments in the Occupational Psychology Service, who reports to the Deputy Director of the service.

Key Responsibilities Include

Assessment Design, Trialling and Evaluation

• Manage test availability, integrity and performance against agreed service standards
• Design and conduct psychometric and statistical analysis using CTT and IRT methods
• Design, conduct and manage research projects, trials and studies
• Act as a subject matter expert for recruitment tests and assessments
• Manage the design, trialing and delivery of content and assembly of tests and assessments
• Work in an agile way with digital colleagues in the design of test integrations, the user interface and test assembly, staging and implementation
• Lead the psychometric implementation of accessibility standards
• Manage item banks to ensure they meet psychometric/statistical needs
• Develop and implement specifications and standardised procedures in test design and form assembly to support the publication process
• Manage and deliver user research, job analysis, online surveys and customer needs engagement initiatives
• Oversee the integrity of tests and assessments to provide accurate item and candidate performance data to support test construction work
• Conduct advanced statistical and psychometric forensic analyses to enhance test security measures
• Present and communicate analytical/research findings and proposals to customers, stakeholders and others, orally and in writing
• Lead initiatives to improve the user journey
• Line management of SEO Psychometric Analyst
• May involve some management of administrative staff
  

Training & development

• Keeping up-to-date with the latest innovations in psychometrics and in particular recruitment testing;
• Developing standard tools for psychometric analyses and providing training to the wider team in their use.
  

Consultancy and Research

• Providing professional psychometric advice and guidance to clients and senior leaders to ensure adherence to industry best practice and the highest data ethical standards;
• Representing the Occupational Psychology Service Team at working groups, committees and other forums, within Civil Service HR (CSHR) and across government to inform policy and operations;
• Researching new and emerging assessment methods/processes; setting up and conducting detailed studies to explore their impact and efficacy.
  

Responsibilities

Person Specification

You’ll be reporting directly to the Head of Online Tests and Assessments. You will work with a variety of clients, departments and professions across government. You will need to be able to demonstrate flexibility, the ability to deliver at pace and be someone who your team can really rely on.

You’ll need to demonstrate real passion for applying psychometric expertise and evidence based practice. Finally, we are a forward-looking team and so you will need to be too. You’ll embrace change and be eager to learn from it to develop and strengthen your role.

Do you see yourself as a talented individual that can achieve all this, progressing yourself and our team to the next level?

As well as a degree in a numerate discipline with strong statistics/psychometrics background, you’ll need graduate training and experience in psychometrics, measurement theory, test development, validation, item bias and standard setting. You’ll also be well versed in psychometric methods, modern statistical and psychometric theories, measurement principles, test design, and validation.

We’re looking for expertise in test development and psychometric methods, along with a good grasp of data analysis for CTT and IRT test analysis. Experience in managing large-scale research and testing programmes is also essential.

If you have a Masters in either Educational Measurement & Statistics, Quantitative Research Methods, Industrial/Organizational Psychology, Psychometrics, or highly-related discipline, so much the better.

In order to be successful in this role you will:

• Excellent oral/written and interpersonal communications, leadership and management skills for multi-functional projects and operations
• Ability to combine theory-based, academic and empirical experiences of developing new research and technical reports
• Ability to work with various stakeholders, committees and governing boards, vendors, consultants, and business partners of different backgrounds
• Ability to focus on technical details while maintaining high level perspectives; and to work with tight schedules and multiple high priority projects
• Must enjoy working in a team environment, be intellectually curious, conscientious, and motivated to work to continually improve the quality and validity of programme deliverables
• Agile project management skills are required to effectively manage multiple project assignments, timelines, deliverables, and milestones in accordance with Government Digital Services standards
  

Essential Knowledge, Skills And Experience

• Degree educated in a numerate discipline with strong statistics/psychometrics background
• Graduate training and experience in psychometrics, measurement theory, test development, validation, item bias, standard setting
• Knowledge of SPSS, survey packages, MS Office/Google Docs are required
• Must be well versed in psychometric methods, modern statistical and psychometric theories, measurement principles, test design, and validation
• Expertise in test development and psychometric methods
• Significant experience working in operational delivery environments where high stakes assessment is carried out using computer-based testing
• Expertise in data analysis for CTT and IRT test analysis
• Good knowledge of Differential Item Function (DIF) analysis
• Experience with managing large-scale research and testing programs, either of norm trialling, volume recruitment testing, certification and/or licensure applications is required
• Registered Test User, BPS or willing to work towards this qualification
  

Desirable Experience And Knowledge

• Masters in either Educational Measurement & Statistics, Quantitative Research Methods, Industrial/Organizational Psychology, Psychometrics, or highly-related discipline
• Solid understanding of WCAG 2.0 standards in relation to user interfaces and test design
• Experience of working in a large, complex organisation with many approval controls, moving parts and stakeholders
• Experience in use of modern measurement models such as cognitive diagnostic models (CDM) or linear-on-the-fly models (LOFT)
• Good knowledge of using the R programming language for statistical computing and graphics
  

Where will I be based?

Roles can be based across all Cabinet Office locations, including: Glasgow, Belfast, Cardiff, Edinburgh, York, and London. There will be some scope for home working across the week.

Eligibility

Applicants must be a British Citizens, citizens of the British Commonwealth, EC nationals or nationals of specified Member States of the European Free Trade Area.

Development Opportunities

Civil Service HR is committed to the development of staff. All employees are encouraged to undertake a minimum of 5 days learning per year. Training in psychometric instruments, and necessary online assessment platforms that are frequently used will be provided as appropriate.

The candidate will be required to undertake BPS Test User training if they do not currently hold this qualification.

For further details, please see the Candidate Pack (attached)