National Psychometrician and Assessments Expert
Reporting to: Team Leader (PESP III)
Location: Full time based in Lahore
Background to the PESP III Technical Assistance
The Third Punjab Education Sector Project (PESP III) is a government-led programme supported by the World Bank (WB) with a loan of $300 million and the provision of technical assistance (TA). The implementing agency for PESP III is Programme Monitoring and Implementation Unit (PMIU) within Schools Education Department (SED) Punjab. The project strategic areas are broken down into nine sub-components, each with an associated objective. Progress in each of these sub-components is measured against annual targets linked to a Disbursement Linked Indicator (DLI). The World Bank is supporting PESP III with a tranche of TA and the PMIU signed the contract with Mott MacDonald (trading as Cambridge Education) to provide TA service in October 2018.
Its objective is to support Government of Punjab (GoPb) to improve school participation, completion and teaching-learning practices with a focus on low-performing districts. The Project has nine disbursement-linked indicators (DLIs) each of which have annual targets over the five-year Project period (Sept 2016 to June 2021). The project is providing a series of coordinated interventions to the School Education Department (SED), Project Monitoring and Implementation Unit (PMIU), Quaid e Azam Academy for Educational Development (QAED), Punjab Education Foundation (PEF), Punjab Examination Commission (PEC) and other relevant institutions in policy, planning, teacher training, examination and assessment, teaching and learning materials, third party validations, impact evaluations and more.
The National Psychometrician and Assessments Expert will work closely with the DLI 2 and 7 Workstream Manager, the short-term international Assessment expert and psychometrician, as well as PEC, PEF and the client – PMIU to:
- Design and execute the plan for support to be provided to PEC and PEF trainees of the capacity building programme on Item writing and test development (IWTD) as well as Psychometric Statistical Analysis Group (PSAG), including planning, implementation and after training tasks.
- With the TL and PESP III Assessment team lead discussions on the restructured TA support plan for PEC after the approval of the Assessment Policy Framework (APF).
- Lead on the development, approval and execution of the Institutional Strengthening Plan for PEC
- Lead on the implementation of the technical support to be provided to PEC as per the agreed PESP III Restructured TA plan for PEC.
Responsibilities of the National Psychometrician and Assessments Expert
The resource would be the technical lead on all our work on assessments across the two DLIs 2 and 7. S/he will be expected to take on the following responsibilities, working in close collaboration with lead international assessment experts and the PESP III Team Lead and Project Management team;
National Psychometrician and Assessments Expert
- Have technical oversight over the work of the DLI 2 and 7 Work Stream Manager.
- As the technical lead, be the first reviewer of the technical outputs produced by the assessments team; including work of the international experts and DLI 2 and 7 WSM
- As the technical expert, where appropriate, lead on key meetings with the client, government counterparts, World Bank and other stakeholders.
- With the DLI 2 and 7 WSM and PM, prepare the ToRs for the visits of the International Consultants and ensure they deliver quality inputs on time.
- In collaboration with the WSM, ensure client relations are being managed. With all complaints, issues, requests from the clients escalated in a timely manner to the PESP III leadership and management.
PEF and PEC Capacity Building Exercise
6.Preparation and Implementation of training – In collaboration with the lead assessment expert and international psychometrician, lead technically on (i) Item Writing and Test Development and (ii) Psychometrics Statistical Analysis Group. This would include but not limited to technical oversight of;
- Development of training plan, modules and selection of materials and training format for both trainings.
- Conducting the cycle 2 Capacity Building trainings PEF and PEC nominated staff – slated for April 2020
- Capturing participant/client feedback and proactively tailoring the training materials format and methodology in order to ensure best value for all.
- Developing and implementing training evaluations instruments to capture evidence of capacity development.
7. After the training – In collaboration with the lead assessment expert and international psychometrician;
- Provide hands-on practices to PEC and PEF staff for item analysis and test writings
- Provide hands-on practices to PEC and PEF staff for drawing samples and test result analysis
- Ensure the continuous professional development of a core group of PEF and PEC trainees in the skills and practices taught in the IWTD and PSAG trainings.
- Ensure a core group within the PEC and PEF teams is continuously using the SAS, BILOG and WesVar softwares for item and test result analysis, test administration, scoring and reporting.
- Develop report of piloting of statistical softwares
- Provide the necessary handholding and support to the PEC and PEF teams in applying learnings from IWTD and PSAG workshops, including; review of work done, giving timely input and feedback, troubleshooting and referring to international consultants where necessary.
- Maintaining a record of the structured support provided to the core group of trainees in PEF and PEC between training cycles.
- Developing tools to capture and document evidence of application and improvement in capacity of the trainees.
- Ensure the technical quality and analytical rigour of the capacity building and training program reports for each year to be submitted the international experts.
Technical Work with PEC and PEF – APF, ISP (PEC) and support to PEF
Working in close collaboration with PESP III Team Lead, PM and assessments team;
8. If required, support the approval of the Assessment Policy Framework and development of its Implementation Strategy.
9. Lead on the work on the development and approval of the Institutional Strengthening Plan (ISP) and its implementation.
10. Lead on the discussion to draft PESP III’s restructured TA plan to PEC. With the plan agreed by all parties and signed off by the client.
11. Ensure restructured plan captures significant level of detail, down to resources committed, work plan, list of activities and deliverables with specifications.
12. Review PEC and PEF’s existing item bank, item development, test construction, scoring, data analysis, reporting etc.
13. Lead on and review the final report on guidelines and processes for Item Development, QA and update of items and question banks by PEC and PEF.
14. Review operational tests drafted by PEC and PEF staff for upcoming examinations (schools based, large scale sample based or QAT)
15. Piloting, preparation and implementation of enhanced templates for analysis and reporting of results.
The National Psychometrician and Assessments Expert is line managed by the Team Leader of PESP III and technical oversight of work is by the Project Director and PESP III National Assessment expert. Management and support on contractual matters is by the PESP III PM.
Candidate must have;
– Masters or equivalent qualification in a relevant programme i.e. education policy, public policy, education, social sciences etc.
– Has up to 15 plus years of experience in working in the education sector in Pakistan, particularly on assessments, policy implementation and/or delivery of systemic change in large education systems.
– Knowledge and experience of working on statistical soft wares
– Has experience and the communication skills to lead and present in meetings to high profile stakeholders in the government, donor agencies and private sector.
– Excellent skills to engage, find common ground and create alignment in the work of different players such as government counterparts, donor and the PESP III TA team.
– Excellent communication skills – able to readily communicate relevant information, feedback intelligence and keep all stakeholders engaged and updated.
The successful candidate will be paid a competitive market salary commensurate with their qualifications and experience.