Quality Assurance Handbook

Self-Evaluation, Monitoring, and Review

12.1 Internal Review, Monitoring, and Self-Evaluation Policy

12.1.1 Purpose

This policy outlines the processes in place to facilitate the ongoing monitoring and review of programmes and related services and the Western Management Centre (WMC) Quality Assurance (QA) Framework.

12.1.2 Policy Statement

This policy establishes available mechanisms that support the ongoing monitoring and periodic review of:

• Programmes and all related services and activities
• The WMC QA Framework

The availability and practical use of this framework provides valuable insights that are used to inform the continuous enhancement of WMC for the benefit of all stakeholders.

Through monitoring, review, self-evaluation, and action, WMC can ensure that effective quality and academic standards are identified, implemented, and maintained and that areas that require improvement are addressed. This process supports and develops stakeholder confidence in the academic qualifications offered at WMC. Monitoring and self-evaluation is based on a cycle of planning, implementation, review, and action.

The Centre utilises a Risk Register which identifies the exposure the Centre has to a range of risks and/or threats at any particular time. The Centre’s governance structure is designed to facilitate the ongoing monitoring of the impacts and to decisions on associated management responses depending on the level of risk which applies at a given point in time.

12.1.3 Responsibility of

This policy is the responsibility of the Registrar

12.1.4 Processes and Procedures

Monitoring

Monitoring is an ongoing internal Centre process that provides insights into areas of service provision that are delivering quality and areas that require planning and action for improvement.

Ongoing monitoring of programmes, essential to ensure academic quality and standards are maintained, is informed by ongoing review of learner and tutor feedback and other data gathered through review and monitoring processes.

WMC has developed a set of key performance indicators which the Centre systematically measures over time. Through the identification and systematic targeting of these processes, the Centre can pinpoint areas for improvement and best-practice and use these as yardsticks for positive change and continuous improvement. Insights are also used to inform the continuous improvement and development of the QA framework.

Monitoring occurs through the setting and evaluation of both ‘hard’ and ‘soft’ metrics. ‘Hard’ metrics provide the Centre with measurements that can be translated into numbers and evaluated. ‘Soft’ metrics provide valuable insights into attitudes.

Monitoring system design is also policy-driven, with the objective of meeting the fundamental policy goals of:

• Managing risks by ensuring financial, legal, and regulatory compliance
• Reducing costs by effective monitoring and audit
• Improving performance by setting and reviewing standards, benchmarks and targets

Ongoing monitoring is documented and used to inform the development of a Centre Improvement Plan. The Centre Improvement Plan defines a schedule of actions to be undertaken following internal evaluation. The plan also identifies the responsible person(s) for actions and follow-up. The ongoing process may result in:

• Changes to existing quality assurance policy or related procedure
• Centre operational adjustments (including administrative, managerial, service provision)

Self-evaluation

Self-evaluation is a key factor that enables WMC to effectively operate to a high level within a rapidly changing landscape and dynamic external environment. Self-evaluation processes and insights also provide a robust method of embedding quality enhancement resulting in a greater impact on learners, and continuous quality service provision. Through self-evaluation, WMC further acquires information to inform the internal development of the Centre and programmes of learning.

Self-evaluation is informed by:

• Dialogue with Centre stakeholders
• Insights gathered through ongoing monitoring and review
• The documented corrective actions reported

Self-evaluation has a broad, systemic focus and is conducted periodically. Self-evaluation at WMC has six stages:

• Review Terms of Reference and update if required
• Creation of Self-Evaluation (SER) Report
• Ratification of SER by Academic Council
• Panel review of SER
• Panel Report and recommendations
• Development of Centre Improvement Plan

Understanding that the process of self-evaluation may disclose interesting information and feedback relevant to the Centre’s stakeholders, WMC will disseminate as appropriate.

Monitoring and self-evaluation criteria

Monitoring and self-evaluation is completed with consideration of:

• Relevant, accurate and up-to-date data
• External factors which may impact WMC
• Feedback and information by the relevant academic or operational unit

Outputs may include (but are not limited to):

• Determining the most appropriate action required to enhance the learning and information environment (for example, programme update, quality assurance update, corrective action)
• Validating that actions are accurate, comprehensive and up to date
• Managing information and knowledge to meet changing needs to ensure that they align with ever-changing requirements
• Collating information for inclusion in the Centre’s Improvement Plan

Table 3 outlines information sources and mechanisms that contribute to effective monitoring, review, and self-evaluation.

Table 3: Information sources and Mechanisms Contributing to Monitoring, Review, and Self-Evaluation.

12.2 Programme Review and Revalidation Policy

12.2.1 Purpose

Programme review and revalidation focuses on the quality of, or impact of a programme’s suitability in meeting the purpose that the selected programme was originally designed for. Programme review and revalidation processes assess programmes in terms of their continued relevance to meet the needs of learners, employers, the labour market, and wider society. Where programme review occurs, this process will assess the suitability of aspects including programmes assessment strategies and creation of a supportive education setting. Programme review and validation may occur for programmes in isolation or for a series of related programmes.

12.2.2 Policy Statement

This policy outlines Western Management Centre’s (WMC) policy and procedures for periodic programme review which normally occurs on a five-year, cyclical basis as part of the Centre’s overall Programmatic Review and Revalidation procedures.

Programme review and revalidation is informed and supported by the regular review of programmes as part of the Centre’s ongoing monitoring processes (for more information, see Section 4.3, Programme Monitoring and Review Policy).

Programme monitoring and review is a means by which WMC reviews the:

• Effectiveness of programmes in the context of meeting learner needs and the creation of an effective learning environment
• Effectiveness of programmes in meeting the needs of employers, industry, the economy, and wider societal developments
• Effectiveness of programme content, teaching and learning methods, and delivery methods
• Accuracy and effectiveness of assessment procedures and learner workload
• Suitability of programme resources available
• Relationships with external stakeholders that impact on the relevance of programmes

Programme review allows for extensive changes to occur on a programme. Where a programme review results in the creation of a new programme, changes must be consistent with the applicable award-standard(s) against which the programme was validated. A transition programme would be introduced by the Centre to assist learners who may wish to transfer from their current programme to the new programme (assuming relevant credits were attained).

Revalidation refers to the QQI process of revalidating a programme that has evolved or developed from a programme that had been previously validated by FETAC/ QQI. The revalidation process is necessary for any programme(s) that wishes to continue to enrol learners following the expiry of the initial duration of enrolment. Where revalidation arises, WMC will plan this process in consultation with QQI in a timeframe that allows for continuity of programme scheduling. WMC recognises that both programmatic reviews and revalidations are significant projects which require planning and significant resourcing.

12.2.3 Responsibility of

This policy is the responsibility of the Registrar.

12.2.4 Processes and Procedures

Application for validation proceeds through three key stages (see Figure 15).

• Self-evaluation of programme(s)
• External peer review of programme(s)
• Application to QQI for validation and decision

Figure 15- Application for Revalidation – Key Stages

Self-Evaluation of Programme(s)

The Registrar is responsible for the creation of a project plan outlining the scheduling, the proposed approach to programme review, and key tasks in each phase of the review process. The project planning in relation to undertaking a programme review will consider the availability of tutors involved in the delivery of the specific programme(s) under review.

Working closely with QQI, the Registrar will agree the Terms of Reference of the review at a date agreed in advance of the commencement of the entire process. The Terms of Reference document is submitted with the project plan and details of the Project Review Team to QQI for approval.

The Programme Review Team work together to compile a self-evaluation report (SER). The creation of an SER is informed by analysis of data gathered during the ongoing monitoring of programmes and reports developed as part of each annual programme review (including the analysis of services, resources, and administration). The Programme Review Team evaluates data insights to determine the success of the programme(s) in meeting initial programme objectives and activities.

Once compiled the SER is presented to the Academic Council. Upon review and ratification by the Academic Council, the process moves to an external peer review phase.

Peer Review of Programme(s)

For peer review, a peer review panel is established. The panel is responsible to form independent judgment to determine the programme(s)’ success in maintaining the overall academic standard required. The peer review panel will consist of members drawn from internal ICE Group expertise and consist of members who have demonstrated and practical understanding of QQI’s awards standards at the relevant NFQ level/s. The Panel will also have experience in the relevant fields of learning and practical understanding in the context for the Centre’s programmes and in particular of the Further Education and Training (FET) sector. Panel members will be familiar with the culture and strategic objectives of WMC.

The peer review panel will examine the SER and consider any proposed programme changes in the context of all information outlined within the report. In addition, the Panel will consider the quality assurance framework relevant to the programme(s) under review.

The Registrar will schedule and communicate a meeting agenda that may include meetings between the Programme Manager, tutors, learners, and the above Panel. The agenda will also facilitate the opportunity for the Panel to review documentation and to hold private discussions.

The Panel will normally present its findings to senior management. Panel findings will determine whether the Panel recommends the programme(s) for further validation and any relevant recommendations for the Centre in respect of the programme(s) under review.
The Panel will compile a formal report of findings based on evaluation of the SER and findings as a result of the meeting with relevant Centre stakeholders. The report will make recommendations concerning the continuing validation of the programme(s) and specify the duration of recommended revalidation (not exceeding five years). The report will also detail any conditions or recommendations associated with the continued validation of the programme(s). The report is submitted to the Registrar for review.

Upon approval, WMC will prepare a project implementation plan that will address findings and recommendations outlined in the SER and Peer Review Panel Report.

Application to QQI for Revalidation and Decision

The Registrar will submit an application for revalidation to QQI adhering to all application criteria. The QQI Programme and Awards Executive Committee will determine final decisions based on validation policy and criteria and indicate the committee’s decision to approve or decline the revalidation application. In the event of a successful validation decision, relevant reports from the revalidation process will be published on the WMC and QQI websites.