SEMTA Standards Assessment Strategy

Introduction
[SEMTA], the Standards Setting Body for the Science Engineering Manufacturing Technologies sector, has taken into account the Qualification and Curriculum Authority (QCA) and Scottish Qualifications Authority (SQA) assessment criteria in producing this Assessment Strategy

[SEMTA] has produced this guidance to:

• endorse Awarding Body applications to award this qualification
• assist assessors, internal verifiers and external verifiers
• encourage and promote consistent assessment of this qualification
• promote cost effective assessment plans
• promote the use of tried and accepted ‘independent assessment’ methods

This guidance provides definitions for:

• the scope of the awards and the characteristics of typical candidates for this qualification
• the qualification structure
• the qualifications and experience required for Assessors and Verifiers of this award
• the assessment environment and standard of equipment that should be used
• access to this qualification
• the evidence required to support competent performance against the standards

and suggestions for:

• carrying out assessments
• performance evidence requirements
• assessing knowledge and understanding
• the arrangements for ‘external quality control of assessment’

The importance which employers and candidates place on having this qualification will provide a key measure of [SEMTA’s] success with this assessment strategy. Another key success factor will be [SEMTA’s] partnership with the relevant Awarding Bodies.

[SEMTA] will use the following guidance to continually improve its strategies for assessment of this qualification.

Scope of the Award
Evidence of competence must be assessed against the requirements of the relevant National Occupational Standards. For this award, the relevant standards are contextualised versions of the Engineering Competence Standards (ECRS).

The specific scope statements for this award are covered in an Appendix 2 where appropriate

Qualification Structure
The qualification structure for this award requires candidates to complete common mandatory units, followed by a choice of pathways. Candidates may then be required to complete further mandatory units within their chosen pathway, followed by a number of optional units from a provided selection. The range of optional units allows for any variations in the occupation in different organisations and across the sector.

The number of mandatory and optional units required to achieve this award, and the available pathways, is defined in the specific qualification structure. This outlines the minimum number of units that must be taken, but additional units can be taken to suit the requirements of particular organisations.

Key or Core Skills at the appropriate level may be taken as additional units where this is considered desirable.

Assessor Requirements
Assessment must be carried out by competent assessors who hold, or are working towards, the nationally recognised Assessor units A1 and/or A2 as appropriate to the assessment being carried out. Assessors that hold units D32 and/or D33 must demonstrate that they are applying the assessment principles and practices set down in A1 and/or A2 as appropriate to the assessment being carried out.

Assessors must be able to demonstrate that they have verifiable, relevant and sufficient technical competence to evaluate and judge evidence for this qualification. This will be demonstrated either by holding a relevant technical qualification or by proven industrial experience of the technical areas to be assessed. The assessor’s competence must, at the very least, be at the same level as that required of the candidate(s) in the units being assessed.

Assessors must also know:

• the content and meaning of the National Occupational Standards against which assessments are to be carried out
• the appropriate Regulatory Body’s system of vocational qualifications
• the relevant Awarding Body’s documentation and system of vocational qualifications within which the assessment is taking place.

Verifier Requirements
Internal verifiers must hold, or be working towards, the nationally recognised Internal Verifier unit V1 and would be expected to be familiar with, and preferably hold, the nationally recognised Assessor units. Internal Verifiers that hold unit D34 must demonstrate that they are applying the verification principles and practices set down in V1.

External verifiers must hold, or be working towards, the nationally recognised External Verifier unit V2 and would be expected to be familiar with, and preferably hold, the nationally recognised Assessor units, and possibly even the nationally recognised Internal Verifier unit. External Verifiers that hold unit D35 must demonstrate that they are applying the verification principles and practices set down in V2

Verifiers, both internal and external, will also be expected to be fully conversant with the standards against which the assessments and verification are to be carried out, the appropriate Regulatory Body’s system of vocational qualifications, and the relevant Awarding Body’s documentation and system of vocational qualifications within which the assessment and verification is taking place.

Specific technical requirements for internal and external verifiers
Internal and external Verifiers of this qualification must be able to demonstrate that have verifiable, sufficient and relevant industrial experience, and must have a working knowledge of the processes, techniques and procedures that are used in the engineering industry.

The tables on the following page show the recommended levels of technical competence for assessors, internal verifiers, and external verifiers.

The tables on the following page show the recommended levels of technical competence for assessors, internal verifiers, and external verifiers.

Note: These levels of technical competence were derived by a project carried out by members of the Awarding Body Forum on the continuous professional development (CPD) of assessors and verifiers. Some of the terminology has since been updated, and relevant terms have been defined in order to eliminate any possible ambiguities or areas that could be susceptible to different interpretations.

Assessment Environment
Evidence for this award should be obtained from the working environment where the work activities or work outcomes to be assessed are clearly attributable to the candidate. However, in certain circumstances, replication of work activities may be acceptable.

Where replication is considered necessary, assessors must be confident that the environment replicates the workplace to such an extent that competencies gained will be fully transferable to the workplace. In this case assessors must clearly identify those aspects of the workplace that are critical to performance, and make sure that they have been replicated satisfactorily. Where replication is involved, assessors must obtain agreement with internal and external verifiers before assessing any candidates.

Examples of critical aspects could be:

• environmental conditions such as, noise levels, lighting conditions and the presence of hazards
• the use of industrial equipment and procedures
• pressure of work such as time constraints and repetitive activities
• carrying out work on actual work pieces and the consequences of making mistakes
• customer/supplier/departmental relationships

Access to Assessment
There are no entry qualifications or age limits required for these qualifications unless this is a legal requirement of the process or the environment. Assessment is open to any candidate who has the potential to reach the standards laid down for this qualification.

Aids or appliances, which are designed to alleviate disability, may be used during assessment, providing they do not compromise the standard required.

Carrying Out Assessments
[SEMTA] strongly recommends that the majority of assessment evidence for the mandatory units is gathered during the performance of the optional units. Evidence should be obtained as a whole, where practically possible, since competent performance in the optional units is often dependent on competence in the mandatory units. Although it is possible to achieve this qualification with the minimum number of optional units, organisations may wish their candidates to be assessed for more than this.

Where Key or Core Skills are required, these may be included as additional units and assessed in conjunction with the mandatory and optional units, where this is appropriate.

The standards were developed to cover a range of activities. The evidence produced for this award will, therefore, depend on the candidate’s choice of ‘scope’ items in the standard, which are intended to help the candidate to seek the appropriate information and to acquire the necessary skills, techniques and knowledge before being able to demonstrate competent performance.

Where the scope section gives a choice (for example ‘any three from five’), assessors should note that candidates do not need to cover the other (in this example, two) items, particularly where these additional items may relate to other activities or methods that are not part of the candidate’s normal workplace activity or area of expertise.

Performance Evidence Requirements
Performance evidence must be the main form of evidence gathered.

In order to demonstrate consistent, competent performance for a unit, a minimum of three different examples of performance evidence must be provided, and must be sufficient to show that the performance requirements of the unit have been carried out to the prescribed standards. The minimum number of items specified in each of the scope statements for a unit (eg, four from a choice of six) must all be covered, and appropriate evidence provided. It is possible that some of the scope items may be covered more than once. If, however, the three examples of performance evidence are not sufficient to cover all the specified scope items, then further examples of performance evidence will be required to ensure this coverage is achieved.

The most effective way of assessing competence, especially for the performance statements in relation to scope items, is through direct observation of the candidate. Assessors must make sure that the evidence provided reflects the candidate’s competence and not just the achievement of a training programme.

Evidence that has been produced from team activities (eg, maintenance and installation) is only valid when it clearly relates to a candidate’s specific and individual contribution to the activity, and not to the general outcome(s).

Items of performance evidence often contain features that apply to more than one unit, and can be used as evidence in any unit where appropriate.

Performance evidence must be a combination of:

• products of the candidate’s work, such as items that have been produced or worked on, and documents produced as part of a work activity

together with:

• evidence of the way the candidates carried out the activities such as witness testimonies, assessor observations or authenticated candidate reports, records or photographs of the work/activity carried out, etc.

Competent performance is more than just carrying out a series of individual set tasks. Many of the units contain statements that require the candidate to provide evidence that proves they are capable of combining the various features and techniques. Where this is the case, separate fragments of evidence would not provide this combination of features and techniques and will not, therefore, be acceptable as demonstrating competent performance.

If there is any doubt as to what constitutes suitable evidence, the external verifier should be consulted.

Assessing Knowledge and Understanding
Knowledge and understanding are key components of competent performance, but it is unlikely that performance evidence alone will provide enough evidence in this area. Where the candidate’s knowledge and understanding (and the handling of contingency situations) is not apparent from performance evidence, it must be assessed by other means and be supported by suitable evidence.

Knowledge and understanding can be demonstrated in a number of different ways, but it is suggested that the most appropriate methods for this qualification are oral questioning and practical demonstrations. Assessors should ask enough questions to be able to determine that the candidate has an appropriate level of knowledge and understanding as required by the unit.

Evidence of knowledge and understanding will not be required for those scope items that have not been selected by the candidate.

The achievement of the specific knowledge and understanding requirements of the standards cannot simply be inferred by the results of tests or assignments from other qualifications or training programmes. Where evidence is submitted from these sources, the assessor must, as with any assessment, make sure the evidence is valid, reliable, authentic, directly attributable to the candidate, and meets the full knowledge and understanding requirements of the standard.

Where oral questioning is used the assessor must retain a record of the questions asked, together with the candidate’s answers.

Awarding Bodies may choose other methods, which must be supported by a suitable rationale.

External Quality Control of Assessment
[SEMTA] has considered the various methods of external quality control of assessment that could be used to assure the quality of assessment of this qualification, taking into account the complexity and diversity of the workplace, commercial acceptability, financial constraints and the Regulatory Body’s criteria.

It is considered that a method of enhanced external verification, coupled with the introduction of a comprehensive strategy for sampling assessment records, will give the most reliable measure of assessment quality. The implementation of this is described in [SEMTA’s] ‘External Quality Control of Assessment’ (EQCA) model (see Appendix 1 for Flow Diagram and explanatory text).

In view of this, [SEMTA] recommends the following strategy for external quality control of assessment:

1. The mechanism of external quality control of assessment will be the application of enhanced verification and the introduction of a comprehensive strategy for sampling assessment records, following [SEMTA’s] EQCA model
2. The Standards Setting Body (SSB) will develop and publish the necessary external assessment criteria for sampling the assessment records
3. The Awarding Body Forum will agree how the criteria are to be used nationally, in line with the published model
4. External verifiers will use the SSB published criteria as a reference document when sampling candidate evidence and assessment records. This will give the SSB a comprehensive and standardised national strategy for sampling and comparing assessment decisions
5. Implementation of the strategy will be through the normal assessor and verifier process, and will take into account any candidates with special needs.

[SEMTA] will require information from Awarding Bodies on the implementation and performance of the EQCA model for future development of external quality control of assessment.

Current indications are that the above methods of external quality control of assessment are proving acceptable and workable within the engineering industry. However, under [SEMTA’s] policy of continuous improvement, the EQCA model will be periodically reviewed, and other methods of external quality control, and the feasibility of using them within the engineering industry, will be explored.