Assessing the Quality and Relevance of Competency-Based Training

Assessing the Quality and Relevance of Competency-Based Training

Sisay Abera Belayneh, Prof Houjun Qi,

Mechanical Design and Automation,

Tianjin University of Technology and Education, 1310, Dagu south Rd, Hexi, Tianjin, China

Prof Doing Xianhui

Tianjin University of Technology and Education, 1310, Dagu south Rd, Hexi, Tianjin, China

Abstract

This study aimed to explore the issues and practices related to quality training at Dire Dawa Poly Technical College, specifically about competence-based training. The participants in the study consisted of 105 trainers and 120 graduate trainees. College Deans, Vice Deans, and registrar heads were also purposefully selected and included in the study. Three different but complementary methods were utilized to gather data from key informants in the study setting: questionnaires, semi-structured interviews, and document analysis. The data collected were analyzed using both qualitative and quantitative analysis approaches.

The findings revealed that there is a lack of adequate and relevant labor market information, insufficient training needs assessment, limited tracer study results, inadequate practical competence among leaders and trainers, and weak stakeholder participation. The study showed that the transfer of relevant technical skills, the study recommends enhancing the competence of leaders and trainers through education and training. The study revealed that emphasizes the importance of stakeholder involvement in designing, implementing, and formulating an internal quality assurance system to sustain training quality in the College.

Keywords: Competency Based Training, quality, TVET,

leaders, trainers, stakeholders, participation

INTRODUCTION

UNESCO and ILO (2002, p.7) propose that Technical and Vocational Education and Training (TVET) encompasses various elements of the educational process. The purpose of TVET is to prepare trainees for specific jobs or types of work, focusing on practical and procedural activities. Its goal is to enable trainees to meet the needs of employers for qualified labor and to fulfill their own production- related needs for goods and services. "Skills training" generally refers to the development of qualifications within a narrower scope and volume of training, often concentrating on the performance of a single task (such as

operating a specific machine) or a limited set of tasks (such as various types of welding) (UNESCO, 2009).

TVET is distinguished by its focus on preparing students for the workforce and equipping them with employable skills, as noted by the AU in 2007. The delivery systems for TVET are designed to produce skilled and entrepreneurial workers who can contribute to a country's economic development and help alleviate poverty. Additionally, TVET can be tailored to meet the needs of various industries and diverse learners from different socio-economic and academic backgrounds, providing them with the necessary training for sustainable livelihoods. A skilled workforce is a fundamental requirement for industrial and economic growth, and TVET plays a crucial role in developing such a workforce. Despite the challenges faced by many African countries, including weak national economies, low job growth, high population growth, and labor force, there are efforts being made at a subliminal level to advance TVET, according to UNESCO in 2001.

TVET is focused on preparing students for the workforce and providing them with employable skills, as noted by the AU in 2007. Its delivery systems are designed to produce skilled and entrepreneurial workers who can contribute to a country's economic development and help alleviate poverty. TVET can be customized to meet the needs of various industries and diverse learners from different socio-economic and academic backgrounds, providing them with the necessary training for sustainable livelihoods. Developing a skilled workforce is essential for industrial and economic growth, and TVET plays a crucial role in this process. Despite the challenges faced by many African countries, there are efforts being made at a subliminal level to advance TVET, according to UNESCO in 2001.

According to UNESCO (2005), quality education is essential for sustainable development as it is a tool for growth. Delors et al. (1996) identified the four pillars of

Education for All (EFA) as learning to know, learning to do, learning to live together and with others, and learning to be. Quality education develops knowledge, life skills, perspectives, attitudes, and values, which enable contemporary societies to transform into more sustainable societies.

To provide high-quality skills training, appropriate equipment and tools, sufficient training materials, and practice by learners are necessary. Relevant textbooks and training manuals and qualified instructors with experience in enterprises are also required (AU, 2007). However, well-qualified instructors with industry-based experience are scarce since they are also in high demand in the labor market. They could be motivated to offer part-time instruction in technical and vocational schools. Meeting all these requirements is essential for quality training.

Competency-based training (CBT) is a learning approach that emphasizes the acquisition of precise skills and knowledge necessary for a particular job or duty. To create effective training programs, it is crucial to identify the required competencies or skills. Learners must demonstrate mastery of specific learning outcomes or abilities to complete the program successfully. This method is commonly used in vocational and technical training programs, although it is applicable in various learning settings. Competency-based training can raise the standard. In reality, CBT involves coaching and active learning. The formal technical and vocational education system must adopt the CBT approach and guiding principles. The development of effective management and leadership skills, as well as an appropriate credentials framework and monitoring mechanism to guide the entire system, are all crucial for the delivery of high-quality TVET.

Competencies, however, often represent more than knowledge and skill levels: they require the effective applications of existing knowledge and skills in

PURPOSE AND SIGNIFICANCE OF THE STUDY

The purpose of the study is to evaluate the effectiveness of the implementation of the ETQF in DDPTC, specifically in terms of practical work opportunities for students, instructor engagement and assessment practices, competency-based training implementation, assessment techniques, training process monitoring system, and factors influencing competency-based training. The study aims to identify areas for improvement and provide recommendations for enhancing the quality of TVET education and training in Ethiopia.

Despite training a substantial number of individuals over the past decade, DDPTC faces challenges in ensuring the quality and competence of its graduates in the Dire Dawa

certain contexts (Egodawatte 2014). Between learning outcomes and competencies, "substitution" can also be recognized. Competencies, according to Klein-Collins (2013), are more than just learning outcomes because they can be measured, supported by proof of student accomplishment, and used in a variety of contexts. The terms mastery-based education, proficiency-based education, and performance-based education are all interchangeable with CBE (Le, Wolf, and Steinberg 2014; Patrick, Kennedy, and Powell 2013). CBE is also used in conjunction with approaches like student- centred learning and personalized learning. Despite the fact that many definitions of CBE share a lot of similarities, there isn't a singular definition that academics have agreed upon (Kelchen 2015; Gervais 2016; Evans, Landl, and Thompson 2021). This study's foundation is the lack of consensus over the definition of CBE, which highlights the significance of the research.

The government of Ethiopia (FDRE) has made significant efforts to promote technical and vocational education, resulting in the establishment of approximately fourteen TVET institutions across the country. These institutions are not centralized in one location, but rather dispersed throughout various jurisdictions to ensure that each region has its own TVET institution. This approach has provided greater accessibility for individuals seeking technical and vocational education opportunities throughout Ethiopia. In 2007, the ETQF (Ethiopia TVET Qualification Framework) was introduced by the government to standardize and qualify TVET institutions in Ethiopia. Its implementation is intended to improve the quality of education and training provided. The ETQF prioritizes an outcome-based TVET system, as emphasized in a report by ecbp in 2006. The framework places great importance on both high-quality education and the practical relevance of training programs, ensuring they are aligned with the demands of the labor market.

administrative region of Ethiopia. While there is no shortage of diploma and certificate holders in various specializations, doubts persist regarding their actual qualifications and readiness for professional roles. These concerns stem from observed weak performance among students within the institution, which indicates deficiencies in the quality of education provided. Therefore, the problem at hand is to assess the issues of quality education in DDPTC in relation to competence- based training. The research aims to investigate key aspects such as the availability and emphasis on practical work, the engagement and assessment practices of instructors, the implementation of competency-based training, the effectiveness of assessment techniques, the

existence of a monitoring system for the training process, and the factors influencing competence-based training in terms of student interest and success. By identifying and addressing these issues, the study aims to contribute towards enhancing the quality and relevance of education in DDPTC, ultimately ensuring that graduates are adequately prepared for their chosen professions.

The study holds significant importance in several aspects. Firstly, it aims to improve the overall quality of education in DDPTC by identifying issues and providing recommendations for enhancement. Secondly, the findings will help align the training programs with the needs of the

OBJECTIVES OF THE STUDY

The general objective of this study is to assess the issues of quality education in DDPTC, with a specific focus on competency-based training. The research aims to investigate practical work availability, instructor engagement, and assessment practices, implementation of competency-based training, the effectiveness of assessment techniques, the existence of a training process monitoring system, and factors influencing competency- based training in student interest and success. By addressing these areas, the study aims to enhance the overall quality and relevance of education in DDPTC, ensuring that graduates possess the requisite skills and competencies for their chosen professions. And also, the study addresses the following Specific Objectives:

• To evaluate the availability and adequacy of practical work opportunities for students in DDPTC, in order to assess the extent to which practical skills are being developed.

• To assess instructor engagement and their utilization of effective assessment practices that align with competency-

  RESEARCH METHODOLOGY

  The research employed a combination of quantitative and qualitative research methods. Quantitative research involves investigating a specific problem by testing a theory, using numerical measurements, and analyzing data through statistical techniques to support or refute predetermined hypotheses regarding the relationships between variables. This approach aims to draw general conclusions by selecting and studying a number of objects or participants (Marczyk et al., 2005).

  The two types of sources provide the data that this study needs to address both its general and specialized aims. The standard questionnaire, which employs a 5-point Likert scale, was given to college teacher leaders and subordinates in order to collect data for the primary source. Focus Group Discussions (FGDs) were conducted as a significant data-gathering method in this research. Reviewing articles, books, journals, research papers, and

  labor market, increasing graduates' employability and reducing unemployment rates. Thirdly, the study will contribute to policy development and implementation in technical and vocational education, guiding decision- makers in formulating effective strategies. Additionally, it will support the capacity building of instructors by identifying areas for improvement in their engagement and assessment practices. Lastly, the study fills a research gap and adds to the existing knowledge in the field of technical and vocational education, providing valuable insights for future studies.

  based training principles, ensuring that students' practical knowledge and skills are appropriately evaluated.

• To examine the implementation of competency-based training in DDPTC, including the incorporation of competency frameworks and standards, to determine the extent to which the training approach is being effectively applied.

• To analyze the effectiveness of assessment techniques employed by instructors in provoking responsible learning and maintaining student interest in the training process.

• To investigate the presence of a training process monitoring system in DDPTC, assessing its effectiveness in capturing information on the ongoing activities and progress within the institution.

• To identify and analyze the factors influencing competency-based training in terms of student interest and success, exploring elements such as student motivation, support systems, and alignment with labor market demands.

other pertinent information from the college served as the secondary source.

A questionnaire consisting of structured and semi- structured questions was developed and administered to the participants of the study who were selected using a stratified random sampling technique. To ensure the reliability and validity of the questionnaire, a pilot testing phase was conducted using non-respondent samples.

In addition to the questionnaire, Focus Group Discussions (FGDs) were conducted as a significant data-gathering method in this research. Trainers, department heads, and other relevant individuals were purposively selected to participate in the FGDs. The insights and information obtained from these discussions proved valuable in identifying potential solutions to the discussed issue or problem. Furthermore, archival sources were utilized for documentary analysis. Any relevant documents related to the practices and issues surrounding quality education and

competence-based training in the college were assessed and analyzed to provide further insight into the research topic.

STUDYGROUP

One non-random selection technique, convenience sampling, was employed to select the study's participants. According to Yldrm and Imek (2013), this sampling strategy is frequently chosen when researchers are unable to employ other sample techniques since it speeds up research and is more practical. The experts who were expected to participate in the study were selected according

DAA COLLECTION TOOL

The questionnaire used in the study was constructed by the researchers by reviewing the relevant literature. The questionnaire is prepared for four different categories, the first part is prepared for trainees second for trainers, the third part for college deans and leaders, and the final part for supervisors. Generally, the questionnaires consisted of two sections. The first section was about demographic

DEMOGRAPHIC PROFILE

Tab. 1 shows the respondents demographic profile, explaining that 120 teachers responses were collected from those who participated in the training program. The rate of response is 87.5% which is categorized as satisfactory. In the data we collected, 71 (67.62%)

responses are from the males, and 34 (32.38%) from the females, respectively Tab. 1 exhibits the remaining demographic analysis.

to their expertise. The trainers, graduate trainees, and college management were chosen by the researchers to serve as indicators of the quality of competency-based training. Due to the diversity of the populations (trainers, trainees, stakeholders, and top managers), all samples for the study were selected according to their expertise.

information for participants, and the second section was 20-30 items that help to assess the current CBT found in the literature and used in various studies. In addition to this, their opinions about CBE were asked through an open- ended question and Focus Group Discussions (FGDs) were conducted as a significant data-gathering method in this research.

Table 1 Demographic information

RESEARCH RESULTS

Descriptive Statistics

The term "descriptive statistic" refers to a statistic that can be used to summarize the characteristics of a collection of data in a quantitative way. Through descriptive statistics, we first look at the fundamental properties of constructs. We retrieved the mean, standard deviation, skewness, and

Exploratory Factor Analysis (EFA)

Exploratory factor analysis (EFA) is a classical formal measurement model that is used when both observed and latent variables are assumed to be measured at the interval level. EFA is executed on the correlation matrix between the items. In EFA, a latent variable is called a factor and the associations between latent and observed variables are called factor loadings. Factor loadings are standardized regression weights.

For the SEM-based multivariate analysis, exploratory factor analysis is initially used to condense and eliminate duplicate items and structures. Therefore, the rotated

kurtosis measurements for this reason. The results of Table. 2 show that the readings for standard deviation, skewness, and kurtosis are all within 1.5 and 3. Byrne (2016) states that the data fits the normalcy pattern if the standard deviation, skewness, and kurtosis are all within the range of 1.5 and 3, respectively. The normality of the data is a need for SEM-based multivariate modeling, according to Lu et al. (2020).

component matrix, which displays the factor loading of each item, is extracted using the main component of the analysis with Varimax rotation. One independent variable makes up the model under consideration. i.e. Pedagogical & Technical components (5 items), Real-world training effectiveness and challenges (6 items), Quality training satisfaction (4 items), Competency-based alignment (3 items), and Job relevance of training (3 items).

The findings of Tab. 3 exhibits that the factor loading of each item is more significant than 0.70. Thus, we can retain all the items and constructs (Ahmed et al., 2021)

Tab. 2: Descriptive analysis

Principal Component Analysis

Principal Component Analysis is a dimensionality reduction method that is often used to reduce the dimensionality of large data sets, by transforming a large set of variables into a smaller one that still contains most of the information in the large set.

The findings of Tab. 3 exhibit that each items readings of factor loading are between 0.75 to 0.97, which meets the pre-requisite of discriminant validity (Hair et al., 2019). According to Fornell and Larcker (1981), the reading of each constructs average variance extracted (AVE) is higher than 0.50, which fulfills the requirement of convergent validity.

Tab. 3; Reliability and validity of constructs

Total Variance Explained

The total variance is the sum of variances of all individual principal components. The fraction of variance explained by a principal component is the ratio between the variance of that principal component and the total variance. For several principal components, add up their variances and divide by the total variance. The results of Tab. 4 confirmed that each construct's initial eigenvalues were more than one, and the

cumulative total variance was 75.031 (75.031%), validating the reliability of the data and considering factors. Hair et al. (2019) claim that if the overall cumulative variance is greater than 0.50, it is sufficient. Because each eigenvalue is greater than one, the total cumulative variance is 75.031, which is good. As a result, it is determined whether the data and structures are reliable.

Table 4. Total variance explained

Findings

The findings suggest that TVET programs require stakeholder participation, adequate funding, continuous monitoring and evaluation, and improved TVET-industry linkage to effectively prepare individuals for the labor market and contribute to economic development.

Specific actions such as arranging more practice time, providing appropriate training machines, improving trainer

competencies, preparing effective training programs, facilitating employer-training provider forums, and introducing a tool-acquisition scheme are necessary to improve the quality of TVET programs.

The college should prioritize arranging more practice time and conducting research to evaluate their graduates. It is also crucial to improve TVET-industry linkage and provide appropriate training machines. Trainers should make efforts

to improve their own competencies, and effective training programs should be prepared by the Dire-Dawa TVET agency.

A forum should be arranged for employers and training providers to discuss TVET training quality issues. Finally, introducing a tool-acquisition scheme will encourage trainees to enter into self-employment, which will help them acquire basic tools to start their own micro- or small-scale businesses.

Finally, the quality of TVET programs in the region needs improvement, particularly in terms of coordination with industry stakeholders, trainer competencies, monitoring and evaluation, and entrepreneurship education.

Discussion and Conclusion

TVET programs are created to give people the technical and vocational skills they need to find jobs and support the growth of their community and nation's economy. However, creating a dynamic, adaptable, and efficient TVET system that can respond to shifting economic needs and demands is crucial for long-term success. To do this, TVET plans should be developed that consider the region's rapid socioeconomic development and the anticipated demand for skilled labor, in addition to supporting the knowledge and skills that individuals require.

TVET organizations, like colleges, are in charge of making sure that the training they are dispensing is of a high caliber and fits the demands of both the individual and the economy. In order to continue offering current and effective training over time, they must think about how to improve the sustainability of their training programs. This could entail putting money into innovative technology and teaching strategies, working with industry partners to keep current on shifting expectations, and making sure their instructors are properly qualified and prepared to provide top-notch instruction.

In general, the long-term goal of TVET programs is to construct a system that can provide the quantity and caliber of technical and vocational human resources required to sustain economic development. This objective necessitates careful planning and continual funding for top-notch training programs that can adjust to shifting demands and needs. TVET institutions can assist people in doing so while also fostering the development and prosperity of their local communities.

Numerous obstacles prohibit the TVET system from being effective and responsive to the needs of the job market. One of the key problems is that training programs might not be in line with company needs or labor market expectations because there is no assessment of training demand. Additionally, it is frequently challenging for TVET colleges to develop programs that are suited to the requirements of certain companies or sectors due to the lack of thorough and pertinent labor market information.

Limiting people's options and failing to address the demands of all industries or sectors can result in concentrating on a

small number of training programs. As a result, there might be an imbalance number of employees in the company in some companies a higher number in the same company shortage of qualified employees, and this led to an economic problem for the country.

Establishing independent entities that can coordinate the collection of data on evaluating training needs and labor market information is crucial to addressing these difficulties. These organizations ought to be in charge of conducting the analysis and providing the findings to TVET organizations, employers, and individuals. By doing this, individuals can receive training that is pertinent to their requirements and in line with the demands of the labor market, as well as the sustainability of outstanding TVET education.

But more than that, it shows that there is little partnership between the college and stakeholders at all levels in planning, allocating resources, providing career guidance services, monitoring training standards, identifying training needs, and providing information that helps maintain training quality. On the other hand, the school has inadequate internet services. In addition, the training facilities such as the size of the school premises, the size of workshops, libraries, and warehouses did not meet the expectations. The training materials, i.e. machines, tools and equipment, computers, books, modules, and raw materials for training, were also rated as better.

In addition, the funds allocated for raw materials, machinery and equipment, and project work were low. In summary, the analysis revealed that the lack of labor market information, the lack of training needs assessment, the problem of adapting the curriculum to the world of work, the low level of stakeholder participation in the VET system, and the lack of a sufficient budget for VET were among the problems affecting the sustainability of training quality in the study area.

The additional problem that affects the proficiency and awareness of the TVET system is the short participation of stakeholders. These are employers, and industry associations that often have valuable insights into the qualified training required for specific industries demands. Without their participation in industry input, TVET institutions may not develop training programs that are aligned and relevant to industry demands, which can lead to a mismatch between qualified and competent graduates with employers' needs.

Furthermore, the lack of adequate finance or raw materials, machinery and equipment, and project work may limit the quality of training provided by VET institutions. This can affect the skills and knowledge of graduates to compete in the market system as well as affect the status of the TVET system as a whole.

It is crucial to promote stakeholder involvement in the development and execution of TVET programs in order to address these issues. This can be achieved through partnerships between TVET institutions and employers,

industry associations, and unions. To ensure that TVET institutions have access to the resources they need to provide quality training, governments should allocate sufficient capital to TVET institutions. The TVET system can become more effective and meet market demands by solving these issues. This will lead to better outcomes for individuals, employers, and the economy as a whole.

Recommendations

In summary, proposals to improve TVET programs focus on stakeholder participation, adequate funding, continuous monitoring and evaluation, and better linkages between TVET and industry. Specific measures include providing more practice time, providing appropriate training machines, improving the skills of sneakers, developing effective training programs, promoting forums between

employers and training providers, and establishing a system for acquiring tools to promote self-reliance. These measures will better prepare people for the labor market and contribute to the country's economic development. The college should prioritize more time for internships and conduct research to evaluate its graduates. In addition, it is crucial to improve the link between vocational training and industry and provide appropriate training machinery. Sneakers should strive to improve their own skills, and the Dire-Dawa TVET Agency should devise effective training programs. A forum should be established for employers and training providers to discuss issues related to the quality of TVET training. Finally, the introduction of a tool acquisition program will encourage trainees to become self-sufficient. This program will help them acquire basic tools to start their own micro or small business.

REFERENCES

• African Union (2007). Strategy to revitalize technical and vocational education and training (TVET) in Africa. Addis Ababa: Final Draft. Bureau of the Conference of Ministers of Education of the African Union (COMEDAF II+).

• Aggrawal, J. C. (2004). Development and Planning of Modern Education. (8thed.). Delhi: Vakas Publishing House.

• Barrie, J., and Pace, R. W. "Competence, Efficiency, and Organizational Learning."Human Resource Development Quarterly 8, no. 4 (Winter 1997): 335-342.

• Brameld, T. S. (1972). Education as Power. New York: Holt, Rinehart and Winstong

• Byrne, B. M. (2016). Structural Equation Modeling with AMOS, Basic Concepts,

  Application and Programming (3rd ed.), London: Routledge.

• Chappell, C. "Quality & Competence Based Education and Training." In The Literacy Equation, pp. 71-79. Red Hill, Australia: Queensland Council for Adult Literacy, 1996.

• Collins, C., ed. Competencies: The Competencies Debate in Australian Education and Training. Curtin: Australian College of Education, 1993. (ED 361833).

• Creswell, J.W. (2009). Research Design: Qualitative, quantitative and mixed methods approaches. (3rd ed) Los Angeles: SAGE Publications Inc.

• D. Bloom, D. Canning and K. Chan (2006): Higher Education and Economic Development in Africa, the World Bank, Africa Region Human Development, Working Paper Series No. 102.

• Edukans Foundation (2012). TVET in Ethiopia. Addis Ababa GTZ, (2000). Report on Vocational and Technical Training Structure: Recommendations and Future Perspective. Addis Ababa.

• Hair, J., Risher, J., Sarstedt, M., & Ringle, C. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 224. https://doi.org/10.1108/

  ebr-11-2018-0203.

• Hodkinson, P., and Issitt, M., eds. The Challenge of Competence. New York: Cassell, 1995.

• HRST Department (June 2006): Recommendations from member

  states on how to revitalize VTE in Africa.

• ILO (2001). Modernization in Vocational Education and Training in the Latin American and the Caribbean Region. Montevideo Juran,

  J.M. and Gryna, F.M. Jr (Eds) (1988). Jurans quality control

  handbook. New York: McGraw-Hill.

• Johansson, R.K and A.V. Adams (2004): Skills Development in

  Sub-Saharan Africa. The World Bank. Washington DC

• Lu, J., Ren, L., Zhang, C., Rong, D., Ahmed, R. R., & Streimikis, J. (2020). Modified Carrolls Pyramid of Corporate Social Responsibility to Enhance Organizational Performance of SMEs Industry.Journal of Cleaner Production,271, 118. https://doi.org/10.1016/j.jclepro.2020.122456

• MOE (2008).National Technical and Vocational Education and Training Strategy Addis Ababa. 52P.

• MOE (2010). Leaders and Trainers Framework. Addis Ababa, Ethiopia.

• treimikien, D., & Ahmed, R. R. (2021). The integration of corporate social responsibility and marketing concepts as a business strategy: evidence from SEM-based multivariate and Toda- Yamamoto causality model. Oeconomia Copernicana, 12(1), 125

  157. https://doi. org/10.24136/oc.2021.006

• Parasuraman, A., Zeithaml, V.A., and Berry, L.L.. (1985). SERVQUAL: a multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing, vol.64, no.1, spring, pp.12-40.

• Roschanski, H. (2007). Deprived Children and Education Ethiopia. Amsterdam. IREWOC

• Tippelt, R. and AntonioA. (2003). Competency Based Training. Retrieved on Marcp7, 13 from http://www.inwent.org.

• UNESCO (2003).Strengthening the Industry Linkages with TVET programs in Arab Countries through sector partnership. Beirut

• UNESCO (2002). Technical and Vocational Education and Training for Twenty-first Century: Paris: UNESCO.

• World Bank (1991): Vocational and technical education and training A World Bank policy paper. Washington, DC.

IJERTV13IS060064

(This work is licensed under a Creative Commons Attribution 4.0 International License.)