Description
Shoroog Agou
Date:
May 15-6, 2024
Your facilitator
An orthodontist
A health professional educator
An educationalist, focused on assessmnet & curriculum development
A consultant, in educational organizational leadership solutions
A member in several national health professional educational
committees
Introduction
&
Expectations
Please tell us more
about you…
1.Name
2.Background
3.Institute
4.Role
5.Current experience with LOs
6.Expectations
Expectations
Please tell us about
your expectations…
1.
2.
3.
4.
Who is Who?
Please introduce yourself, background, & institute
Expectations
&
Preferences
Please tell us more
about you
using the poll
1.Your background
2.Your role
3.Expctations
4.Previous workshop attended
5.reviewing PLOs documents
6.Preferred language
Learning
Outcomes
By the end of this workshop, you
should be able to:
1.Explain the role of learning outcomes
in the quality assurance system.
2.Explain the main sources for
determining learning outcomes.
3.Review the program learning
outcomes matrix as an alignment tool.
4.Formulate the learning outcomes in
accordance with the levels of the
National Qualifications Framework.
5.Assess learning outcomes using direct
and indirect assessment methods.
الوحدة األولى
مخرجات التعلم (األطر المفاهيمية)
تهدف الوحدة إلى تزويد المتدرب بالخلفية المفاهيمية والنظرية للتعلم القائم على المخرجات لفهم منظومة مخرجات التعلم ودورها في تحقيق جودة التعليم
على المستويين المؤسسي والبرامجي.
الجلسة
العنوان
المدة
الجلسة األولى
مخرجات التعلم المفهوم واألهمية
٧٠د
استراحة
١٠د
اشتقاق مخرجات التعلم
٧٠د
الجلسة الثانية
1.1 Individual Activity
Using activity 1 space in your trainng manual,
please write
When you are done, please send it to this email address
5 min
INTRODUCTION
Why Outcomebased Education?
في كلمة واحدة
ما لذي يعنيه لك
مبدأ التعليم القائم على المخرجات
Outcomebased
Education…
Why?
Why OBE?
NQF-KSA
• Unified Goals
• Lifelong learning
• Common language
• Shared expectations
• Practice ready-graduates
• Quality assurance promotion
NQF document, 2020
Outcomebased
Education…
Why OBE?
NQF-KSA
• Mobility & transparency
• Course design
• Students’ guidance
• Quality assurance promotion
Why?
Adam,
2004
Outcomebased
Education…
Sustainability standards: the New Quality Assurance
for Higher Education in the Fourth Industrial
Revolution (4IR), Ramirez, 2018
Program
Specifications
Course
Specifications
Course Reports
Program Reports
The role of LOs in the quality assurance system
Courtesy of Dr. Mohammad Rashid Al-Fuhaid
Program
periodic review
أيهما أهم؟
ما الذي قمت به للحصول على الشهادة؟
أم ما الذي يمكنك أن تفعل اآلن وقد حصلت على شهادتك؟…
Thinking point: What is the difference between aims, objectives, competencies & learning
outcomes?
INTRODUCTION
Where to start
from?
a. Designing instruction
Quiz…
b. Designing assessment
c. Determining content
ما هي نقطة االنطالق؟
d. Articulating outcomes
e. Determining outcomes
When you begin with the assessment in mind,
Begin with the assessment (End) in mind
Backward course design model involves answering three questions:
What do you want the
students to learn?
How will both you and your students
know if they are learning?
What actions will both you
and your students
need to take to learn?
Working definitions
OUTCOMES
Learning outcomes are statements of
• Determining outcomes
• Designing assessment
• Articulating outcomes
what a student is expected to know,
understand and/or be able to
demonstrate after completion of a
process of learning.
READ MORE
• Designing assessment
• Conducting assessment
• Using assessment
ECTS Users’ Guide, 2005
I. Determining Outcomes
Learning
Outcomes
Lifecycle
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
I. Determining Outcomes
Learning
Outcomes
Lifecycle
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
10 MINUTES
Prayer Break
Today’s
Agenda
I. Determining Outcomes
II. Designing Assessment
III. Articulating Outcomes
How do we determine
outcomes?
In the chat… or raise your hand
Standard 3: TEACHING AND LEARNING
Graduate attributes and learning outcomes at the program level must be
precisely defined,
and with the related
The curriculum must
conform to professional requirements. The teaching staff must implement
diverse and effective teaching and learning strategies and assessment
methods that are appropriate to the different learning outcomes. The extent of
achievement of learning outcomes must be assessed through a variety of
means and the results are used for continuous improvement.
التعليم والتعلم
يجب أن تكوننواتجالتعلمفيالبرنامجمحددةبدقة،ومتسقةمعمتطلباتاإلطار الوطني للمؤهالت ،ومع المعايير األكاديمية التخصصية ،ومتطلبات سوق العمل .
ويجب أن يتوافق المنهج الدراس ي مع المتطلبات المهنية ،وأن تطبق هيئة التدريس
استراتيجيات تعليم وتعلم وطرق تقويم متنوعة وفعالة تالئم نواتج التعلم المختلفة ،كما يجب أن يتم تقويم مدى تحقق نواتج التعلم من خالل وسائل متنوعة،
ويستفاد من النتائج في التحسين المستمر.
ALIGN
BENCHMARK
FORECAST
ANCHOR
DETERMINE
BENCHMARK
FORECAST
ALIGN
ANCHOR
DETERMINE
ALIGN
BENCHMARK
FORECAST
ANCHOR
DETERMINE
ALIGN
BENCHMARK
Review PLO-CLO
alignment matrix
ANCHOR
FORECAST
DETERMINE
Anchoring Outcomes to NQF
What is the NQF?
How to ensure alignment with NQF?
Knowledge
Values
Domain
Skills
Knowledge
Values
Skills
Level 5
Level 6
Level 7
Advanced
diploma’s degree
Bachelor’s degree
Master’s degree
How could we ensure consistency
with the NQF?
Verifying Consistency with NQF
Steps to align program learning outcomes with NQF-KSA
Step 1
Identify the level of the qualification in the NQF-KSA that is being dealt with
Ensure that the title of qualification matches this level
Step 2
Identify the essentials of each domain of learning at that level.
Interpret these for the specific program as a series of learning outcome statements.
Step 3
For each domain, identify teaching and learning strategies/methods that will enable students
to develop the knowledge, skills, values and attitudes required and the assessment methods
that are linked to the learning outcomes and will enable them to demonstrate achievement of
the PLOs.
How could we align course outcomes with program outcomes?
In the chat… or raise your hand
ALIGN
BENCHMARK
Alignment matrices
PLOs-CLOs
alignment matrix
FORECAST
DETERMINE
ANCHOR
Designing &
Reviewing
PLOs-CLOs
matrices
PLOs-CLOs Matrix
What is a PLOs-CLOs
matrix?
Where do we find it?
What is it used for?
What levels do we use?
PLOs-CLOs Matrix
Well-represented program learning outcomes are:
·Introduced and assessed in early courses
·Practiced and / or Reinforced, as well as assessed in subsequent courses
·Assessed for level of Mastery in upper level/capstone courses
Why review PLOs-CLOs alignment?
1.2 Large Group Activity
Using the PLOs-CLOs matrix in your trainng
manual, please write review B3 and B4 PLOs
When you are done, Please use the chat or the mic to share
your comments
10 min
Review the PLOs-CLOs matrix for B3 & B4
(critical thinking PLOs)
Reflections
How to review the
PLOs-CLOs matrix?
1
When is each PLO introduced?
look for adequacy of coverage
& timely (early) introduction
2 Is the progrssion logical, I, P,
then M?
There should be a progressive
development of PLOs through
courses over the extent of the
program
3 When is each PLO assessed?
look for demonstration of
achievement of PLOs through
assessment, practically in
final year courses
4 Reviw PLOs coverage per course
CLOs should be varied,
contributing to most domains
CLOs should be approporiate to
the course level
CLOs should be SMART
Review the PLOs-CLOs matrix for B3 & B4
(critical thinking PLOs)
Why review PLOs-CLOs alignment?
The PLOs-CLOs alignement matrix ensures curriculum coherence & progressive
development of knowledge, skills, and values by identifying the following elements which
reviewers must verify:
• Curricula where coverage is incomplete.
• PLOs not adequately addressed by the curriculum
• Courses whose prerequisites are not provided earlier in the curriculum
• Structural coherence: identify overlap and repetition in sets of courses
• Lack of evidence of development from introductory to advanced skills
• Overall lack of thematic organization
• Unclear relationship between courses and program outcomes
الوحدة األولى
مخرجات التعلم (األطر المفاهيمية)
تهدف الوحدة إلى تزويد المتدرب بالخلفية المفاهيمية والنظرية للتعلم القائم على المخرجات لفهم منظومة مخرجات التعلم ودورها في تحقيق جودة التعليم
على المستويين المؤسسي والبرامجي.
الجلسة
العنوان
المدة
الجلسة األولى
مقدمة في قياس مخرجات التعلم
٧٠د
استراحة
١٠د
إجراءات صياغة ومواءمة مخرجات التعلم
بنية ومواصفات مخرجات التعلم
٧٠د
الجلسة الثانية
Today’s
Agenda
I. Determining Outcomes
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
DESIGNING ASSESSMENT
Why & How?
Why assessment first?
before articulating outcome
In the chat… or raise your hand
Designing
Assessment
Designing
my course
assessment
system
Determining
Assessment Method
Designing
my course
assessment
system
Determine the desired assessment
in alignment with NQF
Determine the desired Miller’s level
Choose the right taxonomy level
Choose & develop the tools
Determine the desired performance
level
What NQF domain should I
assess?
Knowledge
Values
Skills
What should I assess in that domain?
DOES
SHOWS HOW
KNOWS HOW
KNOWS
At what cognitive, affective, psychomotor level should I
design my assessment?
Assessment Tools
What tools should I use?
Indirect
tools
examples
Direct
tools
Matching Assessment Tool to Desired Level
Bloom’s
cognitive level
Description
Assessment tools
Matching Assessment Tool to Desired Level
Bloom’s
cognitive level
Description
Assessment tools
Product
Error Identification
reflective writing
Case Study
vignettes
What performance level do you desire?
How can I decide on
the expected level?
Dreyfus & Dreyful Model for Skill Acquisition
Knowledge
Level 5
Level 6
Level 7
Advanced
diploma’s degree
Bachelor’s degree
Master’s degree
without
guidance,
advanced
technique
Values
Skills
Domains &
Sub-domains
with
guidance
selfdirection and
originality
How?
Assessment Tool
Main Purpose
Exams
To measure students’ understanding of a specific set of
information, usually at the end of a unit or term.
Oral Exams
To assess students’ verbal articulation of knowledge,
reasoning, and often their presentation skills.
Working Papers
To review students’ progress and thought processes in a
working context, often used to assess practical or applied
skills.
Dilemmas
To gauge students’ abilities to apply ethical reasoning and
decision-making skills to complex, ambiguous situations.
Student Presentation
To assess students’ abilities to organize, present information
clearly, and communicate effectively to an audience.
Evaluating Research
To critique the design, methodology, and conclusions of
research projects for rigor and academic integrity.
Performance Tasks
To evaluate students’ ability to perform specific tasks that
demonstrate their skills, knowledge, and competencies.
Case-based Discussions
To encourage analysis, application of knowledge, and problemsolving skills related to real-world scenarios.
Graduation Projects
To evaluate the culmination of students’ learning and research
capabilities in a comprehensive project.
Course Portfolio
To collect and review a range of work products from a student
over the course of a term or year to assess growth and
proficiency.
Multi-source Feedback
To gather feedback on students’ performance from multiple
sources (peers, self, teachers) for a well-rounded assessment.
Reflections
To encourage self-assessment and critical thinking about one’s
learning process and personal development.
DOES
SHOWS HOW
KNOWS HOW
KNOWS
Exams
Oral Exams
Working Papers
Dilemmas
Student Presentation
Evaluating Research
Performance Tasks
Case-based Discussions
Graduation Projects
Course Portfolio
Multi-source Feedback
Reflections
2.1 Small Group Activity
We will break into 4 groups, Groups 1 & 2 will
work on skills, Groups 3 & 4 will work on values
1- Redistribute the assessment tools from the list provided
in your manual, according to Miller’s levels
2- Choose one particpant to share with us which
assessment tools the team selected for each level
10 min
DOES
SHOWS HOW
KNOWS HOW
KNOWS
DOES
Projects, feild training, portfolios
SHOWS HOW
Performance tasks, structured exams,
simulations
KNOWS HOW
Written and Oral
Exams
KNOWS
Written and Oral
Exams
DOES
SHOWS HOW
KNOWS HOW
KNOWS
DOES
MSF, projects, feild training, portfolios
SHOWS HOW
Reflections, dilemmas,
debates
KNOWS HOW
Written and Oral
Exams
KNOWS
Written and Oral
Exams
Determining Assessment Method: Summary
Demonstrate integrity and adherence to Islamic values,
ethical principles, legal standards, and professional
behavior.
Exhibit the autonomy and responsibility needed for
continued self-learning, professional planning, strategic
decision making, and self-development
Develop as a leader to manage teams and tasks effectively
and collaboratively
Participate in research and community projects that
demonstrates good civic responsibility and fosters the
quality of life of the community
Demonstrate integrity and adherence to Islamic values,
ethical principles, legal standards, and professional
behavior.
Student Logbook
Daily evaluations
Exhibit the autonomy and responsibility needed for
continued self-learning, professional planning, strategic
decision making, and self-development
Student Logbook
Reflective journal
Personal development
plan
Develop as a leader to manage teams and tasks effectively
and collaboratively
MSF
Assessment of
personal qualities and
graduate attributes
Evaluation checklist
Portfolio
Participate in research and community projects that
demonstrates good civic responsibility and fosters the
quality of life of the community
Research assignment
Student portfolio
Need an example
putting it together
Determining
assessment methods
Skills
Develop critical thinking &
problem solving skills
PLO/CLO
Assessment First Approach Model, SA 2022
Determining
assessment methods
Values
Ethics & integrity
PLO/CLO
Assessment First Approach Model, SA 2022
Determining
assessment methods
Skills
Develop critical thinking &
problem solving skills
PLO/CLO
Analyze research data,
unassisted, using
descriptive statistical
techniques
Assessment First Approach Model, SA 2022
END OF DAY 1
Conclusions &
Feedback
ALIGN
BENCHMARK
Alignment matrices
PLOs-CLOs
alignment matrix
FORECAST
DETERMINE
ANCHOR
Determining
Assessment Method
Designing
my course
assessment
system
Determine the desired assessment
in alignment with NQF
Determine the desired Miller’s level
Choose the right taxonomy level
Choose the tool
Determine the desired performance
level
END OF DAY 1
Take Home Activity
Post-Day 1 Large Group Activity:
putting your new skills into practice
Critique your own program PLOs
We will share and discuss some examples of your reflections tomorrow
DAY 2
What did we talk
about yesterday?
DAY 2
Discussion of the
Take Home Activity
Post-Day 1 Activity:
putting your new skills into practice
We will share and discuss some examples of your reflections & critique your
own program PLOs with respect to:
Activity time:
Workshop
Agenda
I. Determining Outcomes
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
Today’s
Agenda
I. Determining Outcomes
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
DAY 2
Articulating
Outcomes
What do we by articulating outcomes?
In the chat… or raise your hand
What do we mean by SMART?
In the chat… or raise your hand
Articulating Outcomes
Specific:
Measurable:
Acheivable:
Relevant:
M
S
R
A
Time-bound:
T
Thinking point: Should PLOs be SMART?
What are the differences between PLOs & CLOs?
Articulating Outcomes
Specific:
Determining outcomes
1
Measurable: All the steps involved in Designing Assessment
Designing assessment
2
Acheivable: When performance level & guidance was determined
3
Now check, are the outcomes
SMART
M
S
R
A
Relevant: All the steps involved in Determining Outcomes
Time-bound:
T
Thinking point: Should PLOs be SMART?
What are the differences between PLOs & CLOs?
Articulating Outcomes
Specific: AV + task + Achievement level + Performance condition
Determining outcomes
1
Measurable: All the steps involved in Designing Assessment
Designing assessment
2
Acheivable: When performance level & guidance was determined
3
Now check, are the outcomes
SMART
M
S
R
A
Relevant: All the steps involved in Determining Outcomes
Time-bound: “by the end of this …….. students will be able to:”
T
Thinking point: Should PLOs be SMART?
What are the differences between PLOs & CLOs?
Syntax of a learning
outcome
Specific: AV + task + Achievement level + Performance condition
Specific: AV + task + Achievement level + Performance condition
Which of the following verbs, are NOT
measurable?
Specific: AV + task + Achievement level + Performance condition
Condiser Scope of task, Topic, Context
Specific: AV + task + Achievement level + Performance condition
Guidance level
e.g., no critical errors, without
assistance, independantly
Dreyfus & Dreyful Model for Skill Acquisition
Specific: AV + task + Achievement level + Performance condition
Using specific tools or techniques
Setting: reallife, simulations, etc.,
Examples:
By the end of this….., students will be able to:
• use word processing, spreadsheets, databases, and presentation graphics in
preparing their final research project and report.
• select the proper Endodontic instruments to be used in root canal treatment
• design and conduct original and independent biological research.
• present a topic of study in a field–specific language.
• apply medical legal and ethical principles to decision making in patient care.
• explain computer networking concepts to both technical peers and nontechnical management.
• apply critical and logical thought and mathematical principles to the problem
of study.
Examples:
By the end of this….., students will be able to:
• design original and independent biological research.
• present a topic of study in a field–specific language competently.
• apply medical legal principles to decision making in patient care with sound
moral reasoning.
• apply critical thought and mathematical principles to solve discipline-related
problems logically
Let’s practice
3.1 Group Activity
Articulating Learning Outcomes
We will break into groups
1- Use the LO equation and the values assessment table we
used earlier, the reference in the QR code to formulate a LO
targeting senior level undergraduate students
2- Choose one particpant to share with us the LO
formulated by the team
10 min
CLOs Review tips
• Have I focussed on outcomes not processes, i.e. have I focussed on what the students
are able to demonstrate rather than on what I have done in my teaching?
• Have I begun each outcome with an active verb?
• Have I used only one active verb per learning outcome?
• Have I avoided terms like know, understand, learn, be familiar with, be exposed to, be
acquainted with, and be aware of?
• Are my outcomes observable and measurable?
• Are my outcomes capable of being assessed?
• Have I included learning outcomes across the range of levels of Bloom’s Taxonomy?
• Do all the outcomes fit within the aims and content of the module?
• Are the CLOs aligned with the right PLOs, matching the level indicated in the PLOs-CLOs
matrix?
• Have I the recommended number of outcomes (maximum of nine per module)?
• Is it realistic to achieve the learning outcomes within the time and resources available?
3.2 Individual Activity
Please write 2 of your desired program
learning outcomes, peferably a skill and a value
Let’s share examples and critique them together based on
what we learnt together
10 min
Common issues with LOs
•
•
•
•
•
•
•
•
•
DAY 2
Summary
Today’s
Agenda
I. Determining Outcomes
II. Designing Assessment
III. Articulating Outcomes
IV. Conducting Assessment
ALIGN
BENCHMARK
Alignment matrices
PLOs-CLOs
alignment matrix
FORECAST
DETERMINE
ANCHOR
Determining
Assessment Method
Designing
my course
assessment
system
Determine the desired assessment
in alignment with NQF
Determine the desired Miller’s level
Choose the right taxonomy level
Choose the tool
Determine the desired performance
level
Articulating Outcomes
Specific: AV + task + Achievement level + Performance condition
Determining outcomes
1
Measurable: All the steps involved in Designing Assessment
Designing assessment
2
Acheivable: When performance level & guidance was determined
3
Now check, are the outcomes
SMART
M
S
R
A
T
Relevant: All the steps involved in Determining Outcomes
Time-bound: “by the end of this …….. students will be able to:”
Outcome
ASSESSMENT
Outcome assessment is central to the
Determine methods & design plans (including
direct & indirect , formative & summative)
Gather data ensuring: authenticity, credibility,
validity, & reliability
quality assurance process
www.reallygreatsite.com
Analyze data
Review assessment quality
Measure & review achievement of outcomes,
assessment methods, & instructional strategies
Provide & get feedback from & to students, faculty,
curriculum & program administrator
Set & implement improvement action plans
END OF DAY 2
Reflections
Feedback
Post-test
Purchase answer to see full
attachment
