The College is undergoing development of a new, evidence-informed process and tools to assess the continuing competence of occupational therapists (OTs) as part of the Competency Assessment component of the QA program.

Assessing continuing competence is a cornerstone of the College’s legislated mandate to ensure Ontarians receive safe, effective, ethical care from OTs.

The need for a new process was identified when the College initiated a research study evaluating the effectiveness of the previous step in the competency assessment process (which used client and co-worker surveys) and found it was not always effective. The goal is to put in place a new competency assessment process that:

  • More accurately identifies OTs requiring further assessment of competence;
  • Efficiently uses College and registrant resources;
  • Increases engagement of OTs; and
  • Demonstrates the College’s dedication to continued improvement of processes and programs.

The New Competency Assessment Process

Based on research conducted by the College, input by experts in the area of competency assessment, and feedback provided by OTs through the online survey administered at the end of 2019 (note: this survey focused on risks inherent in the OT profession), a new, innovative, risk-based process for assessing OT continuing competency has been determined.

The steps in the new process include:

  1. Risk-based selection of OTs
  2. Additional screening (such as a remote or virtual interview)
  3. Peer and Practice Assessment

In this first step of the new process, all OTs will be screened annually to identify those who may be considered higher risk because of various factors. These factors are referred to as risk indicators.

There are 18 risk indicators as shown below that fall within three types of risk categories. The three risk categories are: Practice Risks, Isolation Risks and QA Behavioural Risks.

Risk Categories and Indicators


Practice Risks

Isolation Risks

QA Behavioural Risks

(Identified through self-assessment and Professional Development Plan)
  • High number of employers (at one time)
  • Temporary primary employment
  • Casual primary employment
  • Low Practice hours (per week)
  • Limited Currency
  • Shift from non-clinical to clinical or mixed practice
  • Suitability to practice (for example: finding/facing misconduct, incompetence or incapacity)
  • Practice includes medically delegated acts
  • Shift in client age range served in the past two years
  • Shift in primary practice setting in the past two years
  • Many years since graduation (more than 25 years)
  • Casual status at primary employer
  • Self-employment/solo practice
  • Rating fewer than one-quarter of competencies
  • Not setting any competency priorities
  • Stating no need for any professional development on any competency
  • Bullet rating - giving more than 90% of competencies the same rating
  • Bullet priorities – giving all competencies the same priority

To conduct the risk-based selection, the level of risk will be calculated for each OT based on the number of indicators they have in each category – as shown below:

  • 0 risks in any category
  • Risks in any 1 category
  • Risks in any 2 categories
  • Risks in all 3 categories
A percentage of OTs will then be randomly selected from each of these risk categories, with a higher percentage of OTs being selected who have risks in two or three categories.

For example, if an OT’s practice includes medically delegated acts (a practice risk) and they are self-employed (an isolation risk), they have risks in 2 categories. They therefore have a greater chance of being selected to participate in the next step of the assessment process – which is the additional screening step.


Note: the risk indicators do not indicate any judgment about an OTs’ practice or abilities. The risk indicators simply provide a first screening to determine OTs who may be at higher risk and require additional screening and assessment to ensure they are meeting the essential competencies of practice. OTs will be selected from each of the risk categories, including those who have 0 risks in any category. By selecting from each category, annually, the College is committed to continuous monitoring and quality improvement of the process.

The College expects to pilot this new and innovative risk-based selection step starting in 2021. In the meantime, additional details, including the percentages of OTs to be selected from each of the risk categories, are being determined. The College will communicate the additional details as they are available.


A new screening step is also being included in the College’s new competency assessment process. Although the details of this step are still being determined, it is expected that it will include a remote or virtual interview with an OT and a College trained Peer Assessor. The goal of this screening step is to assess for any potential gaps in OT competence and trigger when an OT would benefit from a more comprehensive assessment of their practice through the Peer and Practice Assessment (step 3). It is also expected that the interview will be short in length, about one hour, and will be done through teleconference or videoconference.

Note: not every OT who participates in the screening step will participate in the Peer and Practice Assessment; only those determined as having potential gaps in competence and would therefore benefit from a further assessment of their knowledge, skills and judgment.

As it is necessary for this additional screening step to be based on the Essential Competencies of Practice for Occupational Therapists in Canada, 3rd Edition, and these essential competencies are currently being revised through the CORECOM project – to be released in early 2021, the specific interview questions will be developed after the essential competencies have been released.

The College therefore expects to pilot the additional screening step in the summer of 2022 and will communicate details about the additional screening step as they become available.


In the new competency assessment process, it is expected that the Peer and Practice Assessment will be similar to the assessment that is currently done and will continue to be in-person. There are some improvements the College expects to make to the Peer and Practice Assessment, particularly with how questions are phrased and scored.

The College expects to have any revisions to the Peer and Practice Assessment completed in the summer of 2022. In the meantime, the current Peer and Practice Assessment will remain in place and used during the 2020-2021 registration year. Any details about changes to the Peer and Practice Assessment will be communicated as they become available.

Here is how all three steps of the College’s new Competency Assessment process fit together:
new-comptency-assessment-process

OT Input


The results from the 2019 OT survey that focused on risk inherent in an OT’s practice will be used to inform development of the additional screening and any changes to the Peer and Practice Assessment. A group of OTs will also be convened to provide feedback during the review and development of these steps. The College will reach out to OTs when this input is required – likely in early 2021.

Summary of Timelines:


timeline-summary


How often will this new competency assessment process happen?

It is expected that each of the steps of the new competency assessment process will happen annually.

Will the new competency assessment process be the same for OTs working in non-clinical settings (e.g., management)?

Yes, the steps in the new competency assessment process will be the same for OTs working in clinical and non-clinical settings, however, the questions that OTs will be asked in the additional screening step and the peer and practice assessment step will likely differ. These details will be confirmed and communicated when they become available.

How many OTs will participate in each step of the new process?

All OTs will be included in the risk-based selection step of the new competency assessment process. The number of OTs who will participate in the additional screening step and the peer and practice assessment each year are still being determined. This information will be communicated when it is available.

Will the College inform OTs about the number of risk indicators they each have?


At this point in development of the new process, the College does not expect to communicate the number of risk indicators identified for each OT unless an OT is selected to participate in the next step of the process – the additional screening. As the risk categories and indicators are published on the College website, however, all OTs can determine their own number of indicators at any point in time.

Do other professional colleges select participants for screening and/or assessment based on risk?

The new risk-based selection step is an innovative and efficient way to select OTs who may be at higher risk, due to identified risks in practice, and require additional screening and assessment to ensure they are meeting the essential competencies of practice.

The College is not aware of other professional healthcare colleges who currently select members for a competency assessment as part of their respective quality assurance programs – based on known/identified risks. Other non-healthcare professional bodies, however, are using similar risk-based approaches.