Ahead of October’s FST conference, keynote speaker Steven Yule discusses the teaching and assessment of non-technical skills in surgery. Steven is Assistant Professor of Surgery at Harvard Medical School and the Neil and Elise Wallace STRATUS Center for Medical Simulation and will be speaking at the FST conference on Wednesday 22 October
What are non-technical skills?
Non-technical skills are defined as the cognitive and social skills that complement surgeons’ technical expertise, clinical knowledge and procedural ability in the operating room. These skills are often found to be lacking or dysfunctional in analyses of adverse events, and underpin many of the technical errors which injure surgical patients. The Non-Technical Skills for Surgeons (NOTSS) system defines four distinct categories of non-technical skill: Situation Awareness, Decision Making, Communication & Teamwork, and Leadership (see Yule et al. World Journal of Surgery 2008, 32:548-556)
Do all consultants have good non-technical skills at present?
It is difficult to say because we do not have norm data yet, just some small scale studies. There is probably a range of abilities; some consultants will be naturally good at these and some less so. Focus has been on training technical aspects of surgery until recently so that is what surgeons excel at. However, non-technical skills can be trained and improved as well with deliberate practice, expert coaching and skills taxonomies like NOTSS that make explicit the expectations of behavior in the OR.
Why are they important and can they be assessed?
These skills are at the centre of surgical performance. It is not sufficient to be simply technically excellent if your goal is to ensure high quality and safe procedures for patients. Analyses of surgical adverse events often cite failures of awareness, communication and teamwork in the OR so it is important to focus on those and how surgeons can be supported to improve those skills if required. The benefits of having a team that are better able to share information, lead and delegate tasks, be on the same page as others, detect errors when they occur and speak up about them is undeniable.
These skills can be assessed and there are a range of validated assessment tool available such as NOTSS, OTAS and NOTECHS which have all emerged in the past 10 years or so. We finally have tools and vocabulary to focus on the full range of behavior in the OR that make for high performing teams and successful surgery. And we can measure and improve these skills in surgeons and trainees using tools like NOTSS. Assessments are becoming more and more objective as these skills taxonomies are refined and used more regularly.
What is the best way of assessing them and are assessments accurate?
The best way of assessing these skills is direct observation by a trained observer during a real or simulated operative case using a validated assessment tool, either in simulation or the real operating room. For example using NOTSS to observe specific behaviors in the OR, categorising them according to the four categories and then rating how well the surgeon performed in each category can be very insightful.
Saying that, there are a range of options available and questions to answer – such as how many observers; should they be surgeons or social scientists; how much training do raters require; which tool to select; and whether to asses ‘live’ or from video? However, there is now a consensus for training faculty assessors which can help with those decisions (see Hull et al., Annals of Surgery 2013, 258(2):370-375).
Other methods include self-assessment, but people are often not accurate in their self-judgments for a variety of reasons. Harvard hospitals including my own are using 360-degree assessments of surgeons on a range of behaviors in and out of the operating room, including some non-technical skills. These methods are becoming very popular although move away from the workplace assessment model to more general characteristics of the surgeon. There are also knowledge and attitude scales for teamwork and communication. These methods all provide different and often complementary data. However, direct observation using a validated assessment tool is the best way in my opinion.
If they are important, do we teach them to all surgical trainees?
Formal training in these skills for trainees is at an early stage and not comprehensive or universal. However, things are changing – we just created the first non-technical skills curriculum for surgical residents in the USA with modules on cognitive skills (situation awareness, decision making) and social skills (communication and teamwork, and leadership). These are due to form part of the national curriculum for general surgery, and implemented by the Surgical Council on Resident Education (SCORE, www.score.org) in 2014-2015. These modules were created by Doug Smink (Surgeon at Brigham & Women’s Hospital/ Harvard Medical School), Alexandra Briggs (Surgical Resident at Brigham & Women’s Hospital/ Harvard Medical School), and myself.
In the UK, there has been steady progress in development and rollout of training in non-technical skills, particularly in this College, since the inception of NOTSS in 2005. The NOTSS Masterclass - a two-day course, has been run by the College's Patient Safety Board for consultant surgeons and senior trainees to teach identifying and assessing non-technical skills. For the needs of the early years of a surgical career e-learning modules introducing the underpinning concepts of non-technical skills have also been developed by the Patient Safety Board.
This group also developed “NOTSS in a box”, an e-learning package for surgical trainers focusing on assessing surgical trainees. This is available through the College website and also ISCP (Intercollegiate Surgical Curriculum Project) to help support a national assessment trial. There is also a non-technical skills module as part of the ChM on offer through the Edinburgh Surgical Sciences Qualification (ESSQ) in both Trauma and Orthopaedic surgery, and General Surgery.
The Royal Australasian College of Surgeons hold NOTSS courses roughly every month, in collaboration with RCSEd and other groups around the world are formulating this type of training. This continued evolution of non-technical skills featuring in syllabi and curricula points to the value being placed by colleges and other educational bodies on the importance of this skill set as an integral component of intraoperative performance. Expanding the faculty who can teach on this subject and developing a strategic approach for embedding training non-technical skills in the curricula of all surgical specialties in the UK is the next stage.
What is the best way to teach non-technical skills?
There is a distinction between teaching underpinning knowledge and ability to recognise/ assess non-technical skills, and teaching to improve the skills themselves. For teaching knowledge, the curriculum developed in the USA is based around a book chapter with vignettes of surgical scenarios and test questions available online at the ACS Surgery website (see http://tinyurl.com/ms6xe4a). The SCORE national curriculum is focused on underpinning knowledge of the four NOTSS categories and background reading which students read before a ‘flipped classroom’ style group discussion with faculty. A faculty teaching guide with learning and discussion points was developed to help facilitate this.
Other methods of training underpinning non-technical skills in classroom settings focus on small group discussion of videos. These are often surgical scenarios filmed to show specific behaviors and prompt discussion about non-technical skills in surgery. Generalising from videos of team performance and non-technical skills in other industries is also used, highlighting the role of non-technical skill failure in aviation and other industrial accidents. Roleplay, e-learning, and spaced education techniques have also been trialed.
For teaching to improve non-technical skills, there is a systematic review on training non-technical skills by Nicolas Dedy and colleagues in Toronto which provides a good overview of the methods of training, teaching focus, and evaluation (Dedy et al., Surgery 2013,154(5): 1000-1008). They reviewed 23 studies, including four randomised controlled trials and 19 pre- and post-observation studies. The dominant finding is that behavior can be improved by training non-technical skills, and that simulation (either high fidelity operating room or lower fidelity arenas depending on training aims) is a useful method. In fact, high fidelity simulation followed by debriefing sessions have been found to be superior to didactic and practice alone at skill acquisition. Video analysis of own or peer performance in simulation is also commonly used to allow participants to reflect on their own skills and identify learning gaps. Evidence of effectiveness of this type of training is variable but stronger for process measures like teamwork than patient safety outcomes. This is understandable as most efforts are on developing training curricula at present rather than developing sophisticated randomised prospective trials that also take into account clinical outcomes. That represents the future of research efforts in this field.
Some training to improve skill does not require a high fidelity simulated operating room. For example, learning vocabulary of non-technical skills and practicing language for close loop communication or speaking up can be taught in classroom settings. Finally, part-task trainers, virtual reality, bench models, and laparoscopic surgical simulators are usually limited to teaching technical or procedural skills but they can also be used to teach decision making and other cognitive skills.
Should a surgical trainee with great technical skills but poor non-technical skills be allowed to become a consultant?
I think this is a rare situation but no, I do not think that a hospital, health board, university or the College should allow someone to proceed to become a consultant if they know they have substandard non-technical skills. Doing so would reflect a wider issue of values and what makes a proficient surgeon. It may also reflect poorly on the comprehensiveness of the training programme to develop surgeons who are fit for purpose. Taking the question to the next level, I believe that surgical trainees should be assessed on their non-technical skills as part of board examinations. This will help them in the long run as we know that assessment drives learning (and is actually better than repeated study in some cases for long-term retention). No future surgeon wants to be an ineffective communicator, poor leader or incapable of making effective decisions.
How do you see the assessment and teaching of non-technical skills developing over the next five years?
I am convinced that this will become a vital area of concentration as healthcare moves forward. We have made a great start in surgery and the next five years will be exciting. I see demand from trainees for more feedback and strategies for improvement, and desire from surgical trainers for more guidance and tools to help them support trainees’ development.
If we start to train these non-technical skills systematically, and base a national training curriculum on NOTSS, in five years’ time we could have a cadre of surgical residents who graduate from structured training programmes with consistently good non-technical skills as they approach independent practice. They will then become the assessors of the future and develop skills to mentor junior trainees who enter training programmes. Over time they will become faculty with responsibility to train and assess residents coming through the system. This will raise the capacity of knowledge and practical skills across the board.
To achieve this we need assessment tools that are easy to use for faculty and trainers that help them make assessments about trainees in an objective non-personal manner, and specific feedback for trainees on strategies to improve skills. E-learning will be important to this as will cloud technology and smartphones. It is difficult to assess people without training packages in specific areas so new content in advanced non-technical skills will need to be developed. It is important that these are about developing professional skills and not remediation. Simulation will increase in prominence for training in this area as will inter-professional training. It is possible that assessors will need to be accredited in some way.
Of course there are some important research questions around scalability of training, demonstrating impact on performance and outcomes in the OR and longitudinal studies of learning curves in non-technical skills to be conducted if this is to be supported in the most efficient and effective manner. That will also require research funding and continued partnership between the Colleges, surgeons and research groups. Effective leadership, situation awareness, decision making, communication and teamwork will save many lives, lead to less morbidity, improve quality, and result in lowering the cost of surgery. Patients will demand it and we have the ability to make it happen.
Steven Yule will be speaking at Who Makes the Cut? Assessment in Surgical Training at the RCSEd, Wednesday 22 October 2014
For details, contact education@rcsed.ac.uk / 0131 668 9209
Due to essential systems maintenance and upgrades there will be interruptions to some on-line services on Saturday 19th of August.
We apologise for any inconvenience caused.
×