SPEAKER_04 0:07

Welcome to the Thought Broadcast, an Australasian psychiatry podcast created by trainees or for trainees. In previous episodes, we have focused our attention on demystifying the scholarly project and humanizing the research experience by sharing other trainees' stories. In this episode, we are broadening our scope to have a more general look at psychiatry training in Australia, New Zealand. For those unaware, the November sitting of the Objective Structured Clinical Examination, or OSCI, as it is more commonly called, was cancelled at the 11th hour due to a failure of the virtual examination software. The Thought Broadcast acknowledges the serious impact this specific event has had on the candidates who spent months preparing for this examination and the ongoing uncertainty that they face regarding alternative assessment pathways. There are numerous stories currently being shared on social media platforms that bravely paint the picture of how summative psychiatry assessments affect not only the specialist in training but those around them. However, in addition to the immediate stress and uncertainty for candidates, the OSCE cancellation has the potential to amplify pre-existing bottlenecks in the training program. Meaning many fellowship hopefuls will be left waiting to receive their letters with secondary impacts on workforce availability and service provision. To discuss these issues and the education of psychiatry trainees in our region, we have the pleasure of welcoming Dr. Nick O'Connor. Welcome, Nick. Nick, can you just give the audience a bit of a background on who you are and what your role is in the college so that we can sort of start off on the right footing?

SPEAKER_00 1:40

Yes, thanks, Ollie. But I think more important than stuff about myself at this point, I would also like to acknowledge the distress and disruption to trainees and SIMGs, and most particularly to the candidates of the 20th of November, OSCI and their families. But, you know, I'm aware of the level of the distress, and my heart truly goes out to all of you. Trying to repair the trust and provide as much certainty as possible. And I hope that some of the content of this podcast might be able to do that. I think that the exam crisis has made salient. A number of issues that have been bubbling under the surface for a long time. And I think that these also need to be addressed along with recovering and repairing the exam situation. It might sort of appear that not much has happened since the 20th of November. So I really welcome the opportunity later in this podcast to explain what we've been doing and where we're up to.

SPEAKER_04 3:20

So I'll quickly introduce the rest of our panel. We've got our usual group of panel members today. So we have uh the deputy editor of Australasian Psychiatry, Andrew Amos. How are you going, Andrew?

SPEAKER_03 3:35

I'm really good. Thanks for asking.

SPEAKER_04 3:36

Thanks. And then we've got our associate trainee editor. So Michael Waitemans coming in from South Australia. How are you, Michael?

SPEAKER_02 3:43

Hi Oli. Very well, thank you.

SPEAKER_04 3:45

That's good. And we've got Brian Bowie from Queensland. How are you, Brian? I'm good, Olive. How are you? I'm loving the channel in Queensland. Great. Thanks so much. So let's um get on to sort of addressing the issues at hand. Um you've already offered a bit of an introduction, Nick, but could you give us some sort of background what occurred at the November OSCE and the subsequent difficulties that um trainees have been facing who are hoping to attain fellowship?

SPEAKER_00 4:13

Yes, absolutely, Ollie. Um look, I I guess the the background to this, as everybody knows, is that COVID uh sort of really disrupted exams uh during the early part of 2020. Uh, you know, and then we uh in terms of an OSCI, we we stood up a uh a limited in numbers, AV OSCI on Zoom in November. We went a little bit higher in terms of the number of candidates that we could do using that technology in uh April of this year. I think that that was sort of the end of the first phase of the pandemic, and I think many of us and the community were hoping and and and I guess looking back on it somewhat complacent that, you know, um, and I remember the the sort of words that politicians and everybody was using at the time, like we dodged that bullet, and and yet, you know, we could see what was happening overseas. But we we thought that multi-site face-to-face OSCIs were the way to go in July, and that whilst we realized that you know there might be some uh hospitals or even some centres that would be affected, that by and large the every you know, most centres would be okay. And of course, on the eve of that uh Molyside OSCI, New South Wales uh health sort of um lock, you know, went for a hard lockdown and actually instructed colleges that they couldn't do exams. And of course, the New South Wales candidates were very affected by that. And and and I also know there that you know there were Tasmanian candidates who had to have last-minute changes, etc. Following that, you know, the board uh received an options paper that looked at all the various options, and we decided on an audiovisual OSCI that could be sat from either home or work as the most COVID-resistant option. And to do that on a big scale. And this was going to be uh always a stretch for us, but we we gave it sort of really considered thinking, and we we look we looked at you know who we might partner with, we looked at the experience of other colleges here and overseas, and we involved an Australian company that had developed an assessment platform and had a track record of providing assessments, although we were aware, even at the start, that the um scale and complexity of our OSCIS was something that would be uh you know beyond what they they had done to that point. That meant that we had a full window, but one that at the time we judged to be sufficient if we put all our attention and focus into um developing and and implementing this option. Uh, and we we moved the date from 30th of October to the 20th of November to just give us a little bit more space. Um, you know, and and then there there were we expected issues, and there were multiple issues, some expected and some not expected. And we were during this period working through those one by one and finding solutions or workarounds. The the issues continued right up to the point where you know, some weeks before the exam, we were wanting to pilot on a weekend. We're all set up to do that, and we had to delay that as well. Um on the eve of the actual exam, we we thought that we had covered these risks. We we knew there was still risk there, but we thought it was unlikely that you know a failure of the scale that actually happened was going to occur. Um I was myself an examiner. Um I was involved in the morning session, and um I can only relate my experience, which I think will shed some light on on what might have gone wrong. Uh, so we were we were trained, we we'd we'd practiced on the um assess our platform and the uh online scoring sheet. Um we we have sort of uh approached the day with um the sort of uh expectancy that you have before you go into a you know a big sporting event, but we we were you know reasonably confident that it w we're going to do okay, only to find um that I couldn't enter the reading room. Um I couldn't enter the examination room of the first candidate. I'm a little bit tech sappy and I'm thinking, oh, okay, that's just a lot of people trying to get in at the same time, and that that'll correct itself. Just be patient, keep on and keep calm. Uh, and then had the agony, as I know, candidates did, of watching the clock, which once started, uh just goes on for the next couple of hours, and seeing the time uh elapsing for my first candidate, um, still trying to get in, trying to find out what was going on through the chat, uh, the helpline, which were of course both swamped, and sat through you know, six candidates like that. So that that's that's pretty much an account of what actually occurred. Ollie, do you want me to go on to just because I think you're also interested in why we sort of cancelled the PM session?

SPEAKER_04 9:52

Yeah, yeah, absolutely. That would be um very relevant information if you could.

SPEAKER_00 9:56

Yeah, yeah. So for you, you know, the large number of candidates we had across the number of time zones that we we sort of uh operate across, we divided into morning or a m sessions and afternoon PM sessions so that you know the New Zealanders could be in the in the morning session, the the uh West Australians in the afternoon session. Uh there was a an executive group and and you know a sort of disaster response sort of uh process for you know for this and our all our exams. And we got together very quickly. There was about a 90-minute window between the end of the AM session and the commencement of the PM session. We spoke to our um really good IT staff. Um, we spoke with the provider, uh, and we caucused as um an executive sort of disaster management group. And although it was very apparent to us that the issue was we couldn't get into these rooms, um, having had those conversations, um, we were not confident that the PM session would be any better, indeed. Um given the the conversations, uh some of us were even concerned that it might not be even as good as the morning. We had um limited information about how many successful sessions had been done in the morning. We judged that by the number of mark sheets that had been posted online, but we realized that some of those mark sheets were actually empty except for an incident report. The incident, the report being that you know I couldn't get into the examination room. So that sort of um that count wasn't wasn't a valid one. So we really had a very poor idea of how many people had successful stations or or even more than that. And we were also uh aware of the level of distress that this would have caused people who went through that experience. And, you know, to be honest, um I I was part of it and it was pretty, pretty distressing. And we we didn't want to cause more harm, and so we made the decision to pull or cancel the PM session, uh, which meant that some people were able to sit some stations in the morning, uh, a minority, but but uh uh you know, a number that's worth considering. And nobody was able to sit in the PM session because it was cancelled. So that that that was a situation.

SPEAKER_04 12:30

Nick, you made reference to the psychiatry OSCI itself being sort of quite complex and that you made some comparisons to other college uh colleges that were also running either in-person or um audiovisual uh OSCI-style examinations uh through the year. What what is complex about the psychiatry uh OSCI scenario? Um, because I mean on face value, I suppose uh it's sort of more of a talking specialty. You would think that it would be kind of uh readily adaptable to an audiovisual um model. Uh, but I suppose in my mind there's things like maybe the numbers of of candidates is quite high. From what I understand, there's also uh different lengths of stations that might add some complexity as well. But from from your point of view, what what makes it complex?

SPEAKER_00 13:19

Um I I think that um we had reading rooms with reading material. Um we had um examination rooms through which the candidates would sort of pass the examiners and the role players would also be required in. I think four out of six of those stations needed to enter. And the scenarios were very much contingent, uh, you know, dependent on on those role plays. I understand um that there was at least a couple of examiners who actually did the role play and and the examiner role because the role player couldn't get in. Um and the um there there was an assessment app platform uh which has got um you know the the the score sheet and criteria for marking each domain, etc. Um, and has got sort of incident forms and um uh various other things that that are uh eluding me at the moment, and the scale of it. And the fact that um uh this approach required a synergism between the assessment uh platform and the platform that was providing the rooms for examination and reading. So I think that um was the was the complexity uh that we were dealing with.

SPEAKER_01 14:36

Okay, I just have a another question just to clarify what Nick said. Um Nick, you mentioned that you know this hand we did the A V with a scale that is much bigger than you know that what the company was able to provide before and and you know what we did in the past. And you know, what were the key uh factors uh leading to kind of now under college decision to do it up to dark scale compared to something a bit smaller than before?

SPEAKER_00 14:59

Okay, so look, a couple of points there. Firstly, um our uh November and April IDs, which were uh smaller in scale, um it we really learned the um human resource intensity of actually doing these exams, and that was one of the limiting steps for us to just stick with that um that approach and scale it up. That we just wouldn't have enough um uh support people. So e each each um candidate so has has examiners, role players, invigilators while they're in the uh reading room, um, and and then there are a number of support staff, not necessarily a ratio one to one, but a high um ratio of support staff just um supporting the process sort of behind the scenes. Um so um the reason that we went with this new approach uh was we are very conscious that um a large number of candidates did not have the opportunity to sit the OSCI uh during 2020 and and some parts of uh the early part of 2021, and uh also that there were more coming down the line that would add to these numbers. Now that's that's that's um that that's one thing. The other thing is that um in line with the ACER report, which makes recommendations that we need to move to online marking in the OSCI, uh, which we couldn't do in those first two ones that we'd done on Zoom. And secondly, that we need to um build the um proficiency as a as an organization with doing other types of exams like our um essay questions online. Um, this was another attraction to us in terms of the option that we chose.

SPEAKER_03 17:05

One of the points of criticism I've seen even in newspapers and so on, is it's sort of alleged that there wasn't any contingency planning for the exam. And as someone who's somewhat peripheral to the process, I have been involved in the past. It seems that there's quite a bit of contingency planning that goes into these things, but you can't cover every contingency, um, particularly when you're implementing a new process. And um there's a combination of paid staff at the college. But a large part of the work, as I understand it, is done by volunteers from the psychiatric community. I wonder if it's possible to respond to that idea that contingency planning wasn't a part of this process.

SPEAKER_00 17:49

Yes. Thanks, Andrew. Um and look, I hope I my answer does uh the question, Justice, and and I um I take on notice to answer this. But look, um we certainly uh gave great consideration to contingencies, um, and we did have a contingency plan for this exam, and I'll explain what that was in a minute. Um, but I'd just like to preface it with um I think many people don't understand that within the resources that we have, uh, both in terms of our staff and also our fellows who do um contribute to these uh exams on a pro bono basis, um, and also in terms of just the bandwidth of the organization to be doing a number of really, really complex things at the same time. The option of having a fullback face-to-face OSCI with all the stations uh different, and um you need to adapt stations for A B and they're different for face-to-face. You need to offer stations that you know people don't didn't know about in a in the failed OSCI, but the candidates that were able to get into stations, et cetera. So we although we would have liked to have had a you know a rapidly stand-up face-to-face OSCI or other type of OSCI as a contingency, and we did definitely explore that, it was way beyond our capacity. And um and so that's a really important point to make. Secondly, the um the things that guided our contingency plan were addressing those candidates who would be stopped from progressing to fellowship in the immediate sort of period. And so and our contingency was uh uh definitely uh uh uh in in relation to those people, and we had um the contingency of doing a work-based assessment, or even depending on the numbers affected, standing up an OSCI, but you know, a little way down the track on a on a smaller scale as required. And then, of course, um there was the fact that in March there would be uh another OSCI again at scale, which we would have learned from the experience from this one, which you know um uh people would be able to have that opportunity.

SPEAKER_04 20:15

Michael, did you have a question?

SPEAKER_02 20:17

Or um well, just touching on what you mentioned before about this affecting a number of candidates from progressing to fellowship, um, and we've heard stories of that on social media and also of um international graduates whose visas are dependent on um progressing through exams and those kind of things as well. Do we know how many people are affected in in that group?

SPEAKER_00 20:39

Um yes, Michael, we we have good in we have good data on those things, and and let me mention the SIMG candidates first because they always seem to be mentioned second. Um so we do have good information on the SIMG candidates and the particular issues that they are facing in relation to um, you know, expiry on the pathway, visas, etc. Um, we've also got good information about candidates in terms of their progression along the training trajectory. Um, and um that is why uh in the um alternative assessment pathways that we have given a detailed submission to the Australian Medical Council last Friday, um we we have stratified those affected by the um Avioski in November according to those priority groups, and we will deal with them for we will deal with everybody, but we will deal with those people first. Um, in relation to um the issue of how many qualified psychiatrists the college is graduating each year, this this is a continuing concern for us because the Commonwealth Workforce uh group has projected shortfalls of psychiatrists. Significant shortfalls. And so we are trying as a college to increase our training intake and thereby increase our graduation of fellows. Now, one of the things that is I think worth mentioning is we've got good data on how many fellows we have graduated over the last say eight years or so. So in you know, 2013 we graduated 153. Last year, affected by COVID 174. 2019, which was uh COVID began in 2019, but it it didn't have an effect on our um numbers. So we graduated 210. In 2021, thus far, we've graduated to fellowship 213. And those numbers will be added to with the outcome of the alternative assessments from the 20th of November OSCE failure. So I'm not saying there's no problem, but I I am saying that we are extremely mindful of this, and our strategies and responses are sort of focusing on keeping the pathway to fellowship open for those who are in their trajectory ready to do that.

SPEAKER_02 23:24

Thank you. And and more broadly than the OSCE, are there other training bottlenecks that have become apparent in the COVID situation that have prevented other people from achieving fellowship outside of recent situations?

SPEAKER_00 23:37

Look, um I I I think this is I think this issue is also a little bit separate to COVID. So you know, I think we have become increasingly aware that there can be bottlenecks uh in relation to um the positions for uh stage two consultation liaison and child and adolescent um rotations. And this seems to have a bit of an uneven effect uh between jurisdictions. I think smaller jurisdictions are affected more, but there are some centres even in the large states that that are also affected. Um and this is something that we were aware of for a little while, and um I think in fairness to the college, we have secured additional funding from the Commonwealth Workforce package uh that was announced in May in relation to increasing training positions. And in fairness to some of the jurisdictions, there's some good news there. For example, in New South Wales, um, the safeguards program is going to fund permanently uh an additional 25 um positions. Uh in Victoria, there's been under the government's uh recovery workforce program, I think there's 15 additional rotations this year and 30 uh 2022. Um, and I think there have also been some increases in CL positions and child adolescent positions in Western Australia. Um some of the other smaller jurisdictions, I think, uh you know, their their state authorities yet step up, and I think there still are some bottleneck issues in some of the smaller jurisdictions.

SPEAKER_01 25:27

Thanks. I have a uh a question that um but you know I think with the you mentioned a bottleneck of psychiatrists' training, and from what I see, you see that there's a uh increase in demand for psychiatrists, especially in the current COVID uh situation. I work in a private clinic and you know, for even for myself who's a registrar, the waiting time for new patients to come see me in to Wong is like up until May next year, and it's seemed to increase a lot compared to the previous years. Um you think the uh the increase of demand for psychiatrists will continue to to increase even you know with the new COVID not new normal, and you're uh is the college aware of of that, and you know, I'm not sure what you want to comment on that.

SPEAKER_00 26:05

Yeah, look, um uh thanks. Um even pre-COVID and pre-this exam crisis, um, that's been an issue. It's been an issue that has been identified in the Australian sort of national medical workforce um, you know, strategies and and plans. And it has some uh implications, I think, for us as a college and for other medical specialist uh colleges. And that is the issue that we really need to be thinking about um graduating as many generalist specialist psychiatrists as possible. I also think we we we need a you know the required number of subspecialty psychiatrists, but we also need to be thinking about um different ways that psychiatrists can work um so that they have the maximum um uh sort of impact on community need. And um we also need to be, I think, um uh encouraging and sort of um helping other non-specialist uh medical and non-medical professionals to be better able to offer mental health assessments and and basic interventions. And so um one of the other programs that we're we're developing is a um diploma of psychological medicine, for example, that might be very, very helpful for people who are not psychiatrists, they're working as doctors in rural settings or in in settings where they um would benefit from being upskilled in terms of basic uh psychiatry.

SPEAKER_04 27:53

Listening to the way you're talking about this November's AV OSCI and the scale of it being a problem, um, scale in the sense of having lots of candidates trying to sit. Uh, I think the number I've seen quoted is 270. We then are in the next round of OSCIS, assuming we're gonna have more people who are wanting to sit as well. So the problem's sort of getting bigger, and I like it's it's creating an image in my mind of sort of like you know, you've already got a problem, like the water's coming onto the boat, and then you're trying to get the water off the boat, but then there's more water coming in, and it's sort of like it's the problem's gonna get harder to fix. Um, so uh you you flagged earlier that there are alternative pathways that are going to be preferencing um SIMGs and also uh people who are waiting to sit the OSCI as their final hurdle, who otherwise would be um achieving fellowship. Can you comment at all on what those alternatives are going to be? I understand that uh we're recording this episode on the 10th of December. We're gonna be finding out very soon from what you had already mentioned. Is there anything that you can tell us about what's gonna be going on there?

SPEAKER_00 28:59

Yes, thanks, Ollie, and I'm happy to do that with the the caveat that um today is Friday, the 10th of December, and we put our detailed submission for what I'm about to describe to the AMC on last Friday, the 3rd of December, and we are expecting and and we require approval from the AMC before we can proceed. So um, you know, possibly when this goes to air, that result might be known. Um I think the analogy of taking on water is a good one, and we've been very mindful of that in the way that we have um worked up uh detailed alternative assessment pathways, mapping them to the OSCE competencies that would have been um assessed in on the 20th of November and demonstrating uh to the AMC that you know these alternative pathways have some meaning and validity and can give them some confidence that those that come through these pathways we can be confident that they have achieved those uh competencies at the level of an state three um candidate. So this is a little complex to do on a podcast, so bear with me. For those people who were able to complete some OSCE stations in the AM session, we are proposing that we can assess people who've completed two of the six OSCE stations and determine from that whether they've met the standard. For those who uh weren't able to complete two uh OSCE stations, and that means all of the people in the PM session, we're proposing a training portfolio review. So our college training does, I think, more in-training assessment and work-based assessment than I think any other specialist college. And we've looked at, well, what of that, because it's a lot of it is formative, what of that can we use? And we have formed a task force. The task force has allocated um Beth Cotze to lead a group to develop this. We we met yesterday for more than 90 minutes and went through in detail the criterion and the pathways and how we would do this and et cetera. And we expect to be able to make a confident assessment for a majority of candidates who were unable to sit the 20th of November OSCE in relation to their level of competency against the OSCE standards. So I think a significant group of those people will go through that OSCE assessment stage using that pathway. There'll be some that won't because, for example, the OSCI is set at stage three. And so one of the criteria that we're going to use in the portfolio review is that the candidate has demonstrated in at least one stage three in-training assessment and a detailed assessment of all the competencies that are listed in the in-training assessment form, and we've weighted certain competencies domains, you know, like medical expert, etc. And so not all the people who sat uh will have done that. And and we know that some people sit the OSCI pretty early, uh, you know, in stage two. And so it or they there may be people who who sat in in stage three, but they just don't meet the criteria that we've mapped. And in that case, they get a case-based discussion. And case-based discussion is a type of format that you know we're all familiar with, although for trainees it's formative. Um, this will be a summative case-based discussion. Um, we've used summative case-based discussions in our substantial comparability pathway for Synges. It'll be modified to make it less arduous, so we won't be um requiring them to do, you know, write up lengthy three cases, but but shorter um summaries of two cases from which two assessors will choose a case to do the case-based discussion around. One of those assessors needs to be sort of external and independent. And so that that's what we're putting up. And so we've we have in our submission, we have done, and our staff have been you know fantastic in assisting this work. We we've sort of mapped the OSCI competencies to these proposed alternative uh assessment pathways. We think that subject to AMC approval, the portfolio reviews can commence next week. We've stratified the large number of candidates that need to go through these alternative pathways, and we're going to um preference those, as I mentioned before, who are close to fellowship. We also will make provision for SIMGs who've got visa or extension issues, and they'll yeah, so um and and then the case-based discussions themselves, I don't think we can achieve that before Christmas, but we will publish the guidelines and we will be able to advise in the initial tranch who needs to go through for that so that they can actually line up a date in early January or mid-January for their case-based discussion. And as we work through in a stepwise fashion the candidates in the various categories, that information will come to them and then they can look for the case-based discussion, that can be done um from January onwards.

SPEAKER_04 34:53

As you were describing the alternative pathways, you mentioned that the people in the morning sitting who managed to get at least two out of the six stations done could be judged as um whether or not they passed the OSCI based on that assessment. I suppose for some of the people in the afternoon session who didn't get the opportunity to see any stations, they may feel like, well, um, there's a sort of a shortfall there. And I think that the alternative pathways you've described are really um comprehensive. I suppose I'm just interested to get a bit more qualitative information, a bit more of a feel from yourself as to what it was like to be sitting there in the morning session and what how stressful it was as a protest and having the A V system not working, and then how that informed the decision to cancel the afternoon session. Does that make sense?

SPEAKER_00 35:42

Yes, it does. And look, I'm gonna and I I think your question's got two parts. Um, and and the first part is about you know the the the fairness or or the ethics or the or the moral decision to cancel the PM session when there might have been a small number, but significant if you're in that number, who were able to do at least two stations. Um that's the first part, and then you're also asking about you know what was my experience and level of distress, etc. Um, look, um in terms of the first question, all of this went through our minds and was in our discussion, and people can judge whether we made the right call, um, or indeed whether there was a right and wrong call. There's no fairness in when these sort of things happen. It's it's it's um we in and and there's also the question, you know, should we encounting the people who were able to sit some stations in the morning? Is that giving them an unfair advantage? And uh, you know, that's a difficult one as well. On the other hand, um, we felt that we had a moral obligation to try and salvage a confident decision on the performance of those who'd been able to perform because we had those data points, and um we recognized the unfairness of this to the people in the afternoon. In terms of the levels of distress, look, um I I wasn't really aware of the true levels of distress until uh after the whole thing was over, and and and my experience of deeply or starting to deeply understand the levels of distress uh began at the town hall meeting that followed in the afternoon, and I think it was about three o'clock. But, you know, when you've actually sat through it as an examiner, uh, and when you're a psychiatrist and and you have a an empathic and imaginative capacity for others, and and and a number, you know, it wasn't just me, there were a number of us that had um participated uh through that AM session, including our staff who weren't in the rooms, but they're on the sidelines tearing their hair out. Um we had a pretty good idea that this was going to be very, very distressing and traumatizing for people. And we just we did make the call, and people can judge that we made the wrong call. We just didn't want to do any more harm. Um, but it's a wicked problem, isn't it? Because by cancelling it, you're doing harm as well. We and we understood that.

SPEAKER_04 38:23

Yeah, no, thank thanks, Nick. I mean, that really um, yeah, you think you've answered my question really, really um beautifully. Thank you. Um I've Michael, Brian, take take the floor.

SPEAKER_01 38:32

I've asked a lot of questions, so I think I I want to ask a question that you know it might be a kind of warm to few free to let me know that that goes uh or edit it at all. But you know, I I just from what you're saying, you know, there's quite a lot uh that's a way for people who were able to get to um tightened out for people who didn't manage to do any station at all in the in the RP. Uh, does the participant have a choice of what alternative assessment that they can take? Because you know, I imagine that some might argue that yes, they set it too, but they also stressed out that they might have underperformed and they might not want to be judged on those uh performance that they had in those two stations, and they might prefer a you know a new workplace assessment that they could do in the future. Or, you know, people who obviously the trend is that were not affected by this OSCE at all, they might feel that actually I would rather do a workplace assessment at a future date where I can arrange with someone and have someone I know supervise with an external factor uh compared to going to the entire OSCE, for example. You know, would you have any comments on that?

SPEAKER_00 39:43

Yeah, look, that's a really good point. And I'm racking my brains to visualize the flow chart, and and it's been a long day for me. I've I've chaired a three-hour uh education committee preceded by a two-hour clinical directors meeting. Um, and and and my visual um recall is failing me, but but just I I think I understand your your question, and it's a really good one. And and what we're trying to do in everything we're doing now is to demonstrate that we we we want to be flexible, we want to be considerate, and we want to be learning, and um we learn even from just a question asked on a podcast. So the question is if you um did two plus stations and and you failed, what are your options there? And uh my my memory is failing me. Um and so I'll take that one on notice. But you're proposing that uh you know they might re-enter the pathway at the sort of portfolio review and then have the case-based discussion if that didn't sort of um find a result for them. Um we we have thought about whether candidates should have the option to opt out of the alternative assessment pathway? In other words, say, look, thanks guys, I don't I don't like that, I'm I'll sit the next OSCI. And we have um agreed that you know any candidate um has the right to opt out of that alternative assessment pathway uh as offered, but still would have to go through the OSCI, you know, competencies at some point in the future.

SPEAKER_02 41:19

Will the alternative uh pathway count as a failed exam if someone doesn't get through the pathway?

SPEAKER_00 41:25

Uh no. So good point. So um the A, the OSCE experience of November 2021 will be a no-disadvantaged experience. And it's I'm sorry I haven't mentioned some of the really obvious things up front, like you know, full fee recovery and you know, things like that. But uh yeah, thank you for raising that.

SPEAKER_02 41:48

Thanks, Nick, for being so open about um uh the plan for um helping the candidates who sat the most recent OSCE. I'm just wondering if we could turn our thoughts to March next year when we're scheduling the next OSCE. Um and just wondering if you can tell us a bit about what that's going to look like and I guess how confident we can be that that will um go ahead as planned and not have the same issues as this most recent one.

SPEAKER_00 42:12

Okay. Um I wish I had a better answer for you, but this is my answer. So last night the board pushed out the closing date, which was to be today, to this time next week, the 17th of December. The answer to that question is somewhat contingent on the AMC's response to our alternative pathways. The other thing is that um given the COVID situation and where in in some places Delta's actually still, you know, significant, and we have Omicron emerging, and where um, you know, I I I work in a team here in in Sydney, and um uh one of my team is in isolation because uh she was a close contact, and and I know several colleagues who are in isolation because they were close contacts at the moment. I know it's going to be different in all other places, but you know, I don't think there's any certainty that uh COVID in some form will come to a cinema near you, wherever you are, in the second uh in 2022. And the second thing is that I think the board is really determined not to offer a scale A V OSCI before we can with absolute confidence, but after testing it at scale, uh say that you know we think we've got the the right product that's gonna work for us. So stay tuned. We we will advise in relation to March very soon.

SPEAKER_02 43:54

Yeah, so Sunica, earlier this year, the Australian Council for Education Research published their report. Report on the RANZCP training program assessments. Could you please tell us a little bit about their key findings from that report?

SPEAKER_00 44:08

Yeah. So just to remind everybody that the Australian Council of Educational Research review was something that we commissioned for ourselves. And the reason that we did that was in recognition that we have a problem with our essay style examination. And we wanted to understand that and see how we could best fix that. The scope of the review, in addition to the essay style exam, included the multiple choice question paper and also the OSCE. There were considerable sort of strengths and commendations for our exam set, and I think the OSCI did receive quite a bit of positive feedback. However, there were some things around the OSCI that I think we probably need to improve and tweak. One of the things that emerged from that review that I think is really important is concerns across the college that the three examinations, to many people, do not appear to assess all core psychiatric skills. And I think particularly many people feel don't really sufficiently test the sort of formulative assessment. The other thing is that it's apparent, and I think we many of us were sort of aware of this to some extent, that we really don't align our exams to the learning outcomes and our training programs sufficiently. And so, look, there's a lot of issues in the ACER report, and later on a Friday afternoon, they're the things that are salient for me. I I think it's important to say that there's also a context or a background of emerging science around best practice in medical college education and assessment. And that's something that the college is actively involved in. And so we're talking to our Canadian, New Zealand, and particularly UK colleagues about this and about and with other colleges, not just psychiatry. So we we have a plan for implementation of the ACER recommendations, which are numerous, so we've gone through and we've sort of prioritized them in terms of urgency and level of importance, etc. But to give you some idea of some of the things that we have done or in the process of doing, we decoupled the CEQ from the MEQ in August. That was a specific ACER recommendation. I think our feedback and our understanding of that was that that improved things. We want to do another cycle of decoupled CEQ, MEQ before we and we can combine the data to, I think, get a better evaluation of that. We have commissioned a review of critical analysis and critical thinking and how that how that is uh assessed in across our training and and examination processes. And you know, in a craft in a specialty that is very much learnt and taught in an apprentice model, and in a world that there is an increasing amount of new evidence and new sources uh which a psychiatrist needs to be mindful of and evaluating in order to be providing the best and safest care. Um, we actually, I think all of the education, the exam committee, uh and and I think the board feel that critical analysis and critical thinking skills are really important to being a competent, safe doctor in the future. So we don't want to just chuck this out, but what we hope to achieve with our review is to better understand number one, are we currently assessing this well? And number two, if not, what are the different ways that other colleges or that university uh courses, uh medical courses assess this stuff? And is there a need for us to change our the way that we assess it, i.e., the written SA cell exam? We're also mindful that there's a significant burden of assessment in our training, and this is constantly on our mind and needs to be on our mind as we we go through all the processes of improving our assessments. We've got a syllabus review tabled today at the Education Committee. I'd like to acknowledge the fantastic work of Brad Howe and Jackie Moniker. We've got an EPA review starting led by Wayne De Beer, who's um you know, brings fantastic expertise to this. And we've got an assessment framework mapping exercise and a development of an assessment framework that the terms of reference were approved at today's education committee, Lisa Lampy is going to lead that. And so that's going to be looking at mapping our curriculum to learning outcomes and assessments, etc., all mindful of the burden of assessment. And then also tabled at the education committee today. So stuff that gets tabled at the education committee has been in process for a while, let me tell you. Um, the Committee for Education Evaluation, Monitoring and Reporting is going to be providing a special cohort monitoring report for the education committee and for the board so we can track. I mean, we already do this, but this is a refined and a more detailed um monitoring report so that we can track cohorts and and see where where we've got issues. Um there's there's more, but that's about as much as I can remember.

SPEAKER_02 50:15

Certainly a lot going on behind the scenes, so yeah, it's good that that's been looked into. And I've certainly agree that the burden of assessment is quite significant for our college compared to some of the other specialist colleges. So it's good that that's being uh noted and and looked into.

SPEAKER_03 50:29

Um Nick, one of the things that I was thinking when I was watching the meeting where you contained the distress of the people who'd been trying to sit the exam and since then is that the college may be trying to do something that's beyond its current capacities. And that there may need to be either a change in the structure or a change in the sophistication of how the college operates, particularly in changing times. So COVID is one thing, but there's a an explosion in social media, for example. One of the big elements that's been mentioned in the the breakout of uh what happened with the exam is communication with the registrar body and a perception that the college doesn't necessarily hear and then act on what has happened. Now I stood on a few committees where we definitely hear things that are important to registrars, but we might not be as good at communicating that back to them that that has been heard and then something is happening. So um I'm wondering if there's any thought that out of this awful experience there might be improvements in how things are done with the college and whether there might be limiting factors. I mean, that sort of change, for example, might increase fees for fellows and for trainees or the fees associated with assessments and so on.

SPEAKER_00 51:43

Um Andy, thanks for that. And um I can only agree with you. I I think that um one of the things, and depending on how we how we respond to this, it could be one of the good things that comes out of this, that the way that we engage and communicate with our various stakeholders in the foreground at the moment, and very properly are trainees and CMGs, but actually the distress of this event has um rippled through other levels of stakeholder. And you know, so things like our um our sort of communication engagement with with branch committees, etc., comes up as well. Um we we have to learn from this and we have to change. I I have some ideas about how we might do that. I I don't want to sort of preempt uh you know a broader discussion about those. And I think I'm old enough and experienced enough to also know that we might need some expert help in in sort of developing our future from uh yeah, so and there might be a cost to that as you as you mentioned, but I think that that's a cost that will be a really, really good important investment for us.

SPEAKER_03 53:06

Are any of the other colleges that we're talking to from overseas further along that pathway than we are?

SPEAKER_00 53:12

Oh look, that's that's um that's that's a question I I I probably can't do justice to. I think, and and look, I'm giving my this is a personal view. This is not something that um I've sort of digested with the board, and it's not a board position. But actually, I don't think you even need to look overseas to find that we can actually learn from some other medical colleges. And I'm thinking about some of the work the surgeons have done in terms of harassment and bullying, which you know is distressingly high in our own trainee surveys and in the trainee, in the um the the doctor surveys that are done by um you know the Australian authorities. I I also think there are a number of colleges, even within Australia and New Zealand, that have a uh more effective voice at the board level for trainees. So, yeah, there's lots of there, I think there's many places that we can learn from, and it would pay us to actually do a sort of uh a reasonable environmental scan of all of this and see what it can teach us.

SPEAKER_02 54:24

Actually, that's an interesting point about um uh change from board level. Because I know a lot of trainees on social media have been requesting for a trainee rep to be appointed to the board. Is that something that has been discussed? I'm not sure if that's in confidence, if you can respond to that, Nick. But what are your thoughts on that?

SPEAKER_00 54:41

Well, look, it's it's a it is an issue that has emerged in uh correspondence and communications around this, uh, along with other issues. I I I don't want to speak too much to this because I think at the moment the board is appropriately really concentrating on the recovery process and the alternative assessments. But let me say, firstly, that it's not too many years ago that we had a referendum. Uh, and you know, I I'm on the record at that time supporting a greater voice and a voting rights for um trainees and also for affiliates. I think since then we've also discovered the unfairness of SIMG's not actually having member status. And the board has been prior to this exam crisis, the board was giving consideration to these sorts of things. And I I don't want to say more than that. And that's all minuteed in our board meetings, etc. So um there there is, I think, a time, and I and and there's there's also uh many people who would be supportive of this and and see the importance of this. But late on a Friday afternoon, I I don't want to be running ahead of the board on this, and so I'll leave it at that.

SPEAKER_02 56:01

Fair enough. Thank you.

SPEAKER_04 56:03

I mean, what the one thing that's sticking out in my mind as like a final comment on that on that um line of thinking, Nick, and like you know, again, if you don't want to answer this, don't that's fine. But the there was the resignation of the trainee representative committee members, and and that obviously that's an ongoing um issue, and the their representation at board level is something that's uh being discussed uh you know on social media as well. You kind of just answered that, but do you have any comments on that that specific issue with the with the reps um sort of offering in the resignation?

SPEAKER_00 56:39

Yes, I do. Um so I I know uh a number of the TRC reps and and and got to know um both co-chairs, and uh I I have to say that you know they're just such fine people, and they are everything that you'd want uh in in a person who's going to be moving on to become a colleague of mine. I was very saddened to hear that the TRC representatives were the target of some attack and and unpleasantness. I I I don't fully understand that, although I, you know, I'm I'm not stupid. I I I I think you know the the T the TRC was, I think, at times unable to be as open with uh the T the trainee constituents because of our procedures, and and sometimes it is necessary that um things are discussed or uh considered in a confidential uh sort of framework. I could also say that um the board, the whole board has been really feeling for the TRC representatives, and we are planning to meet with both the those that have resigned uh and those that are still remaining on the TRC uh in the in the very next week, basically, in the very near future.

SPEAKER_04 58:05

So, what I understand there was sort of there were two resignations there. There was the the the first um group who uh from what you said had suffered some sort of victimization from other trainees, and then there was another group who uh offered in the the voluntary resignation due to concerns that they weren't being heard. Is that how you understand it?

SPEAKER_00 58:23

Or um c can I um just say no, I I that wasn't my understanding, and but but but I'm not saying that's not correct, it's just that in the last fortnight I have had so many balls in the air that um I'm not completely fully informed. If if that was the case, I'm I'm really sorry to hear that. And I guess my only other comment is it's so important that we all, including me and including the board, but we all uh have got to learn from this, and by that I mean we have to think about and reflect on what went wrong in terms of this engagement, in terms of our structures, etc., um, and how can we make that work better?

SPEAKER_04 59:16

Um, so I'd like to uh offer a great thanks to uh Dr. Nick O'Connor from the team on the Thought Broadcast. Uh it's been a very enlightening and um informative conversation this afternoon, and we really appreciate the time that uh you offered up for us today. So thanks so much, Nick.

SPEAKER_00 59:38

Thank you for the opportunity, Ollie, and and thank you all for support.

SPEAKER_04 59:42

And then thank you, all of uh the rest of our panel. Thanks, Andy, thanks, Michael, and thank you, Brian.

SPEAKER_03 59:48

You're very welcome, Ollie.

SPEAKER_04 59:50

And as always, I'd like to thank David Beal at the College for his support in the production and recording of this podcast. He uh works tirelessly to get these uh podcasts out to all members of the college and also to Zidoni Apprentice for um creating our logo, uh Shadi Dave for our music, and also uh to uh Australasian psychiatry for giving us the opportunity to make this podcast. We will speak to you next time.