AI - some further thoughts

Welcome to the eighth InThinking Physics 'Silent Podcast', written back and forth between Emma and Tim in a shared document. We welcome your suggestions for future topics: put your questions (whether timely or timeless) in the Comments.

For today’s podcast, Emma and Tim are using Richard Allaway's thoughts in the seventh Silent Podcast as a starting point to explore how Physicists can best help their students use AI productively. 

Tim: 
So let's start by putting our cards on the table. Using the IB scale from 1 (Very Poor) to 7 (Excellent), how would you rate your AI capability? I'll start. I think I'm somewhere between a 4 (Satisfactory) and a 5 (Good). I know I need to gain more experience, but I'm not having to engage with students' use of AI on a daily basis. How about you?

Emma:
As tragic as it sounds, I reckon I'm at least a 6 when it comes to using ChatGPT to inform parenting my 6-month-old! That's not to say I'm avoiding expert texts or abandoning my own judgment, but I allow myself one theme for questions every evening - things like planning my weekly shop (so that there's a baby-friendly mini version for each meal), or gauging how clothing size is likely to change over time (so I know which fabrics to choose when second-hand shopping).

I've also used ChatGPT to help with translating between English and Nynorsk (the Norwegian language local to UWC Red Cross Nordic that seems to have evaded Google Translate).

On the other hand, I'm probably only a 4 when it comes to teaching. I'd say I'm skilled at finding ways to prevent my students from overusing AI in their coursework (using these Guidelines and this Process). But I haven't yet produced resources with Google's NotebookLM, for example.

Tim:  
I was particularly struck by Rich's comments on how schools should be supporting their teachers. Is this something that is happening (or going to happen) in your school? 

Emma:
We certainly haven't buried our heads in the sand. When ChatGPT became mainstream (from memory, in late 2023), we allocated an Education meeting for two students to train staff in how to use it and the ways they foresaw their peers engaging with AI. Our IB five-year review came soon after, requiring us to refer to AI (as with existing tools) in our academic honesty policy.

We also have a positive and proactive attitude to more formal professional development, encouraging colleagues to seek out AI-related sessions at conferences, and we have allocated time for staff to read and discuss the IB's examples of acceptable and unacceptable student practices.

I'm grateful to Richard for the suggested links and will bring these to the table when back to work in January.

Likewise, Richard's comments about assuming all students are using AI (all the time?) have really given me things to think about:

• Is AI perpetuating existing academic gaps between our students? Do we have a duty to educate our students in how to use AI to level the playing field? Should we pay for and expect at least a basic level of AI so that everyone has access?
• As a 16-19 DP college, conversation with students about academic topics is regular and formal (there is a weekly meeting between academic student representatives and education leadership, plus a series of six academic evening workshops for each cohort). But, if I'm honest, not all colleagues will recall our AI policy - and 200 students will always have the upper hand over our 30 teachers in avoiding detection. How can we find time and space for all teachers and all students to collaborate and reach a shared understanding?

If you were a physics department lead or DP Coordinator, how would you embark on this kind of project?

Tim:
Great question. For me I think I would want to aim to get students and teachers to be clear about when and where AI can be used to enhance understanding and when it should be left alone. Rich mentioned the work of Leon Furze and I was particularly taken by the idea of agreeing an AI Assessment Scale (AIAS) - the version in this link defines a 5-point scale that allows teachers and students to be clear the extent to which AI can be used. Get the scale debated and agreed across all departments in a school and you then have a shared language that would help everybody to know where they stand. 

This will, of course, take time so maybe a start would be just to get students to routinely be honest about how they used AI in any task that they have been set. Much better to have students tell you that they put the question that you set into ChatGPT and here is the answer you wanted, rather than doing that but passing it off as their own work.

Emma:
Yes, honesty is key.

The Artificial Intelligence topic feels like an addition to already extensive teaching responsibilities. I'd be interested to find out what a wider range of physics teachers think. In your Cat 1 workshops, do you come across recent graduates to whom understanding AI is ‘second nature’? At the other end of the spectrum, there must be teachers with decades of experience who remember the introduction of calculators and the internet in schools.

Tim:
Yes the situation is exactly as you suspect. Most teachers do have catching up with AI on their “to-do list”, but the day job and other commitments keep on getting in the way. In any given workshop there tends to be at least one, probably younger, teacher who is already constantly using AI in their day-to-day work. They are fluent at using AI for lesson planning, researching new topics and ideas, report writing and summarizing information. Interestingly they also probably use AI as the first port of call when dealing with any workshop task that they are set - which is already catching some workshop leaders out. A 45-minute workshop inquiry that involves reading two IB documents and summarizing them for the group turns gets solved by AI in less than 5 minutes. 

Such AI fluent teachers are, however, often in the minority and workshop delegates always have lots of questions about how AI is affecting teaching and learning. All delegates gain a lot by sharing how different schools are approaching the challenges so perhaps the most important message is to talk to other schools and share ideas.

And yes, you've guessed it. I do remember the introduction of calculators into schools (as a student) and the impact that the introduction of the internet had on my teaching. Unquestionably, both of these developments were significant causes for concern for teachers at the time. The good news is, that in the fullness of time and when properly utilized, overall their use improved students' conceptual understanding. I think we can be cautiously optimistic about the benefits that AI will also bring to the students.   

Emma:
Wow, I hadn't thought about the implications of using AI in workshops - even though I consulted it myself in my IBEN EE Upskilling!

Physics teachers (like me) can be complacent because assessed student work often falls into two camps:

1. Problem-solving, where we might think the steps are too small for it to be worth a student's time using AI
2. Investigation reports, where it would take more prompts to engage with the available apparatus and data than, say, a Group 1 or TOK essay title

However, we saw in our Experiments that AI is capable of producing accurate solutions in convincing notation.

Can you help me think a bit more deeply about the importance of process as well as product?

Tim:
I like your distinction between process and product. In the past it's been all too easy to assume that if you get a good product (they hand in perfect solutions to the work) then the processes are sound (they must have understood what they were doing). We have seen that checking the product doesn't guarantee that students have built up a secure understanding of the processes and relevant concepts. 

For me the key is to be with the students during the journey rather than just setting the task and seeing how they do. For example, when students are doing the IA the teacher can only get a real understanding of how much the student is involved by constantly questioning and getting the student to explain the progress. If you wait to the end before reading any draft then it could have been written by IA or anybody else. Questioning the student in a face-to-face conversation will soon allow you to understand if they understand what they have submitted as their own work.

I can imagine the classroom becoming a space where individual students work on their understanding by bringing their difficulties to the teacher. Work away from the classroom being the time where, with the aid of AI, students do their research and build their understandings. Perhaps students are also tasked with teaching other students with the teacher there to oversee the process and ask appropriate questions to challenge their understandings. 

Emma:
So now to the ultimate question: What is education for? How would you like to see the DP physics curriculum change in the next two or three Subject Guides?

Tim:
Great question and it's really interesting that you ask this as effectively this is same question that I tend use to begin a workshop! As previously discussed most teachers never read the first third of the subject guide (where the aims and the objectives of the course are spelt out). Nor do they read the last third where the assessment protocols are spelt out in detail - most teachers jump to the middle section that outlines the knowledge and understanding that the students need to develop.  

Sounds a bit cheesy, but for me the purpose of education is perfectly captured by the IB's mission statement: “…to develop inquiring, knowledgeable and caring young people who help to create a better and more peaceful world through intercultural understanding and respect” and “…[to] encourage students across the world to become active, compassionate and lifelong learners who understand that other people, with their differences, can also be right.”  

The specific aims of the physics DP course are clearly spelt out in the guide and it's quite a fun exercise to see if delegates do know what the course they are teaching is trying to achieve! Few know that there are a total of nine aims. Most do know, that it must be about acquiring a body of Physics knowledge (part of Aim 2 - along with methods, tools and techniques) but what else? It's on printed page 21 of the current guide. I particularly like aim 4: “develop the ability to approach unfamiliar situations with creativity and resilience”.  

Remember these are aims of the whole course and not assessment objectives. The examining team don't want to test students resilience in exams but they do hope that following the course has built up this skill. The assessment objects are heavily weighted towards problem solving and conceptual understanding and pay less attention to learning fact and solving routine problems. 

So how might this develop in the future? I think the trend can only continue. With AI and the internet, finding out facts isn't a particularly challenging skill to develop. AI's ability to problem solve is getting better and better so why shouldn't part of the course and the assessment be about developing a fluency with AI to help solve complex problems?Mathematics courses have embraced the use of graphic calculators and can now set problems that couldn't be solved under examination conditions without them. I've always felt that open book examinations (where the candidate has access to any materials that they want) have a lot of merit. Writing questions that work in this context isn't straight-forward as you can't rely on routine problems that have already been solved in most textbooks. Examiners also have to be careful not to give advantage to lucky students that have access to a wide range of resources over those with limited access.

The IB have, for sometime, wanted to ensure that all their examinations have some on-screen element. Note that this is not necessarily online, but it does allow for adaptive examinations to be set. The second question that the student sees is dependent on how well they answered the first one, and so on...  A really powerful way to check the difference between a silly mistake and a mistaken concept.

Finally, we are missing a trick if future physics courses don't embrace new developments in the subject. Traditionally this has been hard to achieve as modern physics is build on centuries of understanding so the temptation is ensure that the focus is on the basics (Newton's laws, etc.). It would be nice to think that a mid-21st century physics course should contain a significant amount of 21st century physics!