Are testing the use of generative AI in the teaching of law and economics (from left): Sarah Großkopf, Professor Urs Kramer, Simon Alexander Nonn, Deborah Voß and Professor Johann Graf Lambsdorff.
The style of legal opinion is a science in its own right. Forming a major premise, recognising and defining the facts, summarising and drawing a conclusion at the end. Generations of German law students face the challenge of understanding the legal way of thinking and approach in their first semesters and applying it in assignments, but above all in all written examinations up to the first state examination.
Law is a textual science. With the current generative language models, there is now artificial intelligence (AI), which is also making astonishing progress in this area. Does it have the potential to revolutionise the law studies?
Sarah Großkopf talking to Professor Urs Kramer, the scientific director of the DeepWrite project.
The lawyers Sarah Großkopf and Simon Alexander Nonn are familiar with the pitfalls of studying law themselves. As research assistants at the University of Passau in the DeepWrite project funded by the German Federal Ministry of Education and Research, their tools are not thick legal commentaries in the shape and colour of bricks, but above all the computer. In their work with the AI tool, prompting is currently in particular demand as a skill. Formulating the work instructions for the AI so precisely that they reflect the legal procedure for feedback is a core task in the project. In order to be able to assess the structure and quality of student responses, especially in the first semesters, the prompt is being revised and adjusted on a loop-by-loop basis.
The interdisciplinary DeepWrite team, which includes lawyers and academics from the fields of economics, computer science, education and UX, has developed a tool that students will be able to access via the Stud.IP learning management system in the future. A test version - the “legalArgueNiser” - is already available to try out.
One advantage of the DeepWrite application is that it can immediately provide individualised feedback on content and style. This is not always possible, especially in the large lectures of the first semesters. However, the tool still faces the challenge of satisfactorily evaluating deviating approaches.
“It's utopian for the AI to correct real or rigorous exams, such as the intermediate exam, on its own,” says Großkopf. It can already correct smaller, standardised cases. “However, the tool does not yet achieve the same quality and depth as a good human corrector.” But that is not the primary goal. The use of AI in final examinations is not only problematic for technical reasons. This is because the requirements of the AI Act, which has governed the use of artificial intelligence at European level since August 2024, must be observed in the education sector, among others.
Juristinnen und Juristen wie Simon Alexander Nonn bringen in das Projekt ihre Expertise hinsichtlich Datenschutzbestimmungen und den Regulierungsvorgaben ein.
Education falls into the high-risk area and therefore particularly high hurdles apply to the use of AI. For example, its use in the correction of the First State Examination in Law, which is decisive for the further "course of a person's education and professional life", as stated in the AI Regulation, would be problematic. Lawyers such as Sarah Großkopf and Simon Alexander Nonn from Professor Urs Kramer's team are also responsible for such assessments in the DeepWrite research project.
Kramer is Professor of Public Law at the Institute for Legal Didactics at the University of Passau and the overall head of the interdisciplinary project. His team is not only testing how ready the technology is to be used in teaching. It also checks whether its use is legally unobjectionable. It is unproblematic, for example, if the AI supports students in learning and practising or corrects exercises in preparation for exams. This is therefore also the focus of DeepWrite.
Publications from the legal part:
- Braun C., Großkopf S., Nonn S.: Kann ChatGPT Lehrende an einer Universität ersetzen? Juristische Schulung, 2/2024
Hackl V./Braun C./Großkopf S./Nonn S. A./Müller A./Zwickel M.: KI-Feedback in der Rechtslehre: Eine explorative Studie zur Wahrnehmung und Bewertung durch Studierende, ZDRW 4/2024, 320 i.E.
Deborah Voss and Professor Johann Graf Lambsdorff from the economics part of the DeepWrite project work with the "econArgueNiser" application in teaching.
AI as a tutor in economics
The “legalArgueNiser” is a further development of the “econArgueNiser” from the economics part of the project. Economists led by Professor Graf Lambsdorff, holder of the Chair of Economic Theory, have been experimenting with online applications in teaching for more than ten years. It all started with classEx, a software programme that is now used in lecture halls and classrooms around the world. Students and pupils can use it to playfully experience experiments such as the prisoner's dilemma.
Melika Mirza Agha Khan (back left) is part of the team at the Centre for Information Technology and Media Management (ZIM), which ensures that applications such as the ‘econArgueNiser’ are integrated into the Stud.IP learning platform.
The DeepWrite project has given classEx powerful interfaces to various AI models - a development that inspires Professor Graf Lambsdorff: “It's incredible what the technology can already do.” As part of the project, the team has developed various applications with which students receive AI-generated, individualised feedback tailored to teaching content in real time. This includes the “ArgueNiser”, which students can use to specifically train their argumentation skills in an economic context. To do this, they enter their answer to an open question using various relevant argumentation modules and the application provides customised feedback on content and stylistic issues. DeepWrite tools such as the “econArgueNiser” are now being used in teaching in the field of institutional economics and economics.
In addition to developing applications, Professor Graf Lambsdorff's team is particularly involved in the scientific evaluation of the AI tools - as a substitute for tutors, but also as a possible correction assistant. "In principle, AI seems to be a little more generous than human proofreaders when evaluating student answers," says Deborah Voß, who works for DeepWrite together with Stephan Geschwind and Elaheh Alinezhad as a research assistant in Professor Graf Lambsdorff's department. She rates the potential of AI tutors as a supplement to teaching highly: Because with a large number of students, it is virtually impossible to do justice to each and every one individually. AI, on the other hand, can do this.
To keep the technology running, Professor Graf Lambsdorff's team is supported by programmers who have been specially assigned to the Chair of Data Science at the Faculty of Mathematics and Computer Science. This is because it is important to keep an eye on the rapid developments in the field of generative artificial intelligence. This is precisely the task of the computer scientists involved in the project.
Publications in the field fo economics, education and data science:
- Geschwind S., Graf Lambsdorff J., Voß D., Hackl V.: GPT-4 Feedback Increases Student Activation and Learning Outcomes in Higher Education, Research Square, Preprint, 11/2024
- Zubaer a., Granitzer M., Geschwind S., Graf Lambsdorff J., Voß D.: Can GPT-4 Replace Human Examiners? A Competition on Checking Open-Text Answers, Research Square, Preprint, 07/2024
Close to the developments in the field of generative artificial intelligence: the data science team led by Professor Michael Granitzer.
When an AI revolution requires reorientation
DeepWrite was launched in December 2021, one year before the US software company OpenAI brought the language AI ChatGPT onto the market and thus made generative AI in the form of large language models known to a wider audience. The fact that developers were experimenting with new possibilities for online searches was previously only known in specialist circles. This includes the team at Prof Dr Michael Granitzer's Chair of Data Science. It attracts scientists from all over the world to Passau who are leaders in areas of human-machine collaboration. For example, the CAROLL junior research group, led by Dr Jelena Mitrović, is developing ontologies that enable machines to recognise rhetorical figures. This knowledge is also used in DeepWrite, for example when - as in the example above - the aim is to teach the machine the legal style of expert opinions or to improve argumentation in the subjects of law and economics.
The DeepWrite concept originally envisaged that the interdisciplinary project team would use deep learning, a method of machine learning, to develop its own AI-supported assistance system. However, with the launch of the language AI ChatGPT, a generative artificial intelligence was suddenly available that already had these capabilities with regard to analysing language and producing text.
This is a problem for a third-party funded scientific project. This is because the approved funds are fundamentally tied to the application and the work packages envisaged therein. Scientists have to explain in detail what they intend to use the requested funds for during the project period.
“If a work package changes so fundamentally that you basically have to replace it, then that's cause for excitement,” explains legal scholar Sarah Großkopf. There is an obligation to justify this to the project sponsor, for example in the form of interim reports.
DeepWrite is funded as part of the “Artificial Intelligence in Higher Education” initiative. “We weren't the only projects that had to rethink,” says Großkopf. The BMBF brought the affected projects together across universities. They discussed internally, and in some cases also in joint workshops, how to proceed.
In the case of Passau, the computer science team is no longer focussing on developing its own prototype, but rather on observing and testing new models that have been coming onto the market on a weekly basis since the launch of ChatGPT. Which of these are particularly suitable for use in law and economics? How can applications such as the “ArgueNiser” be built on them? The computer scientists also experimented with the model from Aleph Alpha, the German AI start-up from Heidelberg, for example. The conclusion: although the model fulfils European data protection regulations, it cannot yet keep up with the capabilities of ChatGPT. Open source models are increasingly coming to the fore; a smaller model from DeepSeek is already being evaluated in the project.
Publication from the field of Natural Language Processing:
Links with other areas at the University of Passau
The fact that the applications are already being used in teaching at the University of Passau is thanks to the involvement of the Centre for Information Technology and Media Management, which is responsible for the management, development and provision of IT services around the campus. The centre works closely with academia to develop hybrid learning formats. The staff are developing ways in which the tools specified for law and business can be used on the Stud.IP learning platform in so-called asynchronous teaching. This means that students can use them to consolidate and practise what they have learnt in the lecture hall at home - with immediate individual feedback.
Yujin Kang is also researching how students and lecturers can positively perceive DeepWrite's offerings and the associated system and service experiences in the area of user experience (UX). The aim is to make it easier for users to utilise the new DeepWrite tool with the help of training materials.
The project is also being closely monitored from an educational perspective. The Didactic Innovation Labs are responsible for this with a team led by Dr Christian Müller and Veronika Hackl. This team designs scenarios for the use of AI in university didactics and analyses the effectiveness of AI-generated feedback. Research associate and entrepreneur Hackl also sees herself as a kind of AI ambassador: she is not only investigating how AI can be used to enhance competences. She also recognises possible reservations about its use. Educational work is therefore important to her.
She does this with the help of her Linkedin profile, providing an example of how an interactive form of science communication can succeed on social media. She regularly posts about new developments and studies; she formulates them briefly and concisely, categorises them and gets to the point in just a few sentences. The posts are met with lively interest and numerous comments. She discusses with users and also points out undesirable developments.
It is important to her to make it transparent how artificial intelligence works, keyword AI literacy: “We all need to understand even better what is and isn't possible with AI and what potential and risks are associated with it,” she says, adding that her expertise in the field of prompting means she also provides support in the areas of law and business time and again.
The DeepWrite project also emphasises transparency: the team makes the results open access, i.e. freely available, to the scientific community so that other disciplines can build on them and continue working with the findings from Passau.
This text was machine-translated from German.
Publication from the educational field:
