AI Expert Position Assessment - Round 2

Welcome to Round 2

We look forward to seeing your innovative approach to integrating AI technology in higher education. In your role as an AI Expert at our university, you will be instrumental in educating lecturers on AI applications for teaching and helping develop AI-powered educational tools.

Your Task

For this round, you will:

  1. Choose one of the two assignments detailed below
  2. Prepare and deliver a 20-minute presentation in person, followed by a discussion. The presentation format is up to you (screen/projector are available). Language can be either English or German.

Both assignments involve working with the OpenAI API to generate Anki flashcards using structured output. The two assignment options focus on different but complementary aspects of AI in education:

  • Teaching AI Development (Workshop Design)
  • Quality Assurance for AI-Generated Content

For both assignments, we provide the code examples for generating structured output from an LLM, and for using structured output to generate Anki flashcards.

Python code is provided here: anki-deck-generator.zip. Installation instructions are provided in the README.md file.

Before reviewing the assignments, we recommend reading the background information about Anki flashcards.

Background: Anki Flashcards

Anki is a powerful open-source flashcard program that uses the principles of spaced repetition and active recall to enhance learning and long-term retention. It presents flashcards at optimal intervals, adjusting the frequency of review based on the user’s performance. This is thought to ensure that information is reinforced before it is forgotten.

One of the key strengths of Anki is its flexibility. Users can create their own decks of flashcards tailored to their specific learning needs, or they can download pre-made decks from shared repositories covering a vast array of subjects, from languages and vocabulary to complex medical terminology.

However, Anki requires a lot of manual work to create effective flashcards. In this assignment, we will explore how we can use an LLM to generate the flashcards for us.

Flashcards

The simplest type of Anki flashcard is a pair of text fields: a question and an answer (or a term and a definition).

Anki can import flashcards from a variety of formats, including comma-separated values (CSV) files. A CSV file is a text file that uses a delimiter, e.g. a comma, to separate columns. Each line of the file is a row, and each row has of one or more columns.

If we have the following list of terms and definitions:

term definition
Metacognition “The ability to think about one’s own thinking processes”
Self-Regulated Learning “The process of taking control of and evaluating one’s own learning”
Executive Functions “Higher-order cognitive processes like working memory and cognitive flexibility”
Scaffolding “Support provided by teachers or peers to help students learn new skills”

we can save these in a CSV file:

term,definition
Metacognition,"The ability to think about one's own thinking processes"
Self-Regulated Learning,"The process of taking control of and evaluating one's own learning"
Executive Functions,"Higher-order cognitive processes like working memory and cognitive flexibility"
Scaffolding,"Support provided by teachers or peers to help students learn new skills"

If we save the contents above in to a file, we can import them into Anki by selecting the File -> Import... menu item.

Important Information

Presentation Focus

Your presentation should focus on high-level concepts rather than technical implementation details. We don’t expect you to create new code, or to modify or improve the example code. Neither do you need to present a live code demo. Key areas to address include:

  • Your conceptual approach to either:
    • Teaching AI development to educators, or
    • Developing quality control systems for AI-generated content
  • Your educational philosophy and teaching methodology
  • A critical analysis of AI’s potential and limitations in education
  • Specific strategies for making complex technical concepts accessible to non-technical audiences

Time Investment & Preparation

  • Recommended total time investment: <8 hours (don’t spend more than 1 day on this)

Assessment Focus

While we provide example code and technical materials, your presentation will be evaluated primarily on:

  • Effectiveness of your teaching strategies
  • Understanding of AI’s role in education
  • Ability to communicate complex ideas to diverse audiences

Assignment 1: Teaching AI Development Workshop

Demonstrate how you would teach lecturers to create an AI-powered Anki flashcard generator using an LLM. Give a presentation of your workshop concept.

Context

Many lecturers want to use AI in teaching but lack technical expertise. With varying technical abilities among faculty and limited training time, we need practical, hands-on approaches that enable immediate classroom application within a 3-hour workshop format.

Your Task

Design and present a concept for a 3-hour workshop teaching participants how to create an AI-powered Anki flashcard generator using the OpenAI API. While example code is provided for reference, you are not expected to write any new code or make technical improvements. Focus instead on how you would effectively teach the concepts and guide participants through understanding the existing implementation.

Workshop Goals

Participants should learn to:

  • Set up a Python environment for working with the OpenAI API
  • Use the OpenAI API to generate flashcards from educational content
  • Export generated flashcards to CSV format for Anki import

Presentation Requirements

Include:

  • Workshop structure and timeline
  • Teaching approach for different skill levels
  • Hands-on exercises
  • Approach to dealing with technical issues
  • Approach to dealing with questions about LLMs

Evaluation Criteria

  • Clear teaching strategy
  • Technical accuracy
  • Ability to explain complex concepts
  • Practical feasibility
  • Engagement with audience needs

Assignment 2: Quality Control for AI-Generated Content

Present a concept for a system that would ensure the quality of AI-generated educational materials. Focus on the conceptual design rather than implementation details.

Context

As AI-generated educational content becomes more prevalent, ensuring its quality and effectiveness has emerged as a critical challenge. Educational institutions need robust quality control systems and clear metrics to evaluate and improve these AI-generated materials.

Your Task

Design and present a concept for a system that evaluates and improves the quality of AI-generated Anki flashcards. Feel free to use the example code provided to help you. You may also present your ideas for how to improve the example code.

System Goals

Design a quality control system concept that:

  • Describes how to evaluate the educational effectiveness of generated flashcards
  • Proposes methods to ensure accuracy and appropriateness of content
  • Suggests approaches for improvement and iteration processes

Presentation Requirements

Include:

  • Quality assessment criteria
  • Technical implementation approach
  • Methods for automated evaluation
  • Examples of quality improvements

Evaluation Criteria

  • Technical feasibility
  • Quality metrics design
  • Consideration of educational effectiveness limitations of LLMs
  • Ability to explain complex concepts

Next Steps

  1. Review both assignments
  2. Select your preferred assignment
  3. Download and explore the provided code
  4. Prepare your presentation

Questions?

If you need any clarification or have questions, please contact: andrew.ellis@bfh.ch

Back to top