Conventional AI benchmarks typically focus on the content of responses, for example checking factual (e.g. MMLU ) or mathematical correctness (e.g. GSM8k). However, for many language model applications, the manner (or “personality”) of a model’s responses also matters to users, for example how friendly or confident responses are. Recent issues with model releases highlight the limited ability of existing evaluation approaches to capture such personality traits: a ChatGPT model version was rolled back over sycophant personality issues, other models’ personalities have been critised to overfit to the Chatbot Arena leaderboard.

In this talk, I will introduce Feedback Forensics: our newly released toolkit to measure AI personality traits. Using our toolkit, I will first share results detecting the personality traits currently encouraged by popular human feedback datasets (incl. Chatbot Arena). Next, I will discuss changes and trends in personality traits exhibited across model families and versions. Finally, I will take a closer look the personality differences between the Chatbot Arena and publicly released version of Llama-4-Maverick.

The talk will feature a live demo of our personality visualisation tool and attendees are invited to follow along via our online platform https://feedbackforensics.com/ (laptops are encouraged).

• Speaker: Arduin Findeis (University of Cambridge)
• Tuesday 17 June 2025, 13:00-14:00
• Venue: Lecture Theatre 2, Computer Laboratory, William Gates Building.
• Series: Artificial Intelligence Research Group Talks (Computer Laboratory); organiser: Mateja Jamnik.

In both federated learning (FL) and large language model (LLMs) optimization, a central challenge is effective learning under constraints, ranging from data heterogeneity and personalization to limited communication and black-box access. In this talk, I present three approaches that address these challenges across different settings. FilFL improves generalization in FL by filtering clients based on their joint contribution to global performance. DPFL tackles decentralized personalization by learning asymmetric collaboration graphs under strict resource budgets. Moving beyond FL, I will present ACING , a reinforcement learning method for optimizing instructions in black-box LLMs under strict query budgets, where weights and gradients are inaccessible. While these works tackle distinct problems, they are unified by a common goal: developing efficient learning mechanisms that perform reliably under real-world constraints.

• Speaker: Salma Kharrat, Kaust
• Monday 04 August 2025, 11:00-12:00
• Venue: Computer Lab, FW26.
• Series: Cambridge ML Systems Seminar Series; organiser: Sally Matthews.

abstract

Formal verification of hardware often requires the creation of clock-cycle accurate properties that need tedious and error-prone adaptations for each design. Property violations further require attention from verification engineers to identify affected instructions. This oftentimes manual effort hinders the adoption of formal verification at scale. This talk introduces Microarchitectural Control-Flow Integrity (𝜇CFI), a new general security property that can capture multiple classes of vulnerabilities under different threat models, most notably the microarchitectural violation of constant-time execution and (micro-)architectural vulnerabilities that allow an attacker to hijack the (architectural) control flow. We show a novel approach for the verification of 𝜇CFI using a single property that checks for information flows from instruction operands to the program counter by injecting taint at appropriate clock cycles.

To check arbitrary sequences of instructions and associate property violations to a specific Instruction Under Verification (IUV), we propose techniques for declassifying tainted data when it is being written to registers and forwarded from the IUV through architecturally known paths. We show that our verification approach is low effort (e.g., requires tagging six signals) while capturing all interactions between unbounded sequences of instructions in the extended threat model of 𝜇CFI. We verify four RISC -V CPUs against 𝜇CFI and prove that 𝜇CFI is satisfied in many cases while detecting five new security vulnerabilities (4 CVEs), three of which are in Ibex, which has already been checked by state-of-the-art verification approaches.

bio

Katharina Ceesay-Seitz gained several years of experience as embedded software developer and in formal verification related to functional safety at CERN and in the automotive field. She holds a Bachelor in software engineering and a Master’s in embedded systems, both from TU Wien. For her master thesis she received the Award of the Faculty of Electrical Engineering and Information Technology from TU Wien and the TÜV Austria Science Award. Currently she pursues a doctoral degree in hardware security at ETH Zurich, working on formal verification of CPU microarchitectures.

• Speaker: Katharina Ceesay-Seitz, ETH Zurich
• Monday 30 June 2025, 13:00-14:00
• Venue: Computer Laboratory, William Gates Building, LT2.
• Series: compiler socials; organiser: Luisa Cicolini.