Hi everyone. I am Shao-Yu. I'm a PhD student from UC San Diego. Today I would like to share an educational game we've been building.

[click] Let me start with Premom — it's a fertility app. [click] People use it to log their periods, so they're trusting it with some pretty personal data. [click] And the company had promised users it would never share that data. [click] But in 2023, the FTC found it had been sharing it with third parties anyway. [click] The whole story's in this complaint: 29 pages of what happened and where it went wrong. Real, concrete lessons. But honestly — no developer is ever going to sit down and read something like this. So we turned cases like it into a game.

In this game, players learn about privacy incidents. These are the stories of real companies and real enforcement cases. The players take the role of an investigator. They navigate the large evidence space by choosing among investigation actions. So they learn about the company's privacy policies, data collection practices, etc. Let's say they choose to review their data collection behavior. Then the evidence automatically gets stored in the evidence panel. They can freely explore by choosing these investigation actions. Over time, they collect more and more evidence. Whenever they feel things are off, they can file a violation report. This can happen at anytime. They don't nee to wait until the end to make all judgement at a time. So they chose the misrepresentation of practices. Then they follow the structured form to assign the evidence to each slot. For this one, they need one for the claim, and one for the actual practice that violations the claim. On an incorrect report, the feedback tells them what parts to reconsider. Give them another try. So when they get it correct, the game explains the violation.

For learning to stick, we bet on three things: First, it has to hold the learners' attention. Second, you learn more working it out yourself than being told. And third, we learning transfers better when practices are grounded in realistic scenarios.

Privacy is everyone's job. Developers sit at the center of it. The everyday decisions they make — what data to collect, what to share, what the defaults are — directly shape users' privacy. But most developers were never taught to see that. We view this as an education problem. We ask: How can we help developers and the students who will become them be more privacy-sensitive?

Developers learn about privacy in a couple of ways. The first is privacy training courses, or materials for certification test prep. They prepare developers with concept-based and scenario-based questions like these. But the issue is that they are not realistic. They are mainly overly simplified, made-up scenarios. And the other issue is that people do not spend much time on them. Each question takes at most a minute to answer. A short attention window does not offer much in learning.

On the other side, some developers learn privacy informally — through reading the news. News about other companies' privacy incidents helps them build awareness, and see how the public thinks about privacy. These are real cases. But the signal is sparse — there isn't much news like this everyday. And it's easy to read these stories and feel it's someone else's problem, not theirs.

We've been working on privacy training for developers. We've built Panopticon and Privacy Jury — and Privacy Detective is our most recent addition.

Panopticon is a board game for teaching privacy design — a hands-on activity for educators. We took Monopoly, and reimagine its financial system as a data economy. Instead of buying properties, players own digital services. They propose their design, critique others, and refine their owns, so that their digital service don't lose trust from the users.

Privacy Jury is a global registry of real privacy enforcement decisions — over eleven hundred of them, across seven jurisdictions. By curating real enforcement outcomes, we help developers understand what consequences they could face when privacy decisions go wrong.

Building on top of these two, our new Privacy Detective turns enforcement cases into a game that you can really play through it. There are three key ideas behind it. First, it's a detective game. Second, it guides your reasoning. And third, it's built on real enforcement cases.

The first idea — why a detective game? The cause of these problems is usually not as obvious as just asking "what data do they collect." Take the health app from the demo. When you investigate, you run into all sorts of things — permissions, logins, reminders, an old privacy policy. Most of it is noise. [click] A few pieces are the ones that matter. It collects the health data users enter. [click] It brings in third-party analytics SDKs. [click] It logs events with descriptive titles, like "Log period-save." [click] And it told users it would never share their health data. [click] On its own, none of these looks alarming. The problem only shows up when you put two of them together — the event that gets logged and sent out, against the promise never to share. That mismatch is the misrepresentation. So to catch it, you have to dig in and decide where to look. [click] And a game can hold someone's attention long enough for that to happen.

The second idea — guided reasoning. Being able to explain sth is different from simply recognizing it. It requires a deeper skill to actually tell the reason behind. The structured report helps them build such explanation. You pick the kind of violation, and the form tells you what's needed to argue for such violation. And because the answer is structured, it can be graded automatically. So the players get immmediate feedback to fix their reasoning when they fall short.

The third idea — building on real enforcement cases. When we read these legal documents, we were pretty surprized at how concrete the descriptions were provided. For privacy policies, they usually include word-by-word quotes. For data practices, they go into details like what exact titles are used when logging the users data. [click] With these details, we provide contextualized and concrete scenarios for players to practice on. On the other hand, base on these enforcement cases, they can calibrate their sensitiviy to what the regulators really act on.

To build these game levels, we took the official complaints, mainly from FTC cases. We look into the counts, where it usually start by naming the type of violation, then points back to allegations back in the previous paragraphs. We map these counts to the report forms in the game. Where each cited fact becomes a field under the violation type.

And then we assemble the allegations and violations into a playable search tree. We cluster the facts into investigation actions, ensure that there are distractors to mimic the messy environment, and make sure that these descriptions make sense in arbitrary reading order.

To get some early insights on its learning outcome, we ran a user study with 24 student developers. Each of them started with identifying and explaining violations in a given scenario. Then half of them played the game. The other half read the complaint and the official press release. After that, they get a new scenario to identify and explain the privacy violations. We measure their improvement by comparing before and after the learning.

We used two metrics. The first one is the recall, which is among all the violations FTC alleged, how many did they catch? And the other one is reasoning completeness. For a violation they caught, did they give a full argument on all the cited facts?

Both groups improved on both metrics. For recall, the game group went up 24%, the reading group 10%. And for reasoning completeness, 22% vs. 12%. Both groups improved. And those who played the game improved more.

Here's the link to our Privacy Detective game. Feel free to try it out! There's also Privacy Jury and the Panopticon game we built. If you're interested in developers’ privacy education, I'd love to chat and collaborate. Thank you. I'm happy to take any questions.