Personalised Persuasion: The hidden risks of AI and Consumer Protection Gaps
Corporate + Commercial Lawyer, Vladimir Kravchenko, offers a vivid scenario below to illustrate the risks of AI-enabled personalised persuasion and why nuanced consumer protection is critical—an issue explored in Law Squared's recent Review of AI and the Australian Consumer Law submission to the Treasury.
Personalised persuasion - picture this:
Imagine waking up from a bad dream and sharing it with your partner over breakfast. By lunchtime, as you scroll through your favourite social media app, the dream’s narrative reappears—this time as a movie clip styled by your favourite film director. The twist? The story ends on a high note, thanks to you buying Product X.
Later in the day, you share with a friend how much you miss a family member who has passed on. Within the hour, your phone delivers an ad—again in the style of a beloved film—featuring that family member endorsing Product X.
From then on, every online interaction seems haunted by deeply personal and evocative messages urging you to buy Product X. The messaging suggests that without it, you’ll never be fulfilled—and might even face a life of misery. Though you had no prior interest in Product X, the relentless pressure compels you to buy it, despite a hefty price tag.
Science fiction becomes fact
Ten to fifteen years ago, this would have sounded like science fiction. Today, it’s a plausible scenario thanks to the capabilities of AI in data processing and generation. This phenomenon has a name: personalised persuasion.
Personalised persuasion leverages AI to push consumers toward decisions they wouldn’t normally make. It can create internal conflicts, reinforce unhealthy consumption patterns, and exacerbate mental health issues. While still in its infancy, personalised persuasion is just one of the many risks posed by AI-enabled technologies.
But here’s the pressing question: Does Australian Consumer Law (ACL) adequately protect us from these risks? Should it?
AI and gaps in Australian Consumer Law?
In our submission to the Treasury, Law Squared highlights how AI-enabled goods and services expose gaps in the ACL. We argue that nuanced regulation is urgently needed to address these risks and safeguard consumers. Read our full submission on this emerging issue here.