AI scams – damned clever and hard to spot

AI scams are now being used by criminals to enhance their success rate. They make scams more difficult to detect and increase how often you are targeted.

They also lower the bar for scammers, who once needed to be good con people to spin a convincing yarn. AI scams are increasingly used to fund ‘war efforts’.

Where AI scams begin

If you use the internet for social media (Facebook, Instagram, TikTok), forums (Reddit, Quora), online purchases, or have ever been in a database hack (Medicare, etc), you are very likely to have a dark web profile.

Additionally, Bad Bots continually patrol the internet, gathering disparate data that is filtered and sorted into your profile. They might, for example, know your children’s or pet’s names, where you like to holiday, where you shop, or even more personal data just by scraping your Facebook profile.

Data harvesters

Before we get into breaches, data harvesting has become the new river of gold. Here is what TikTok knows about you and, by inference, the Chinese Communist Party (CCP).

  • Name
  • Address
  • Birthday
  • Email address (especially those using .gov, .edu, .asn or companies of interest).
  • Biometric data – face ID, skin, hair and eye colour, height, weight, age, and sentiment (it parses all images and uses recognition and sentiment tools).
  • Biometric data – voice print and audio transcriptions- can create an account using that.
  • Object and environment recognition and content
  • Username and password (and as so many people re-use passwords, it is an open invitation to hackers)
  • Credit card details (where appropriate)
  • Phone number/type/IMEI
  • Contacts (Find Friends)
  • Private messages using the platform
  • Users you interact with
  • Network information, IP address (mobile or Wi-Fi)
  • Location at any time
  • Website visits via hidden, transparent tracker pixels
  • Websites you visit, what you view, what forms you fill in and what you type
  • What you are looking at – user content and what you comment on
  • What TikTok categories you most use
  • Facebook and Google profiles if you sign in via those. Worse, its data collection extends beyond users via other profiles, cookie tracking, etc.

TikTok is not what it appears to be on the surface. It is not just an app for sharing funny videos or memes. That’s the sheep’s clothing. At its core, TikTok is a sophisticated surveillance tool that harvests extensive personal and sensitive data. Australian Senator James Patterson

Then, you have data harvesters like Temu and Shein masquerading as online marketplaces. All that data is monetised.

Data breaches

If you were unlucky to have personal data as part of over 1000 breaches reported to the Office of the Australian Information Commissioner  in 2023, then your profile could already include

  • Full name, title
  • Date of Birth
  • Gender
  • Phone number
  • Physical address
  • Email address
  • Driver’s license
  • Emergency contacts like next of kin and their details
  • Medicare number or Health Care Identifier and expiry date
  • Tax File Numbers
  • Bank Account details
  • Pensioner Card
  • Commonwealth Seniors Card and expiry date
  • Department of Veterans Affairs DVA Card and expiry
  • Medical prescriptions
  • Passport
  • Bank and credit card number and CVV
  • Medical records
  • Academic results
  • Passwords

The biggest recent breaches in Australia include

  • ANU, November 2019, 19 years of student and academic data
  • Australian Parliament House – 2021, Liberal, Labor and Nationals databases
  • Canva, May 2019, 137 million users
  • Eastern Health, March 2021, 4 Melbourne hospitals
  • Latitude, March 2023, 14 million customers
  • Medibank, December 2022, 9.7 million people
  • Medisecure, 2024, 12.9 million Australians
  • Melbourne Heart Group, February 2019, 15,000 patients
  • Northern Territory Government, February 2021 – 4400 emails about COVID-19 check-in
  • Optus, September 2022, 9.8 million customers
  • ProctorU, July 2020, 444,000 remote students of many of Australia’s universities in every state
  • Service NSW, April 2020, 5 million documents, with 104,000 containing sensitive personal data
  • Tasmanian Ambulance, January 2021 – all users of this service from November 2020 to January 2021
  • West Australian Parliament, March 2021, Mail server (unknown)
  • Western Sydney University, 2024, 7500 email accounts

The biggest corporate data breaches  include (recent Australian relevant only)

  • AOL, 2024, 92 million users
  • Apple/Blue Toad, 2021, 12,367,232 Apple users
  • CardSystems (MasterCard, Visa, Discover Financial Services and American Express) 40 million users
  • Dell, 2024, 49 million customer records
  • Disney,2024, 1.2TB of undefined data
  • Equifax, 2017, 143 million users
  • EventBright/TicketFly, 2018, 26,151,608 users
  • Facebook Marketplace, 2023, 200,000 users
  • Facebook, 2019, 267 million users
  • Insomniac Games, 2023, 1.67TB including 1.3 million records and Wolverine game data]
  • Instagram, 2020, 200 million users
  • Lyca Mobile Global, 2023, 16 million users
  • Marriott/Sheraton, 2020, 500 million records
  • Microsoft, 2019, 250 million users
  • MyFItnessPal (Under Armour), 2018, 150 million users
  • MySpace, 2016, 360 million users
  • PayPal, 2020, 20,076,016 users
  • Quora, 2018, 100 million users
  • Reddit, 2021, not disclosed
  • Roll20, 2024, address, IP, gaming and credit card
  • Tesla, 2023, 75,000 purchasers
  • Ticketmaster, 2024, 560,000 ticket accounts
  • Trello, 15,111,945 user details
  • Yahoo, 2013, 3 billion records, then later 500 million records

OK, now you know where some of that data in your dark web profile came from.

Until recently, dark web mega-computers could not process all that data – sifting for gold and occasionally finding small nuggets. Computers could only sort and identify profiles ripe for scamming and sell lists to scammers. The success rate was a fraction of a percent, reflecting the list prices. Countries like Nigeria, Russia, China, North Korea, the Philippines, India, Brazil, Venezuela, and South Africa are prevalent.

AI has turned those occasional nuggets into rivers of gold, and now new players, such as the war-affected nations, including Iran, Lebanon, Palestine, Syria, Sudan, Yemen, Libya, and more, are scammers funding war efforts.

So, let me spell out an AI scam

A group in Yemen has just scammed me – at least, that is where the money flowed. You can read more PayPal scam – Happened to me, it could to you

I am a private person and highly cautious about the information I place online. I don’t have social media, and there are precisely three photos of me online. Why? I have seen the emergence of scams since the mid-90s and know what to avoid.

The PayPal scam’s AI-generated script was excellent. In my clear-headed post-analysis, I can see why I was convinced, and no alarm bells rang until it was too late.

Set the scene for an AI scam

  • The SMS came from PayPal’s registered number (it was spoofed, of course).
  • The call to action was that if I did not contact PayPal, the transaction would be approved (I should have known that PayPal always emails any transaction to me and requires multi-factor authentication). This was the bait!
  • The SMS requested that I contact PayPal, giving me a 61+ 2400 XXXX phone number. Australia does not have 2400 numbers, so I did not call.
  • I went online to PayPal, and there was a scam transaction and a different number: 61+ 1800 (we have 1800 numbers here). I stupidly did not check the PayPal number in the contact section of its website.
  • A professional auto-answer offered various legitimate options and one for reporting fraudulent transactions. No alarm bells!
  • The ‘consultant’ asked for my phone number and told me there had been a charge (correct details), where I lived, and other relevant information. He explained that to reverse the charge, PayPal would have to lodge an electronic dispute with my bank (logical). I said I could do that, and he said it was faster and would immediately prevent other scam transactions if PayPal did it. He correctly identified the first four and last four characters of the linked credit card number. I was a little apprehensive and said he would get his supervisor to talk to me. The supervisor was excellent, joked about the weather in Sydney today, and explained that PayPal was experiencing a huge number of these scams. Yes, I could go to my bank and wait a long time to get through, or PayPal could lodge the dispute immediately. Hooked and I gave them the missing pieces to scam me!

AI scams – coming to an SMS or email near you

So, AI concocted the scenario, the script, and Plan B (supervisor and local weather references) and put enough pain points in the way to let PayPal scammers ‘do it’.

Put yourself in my shoes. You get an SMS from, say, Myer/David Jones, Harvey Norman, JB Hi-Fi, Bunnings, Office Works, Tax Office, Centrelink, Education provider, church, et al., that there has been a transaction, and you need to contact the company to authorise or deny.

AI knows you too well and concocts a highly believable script for which even the best will fall.

We have only scratched the surface using an SMS scam as an example. AI scams include

  • Voice cloning – it only takes a few seconds of voice to replicate someone’s voice convincingly. AI can concoct a life or death scam (hospital, bail, accident, etc).
  • Deepfake video—This is more about using celebrities like Kochie, Twiggy Forrest, et al. to convince you to invest. AI scams have been known to use voice cloning to call you to add credibility.
  • Deep-fake romance scams
  • Generative AI images to reinforce the location
  • AI-generated websites that are indistinguishable from the real ones except that they steal your passwords and data.
  • AI Phishing emails using data about you to convince you that the email is legitimate.
  • AI marketplace scams that steal your data.

AI is constantly devising new ways to mine data and create emotional appeals you cannot resist.

Taking AI scams to the next level

The fusion of AI technologies takes scamming to a new level. It can generate entire fraud campaigns that combine code, text, images, and audio to build hundreds of unique websites and their corresponding social media advertisements. The result is a potent mix of techniques that reinforce each other’s messages, making it harder for individuals to identify and avoid these scams.

What to do

  • Scammers work on the law of averages – at least some will react to the SMS or email.
  • Stay calm, don’t react in haste and repent at leisure. Think hard about whether this could be a scam and verify it directly with the company or agency involved.
  • Never click on a link or call a number in an SMS or email. Again, verify the number and go directly to that.
  • Regularly change passwords and never use the same password for multiple online accounts.
  • If you use social media, do not overshare. Think twice about whether that information could be used against you. Many parents report getting SMS or Emails using their children’s names as a hook.
  • Set up a junk email address only used online.
  • Get a low-limit card for online purchases to limit exposure.
  • While I was ‘PayPal scammed’, it is still the safest option, as your credit card details are never used online.
  • CyberShack consumer advice
  • ACCC Scamwatch
ai scams
Previous Post
Next Post