Big tech tackles online safety for children and teens with new AI powered apps and controls

Parents concerned with protecting their kids online can take advantage of the newest AI-powered apps and controls to hit the market. Here’s the latest roundup.

Online predators, sextorsion, pornography exposure, cyber bullying, phone addiction are just some of the everyday dangers facing children and teens in cyberspace today.

US Surgeon General Vivek Murthy put out a warning to parents in February that the internet was no place for kids age 13 and under.

Murthy, instead, suggested parents wait until children are at least 16 years old before exposing them to what is on the internet due to the harmful effects on teen self-worth, relationships, and personal development.

Meanwhile, almost 50% of teens have been bullied, threatened, or harassed online, while more than half of all young adults believe they are addicted to their mobile phones, according to recent US surveys and reports.

For apps such as Snapchat, TikTok, Instagram, Facebook and Twitter, users 13 years old and over are allowed to sign up and create an account without parents permission.

To note, YouTube – currently facing accusations and legal ramifications, in both the US and UK, for illegally collecting children's private data – has its own YouTube Kids channel. Still, studies show many kids use their parents' accounts.

Even more disturbing, the average age at which children first see pornography is just 13 years old, according to a January study for the Children's Commissioner for England.

Surprisingly – or not – 40% of those teenagers surveyed said the most common online platform exposing them to illicit images and videos was on Twitter, with Instagram and Snapchat not following far behind.

What’s more, the survey showed that over half of all teenage girls, 16 years old and up, had been sent or shown explicit content involving someone they know in real-life.

"I truly believe that we will look back in 20 years and be shocked by the content to which children were exposed. The adult content which parents may have accessed in their youth could be considered 'quaint' in comparison to today's world of online pornography," said Children's Commissioner for England Dame Rachel de Souza, about the report.

AI powered solutions can help teens stay safe online

Technology companies have been paying attention to the dismal state of online security for children, most recently with the launch of two new machine learning inspired apps - one that will help teens remove explicit images and videos of themselves from the internet, and the other helps parents to filter content and monitor social media chats on their kids phones.

Regarding parental controls, the popular (and controversial) social video app TikTok announced Wednesday its plans to roll out a new feature designed to restrict daily screen time to just 60 minutes per day for all users under the age of 18 years.

Some see the move as a way to gain favor among US adults, who overwhelmingly dislike the app due to its invasive privacy collection practices.

The app has been recently banned throughout the US federal government.

“We're improving our screen time tool with more custom options, introducing new default settings for teen accounts, and expanding Family Pairing with more parental controls” said TikTok.

TikTok already automatically sets user accounts private by default for those ages 13-15, and only users age 16 or over are allowed to use direct messaging on the app.

The company said the family pairing feature will give parents a dashboard to breakdown statistics even more, such how often the app was opened and what times.

Another site making waves this week is the Take It Down online safety tool, provided by the National Center for Missing & Exploited Children (NCMEC) and funded by Zuckerberg’s Meta Platforms.

The service can help teens remove or stop the online sharing of nude, partially nude, or sexually explicit images or videos taken when the person was under 18 years old, according to the Take It Down website.

The free and anonymous tool works by assigning a unique digital fingerprint, or hash value, to the illicit image or video, which other online platforms can use to detect the same images or videos on their services or removal.

“Take It Down is made specifically for people who have an image that they have reason to believe is already out on the Web somewhere, or that it could be,” said NCMEC spokesperson Gavin Portnoy.

“You’re a teen and you’re dating someone and you share the image. Or somebody extorted you and they said, ‘if you don’t give me an image, or another image of you, I’m going to do X, Y, Z,’” said Portnoy.

The picture or video itself is never uploaded or revealed to any platform, only the hash value - something the makers say is an incentive for teens who may be too embarrassed to reveal the presence of the illicit material to friends, family, or aw enforcement agencies.

Take it Down also works with AI-generated images and deepfakes.

Facebook, Instagram, OnlyFans, Yubo, and Pornhub are all participating members of the service.

In 2017, Zuckerberg had the idea for a similar, adults only service to combat revenge porn, but it never took off.

Meta did help launch an alternate tool in the UK to support adult victims of revenge porn called the Stop Non-Consensual Intimate Image Abuse (NCII) tool, which also uses hash values to identify illicit material.

Both the Take it Down tool and NCII helpline can be used by any person across the globe.

The NCMEC also runs a CyberTipline to report online sexual abuse of children, which has seen a drastic increase in reports over recent years. ly, Family Keeper is another recently launched app designed to help parents navigate their children’s digital world using the power of AI.

The last launch of the week belongs to the Family Keeper app, an app designed to help parents navigate their children’s digital world using the power of AI, according to the site.

The app is synced to both the parent's and child's devices and offers geolocation, screen time scheduling, and keyword monitoring.

Family Keeper “enables parents to monitor social media chats and track their child's location; filters content to ensure kids aren't exposed to anything inappropriate or harmful,” said Noy Haber.

The app also has an educational aspect to it for parents and children to use together.

"Kids today are spending more time online than ever, and as online platforms—from social media to gaming, and more—continue to rapidly evolve, it's a constant challenge for parents to ensure their kids are protected,” Reason Labs CEO Kobi Kalif.

The company claims the app regularly shares safety alerts and advice from experts, including depictions of different scenarios children could face online and how best to deal with them.

Some critics state that none of the newly released apps and controls are foolproof, but it is a start in the right direction.

For example, Meta’s Take it Down tool can not remove images if they have been sent over encrypted apps, such as WhatsApp or Telegram, and the Family Keeper has to be installed on a child’s phone to work properly. TikTok’s teen users can log into other accounts or rotate devices while watching videos with friends to bypass the daily screen limit.