Child protection app SafeToNet has denied spying on children after a news report suggested it had identified new sexual slang terms after accessing millions of children’s text messages.
Privacy campaigners raised concerns about the app after the company claimed it had found children were referencing Nando’s “peri-peri” sauce and “coleslaw” as sexual slang terms.
An article in The Guardian newspaper suggested the app had screened more than 65 million texts and identified girls as young as 10 using the slang terms in sexually explicit conversations.
This appeared to contradict the company’s claims on its website that its users were anonymous and that it did not collect any data from their devices, such as the content of their conversations.
SafeToNet’s website explains how the app uses a software keyboard “which suppresses all other keyboards running on your child’s device” in order to filter out harmful activities.
This keyboard “analyses in real-time what they are typing and guides them in-the-moment if risks arise” by sending prompts on the screen if their language is considered to be hostile.
The company’s chief executive Richard Pursey had also appeared in an interview to claim that the app monitored “the messages that children send, share and receive”, even though incoming messages could not themselves be accessible via the keyboard app, prompting accusations that the app was “spyware”.
A cyber security researcher who asked to be referenced by his Twitter handle @NetworkString found that the app contained a whitelist allowing advertisers clean access to monitor children’s web browsing habits.
Asked about these irregularities by Sky News, Mr Pursey explained that the app had been through many versions in recent years, but did not spy on children.
He said reading a description of the app as “spyware” was a “dagger in our heart” and said critics, whom he invited to get in touch with him, misunderstood the moral backbone of our company”.
The way that SafeToNet works is primarily not by filtering particular words which children might use on their keyboards – although some words are blocked – but through behavioural analytics rather than natural language processing, a different type of algorithm which measures behaviour instead of words.
An algorithm which runs on the phone analyses how children are typing. It compares this activity to what it expects of the child – for instance if they are sending shorter, terser, more regular messages – which is typically a signal that they are arguing.
These messages are then blocked, but they never normally leave the device and do not enter the cloud to be analysed by the company’s software.
But if the child interprets that the algorithm has made an incorrect decision, they can press a button on the keyboard which effectively appeals the block.
These alerts are sent off the device in what Mr Pursey said was an anonymous fashion – only the child’s age, gender and potentially location are viewable by SafeToNet – before being analysed by the app’s staff.
Mr Pursey also explained that a previous version of the app had been able to see incoming messages although this was no longer part of what it allowed, as it would be considered a criminal interception to access and change a message before it was displayed to the child.
The whitelist had also been part of a previous version of the app and newer versions had no mention of advertising technology at all, something which @NetworkString confirmed.
“We switched off a feature which could allow parents to turn off a child’s app or device,” Mr Pursey said of a previous version of the app, noting that it being used as a punishment by parents and “upset children” which went against their safeguarding mission.
In an email to Sky News, Mr Pursey said: “You are absolutely right to question our approach to privacy – there is nothing more important to us.
“We have built our entire app on a ‘privacy-by-design’ basis and so it cuts deeply when we are questioned about our moral code and ethical stance.”
“We clearly have work to do when it comes to getting that message across,” he added.