Facial Action Coding System (FACS) assessing emotions - is this a biometric?

When is facial recognition not facial recognition?  And when a school uses a facial action coding system (FACS) to analyse, in real time, a class of children's muscle movements from their face and body, not using a child's image, is this classed as biometric data processing?  This is a question I shall be asking the Information Commissioner's Office (ICO).  

Just to note, a child's fingerprint is not stored as an image but an algorithm - classed as a biometric, when processed in a school setting.

From a lay persons point of view, for the FACS to work it would have to first recognise that there are faces in front of the lens to process - hence facial recognition, therefore a biometric.  But as the system is not using the biometric to identify a specific human can we say that the school is processing an individuals biometric?  I would say yes, because without the collective children or individual child in front of the lens FACS would not work.  FACS can only work with biometric analysis.  Bio 'life' metric 'measure'.

FACS are employed in the Sens equipment being used in Smestow Academy.  Confirmed by the supplier ViewSonic's tweet.

The last couple of sentences of the second tweet is interesting:

"...we are in the early days yet.  Machine learning models take time to evolve."

Are the kids at Smestow Academy helping a FAC system "evolve"?  Is that ethical?  Do the parents and kids know they are helping to "evolve" a private company's commercial product, using the children's biometric muscle movements? 

-----------------------

How a Facial Action Coding System concludes - happiness, sadness, surprise, fear, anger, disgust, contempt.  

FACS potentially looks at lip, brow, nose, chin, neck data, nostril, jaw data whether muscles are tight or loose, head and eye movement, eyes blinking open shut, how much of the face is visible, shoulder shrug, sniffing and whether or not a smile is genuine... a lot of data.  

Here is the FACS inventors description of the technology:

UK schools using classroom biometric technology, data-scraping students faces to detect emotional engagement in lessons

I am writing this in frustration.  There is a video doing the rounds on Facebook and Twitter about schools in China using facial recognition to determine children's attentiveness in class.  It is understandably shocking to see this type of biometric facial/body scanning technology being used on children and the comments relating to the videos relay this.  

Little known is the fact we have the very same biometric technology operating here in UK schools, using biometric facial and body recognition data capture of students to generate 'mood indexing' information relayed in real time to the teacher in the class and school leaders.  It is called Sens and it is supplied by ViewSonic.  ViewSonic's Sens uses Intel's 'OpenVINO toolkit'. 

"To improve engagement rates, Sens measures every student’s facial expression for real-time tracking.  

Track reaction in real-time. From the front row all the way to the back of the room, dual lenses collect data from a full view of the classroom."

Sens biometrically captures a child's 5 emotions:  Happy / Sad / Upset / Amazed / Attentive and dutifully sends the teacher a real time alert to their phone to tell them how the lesson is going.  Placed on top of the white screen the lens silently acquires the breadth and depth of the classroom's human participants emotions.

(Above images from ViewSonic UK Youtube video detailing their collaboration with the City of Glasgow College)

How have we got to this type of technology being used in UK schools?  Simple - through a complete lackadaisical attitude from the UK Department of Education and an Information Commissioner's Office that is, and has been, devoid of regulating children's use of biometrics in schools over the past 2 decades.

The lack of due diligence displayed by these two institutions has enabled the use of biometric technology in schools to step over the boundary of services - using a face to buy a bag of chips, a fingerprint to access a library book - into the classroom to monitor students emotions.  Chinese style.

Okay, we have the consent in the Protection of Freedoms Act, the only legal right children have in schools to say no to their data sharing, but this is not well known and parents and students lack the knowledge of this IMHO, which is a shame because now we have sleepwalked into this situation where pupils emotions are being logged via biometric technology.

Biometric data is the most sensitive and irreplaceable data a child will ever own.  Why are schools using this data to see how students are engaging in a lesson?   

If a teacher cannot humanly know how their class is responding to their delivery of a lesson, either they are in the wrong job or maybe use the money spent on 'mood indexing' children to create smaller class sizes so teachers can 'get' how the lesson is going?

If 'mood indexing' biometric technology is accepted in UK primary and secondary schools by the Department of Education and ICO - then let's lead by example!  The next deployment of this surveillance technology surely should be installed in the House of Commons and Lords... live real time emotional engagement of our elected representatives.  Fantastic!

------------------------------

Smestow Academy, Wolverhampton - Youtube Hybrid learning at the academy, Freedom of Information request, written question asked in the House of Lords (Sept 2022) by Lord Scriven.  ViewSonic's case scenario.

City of Glasgow College - Youtube, Freedom of Information request, ViewSonic's case scenario 

There are another 3 schools, primary and secondary, in the UK that may be using Sens.  More on this to follow.

Facial recognition in UK schools - the 'right' or 'wrong' type of biometric to be used with children?

10 months after the suspension of facial recognition (FR) at all the secondary schools in North Ayrshire Council, the Information Commisisoner's Office (ICO) has not yet issued any opinion on the use of FR in schools but are expected to publish this on their website in the near future (from their response to a Freedom of Information request received August 2022).  

The very much sooner the ICO issues their opinion the better.  We are seeing biometric analysis of children's bodily and facial data taken in schools, move from services, such as canteen and library, to classrooms monitoring emotional engagement.

A brief history of facial recognition in UK schools:

2010

Facial recognition was first used in UK schools.  Schools from Northamptonshire, Hertfordshire and Cambridgeshire used FR from September 2010 for registration.  The FR system was ‘faceREGISTER’ supplied by Aurora Computer Services, now part of Causeway.

Enquiries to the then Sir Christopher Hatton School, a comprehensive in Wellingborough, Northamptonshire, (now an academy) revealed that the system was scrapped after a few terms.

Since this instance, FR was not used in UK schools until 2020.

2020

FR was trialled in the summer term of 2020 at Kingsmeadow Community College for canteen payments.  It was fully implemented in the Autumn term of that year. 

The FR uses the School Information Management System (SIMS) photograph as a template.
The students present their face at a scanner at the point of sale (POS) to pay for their food.

The school is part of a pilot scheme and no funds were spent on the FR system.

If students do not wish to participate the person at the POS can identify the child by their photograph.

More information here.

FR supplied by CRB Cunninghams.

2021

CRB Cunninghams are supplying around 70 schools with FR technology.

In October North Ayrshire Council (NAC) implemented FR from CRB Cunninghams into 9 of its secondary schools.  The technology last a week.  It was suspended due to concerns from civil liberties groups and and parents.  The ICO paused the technology to review its acceptability under Data Protection Act 2018 and GDPR.

West Lothian Council (WLC) were planning on implementing FR in their 11 secondary schools.  In a January 2022 response to a Freedom of Information request they state, “WLC had begun to consider cashless catering in secondary schools using facial recognition, but this has not been progressed”

It was unclear by the end of 2021 how many of the nearly 70 schools had gone ahead with their FR technology.  David Swanston, managing director of CRB Cunninghams stated, “some schools had shown concerns about the system as a result of the reporting on the North Ayrshire schools and the company is planning on issuing a statement to reassure them next week.”  That statement to schools, as far as is known, never appeared on CRB Cunninghams website.

2022

March - PRNewswire reported that Smestow Academy, in Wolverhampton, are using a brand new type of facial/bodily scanning of students to determine their wellbeing in the classroom… “the first school in the UK to deploy the AI-powered myViewBoard Sens analysis tool in the classroom.” supplied by ViewSonic.

“Through the real-time insights generated by the intelligent sensor, the school can ensure the classroom follows the wellness compliance, increase students' engagement, and facilitate a safe and active learning environment for the best possible learning outcome.”

Smestow Academy are working with Intel and ViewSonic to develop “experimental hybrid learning rooms” within the academy - see 1 minute 59 seconds in this video - creating a "software ecosystem" to "add facial-expression recognition and mood indexing to better respond to learners' needs based on non-verbal cues"

April - Biometric Update reported that Stonyhurst College would be implementing FR in 2023 which “will have mobile and facial recognition with two-factor authentication to minimise occurrences of students and staff copying PINs” for door access.  The FR entry system is supplied by Suprema.  A case study video from Suprema ID showcases the company’s 50 biometric and mobile security reader devices in use at Stonyhurst College.

Suprema also provide Wolverhampton Grammar School with FR door access with an additional feature to read the body’s temperature.

June - A parliamentary House of Lords debate on Biometrics Technologies on the use of FR in schools.  Lord Scriven asks:

“Facial recognition technology is now used in classrooms to monitor children’s mood and engagement, despite some parents objecting. The biometric regulator has no powers to enforce compliance with the law in schools and the department does not even monitor the use of this technology. Why are the Government taking this approach, allowing private companies’ marketing departments to determine the parameters of our children’s civil liberties and privacy in the classroom?”

July - the Department for Education (DfE) issued new guidance on the use of biometric technologies in schools, with notable changes to include FR.  Seemingly some types of FR are acceptable:

“Schools and colleges must establish that facial recognition is both necessary and proportionate within the school and college environment.”... with no explanation from the DfE how FR could be necessary or proportionate in schools.

“Live facial recognition is not appropriate in schools or colleges.” (A tiddybiddy round of applause here I suppose).  It appears the the DfE is deciding which biometrics are acceptable for use on children in schools when legislations, such as the Data Protection Act 2018 and GDPR, do not offer parameters of acceptable levels.  

Other data regulators in the EU, under GDPR, have halted the use of FR and fingerprints scanners in schools, with fines issued to schools using biometric technology in Sweden, France, Bulgaria and Poland.

August - CRB Cunninghams removed a webinar, detailing the learning capabilities of the FR system they are supplying to schools, from their website and Vimeo.  In the spirit of transparency for students and parents to make an informed decision on consent as to whether they participate in this FR technology, this is how students facial biometric data is used and processed - the webinar exists here:

"the idea with the third template is that this will be updated every 3 months" 17mins 1s

"the algorithm grows with the child" 17mins 20s

"the system will match that for you by constantly evolving the algorithm to match the child's growth and change of appearance" 17mins 49s

----------------------------------------------------------

Our data regulator, the ICO, has yet to publish their opinion on schools processing students facial biometric data - or body movements to give emotional data to school leaders and Intel (if they even know about this?)

Though I’m not sure how an opinion can be had on the use of biometric technology in schools when legislations explicitly detail how this technology should not be used with children when other less data intrusive systems will suffice -  and less data intrusive systems do exist in a lot of UK schools... PINs, swipe cards, etc.  And especially when other equivalent EU data regulators follow those legislations and do not hold opinions on the 'right or wrong biometrics' to be used with children.

State of Biometrics 2022 - a review of policy and practice in UK education

I co-authored a report with Jen Persson from DefendDigitalMe, 'The State of Biometrics 2022 - a review of policy and practice in UK education', which was published last week.  There is no collection, monitoring, record of biometric technology in schools held by any UK government department, regulator or commissioner so information in the report was obtained by Freedom of Information requests and research collated over the number of years I've been doing this blog.

There is definitely further research and questions to be answers and issues to be resolved following the publication.  

Biometric technology has crept from school services, canteen, library, etc, into the classroom - where 'sensors' (cameras) now scrape data from children into a group view for teachers and management to gauge real time interest in lessons, engagement, attentiveness and emotion.  This behavioural data capture is classed as biometric data under the Protection of Freedoms Act, Chapter 2 (28)(2).

From PRNewswire:

"ViewSonic Corp., a leading global provider of visual solutions, has partnered with the Smestow Academy in Wolverhampton, England, as the first school in the UK to deploy the AI-powered myViewBoard Sens analysis tool in the classroom. Through the real-time insights generated by the intelligent sensor, the school can ensure the classroom follows the wellness compliance, increase students' engagement, and facilitate a safe and active learning environment for the best possible learning outcome."

 

The Report in summary

The Protection of Freedoms Act 2012 and weak enforcement of data protection law are failing to protect millions of children from the normalisation of routine use of biometric data for everyday canteen and library transactions, and increasingly intrusive bodily surveillance in the classroom. 

Pupils in the UK are effectively guinea pigs in the use of emerging technology for companies from around the world, including facial recognition technology that has already been banned from schools in other countries.

Some companies claim that their products can measure mood and attentiveness, or use artificial intelligence to manage behaviour-based classroom planning. Some companies even claim to be able to detect autism without any child development expertise.

How this normalisation and these emerging products may affect the full and free development of the child are yet to be seen.

The fast-growing uses of intrusive technology involving bodily data in educational settings worry child rights advocates and law makers alike. 

Lord Paul Scriven said, “As parents you should be very worried and angry that private companies are seeking to make a profit from your child’s face, fingers, eyes and other personal characteristics while trying to pretend that it is all to aid their educational attainment. Where do we draw the line?”

• Data from FOI requests to schools (all UK) Our findings from enquiries to 550 schools with over a quarter of a million pupils in total, suggests that around three quarters of secondary schools are using fingerprint technology or other biometrics, and where used, uptake is routinely 85%, or more where use is restricted to only certain year groups.

• Despite the law requiring consent some schools in England are discriminating against children in receipt of Free School Meals (FSM) by obliging them to use the fingerprint systems, and others make it obligatory for all pupils. (We are yet to quantify these issues and plan to continue further research.)

• Emerging technologies, school trials and scope creep including under COVID. Schools started using facial recognition more widely in 2020. At least one school got new facial recognition technology free, “as part of a trial”. Some schools combined entry access readers with thermal and facial detection. Some are trialling “experimental” products including attentiveness and mood detection that are unevidenced in their intended outcomes or in any unintentional effects on children’s behavioural and cognitive development in UK classrooms. Voice is rarely considered under school biometrics policies whereas fingerprint technology is now routine.

• Lack of regulatory enforcement. Six months after North Ayrshire schools in Scotland put their facial recognition rollout on pause, there has been no visible ICO regulatory enforcement action. We include a latest position statement in the report.

• Large multinational companies have bought out the originally small school biometrics suppliers and many significant UK school providers are owned in the US, Canada and Israel.

• Parents’ survey findings (2018) Survation polled 1,004 parents with children in state schools on behalf of defenddigitalme about their experience of technology in schools. Over a third of parents (38%) whose children were using biometrics in school, said they had not been offered any choice despite the law that requires parental consent, the Protection of Freedoms Act 2012, and over 50% of parents had not been informed how long their child’s  fingerprints or other biometric data is retained or when the data would be destroyed.

Professor Fraser Sampson, Commissioner for the Retention and Use of Biometric Material and Surveillance Camera Commissioner and author of the report foreword, suggests there is inadequate oversight of school procurement among wide ranging comments.  “Some – including, surprisingly, the Department for Education – appear to have taken the view that bare compliance with Chapter 2 of the Act is all that is required to ensure the lawful, ethical and accountable use of biometric surveillance in schools.” He asks five key questions of practice in schools: Who's benefiting? Who's watching? Whose company are you keeping? Where's the push? And, Why the rush?

There are no UK national requirements for any quality or health and safety standards of biometric or AI technology when used by state schools, and no oversight or record of what is used where.

Current legislation is ineffective in protecting children’s and students’ rights in educational settings from age 2-25 and change is needed now. defend digital me is calling for a ban on biometric systems in educational settings.

Authors: Pippa King and Jen Persson

Artwork: Hannah Mallory

Update on facial recognition in UK schools

Since 2000 biometrics deployed in UK schools are/have been - fingerprinting, facial recognition, iris scanning and infrared palm scanning.

Department of Education and the Information Commissioner's Office (ICO)

• Do not list any schools using facial recognition (or any other type of biometric, i.e. fingerprint)
• Have not had any communication with companies that supply facial recognition in schools
• Have not issued any advice to educational establishments with reference to the use of facial recognition
• Have not logged any instances of violations of the Protection of Freedoms Act 2012, Chapter 2 - consent to use of a child’s biometric data
• Have not ever seen biometric hardware or software that are being used in schools

Evidence for this has come from Freedom of Information requests.

Biometrics Commissioners (and Surveillance Camera Commissioner)

• Have no jurisdiction with regards to children's biometrics, use and processing in education 

Children’s Commissioners UK

• Have no jurisdiction with regards to children biometrics, use and processing in education 

Legislation - Protection of Freedoms Act Chapter 2, Data Protection 2018 and GDPR

• Why does the ICO not enforce or rule as other EU countries (France, Sweden, Bulgaria and Poland) have done on the use of biometrics in schools?

What is the point of having legislation, if legislation is not being used or at least monitored for compliance?

Why has the use of children's biometrics slipped through the net of every single government department, various commissioners and an information regulator, ICO, which is meant to scrutinise the use of data - especially when this involves children’s data, with a use of technology that adults do not routinely use?

For over two decades now - biometrics in schools - there should be no excuses of this neglect of responsibility.  An immediate duty in this needs to be allocated to at least one of the above positions and organisations - the fact this has never been done is disgraceful.  

The ICO has yet to rule on the use of facial recognition by North Ayrshire Council, I await with interest its decision (if it is capable of making one).