password123: Applying behavioural insights to cyber security advice

Authored on
5 years 1 month ago
Complete
Project Type
Evaluation report
Policy Area
Cyber
Partner agencies
The Australian Cyber Security Centre (ACSC)
Registration date
Wednesday, 06 November 2019

To help improve the impact of cyber security advice for individuals and small businesses, BETA partnered with the Australian Cyber Security Centre (ACSC) to design and test different formats of advice. We conducted focus groups and two survey experiments (surveys with embedded randomised controlled trials) to understand whether behavioural insights concepts are effective in shifting people’s intentions to enact safer cyber security practices. We surveyed small and medium business (SMB) owners and operators and tested the effect of different formats of advice. We found some evidence messengers may have a small positive impact on people’s intentions to update their software, but we only have moderate confidence in this finding. We also found no effect from messengers on people’s intentions to use strong and different passwords across important accounts, and no effect on either cyber security practice from using different financial or non-financial loss framing. Overall, our research suggests making cyber security advice salient and engaging can help make key messages stand out. However, simply providing advice alone is insufficient to change behaviour, and further research is needed to better understand which formats, framing, and channels are most impactful for different groups.

This is part of a series of reports on applying behavioural insights to improve cyber security advice for individuals and small businesses in Australia. Other related reports are available here:

Registration date:

Trial 1: Wednesday, November 6, 2019

Trial 2: Monday, March 2, 2020

Intervention start and end date:

Trial 1: 26 August 2019 to 23 October 2019

Trial 2: 3rd March 2020 - 3rd April 2020

Ethics approval:

Trial 1: Bellberry Human Research Ethics Committee, BETA ETH 2019-001, 3 June 2019

Trial 2: Bellberry Human Research Ethics Committee, BETA ETH 2019-05, 11 September 2019

Experimental design including randomisation:

Trial 1:

Four-arm individually randomised survey experiment delivered as part of a survey collecting information on the cyber security behaviours of small and medium enterprises (SMEs). The initial experimental design was piloted and interventions were refined based on this pilot.

Trial 2:

Two separate (consecutive) experiments each with a 2x3 factorial design. After being individually randomised, participants will see advice on two cyber security behaviours. Advice will be varied as a factorial design along two axes, relating to the advice messenger (3 levels) and the framing of consequences (2 levels).

Randomisation for the second experiment is blocked on randomisation for the first experiment.

Intervention(s):

Trial 1:

After completing a survey, participants were randomised to one of three treatment arms to receive cybersecurity information in one of three formats: plain text, infographic or interactive quiz.

Trial 2:

During a framed field experiment, participants will be exposed to cyber security advice relating to improving cyber-security behaviours surrounding strong passwords and timely updating of software and apps.

We will test different ways of framing this information by varying the messenger delivering the information and the framing of potential consequences of poor behaviour.

Control condition:

Trial 1: The control group do not receive cybersecurity information.

Trial 2: It is an attentional control; participants will see the same advice as those in the treatment condition, but without a messenger effect or financial consequences.

Outcome(s):

Trial 1:

  • Average number of correct answers on a test detecting whether 3 emails are genuine or fake.
  • Self-reported intention to update business software
  • Self-reported intention to backup business data

Trial 2:

  • Participants’ self-reported intentions to create strong and different passwords across their important accounts
  • Participants’ self-reported intentions to update software on their devices immediately after being prompted.
  • Participants cyber-security knowledge (password and update behaviours) at the time of exposure to our intervention
  • Participants’ cyber-security knowledge (password and update behaviours) at the time of our follow-up survey (2 weeks later)
  • Participants’ self-reported behaviours around password and update behaviours at the time of the follow-up survey (2 weeks later)

Expected sample size: 

Trial 1: 1,186 small and medium businesses assigned equally to each of the four arms.

Trial 2: 4500 recruited survey participants, representative of the Australian population.

Other:

Trial 1:  AEA registration

Trial 2: AEA registration