Survey Sentinel FAQ

What is the Survey Sentinel? 

The Survey Sentinel is a model that we (Angus Reid Group) have developed ourselves to identify and filter out potential bots and unqualified respondents. It utilizes a combination of respondent metadata, demographic information, and specific trap questions to calculate a score for each participant. This score is then assessed against a predefined threshold to determine whether a respondent qualifies or is disqualified. 

Does the Survey Sentinel guarantee the detection of all unqualified respondents? 

The Survey Sentinel will catch most bots/bad actors but not all – data should still be reviewed by project leads before closing field. In Canada, the model is optimized to detect about 89% of the bad respondents while flagging a minimal number of good respondents (~<1%) in a gen-pop sample. In the USA, the model is optimized to detect about 94% of the bad respondents while flagging a minimal number of good respondents (~<1%) in a gen-pop sample. 

Does the Survey Sentinel analyze open-ended responses? 

No, the Survey Sentinel does not currently analyze open-ended responses. Monitoring of these responses is managed by the project owner.  

Is the Survey Sentinel employed in every study? 

Yes Survey Sentinel is always on integrated into every study conducted. It is activated for internal panelists who have not previously been tested by it and is always run for external vendors.  

How do you catch the bad respondents in a survey if it only runs for some panelists? 

The Survey Sentinel is a key profiling tool, updated every 6-8 months and applied across the entire panel in Canada and the USA. Panelists who fail the Survey Sentinel’s criteria will be purged. Those not tested during the welcome survey or large panel deployment will be evaluated in their first participating survey. 

Can the Survey Sentinel be used in international studies? 

Currently, the Survey Sentinel is specifically tailored and calibrated for North America with a version deployed for international studies. 

How do I onboard a vendor with the Survey Sentinel? 

The Survey Sentinel should remain concealed until tested by a vendor’s project manager. After it is reviewed and tested by the external vendor, then the Survey Sentinel is made visible for further processing. 

Am I able to see which respondents are flagged by the Survey Sentinel? 

Absolutely. You can access the “SentinelSeg” variable in the portfolio and data exports, which categorizes respondents as: 

Bot >85: Flag for internal sample. 

Bot>25: Flag for external sample. 

Not a Bot. 

You can always review the respondents flagged as bots before removing. If you disagree with some of the flagged respondents, please let your PM know so that they don’t get removed from the survey. 

Is the Survey Sentinel intrusive or does it bias my data collection? 

Extensive testing has confirmed that the Survey Sentinel does not impact or bias survey results. To ensure the integrity of your data, the visible testing component of the Survey Sentinel is conducted at the end of the survey. 

How much length does the Sentinel block add to the survey? 

The test adds 1-1.5 minutes to the length of the survey. 

Do I have a choice of which questions to include in Sentinel in my survey? 

No, the model doesn’t offer any flexibility, but it is tailored to have the least burden on respondents. 

Are Sentinel questions the same for all Canadian studies? Do we plan to update them and replace them with new ones occasionally?   

The model doesn’t change and hence why it’s updated every 6-8 months. 

Does the Sentinel block of questions have a transitional introduction for respondents? 

Yes, it does: Lastly, we have a few general questions for you.