What’s wrong with online survey research methods?
Perhaps the most significant trend in market and public opinion research in the past decade has been the emergence of online research methods as the dominant form of survey data collection. This trend has taken for three reasons: a) to leverage the expanding array of digital technologies and their rapid adoption across the population; b) to realize greater efficiencies and lower costs in collecting survey data; and c) to avoid having to deal with the challenges associated with telephone interviewing.
But what about the quality of online research? Is something critical being lost by this expanding reliance on online research methods? This question is addressed in the latest issue of MRA’s Alert Magazine, with a critique of online research methods by author Neil Chakraborty. He addresses a number of issues, but zeroes in the reliance on non-probability samples, which is widely considered to be the greatest limitation of online survey research.
Survey research blossomed in the latter half of the 20th century largely on the strength of the science of probability sampling that provided a statistically-credible basis for extrapolating results from small representative samples to the populations they are drawn from. This approach requires that every member of the population has a chance of being selected to be surveyed. This could be more or less accomplished when surveys were conducted by telephone, but cannot be done with Internet-based surveys because there is no online equivalent to telephone numbers (unlike telephone numbers, e-mail addresses cannot be randomly generated). This means that most online surveys rely on drawing samples from established panels of individuals who are recruited to participate through website promotions. However balanced such online samples might be, in terms of their demographic and regional characteristics, they do not possess the qualities of probability samples, and cannot be treated as such.
Chakraborty’s critique is not new, and his points have been covered by others (see for instance AAPOR’s 2010 report on online panels and 2013 report on non-probability samples). But it offers a useful overview of key issues, and includes an important admonishment to research practitioners to both focus on reducing survey errors, and be transparent about the limitations of their methods.
Practical advice about survey research goes video
When organizations, students and newcomers to the survey research business look for “how to” guidance, the choices are largely limited to textbooks, consultants and on-the-job experience. Now there is a new resource in the form of short practical advice videos that have been created by Elon University in Greensboro North Carolina (home of the Elon University Poll).
- Surveys in society
- What is sampling error?
- Methods of collecting survey data
- How to read a crosstab
- 7 tips for good survey questions
The material is basic, and the audience is primarily for beginners. But even seasoned professionals can benefit from refreshing their knowledge.
2013 Survey of American Jews -- Lessons for Canada
The survey addresses many themes, but chief among them is how Jewish identity and practice are changing among Americans, and the results may also be relevant to what is happening in the Canadian Jewish community. This was the focus of the Seventh Annual Elka Klein Memorial Lecture held at Congregation Darchei Noam in Toronto on June 23, 2014.
The event featured a presentation on the American survey results by Greg Smith, one of the senior Pew researchers responsible for the study. This was followed by a panel of Canadian experts who discussed how the American findings may or may not be relevant to the Canadian Jewish community (in the absence of any comparable survey of Canadian Jews).
The panel was moderated by Janice Stein (Director, Munk School of Global Affairs), and will include our member Frank Bialystok (University of Toronto), Bernie Farber (former CEO of the Canadian Jewish Congress) and Aaron Levy (Founder and Executive Director of Makom: Creative Downtown Judaism).
AAPOR launches new initiative on survey methods
The American Association of Public Opinion Research (AAPOR) righly bills itself as the leading association of public opinion and survey research professionals. In keeping with this role, AAPOR has just announced the launch of its latest special task force initiatives, in this case to encourage dialogue and deepen our understanding of today's survey methods.
The current research environment is characterized by an expanding array of methodologies, each with its own challenges — increased cost, under-coverage, low participation, mixing of modes, uncertainty in the links between theory and practical application. This trend is producing numerous reactions, from those who are developing and testing new techniques and innovative methods to address many of these issues, to those who have grown skeptical of contemporary survey methods, whether new approaches or traditional methods.
The Task Force will be composed of leading research methodologists, and will have three principal responsibilities:
- Organizing a "mini-conference" at the 2015 AAPOR Annual Conference (May 2015).
- Develop guidance to help consumer of survey data assess the quality of what they commission.
- Identify other ways to feature innovation and learning in this critical area of research methodology.
This type of independent, expert-driven initiative is critically needed at this point in time, and is well worth paying close attention to as it evolves over the next year or so. Watch for updates in this space.
Gallup Tracks Americans: Concerned Believers and Cool Skeptics on Climate Change
By Kristen Pue
Gallup recently concluded a series of public opinion reports on climate change, culminating in a final report that established three categories to articulate American public opinion about climate change. The report showed that age and political affiliation, but not level of education, are important in explaining public opinion on the subject. It also included trend data that can be used to compare the story of climate change skepticism in Canada and the US.
WHAT GALLUP FOUND
Gallup has released ten reports since March as part of its ongoing climate change series. Each of the reports addressed one aspect of American public opinion about climate change. The reports concluded that:
· Americans value the environment and a majority believes climate change is man-made. Fifty-seven percent of Americans believe that human activity is responsible for climate change, a proportion that has remained steady since 2001. Likewise, Americans prioritize environmental protection over economic growth and favor energy conservation over energy production to solve the nation’s energy problems.
· However, climate change is not a top of mind issue for most. Americans express a low level of concern about climate change, particularly when compared with other environmental concerns such as polluted drinking water, soil contamination, and air pollution. A majority of Americans believe the seriousness of global warming has been exaggerated.
· Terminology doesn’t matter. The term climate change has increasingly been used instead of global warming, in part because it is believed that climate change will be a more persuasive term. But Gallup found that there is no substantial difference in how the American public responds to the terms global warming and climate change.
Each of the first nine reports, summarized above, highlighted individual aspects of climate change public opinion. Gallup’s final climate change report attempted to pull these different snapshots into a single, coherent narrative. In particular, it determined that American public opinion about climate change is polarized, with two prominent clusters and a “mixed middle.”
CONCERNED BELIEVERS, MIXED MIDDLE, COOL SKEPTICS
Gallup organized Americans into three categories, according to their views on climate change.
· “Concerned Believers” (39%) believe that climate change is attributable to human activity, are concerned about it, and believe its seriousness has either been underestimated or given a correct level of attention in the news.
· “Cool Skeptics” (25%) do not believe that climate change is man-made, worry little about it, and believe news coverage has exaggerated the seriousness of the threat.
· The “Mixed Middle” (36%) constitutes those people that do not fit into the other two categories. They hold a range of views on the three questions.
Gallup then used this segmentation to offer some conclusions about demography and perceptions about climate change. They found that age and political affiliation are the key dividing factors on climate change opinion. 65% of Cool Skeptics identified as conservative, compared with only 18% of Concerned Believers. The majority of Concerned Believers are under fifty years old, whereas the majority of Cool Skeptics are older than fifty years old. It is worth noting that there is a relationship between age and political affiliation in the US (the average Democratic Party voter is younger than the average Republican Party voter). Interestingly, Gallup also found that climate change opinions in the US are not divided according to level of education.
TREND DATA: CONCERNED BELIEF REMAINS STEADY, COOL SKEPTICISM GROWS
Gallup also issued trend data for its three clusters, demonstrating the general trajectory of American opinions about climate change. The proportion of Concerned Believers has remained largely the same throughout this period. The Mixed Middle is shrinking, while the Cool Skeptics category has grown.
CLIMATE CHANGE OPINION TRENDS: COMPARING CANADA AND THE US
This section compares Gallup’s trend data on this issue to similar data for Canada. Although the Environics Institute’s Focus Canada 2013 Climate Change Report is based on answers to different questions, it is possible to qualitatively compare the two reports. Both attempt to measure and track climate change skepticism over time.
The Focus Canada data asks to what extent Canadians believe science has conclusively proven that climate change is caused by human activity. Notably, a clear majority of Canadians believe that climate change is man-made. Climate skepticism has risen and subsequently fallen, returning to just above 2007 levels.
The two surveys asked different questions, so direct comparison isn’t possible. But it is possible to draw some conclusions from analysing the trajectory of public opinion in each. In the US, climate change skepticism has been rising since since at least 2006, but in Canada views on climate change science have reverted, and are roughly the same today as they were in 2007.
Further, it is worth highlighting American and Canadian climate change public opinion in the context of major events and public discussions on the subject.
Climate Skepticism Began to Grow in 2006/2007
Uncertainty about whether climate change is attributable mostly to human activity began to grow in 2006 in the US and in 2007 (or earlier, as Focus Canada trend data on this issue only captures the period of 2007-2013) in Canada. There is no obvious reason for this to have occurred, but it may reflect the general intensification of public debate about climate change.
2006 would seem to have been an excellent year for climate change activists. An Inconvenient Truth was released and the Stern Report controversially argued the costs of adapting to climate change will be greater than the costs of prevention. That same year, the Environmental Protection Agency (EPA) in the US was sued for its failure to regulate greenhouse gas emissions. Massachusetts v. EPA was decided by the Supreme Court in 2007. The Court ruled in a 5-4 decision that the EPA has the power to regulate greenhouse gas emissions from automobiles and that it could not sidestep its authority unless there was a scientific basis for its refusal. It is surprising -- given these developments -- that American climate change skepticism grew during this period.
The 2007 Intergovernmental Panel on Climate Change Report is seen by some as a watershed moment for climate change science: it declared with 90% certainty that human activities cause climate change. However, climate change skeptics had some notable moments in 2007: questions were raised surrounding factual inaccuracies in An Inconvenient Truth and The Great Global Warming Swindle, a divisive climate denialist UK documentary, was released. Both Canadians and Americans became less certain during 2007 that climate change is caused by human activity, suggesting that rebuttals to An Inconvenient Truth and other climate change denialist publications were successful in calling into question the conclusive nature of climate change science.
“Climategate” Deepened Skepticism
In both data sets, climate change skepticism accelerated between 2009 and 2010, likely in response to the “climategate” scandal in which over 1000 emails between climate change scientists were stolen. Although this scandal did not actually unveil evidence that scientists had been manipulating climate change data, they did show that the scientists were very reluctant to share their data with people they saw as only wanting to make trouble. It is likely that this scandal impacted public opinion in both Canada and the US. In each case, the percentage of climate change believers -- in the US, “Concerned Believers” and in Canada those believing the science is conclusive that climate change is caused mostly by human activity -- reached its nadir in early 2010.
The Copenhagen Accord, which was signed in late 2009, likely contributed to the high rates of climate skepticism in 2010. The meeting in Copenhagen failed to deliver a global agreement on climate change, and was seen as a spectacular disappointment. This failure halted momentum on climate change action, and was followed with negative media coverage.
Belief in Climate Change is Slowly Recovering
In Canada and the US, the view that climate change is attributable to human activity is regaining popularity, but has not returned to 2007 levels.
The percentage of Concerned Believers in the US began to grow again in 2012. This may be because the sense of urgency expressed in the IEA report was made tangible for many in North America. Summer ice cover over the Arctic Ocean reached a record low. Hurricane Sandy catalysed discussion on climate change; most Americans now link climate change and extreme weather.
2013 proceeded with a leaked draft of the UN Intergovernmental Panel on Climate Change that made headlines, particularly for its findings on the implications of climate change for food security. This draft was timely, given the protracted heat wave across the Southern US in 2012, which prompted discussion about climate change and concerns about current and future wheat yields. The Intergovernmental Panel also declared 95-100% confidence that human activity is the primary influence on planetary warming.
Increasingly since 2010, Canadians and Americans believe that climate change is man-made. This may be due to a combination of increased scientific data about climate change – and efforts to articulate these findings in publicly salient ways – and extreme weather occurring in 2012.
For a comprehensive timeline on climate change, click here.
For more on climate change and American public opinion, check out the Yale School of Forestry and Environmental Studies’ Project on Climate Change Communication. This project, which has been ongoing for nearly a decade, conducts social and public opinion research on climate change; designs and tests strategies to engage the public on the issue of climate change; and empowers opinion leaders by giving them tools to more effectively engage audiences on climate change. As part of this mandate, they recently released the following report: Public Support for Climate and Energy Policies in November 2013.
The World Bank surveys the globe on open government
Sometimes important public opinion research does not receive the attention it deserves. The Global Opening Government Survey – a study undertaken through collaboration between the World Bank’s Open Government Practice and RIWI Corporation, a Toronto-based company – is one such survey.
Global in scope, the Global Opening Government Survey spanned 63 countries on every continent. It asked the fundamental question: Do citizens perceive their governments to be open, as well as why, and how they want government to be more open.
WHY DOES TIHS RESEARCH MATTER?
It’s an important topic. Government openness is an important democratic requirement; it enables citizens to make informed choices, to question policies and hold politicians to account. In this sense, the extent to which citizens perceive government as open may be a litmus test for a healthy, democratic environment.
This topic is also timely, given controversy in various countries surrounding how much citizens know about what their governments are doing – from news that the NSA’s PRISM surveillance system is being used on American citizens, to reports that CSEC (Communications Security Establishment Canada) has been using free airport wifi to keep watch on Canadians.
The survey method is innovative. The Global Opening Government Survey uses an innovative new technology for conducting online surveys with citizens on a global scale without relying on recruited opt-in panels. The method was developed from RIWI’s random domain intercept technology (RDIT), which connects with Internet users across devices and platforms who mistype a URL or type a URL for a website that no longer exists. On the error page, users are asked to answer a short set of questions.
It’s a hidden gem. The Global Opening Government Survey is worthy of attention because it hasn’t received any. If you haven’t heard about this study you are not alone; the data was presented at the World Bank’s Open Government Partnership Summit in London last year and then posted on a website, but has received very little media coverage.
One reason this study may have attracted little attention is that its publication was not accompanied by a comparative analysis of the findings that presented a global picture or cross-country comparisons. So the key insights and value of this research take some effort to identify.
WHAT DOES THE RESEARCH TELL US?
Perceptions of government openness do not correspond to measures of democratic quality. One might expect that people in well-established democracies would be the most likely to say their governments are open, since democratic systems rely on transparency and open debate. But this does not prove to be the case. The five countries whose citizens most believe their governments are open can all be found on the African continent: Tanzania, Ghana, Malawi, Liberia, and Kenya.
Developed democracies – countries that score well on democracy indexes like the Global Democracy Ranking – varied considerably in terms of public perception of government openness.Norway, Finland, Canada, and the Netherlands were near the top of the list in terms of citizens rating their government as the most open, while publics in the United States, Spain, and Italy were least apt to share this view. This discrepancy between actual level of democracy and perceived level of government openness suggests that populations across countries have different ideas and expectations about what it means to have an “open” government.
Citizens in developed democracies are less demanding for more government openness. More often than not citizens in developed democracies are less likely than others to express the desire for their government to become more open and to provide more information about what it does. They are also less likely to say they would be more likely to trust government or pay taxes if their government was more open. The results suggest that citizens in developed democracies are, comparatively, satisfied with current levels of government openness.
A few other notable findings:
Mongolia stands out. Of all countries surveyed, Mongolians have the least optimistic assessment of how open their government is, and the highest level of demand for more government. This is noteworthy given that Mongolia has a healthy democratic system with open information, a lively multi-party system with elections that are generally free and fair and an active civil society sector.
Only 25% of Latvians – less than any other country by far – think citizens should have a say in government spending. Denmark (42%) had the second lowest proportion of its public saying citizens should have a say in government spending. By comparison, this expectation is expressed by at least eight in ten citizens in such countries as Mongolia, Brazil, and Kenya.
HOW DO CANADIANS VIEW THE OPENNESS OF GOVERNMENT?
Canadians are generally positive about the openness of their governments. Of the 63 countries surveyed, Canada is 13th highest proportion of its citizens rating their government to be open; third among developed democracies.
Canadians also expressed a greater than average demand for more government openness, amongst developed democracies. Two-thirds (64%) of Canadians think citizens should have more say in government spending and contracting, which is a high proportion compared with publics in other developed democracies. Canadians are also more likely than citizens in most other developed democracies to say they would be more likely to pay their taxes if government were more open.
While these numbers are strong compared with other developed democracies, they are less than compelling in absolute terms. Just over half of Canadians would like government to be more open (53%), and to have more information about government (53%). A sizeable minority were unable to even offer opinions on these questions.
The Global Opening Government Survey provides valuable information about an important topic, and was conducted using an innovative new method. The raw data, publicly available online, is potentially useful for policy-makers, activists, and journalists. It is worthy of further attention and comprehensive analysis.
Majorities in 22 of 40 Countries Say Believing in God is Essential to Morality
To be a moral person and to have good values is it necessary to also believe in God? Many people from around the world answered “yes” in a recent Pew Research Center report; a clear majority in 22 out of 40 countries believe that morality emanates from one’s religious views.
In some countries, particularly in Africa and the Middle East, the view that it is necessary to believe in God to be moral and have good values was nearly universally held – for example, in Ghana there is 99% support for this perspective. In other countries, however, most people disagreed: in France only 15% believe morality is contingent upon belief in God.
The Asia-Pacific exemplifies this diversity of opinion: perspectives range from Indonesia and Pakistan on one end (where 99% of people believe it is necessary to believe in God to be moral) to China on the other (with only 14% support for this view).
Beyond the range of global perspectives on religion and morality, there are two additional findings worth highlighting.
Views about morality and God are closely linked to wealth.
People in richer countries are more likely to believe that morality does not require belief in God than people in poorer countries, a correlation shown below in this chart from the Pew Research Center’s report. For example, a majority of people in every European country surveyed hold that it is not necessary to believe in God to be moral.
Although people in wealthier countries are more likely to believe that morality exists independent of one’s belief in God, the USA and China are notable outliers. Just over one in ten people in China – a country of average wealth amongst those surveyed – hold that belief in God is necessary for morality. And, unlike all other western liberal democracies surveyed, a majority of Americans believe that theism is required for morality.
Viewpoints on this question remained remarkably stable.
Pew Research asked this same question in previous years (2002, 2007, and 2011), and opinions have changed very little in most cases. For example, Mexico’s responses have shifted by two percent between 2002 and 2013. Ghana is a notable exception: the percentage of people that believe it is necessary to believe in God to be a moral person was 73% in 2007 but skyrocketed to 99% in 2013.
This stability in opinions about whether belief in God is necessary to one’s moral character is notable given the substantial social changes that have taken place over the past decade. Austerity has done little to change the perspective of most Europeans on this question. The United States has adopted a more secular outlook on this question over the decade, but the change is slight – just 5%. And despite seismic societal shifts emanating from the Arab Awakening, the Middle Eastern countries included in this study retain the same aggregate views about morality and belief in God.
Where Canada Fits
Canada fits the trend that wealthier countries take a secular perspective on this question: just 31% of Canadians think it is necessary to believe in God to be moral, a view that has not changed since 2002. Of all countries surveyed, Canada is most similar to Germany. However, the percentage of people who believe that morality is inseparable from religion is smaller in most European countries than in Canada – among them France (15%) as well as countries like Italy (27%) that have large religious populations.
Younger and college educated Canadians are least likely to agree that morality is contingent upon belief in God.
How does this report relate to what we know about religion in Canada?
According to the Environics Institute’s Focus Canada data from 2011, eight in ten Canadians believe in God or a universal spirit. Likewise, most Canadians express a religious affiliation (69%), while just over a quarter identify as atheist, agnostic, or without a religion. So, most Canadians believe in God; at the same time, most believe that morality is not contingent upon believe in God. What explains this difference?
One possible explanation is that Canadians distinguish religious questions from certain other values. Lending some evidence towards this view, Canadians were asked in 2011: if they have a religious affiliation, do they share common values with those that don’t; if they don’t have a religious affiliation, do they share common values with those that do? A majority in both groups agreed that they share common values on most things.
Another possibility is that many Canadians may separate theism from morality because religion plays only a small role in their lives. For example, half of Canadians reporting a religious affiliation rarely if ever attend religious services.
The results of this Pew Research report reflect the complex relationship between religion and morality, a question that has become increasingly poignant as more people in rich, western countries identify as atheist or agnostic, while other parts of the globe are becoming more religious.
Asia Foundation Publishes Survey of Afghans as International Withdrawal Begins
In December 2013, the Asia Foundation released its ninth annual public opinion Survey of Afghans, which polled over 9000 Afghan men and women from all 34 provinces of the country. The Asia Foundation works throughout Asia to improve lives through projects supporting governance and law, economic development, women's empowerment, environmental stewardship, and regional cooperation.
Unlike a decade ago, Afghanistan is no longer front page news. But it might be soon, as the country is approaching a critical juncture at which it will either thrive or falter.
About Afghanistan: Why Survey Afghans?
Canada has been a part of the multilateral intervention in Afghanistan (the International Security Assistance Force) since 2001. Over 50 states have contributed to the mission, which is engaged in training the Afghan National Security Forces, providing civilian protection and conducting counter-terror operations. Afghan forces have now taken over leadership in securing the country, and the intervention is set to conclude at the end of 2014. Afghan government and military capacity has been steadily increasing, and sustained economic growth is encouraging. Still, considerable uncertainty lingers about what will happen to the country when international forces leave (although a scaled-back NATO train, advise, and assist mission will remain) -- particularly given difficulties with neighboring Pakistan, lack of progress in national reconciliation, and a recent suicide bombing incident. Noting this, and in light of upcoming elections in spring 2014, it is critically important to ask Afghans what they think about their country, the direction in which it is headed, and prospects for the upcoming year.
Surveying Afghans: It’s Possible
As you might expect, there are steep challenges attendant to survey research in Afghanistan. Because access to technology is spotty, interviewers must conduct face-to-face interviews. Getting a nationally representative sample is difficult given extreme insecurity in some portions of the country, and it is often difficult to survey women. Nonetheless, it is possible to do public opinion research in Afghanistan because over the past decade outside organizations have invested in the developing the requisite infrastructure nationwide.
The primary actor in this area is the Afghan Centre for Social and Opinion Research (ACSOR-Surveys). It conducts the field work for the Asia Foundation’s Survey of Afghans. ACSOR has a well-established survey research operation for in-person interviewing to a high standard across Afghanistan, which it cultivates through recruiting and training indigenous Afghan professionals.
Using the opportunity provided by this framework, in 2007 the Environics Institute teamed up with CBC, the Globe and Mail, La Presse, and the Munk School of Global Affairs to survey Afghans, asking what they think about their country and the presence of Canada and international forces.
Key Findings of the 2013 Survey of Afghans
The 2013 Survey of Afghans includes 89 questions on a variety of subjects, yielding a textured depiction of national and local public opinion. However, findings on four key issues stand out:
1. The Future of Afghanistan: A majority of Afghans believe the country is moving in the right direction, relatively unchanged since 2012. When asked why the country might be moving in the right direction, the top reasons were: reconstruction, schools for girls, good security, the active presence of the Afghan National Army/Afghan National Police, and an improved education system. When asked why the country might be moving in the wrong direction, Afghans are most likely to emphasize insecurity, corruption and unemployment. That security was listed as a reason for both optimism and pessimism suggests that Afghans may at some level feel safer but there remains a pervasive uneasiness about security. Interestingly, when asked about the biggest problems locally (rather than nationally), insecurity was well down the list of concerns (behind: unemployment, electricity supply, roads, and the availability of drinking water. There are, of course, substantial regional differences in how Afghans responded to these questions.
2. International Presence: The survey suggests Afghan trust in international civilian actors. Half of Afghans have some or a lot of confidence in international NGOs – for context, this is much lower than electronic media (68%), but higher than Parliament (47%) and equivalent to national NGOs. However, Afghans are afraid of international forces. 77% of respondents said they would experience some or a lot of fear when encountering international forces, the highest percentage for any activity listed.
3. The Upcoming National Elections: The survey reflects a mixed view of the upcoming elections. Although a majority of Afghans (61%) believe the elections will be free and fair, a similar proportion (59%) anticipated experiencing fear when voting in a national or provincial election. Moreover, Afghans described corruption as the main reason that elections might not be free and fair.
4. Women’s Rights: 90% of Afghans agree with the notion that, regardless of gender, everyone should have equal rights; although questions on particular rights yield lower levels of public support, the survey suggests consistent majority support for women’s rights.
· Political rights: most (83%) Afghans agree that every person should vote for himself regardless of what the community thinks, but only half (53%) say women should decide who to vote for on their own.
· Right to education: consistent with the priority placed on schools for girls as a reason that Afghanistan is moving in the right direction, Afghans see illiteracy as the most pressing problem facing women. The vast majority of Afghans (83%) believe that men and women should have equal opportunities in education.
· Economic rights: lack of job opportunities for women was seen as another pressing problem for Afghan women. Although women’s participation in the formal economy is extremely low (only 5% of women are employed), 63% of Afghans agree that women should be allowed to work outside the home; this has remained consistent since 2006.
The Asia Foundation's 2013 Survey of Afghans represents an important example of how it is possible to conduct high quality and relevant public opinion research in a relatively undeveloped war torn country like Afghanistan. It is critically important, for two reasons. First, it gives voice to Afghans; shielded by the confidentiality of a properly conducted survey, individuals are able to express themselves in a way that might not otherwise be safe. Second, with the question of “what will happen to Afghanistan?” looming, survey data – an aggregative instrument that can nonetheless capture nuance – provides us with the tools to contextualize new developments.
Download the full report here.
“What are we thinking?” Who is asking? Surveying the landscape of public opinion organizations worldwide
Market research: it’s ubiquitous – an industry that employs people to produce myriad surveys on diverse subjects from political preferences to cellphone usage to dessert food choices and whether you like the programs on TV. They are solicited by diverse groups – governments, companies, and the media – and conducted everywhere, even in war-torn countries. Millions of polls are out there, but most of them are proprietary, because they are done by private companies for individual clients, and as such never see the light of day.
Although market research makes up the vast majority of polling, many surveys are also conducted in the public interest. It is in this specific but important space that the Environics Institute works, undertaking public opinion and social values research on issues of interest to society writ-large, the findings of which are released to the general public. The questions are different and everybody gets to see the results, building our collective understanding of who we are as a society; for these reasons, although public opinion research undertaken for public consumption makes up only a fraction of all of the polls that are out there, it is vital.
Who in the world is active in this sphere? The Environics Institute recently conducted an analysis to find out who else is conducting public interest research.
Public opinion research in the public domain is produced by several different types of organizations:
- Not-for-profit organizations;
- Organizations with links to academic institutions;
- Organizations with links to governments and international organizations; and
- Private companies.
Global representation. Possibly due to language constraints (the search was conducted in English and French only) the search yielded 19 organizations from Anglo-Saxon countries – from the United States, the United Kingdom, Australia, and New Zealand. Of seven European organizations that were found, two were Europe-wide in scope while the remaining five were French, Swedish, Polish, and Italian.
The search also identified 29 organizations from Latin America, most of which are part of the LAPOP network and collaborate on the AmericasBarometer project (a hemisphere-wide survey on democracy, governance and civic engagement every two years). The Environics Institute is the Canadian partner for the AmericasBarometer.
Elsewhere in the world, there are active research organizations in Asia (one each in Hong Kong, Thailand, and India, the Middle East (Qatar, Palestine, and Jordan), as well as one each in Russia and South Africa.
Not-For-Profit Organizations. There are a number of not-for-profit groups doing research on public opinion and social values across the globe. Not-for-profit survey organizations fall into three broad categories:
- Those whose primary aim is to conduct public opinion research (i.e.: the Pew Research Centre). This group is the smallest, with only a handful of organizations worldwide.
Those for whom survey research is a stream of work, fitting into a broader, usually policy-related, purpose (i.e.: the Asia Foundation, the National Democratic Institute or the Chicago Council on Global Affairs).
- Those for whom survey research is not a primary function, but who might occasionally commission a survey on a topic of interest (i.e.: the Asia Pacific Foundation of Canada, Samara or the Broadbent Institute). This category includes by far the most number of organizations.
In Canada, the Environics Institute is the only organization in the first category, because the Institute’s primary aim is promoting relevant and original public opinion and social research on important issues of public policy and social change. The Institute aims to ask the important questions that no one else has asked. Organizations in the third category include the Broadbent Institute, the Manning Centre, and the Asia Pacific Foundation of Canada; these organizations typically conduct one or two surveys a year.
The search also uncovered two organizations that promote public opinion research worldwide. The World Association for Public Opinion Research promotes public opinion research through events, seminars, and as a hub for research. WorldPublicOpinion.Org is a collaborative project supported by a number of foundations, academic institutions, and not-for-profits; it aims to give voice to public opinion research that engages international issues.
Organizations Linked with Academic Institutions. Of 67 groups found through our search, 25 were organizations that are nested within academic institutions. Because social science researchers find public opinion data very valuable – we regularly grant requests from academics that would like to use our Focus Canada data! – universities often house their own institutes for public opinion research. For example, the ABAC Poll Research Centre at Assumption University is a Thai institute; Laboratory Analysis of Political and Social (LAPS – title is a translation) is Italian; NORC at the University of Chicago is American; and the Public Opinion Programme at the University of Hong Kong conducts regular public opinion research, eliciting responses from across China. A notable hybrid, YouGov has partnered with the POLIS Department at Cambridge.
Organizations Linked with Governments and Intergovernmental Organizations. Finally, public opinion research organizations can be linked with governments, to varying extents. Some organizations are essentially not-for-profits but they are established by statute and receive regularized funding; others can be run directly by government institutions. Our search yielded three such organizations:
- Eurobarometer is run under the European Commission; it publishes surveys twice per year on issues of importance to the European Union and has been in existence since 1973;
- The Public Opinion Research Center (CBOS) is a publicly-funded Polish research institute. CBOS primarily researches domestic social, economic and political issues;
- Finally, the Human Sciences Research Council was established in 1968, by statute, as South Africa’s statutory research agency. It has grown to become the largest dedicated research institute in the social sciences and humanities on the African continent. It conducts research on a variety of social issues, often using comparative analysis. Though survey research is not the only method that it uses, it does conduct public opinion surveys.
Private Companies. Although the public opinion research of private firms is often proprietary, some companies do frequently publish polls as well. Gallup International, perhaps the most famous name in survey research worldwide, fits in this category. It does regular surveys in 140 countries and publishes results. Two others that you might not have heard of are YouGov, a British firm, and Ifop, which is French.
Polls and surveys surface in the news frequently, but these often have to do with political horseraces or cover social issues in a superficial way, because the function of these polls is to provide click-bait to augment the public profile of a private company. Although there are few organizations engaged in public opinion research for the public domain – compared to those engaged in market research, at least – the work that is done by these organizations is critically important to the sustainment of vibrant, pluralistic societies.
Found an interesting public opinion study? Share it with us on Twitter at @Environics_Inst , or send us an email and we might post about it!
Public trust in institutions: Latest Global Trust Barometer finds widening gap between trust in business and governments
The results of the 2014 Edelman Trust Barometer were released January 20, 2014, the latest edition of an annual global study that has been conducted by the world’s largest public relations firm since 2000. The study taps public opinion around the world and asks about general trust in four major institutions: government, media, NGOs and business.
This year’s survey polled 33 000 respondents in 27 countries in late 2013 (although the survey is labelled as 2014). In addition to its general sample size of 1000 per country, it focused on a smaller group of 200 to 500 people in each country labelled “informed publics” – people aged 25-64 with a college education and top quartile in income, and report significant media consumption and engagement with news and public policy.
The following countries were surveyed in 2014: Argentina, Australia, Brazil, Canada, China, France, Germany, Hong Kong, India, Indonesia, Ireland, Italy, Japan, Malaysia, Mexico, the Netherlands, Poland, Russia, Singapore, South Africa, South Korea, Spain, Sweden, Turkey, the United Arab Emirates (UAE), the United Kingdom, and the United States.
Main Global Findings. In its latest survey, the Trust Barometer found that worldwide trust in business and non-governmental organizations (NGOs) has remained stable while trust in government and the media had decreased. The global public now trusts NGOs the most and governments the least.
More Informed, More Trust: Consistent with previous years, the survey found that in all countries the “informed publics” group was more trusting than the general public, based on the aggregate trust levels for all four institutions. For Canada, the “informed publics” group is 6% more trusting than the general public, a divide that is on par with most other countries. Data reported by Edelman, with a few exceptions, based its findings on data from the informed publics group.
Business More Trusted than Government: 2014 revealed the biggest ever gap between trust in business and trust in government; now a gap of 14 percentage points globally and the largest since the study began. This latest result reflects trust in business remained unchanged from 2013, while trust in government is down four points globally.
The gap between business and government has grown in a number of countries but notably in the US the US, where trust in government declined by 16 points over the past year (to 37%). By comparison, Americans trust in business is down only five points (to 58%).
Canadian Companies Trusted Highly: Companies headquartered in Canada are the fourth most-trusted worldwide (78%), very close behind Germany (80%), Sweden (79%), and Switzerland (79%). This high rating is despite recent scandals surrounding SNC Lavalin, which resulted in a huge jump in the number of Canadian firms listed on the World Bank’s blacklist (from 0 to 115). American companies are in the middle of the pack, while the BRIC countries and Mexico headquarter the least trusted companies worldwide.
Google Can Rejoice: globally, the public trusts the information gleaned from online search engines as much as traditional media, and more than hybrid media, social media, and owned media.
Canada-Specific Findings. As in past years, Canada places highly in the Global Trust Barometer. Although overall institutional trust declined marginally over the past year (from 62% to 60%), the country’s ranking inched up from 8th to 7th place, and is now tied with the Netherlands and higher than every other western democracy included on the survey. The latest year-over-year rating interrupts several years of consistently rising trust levels; between 2008 and 2013 Canadians’ trust in their institutions rose by 14 percentage points. It remains to be seen whether this past year reflects an aberration or a new downward trend.
Canadian trust in specific institutions:
- NGOs: While trust in NGOs rose globally in 2014, it declined in Canada (from 73% to 67%). Canada continues to be in the middle of the pack among countries surveyed.
- Media: Consistent with global trends, Canadian trust in media decreased slightly, but remains above the global average. Canadians trust media more than the US and all European countries except for the Netherlands. By comparison, trust in the media is much higher in such countries as China, India, Indonesia, UAE, and Singapore.
- Business: Canadians’ trust in their business community has strengthened over the past year (from 58% in 2013 to 62% in 2014), with this growth stronger than in other western democracies.
- Government: Canadians’ trust in government declined this year, by slightly more than the global average (from 58% to 51%). Canada has not experienced the steep drop in government trust recorded over the past year in such countries as the US, Mexico, France, Ireland, and Italy. At the same time, trust levels in Canada are now well below the UAE, China and Singapore, and is comparable to such countries as India and Indonesia.
Conclusions. Measuring trust is useful for two reasons:
Trust as socially desirable: Trust is important because it shapes how we engage with one another and enables us to rely on others, an essential characteristic for complex societies. As Nobel Prize winning economist Kenneth Arrow has pointed out, trust is important to nearly all economic activities, and so it is in everyone’s benefit when trust levels are high. The 2014 Trust Barometer shows that, globally (and in Canada), public trust in institutions has declined since 2013, though NGOs have retained high levels of trust. Moreover, some countries – such as UAE, Singapore, China, and Indonesia – have higher trust levels across the board.
Trust as a litmus test: Changes in trust levels (over time or between institutions) provides important feedback on how well these institutions are meeting societal needs and citizens’ expectations. From this perspective, trust levels should be responsive to events because trust indicates that the population believes institutions are doing a good job. The 2014 Trust Barometer reveals that governments in many countries are losing ground in earning the trust of their citizens, while trust in other institutions are holding steady.
At the same time, it is important to note that trust is not always the same thing as public satisfaction or support for institutional performance or policies. In Canada, for instance, overall trust in government as an institution (51%) remains well above public approval of the federal government’s performance (based on recent national surveys).