Objectivity in Public Life

Public officials are explicitly intended to be ethical role models. The Nolan Principles of integrity, leadership, selflessness, honesty, openness and accountability also include objectivity, noting:

Holders of public office must act and take decisions impartially, fairly and on merit,
using the best evidence and without discrimination or bias

Nolan Principle 3 – Objectivity (as revised in 2013)

The notions of impartiality, fairness and merit are fairly well understood, even as they can be difficult to specify in particular circumstances, especially when information is limited. Similarly, the notions of discrimination and (prejudicial, as opposed to statistical, although analysis may offer some assurance) bias relate to established legal prohibitions, and are consonant with impartiality in public imagination. So the significance of “using the best evidence” is in the process of impartially and fairly assessing merit, but what does it mean to use the best evidence?

Evidence

Discussion of quality of evidence takes for granted that this is well defined, largely because evidence is understood to be judged with regard to support for a specific conclusion. However, determining the merit of a particular decision first requires identifying credible options, and in fact that a decision is required. Hence the framing of any action is intrinsic to the role of any public official, and pre-determined analysis is often not going to conclusively resolve the action which evidence indicates is fair etc. Moreover, the choice of framing can specify trade offs, whose impacts bear on both merit and fairness, or obscure consequences which are important but at some remove from available evidence.

Thus although it is conventional to set evidence within a hierarchy, the best evidence is from robust studies of the same question (at least for more fixed factors). To the extent that practicalities usually make decisions considerably more complicated, or arise on (or because of) entirely new questions, synthesis is necessary. But further research can also be commissioned, and initial decisions staged to proceed as evidence becomes more clear cut, with reference to the timing constraints of decisions. Conversely, decisions can be presented as advice or precautions while emerging risks are evaluated, again framing the question (and structuring sub-questions) is a key responsibility of public officials.

Robustness

Notwithstanding difficulties of mismatch, evidence can still be assessed on its own terms (as well as its relevance), there is validity in respect of addressing the intended design. For a point estimate of an effect this can be presented as a risk of bias i.e. getting an estimate distorted in a particular direction, but the influence of funding etc compromising evidence is also a concern. The broader vernacular understanding of bias attends to the provenance of evidence as well, as documented procedural and technical qualities. Thus public officials will take an interest in the oversight and management of interests in evidence, as well as the process which brought it to their attention. Where limited and informal evidence is available this is acute, but in other cases they can rely on good practice.

Researchers are well aware that most interventions are difficult to evaluate objectively, both in terms of designing suitable evaluation approaches and putting them into practice. Randomisation is employed because incentives to allocate innovations are always strongly in favour of those with most potential benefit, despite the uncertainty (which equipoise overcomes ethical objections). Making the case for the need to rigorously evaluate remains challenging, especially when one hurdle for getting to this stage is advocacy, for the scientific potential and concomitant funding. Blinding of investigators to the allocations is also essential because other biases reliably creep in, and clinical trials have a separate data monitoring committee, entirely independent of the research team. Even so, methodology is not discussed enough.

Triangulation

Evaluating social complexity is largely at the heart of challenges for public officials to make decisions fairly, and typically a decision is needed because evidence is difficult to reconcile or the available options, and public views about them, are not readily elicited. The notion that disparate sources should be synthesised to come to a coherent view on the most meritorious course of action is appealing but simplistic. There may well be different sources which triangulate to offer a complete warrant for a particular choice, subject some points being more robust than others, indicating a need for piloting of interim evaluation of the implementation, but it is also the case that evidence is contradictory.

Triangulation of evidence to review what is consistent with an assessment of the most coherent decision means choosing what to set aside because it is inconsistent. This speaks to the challenge of impartiality in reviewing evidence, as opposed to ‘cherry-picking’ what fits, and it also means being clear about what is included for consideration as relevant to the decision. While these choices are normally made in the standard, systematic review of evidence, the focus there tends to on robustness, with key word searches used to identify what is relevant to be screened for inclusion, assuming all such material is indexed suitably. Having assembled material, however, officials are likely to be practised in synthesis, to build a narrative, including making some allowance for implicit value judgements in measurement choices etc.

Provenance

Assessing the merits of evidence fairly and bringing that together ought to be feasible given technical capabilities and access to information. But limits to capability are actually less of a problem in many cases because of such poor access to material sufficient to assess the quality of evidence. Although scientific work, the kind of material considered both robust and disinterested, is by convention published to develop the scientific record, and for peer scrutiny, it is not available to others. This means that many other aspects of the publication, such as status of journals, authors and funding sources, are used in proxy of assurance. This is even before observing that necessary materials, from data and computer code to measurement protocols and other instrumental standards are often held privately.

Limits to relying on provenance, in the reputation of parties to the commissioning, production and publication of evidence, are that such markers are not necessary to problem solving. So the effects of relying on them are to intensify any social selection patterns in the development of scientific practices, and also to reinforce them further. However, the alternative, to access individual studies from authors and then to try to assess the quality directly, relies on extensive technical capability, and sufficient time to do it. And so there will be less risk involved in standard practices, especially as the public officials are never likely to have the personal capacity to assess information relevant to decisions (more than a couple of times).

Impartiality

Trends proceed in fields about public behaviour, from ‘unconscious bias’ to ‘diversity of thought’, so that a shift to ‘evidence informed decision making’ supplanted the founding concern for ‘impartiality’ (for now). Objectivity was originally conceived as an expectation to set aside political and social biases and assess on merit, for award of public contracts and making appointments. Indeed this need for independence from political pressures and prioritisation of technical soundness is one of the justifications for establishing public bodies independent for government in the first place. However political ideologies can seek to influence indirectly, and research institutes have been founded with such an ideological basis to their reports.

There are several components, some of which public officials will have strong experience in, from setting questions to synthesising summaries of evidence. Others like reviewing, assessing and even accessing evidence are considerably more difficult, particularly to serve the purpose of making decisions rather than scientific analysis. But all of these have their established practices and thus favoured forms, which can introduce bias into decisions, without that being apparent. The latest idea to disrupt lazy assumptions about what good decisions look like is diversity of thought, which means something about professional habits and epistemic abstractions, not maverick, dilettante or heterodox attitudes, testing consensus.

Good Intentions

That all public officials should be role models, embodying the values of society, is of course aspirational, and in the objective use of the best evidence, perhaps unrealistic. Scientists also struggle to set aside personal concerns in doing their work, and it is the broader scientific enterprise which follows the Mertonian principles, perhaps with some false steps and corrections along the way. But challenges to using evidence coming down to deciding to look for it, looking for it, evaluating its quality, and synthesising it in respect of the decision at hand. And of course that all depends on what is known and what is needed – societal mores and context change, and so policy will do too.