Reliability as Ethical Commitment
Trustworthiness is defined as a duty to be honest, competent and reliable when applied to the corporate values of an institution. Honesty and competence are attributes that are straightforward especially as institutional duties envisioned by Onora O'Neill, even as they can be hard to pin down. But the "ethical commitment" to trustworthiness is really where reliability insists integrity dominates against other pressures. When incentives are to cut corners, oversimplify or skip assurance to expedite delivery of save resources, the commitment to reliability insists on professional principles. Similarly, when revelations are socially or politically awkward to admit, various kinds of lies and dissembling undermine trustworthiness.
Public bodies are often accused of deceit and incompetence, suspicion born of too little information being available, and so statisticians promote the idea of intelligent transparency. This earnest attention to transparency is predicated on a belief that practices are trustworthy, and assumes public mistrust is due to misconceptions. But that means a corporate commitment to transparency has to be honoured reliably, i.e. exceptions are unrelated to factors which undermine assertions of trustworthiness. Hence security and economic constraints restrict some publications, and release of commercially and politically sensitive information is carefully scheduled, but these can also be foreseen, and mitigated in some ways.
The concept of reliability is weakly developed, and specifically how it is achieved and sustained. Habits might have set this problem aside, and customary practices are likely to do this even as professionals sometimes adopt processes that have a ritual character. A much bigger challenge is sustaining good practice in the face of new technologies, techniques and circumstances, exactly the challenge of innovation. Regulation of new work broadly labelled as data science is already strained by grand claims which are not really honest, and skills shortages which put a premium on competence. What to do to remediate these challenges is a matter of discussion, but will benefit from a consideration of principles.
Regulation of AI
The problem of reliability is clearer in the concern for the regulation of artificial intelligence (AI) and the role of public bodies in developing and deploying such technologies and systems. Hype accompanies any innovation but any product must deliver on its promises, and will fail in the market if it causes damage in the context it is deployed unless that can be managed and mitigated. Regulation of AI has been stymied by difficulties not only understanding impact in practical applications, but also characterising the behaviour of systems as they react with a dynamic environment. Public bodies face further challenges if procuring these systems without the evidence to judge how robust and interoperable they may be.
Political and economic incentives support success wherever it may be seen, and obscuring other aspects of performance, variously described as glitches, bugs, calibration, updates etc. In the open market these might be exposed in competition or remedied as innovation, but a public task is typically not competitive in that sense and contracts are often more like partnerships. Although there can be well-designed technical systems, and well-evidenced claims about their utility, cutting corners to offer a system before it is ready and more widely than it has been tested is a natural temptation. AI proponents will struggle considerably to assure yielding to neither pressure without further commitments than advertising or framing releases as a 'public beta', openly exposing unfinished products to testing in the real world.
Although technical and evidence standards may exist, reliably honouring them in practice depends on the integrity of processes for assurance, and the behaviour of individuals. In the UK, the aim is that regulators will acquire the requisite technical skills by drawing on a pooled central capacity, but more political engagement will also be needed to keep people honest. Here the parallel with statistics is clearer, that there may be expertise, and there may be scrutiny, but if these are not combined effectively something can slip through. Although media and other civil society may attend to matters of interest, they may not be well informed about technical functions, and regulators lack a mandate to criticise political choices.
Enforcement
Regulation of AI is incoherent at present, and steps towards principles are often criticised for their broad, voluntary nature, even as corporate statements seem sincere. But these probably miss the point, that it remains unclear what is expected, and how to evaluate and assure trustworthy implementations of AI. Honesty and competence are vulnerable in this situation, to narrative after the event in the context of innovation that is predicted to transform many aspects of activity. Although there are notions of responsible innovation, these mainly focus on entirely novel activity, and particularly technologies with dual-use potential, i.e. obviously dangerous in uncontrolled applications. So although we might hope professionals such as engineers take care for their impact on society, they lack a guiding framework.
Enforcing prohibitions on something that is continually evolving is unrealistic without the support of professionals who are doing the work, and also brings protests about stifling innovation, so an alternative, dynamic but consensual approach is necessary. But the transformative potential of AI has an advantage in that there is political interest and from economic and social perspectives on benefits as well as risks. The many stakeholders and high level of public discussion make for a natural representative forum such as parliament to hold dialogic hearings about progress with regulation. The big challenge there is that even if a few parliamentarians do have relevant education and experience, the rate of technological development and the specificity of impact will require technical staff advice, at a level that may be difficult to secure on a project basis, so a specialised committee is needed.
Professionals bear responsibility for the work they do and assure for their contracts, whether these are to large corporate customers, public sector employers or individual consumers. That responsibility extends from the suitability of the work to the integrity of delivery: the licence of professionals to society is for a service of value which they can regulate themselves. This professional self-regulation is often derided when viewed in the context of powerful private sector organisations, but even in these capability is defined by what staff can offer, and so they have leverage over the activity. Engineers are more familiar with this challenge of standing up to their employer and explicitly acknowledging the interests of other stakeholders, perhaps because failures are more visible, dangerous and so obviously at odds with the societal contract, but technical staff in AI all ought to offer that assurance too.
Practising
Low stakes tasks, which can fail without consequence, and outputs that can be easily revised and updated, using familiar low cost tools which are well understood by users, are no trouble. Indeed these may be toy problems used in training, induction and testing of new ideas, but they are unlikely to offer a specialised or sophisticated product justifying professional input. Where technical quality justifies the professional assurance this also requires integrity, because that complexity sets the appreciation beyond lay assessment. And so the higher stakes applications require greater commitment to understanding the problem, mastering suitable tools to resolve it, and make a comprehensible account of its limitations.
Student exercises in support of professional training still require the production of a solution which is final and assessed. Entering practical employment they have to learn that this is not what is expected, just as research has stages even as external review may only come ahead of final publication. But academic competitors can have a range of reasons to criticise methods used, just as clients may prefer a certain presentation, emphasis or even secrecy. Negotiating solutions and resisting pressure, by explaining reasons for technical approaches and citing authoritative standards are an essential part of ethical analytical practices. But there may be contracting and other agreements that mean professionals have no apparent recourse, save to withdraw.
Protected disclosures are designed to allow UK employees to raise a concern, both internally and externally if that does not resolve the situation, with fear of consequences. This protection is narrow, so employment rights are protected in respect of the specific disclosure, but obviously workplaces are such that roles and activities are connected, and fairly arbitrary changes are often necessary. Professionals ought to have stronger protection as their registration requires them to have some regard to the integrity of their practice, for the reputation of the profession and impact on society. All of these ideas about whistleblowing are littered with examples of procedures not protecting anyone, especially the public, so analytical work whose details are inscrutable to outsiders if they do have access seems vulnerable.
Consequences
Reliance can be an implicit feature of trust, that behaviour sustains in adversity; contra cynicism that interests are favoured instead, and any failures are to be expected. Trust can be built by demonstrating resilience and adaptability, with evidence that systems serve general principles, but it remains a social characteristic rather than a contract. Expressions of cynicism can also be a social articulation rather than a trenchant view which can be addressed with communication, especially responsively. Where such exigencies are very rare, this communication including hypothetical situations may have more salience, but partial demonstrations or exercises and community forums substitute.
Priorities and interests bear on human activity, and choices are made in terms of distribution and strategy. But some priorities are considered fair, and some interests conflicting, the former justified by open consultation and transparent processes, the latter supported by declarations and mitigations. However there are always aspects of discretion over what concerns to take up, what issues to escalate, and whose reactions to manage more carefully. Expedience can also short cut points of process that may seem like they are bureaucratic irrelevancies, but actually not just support but embody the values of the people the system serves. Other mitigations may be available but dispensing with checks and balances is not justified without rehearsing the values that they support, as applied to the particular eventuality.
"It might seem ridiculous to have been bothered by what seem like small process questions in the face of such an enormity of decision making, but the ends do not justify the means. And the governance of our country does not normally allow for operating without the agreement of Cabinet Ministers - albeit through norms and conventions rather than law. When it comes to it, we have surprisingly thin threads that connect what No 10 wants to do to the democratic mandate. And the point to hold onto them is precisely when it is difficult. This felt like a minority view."
Helen MacNamara, witness statement to the UK pandemic public inquiry
When there are no easy choices it becomes considerably more important. There is an aphorism 'penny wise and pound foolish', inverting the concern for procedure against the impact of the decisions might be a neat analogy. Developing systems to apply proportionate concern and sustain the integrity of decisions is really the point of regulation, and reliability has that intention. In aspects of innovation, it is not clear that it is happening, and burdens can fall rather heavily on individuals as leaders or professionals. While escalating more serious issues is important, where the aim is to support the public interest, another dimension is indicated, starting with making things open. It would be easy enough to apply this to examples recently, in official statistics and other public data science projects, but that is not the point.