Lakhasly

5. Linking the Two Maps: Viewing the Other Concepts Through the Lens of Accountability
   Not only does this more structured understanding of accountability help us to avoid disorder in that
   context, it also allows us understand more specifically what might be entailed in each of the other concepts
   outlined above. Thus, for example, if a company uses a system which causes harm, this may well result in
   some form of bureaucratic or professional accountability within the organization. Those responsible for
   designing or deploying the system will need to make readily available (push) information about what the
   system was supposed to do; as far as possible what has happened; who was involved in any relevant
   processes or choices and who has been affected by it. This information must be interpretable or intelligible
   to those further up the organization. In other words those lower down the chain (the “who?”) must provide
   (push) sufficient transparency and interpretability/intelligibility for those higher up the chain (the “to
   whom?”) to be able to establish the answers to “what” has happened and what should be done about
   it. They should also be able to supply some kind of account or explanation of the “what” (explainability).
   Those higher up the chain must also be able to obtain (pull) any further information they find to be
   necessary (reviewability) in order to assess the functioning and compliance of the system and to enable
   satisfactory oversight of it (the standard of accountability in this context).
   If such harm has occurred there may also be a desire for some form of punishment or compensation (the
   why) which would suggest legal liability of some kind (the how). Those responsible (the who) may well
   be those with a causal input into the chain of events leading up to the harm, who would be accountable to
   e7-16 Rebecca Williams et al.
   https://doi.org/10.1017/dap.2021.37 Published online by Cambridge University Press
   Why?
   How?
   (Mechanisms
   and attribute of
   remediability)
   Who? (Responsibility
   attribute)
   To whom?
   (Responsiveness
   attribute) For what By what measure?
   Resulting in what
   practices?*
   Incentives, monetary
   compensation,
   punishment
   Law Using
   sanctions,
   obligations,
   liability, and
   other
   remedies
   Those causally or
   otherwise legally
   responsible for harm
   Those with
   standing to
   bring the claim
   or to the public
   in general
   (criminal law)
   Breach of
   specific
   legal
   obligations
   Legal standards
   applicable from the
   various forms of law;
   civil, public, criminal,
   etc.
   Governance and
   Regulatory
   compliance, due
   diligence, insurance,
   etc.
   Structured, internal
   incentive scheme;
   accountability
   compatible with
   efficient
   decentralization and
   an optimal balance
   between technical
   expertise and overall
   governance
   Bureaucratic** Those further down the
   bureaucratic chain
   Those at the top
   of the
   bureaucratic
   chain
   Not always
   clearly
   defined
   Not always clearly
   defined
   Rules/standards,
   reporting, auditing,
   incentives,
   governance
   Professional** Technical experts Lay manager, but
   deference to
   expertise
   Technical
   decisions
   Best scientific evidence Focus on justification
   according to
   evidence, trust in
   expert judgment and
   deference to it
   Need to achieve change
   in legal regime,
   change in funding or
   other larger, structural
   societal change
   Political** Elected representatives Constituents Existing
   promises,
   policy
   priorities,
   competent
   governance,
   societal
   benefit, etc.
   Public opinion Concern with majority
   public opinion,
   responsiveness to
   constituents,
   oversight by
   legislative
   committees, freedom
   of information and
   openness,
   importance of media
   (Continued)
   Data & Policy e7-17
   https://doi.org/10.1017/dap.2021.37 Published online by Cambridge University Press
   (cf. Diakopoulos, p.

Accountability
compatible with
Economic efficiency,
consumer decisionmaking, wealth
generation
Markets Sellers/producers Consumers Quality of
products
Public opinion and
consumer choice
Governance and rules
focused on delivery
of efficiency,
publication of data to
consumers (see
Diakopoulos, p. 31)
Generation of trust and
understanding on the
part of members of the
public interacting
with the system
Information
transparency
(including
system
transparency)
System operators Public
stakeholders
Relevant
information
to enable
trust and
autonomy
Public opinion Information giving
*Practices listed as: (a) Defining governance to comply in a responsible manner with internal and external criteria; (b) Ensuring the implementation of appropriate actions to actualise such governance; (c) Explaining and
justifying those actions, namely, demonstrating regulatory compliance, and (d) Remedying any failure to act properly (p. 2).
**The details in the adjacent cells are inferred from and build on the scheme devised by Romzek and Dubnick, p. 37.
e7-18 Rebecca Williams et al.
https://doi.org/10.1017/dap.2021.37 Published online by Cambridge University Press
the victims of such harm or to the state (to whom) for failure to comply with the standards of tort law or
criminal law respectively. In such instances the court will need transparency of information, which must
be intelligible to or interpretable by those involved in the legal system (including by a lay jury in the
context of criminal law), those defending the system will need to be able to explain its operation, while
claimants, prosecutors and the court in general will need to be able to ascertain the information necessary
to allow them to assess the functioning and compliance of the system with any applicable rules. In this
context the “push” and “pull” aspects of these concepts will be governed by specific legal duties of
disclosure as well as burdens of proof which rest to varying degrees on prosecutors and claimants.
Indeed in some instances, such as the GDPR for example, the form of accountability will affect not only
the quality of the information available when required (reviewability, transparency and interpretability)
but also enhance the onus on the system operator to provide (push) that information (explainability, or
indeed explanation). There is thus a direct and symbiotic relationship between these other concepts and
accountability, in that their content and requirements are directly informed by what is necessary for the
particular form of accountability at issue, but it is also that accountability which provides the vehicle
which renders them enforceable and practically useful. Accountability is thus the key lens through which
the concepts as a whole should be viewed.
6. Challenges to Accountability
This does not mean, of course, that accountability will always be straightforward. Nissenbaum, for
example, lists four main barriers to accountability in “a computerized society” as being the problem of
many hands; a complacent tendency to accept software flaws as inevitable; a tendency to use “the
computer” as a scapegoat and the tendency of software producers to deny accountability while leaving it
to their software licensees who are least well placed to be accountable (Nissenbaum, 1996). Diakopoulos
adds that problems also arise from lack of enforcement of accountability mechanisms that might be in
place and from a tendency to game and manipulate any standards used, to which we might add the
difficulty of specifying sufficiently precisely the level of compliance necessary in any given case.
Diakopoulos also notes the lack of accountability that can arise from trade secrets (Diakopoulos, 2015)
(a clash also noted by Bennett, 2013), the use of legacy code which cannot easily be reconstituted, or by
the pure complexity of the scheme used. Elish suggests that in turn the resulting gaps in accountability
tend to be filled by a “moral crumple zone” in which “the human in a highly complex and automated
system may become simply a component… that bears the brunt of the moral and legal responsibilities
when the overall system malfunctions” (Elish, 2019) But even here it is evident that a better understanding
of the concepts defined above can help. The problem of many hands, for example, or those of
scapegoating and buck-passing are essentially problems of attribution of causal liability which can be
addressed at least to some extent by greater transparency and explanation. A tendency to accept software
flaws as inevitable suggests a failure in accountability that could be addressed directly, while trade secrets
and scheme complexity go directly to requirements of transparency and interpretability respectively. And
Elish’s moral crumple zone arises precisely because of an inaccurate placement of liability resulting from
the failure of those earlier concepts; a situation that a more accurate form of accountability, supported by
the other integral concepts, could prevent. If the correct form of accountability is thus identified, and the
related concepts of transparency, interpretability, and accountability are deployed as necessary to support
it they can as a whole provide a structure which renders those deploying a system “answerable” to those
affected by it in a manner which provides concrete remedies and incentives.
It is clear, however, from the context-specific nature of accountability that achieving it will require the
input of the relevant discipline for each context. But it is also clear that when it is accountability for an
autonomous system that is at stake this process must be fully interdisciplinary, involving both the relevant
discipline and technologists or computer scientists on both substantive and procedural fronts. This is
imperative both because the accountability discipline (law, politics, etc.) must fully understand the
relevant technology in order to provide an optimal form of accountability and because, conversely,
technology can in fact underpin and help to realize that accountability (Naja et al., 2021).
Data & Policy e7-19
https://doi.org/10.1017/dap.2021.37 Published online by Cambridge University Press
7. Conclusion
In conclusion, as intelligent systems are deployed in an ever wider variety of contexts, those responsible
for their governance have responded by developing a series of overlapping abstract concepts which aspire
to regulate its operation. However, if those concepts are to do the work expected of them in regulating and
governing such systems, they must become specific and enforceable. We have in this article taken two
steps toward achieving that aim. First, we have identified more precisely what each concept requires and
in particular we have examined how the concepts fit together in terms of chronology and the extent to
which they require the provision (push) or active seeking (pull) of information. And second, we have
argued that the key concept in rendering them enforceable is that of accountability. There is a variety of
taxonomies of accountability in the literature. However, at the core of each account appears to be a sense
of “answerability”; a need to explain or to give an account. It is this ability to call an entity to account
which provides the impetus for and ability to enforce each of the other concepts. Conversely, if we divide
accountability more specifically, as suggested above, into questions of who is accountable, to whom, for
what, by what measure and why, this in turn will inform what precisely is required in terms of
transparency, interpretability, explainability, or reviewability in a particular instance. This understanding
will then enable us to develop the supporting sociotechnical infrastructure needed to realize these more
concrete concepts and enable them to fulfill their intended roles.
Funding Statement. This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) under
research grants EP/R033846/1, EP/R033501/1 and EP/R03379X/1. The funder had no role in study design, data collection and
analysis, decision to publish, or preparation of the manuscript.
Competing Interests. The authors declare no competing interests exist.
Author Contributions. The authors have all contributed to the work in accordance with the Publishing Ethics Guidelines.
Data Availability Statement. This study does not contain empirical data. The other resources on which the article draws are listed
in the references section.
References
AI Now Report (2018) New York University.
AI Regulation COM(2021) 206 Final 2021/0106 (COD) European Commission, Brussels 21.4.2021, Recital [38] and
Article 17(1)(m).
Ananny M and Crawford K (2018) Seeing without knowing: Limitations of the transparency ideal and its application to
algorithmic accountability. New Media & Society 20(3), 973–989.
Aristotle (n.d.) Nichomachean Ethics, Book III 1109b30-1111b5.
Auel K (2007) Democratic accountability and national parliaments: Redefining the impact of parliamentary scrutiny in EU affairs.
European Law Journal 13, 487.
Behn R (2001) Rethinking Democratic Accountability. Washington, DC: Brookings Institution Press.
Bennett C (2013) Accountability for privacy in cloud computing: Is this a new problem? In Pre-Proceedings of TAFC, p. 3, 125.
Bhatt U, Xiang A, Sharma S, Weller A, Taly A, Jia Y, Ghosh J, Puri R, Moura JMF and Eckersley P (2020) Explainable
Machine Learning in Deployment. arXic:1909.06342v3 [cs.LG}, 22 May.
Binns R (2018) Algorithmic accountability and public reason. Philosophy and Technology 31, 543–546.
Birkinshaw P (2005) Government and Information: The Law Relating to Access, Disclosure and Their Regulation, 3rd Edn.
Haywards Heath: Tottel.
Birkinshaw P (2010) Freedom of Information, The Law, the Practice and the Ideal, 4th Edn. Cambridge: Cambridge University
Press.
Blumenstock J (2020) Machine learning can help get COVID-19 aid to those who need it most. Nature World View. Available at
https://www.nature.com/articles/d41586-020-01393-7; https://cGovid.joinzoe.com/ (accessed 5 January 2022).
Bovens M (2007) Analysing and assessing accountability: A conceptual framework. European Law Journal 13, 447–449.
Bovens M (2010) Two concepts of accountability: Accountability as a virtue and as a mechanism. West European Politics 33,
946–947.
Bovens M, Goodin R and Schillemans T (2014) Oxford Handbook of Public Accountability. Oxford: Oxford University Press.
Bryson J and Winfield A (2017) Standardizing ethical design for artificial intelligence and autonomous systems. Computer 50,
116–119.
e7-20 Rebecca Williams et al.
https://doi.org/10.1017/dap.2021.37 Published online by Cambridge University Press
Cattedu D, Felici M, Hogben G, Holcroft A, Kosta E, Llenes R, Millard C, Niezen M and Nu D (2013) Towards a model of
accountability for cloud computing services. In Pre-Proceedings of International Workshop on Trustworthiness, Accountability
and Forensics in the Cloud (TAFC). Malaga: Trust Management, University of Malaga.
Coeckelbergh M (2012) Moral responsibility, technology, and experiences of the tragic: From Kierkegaard to offshore engineering.
Science and Engineering Ethics 18, 35.
Coeckelbergh M and Wackers G (2007) Imagination, distributed responsibility and vulnerability: The case of Snorre, A. Science
and Engineering Ethics 13(2), 235.
Craig P (2012) EU Administrative Law, 2nd Edn. Oxford: Oxford University Press, p. 356.
Curt