Human experimentation is essential to medical innovation. It is through human experimentation that we get the best understanding of the risks (and benefits) of a potential therapy. But, importantly, we also only conduct human experimentation when we are unsure of the potential hazards and/or are uncertain of their likelihood. Strictly speaking, a risk is a measure of the probability of some hazard occurring multiplied by its magnitude – that is, a measure of how bad the hazard would be if it did occur. Sometimes we are ignorant of potential hazards – we simply do not know what effects something may have and therefore cannot determine magnitude – and other times we know what may occur but are uncertain as to its likelihood and therefore cannot determine probability. And so, when we conduct human medical experiments, we are doing so in order to better understand what hazards are possible and/or how likely the hazards are.

But because human medical experimentation is carried out under these conditions of ignorance and uncertainty, we insist on strict ethical controls. We are likely all familiar with some of these ethical controls: all research subjects must be informed of the potential risks (as well as the lack of complete knowledge) and voluntarily agree to be exposed to them; research can only be carried out when there is good reason to believe the therapy could in fact provide a meaningful benefit. There are others as well, which we need not detail now. The central point is simply that we require strict ethical scrutiny of human medical experiments precisely because they involve exposure to unknown hazards with unknown probabilities.

But isn’t the same true of many technological innovations? Recall the Control Dilemma from a previous chapter: it shows us that for many technological products and projects, we find ourselves in the same position as medical researchers: we are simply unaware of some of the potential effects and we are uncertain of the likelihood of other effects that we are aware of. Moreover, just like medical therapies, many technologies can have a deep and lasting impact on individual lives and on the trajectory of society. Just think which has had a larger social impact: the invention of vaccines or the invention of the internet? Whatever the answer to that question, it is not an obvious one, and that alone should show the affinity between medical therapies and technological innovations. And if there is this sort of meaningful parallel, and we hold human medical research to high ethical standards, ought we not similarly hold the introduction of technology to high ethical standards?

1. Experimental Technology

At the time of its launch, the iPhone had two key properties that are shared with many other types of technology:

1. Significant Impact Potential: It had the potential to cause significant social changes if it took off; and
2. Uncertain and Unknown Impact: Many of the potential impacts could be speculated about but not actually predicted or quantified in any meaningful way while some of the potential impacts were complete unknown at the time

These two properties make the launch of the iPhone rather different from some other technological developments. For example, the construction of a new bridge could be seen as a technological development, but the potential impacts are usually more easily identified and quantified (although this can vary). Since we have a lot of experience building bridges and long running familiarity with the scientific principles that underly their design and construction, there is a lot less uncertainty involved. But, of course, if the bridge uses new techniques or new materials, or a bridge is used for a new purpose, then even a bridge can start to have properties similar to the iPhone. Nonetheless, what we can say here is that some technologies fall into a special class that merit greater ethical scrutiny:

Experimental technologies, more than others, are subject to the Control Dilemma: they are those technologies where we are substantially ignorant and uncertain of the risks and therefore are not in a position to control them early on. But it is also worth noting that whether a given technological product or project is “experimental” is really a matter of degree: even a new bridge may be experimental, at least to some degree.

So, many technological products and projects are (to some degree) human experiments. When a human experiment involves medical therapies, we insist on strict ethical controls. But these experimental technologies can have at least as great, if not a greater, effect on individuals and society than the medical therapies. Thus, if we think it is right to have strict ethical controls for medical experimentation, we should also have strict ethical controls for engineering and technology experimentation.

But what does that look like?

2. The Principles of Engineering Ethics

A good starting point for developing an ethical framework for thinking about experimental technologies is to consider the ethical framework most commonly used to think about human medical experiments, a framework known as Ethical Principlism. In broad outline, ethical principlism is an ethical framework centered around multiple equally important, but sometimes conflicting, ethical principles. In the context of medicine, there are three major principles:

These three principles are intentionally general and somewhat vague. The basic idea is that they represent widely shared ethical commitments and provide the basis for establishing more precise and practical guidelines for medical researchers.

By and large, we can apply these same three principles to thinking about experimental technology. But, of course, out context is a bit different (even if similar) and so we will want to tweak things a bit. Additionally, for the sake of simplicity, we can rename the principles. In so doing, we end up with the following three Principles of Engineering Ethics:

We rename the Principle of Beneficence to the Principle of Welfare to make clearer that its focus is on promoting and protecting welfare, thereby underwriting the most fundamental principle of engineering ethics. Additionally, we spell out what it means to respect “human autonomy and agency” and, finally, rename the Principle of Justice to the Principle of Fairness to avoid confusion with thinking about (for instance) criminal justice or the law more generally.

3. Applying the Principles

We now have three principles for scrutinizing the design and implementation of technology into society. But just as the principles of biomedical ethics are general and require specification for context, so too do the principles of engineering ethics. So, the question arises, how do we work with these principles?

Technological design and implementation already involves working with a variety of necessary but sometimes conflicting design constraints. For instance, buildings must be able to resist wind forces up a certain speed and bridges must be able to hold at least a certain weight level. But these considerations often butt up against other constraints regarding, for instance, total weight of the bridge or building. The project must be kept under a certain total weight but also must display a certain level of strength. Both of these are constraints on the design, and must be adhered to, but also must be balanced against one another – you cannot build the bridge maximally strong, for it will be too heavy, and you cannot build it maximally light, for it will be too weak. Similar considerations apply when we aren’t talking about safety: an interactive device must balance simplicity with power, other products may need to balance longevity with accessibility.

And so, we can think of the ethical principles as Ethical Design Constraints that function in a similar manner. Each principle must be adhered to for a project to be (ethically) successful, but the requirements of the principles will sometimes come into conflict. Indeed, one basic conflict applies to both engineering and medicine and is illustrated with the iPhone example: we could, hypothetically, make significantly greater medical progress if we did not inform people they were research subjects and did not worry about whether the experiment would be harmful to them. In this way, if we ignore the principle of autonomy and half of the principle of welfare we would be better able to achieve what the first half of the welfare principle requires. But such use of people for the benefit of others should strike us all as morally suspect.

In functioning as ethical design constraints, the principles provide broad guidelines that must always be accounted for in design and implementation. So, then, how do we account for them? To do that, it can help to recognize that the principles really engage us in two ways: they both tell us what to pay attention to as well as what to do with the information we are paying attention to.

3.1. Thinking about Welfare

The Welfare Principle, as its name suggests, directs us to pay attention to the ways in which welfare may be positively and negatively affected. In this way, it functions quite similarly to Canon 1 of the NSPE Code of Ethics (and the first principle of many other engineering codes of ethics). But, importantly, it also directs us to pay attention to two distinct ways that a technology may affect welfare: it may enhance it by either providing a new benefit or by reducing or eliminating existing harms or risks. So, for instance, the introduction of FaceTime provided a new benefit: the ability to visually converse via phone. On the other hand, new water treatment technology can be understood as reducing or eliminating the existing risk of water-borne illness. These are two ways of “doing good”. But the Welfare Principle also directs us to “do no harm”, which means not introducing new harms or risks through the technology. Thus, this second part of the principle directs us to (for instance) not introduce a technology that will pollute the water and thereby create (or increase) the risk of water-borne illness.

This point about introducing new risks and reducing existing risks is important because they influence how we think about what to do once we have identified the potential effects on welfare. There is a millennia old debate over whether it is morally worse to cause harm than to fail to benefit. On one view, so long as the consequence is identical, then the moral evaluation is identical. Thus, were you to fail to save a drowning child from a pond when you were able to, that would be just as bad as if you had actually drowned the child in the pond. According to this symmetry thesis, harms caused are symmetrical with benefits denied – so long as their likelihood and effect are identical, they are morally identical. In contrast, the asymmetry thesis holds that causing harm is morally worse than failing to benefit. While it may still be morally bad to fail to save the drowning child from the pond, it is not as morally bad as if you were to have drowned the child.

We will not resolve this debate here (there is a reason it has been raging for thousands of years). Instead, we point it out so that we all may understand the competing perspectives. But what position one takes on this “doing/allowing” debate does often influence how one will apply the welfare principle. If you believe that it is morally worse to cause harm, then you will “err on the side of caution” and be more willing to (for instance) stop a project or reduce the power of a technology so as to limit the introduction of new risks. On the other hand, if you accept the symmetry thesis then you are much more willing to say “the benefits outweigh the risks” and therefore be willing to justify almost any risks so long as the benefits are greater.

Our stance on the doing/allowing debate will influence how we apply the welfare principle to some degree, but there are nonetheless some general guidelines that the principle offers us. These include:

• Only pursue a product or project if it is reasonable to expect social benefits
• Aim to reduce the potential for harm to the public as far as reasonably possible
• Conduct controlled “experiments” of the technology whenever possible
• Provide safeguards that allow a project to be halted or a product to be recalled if excessive harm is discovered
• Avoid products and projects that may produce irreversible harm
• Pursue products and projects that have the potential to improve social or environmental resiliency (the ability to ‘bounce back’ from bad situations)

Like the principle itself, these guidelines are still a bit general and vague: it leaves open, for instance, what counts as “reasonable possibility”. But they are more precise than the principle itself and help us understand, in more detail, how to reason with it.

3.2. Thinking about Autonomy

Autonomy is derived from Ancient Greek and literally means “self-legislate” (auto-nomos). Although it hasn’t always been the case, in contemporary society we value individual autonomy, even at the expense of individual and social welfare. For instance, we leave people quite free to make their own decisions even knowing that many of them will make decisions that reduce their welfare. This is because we value people being the authors of their own lives alongside valuing their welfare. Put another way, we think it important not just that people live good lives but that they control their own lives. Obviously, this control is not absolute, but it matters, nonetheless.

In the medical context, the autonomy principle grounds the practice of informed consent: we only allow researchers to research on human subjects who have been properly informed of the risks involved in the experiment and have voluntarily agreed to be exposed to those risks. It is not the role of the researcher to decide whether someone else is exposed to risks.

The principle functions similarly in the technology case. First, it directs us to pay attention to what information people have about the technology and whether they are in control of their exposure to it or not. Relatedly, and as we will explore in more detail in a later chapter, it encourages us to think about their privacy, for privacy is a means by which we protect our autonomy. My ability to control what others know about me, for instance, helps me limit the ways in which others may try to interfere with my informed choices and the development of my life plans. Thus, whereas the principle of welfare directs us to focus on all those values associated with welfare, the principle of autonomy directs us to focus on those values associated with autonomy: honesty, adequate information, good reasoning and decision-making, control over our bodies and personal information, and individual freedom more generally.

An important note must be made about respecting autonomy. The principle directs us to respect the informed choices of autonomous people. But not everyone is autonomous all the time. If I have been systematically deceived and am therefore making choices based on false information, then I am not making autonomous choices. Relatedly, even if I have accurate information, if I am (for instance) drunk then I may not be reasoning autonomously. Thus, autonomy can be imperiled both by false information and by various barriers to good reasoning. Respecting autonomy is not about just letting people do whatever they want. However, we must also be careful not to judge someone as non-autonomous just because they are making choices we don’t agree with. Autonomous choice does not require making the best or right choice either.

Like the principle of welfare, the principle of autonomy also provides a few guidelines for thinking through technology’s impact on autonomy:

• Ensure all those exposed to the risks of a technological product or project are informed
• Ensure all those exposed to the risks of a technological product or project have voluntarily agreed to be exposed
• Introduction of experimental technologies should be approved by democratically legitimized bodies
• Those involved in the “experiment” should be able to withdraw from the experiment
• Reduce, as far as possible, any exposure to risk for people who have not been informed and/or have not approved the product or project

Once again, these guidelines are intentionally general so as to be widely applicable. It should also be noted that all these guidelines (including those for the other principles) are often statements of what is ideal. As a matter of fact, it may be impossible to “ensure all those exposed to the risks of a technological product or project are informed” since, unlike medical experiments, technologies are often simply “tested on society” via their introduction as a consumer product or their implementation as a public works project. We simply do not know who all will be affected and therefore cannot inform them in advance. That does not mean the guideline is irrelevant, though. By setting the ideal that all will be informed, we know what we should aim for, even if we cannot ever quite reach it. This is certainly better than not even attempting to meet the guideline at all.

3.3. Thinking about Fairness

The Principle of Autonomy is fundamentally about protecting individuals, for it is only individuals that have autonomy. The Principle of Welfare is about both individuals and groups, since we can meaningfully speak about both individual and social welfare. The Principle of Fairness, on the other hand, is fundamentally about groups and not individuals. Whereas the Principle of Autonomy augments our pursuit of social welfare by basically telling us not to sacrifice the individual, the Principle of Fairness augments our pursuit of social welfare by telling us to focus not just on the social benefits but also on how they are distributed across society. In this way, both autonomy and fairness help us avoid the sort of “tyranny of the majority” potential that characterized medical research before the introduction of the ethical principles.

The Principle of Fairness, then, directs us to focus on values like fairness, equality, democracy, and the protection of the vulnerable. It is unfair for only a select few people to gain all the benefits of a new technology whilst everyone else pays all the costs. More positively, technology has the potential to, and has in fact, promoted greater equality in society and “democratized” society. Various social goods that were once only available to nobility are now available to everyone while the creation of new means of communication have enhanced peoples’ abilities to have a say in how their society progresses.

Once we have identified the potential for unfair distribution or potentially affected vulnerable populations, the Principle of Fairness offers us the following sorts of guidelines for moving forward:

• Avoid imposing risks on vulnerable populations or, at the least, offer additional protections and/or compensation if they are exposed
• Avoid imposing risks on specific populations without any corresponding benefit to that same population
• Distribute the benefits and risks of a technology fairly across all those affected by the technology
• Avoid imposing irreversible risks on society

4. The Principles & Ethical Design

As should now be clear, these ethical principles are applicable to our thinking about technology at multiple levels. As policymakers or members of society, we can use the principles to think about existing or emerging technologies and how they are implemented into society. As designers of technology we can also use the principles to help guide the design and implementation of technology in society.

When using the principles to think about design, we should think broadly. That means considering the immediate risks the technological artifact may pose – for instance, the potential health effects of some new building material. But it also means considering the wider social effects of the use of technology in society – for instance, the potential health effects of the spread of misinformation due to the widespread use of social media as a result of the easy accessibility of smartphone devices. This is why keeping in mind the social context of technology and technological mediation is important – they both help expand our thinking beyond the technological artifact to the techno-social system as a whole.

We should thus endeavor to speculate, within reason, about the potential long-term social effects of a technology on existing social institutions and systems. For instance, the principle of welfare does not only direct us to think about the welfare effects of the technology but also how the technology may affect existing social institutions and practices that protect or promote welfare. The spread of misinformation via social media both directly affects health, a welfare value, and indirectly affects existing health-oriented institutions by, for instance, leading to an increased burden on health resources, healthcare workers, and the health system as a whole. Similarly, the rise of consumer surveillance – Ring doorbells, etc. – raises both immediate privacy concerns with the device itself but also has a long-term effect on how we think about privacy in society. And, finally, existing institutions that promote fairness such as democratic elections can be imperiled both directly and indirectly by technology.

In short, then, a major value of these principles is their wide applicability. They are simply statements of nearly universal ethical principles, with application to thinking about technology. As such, in applying them, we should keep in mind both the narrow applications and the wider applications.