ETHICS, COMPUTING, AND AI | PERSPECTIVES FROM MIT
Two Commentaries | Susan S. Silbey, Chair of the MIT Faculty
Susan S. Silbey; photo by Jon Sachs, MIT Spectrum
"The College is envisioned to be the nexus connecting those who advance computer science, those who use computational tools in specific subject fields, and those who analyze and write about digital worlds."
— Susan S. Silbey, Goldberg Professor of Humanities, Professor of Sociology and Anthropology, and Professor of Behavioral and Policy Sciences
Series: ETHICS, COMPUTING, AND AI | PERSPECTIVES FROM MIT
Susan S. Silbey is Leon and Anne Goldberg Professor of Humanities, Professor of Sociology and Anthropology, and Professor of Behavioral and Policy Sciences. She serves as chair of the MIT faculty (2017-2019). Her research focuses on governance, regulatory and audit processes in complex organizations. Her current research focuses on the creation of management systems for containing risks, including ethical lapses, as well as environment, health and safety hazards. Her books include The Common Place of Law: Stories from Everyday Life (with Patricia Ewick) (1998), and Law and Science (2008).
Here follow the full texts of two commentaries by Susan Silbey, originally published in the MIT Faculty Newsletter. You can also read the original publications at these links: How Not to Teach Ethics (Sept/Oct 2018 edition); and Forming the MIT Stephen A. Schwarzman College of Computing (Nov/Dec 2018 edition).
• • •
How Not to Teach Ethics
Susan S. Silbey, MIT Faculty Chair
from the MIT Faculty Newsletter, September/October 2018 edition
"Rather than thinking about ethics as a series of anecdotal instances of problematic choice-making, we might think about ethics as participation in a moral culture, and then ask how that culture supports or challenges ethical behavior."
— Susan S. Silbey, Leon and Anne Goldberg Professor of Humanities, Professor of Sociology and Anthropology, Professor of Behavioral and Policy Sciences, and Chair of the MIT faculty (2017-2019)
I write this month about stories and ethics, more specifically about the stories we tell about ethics. There is increasing talk lately, coming from unexpected and unaligned voices, about the importance of stories for understanding what we do, and what we should do. For example, Gary Saul Morson and Morton O. Schapiro write in their book, Cents and Sensibility: What Economics Can Learn from the Humanities, that “to understand people one must tell stories about them” and, while we can learn much from economics, culture and ethics cannot be reduced to economic equations.
In the popular press, New York Times columnist David Brooks recently recalled philosopher Alasdair MacIntyre’s argument that you can’t know what is the right thing to do unless you know what story you are a part of, that the story we tell about what we do can be more powerful than the specific details of programs and policies. Yet, in the intensifying calls for the teaching of ethics as part of both undergraduate and professional education, here at MIT and across the nation, the story being told about ethics is disturbingly banal and wrong-headed.
We might begin by noting that crises of corporate and professional responsibility have been endemic to American society, at least since the last quarter of the nineteenth century. With each chapter of professional misconduct – from the robber barons and the Teapot Dome scandals, through the progressive era up through Watergate, Iran Contra, the financial crisis of 2008, and to the recent epidemic of research scandals in political science and psychology – the response has been the same: calls for education in ethical responsibilities, and specifically training in ethics as part of professional education.
For example, a 2017 article about bridge failures recommended that “engineering schools should do more to prepare students for the ethical challenges they’ll face as individual workers – and as an industry.” Another 2017 Atlantic article by Irina Raicu addressed the ethics of Silicon Valley directly, also recommending ethics education. “A growing chorus has argued that we need a code of ethics for technologists.
That’s a start, but we need more than that. If technology can mold us, and technologists are the ones who shape that technology, we should demand some level of ethics training for technologists. . . . Such training . . . would prepare them to make more thoughtful decisions when confronted, say, with ethical dilemmas that involve conflicts between competing goods. It would help them make choices that better reflect their own values (my emphasis).” Natasha Singer wrote in the New York Times in February 2018 about new efforts needed, and forthcoming, to incorporate ethics into computer science education. Most of these courses focus on getting students to reflect on their personal choices. Singer recounts how such courses are emerging at a moment when big tech companies have also been struggling to handle the side effects of Silicon Valley’s build-it-first mindset. The message is clear: universities should embrace, not shun, teaching about values in the classroom.
Models for Teaching
This cycle of scandal and responsive calls for better training has been so often repeated that one can be surprised only by the paucity of models for providing that education. The standard model – required in law and medical schools now leaking into engineering and computer science programs with minor variations – teaches ethics as problems in individual decision-making, personal values, and choices. Training focuses on formalized rules of professional conduct, punctuated by appeals for social responsibility. It has not proved to be a successful regimen, if the repeated cycles of corporate and professional misconduct are any gauge.
Such standard models fail because the diagnosis and cure share a basic misconception: that corporate and professional misconduct are problems caused by rotten apples; some few weak, uninformed, or misguided individuals making independently poor choices.
What is the source of this misconception? A great deal of education propagates this misunderstanding by focusing exclusively on two forms of causality familiar to engineers and scientists: physical forces or atomic structure and human will or intention. Although processes of aggregation and ecology for matter and mechanical systems are well understood, many seem unable to recognize patterns of aggregation when it comes to human action. Thus, when asked to interpret or explain social phenomena, including professional misconduct or inattention to competing interests, historical examples and possible precedents, the well-educated technologist as well as the popular pundit will more often than not offer accounts that rely on individual agency, choice, and personality. Unable to recognize or describe forms of social organization, many adopt a rationalist, often reductionist model of social action that in effect constitutes a powerful and unreflexive orthodoxy.
Consider an alternative account produced nearly 70 years ago when sociologist Edwin Sutherland published his now canonical work, White Collar Crime, in which he documented that American corporations constituted the most numerous population of criminal recidivists. This counter-intuitive observation flowed from Sutherland’s earlier work outlining a theory of criminal behavior as normal behavior in situations publicly defined as undesirable, illegal, or unethical. Sutherland described criminal behavior as normal learned behavior in situations and transactions where there is an excess of circulating definitions favorable to violation of norms or law over definitions unfavorable to the violation of law. He called this the principle of "differential association."
Although Sutherland’s work focused on criminal behavior, the insights merit our attention when considering what stories to tell about ethical and unethical professional behavior. Sutherland’s principal account describes all behavior, deviant as well as normative, as habits learned in interaction with others, most often within intimate personal settings and organized groups. That learning includes the motivations, drives, and rationalizations for the action as well as the techniques of committing the act, which can be complex, especially in white collar crime, financial, scientific, or computer-based fraud.
If we understand both ethical as well as criminal misconduct as consequences of normal learning, we might offer different kinds of ethics education, as well as different kinds of experiences, telling different stories than ones about individual, rational, and isolated decision-making. First, we would, of course, attend to the content of what is learned, which includes both motives and techniques. This would include in our local domain, scientific theories and engineering methods, but also various modes of collegiality, status hierarchies, gender performances, and appropriate degrees of ambition as well.
Second, we would focus on pedagogy and the process of learning. Although Sutherland, following George Herbert Mead, focused on interpersonal and symbolic interaction, I might put as much emphasis in twenty-first century learning on mediated communication as on intimate personal transactions. Engineers learn to become engineers not only by doing problem sets and working in laboratories, but by mimicking what they observe as conventional, accepted and rewarded demeanors, conversational practices, and career expectations, whether observed face to face or through public media.
The third, and most important lesson for ethics education is Sutherland’s emphasis on context and social organization as an antidote to an exclusive focus on individual choice-making activity. In other words, while we might want to acknowledge human agency and decision-making at the heart of ethical action, which cannot be avoided for sure, nonetheless, we blind ourselves to the structure of those choices – incentives, content, and pattern – if we focus too closely on the individual and ignore the larger pattern of opportunities and motives that channel the actions we call ethics or occupation we call career.
Ethical Lapses
Perhaps the simplest way to think about how attention to context and social organization might challenge the individualist story of ethics is to consider the popular American narrative of ethical lapses. For example, the stories of Enron, drug trials for Actonel, the Schon affair at Bell Labs, and the Cambridge Analytica debacle at Facebook are usually narrated as the story of a few rotten apples giving the barrel a bad name. In other words, such bad apple narratives tell us that we need not worry about increasing evidence of financial misconduct, student cheating, scientific fraud, or the digitized threats to liberal democracy, because the grand narrative of well-functioning institutions (the market, meritocratic higher education, peer review, or digital connectivity through anonymous participation) remains in place, unsullied by the random bad apple.
Each of these examples is reported and interpreted as an anecdote. As separate accounts, anecdotes claim particularity, not typicality, and as such, anecdotes obscure the links connecting one event to another. The social organization that arranges the individual cases into a structure of action we might call professional or market failure, or digital warfare or social disintegration is suppressed and thus overwhelmed by the exclusive focus on personal motive, action, and fault.
How is this relevant for teaching ethics? Rather than thinking about ethics as a series of anecdotal instances of problematic choice-making, we might think about ethics as participation in a moral culture, and then ask how that culture supports or challenges ethical behavior. Or, in Sutherland’s terms, what are the transactions among the cultural members, what are the communicated messages, how often and for how long, and thus how is that culture learned? How is the system of incentives and rewards organized, what is the structure of resources and rewards? More particularly, what do we describe as a good life?
"To the degree that this neglect of institutional analyses is a result of the way degree requirements are specified, as well as the way course offerings are organized, named, and announced, we fail to provide students who will soon be professionals with the tools they will need to recognize the social structures through which individual action is channeled—skills they need to make their way in the world."
— Susan S. Silbey, Leon and Anne Goldberg Professor of Humanities, Professor of Sociology and Anthropology, Professor of Behavioral and Policy Sciences, and Chair of the MIT faculty (2017-2019)
Influences on Social Structures
Although studying culture is the adopted subject of many disciplines, sociology and anthropology specifically attempt to trace the links between the particular and the general to identify the mechanisms for aggregating individual actions or persons into collectivities and collective action, and as practices of a circulating culture. From this perspective, we might think of the task of ethics education as socio-cultural analysis, and preparation for a career as a scientist or engineer as requiring lessons in history, organization, cultural exploration, and management. Ethics education needs to be, following MacIntyre, historically contextualized, including analyses of what happens in particular situations to identify the logics operating in that historically located situation.
Preparation for a career in science, for example, might include attention to the organization of laboratories (including perhaps how they have changed over time), the incentives and pitfalls of different forms of funding (including the differences between grants and contracts), as well as the role of gender in both local group and external professional activities. Preparation for a career in computer science, for example, might pay special attention to historical examples of technological catastrophes and the transformation of technologies into systems of social control. In some of our research, we refer to these kinds of accounts, which I am suggesting ought to be the subject of ethics education, as subversive stories: narratives that subvert or undermine claims of individualist causality by revealing how social structures link the general and the particular. Subversive stories reveal the patterns of aggregation through systems of opportunity and reward as well as structure and constraint.
For a moment, let’s imagine repairing the conventional individualist narrative of ethical lapses by adopting the usual modifications: suppose we change the story of a few bad apples, to one about many bad apples, or one about all bad apples. This will not suffice, however. As long as we are describing the apples, we have preserved a system, a set of practices, idealizations, and cultural resources that support misconduct. We have helped to tell what we call a hegemonic tale, a story that buries the social organization of action and power, and thus absolves us all of a deeper responsibility.
If we talk about ethics as individual decision-making without history, context, social structure and culture, we have not explained how the organization of apples in the barrel is part of why we see only an occasional bad apple and how those bad apples can infect the other apples. What are the mechanisms of infection and spread? This is the missing structural element that conventional accounts of ethics as bad apples usually miss, and an alternative approach to ethical education and responsibility might offer.
When a curriculum lacks a solid grounding in organizational and institutional analysis, it encourages the hyper-individualism characteristic of American culture, media, and politics generally. Rather than provide students with subversive stories and the tools for critical inquiry, the curriculum, inadvertently perhaps, becomes a vehicle reinforcing popular ideologies. As Karen Levy, Professor of Information Science at Cornell recently noted, “... if data science ethics training focuses entirely on the individual responsibility of the data scientist, it risks overlooking the role of the broader enterprise, ...” which is also making choices about its products and policies.
Accurate Stories
Too often, history, arts, corporate behavior, public affairs are understood as merely a series of individual actions, the product of human decision, utility, invention, malfeasance, avarice, or creativity: a series of particularly special and delicious apples with perhaps some attention to which trees grow better apples, but little attention to the organization of the orchard, especially the cultivation, resources, and weather that sustains the orchard. The culture writ large is understood, implicitly if at all, as an aggregation of individual preferences and attitudes.
Too often, stories are framed primarily as tools for obtaining desired ends (normally power – i.e., tell a good story and you can convince others of what you want, successfully market yourself or your product). The mechanisms and processes of aggregation that provide intervening conditions that influence, channel, and organize human action are the subject of only a few elective, easily overlooked courses. It is common for students to complete degrees without any notice, no less concerted attention, to the processes and structures organizing human action, and accumulating power. And then, when crises of public confidence and professional irresponsibility erupt, we hear calls for training in (personal) ethics.
To the degree that this neglect of institutional analyses is a result of the way degree requirements are specified, as well as the way course offerings are organized, named, and announced, we fail to provide students who will soon be professionals with the tools they will need to recognize the social structures through which individual action is channeled, skills they need to make their way in the world.
Should students leave college and professional training believing that their individual will and personal resources are the major opportunities and limits determining success and failure, they will find themselves frustrated when they butt up against those very powerful, yet invisible social structures. This naivite, or ignorance, is one crevice in which unethical behavior germinates. Should students, however, have an understanding of the constraints and resources of organizational structures and institutionalized cultures as well as worthy individual attributes, they will be more effective, perhaps more ethical professionals and citizens.
What story of ethics do we tell MIT students? Is it the familiar story of great men and a few women individually overcoming great obstacles and ignorance to push back the frontiers of knowledge? Is it the reductionist and one-dimensional story of initiative and creativity working to disrupt conventional modes of production? Let’s stop telling this one hegemonic tale and ask instead what stories (plural) we should be using to help us interpret our world with greater precision and complexity. We might then, for example, demonstrate alternative modes of organizing technological and scientific production.
For example, how might we organize to limit winner-take-all dynamics? What would happen if our intellectual property regime became more consistent with actual creative practices? Let’s tell stories that interpret the world with greater nuance and empirical validity, and expose us to multiple voices that push us out of our habitual ways by seeing the world through the subjectivity – eyes and voices – of others. In Sapiens: A Brief History of Humankind, the Israeli historian Yuval Noah Harari makes a very similar claim as does the Nigerian novelist Chimanda Ngozi Adichie when she writes, “Stories matter. Many stories matter. Stories have been used to dispossess and to malign, but stories can also be used to empower and to humanize. Stories can break the dignity of a people, but stories can also repair that broken dignity.”
• • •
Forming the MIT Stephen A. Schwarzman College of Computing
Susan S. Silbey, MIT Faculty Chair
from the MIT Faculty Newsletter, November/December 2018 edition
MIT Building 10, Maclaurin Buildings
"The college is envisioned to be the nexus connecting those who advance computer science, those who use computational tools in specific subject fields, and those who analyze and write about digital worlds."
— Susan S. Silbey, Leon and Anne Goldberg Professor of Humanities, Professor of Sociology and Anthropology, Professor of Behavioral and Policy Sciences, and Chair of the MIT faculty (2017-2019)
On October 15, 2018, MIT announced a gift from Stephen A. Schwarzman to create a college of computing. Eleven months earlier, in November 2017, President Rafael Reif began discussing with the faculty officers the possibility and prospects for a School of Computing. He sought our advice about how the proposal from some members of the computer science faculty might be explored with the faculty at large. We recommended extensive conversations across the Institute to gather the faculty’s collective wisdom and hear their worries.
In my role as Chair of the Faculty, I accompanied Rafael, Provost Martin Schmidt, and Engineering Dean Anantha Chandrakasan during the spring semester 2018 to meetings with all School councils and several large departments. I also joined conversations in Rafael’s office with several department heads and program directors. The MIT Schwarzman College of Computing has now been created; it has not yet been designed.
Collective Intelligence
A vision for the College evolved over the course of these conversations. Most importantly, faculty beyond EECS wanted any new structure to be porous with unfettered collaborations across its boundaries, so that faculty using the computational methods which have become commonplace within most disciplines would be in more regular conversation with the computer science faculty enabling adoption, and adaptation, of new methods. Equally, faculty across the Institute wanted to engage computer scientists on the substantive disciplinary problems appropriate for computer analytics, as has been the case, for example, in computational biology.
In addition, faculty across several departments offered to share the burden of teaching some basic courses in areas that involve computing and computational methods. The significant increase in undergraduate enrollments in these courses are stretching and diluting existing resources for making fundamental advances at the frontiers of computer science. Faculty consistently said that, whatever was proposed, the plan should permit more flexible collaborations for teaching, degrees, and research while providing increased manpower and material resources.
Concept
This semester, I have been accompanying Marty and Anantha on another tour of School councils, departments, and labs as they have been presenting a vision for the College, again seeking feedback and suggestions on how to enact this bold and challenging concept of a new college for MIT. The Provost’s presentation is presently a sketch of what might be put in place after further consultation and design.
The task before us now — to develop and propose designs for the College — is daunting because the College is envisioned to have the status of a School, similar to the current five MIT Schools, but with a broader mandate: to be the nexus actively connecting across the Institute those who advance computer science, those who use computational tools in specific subject fields, and those who observe, analyze, and write about digital worlds.
There are certainly observable tensions pulling and pushing the vision of the College in competing directions. Some expect the College to be a bigger and better extension of the existing Department of Electrical Engineering and Computer Science with auxiliary research labs and centers. Others want the College to be a different kind of organization devoted not only to advancing computation and computer science but to fully accept — by making visible, rather than tacit — the inescapable human preferences embedded within digital technologies, both the unintended as well as predictable consequences of these technologies. To his credit, the Provost has publicly acknowledged these tensions that we must now embrace and mediate.
Research
Facing this daunting mission, I am reminded of Admiral Rickover’s cautionary advice concerning academic designers, if not specifically about designing academic organizations:
The academic [nuclear reactor] designer is a dilettante. He [sic] has not had to assume any real responsibility in connection with his projects. He is free to luxuriate in elegant ideas, the practical shortcomings of which can be relegated to the category of ‘mere technical details.’ The practical [reactor] designer must live with these same technical details. Although recalcitrant and awkward, they must be solved and cannot be put off until tomorrow. Their solutions require manpower, time and money [1953 letter].
There is much that needs to be researched and decided in a short period of time if the College is to open in fall 2019, demanding manpower, time, and money. We have not specified what opening looks like, other than perhaps a web page. We should investigate historical and contemporary examples here and at other universities with respect to, for example, component units (shall there be departments as well as labs and centers?); roles and responsibilities (shall a member of the new College sit on each of the existing School councils and in turn members of the five Schools sit on a governing council of the new College?); distribution of resources (how shall College funds be distributed for computational teaching across the Institute, for support staff, and for hardware?); decision-making processes for faculty appointments and student degrees (how shall faculty be appointed, evaluated, and promoted if they sit in more than one department or School; what degrees shall be offered; how shall graduate students be admitted and funded?).
Working Groups
In active collaboration with the faculty officers and the new Dean, the Provost has created a task force comprised of multiple working groups to generate recommendations on the design of the Schwarzman College of Computing, a process successfully implemented during the 2008 financial crisis. Each working group will have a charge designating the central topics for its inquiry to culminate in a set of proposals and recommendations. The Provost has already created a search committee with five School representation to identify candidates for Dean of the new college; a second committee will interview the identified candidates.
We currently expect to stand up working groups (in canonical MIT fashion with representation from the five Schools) to consider and address the following issues. The particular charges and compositions are in process:
- Organizational structure: to explore organizational configurations or structure with two tracks. One subcommittee will consider what kind of organization will provide agility while ensuring stewardship of key elements such as degrees, graduate student supervision and mentoring, faculty hiring and promotion. Should there be project-centered interdisciplinary labs? How will the internal structural elements relate to departments in the other Schools? A second subcommittee will explore the preferences and expectations of the faculty who will reside within the College.
- Faculty appointments: to specifically consider how bridge appointments will be structured, hired, mentored, promoted. What are the categories of appointments in the College, and how are these the same or different from appointments in the current Schools? What are the rights and responsibilities of faculty appointed in more than one unit? How are they hired, mentored, promoted? How are transitions managed as faculty may move from one unit to another within the College and in other Schools? Might we consider some appointments with limited terms?
- Landscape of available models: to gather intelligence about similar colleges and schools around the nation and globe. What has been tried elsewhere and how is it working? What might we envision as distinct for MIT?
- Social Implications: to explore the social implications of computing. The College aspires to integrate thinking, research, and teaching about the social implications of computing into everything it does, in education and in research. How might this be instituted?
- Curriculum and degrees: to conduct a census of all courses offered on or about computation. What degrees will be offered by units in the College? How will the courses be coordinated within the College and with the existing departments and Schools?
- Computing infrastructure: to identify best practices with respect to sharing common resources. How does the College ensure that everyone has access to hardware and professional staff?
Clearly, the topics of these working groups intersect. The multiple groups will provide greater opportunity for faculty, staff, and student participation and feedback than would a smaller set. The chairs of the working groups will meet to coordinate and communicate across the groups and with the new Dean (or Acting Dean) and Provost. We expect the working groups to be in place by December 2018 with recommendations in June 2019, following interim reports during the spring semester.
Guiding Questions
From the many conversations over these two semesters, we have generated a set of questions that seem to represent the central concerns and multiple goals we have heard. In developing proposals and recommendations, we hope that the working groups will be guided by their charge and these questions, which can function as a kind of “catechism” for the design of the College.
- Will the recommendation facilitate collaboration and promote integration across MIT departments and other units in curricular planning and research?
- Will the recommendation acknowledge and maintain respect for the demonstrated expertise of colleagues with regard to computing arts and sciences?
- Will the recommendation ensure that faculty with appointments in more than one unit have clearly defined responsibilities that do not impede the normal progression of an academic career?
- Will the recommendation create an unusual burden on any unit, benefit one group at expense of others, or disenfranchise anyone?
- Will the recommendation increase MIT competitiveness with regard to faculty and student recruiting and retention?
- Will the recommended administrative structure sustain these principles for fair and appropriate allocation of resources (space and funding), appointments, teaching and related assignments within the College and with respect to the five MIT Schools?
- Will the recommended design of the College incorporate flexibility to accommodate the possibility that some current trends (e.g., in enrollments) might shift dramatically so that changes will be appropriate?
Commitment
We, the faculty, are eager to take up this challenge and enthusiastically support the Provost’s commitment that the MIT Schwarzman College of Computing be our next step toward a better world through more socially responsible and yet even more adventuresome computing. We commit ourselves to the work of realizing this bold vision.
• • •
Suggested links
Series:
Ethics, Computing, and AI | Perspectives from MIT
Susan S. Silbey:
MIT webpage
MIT Anthropology
Stories:
The Human Factor: Anthropologist Susan Silbey on solving major global issues
"Transferring the models of physical matter or rational calculation to these massive global problems can do a great deal to help solve our current issues — but only when they are informed by a nuanced understanding of how humans and human organizations operate."
Anthropologist Silbey receives award for research on forging safety in science labs
Aligning ideals and real-world conditions
Female scholars led by MIT anthropologist Susan Silbey illuminate path to commonsense regulation
Journal volume advances research stalled by political agendas.
Why do women leave engineering?
Study analyzes the issue and proposes remedies.
Culture Shift
Study shows that expanding engineers’ horizons is essential to addressing the field’s gender imbalance.