Exploring Values and Norms of Engineering Through Responsible Innovation and Critiques of Engineering Cultures

Paper ID #34487Exploring Values and Norms of Engineering Through Responsible tion Innova-and Critiques of Engineering Cultures Dr.. Exploring values and norms of engineering through resp

Paper ID #34487

Exploring Values and Norms of Engineering Through Responsible tion

Innova-and Critiques of Engineering Cultures

Dr Rider W Foley, University of Virginia

Dr Rider W Foley is an assistant professor in the science, technology & society program in the partment of Engineering and Society at the University of Virginia He is the principal investigator at University of Virginia on the ’4C Project’ on Cultivating Cultures of Ethical STEM education with col- leagues from Notre Dame, Xavier University and St Mary’s College He is also the co-leader of the

De-’Nano and the City’ thematic research cluster for the Center for Nanotechnology in Society at Arizona State University Rider is a Research Collaborator with the Sustainability Science Education program at the Biodesign Institute His research focuses on wicked problems that arise at the intersection of society and technology Rider holds a Ph.D in Sustainability from Arizona State University, and a Master’s de- gree in Environmental Management from Harvard University and a Bachelor’s degree in Environmental Science from University of New Hampshire Before earning his doctorate, he has worked for a decade in consulting and emergency response for Triumvirate Environmental Inc.

Rachel Sinclair, University of Virginia

Rachel Sinclair is a graduate with a Master of Public Health and Bachelor of Arts, major in Psychology, from the University of Virginia She is beginning her professional career as an Associate Clinical Research Coordinator at the Mayo Clinic Prior research experience has involved neurodegenerative disorders, pathogens, mental health outcomes and policies, and engineering ethics education.

Araba Dennis, Purdue University

Araba Dennis is a second-year PhD student studying race, culture, and institutional definitions of sion.

inclu-c

Exploring values and norms of engineering through responsible

innovation and critiques of engineering cultures

disengagement” and the pillars of depoliticization, socio-technical dualism and meritocracy demonstrated shifts in values among students at four very different universities in Massachusetts Out of political science and technology assessment the concept of responsible innovation is gaining traction as a means to bring values and norms into the center of innovation activities, exemplified in works by Rene von Schomberg and David Guston However, the critiques of engineering cultures are often absent from concepts of responsible innovation For if responsible innovation is a normative shift towards a better future, then understanding what is undesirable about the cultures of engineering and innovation must be equally important This research

explores how engineering students at a university in Virginia express values and norms when asked the question: “What is engineering?” The research design captured long-form essays prior

to and after taking an engineering ethics course Those essays were coded thematically for

dimensions of irresponsible innovation, e.g., myth of objectivity, depoliticization and

reductionism, as well as for dimensions of responsible innovation including stakeholder

engagement, future anticipation of consequences, and adaptiveness Methodologically, the results suggest that long-form essays, in addition to the surveys used by Cech and the case

studies offered by Riley, offer an intriguing method to analyze the emergent values and norms among engineering students Secondarily, the empirical results suggest subtle shifts in the

discourse about what engineering is and, thus recognition of values that might underpin cultures

of responsible innovation

Keywords: Content Analysis, Engineering Education, Engineering Ethics

professional society’s values and ethical obligations Macroethical dilemmas result in the

“problem of many hands”, as described by van der Poel and Royakkers [3] This brings to light the notion that individuals or even large organizations are not solely responsible for engineering processes and uncertain outcomes For it is clear that no individual or discrete organization has complete control and authority for the complex socio-technical innovation process from design

to implementation, nor for the maintenance and disposal of engineered systems [4-6]

The challenges associated with such a lack of complete control or authority are often handled in one of two ways that align with distinct philosophies of engineering The first of these

is the pursuit of greater control and authority as exercised by myriad forms of power, which is much discussed in engineering ethics and manifests in multiple forms Scholars focused on the study of engineering practices have detailed how the elusive sense of control can create even greater vulnerabilities when macroethical challenges arise As Bryan Newberry [7] detailed in the case of New Orleans and Hurricane Katrina the desire for greater human control may very well have contributed to many of the compounding errors that yield catastrophic results More recently, a second approach has emerged in the teaching and scholarship and stands in stark contrast to aspirations for greater control This approach seeks to foster reflexivity and learning about one’s own context and broader societal implications of engineering practice Robbins [8] offers the notion of the “reflexive practitioner” as an emergent theme in engineering ethics However, there are few examples for how such reflexivity can be demonstrated in the education and maturation of engineers

This project aims to address that knowledge gap in a small, but important way, by

assessing reflective writing by engineers in an undergraduate program This paper offers data from 65 students that wrote 1-page essays in response to the question: “What is engineering?” The essays were analyzed using the mirrored concepts of (ir-)responsible innovation, which are reviewed in the next section The results offer but one example of how reflective writing can offer a means to assess students’ learning outcomes The discussion returns to the novelty of the mirrored concept of (ir-)responsible innovation and offers pathways forward for future research

to address the many limitations of the research design for this study, namely the single-university focus and small sample This paper aims to demonstrate an assessment technique that moves from rule-following ethics education to an approach that represents the students’ values and beliefs

Conceptual Framing

The following review of literature serves as an overview of the conceptual foundations that inform the (ir-)responsible innovation as a framework for analysis Joseph Herkert’s [2] discussion of microethics and macroethics provides a foundational lens for understanding how students might become oriented towards expressions of responsible innovation Herkert explains this distinction as microethics being primarily, “concerned with ethical decision making by individual engineers and the engineering profession’s internal relationships ” [2, p 374] While microethical decisions are those that are generally considered to be “the right thing” with regard

to one’s particular duties or responsibilities, they fail to consider the organizational or social conditions that engender the need for such individual decisions Conversely, macroethics

revolves around the “collective social responsibility” and “societal decisions about technology” relating to the engineering profession as a whole Stressing a macroethical approach to

engineering pedagogies allows engineering students to consider responsible innovation as a conduit for fitting individual subjectivity onto a larger social context While this micro and macroethical framing sets the stage for this research, it is not used directly for analysis

Irresponsible innovation: Foundations and facets

Donna Riley’s book, Engineering and Social Justice, builds upon Herkert’s proposed

framework of macroethics and how it relates to an established engineering culture Riley’s [9] critiques of engineering culture take a normative position on the values and principles that

inform engineering practice, which represents an important shift in the field Prior work on engineering cultures, led by Gary Downey and others, offers a detailed and descriptive analysis

of engineering cultures and how nationalism and shared identity were important aspects that differentiated German, French and English cultures of engineering [10] Riley [9] shifts away from such a nationalistic perspective and instead focuses on six main characteristics of

engineering culture that appear to transcend nationalistic differences The first is a positivist epistemology that is centered upon a myth of objectivity, such that engineering can be performed without any inputs based upon subjective knowledge and the flaws associated with that way of understanding the world This myth builds upon a commitment to problem solving that aims to reduce complex social problems into discrete technical components and then focuses on solving those technical problems Such a reductionist approach is typified in Alvin Weinberg’s essay that was reproduced and contextualized as a core lesson for engineers [11] Riley’s book on social justice and engineering highlights the narrow focus on technical skills and competencies centered around the “hard” sciences and mathematics as a way of reinforcing this principle

Regarding the career orientation of aspiring engineers, there is a long tradition of

engineering as it relates to the military, with Virginia Military Institute being one of the first secondary schools with a dedicated focus on engineering [10] That tradition is alive and well, yet there is an emergent emphasis on corporate organizations and partnerships that fund

engineering research at universities and offer clear career pathways for university graduates What Riley [9, p 39-40] strongly critiques is how this facilitates an uncritical acceptance of authority and foregrounds profit-oriented engineering with economic calculations as the only proxy for social acceptability The “rule-following” ethics education offered in many

universities places an emphasis on the individual’s decisions and de-emphasizes the

organizational impacts and broader implications of any given design or project As Jessica Smith points out in her research at the Colorado School of Mines, the ethics associated with material provisioning is often used to justify (or set aside) local concerns about the impacts of mining and other processes that extract natural resources [12] Those impacts are often deemed as hyper-local and offset by the materials provisioned to the global community, such as gasoline or

plastics from petroleum extraction

Cech [13] provides evidence of a “culture of disengagement” that uses longitudinal survey data from four different engineering education programs She offers three main

“ideological pillars” as the following beliefs: (i) an “ideology of depoliticization” wherein

science and engineering are “pure” spaces free of social and cultural concerns, (ii) a

“technical/social dualism” wherein technical knowledge and competencies have more value than social ones, and (iii) a “meritocratic ideology” wherein scientific systems are unbiased, with fair systems of advancement [13, p 45] Amy Slaton [14] builds upon these notions and offers

additional evidence that links the premise that technological decisions are not influenced by political power and structural relationships with the ideals of individual achievement Slaton [14] argues that the neo-liberal ideals of meritocracy and technocracy reinforce bias against non-White and non-male persons, thus perpetuating historical and cultural inequities that track to racial and gender identities It is not surprising that those three scholars collaborated on a recent book chapter that nicely summarized and synthesized their research efforts [15] That work highlights the concept of grit or persistence as foundational to the experience of undergraduate engineers, whose educational journey is often described with analogies like “gauntlet” or “trial

by fire” This enculturation process is underpinned by the ideals of meritocracy, yet yields inequitably outcomes for minority populations, as seen in the admissions and graduation data that are routinely cited in reports on the “leaky pipeline” in STEM education

Together this critically reflexive scholarship on engineering education and culture offers

a conceptual foundation of irresponsible innovation in engineering, which includes the following facets:

1 Positivist epistemology and the myth of objectivity

2 Commitment to problem solving based on reductionism (reductionist thinking)

3 Persistence or “grit” as a personality trait

4 Centrality of military and corporate organizations and decision-making that privileges command and control models of authority

5 Technocratic or narrow technical focus regarding decision-making and design

6 Uncritical acceptance of authority and rule-following behaviors

Responsible Innovation: Foundations and Facets

The roots of responsible innovation extend into the philosophies of deliberative

democracy Richard Sclove’s book Technology and Democracy offers an entry to the

relationship between deliberative democratic theory, technological designs and decision-making Attempts to apply deliberative democratic theories to the research and development of

technology have been experimented with around the world with examples readily apparent in the Netherlands [16], France [17], the United States [18], Brazil [19], India [20] and Japan [21] Those experiments were seen as a means to address the negative consequences and perceptions

that can accompany the emergence of novel scientific and engineering research associated with nuclear energy and waste [22], genetic engineering [23], and more recently nanotechnology [24]

Recently, Rene von Schomberg’s book chapter [25] offers a conceptual definition of responsible innovation that many scholars reference in order to anchor (or critique) the notion The phrase most frequently pulled from that book chapter is, “A transparent, interactive process

by which societal actors and innovators become mutually responsive to each other with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products (in order to allow a proper embedding of scientific and technological

advances in our society)” [25] von Schomberg’s work is important as it expresses an explicitly normative position and, yet offers deliberative democratic theory as a means to arrive at the

“good” outcomes desired from innovation

Building upon that book chapter, Stilgoe and colleagues [26] offered four dimensions of

responsible innovation The first dimension offered was anticipation that can be briefly defined

as a predisposition towards the future that explores desired outcomes and associated uncertainties

that are plausible through foresight methods [27,28] Second was reflexivity, a form of

self-critique that can attend to the individuals, organizations or societal structures that are supporting (or constraining) innovation activities [8, 29] This can be achieved through various methods that are internal to the individual or organization or facilitated by external means [30] Third is

inclusion (or inclusive stakeholder engagement), which draws from Brian Wynnes [31] research that demonstrated that stakeholders possess different forms of knowledge and that a greater diversity of knowledge can avoid devastating outcomes Andy Stirling [32] and Jason Chilvers [33] scholarship articulates the demands for inclusive stakeholder engagement, while Frewer and Rowe [34] articulate criteria for assessing the inclusivity of stakeholder participation This

framework is activated by the notion that stakeholders need to be responsive to the knowledge

acquired from future-oriented, self-critical and diverse stakeholder perspectives They argue that

it is not enough to conduct foresight activities and anticipate desired and undesired futures, but the important next step is to take actions, adopt rules or make other changes based upon that new knowledge

Foley and Gibbs [35] review of prior literature aligned four additional dimensions of responsible innovation with engineering practice The first being systems thinking, which

requires engineers to consider the future implications of their designs, as well as how a novel technology will create social and environmental implications from the extraction of resources to the ultimate disposal or recycling [36] By taking a systems approach, engineers need to work in coordination with diverse stakeholders and recognize and work to negotiate trade-offs between the technical designs and the demands of other stakeholders [37] The coordination and conflict negotiation needs to be conducted in a manner that is transparent to the involved stakeholders [38,39] The intended outcome is to design and process technology that contribute to human flourishing and social justice [40, 41]

Research Design

The concepts discussed in the literature review demonstrate a shift from individual

responsibility to the collective responsibility Further, the facets of irresponsible innovation and

dimensions of responsible innovation offer an analytical framing for this research Scholars faced with the challenge of conducting research on responsible innovation often look to discourse and content analysis as a means to glean insights about the perceptions and beliefs of individuals and discrete social groups [42] This informs the research question and methods of analysis for this research We ask: How is (ir-)responsible innovation represented in the writings of

undergraduate engineers?

Course context: Bringing science, technology and society into engineering education

The University of Virginia, has structured course requirements to ensure that all

engineering students will graduate with the technical knowledge of their field, accompanied by

an understanding of the social context, engineering ethics and the contemporary issues in the field Students are required to take four courses in science, technology, and society (STS) The first course is taken during undergraduates’ first semester as an introduction to the relationship between engineering and society The second required course is at the 2XXX or 3XXX level (second or third year), and students can select from the course catalog that addresses a number of topics such as, data ethics, entrepreneurship, laboratory life, for example These courses use approaches aligned with the humanities and social sciences to further investigate the social and ethical issues related to engineering and engineered artifacts In their fourth-year all engineering students take a yearlong course sequence in both their fall and spring semesters This is where they learn about STS theories, consider various ethical frameworks and apply these concepts to their own research topics A graduation requirement is for all students to generate a written portfolio that includes a report on their technical capstone project and STS research paper that addresses aspects of engineering and the social and ethical context This curriculum has been previously recognized by the National Academies of Engineering as exemplary [43]

The in-person course uses activity-based learning that is student-oriented, such activities include worksheets, concept mapping, and role playing, for example The students are

introduced to core concepts from science, technology and society (STS) and then readings on responsible innovation are brought into the course The classroom activities involve stakeholder mapping, analysis of case studies and activities designed to demonstrate how the STS

frameworks can be used to assess socio-technical systems For example, one activity involves using actor network theory to analyze the relationships between electronic healthcare records within the healthcare setting This activity is performed in small groups and then students share their analysis and co-construct a large network as a full class This activity demonstrates how a single technology is related to the broader network of human and non-human actors and the ways that power and authority are structured Another activity takes up Winner’s [44] paper and students consider how the designs of bridges in Long Island and the tomato harvester designed for the central Valley in California exerted power through the designs of the technology In small groups, the students visualize the design elements of the tomato harvester and then connect them to the designers, funders, land-owning farmers and labors A reading of Pinch and Bjiker’s [45] about the historical influence of stakeholders on the design of bicycles helps students

understand how the physical infrastructure, values and preferences and cultural norms informed the design of the “safety bike.” Those lessons are then carried forward with an exercise that takes focuses on five different bicycle designs: mountain bikes, electric bikes, carbon fiber racing bike, beach cruiser bikes and children’s bicycles That secondary activity encourages the students to

take the historical lessons from Pinch and Bjiker [45] and to apply them to contemporary designs

of the same technology Those activities do not inform the results directly, rather they serve to provide greater context for the educational intervention in question

Data Collection and Thematic Coding

Students were invited to participate in a research study to evaluate their representations of engineering and demonstrate their values of engineering and engineering education No

incentives were offered to the participants and all students in the course were required to

complete the same amount of work Per the requirements set forth by the School of Engineering, the instructor was not present during the recruitment for the research study The assignments that served as the data for this study took the form of two long-form essays At the start of the fall semester on the first day of class all students were assigned a long-form essay (1-page) that would be graded pass/fail The prompt for that essay was, “What is engineering?” At the

conclusion of classes in December, the same assignment was given Those two essays comprise the pretest and posttest for this research study

The participants two writing samples offer the evidence that is brought to bear on the research question The population of students solicited for participation was 72, of which 65 gave their consent and completed at least one of the requested tasks The remaining students either did not consent or did not complete both the pretest and posttest required tasks for full participation The pretest and posttest yielded approximately 130 pages of writing for analysis The writing samples were cataloged before being uploaded into Dedoose®, a software program that supports thematic content analysis The literature review above was converted into a

thematic coding schema The thematic codes were clustered into two groups based upon their normative orientation: Irresponsible and Responsible Innovation, see left-hand column of Table

1 below The analytical categories were then broken into two tiers based upon the theoretical framing suggested by the prior literature introduced earlier This coding schema was first

proposed and explored by Araba Dennis, and presented at the Forum on the Philosophy of

Engineering and Technology in 2018 After a process of iterative reflection and refinement, the research team settled on the themes offered in Table 1, below

Once the analytical categories were finalized, the writing samples were analyzed by two graduate student researchers When coding the student writings with this analytical framework, the researchers looked for specific words, phrases or references to the thematic codes One researcher first went through each sample and highlighted relevant excerpts and assigned codes The second researcher reviewed these annotated documents, noting new excerpts and adding additional codes to those that fit into multiple frameworks Agreement, disagreement or

uncertainly in regards to the previous notations were also marked After a secondary review of the thematic codes, both researchers went through each writing sample in Dedoose® to annotate the excerpts and codes for ease of analysis The final codes entered into the software were based upon the researchers’ previous analysis and the secondary review This process resulted in consensus coding of the data The lead author, Rider Foley, did not participate in the application

of codes, but rather consulted when debates arose between the two researchers conducting the thematic coding The primary reason for this was due to unconscious bias of the lead author, who was the instructor of the course and recognized that they would read the essays with a more

interpretive lens Secondarily this offered to two graduate research assistants an opportunity to practice and learn about this methodology

Statistical analysis

Data was collected from 65 participants in both a pretest and posttest for a total of 125 records Four participants completed only the pretest and one participant completed only the posttest This resulted in the two researchers creating 641 excerpts for the 60 pretest and posttest records Each individual record had between 5 and 27 codes applied, and the mean number of codes applied per record was 14 with a standard deviation of 4.3 To investigate the difference

in thematic codes applied to the pretests and posttests, the rate of codes applied across all

excerpts was compared for individual codes as well as by two broader categories, as shown in Table 1 Two different statistical tests were run to analyze the significance of the changes The first was a paired t-test that compared only the records of the 60 participants that completed both the pretest and posttest Secondarily, a two-sample t-test was used to test for significance among the aggregate pretests and posttest Since the number thematic codes applied were different in the pretest and posttest treatments both the total counts and occurrence––i.e., presence or absence––

of the codes applied were analyzed in both the paired and two-sample t-test For example, for the

code reflexivity, tests were conducted based upon the total number of times the code was applied

across all excerpts which is reported as mean count and tested with both the paired t-test and a two-sample t-test for significance A second statistical test was run on the binary variable of occurrence (presence or absence) of the thematic code within each record (meaning within each

student writing sample) So, if a student’s writing was coded for reflexivity at least once, then it would count as 1 and if no occurrences of reflexivity were identified by the researchers, then it

would count as 0 This binary representation of the data was also tested with the paired t-test and the two-sample t-test Tests were conducted on the total number of aggregate count of codes across all excerpts, as well as for a binary representation of the presence or absence of those code For both the paired and two-sample t-tests, an α = 05 threshold was used to determine significance of results The co-occurrence of thematic codes applied to the students’ writing was extracted from Dedoose® to give the research team an understanding of the relationship between the applied codes The results are then enriched and interpreted in a more qualitative manner by looking at specific excerpts and the thematic codes applied, as a means to interrogate the method and explain the results further

Normative Orientation Tier 1 Codes (Parent Codes)

Irresponsible

Innovation

Positivist Epistemology / Myth of Objectivity Commitment to Problem Solving via Reductionism Persistence/ Grit

Centrality of Military and Corporate Organizations Technocracy

Narrow Technical Focus Uncritical Acceptance of Authority

Transparency Socio-technical Connections Systems Thinking

Coordination Stakeholder Engagement Anticipation

Reflexivity Responsiveness

Table 1 Thematic codes applied to the writing samples

Results

This research offers two key findings from this analysis of the students’ discourse on engineering First, this exploratory project demonstrates that the reflective writing assignment coupled with a coding schema for both responsible and irresponsible innovation offers a method for assessing the change in discourse on the ethics and values of engineering Second, the

quantitative results demonstrated a shift from discourse from irresponsible to responsible

between the pretest and posttest writing samples The qualitative evidence makes transparent how the codes were applied and further suggests that some students layered knowledge from the course onto their existing understanding of engineering, while other excerpts articulate the nature

of that shift in the discourse While the scope and boundaries of this study has limitations, this exploratory project generates productive questions and offers a path forward for future research

Quantitative results

The students’ writing samples demonstrated statistically significant shifts in both of the major categories when analyzed in in the aggregate and as binary (presence / absence) variables The mean aggregate score in the posttests for codes that were thematically aligned with

irresponsible innovation were lower than in the pretests when analyzed with both the two-sample t-test and the paired t-test, see Table 2 and 3 below While there is no difference in the mean pretest or mean posttest between the two-sample t-test and paired t-test, there are subtle

differences in the associated p-values This is due to the very small difference between total participants and paired participants Inversely, the mean aggregate score for thematic codes associated with responsible innovation were higher in the posttest On average, fewer statements associated with the core concepts of irresponsible innovation are present in the essays written at the end of the course, while more statements align with responsible innovation appear in the posttests When the application of codes was converted to a binary variable, the changes in pretests and posttests were still statistically significant for the two main clusters using both the paired t-test and two-sample t-test, as shown in Table 4 and 5 This suggests that the educational intervention induced a near-term changes in the discourse of engineering students about the values associated with their profession

Aggregate Two-Sample T-test Clusters Mean Pretest Mean Posttest Significance P-Value

Table 2 Two-sample t-test of clusters analyzed by aggregation Note: Cells colored red indicate

a lower mean posttest, while green indicates higher mean posttest

Aggregate Paired T-test Clusters Mean Pretest Mean Posttest Significance P-Value

Table 3 Paired t-test of clusters analyzed by aggregation Note: Cells colored red indicate a

lower mean posttest, while green indicates higher mean posttest

Binary Two-Sample T-test Clusters Mean Pretest Mean Posttest Significance P Value

Table 4 Two-sample t-test of clusters analyzed as binary Note: Cells colored red indicate a

lower mean posttest, while green indicates higher mean posttest

Binary Paired T-test Clusters Mean Pretest Mean Posttest Significance P Value

Table 5 Paired t-test of clusters analyzed as binary Note: Cells colored red indicate a lower

mean posttest, while green indicates higher mean posttest