Exploring the Unintended Consequences of Automation in Software

Automation is ubiquitous—it is entwined in our daily lives in ways that we aren’t always aware of. It has been woven into all aspects of modern software by being presented as a utopian vision: a way of making human lives easier, doing repetitive tasks faster and with fewer errors, freeing us fallible humans up to do other ostensibly more important work. But anyone who has worked directly with automated systems knows that we are still very far from such a dreamy reality.

This talk delves into detailed research about how automation is involved in software incidents. My focus on this area stems from the growing portrayal of automation as a panacea for various software incident issues, despite its limitations in effectively addressing these challenges, such as reliable detection and resolution of software issues or analyzing and disseminating learnings from these incidents back into the organization and its products and services.

Drawn directly from public incident reports (collected in the VOID), the research revealed multiple, often competing, roles that automation can play over the course of an incident, and most importantly underscored how important humans are at understanding, troubleshooting, and recovering from automated software issues. It’s a slightly vexing perspective, but all is not lost! I close with research-backed suggestions for ways we can coexist and build better systems in partnership with automation.


Speaker

Courtney Nash

Internet Incident Librarian & Research Analyst, Previously @Verica, @Holloway, @Fastly, @O’Reilly Media, @Microsoft, & @Amazon

Courtney Nash is a researcher focused on system safety and failures in complex sociotechnical systems. An erstwhile cognitive neuroscientist, she has always been fascinated by how people learn, and the ways memory influences how they solve problems. Over the past two decades, she’s held a variety of editorial, program management, research, and management roles at Prowler, Verica, Holloway, Fastly, O’Reilly Media, Microsoft, and Amazon. She lives in the mountains where she skis, rides bikes, and herds dogs and kids.

Read more
Find Courtney Nash at:

From the same track

Session Change Management

Systems Thinking for Building Resilient Engineering Organizations

Monday Nov 18 / 10:35AM PST

The introduction of mainstream LLMs is not only changing the products we build, it's also changing the way we build those products, the way we hire and the way we build our teams.

Speaker image - Michelle Alexander

Michelle Alexander

VP Engineering @Fanatics, Previously Platform and Product Engineering Leader @SeatGeek and @Spotify, Engineering Roots in Mobile Development

Session Generative AI

What to Pack for Your GenAI Adventure

Monday Nov 18 / 11:45AM PST

🎥 The recording of this session will become available within the next few weeks.

Speaker image - Soledad Alborno

Soledad Alborno

Sr. Product Manager @Google, Building Transformative AI Experiences, Latina in Tech

Session Mental Health

Achieving Sustainable Mental Peace at Work Using GenAI

Monday Nov 18 / 03:55PM PST

This presentation explores how generative AI technology can be leveraged to enhance mental health and build sustainable mental peace in high-pressure work environments, particularly for achievement-oriented and neurodivergent individuals.

Speaker image - John  Gesimondo

John Gesimondo

Senior Software Engineer @Netflix

Session Resilience

Built to Outlast: Cultivating a Culture of Resilience

Monday Nov 18 / 01:35PM PST

In the tech industry, where the only constant is change, the companies and individuals who survive the longest are those who have learned to reinvent themselves over and over again by learning to thrive in chaos and uncertainty.

Speaker image - Kathleen  Vignos

Kathleen Vignos

VP, Software Engineering @Capital One, Previously Engineering Director @Twitter and @WIRED, Mentor, Facilitator, Speaker, 25+ Years in Tech

Session

Unconference: Sociotechnical Resilience

Monday Nov 18 / 05:05PM PST