Does soft­ware automa­tion improve or ham­per your life?

Most peo­ple embrace the idea of tech­nol­o­gy. How many peo­ple real­ly would rel­ish writ­ing a let­ter or even an email with­out spell check? Or endure a rush-hour dri­ve into a for­eign city with­out a GPS system?

Nicholas Carr, author of The Glass Cage, argues that for all of technology’s ben­e­fits, it has its share of per­ils. When peo­ple use com­put­ers, they often fall vic­tim to a pair of cog­ni­tive ailments—automation com­pla­cen­cy and automa­tion bias.

Technology Can Harm our Careers and Threaten Lives

Automa­tion com­pla­cen­cy lulls us into a false sense of secu­ri­ty. Carr writes: “We become so con­fi­dent that the machine will work flaw­less­ly, han­dling any chal­lenge that may arise, that we allow our atten­tion to drift. We dis­en­gage from our work, or at least the part of it that the soft­ware is han­dling, and as a result may miss sig­nals that some­thing is amiss.” Any­one who uses a word proces­sor will have expe­ri­enced this: per­haps the spell check doesn’t catch an embar­rass­ing error. Or a foot­er does not dis­play the cor­rect information.

Automa­tion bias occurs when peo­ple give too much cre­dence to the infor­ma­tion dis­played by a com­put­er. Even when the infor­ma­tion is wrong or mis­lead­ing, peo­ple may still believe it. One alarm­ing exam­ple is a dri­ver of a school bus who mis­tak­en­ly rammed into a con­crete bridge in Seat­tle, injur­ing 21 stu­dents. The dri­ver told police that he had been fol­low­ing the GPS instruc­tions and “did not see” signs and flash­ing lights warn­ing of the impend­ing bridge.

Soft­ware automa­tion is every­where in our soci­ety. Accoun­tants use deci­sion-sup­port soft­ware in cor­po­rate audits. The soft­ware expe­dites the work but “there are trou­bling signs that as the soft­ware becomes more capa­ble the accoun­tants become less so,” writes Carr. One study, con­duct­ed by a group of Aus­tralian pro­fes­sors, exam­ined the effect of the soft­ware sys­tems used by three account­ing firms. Those accoun­tants who had to make more of the deci­sions them­selves had “a sig­nif­i­cant­ly stronger under­stand­ing of dif­fer­ent forms of risk” than the oth­er accoun­tants who relied on the soft­ware. “The decline in learn­ing asso­ci­at­ed with advanced soft­ware affect­ed even vet­er­an audi­tors,” writes Carr.

A ten­den­cy to be increas­ing­ly reliant on soft­ware is not unique to accoun­tants. Finan­cial pro­fes­sion­als, lawyers, doc­tors, human resource pro­fes­sion­als, pilots all rely on soft­ware to com­plete aspects of their jobs. The book con­tains many exam­ples of how these well-edu­cat­ed pro­fes­sion­als become over­ly reliant on tech­nol­o­gy and make mistakes—sometimes life-threat­en­ing ones.

Online Information May Weaken our Memory

In the devel­oped world, most of can­not imag­ine our lives with­out an Inter­net con­nec­tion and a search engine.

When you type in a search query using Google, the search engine does a great job of fig­ur­ing out what you want. It cor­rects your spelling. It sug­gests search terms. It even antic­i­pates your needs based on who you are and the terms you have used in the past. Over time, you might think that users are learn­ing from Google’s sug­ges­tions so that they are more savvy when typ­ing search terms.

The oppo­site is true. A series of exper­i­ments in Sci­ence mag­a­zine show that all the avail­able online infor­ma­tion weak­ens our mem­o­ry for facts. Why remem­ber fac­tu­al con­tent when a search engine will do it for you? The researchers con­clud­ed: “Since search engines are con­tin­u­al­ly avail­able to us, we may often be in a state of not feel­ing we need to encode the infor­ma­tion internally.”

Amit Sing­hal, Google’s lead engi­neer, notes that the bet­ter Google’s search engine becomes, the lazier we are with our typ­ing and queries.

Most of us wouldn’t accept a world with­out soft­ware, whether it’s a GPS, bank­ing soft­ware, or a smart­phone. Tech­nol­o­gy has rev­o­lu­tion­ized our lives in many pos­i­tive ways. But for those of us enam­oured with tech­nol­o­gy, Carr’s book is a a cau­tion­ary tale—don’t let your guard down too much when using it.