Month: August 2020

Bias in, bias out: many algorithms have inherent design problems. Vintage Tone/Shutterstock

Could this be the first line of a “Hippocratic Oath” for mathematicians and data scientists? Hannah Fry, Associate Professor in the mathematics of cities at University College London, argues that mathematicians and data scientists need such an oath, just like medical doctors who swear to act only in their patients’ best interests.

“In medicine, you learn about ethics from day one. In mathematics, it’s a bolt-on at best. It has to be there from day one and at the forefront of your mind in every step you take,” Fry argued.

But is a tech version of the Hippocratic Oath really required? In medicine, these oaths vary between institutions, and have evolved greatly in the nearly 2,500 years of their history. Indeed, there is some debate around whether the oath remains relevant to practising doctors, particularly as it is the law, rather than a set of ancient Greek principles, by which they must ultimately abide.

How has data science reached the point at which an ethical pledge is deemed necessary? There are certainly numerous examples of algorithms doing harm – criminal sentencing algorithms, for instance, have been shown to disproportionately recommend that low-income and minority people are sent to jail.

Similar crises have led to proposals for ethical pledges before. In the aftermath of the 2008 global financial crisis, a manifesto by financial engineers Emanuel Derman and Paul Wilmott beseeched economic modellers to swear not to “give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.”

Just as prejudices can be learned as a child, the biases of these algorithms are a result of their training. A common feature of these algorithms is the use of black-box (often proprietary) algorithms, many of which are trained using statistically biased data.

In the case of criminal justice, the algorithm’s unjust outcome stems from the fact that historically, minorities are overrepresented in prison populations (most likely as a result of long-held human biases). This bias is therefore replicated and likely exacerbated by the algorithm.

Machine learning algorithms are trained on data, and can only be expected to produce predictions that are limited to those data. Bias in, bias out.

Promises, promises

Would taking an ethical pledge have helped the designers of these algorithms? Perhaps, but greater awareness of statistical biases might have been enough. Issues of unbiased representation in sampling have long been a cornerstone of statistics, and training in these topics may have led the designers to step back and question the validity of their predictions.

Fry herself has commented on this issue in the past, saying it’s necessary for people to be “paying attention to how biases you have in data can end up feeding through to the analyses you’re doing”.

But while issues of unbiased representation are not new in statistics, the growing use of high-powered algorithms in contentious areas make “data literacy” more relevant than ever.

Part of the issue is the ease with which machine learning algorithms can be applied, making data literacy no longer particular to mathematical and computer scientists, but to the public at large. Widespread basic statistical and data literacy would aid awareness of the issues with statistical biases, and are a first step towards guarding against inappropriate use of algorithms.

Nobody is perfect, and while improved data literacy will help, unintended biases can still be overlooked. Algorithms might also have errors. One easy (to describe) way to guard against such issues is to make them publicly available. Such open source code can allow joint responsibility for bias and error checking.

Efforts of this sort are beginning to emerge, for example the Web Transparency and Accountability Project at Princeton University. Of course, many proprietary algorithms are commercial in confidence, which makes transparency difficult. Regulatory frameworks are hence likely to become important and necessary in this area. But a precondition is for practitioners, politicians, lawyers, and others to understand the issues around the widespread applicability of models, and their inherent statistical biases.

Ethics is undoubtedly important, and in a perfect world would form part of any education. But university degrees are finite. We argue that data and statistical literacy is an even more pressing concern, and could help guard against the appearance of more “unethical algorithms” in the future.

For more insights like this, visit our website at www.international-maths-challenge.com.
Credit of the article given to Lewis Mitchell, Joshua Ross

Credit: Getty Images

Processing high-level math concepts uses the same neural networks as the basic math skills a child is born with

Alan Turing, Albert Einstein, Stephen Hawking, John Nash—these “beautiful” minds never fail to enchant the public, but they also remain somewhat elusive. How do some people progress from being able to perform basic arithmetic to grasping advanced mathematical concepts and thinking at levels of abstraction that baffle the rest of the population? Neuroscience has now begun to pin down whether the brain of a math wiz somehow takes conceptual thinking to another level.

Specifically, scientists have long debated whether the basis of high-level mathematical thought is tied to the brain’s language-processing centers—that thinking at such a level of abstraction requires linguistic representation and an understanding of syntax—or to independent regions associated with number and spatial reasoning. In a study published this week in Proceedings of the National Academy of Sciences, a pair of researchers at the INSERM–CEA Cognitive Neuroimaging Unit in France reported that the brain areas involved in math are different from those engaged in equally complex nonmathematical thinking.

The team used functional magnetic resonance imaging (fMRI) to scan the brains of 15 professional mathematicians and 15 nonmathematicians of the same academic standing. While in the scanner the subjects listened to a series of 72 high-level mathematical statements, divided evenly among algebra, analysis, geometry and topology, as well as 18 high-level nonmathematical (mostly historical) statements. They had four seconds to reflect on each proposition and determine whether it was true, false or meaningless.

The researchers found that in the mathematicians only, listening to math-related statements activated a network involving bilateral intraparietal, dorsal prefrontal, and inferior temporal regions of the brain. This circuitry is usually not associated with areas involved in language processing and semantics, which were activated in both mathematicians and nonmathematicians when they were presented with the nonmathematical statements. “On the contrary,” says study co-author and graduate student Marie Amalric, “our results show that high-level mathematical reflection recycles brain regions associated with an evolutionarily ancient knowledge of number and space.”

Previous research has found that these nonlinguistic areas are active when performing rudimentary arithmetic calculations and even simply seeing numbers on a page, suggesting a link between advanced and basic mathematical thinking. In fact, co-author Stanislas Dehaene, director of the Cognitive Neuroimaging Unit and experimental psychologist, has studied how humans (and even some animal species) are born with an intuitive sense of numbers—of quantity and arithmetic manipulation—closely related to spatial representation. How the connection between a hardwired “number sense” and higher-level math is formed, however, remains unknown. This work raises the intriguing question of whether an innate capability to recognize different quantities—that two pieces of fruit are greater than one—is the biological foundation on which can be built the capacity to master group theory. “It would be interesting to investigate the causal chain between lower-level and higher-level mathematical competency,” says Daniel Ansari, a cognitive neuroscientist at the University of Western Ontario who did not participate in the study. “Most of us master basic arithmetic, so we’re already recruiting these brain regions, but only a fraction of us go on to do high-level math. We don’t yet know whether becoming a mathematical expert changes the way you do arithmetic or whether learning arithmetic lays out the foundation for acquiring higher-level mathematical concepts.”

Ansari suggests that a training study, in which nonmathematicians are taught advanced mathematical concepts, could provide a better understanding of these connections and how they form. Moreover, achieving expertise in mathematics may affect neuronal circuitry in other ways. Amalric’s study found that mathematicians had reduced activity in the visual areas of the brain involved in facial processing. This could mean that the neural resources required to grasp and work with certain math concepts may undercut—or “use up”—some of the brain’s other capacities. Although additional studies are needed to determine whether mathematicians actually do process faces differently, the researchers hope to gain further insight into the effects that expertise has on how the brain is organized.

“We can start to investigate where exceptional abilities come from, and the neurobiological correlates of such high-level expertise,” Ansari says. “I just think it’s great that we now have the capability to use brain imaging to answer these deep questions about the complexity of human abilities.”

For more insights like this, visit our website at www.international-maths-challenge.com.

Credit of the article given to  Jordana Cepelewicz