Statistics: A Dying Art

AI is making traditional statistics irrelevant.

This is a statement I have been hearing and reading about. The premise is that AI models perform so well at predictive analytics that the traditional statistical models are only still used because it is what people know and that statistics has no real value in the modern world of AI driven analytics.

I disagree...

Fundamental statistical theory is and will continue to be a useful tool for data work. The basic models and all of their assumptions and shortcomings will continue to help people learn about and understand the data they work with. This is because these models are relatively simple mathematical abstractions of real probabilities. When our mathematical assumptions differ from reality this can be observed and understood through the models we apply. With the tools of statistics we can test hypotheses and build predictive models that reinforce how we look at and interpret the data. Are we always right? Of course not! (Lies, damn lies, and statistics! If only Mark Twain new about the coming of AI)

The analogy that I like to draw here is to compare a CNC machine to a modest chisel. The CNC (computer numerical control) machine is sort of the opposite of a 3D printer, it cuts material down very precisely following digital instructions. In theory it can make just about any 3D object and is popular for manufacturing wood products (there are also metal machining versions that operate with the same principles). The chisel on the other hand is about the simplest of tools; it's made of steel and one end is sharp. A skilled craftsman can use a chisel to make just about anything with a combination of precision, the proper application of force, and patience. Guess which tool is in more workshops around the world?

The CNC machines do not make chisels obsolete. I'm not even sure how much they compete with one another in the end. People reach for the tool that suits their needs and skills. AI and statistics, I think, share the same fate. Both will continue to be valuable in data analytics. Both will drive exploration and insights. The careful application of the modest tools of statistics will always have their place in my shop.