Debugging Artificial Intelligence

Computer programs often contain errors. They are nicknamed bugs because many years ago, a moth flew into computer pioneer Grace Hooper's hardware and her program produced an incorrect result. Insects aside, why do software bugs happen? It's because computer programming is only as good as the logic a programmer envisions in his mind.

Let's take an example. This is fictitious and is only used to illustrate a point:

• Let's say you're a programmer, and I am the customer.
• I ask you to write a program that tells me if a number is positive or negative.
• So you write some code that looks like:
  if x > 0
  then print ("The number is positive.");
  else print ("The number is negative.");
• You test the program using some numbers like -1 and 1 and get the correct results. You are happy. You release the program.
• Lots of people use your program, and everyone is getting correct results. Everyone is happy.
• Along comes a user who tries the number 0 and gets "The number is negative." as a result. This is an error. He is not happy. He reports a bug.
• You get the bug report and fix your code.
  if x = 0
  then print ("The number is neither positive nor negative.");
  else if x > 0
       then print ("The number is positive.");
       else print ("The number is negative.");
• You retest the program using numbers -1, 0, and 1 and get the correct results. You are happy. You release the update.
• Lots of people use your updated program, and everyone is getting correct results. Everyone is happy.

This is an overly simplistic example but makes a point. The customer didn't think to specify the zero case as in "Tell me if a number is positive, zero, or negative." and the programmer didn't think of "What if x equals 0?" either. Though simple, that's how these things happen in real programming but on a grander and more complicated scale. Programmers can't test every possible input to their programs, so they have to break up the problem space into "equivalence classes" and test one value from each of the classes to represent the entire class. Sometimes they get the classes wrong because there is more data to consider.

Normally a programmer makes sure that his code is correct by using a debugger. A debugging tool allows a programmer to step through each line of code and see what the values of the variables in his program are. He can trace through his code and identify exactly where his program goes wrong; however, there are some cases where the code is part of an embedded system and using a debugger is not practical. In the olden days, programmers would insert print statements into the code, execute the code, and look at the printed output to see is happening.

In our fictitious example, a programmer may add:

• print ("entering my code and the value of x is ", x);
  if x = 0
  then print ("The number is neither positive nor negative.");
  else if x > 0
       then print ("The number is positive.");
       else print ("The number is negative.");
  print ("exiting my code");

When I was worked at Ithaca Software, my mentor was a guy named Milt Capsimalis. As Milt likes to say, "I taught Scott everything he knows about 3D graphics, but I didn't teach Scott everything that I know about 3D graphics." One day Milt was working on a graphics driver that had multiple buffers for displaying pixels on the screen. The graphics card used two buffers to increase device throughput. Milt and I would routinely put in print statements into our code so that we could see what our driver was doing while we watched pixels being colored on a separate display screen. In the case of this device, when Milt used print statements, the extra code to do the printing threw off the timing of the buffering, and the display was a total mess. So Milt got the idea to add beeps, as in sound, to mark different spots in the code. The beeps took less time to process than printing the debugging text. He had to carefully listen as in "Was that one beep or two beeps?" to tell what part of his driver code was being used to color the pixels on the display. Milt is brilliant like that. I am not brilliant but systematic. We made a great team.

Programming has bugs because it is a human activity where programmers embody rules of program operation based on their understanding of the data that their code will be receiving. Sometimes they get it wrong. Machine learning is a machine activity where machines embody rules of operation based on their analysis of data that they have received. In other words, humans programmers used to make the rules and then get the data. For machine learning, machines get the data and then make the rules. So how does one debug the rules that were made by the machine?

In the past, I have worked with a guy named Tom Wujec. As Tom likes to say, "I taught Scott everything he knows about visual thinking, but I didn't teach Scott everything that I know about visual thinking." Actually, Tom does not say that, but he could.

Recently, Tom sent me a link to this article:

Read the article.

Tom noted that:

• In 2018, the European Union will begin enforcing a law requiring that any decision made by a machine be readily explainable.
• Even if a machine made perfect decisions, a human would still have to take responsibility for them.
• Is the deep neural network really seeing a world that corresponds to our own?

For me, the article brings up some interesting points:

• Researchers are developing a tool that is basically an ingenious way of testing a deep neural network.
• Using the tool is the key to unlock what a deep neural network is doing because it represents a new kind of science where machines are accessing truths that lay beyond human intuition.
• This is vital because understanding Artificial Intelligence (AI) requires reducing what a computer knows into a single conclusion that humans can grasp and consider.

Based on my experience, the essence is that society wants AI to have print statements in AI code so humans can see what's going on. We have to carefully listen to calls for caution regarding code generated for the machine by the machine. Beep. Beep.

Debugging is alive in the lab.