Intelligent recognition: the case of xThink pen-based math

xThink intelligent recognition software empowers learning and discovery for students. To achieve this goal, the software radically reduces complexity for the end user.

For example, xThink’s first products — 2003 and 2004 — provided pen-based math, eliminating the need for keyboard and mouse. This blog entry reveals the technicalities behind this first expression of xThink intelligent recognition technology (IRT). Maybe this information will satisfy the inner geek of some MathJournal enthusiasts out there.

Note:   Keep in mind that math is only one of the places where IRT brings benefits. IRT adapts to other types of recognition, too, such as recognition of flow charts, graphical programming languages, schematics, and images. xThink’s math products exploit only a small facet of what IRT can do.

Consider the following handwritten math expression:

IRT recognizes it as follows:

1. Recognize each individual symbol in the math expression, with reference to xThink’s massive database of recognized symbols.
2. Recognize the 2D spatial relationships of the symbols as a mathematical graph, which is a set of nodes and connectors.

The graph organizes the information as follows:

• Nodes represent the symbols and their relationships.
• Edges connect nodes.
• In complex expressions, the edges can carry semantic details about the expression.

Note: Very complex expressions, can be represented and analyzed as hypergraphs (overlapping sets of nodes and connectors).

(The reader must not confuse a mathematical graph with the 3D graphs used in geometry. They are distinct.)

3. Create an adjacency matrix for the graph. Unlike a graph, the matrix format is easy to process mathematically. Use mathematical rules to simplify the matrix, without loss of detail. The following rules simplify a specific adjacency matrix.

4. Transfer the simplifications of the matrix back to the graph to create the simplified, unambiguous essence of the original graph. If a basic ambiguity remains, apply a minimum spanning tree to resolve ambiguity.

5. Recognize the specific type of mathematical problem.

Note: In contrast, this type of recognition is not available in scientific calculators. Instead, you must configure the calculator to process your inputs in a specific way. For example, you must take steps to configure a graphing calculator to solve an algebra problem or statistics problem or plotting problem or integration problem, and so on.

6. Apply the appropriate math and display algorithms to generate the solution that the user requests.
7. Present all available solution options to the user, such as numerical, zeroes, extrema, plots, and graphs.

Pen-based math was only the beginning for xThink’s IRT. xThink Intelligent Tutor is the newest expression of xThink’s IRT. (More information on Tutor is here: xThink Videos, January 2011 blog entry) This new software exists in prototype form, and it is quite different from the IRT for pen-based math. However, the Tutor follows the general xThink mission: Empower learning and discovery; hide the technology.