Knowm First to Deliver Configurable Artificial Neural Networks using Bi-Directional Learning Memristors
By Ed Korczynski, Sr. Technical Editor
Knowm Inc., a start-up pioneering next-generation advanced computing architectures and technology, today announced the availability of artificial neural-network (ANN) chips built using memristors with bi-directional incremental learning capability. “We have been dreaming about this device and developing the theory for how to apply them to best maximize their potential for more than a decade,” said Alex Nugent, CEO and co-founder of Knowm. “The problem I set out to solve in 2001 was the massive discrepancy between how computers model brains and how neurons function. This result is truly a monumental technical milestone.”
Memristors with the bi-directional incremental resistance change property are the foundation for developing ANN such as Knowm’s recently announced Thermodynamic RAM (kT-RAM). Intended for high computing power jobs like machine learning (ML), autonomous robotics, and intelligent internet assistants, kT-RAM radically increases the efficiency of synaptic integration and adaptation operations such as pattern recognition and inference. The company has released an API for organizations and individual developers.
The Figure shows how Knowm’s architecture is adaptive, and based on the principle of Anti-Hebbian and Hebbian (AHaH) learning in neurons—following Hebb’s famous observation that “neurons that fire together wire together.” Hebbian learning reduces the synaptic resistance, while anti-Hebbian learning increases the resistance. The adaptive architecture means that thousands of memristors could be connected in parallel to do large-scale pattern recognition, or individual memristors could be mapped into a decision-tree to produce combinatorial optimization.
The most famous ANN chip had been True North, but IBM could not develop memristors technology so that chip uses a fixed architecture with digital SRAM transistor arrays. The use of SRAM arrays means that True North chips require 21 pJ energy per synaptic integration, while Knowm’s memristor arrays can perform the same function with less than a thousandth of the energy (1-10 fJ).
Since the principle of Hebbian-learning is well known, many R&D teams around the world have tried and failed to find a material stack that allows for controlled incremental increase and decrease in resistance. An ideal memristor with the following properties would allow a single 2-terminal device to provide both Anti-Hebbian and Hebbian learning in artificial synapses:
- BEOL CMOS compatible fabrication,
- Voltage dependent and low voltage thresholds of adaptation,
- Non-Volatile with high cycle endurance,
- Resistance ranges from ~100kΩ to ~100MΩ, and
- Bi-directional incremental changes in resistance.
Fortunately, Dr. Kris Campbell of Boise State University had been researching the electronic properties of chalcogenide compounds, and her group was able to find the right material. Working with Knowm on this patent-pending technology, Campbell can create memristors that adjust resistance in incremental steps in both directions, instead of being limited to incremental change in only one direction alternating with a non-incremental “re-set” step. Using voltage pulses nominally 100 ns allows for learning (though shorter pulse lengths also work) using 0.6V to decrease resistance or -0.3V to increase resistance, and then a 20 mV pulse can easily read the learning level. The memristor stack of materials—including a silver layer as source of ions to diffuse through and alter the resistance of the calchogenide layer (still secret)—is only a few tens of nanometers thick, and can be formed in a single physical-vapor deposition (PVD) chamber in a time-scale of minutes.
Knowm has cycled these memristors billions of times, so reliability has been shown and the company is confident that it now has the building blocks in place for the creation of powerful and efficient ANN chips. “This is a low-level resource for adaptive learning,” explained Nugent to Solid State Technology. “It’s important to say that we’re not trying to do what others are doing with digital non-volatile memory. What we started out to do is to use memristors as synaptic connectors.” Earlier this year, Knowm announced the commercial availability of the first kT-RAM products: discrete memristor chips, a Back End of Line (BEOL) CMOS+memristor service at Boise State, and the first “Knowm Anomaly” application.
Non-Volatile Memory (NVM) using memristors in cross-point arrays for digital Resistance RAM (ReRAM) has been pursued by many companies for many years. While there are inherent specification differences between digital NVM and analog ANN, in general it is more difficult to meet the device requirements for ANN. “In terms of the NVM, I feel pretty good about it already,” said Campbell to Solid State Technology. “Work I had been doing with Air Force Research Lab starting in about 2008 had been studying NVM and cross-point arrays.”