Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and... fnins-15-661261 June 26, 2021 Time: 19:12 # 1 ORIGINAL RESEARCH published: 02 July 2021 doi: 10.3389/fnins.2021.661261 Exploring Area-Dependent Pr Ca MnO -Based Memristive 0:7 0:3 3 Devices as Synapses in Spiking and Artificial Neural Networks Alexander Gutsche , Sebastian Siegel, Jinchao Zhang, Sebastian Hambsch and Regina Dittmann Peter Grünberg Institut (PGI-7/10), Forschungszentrum Jülich GmbH & JARA-FIT, Jülich, Germany Memristive devices are novel electronic devices, which resistance can be tuned by an external voltage in a non-volatile way. Due to their analog resistive switching behavior, they are considered to emulate the behavior of synapses in neuronal networks. In this work, we investigate memristive devices based on the field-driven redox process between the p-conducting Pr Ca MnO (PCMO) and different tunnel barriers, 0:7 0:3 3 namely, Al O , Ta O , and WO . In contrast to the more common filamentary-type 2 3 2 3 switching devices, the resistance range of these area-dependent switching devices can be adapted to the requirements of the surrounding circuit. We investigate the impact of the tunnel barrier layer on the switching performance including area scaling of the Edited by: Sabina Spiga, current and variability. Best performance with respect to the resistance window and National Research Council (CNR), Italy the variability is observed for PCMO with a native Al O tunnel oxide. For all different 2 3 Reviewed by: layer stacks, we demonstrate a spike timing dependent plasticity like behavior of the Martin Ziegler, investigated PCMO cells. Furthermore, we can also tune the resistance in an analog Technische Universität Ilmenau, Germany fashion by repeated switching the device with voltage pulses of the same amplitude Brian Douglas Hoskins, and polarity. Both measurements resemble the plasticity of biological synapses. We National Institute of Standards and Technology (NIST), United States investigate in detail the impact of different pulse heights and pulse lengths on the shape *Correspondence: of the stepwise SET and RESET curves. We use these measurements as input for the Alexander Gutsche simulation of training and inference in a multilayer perceptron for pattern recognition, to a.gutsche@fz-juelich.de show the use of PCMO-based ReRAM devices as weights in artificial neural networks Specialty section: which are trained by gradient descent methods. Based on this, we identify certain trends This article was submitted to for the impact of the applied voltages and pulse length on the resulting shape of the Neuromorphic Engineering, measured curves and on the learning rate and accuracy of the multilayer perceptron. a section of the journal Frontiers in Neuroscience Keywords: PCMO, memristive devices, perceptron learning, resistive switching, multilevel switching Received: 30 January 2021 Accepted: 21 May 2021 Published: 02 July 2021 INTRODUCTION Citation: Gutsche A, Siegel S, Zhang J, Most modern computer architectures are based on the von Neumann principle, which separates Hambsch S and Dittmann R (2021) the data processing unit from the data storage. As the performance of processors increased strongly Exploring Area-Dependent over the last decades, the bandwidth for the communication between processor and data storage Pr Ca MnO -Based Memristive 0:7 0:3 3 became the limiting factor for the overall computational performance. This is called the von Devices as Synapses in Spiking Neumann bottleneck (Backus, 1978) (Wolf and McKee, 1994). and Artificial Neural Networks. The limit is especially problematic for tasks, where simple operations are performed on large Front. Neurosci. 15:661261. doi: 10.3389/fnins.2021.661261 sets of data, e.g., learning tasks in massively parallel systems mimicking brain-like functionalities Frontiers in Neuroscience | www.frontiersin.org 1 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 2 Gutsche et al. Neuronal Networks With PCMO Devices or vector-matrix multiplications in artificial neural networks over the whole device area (Herpers, 2014; Bagdzevicius et al., (ANNs) during the inference step. The multiplication of the input 2017). Since the current of the area-type switching devices nodes of a layer with the weight matrix yields the output of scales for both the high resistive state (HRS) and the low this layer. A possible strategy to overcome this von Neumann resistive state (LRS) with the device area, the resistance bottleneck for ANNs is the usage of resistive arrays as weight values can be adapted to the given circuit requirements. matrices (Xia and Yang, 2019). To achieve tunable weights, This is not the case for the most common filamentary-type one approach is the so-called memristive device, an electrically memristive devices. Moreover, filamentary-type switching is tunable resistor. Previous works already show that memristive usually indicated by a sharp SET process. In contrast, area- crossbar arrays allow for efficient vector-matrix multiplication type switching devices exhibit a gradual SET and RESET (Cai et al., 2019). The use in ANNs was demonstrated on many that enhances their ability for analog switching in comparison network types such as single-layer perceptrons (Alibart et al., with filamentary memristive devices. Due to their analog 2013; Prezioso et al., 2015) as well as multilayer perceptrons switching behavior, PCMO-based resistive switching devices are (Moon et al., 2015; Burr et al., 2017; Babu et al., 2018; Go et al., considered hardware representation for synapses in artificial 2019; Wu et al., 2020) and convolutional neural networks (CNNs) neural networks as described above. In particular, it has been (Yakopcic et al., 2017). Many groups show that memristive shown that they can emulate aspects of synaptic plasticity (Park devices can already today replace conventional networks trained et al., 2012, 2013, 2015; Moon et al., 2014; Fumarola et al., in software for many applications. Li et al. report a recognition 2018). accuracy of more than 97% on the MNIST dataset, which is In this work, we compare in detail the performance and common for benchmarking of pattern recognition tasks. Also, analog behavior of PCMO-based devices with different interface more complex tasks like face recognition have been demonstrated configurations. In particular, we compare the more common (Yao et al., 2020). These similar network performances are Al/PCMO devices with a natively formed Al O oxide to 2 3 often achieved at higher-energy efficiencies and make memristive devices with a directly sputtered Ta O and WO as interface 2 5 3 device-based ANNs most useful for low-energy applications layer. For all devices, we can demonstrate analog switching at the edge and in the IoT sector (Chowdhury et al., 2018) behavior. We demonstrate a STDP-like behavior on single PCMO (Krestinskaya et al., 2020). A large variety of different types of devices. This learning rule for spiking neural networks (SNN) memristive devices have been proposed for neuronal networks stems from neuroscience and neurophysiology. Furthermore, so far in the literature mimicking behavior of biological synapses we investigated in detail the impact of the material stack as like, e.g., long-term potentiation and depression (LTP/LTD) and well as pulse length and height on the shape of the analog even more complex aspects of synaptic plasticity like simple stepwise SET and RESET curves. This stepwise change of forms of spike timing dependent plasticity (STDP), but no conductance mimics aspects of LTP/LTD of biological synapses. optimal memristive device type has been identified yet. For a We use the experimental data as input for simulations of the given choice of materials, the ANN, the learning rule and the training of a multilayer perceptron for pattern recognition and update rule have to be adjusted to obtain best performance. In reveal how the different electrical stimuli and the resulting this work, we propose an update rule for a specific memristive shapes of the stepwise SET and RESET measurement (SPM device based on Pr Ca MnO (PCMO) after a thorough and RPM) curves affect the learning rate and the accuracy 0:7 0:3 3 investigation of its switching behavior and the influence of of the network based on a gradient descent learning rule, different material stacks. which is a learning rule for conventional ANNs. Comparing In memristive devices, information is stored by the change in STDP and gradient decent methods, STDP only requires local the resistance that can be switched by an applied bias in a non- information processing between the two neurons adjacent to volatile manner. Different mechanisms and materials that show the very synapse, while gradient descent methods take the resistive switching have been reported in literature (Simmons global error of the complete network into account. Here, we and Verderber, 1967; Asamitsu et al., 1997; Sawa, 2006, 2008; present how both learning rules can be achieved with the same Tsymbal and Kohlsted, 2006; Jooss et al., 2007; Waser et al., 2009; memristive device. Herpers, 2014). In this work, we will address the mixed valence manganite (PCMO) in combination with a tunnel oxide that has EXPERIMENTAL been either deposited directly by physical vapor deposition or that has been formed by the redox process with an oxidisable metal top electrode. Combinations of PCMO with many different Sample Preparation metals are reported in literature so far: e.g., Al (Seong et al., The memristive devices consist of a 25-nm-thick Pt bottom 2009a), Ta (Seong et al., 2009b), Ti (Seong et al., 2009b), W (Liu electrode, a 20-nm PCMO film grown by pulsed laser deposition et al., 2011), and others (Moon et al., 2014, 2015; Baek et al., 2017; (PLD), a 7-nm-thick interface layer, either Al, Ta O or WO 2 5 3 Go et al., 2019). It is proposed that the field-driven movement of and a 25-nm-thick Pt top electrode as sketched in the insets of oxygen anions between the PCMO layer and the reactive metal Figures 1A–C. The Pt layer that serves as bottom electrode is electrode is the underlying switching mechanism (Sawa et al., DC sputtered on top of a 5 nm Ta adhesion layer on a thermally 2004; Asanuma et al., 2009; Seong et al., 2009a). oxidized Si wafer. PCMO is known for its area-type resistive switching The PLD growth of PCMO is performed with an O pressure properties, namely that the change of the resistance happens of 0.133 mbar at room temperature (RT). A laser fluence of Frontiers in Neuroscience | www.frontiersin.org 2 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 3 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 1 | (A–C) I-V curves for three different materials, (B) Al/Al O , (C) Ta O , and (D) WO , like it is indicated by the layer stack in the lower right corner. The 2 3 2 5 3 voltage is applied on the top electrode. For every device, three different sizes—100 100, 50 50, and 20 20 mm —are measured. The red dotted line indicates the read out voltage for the area scaling. The switching voltages differ between the three stacks: (A) Al O : 2.2 V/–3 V, (B) Ta O :  2.5 V, and (C) WO :  2 V. 2 3 2 5 3 (D–F) Area dependence of the LRS and HRS. The resistance value is scaling with the device size, for all of the three materials, (D) Al O , (E) Ta O , and (F) WO . 2 3 2 5 3 The slopes of the linear fit for all of the devices and the HRS and LRS are around –1 /mm . Slopes can be seen in Table 2. TABLE 1 | Voltages for the conducted measurements. Material Set voltage (V) Reset voltage (V) Read-out voltage (V) SPM voltages (V) RPM voltages (V) Read-out (V) Al/Al O 3.0 2.2 0.5 1.5 to 2.0 1.2 to 2.0 0.3 2 3 Ta O 2.5 2.5 1.0 to 1.5 0.8 to 2.0 2 5 WO 2.0 2.0 2.0 to 2.6 1.2 to 2.0 Electrical Measurements 1.33 J/cm and a frequency of 5 Hz are used during PLD growth. Around 2,800 pulses are needed to grow a 20-nm In preparation of the electrical measurements, the samples are amorphous PCMO layer. Afterward, the PCMO thin film is glued to a large sample carrier chip with Pt pads. The BE is annealed in N atmosphere at 650 C for 2 min in order to contacted to one of the Pt pads on the sample carrier using crystallize the PCMO layer. aluminum wire bonding. Two different setups are used to The Ta O and WO layers are deposited by RF sputtering at characterize the samples electrically, namely one to perform the 2 5 3 RT. Both depositions are performed at 200 W with 5 10 mbar quasi-static current-voltage (I-V) measurements, the other one pressure and an Ar/O ratio of 3/2. Afterward, the sample is to apply pulses to the devices. A Keithley 2611B is used to transferred in situ into an e-beam evaporator to deposit the Pt measure the I-V characteristics of the devices. The connection top layer which is used as top electrode. During the Pt deposition between the measurement unit and the device is performed in vacuum, the e-beam process heats the sample up to 180 C. For by soft tungsten needles. Every measurement starts with an the Al device stack, a 7-nm layer is also deposited on top of the initialization curve: 0 V!2.5 V!–2.5 V!0 V. During this PCMO layer by e-beam evaporation and capped in situ with the initialization procedure, the oxide layers of the metal are Pt layer. During the short deposition of the Al layer, no significant presumably homogenized (Arndt et al., 2017). Afterward, the increase in temperature can be detected. Here also, a 25-nm Pt regular switching cycle can be performed: 0 V!RESET voltage capping layer is used. (positive)!SET voltage (negative)!0 V. The SET and RESET The patterning of the top electrode and the active interface voltages have to be adapted for the different interface layer layer of the devices is performed by optical lithography and materials. In Table 1, the writing voltages that show the most Ar ion-beam etching. The pad size varies between 100  100, stable switching for the different interface layers can be found 50 50, 20 20, and 10 10 mm . along with the read voltage. Frontiers in Neuroscience | www.frontiersin.org 3 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 4 Gutsche et al. Neuronal Networks With PCMO Devices The pulse measurements for the multilevel SPMs and RPMs with voltage in the negative branch is much higher, compared are performed with a Keithley 4200A. Different pulse lengths with Figures 1A,B. For the WO devices, a stable switching curve between 1 and 100 ms are employed. A large variety of can be found with symmetric switching voltages at  2 V. At combinations of SPM and RPM voltages are investigated. The around –1.8 V, a change in the slope can be seen at least for parameter ranges that are used for the different devices are the 50 mm  50 mm and the 100 mm  100 mm devices. For displayed in Table 1. the 20 mm  20 mm devices a similar change in slope can be The STDP measurements are performed with an Arc One surmised, but not clearly determined. from Arc Instruments. For all three device types, a pulse length Each device state, HRS and LRS, for the Al, Ta O , and 2 5 of 100 ms with a pulse voltage of 2 V/–2 V is investigated. WO interface devices are tested regarding their retention time. Over a period of several days, no change in the states can be determined. The samples are stored at room temperature and in ELECTRICAL CHARACTERISATION OF ambient atmosphere. To prove that all of the devices show area type resistive PCMO MEMRISTIVE DEVICES switching, we read out the resistance at –0.5 V since switching effects can be excluded at this voltage and the resistance at this Quasi Static I-V Measurements voltage is plotted against the device area (see Figures 1D–F). The In Figure 1A, the I-V measurements of a typical sample with read out is chosen to be on the negative branch due to a higher an Al interface layer can be seen. A clear hysteresis of the I-V ON/OFF ratio. A clear linear relationship between the device curve on both the positive and the negative branch is visible. The resistance and the device area can be seen for the HRS and the SET takes place at negative voltages and the RESET at positive LRS for all of the devices with a slope around 1 /mm , as voltages. For negative applied voltages, the difference between the expected by Ohm’s law. The exact values of the fitted slopes can LRS and the HRS, called ON/OFF-ratio is higher. Concerning be found in Table 2. the gradual switching of the area type switching devices, no Additionally, we studied the device-to-device (d2d) and distinct SET or RESET voltage can be defined. Therefore, we cycle-to-cycle (c2c) variability of the devices during the quasi- always choose a voltage pair that allows stable switching of the static I-V measurement. For these measurements, we used the devices without any change of the I-V curves during the repeated 20 mm 20 mm devices. Figures 2A–C shows the combined c2c switching, e.g., 2.2 and 3 V for RESET and SET, respectively, and d2d Weibull distribution for the different devices, namely, in case of the Al devices. For simplicity reasons we will call (A) Al/Al O , (B) Ta O and (C) WO . For the Al/Al O 2 3 2 5 3 2 3 the maximum voltage in the different voltage directions SET interface layer, it can be seen that the HRS and LRS are clearly and RESET. During the RESET, the slope changes at 1.8 V. separable over their whole resistance range. The spread of the A similar but smaller change in slope can be seen during the HRS and the LRS is half an order of magnitude. For the Ta O 2 5 RESET at 2 V. Furthermore, the RESET and SET are both interface layer (Figure 2B), the spread for the different devices gradual, with no abrupt jumps into the HRS or LRS. The I-V and cycles is smaller. However, due to the smaller ON/OFF ratio, curves for different pad sizes all have the same shape with smaller the overlap of the two states is around a few percent. For the WO differences, like the opening on the positive side. For smaller devices, the variability plot (see Figure 2C) differs from the plot of devices, the opening becomes smaller in the positive branch. This the other devices. It can be seen that the LRS shows a much higher effect is not observed for the negative branch. variability than the HRS. The variability of the HRS is as small The I-V curves of a device with a Ta O interface layer are 2 5 as for all of the other device types. Comparing all three device shown in Figure 1B. This switching polarity is the same as for the stacks, the Al/Al O devices show a higher ON/OFF ratio than 2 3 Al devices, and HRS and LRS are clearly separable on the negative the Ta O devices and a lower variability than the WO devices. 2 5 3 side. Also, a change in the slope of the I-V curve can be found around –2 V. On the positive branch, no opening and no change Spike Timing Dependent Plasticity in slope can be seen. In STDP, the change of a synaptic weight between neurons The WO devices show a different shape of the I-V curve depends on the time difference between two spikes, the pre- compared with the Al and Ta O devices. The positive and the 2 5 and post-synaptic neuron pulse. The memristive devices act as negative branches both show two clearly separable resistive states, synapses, and the pre- and post-synaptic pulse are applied at the see Figure 1C. In contrast to the case of Al and Ta O devices, 2 5 top/bottom electrode, respectively. the I-V curves are very symmetric for positive and negative Figure 3 shows the relative change in conductance of the polarities. In particular, the increase in current in the LRS state three different memristive devices for different time delays between the pre- and post-synaptic pulse. All devices show an increase/decrease in conductance for a negative/positive time TABLE 2 | Slope of the linear fit of the resistance vs. area plot for all the delay between the pulses, respectively. The Al STDP curve different materials. (Figure 3A) shows a symmetric increase or decrease of the 2 2 Material HRS (/mm ) LRS (/mm ) conductance for the time delay between the pulses compared with the STDP curves of the Ta O and the WO (Figures 3B,C). Al/Al O 1.09  0.01 1.12  0.07 2 3 2 5 3 Ta O 0.98  0.18 0.95  0.05 The WO (Figure 3C) shows a clear asymmetry between the 2 5 3 WO 1.25  0.18 1.02  0.22 increase and decrease of conductance. The maximum increase in Frontiers in Neuroscience | www.frontiersin.org 4 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 5 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 2 | (A–C) The Weibull plot of the combined cycle to cycle variability (c2c) and device-to-device variability (d2d) for the three different stacks, (A) Al O , 2 3 (B) Ta O , and (C) WO , indicated by the stack in the lower right corner. For each stack, 10 different devices with each up to 100 cycles have been investigated. 2 5 3 FIGURE 3 | Relative change of conductance for different time delays between the pre- and post-synaptic pulse during STDP measurement for the three different devices. (A) Al/Al O , (B) Ta O , and (C) WO . 2 3 2 5 3 conductance is around twice as high as the decrease. Therefore, For the positive voltage curves, the resistance saturates after all three types of devices are suitable for the implementation in 20 pulses for both voltages but with different saturation SNNs based on the STDP learning rule. level, a higher/lower resistance for the higher/smaller voltage, respectively. Furthermore, the increase in resistance at the beginning of the curve is higher with a higher pulse voltage and Stepwise SET and RESET Pulse smaller with smaller pulse voltage. After the steep increase in the beginning, the resistance only slightly increases. The SPM curves Measurements that are measured with positive pulse voltages saturate after10 We perform stepwise SET and RESET pulse measurements by pulses. Both curves show a clear non-linear behavior. applying the same voltage pulse multiple times to one device without switching the device back into a predefined state. By The SPM and RPM curves of the Ta O device are depicted 2 5 in Figure 4B. The RPM curve is shown for two different pulse applying pulses with a lower voltage, compared with the voltages used during the IV measurement, it is possible to tune the voltages, namely, 0.8 and 1.4 V, each with 100 ms pulse length. The SPM pulses have a height of –1.0 or –1.5 V, also with a resistance of the devices in a gradual way between the HRS and the LRS and vice versa. The transition from the HRS to the LRS in pulse length of 100 ms. Again, both the RPM and SPM curve characteristics are non-linear. The SPM curves saturate after10 the SPM and the transition from the LRS to the HRS in the RPM pulses, similar to the Al/Al O , but the maximum resistance happen stepwise. With these measurements, we can show that it 2 3 reached is different. The larger negative voltage leads to a lower is possible to write different resistance states into the investigated resistance value, compared with the smaller negative voltages. devices, resembling the LTP/LTD behavior of biological synapses. For the RPM curves, the resistance increases less with each pulse In Figure 4A, the SPM and RPM measurements of PCMO for the 0.8 V pulses as for the 1.4 V pulses. Furthermore, the with the Al/Al O interlayer are depicted. The chosen voltage 2 3 obtained saturation resistance is also smaller and therefore the for the SPM and RPM are 1.8 V/2.0 V and –1.5 V/ 2.0V, ON/OFF ratio is smaller. Beside the smaller ON/OFF ratio, the respectively, at a pulse length of 100 ms. Every pulse was applied 50 times without going back to the initial state. The largest curve shows a more linear increase in resistance during the pulse measurement with the smaller SET voltage compared with the resistance change for the SPMs and RPMs of the Al/Al O 2 3 interface device occurs during the first few pulses of a cycle. larger SET voltage. Frontiers in Neuroscience | www.frontiersin.org 5 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 6 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 4 | Results of the SPM and RPM for the different devices. The RPM curve (red) and the SPM curve are plotted for the same material into the same coordinate system. The SPM curve starts at higher pulse numbers and goes down to lower pulse numbers; the RPM curve can be read as normal. The pulse voltages for the different stacks can also be seen in Table 1. (A) Al O , (B) Ta O , and (C) WO . Every pulse has a length of 100 ms. For the shown curves, the fits 2 3 2 5 3 are shown in the plots. In Figure 4C, measurements for the WO stack with The fit function is used to determine the change of the 1.2 V/1.4 V and –2 V/–2.6 V for RPM and SPM, respectively, resistance of a memristive device upon the application of either are shown. All measurements are performed with a pulse length a SET or RESET pulse. For a SET pulse, the fit function for of 100 ms. The RPM curves of WO are more linear compared the respective SPM curve y .n/ is inverted and the current 3 SPM with the RPM curves of the Al and the Ta O devices, and no conductance (before the update pulse) of the device is used 2 5 clear saturation can be seen for the shown RPM curves. The to determine the pulse number n , which resembles this current SPM curves show a saturation after 20 pulses with a slight conductance. Following to this, y .n/ is evaluated at n C SPM current resistance decrease afterward. Here, a clear separation between 1 to yield the conductance of the device after the SET update the saturation levels for the different voltages can also be seen. pulse. For a RESET pulse, the inverse of the RPM curve’s fit function y .n/ and the current resistance yield n and RPM current y .n C 1/ gives the resistance value of the device after RPM current Behavioral Modeling of the Resistance the update pulse. Changes of the PCMO Devices Another common approach for a behavioral model in the To better analyze the impact of the material stack and applied literature is fitting the resistance change [e.g., (Suri et al., 2015)] voltages on the shape of the SPM and RPM pulse measurements instead of the actual resistance as it is proposed in this work. and to use these measurements in the ANN simulations in However, this approach showed a similar fitting accuracy for the Section “Perceptron Learning of Mnist Dataset,” the evolution data used here but a lower computational performance in the of the resistance for the devices with an Al, Ta O , and WO 2 5 3 TensorFlow environment. interlayer is mathematically fitted. Similar to other approaches in The fit parameters ý and ý correspond to the SPM RPM literature (Suri et al., 2015), a logistic function saturation value of the resistance in the SPM and RPM, respectively. Therefore, the maximum ON/OFF ratio of a pair of yK SPM and RPM curves can be calculated from these parameters. y.n/ D (1) 1C exp.a  n c/ The ON/OFF ratio strongly depends on the used material stack. With the Ta O interlayer samples, the lowest ON/OFF ratios 2 5 is employed, where y is the fitted resistance for the RPM curve can be reached, while the WO samples show the highest values and conductance for the SPM curve. ý is the maximum value at and with an Al interlayer, intermediate values can be reached. which the function saturates, and a and c determine the steepness Additionally, higher SET and RESET voltages lead to a higher of the increasing swing. In the following, parameters concerning maximum ON/OFF ratio in the pulse measurements, except for the SPM fit are equipped with the index SPM and parameters the SPM with –2.6 V for the WO interlayer samples and the 1.6- V RPM for the Ta O interlayer samples. It can be found that an concerning the RPM fit with the index RPM. This formula shows 2 5 a strong saturation for high values of the pulse number n as increase of the SET voltage leads to a lower saturation resistance in the SPM and a higher RESET voltage to a higher saturation observed in our experiments and reasonably good fitting of the measured resistance values. A fitted resistance value can therefore resistance of the RPM. For the a parameter of the fit function (1), which resembles be attributed to each measured resistance, leading to a total of 50 different resistance values for every SPM and RPM curves and the steepness of the initial change of the resistance, no clear therefore in total 100 resistance levels for every material stack and dependency on the applied voltage can be found, neither for the pair of SPM and RPM voltages. The different resistance levels are SPM nor the RPM. Concerning the a parameter for the fit SPM not evenly spaced. of the SPM, a clear separation between the different material Frontiers in Neuroscience | www.frontiersin.org 6 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 7 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 5 | (A) RPM of the Ta O devices with 1.6 V per pulse and different pulse lengths –1, 10 and 100 ms. (B) Resistance of the RPM plotted against the total 2 5 applied pulse length for three different pulse lengths with a pulse voltage of 1.6 V. (C) SPM curves of the Ta O devices with the resistance plotted against the total applied pulse length, with a pulse voltage of –2.5 V. stacks can be observed, where the Al interlayer samples show that can be used and the resistance change that one pulse triggers. the highest, Ta O intermediate and WO the lowest values. In In addition to the former described current adaptability of the 2 5 3 the fit of the RPM curves, a trend to increasing a parameters devices by their sizes, this gives the opportunity to adapt the pulse RPM with an increasing reset pulse voltage can be observed. However, length to the network requirements. the height of the increase is the largest for the Ta O interlayer 2 5 samples and comparably small for the WO interlayer samples. Discussion of the Impact of Pulse Height In conclusion for the fit parameters, in the most cases, a higher and Length on SPM/RPM SET voltage leads to a lower saturation resistance in the SPM As described in the previous section, the final ON/OFF ratio that and a higher RESET voltage to a higher saturation resistance in can be reached after saturation increases with the pulse height. the RPM. a , corresponding with the steepness of the initial SPM The tradeoff is to find a voltage that allows a high saturation increase of the SPM, depends mostly on the material and not on ON/OFF ratio but without reaching the saturation immediately, the SET voltage, while a , for the RPM steepness, depends on RPM which would lead to a binary synapse. With respect to the pulse the RESET voltage. length dependence (see Figure 5A), the reduction of the pulse length results in a smaller increase in resistance and the final Impact of Pulse Length on SET and ON/OFF ratio that can be reached after saturation increases only RESET Pulse Measurements with the pulse height and not with the pulse length. To avoid In order to study the impact of the pulse length on the shape switching the device completely with only one pulse, the voltage of the SPM and RPM curves, we vary the pulse length between and the applied pulse time has to be reduced. This way we can 1 ms and 100 ms. Figure 5A shows that using shorter pulses reach a high number of intermediate resistance states. for the RPM, the maximum reached resistance after 50 pulses In filamentary systems, the switching current is confined to the is smaller, similar to what we have observed for smaller pulse filament resulting in high current densities and self-heating up to amplitudes (see Figure 4B). Moreover, Figure 5A implies that 800 K (Menzel et al., 2011). The increase in temperature leads to a the total change of the resistance only depends on the total time to a self-enhanced, abrupt SET process. In area type switching, the of the applied pulses irrespective of the length of a single pulse. current is distributed over the whole area resulting in low current For example, for the application of a single pulse of 100 ms or 10 densities and a large dissipation area. Simulations confirm that pulses of 10 ms pulse length, the total applied time is the same self-heating is not important in area type devices, and we neglect and the observed resistance change is the same. Figure 5B shows its influence on the switching process (Menzel et al., 2019). As a the measured device resistance at –0.3 V, plotted against the total result, the velocity of the oxygen ions only depends on the force applied pulse length for the Ta O RPM curve starting in the LRS. of the applied electric field and the diffusion force. It does not 2 5 A clear trend of increasing resistance with increasing total applied depend on the length of the applied voltage pulse. pulse time can be seen. Also the measurements with the different We assume that the resistance changes with the amount of pulse lengths show a continuous behavior. For the Ta O devices, oxygen in the tunnel oxide and in the PCMO. An oxygen ion 2 5 this behavior is also depicted for the SPM curve in Figure 4C. transfers from the tunnel oxide into the PCMO (or vice versa) This proves that the total change in resistance is indeed only if it overcomes the distance to the interface between the two dependent on the total applied time for Ta O . This behavior is materials. If we apply an electric field, oxygen ions begin to move. 2 5 also observed for the Al and the WO devices (not shown here). The higher the distance each oxygen ion can travel, the more ions In summary, the combination of different pulse lengths and can move in total from the tunnel barrier to the PCMO (or vice pulse voltages makes the devices flexible in the resistance range versa) if the drift process takes place via vacancy sites. Therefore, Frontiers in Neuroscience | www.frontiersin.org 7 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 8 Gutsche et al. Neuronal Networks With PCMO Devices the change in resistance is directly related to the total distance an In order to determine whether a SET or RESET pulse must be oxygen ion has moved. applied to a ReRAM device, a form of gradient descent learning This distance is the time integral of the velocity that only using the backpropagation algorithm is employed. depends on the total time a voltage is applied as long as the In the forward pass, a sample image is presented to the input velocity itself is not a function of the pulse length. This should be of the network with the grayscale values of each pixel being the case if no Joule heating takes place. Therefore, the distance converted to an input voltage. By Ohm’s and Kirchhoff ’s Law, the oxygen can be moved does not depend on the number of this vector of input voltages is transformed to a vector of output pulses and their length. For example, one 100 ms pulse has the currents by the memristive weight matrix. A Rectified Linear Unit same effect as ten 10 ms pulses (see Figures 4B,C). (ReLU) function determines the input voltage to the next weight layer from these currents. The gradient for the gradient-descent algorithm is calculated PERCEPTRON LEARNING OF MNIST in the backward pass of the total network error E for every @E DATASET weight as . This calculation is executed within the TensorFlow @g i;j framework. The calculated gradients for every weight are The presented devices allow for usage with two different accumulated within each epoch. In every training epoch, a subset learning rules. The measurements presented in Section “Spike of 60,000 samples from the MNIST dataset is shown to the Timing Dependent Plasticity” resemble a synaptic STDP behavior network, resulting in a batch size of 60,000. Previous works by and therefore suggest the use of the proposed devices for Gao et al. (2020) showed that larger batches can lead to a better Hebbian style learning in a spiking neural network. The second recognition performance. With this large batch size, only one approach uses the stepwise resistance change in the pulse update cycle per epoch is performed. measurements presented in Section “Stepwise SET and RESET In the update operation of the PCMO devices we propose Pulse Measurements”. The gradual nature of the resistance here, either a gradual SET, RESET, or no pulse can be applied change can be exploited in an artificial neural network, which is to a device. The pulse height for SPM and RPM are fixed. trained by a gradient descent learning rule. This duality shows Therefore, only the sign of the accumulated gradient would the wide range of applications for the proposed devices in determine whether a device receives a SET or RESET pulse neuromorphic systems, as these learning rules differ significantly for update. With this sign update rule, an infinitesimal small in the scope of processed information (local comparison of gradient would have the same effect as a large gradient, what pre- and post-synaptic activity for Hebbian learning and global can be expected problematic for the training of the network. error minimisation for gradient descent) and the initial point Therefore, the set of updated conductances is restricted to only of their derivation (neurophysiology for Hebbian learning and the largest positive gradient and the smallest negative gradient in mathematical optimisation theory for gradient descent). In the every layer. Another benefit of such a very sparse update matrix following, an exemplary ANN trained by a gradient descent in a matrix-shaped weight layer is that it is much more time learning rule is shown. consuming to update a large number of conductances than to To investigate the use of PCMO resistive switching devices as infer the whole network. Using a matrix structure of the PCMO presented above as weights in an ANN, we conduct simulations devices, the inference of a complete layer takes one step, whereas of multilayer perceptrons in a TensorFlow (Abadi et al., 2016) the update of device is performed sequentially. The very low environment in Python. Furthermore, the impact of the different number of updated devices on the other hand leads to a large material stacks and the hyperparameters SET/RESET pulse number of learning epochs necessary to reach the maximum voltage and pulse length on the learning and recognition accuracy recognition accuracy. are analyzed. To compare and benchmark the results of our After each learning epoch, a validation subset of 10,000 network, the common dataset of hand-written digits MNIST samples from the MNIST dataset, which is different than the is used for training of the network and validation of the learning set is shown to the network, and the fraction of correctly recognition performance. recognized numbers is calculated as the recognition accuracy after this epoch. To investigate the impact of the PCMO-based Gradient-Descent Learning of the MNIST ReRAM devices in this network, we also performed a benchmark Dataset test with the described network structure and learning rule, but The investigated perceptron network consists of four layers of floating point weights instead of resistive switching weights. In neurons with the second and third being hidden. The input layer this benchmark, the network showed a maximum recognition has 784 neurons, the first hidden layer 250, the second hidden accuracy of 96.3%. layer 125 and the output layer 10 neurons. This structure is chosen to make the network comparable with similar memristive Results of Multilayer Perceptron networks in the literature. Similar to previous works, a matrix Simulations With PCMO-Based structure of PCMO-based ReRAM devices is assumed as the Memristive Switching Weights weight layer between two neuron layers (Xia and Yang, 2019). A more detailed description of the network can be found in the For all combinations of measured SPMs and RPMs, multilayer Supplementary Material. The weights are initialized randomly perceptron simulations are conducted for 6,000 epochs. from the range the employed SPM and RPM curves provide. As described in Section “Stepwise SET and RESET Pulse Frontiers in Neuroscience | www.frontiersin.org 8 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 9 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 6 | Results of the neural network simulations. (A) Recognition accuracy of the simulated network depending on the fitted ON/OFF ratio and the steepness of the SPM fit function. (B) Network recognition accuracy on a subset of the MNIST dataset. Red: Al interlayer. Blue: Ta O interlayer. Yellow: WO interlayer. (C) 2 5 3 Recognition accuracies of similar networks with ReRAM devices from literature depending on the ON-OFF ratio of the used ReRAM devices (Moon et al., 2015; Suri et al., 2015; Ambrogio et al., 2016; Babu et al., 2018; Fumarola et al., 2018; Go et al., 2019; Wu et al., 2020; Yin et al., 2020). (D) Learning curves for PCMO devices with Al and Ta O interlayers with 100 ms (red) and 1 ms update pulses. 2 5 Measurements,” the material stack of the PCMO devices and RPM. On the y-axis, the steepness value from the fit of the SPM choice of the voltage for the SET and RESET pulses in the pulse a is shown. This parameter is mostly dependent on the used SPM measurements lead to significantly different evolutions of the material stack (see Section “Stepwise SET and RESET Pulse resistance in these measurements. This also has an influence Measurements”). on the maximum accuracies that can be achieved using these The lowest accuracies resulting from the training simulations resistance curves for ANNs. The maximum accuracies of all can be found in the lower left corner for low ON/OFF ratio and simulations are plotted in Figure 6A. The x-axis shows the fitted low steepness of the SPM curve. Many of these simulations yield maximum resistance in the SPM and RPM curves. The higher accuracies below 70%, and for the data points in black no learning this value, the higher also the measured ON/OFF ratio in the at all with accuracies around 10% is even reached, which would pulse measurements is. As discussed in Section “Spike Timing be like a random drawing. However, for higher values of the a SPM Dependent Plasticity,” in most cases, a higher ON/OFF ratio parameter around 0.25, the maximum accuracy increases to 75% can be reached by choosing higher voltages for the SPM and to 80%, even for comparably low ON/OFF ratios. This means Frontiers in Neuroscience | www.frontiersin.org 9 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 10 Gutsche et al. Neuronal Networks With PCMO Devices that an initially steeper increase of the conductivity of a weight software. To achieve higher recognition accuracies, the ON/OFF leads to an increase in recognition accuracy. This observation ratio must be increased further. With an ON/OFF ratio of around appears contrary to previous observations that a more linear 30, Wu et al. (2020) showed an accuracy of about 95%. Even update behavior of the resistance is beneficial for the recognition higher accuracies can be reached using memristive devices as accuracy (Cai et al., 2020). Burr et al. (2017) attribute this to the storage for weights in a fashion of digital numbers instead of non-reversibility of a weight update that follows from a strong analog weights (Moon et al., 2015) (Babu et al., 2018). non-linearity. However, the difference here can be explained by As described in Section “Stepwise SET and RESET Pulse the different update rules. With the large batch size and sparse Measurements,” by varying the length of the applied pulses in update matrix used in this work, updates on the same device the SPM and RPM, short pulses lead to slower and longer mostly happen in the same direction and update pulses with pulses to a faster progression on the SPM or RPM curve, different polarities on the same device rarely occur. Therefore, respectively. Figure 6D shows a comparison of the learning the non-reversibility of a weight update is not an issue. of the MNIST dataset with long pulses of 100 ms (red) and Another path to higher accuracies is a higher ON/OFF ratio. short pulses of 1 ms (blue) using PCMO devices with Al and For a low a of around 0.1, the accuracy increases from low Ta O interlayers. Initially, for both interlayers, the accuracy SPM 2 5 to higher ON/OFF ratios. The same trend can also be found for for the network using the long pulses increases faster. In the high values of the RPM steepness factor a of about 0.4. The end, both saturate at about the same values, 82% for the Al SPM ON/OFF ratio has two effects here. Since all pulse measurements interlayer devices and 78% for the Ta O interlayer devices. 2 5 consist of an equal number of pulses, a higher ON/OFF ratio In the simulation with the latter devices, for the long pulses, means that the difference between the resistance steps is larger, as the recognition accuracy oscillates at high pulse numbers. This long as the curve reaches the saturation within the same number is not the case for the short pulses. In conclusion, the use of of pulses. On the other hand, a low ON/OFF ratio of, e.g., 3 means shorter update pulses can lead to a more stable, but slower that three OFF switched devices contribute the same activation to learning process. A gain in accuracy is not reached. However, a neuron in the subsequent layer as one ON switched device. For in conventional perceptron networks, the learning rate is an higher ON/OFF ratios, e.g., of 10, 10 devices can be in the OFF important factor for a successful learning and has a large impact state with one ON switched device still having a relatively high on the convergence and accuracy of the network. In this work, impact on the activation of the next neuron layer. The ability of we present one approach to implement a variable learning rate a device to differentiate the activation of the next neuron layer for resistive switching ReRAM devices by changing the pulse decreases with decreasing ON/OFF ratio. length. Such adaptive learning rates are not only beneficial for As described in Section 2.4, a parameter strongly depends artificial neural networks like perceptrons but can also be used in SPM on the material stack. Therefore, a separation between the brain-like learning systems to realize more biologically plausible materials can also be observed in Figure 6A, with the WO learning rules from neuroscience. interlayer samples for low, the Ta O interlayer samples for In conclusion for the neural network simulations, PCMO 2 5 intermediate and the Al interlayer samples for high values of ReRAM devices with Al, Ta O and WO interlayers can be used 2 5 3 a . For each material, the complete learning curves for those as weights in ANN learning to MNIST dataset. For devices with pot pulse measurement voltages resulting in the highest maximum an Al interlayer, the highest recognition accuracy of about 82% recognition accuracy are displayed in Figure 6B. After 6,000 could be achieved. A parameter optimisation showed how the training epochs, the network using the PCMO devices with the shape of the resistance evolution curve of pulse measurements Al interlayer reaches the highest recognition accuracy with 82%. affects the maximum accuracy. A high steepness of the SPM and The network using the Ta O interlayer devices, which initially the maximum ON/OFF ratio were identified as most important to 2 5 shows a faster increase of the recognition accuracy, exhibits a reach the highest accuracy values. While the steepness of the SPM saturation at a lower level of 76%. The WO interlayer devices depends mostly on the material stack used, the ON/OFF ratio can lead to a much lower learning speed but a similar recognition be maximized by choosing greater voltages for the SPM and RPM. accuracy of about 76%. Finally, the concept of a variable learning rate was implemented To benchmark the performance of the proposed network using different pulse lengths and the effect on the learning speed and memristive devices, the maximum accuracy reached can be and accuracy investigated. compared with similar networks. In Figure 6C, a comparison of previous works on ANNs with memristive devices training MNIST is provided separated by the maximum ON-OFF ratio of CONCLUSION the used devices on the x-axis. Most networks here have a similar layer and neuron-per-layer count as the network proposed in this In this work, we compared the performance of area-dependent work. The maximum accuracy of 82% reached in our simulations memristive PCMO devices with Al interlayer, where Al O is 2 3 can be found in the lower left corner of Figure 6C, which means formed naturally at the interface with devices where Ta O 2 3 a 14% higher error rate compared with the same network with and WO have been deposited intentionally. All investigated ideal floating point weights, as described above. It can be seen that devices show area-dependent switching and exhibit a STDP- this value is comparable with other networks using memristive like behavior. devices with a similar ON-OFF ratio, which is still lower than Furthermore, for all three types of devices, we performed accuracy values one would expect from conventional ANNs in SPMs and RPMs. The shape of the SPM and RPM curves differs Frontiers in Neuroscience | www.frontiersin.org 10 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 11 Gutsche et al. Neuronal Networks With PCMO Devices significantly for the different materials. In particular, the WO DATA AVAILABILITY STATEMENT stack showed a better linearity than the other two types of devices. Moreover, we showed that we can adapt the SPM and The raw data supporting the conclusions of this article will be RPM curves with the pulse parameters. By reducing the pulse made available by the authors, without undue reservation. height and the pulse length, we could adapt the step width of the resistance change and the ON/OFF ratio. Additionally, we showed that the amount of resistance change during the SPM or AUTHOR CONTRIBUTIONS RPM depends on the total time a voltage is applied irrespective of the number of pulses. All authors listed have made a substantial, direct and intellectual For the neural network simulations, the application of directly contribution to the work, and approved it for publication. deposited Ta O and WO layers does not lead to an increase 2 5 3 in recognition accuracy or increased learning speed compared with the Al interlayer devices despite of the better linearity FUNDING of the SPM and RPM curves of the WO . A hyperparameter optimisation shows the influence of the pulse lenght and height This work was supported by the DFG (German Science on the SPM and RPM curves and the influence of their shape Foundation) within the collaborative research center SFB on the maximum accuracy. The ON/OFF ratio and the SPM 917 “Nanoswitches” and by the Helmholtz Association Initiative and Networking Fund under project number SO-092 steepness are identified as the most crucial for high accuracies. Furthermore, it was shown that using shorter update pulses leads [Advanced Computing Architectures (ACA)] and the Federal to a slower initial increase of the recognition accuracy but a more Ministry of Education and Research (project NEUROTEC stable learning process, with less oscillations. Based on this, we grant no. 16ES1133K). propose a new approach to implement a variable learning rate for resistive switching ReRAM devices by changing the pulse length that might be interesting for perceptron networks in the future. SUPPLEMENTARY MATERIAL In conclusion, we demonstrated how two fundamentally different learning rules for neural networks, STDP in SNN and The Supplementary Material for this article can be found gradient descent learning in ANN, could be realized in the same online at: https://www.frontiersin.org/articles/10.3389/fnins. memristive devices. 2021.661261/full#supplementary-material Bagdzevicius, S., Maas, K., and Boudard, M. (2017). Interface-type resistive REFERENCES switching in perovskite materials. J. Electroceram. 39, 157–184. doi: 10.1007/ Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., et al. (2016). s10832- 017- 0087- 9 “TensorFlow: a system for large-scale machine learning,” in Proceedings of the Burr, G. W., Shelbya, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., et al. (2017). 12th USENIX Symposium on Operating Systems Design and Implementation, Neuromorphic computing using non-volatile memory. Adv. Phys. 2, 89–124. (Savannah, GA). Cai, F., Correll, J. M., Hwan Lee, S., Lim, Y., Bothra, V., Zhang, Z., Alibart, F., Zamanidoost, E., and Strukov, D. B. (2013). Pattern classification by et al. (2019). A fully integrated reprogrammable memristor– CMOS memristive crossbar circuits using ex situ and in situ training. Nat. Commun. system for efficient multiply–accumulate operations. Nat. Electron. 2, 4:2072. 290–299 Ambrogio, S., Balatti, S., and Milo, V. R Carboni, Z-Q Wang, A Calderoni, Cai, F., Correll, J. M., Lee, S. H., Lim, Y., Bothra, V., Zhang, Z., et al. (2020). A fully et al. (2016). Neuromorphic learning and recognition with one-transistor-one- integrated reprogrammable memristor- CMOS system for efficient multiply– resistor synapses and bistable metal oxide RRAM. IEEE Trans. Electron Devices accumulate operation. Nat. Electron. 2, 290–299. doi: 10.1038/s41928- 019- 63, 1508–1515. doi: 10.1109/ted.2016.2526647 0270- x Arndt, B., Borgatti, F., Offi, F., Phillips, M., Parreira, P., Meiners, T., et al. (2017). Chowdhury, A. P., Kulkarni, P., and Bojnordi, M. N. (2018). MB-CNN: memristive Spectroscopic indication of tunnel barrier charging as the switching mechanims binary convolutional neural networks for embedded mobile devices. J. Low in memristive devices. Adv. Func. Mater. 27:1702282. doi: 10.1002/adfm. Power Electron. Appl. 8:38. doi: 10.3390/jlpea8040038 201702282 Fumarola, A., Sidler, S., Moon, K., Jang, J., Shelby, R. M., Narayanan, P., et al. Asamitsu, A., Tomioka, Y., Kuwahara, H., and Tokura, Y. (1997). Current (2018). Bidirectional non-filamentary RRAM as an analog neuromorphic switching of resistive states in mgnetoresistive manganites. Nature 388, 50–52. synapse, part II: impact of Al/Mo/Pr0.7Ca0.3 MO device characteristics on doi: 10.1038/40363 neural network training accuracy. IEEE J. Electron Devices Soc. 6, 169–178. Asanuma, S., Akoh, H., Yamada, H., and Sawa, S. (2009). Relationship between doi: 10.1109/jeds.2017.2782184 resistive switching characteristics and band diagrams of TiÕPr1–xCaxMnO3 Gao, Y., Wu, S., and Adam, G. C. (2020). “Batch training for neuromorphic systems junctions. Phys. Rev. B 80:235113. with device non-idealities,” in Paper Presented at the International Conference on Babu, V., Lashkare, S., Ganguly, U., and Rajendran, B. (2018). Stochastic learning Neuromorphic Systems, Washington: DC. in deep neural networks based on nanoscale PCMO device characteristics. Go, Y., Kim, M., Kwak, J., Song, S., Chekol, A., Kwon, J.-D., et al. (2019). W/WO3– Neurocomputing 321, 227–236. doi: 10.1016/j.neucom.2018.09.019 x based three-terminal synapse device with linear conductance change and Backus, J. (1978). Can programming be liberated from the von neumann style? high on/off ratio for neuromorphic application. Appl. Phys. Express 12:026503. Commun. ACM 21, 613–641. doi: 10.1145/359576.359579 doi: 10.7567/1882- 0786/aafc74 Baek, K., Park, S., Park, J., and Hwang, H. (2017). In situ TEM observation on Herpers, A. (2014). Electrical Characterization of Manganite and Titanate the interface-type resistive switching by electrochemical redox reactions at a Heterostructures, Jülich: Forschungszentrum Jülich GmbH Zentalbibliothek. TiN/PCMO interface. Nanoscale 9, 582–593. doi: 10.1039/c6nr06293h Ph.D. thesis, Aachen: Verlag. Frontiers in Neuroscience | www.frontiersin.org 11 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 12 Gutsche et al. Neuronal Networks With PCMO Devices Jooss, L., Wu, T., Beetz, R., Klie, F., Beleggia, M., Schofield, M. A., et al. (2007). reactive metal/polycrystalline Pr0.7Ca0.3MnO3 device for nonvolatile memory Polaron melting and ordering as key mechanism for colossal resistance effects applications,” in Proceedings of the IEEE International Electron Devices Meeting, in manganites. Proc. Natl. Acad. Sci. U.S.A. 104, 13597–13602. doi: 10.1073/ (Baltimore, MD: IEEE). pnas.0702748104 Seong, D.-J., Hassan, M., Choi, H., Lee, J., Yoon, J., Park, J.-B., et al. (2009a). Krestinskaya, O., Choubey, B., and James, A. P. (2020). Memristive GAN in analog. Resistive-switching characteristics of Al/Pr0.7Ca0.3MnO3 for nonvolatile Sci. Rep. 10:5838. memory applications. IEEE Electron. Device Lett. 30:919–921. Liu, X., Kim, I., Siddik, M., Sadaf, S. M., Biju, K. P., Park, S., et al. (2011). Simmons, J. G., and Verderber, R. R. (1967). New conduction and reversible Resistive switching mechanism of a Pr0.7Ca0.3MnO3-based memory device memory phenomena in thin insulating films. Proc. R. Soc. Lond. A Math. Phys. and assessment of its suitability for nano-scale applications. J. Korean Phys. Soc. Sci. 301, 77–102. doi: 10.1098/rspa.1967.0191 59, 497–500. doi: 10.3938/jkps.59.497 Suri, M., Parmar, V., Kumar, A., Querlioz, D., and Alibart, F. (2015). Menzel, S., von Witzleben, M., Havel, V., and Böttger, U. (2019). The ultimate “Neuromorphic hybrid RRAM_CMOS RBM architecture,” in Proceedings of the switching speed limit of redox-based resistive switching devices. Faraday 15th Non-Volatile Memory Technology Symposium, Beijing. Discuss. 213, 197–213. doi: 10.1039/c8fd00117k Tsymbal, E. Y., and Kohlsted, H. (2006). Tunneling across a ferroelectric. Science Menzel, S., Waters, M., Marchewka, A., Böttger, U., Dittmann, R., and Waser, R. 5784, 181–183. doi: 10.1126/science.1126230 (2011). Origin of the ultra-nonlinear switching kinetics in oxide-based resistive Waser, R., Dittmann, R., Staikov, G., and Szot, K. (2009). Redox-based resistive switches. Adv. Func. Mater. 21, 4487–4492. doi: 10.1002/adfm.201101117 switching memories - nanoionic mechanims, prospects, and challenges. Adv. Moon, K., Cha, E., Park, J., Chu, S. M., Gi, H., Wang, H., et al. (2015). “High Mater. 21, 2632–2663. doi: 10.1002/adma.200900375 density neuromorphic system with Mo/Pr0.7Ca0.3MnO3 synapse and NbO2 Wolf, W. A., and McKee, S. A. (1994). Hitting the memory wall: implications IMT oscillator neuron,” in Proceedings of the IEEE International Electron Devices of the obvious. Dep. Comput. Sci. 23, 20–24. doi: 10.1145/216585.21 Meeting, (Washington, DC: IEEE), 463–466. 6588 Moon, K., Park, S., Lee, D., Woo, J., Cha, E., Lee, S., et al. (2014). “Resistive- Wu, F., Mo, T., Saraya, T., Hiramoto, T., and Kobayashi, M. (2020). A monolithic switchiing analogue memory device for neuromorphic applications,” in 3-D integration of RRAM array and oxide semiconductor FET for in-memory Proceedings of the 2014 Silicon Nanoelectronics Workshop, (Honolulu, HI: computing in 3-D neural network. IEEE Tran. Electron Devices 67, 5322–5328. IEEE). doi: 10.1109/ted.2020.3033831 Park, S., Chu, M., Kim, J., Noh, J., Jeon, M., Lee, B. H., et al. (2015). Electronic Xia, Q., and Yang, J. J. (2019). Memristive crossbar arrays for brain-inspired. Nat. system with memristive synapses for pattern recognition. Sci. Rep. 5:10123. Mater. 18, 309–323. doi: 10.1038/s41563- 019- 0291- x Park, S., Kim, H., Choo, M., Noh, J., Sheri, A., Jung, S., et al. (2012). “RRAM- Yakopcic, C., Alom, M. Z., and Taha, T. M. (2017). “Extremely parallel memristor based synapses for neuromorphic system with pattern recognition function,” in crossbar architecture,” in Proceedings of the International Joint Conference on Proceedings of the 2012 International Electron Devices Meeting, (San Francisco, Neural Networks, (Anchorage, AK: IEEE). CA: IEEE), 231–234. Yao, P., Wu, H., Gao, B., Tang, J., Zhang, Q., Zhang, W., et al. (2020). Fully Park, S., Sheri, A., Kim, J., Noh, J., Jang, J., Jeon, M., et al. (2013). “Neuromorphic hardware-implemented memristor convolutional neual network. Nature 577, Speech systems using advanced ReRAM-based synapses,” in Proceedings of the 641–646. doi: 10.1038/s41586- 020- 1942- 4 2013 IEEE International Electron Devices Meeting, (Washington, DC: IEEE), Yin, L., Cheng, R., Wang, Z., Wang, F., Sendeku, M. G., Wen, Y., et al. (2020). Two- 625–628. dimensional unipolar memristors with logic and memory functions. Nano Lett. Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., and 20, 4144–4152. doi: 10.1021/acs.nanolett.0c00002 Strukov, D. B. (2015). Training andoperation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64. doi: 10.1038/ Conflict of Interest: The authors declare that the research was conducted in the nature14441 absence of any commercial or financial relationships that could be construed as a Sawa, A. (2006). Interface resistance switching at a few nanometer thick perovskite potential conflict of interest. manganite active layers. Appl. Phys. Lett. 88:232112. doi: 10.1063/1.2211147 Sawa, A. (2008). Resistive switching in transition metal oxides. Mater. Today 11, Copyright © 2021 Gutsche, Siegel, Zhang, Hambsch and Dittmann. This is an open- 28–36. doi: 10.1016/s1369- 7021(08)70119- 6 access article distributed under the terms of the Creative Commons Attribution Sawa, A., Fujii, T., Kawasaki, M., and Tokura, Y. (2004). Hysteretic current–voltage License (CC BY). The use, distribution or reproduction in other forums is permitted, characteristics and resistance switching at a rectifying Ti/Pr0.7Ca0.3MnO3 provided the original author(s) and the copyright owner(s) are credited and that the interface. Appl. Phys. Lett. 85:4073. doi: 10.1063/1.1812580 original publication in this journal is cited, in accordance with accepted academic Seong, D.-J, Park, J., Lee, N., Hasan, M., Jung, S., Choi, H., et al. (2009b). “Effect of practice. No use, distribution or reproduction is permitted which does not comply oxygen migration and interface engineering on resistance switching behavior of with these terms. Frontiers in Neuroscience | www.frontiersin.org 12 July 2021 | Volume 15 | Article 661261 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Frontiers in Neuroscience Pubmed Central

Exploring Area-Dependent Pr0.7Ca0.3MnO3-Based Memristive Devices as Synapses in Spiking and Artificial Neural Networks

Frontiers in Neuroscience , Volume 15 – Jul 2, 2021

Loading next page...
 
/lp/pubmed-central/exploring-area-dependent-pr0-7ca0-3mno3-based-memristive-devices-as-S7vcW1uPWV

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Pubmed Central
Copyright
Copyright © 2021 Gutsche, Siegel, Zhang, Hambsch and Dittmann.
ISSN
1662-4548
eISSN
1662-453X
DOI
10.3389/fnins.2021.661261
Publisher site
See Article on Publisher Site

Abstract

fnins-15-661261 June 26, 2021 Time: 19:12 # 1 ORIGINAL RESEARCH published: 02 July 2021 doi: 10.3389/fnins.2021.661261 Exploring Area-Dependent Pr Ca MnO -Based Memristive 0:7 0:3 3 Devices as Synapses in Spiking and Artificial Neural Networks Alexander Gutsche , Sebastian Siegel, Jinchao Zhang, Sebastian Hambsch and Regina Dittmann Peter Grünberg Institut (PGI-7/10), Forschungszentrum Jülich GmbH & JARA-FIT, Jülich, Germany Memristive devices are novel electronic devices, which resistance can be tuned by an external voltage in a non-volatile way. Due to their analog resistive switching behavior, they are considered to emulate the behavior of synapses in neuronal networks. In this work, we investigate memristive devices based on the field-driven redox process between the p-conducting Pr Ca MnO (PCMO) and different tunnel barriers, 0:7 0:3 3 namely, Al O , Ta O , and WO . In contrast to the more common filamentary-type 2 3 2 3 switching devices, the resistance range of these area-dependent switching devices can be adapted to the requirements of the surrounding circuit. We investigate the impact of the tunnel barrier layer on the switching performance including area scaling of the Edited by: Sabina Spiga, current and variability. Best performance with respect to the resistance window and National Research Council (CNR), Italy the variability is observed for PCMO with a native Al O tunnel oxide. For all different 2 3 Reviewed by: layer stacks, we demonstrate a spike timing dependent plasticity like behavior of the Martin Ziegler, investigated PCMO cells. Furthermore, we can also tune the resistance in an analog Technische Universität Ilmenau, Germany fashion by repeated switching the device with voltage pulses of the same amplitude Brian Douglas Hoskins, and polarity. Both measurements resemble the plasticity of biological synapses. We National Institute of Standards and Technology (NIST), United States investigate in detail the impact of different pulse heights and pulse lengths on the shape *Correspondence: of the stepwise SET and RESET curves. We use these measurements as input for the Alexander Gutsche simulation of training and inference in a multilayer perceptron for pattern recognition, to a.gutsche@fz-juelich.de show the use of PCMO-based ReRAM devices as weights in artificial neural networks Specialty section: which are trained by gradient descent methods. Based on this, we identify certain trends This article was submitted to for the impact of the applied voltages and pulse length on the resulting shape of the Neuromorphic Engineering, measured curves and on the learning rate and accuracy of the multilayer perceptron. a section of the journal Frontiers in Neuroscience Keywords: PCMO, memristive devices, perceptron learning, resistive switching, multilevel switching Received: 30 January 2021 Accepted: 21 May 2021 Published: 02 July 2021 INTRODUCTION Citation: Gutsche A, Siegel S, Zhang J, Most modern computer architectures are based on the von Neumann principle, which separates Hambsch S and Dittmann R (2021) the data processing unit from the data storage. As the performance of processors increased strongly Exploring Area-Dependent over the last decades, the bandwidth for the communication between processor and data storage Pr Ca MnO -Based Memristive 0:7 0:3 3 became the limiting factor for the overall computational performance. This is called the von Devices as Synapses in Spiking Neumann bottleneck (Backus, 1978) (Wolf and McKee, 1994). and Artificial Neural Networks. The limit is especially problematic for tasks, where simple operations are performed on large Front. Neurosci. 15:661261. doi: 10.3389/fnins.2021.661261 sets of data, e.g., learning tasks in massively parallel systems mimicking brain-like functionalities Frontiers in Neuroscience | www.frontiersin.org 1 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 2 Gutsche et al. Neuronal Networks With PCMO Devices or vector-matrix multiplications in artificial neural networks over the whole device area (Herpers, 2014; Bagdzevicius et al., (ANNs) during the inference step. The multiplication of the input 2017). Since the current of the area-type switching devices nodes of a layer with the weight matrix yields the output of scales for both the high resistive state (HRS) and the low this layer. A possible strategy to overcome this von Neumann resistive state (LRS) with the device area, the resistance bottleneck for ANNs is the usage of resistive arrays as weight values can be adapted to the given circuit requirements. matrices (Xia and Yang, 2019). To achieve tunable weights, This is not the case for the most common filamentary-type one approach is the so-called memristive device, an electrically memristive devices. Moreover, filamentary-type switching is tunable resistor. Previous works already show that memristive usually indicated by a sharp SET process. In contrast, area- crossbar arrays allow for efficient vector-matrix multiplication type switching devices exhibit a gradual SET and RESET (Cai et al., 2019). The use in ANNs was demonstrated on many that enhances their ability for analog switching in comparison network types such as single-layer perceptrons (Alibart et al., with filamentary memristive devices. Due to their analog 2013; Prezioso et al., 2015) as well as multilayer perceptrons switching behavior, PCMO-based resistive switching devices are (Moon et al., 2015; Burr et al., 2017; Babu et al., 2018; Go et al., considered hardware representation for synapses in artificial 2019; Wu et al., 2020) and convolutional neural networks (CNNs) neural networks as described above. In particular, it has been (Yakopcic et al., 2017). Many groups show that memristive shown that they can emulate aspects of synaptic plasticity (Park devices can already today replace conventional networks trained et al., 2012, 2013, 2015; Moon et al., 2014; Fumarola et al., in software for many applications. Li et al. report a recognition 2018). accuracy of more than 97% on the MNIST dataset, which is In this work, we compare in detail the performance and common for benchmarking of pattern recognition tasks. Also, analog behavior of PCMO-based devices with different interface more complex tasks like face recognition have been demonstrated configurations. In particular, we compare the more common (Yao et al., 2020). These similar network performances are Al/PCMO devices with a natively formed Al O oxide to 2 3 often achieved at higher-energy efficiencies and make memristive devices with a directly sputtered Ta O and WO as interface 2 5 3 device-based ANNs most useful for low-energy applications layer. For all devices, we can demonstrate analog switching at the edge and in the IoT sector (Chowdhury et al., 2018) behavior. We demonstrate a STDP-like behavior on single PCMO (Krestinskaya et al., 2020). A large variety of different types of devices. This learning rule for spiking neural networks (SNN) memristive devices have been proposed for neuronal networks stems from neuroscience and neurophysiology. Furthermore, so far in the literature mimicking behavior of biological synapses we investigated in detail the impact of the material stack as like, e.g., long-term potentiation and depression (LTP/LTD) and well as pulse length and height on the shape of the analog even more complex aspects of synaptic plasticity like simple stepwise SET and RESET curves. This stepwise change of forms of spike timing dependent plasticity (STDP), but no conductance mimics aspects of LTP/LTD of biological synapses. optimal memristive device type has been identified yet. For a We use the experimental data as input for simulations of the given choice of materials, the ANN, the learning rule and the training of a multilayer perceptron for pattern recognition and update rule have to be adjusted to obtain best performance. In reveal how the different electrical stimuli and the resulting this work, we propose an update rule for a specific memristive shapes of the stepwise SET and RESET measurement (SPM device based on Pr Ca MnO (PCMO) after a thorough and RPM) curves affect the learning rate and the accuracy 0:7 0:3 3 investigation of its switching behavior and the influence of of the network based on a gradient descent learning rule, different material stacks. which is a learning rule for conventional ANNs. Comparing In memristive devices, information is stored by the change in STDP and gradient decent methods, STDP only requires local the resistance that can be switched by an applied bias in a non- information processing between the two neurons adjacent to volatile manner. Different mechanisms and materials that show the very synapse, while gradient descent methods take the resistive switching have been reported in literature (Simmons global error of the complete network into account. Here, we and Verderber, 1967; Asamitsu et al., 1997; Sawa, 2006, 2008; present how both learning rules can be achieved with the same Tsymbal and Kohlsted, 2006; Jooss et al., 2007; Waser et al., 2009; memristive device. Herpers, 2014). In this work, we will address the mixed valence manganite (PCMO) in combination with a tunnel oxide that has EXPERIMENTAL been either deposited directly by physical vapor deposition or that has been formed by the redox process with an oxidisable metal top electrode. Combinations of PCMO with many different Sample Preparation metals are reported in literature so far: e.g., Al (Seong et al., The memristive devices consist of a 25-nm-thick Pt bottom 2009a), Ta (Seong et al., 2009b), Ti (Seong et al., 2009b), W (Liu electrode, a 20-nm PCMO film grown by pulsed laser deposition et al., 2011), and others (Moon et al., 2014, 2015; Baek et al., 2017; (PLD), a 7-nm-thick interface layer, either Al, Ta O or WO 2 5 3 Go et al., 2019). It is proposed that the field-driven movement of and a 25-nm-thick Pt top electrode as sketched in the insets of oxygen anions between the PCMO layer and the reactive metal Figures 1A–C. The Pt layer that serves as bottom electrode is electrode is the underlying switching mechanism (Sawa et al., DC sputtered on top of a 5 nm Ta adhesion layer on a thermally 2004; Asanuma et al., 2009; Seong et al., 2009a). oxidized Si wafer. PCMO is known for its area-type resistive switching The PLD growth of PCMO is performed with an O pressure properties, namely that the change of the resistance happens of 0.133 mbar at room temperature (RT). A laser fluence of Frontiers in Neuroscience | www.frontiersin.org 2 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 3 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 1 | (A–C) I-V curves for three different materials, (B) Al/Al O , (C) Ta O , and (D) WO , like it is indicated by the layer stack in the lower right corner. The 2 3 2 5 3 voltage is applied on the top electrode. For every device, three different sizes—100 100, 50 50, and 20 20 mm —are measured. The red dotted line indicates the read out voltage for the area scaling. The switching voltages differ between the three stacks: (A) Al O : 2.2 V/–3 V, (B) Ta O :  2.5 V, and (C) WO :  2 V. 2 3 2 5 3 (D–F) Area dependence of the LRS and HRS. The resistance value is scaling with the device size, for all of the three materials, (D) Al O , (E) Ta O , and (F) WO . 2 3 2 5 3 The slopes of the linear fit for all of the devices and the HRS and LRS are around –1 /mm . Slopes can be seen in Table 2. TABLE 1 | Voltages for the conducted measurements. Material Set voltage (V) Reset voltage (V) Read-out voltage (V) SPM voltages (V) RPM voltages (V) Read-out (V) Al/Al O 3.0 2.2 0.5 1.5 to 2.0 1.2 to 2.0 0.3 2 3 Ta O 2.5 2.5 1.0 to 1.5 0.8 to 2.0 2 5 WO 2.0 2.0 2.0 to 2.6 1.2 to 2.0 Electrical Measurements 1.33 J/cm and a frequency of 5 Hz are used during PLD growth. Around 2,800 pulses are needed to grow a 20-nm In preparation of the electrical measurements, the samples are amorphous PCMO layer. Afterward, the PCMO thin film is glued to a large sample carrier chip with Pt pads. The BE is annealed in N atmosphere at 650 C for 2 min in order to contacted to one of the Pt pads on the sample carrier using crystallize the PCMO layer. aluminum wire bonding. Two different setups are used to The Ta O and WO layers are deposited by RF sputtering at characterize the samples electrically, namely one to perform the 2 5 3 RT. Both depositions are performed at 200 W with 5 10 mbar quasi-static current-voltage (I-V) measurements, the other one pressure and an Ar/O ratio of 3/2. Afterward, the sample is to apply pulses to the devices. A Keithley 2611B is used to transferred in situ into an e-beam evaporator to deposit the Pt measure the I-V characteristics of the devices. The connection top layer which is used as top electrode. During the Pt deposition between the measurement unit and the device is performed in vacuum, the e-beam process heats the sample up to 180 C. For by soft tungsten needles. Every measurement starts with an the Al device stack, a 7-nm layer is also deposited on top of the initialization curve: 0 V!2.5 V!–2.5 V!0 V. During this PCMO layer by e-beam evaporation and capped in situ with the initialization procedure, the oxide layers of the metal are Pt layer. During the short deposition of the Al layer, no significant presumably homogenized (Arndt et al., 2017). Afterward, the increase in temperature can be detected. Here also, a 25-nm Pt regular switching cycle can be performed: 0 V!RESET voltage capping layer is used. (positive)!SET voltage (negative)!0 V. The SET and RESET The patterning of the top electrode and the active interface voltages have to be adapted for the different interface layer layer of the devices is performed by optical lithography and materials. In Table 1, the writing voltages that show the most Ar ion-beam etching. The pad size varies between 100  100, stable switching for the different interface layers can be found 50 50, 20 20, and 10 10 mm . along with the read voltage. Frontiers in Neuroscience | www.frontiersin.org 3 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 4 Gutsche et al. Neuronal Networks With PCMO Devices The pulse measurements for the multilevel SPMs and RPMs with voltage in the negative branch is much higher, compared are performed with a Keithley 4200A. Different pulse lengths with Figures 1A,B. For the WO devices, a stable switching curve between 1 and 100 ms are employed. A large variety of can be found with symmetric switching voltages at  2 V. At combinations of SPM and RPM voltages are investigated. The around –1.8 V, a change in the slope can be seen at least for parameter ranges that are used for the different devices are the 50 mm  50 mm and the 100 mm  100 mm devices. For displayed in Table 1. the 20 mm  20 mm devices a similar change in slope can be The STDP measurements are performed with an Arc One surmised, but not clearly determined. from Arc Instruments. For all three device types, a pulse length Each device state, HRS and LRS, for the Al, Ta O , and 2 5 of 100 ms with a pulse voltage of 2 V/–2 V is investigated. WO interface devices are tested regarding their retention time. Over a period of several days, no change in the states can be determined. The samples are stored at room temperature and in ELECTRICAL CHARACTERISATION OF ambient atmosphere. To prove that all of the devices show area type resistive PCMO MEMRISTIVE DEVICES switching, we read out the resistance at –0.5 V since switching effects can be excluded at this voltage and the resistance at this Quasi Static I-V Measurements voltage is plotted against the device area (see Figures 1D–F). The In Figure 1A, the I-V measurements of a typical sample with read out is chosen to be on the negative branch due to a higher an Al interface layer can be seen. A clear hysteresis of the I-V ON/OFF ratio. A clear linear relationship between the device curve on both the positive and the negative branch is visible. The resistance and the device area can be seen for the HRS and the SET takes place at negative voltages and the RESET at positive LRS for all of the devices with a slope around 1 /mm , as voltages. For negative applied voltages, the difference between the expected by Ohm’s law. The exact values of the fitted slopes can LRS and the HRS, called ON/OFF-ratio is higher. Concerning be found in Table 2. the gradual switching of the area type switching devices, no Additionally, we studied the device-to-device (d2d) and distinct SET or RESET voltage can be defined. Therefore, we cycle-to-cycle (c2c) variability of the devices during the quasi- always choose a voltage pair that allows stable switching of the static I-V measurement. For these measurements, we used the devices without any change of the I-V curves during the repeated 20 mm 20 mm devices. Figures 2A–C shows the combined c2c switching, e.g., 2.2 and 3 V for RESET and SET, respectively, and d2d Weibull distribution for the different devices, namely, in case of the Al devices. For simplicity reasons we will call (A) Al/Al O , (B) Ta O and (C) WO . For the Al/Al O 2 3 2 5 3 2 3 the maximum voltage in the different voltage directions SET interface layer, it can be seen that the HRS and LRS are clearly and RESET. During the RESET, the slope changes at 1.8 V. separable over their whole resistance range. The spread of the A similar but smaller change in slope can be seen during the HRS and the LRS is half an order of magnitude. For the Ta O 2 5 RESET at 2 V. Furthermore, the RESET and SET are both interface layer (Figure 2B), the spread for the different devices gradual, with no abrupt jumps into the HRS or LRS. The I-V and cycles is smaller. However, due to the smaller ON/OFF ratio, curves for different pad sizes all have the same shape with smaller the overlap of the two states is around a few percent. For the WO differences, like the opening on the positive side. For smaller devices, the variability plot (see Figure 2C) differs from the plot of devices, the opening becomes smaller in the positive branch. This the other devices. It can be seen that the LRS shows a much higher effect is not observed for the negative branch. variability than the HRS. The variability of the HRS is as small The I-V curves of a device with a Ta O interface layer are 2 5 as for all of the other device types. Comparing all three device shown in Figure 1B. This switching polarity is the same as for the stacks, the Al/Al O devices show a higher ON/OFF ratio than 2 3 Al devices, and HRS and LRS are clearly separable on the negative the Ta O devices and a lower variability than the WO devices. 2 5 3 side. Also, a change in the slope of the I-V curve can be found around –2 V. On the positive branch, no opening and no change Spike Timing Dependent Plasticity in slope can be seen. In STDP, the change of a synaptic weight between neurons The WO devices show a different shape of the I-V curve depends on the time difference between two spikes, the pre- compared with the Al and Ta O devices. The positive and the 2 5 and post-synaptic neuron pulse. The memristive devices act as negative branches both show two clearly separable resistive states, synapses, and the pre- and post-synaptic pulse are applied at the see Figure 1C. In contrast to the case of Al and Ta O devices, 2 5 top/bottom electrode, respectively. the I-V curves are very symmetric for positive and negative Figure 3 shows the relative change in conductance of the polarities. In particular, the increase in current in the LRS state three different memristive devices for different time delays between the pre- and post-synaptic pulse. All devices show an increase/decrease in conductance for a negative/positive time TABLE 2 | Slope of the linear fit of the resistance vs. area plot for all the delay between the pulses, respectively. The Al STDP curve different materials. (Figure 3A) shows a symmetric increase or decrease of the 2 2 Material HRS (/mm ) LRS (/mm ) conductance for the time delay between the pulses compared with the STDP curves of the Ta O and the WO (Figures 3B,C). Al/Al O 1.09  0.01 1.12  0.07 2 3 2 5 3 Ta O 0.98  0.18 0.95  0.05 The WO (Figure 3C) shows a clear asymmetry between the 2 5 3 WO 1.25  0.18 1.02  0.22 increase and decrease of conductance. The maximum increase in Frontiers in Neuroscience | www.frontiersin.org 4 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 5 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 2 | (A–C) The Weibull plot of the combined cycle to cycle variability (c2c) and device-to-device variability (d2d) for the three different stacks, (A) Al O , 2 3 (B) Ta O , and (C) WO , indicated by the stack in the lower right corner. For each stack, 10 different devices with each up to 100 cycles have been investigated. 2 5 3 FIGURE 3 | Relative change of conductance for different time delays between the pre- and post-synaptic pulse during STDP measurement for the three different devices. (A) Al/Al O , (B) Ta O , and (C) WO . 2 3 2 5 3 conductance is around twice as high as the decrease. Therefore, For the positive voltage curves, the resistance saturates after all three types of devices are suitable for the implementation in 20 pulses for both voltages but with different saturation SNNs based on the STDP learning rule. level, a higher/lower resistance for the higher/smaller voltage, respectively. Furthermore, the increase in resistance at the beginning of the curve is higher with a higher pulse voltage and Stepwise SET and RESET Pulse smaller with smaller pulse voltage. After the steep increase in the beginning, the resistance only slightly increases. The SPM curves Measurements that are measured with positive pulse voltages saturate after10 We perform stepwise SET and RESET pulse measurements by pulses. Both curves show a clear non-linear behavior. applying the same voltage pulse multiple times to one device without switching the device back into a predefined state. By The SPM and RPM curves of the Ta O device are depicted 2 5 in Figure 4B. The RPM curve is shown for two different pulse applying pulses with a lower voltage, compared with the voltages used during the IV measurement, it is possible to tune the voltages, namely, 0.8 and 1.4 V, each with 100 ms pulse length. The SPM pulses have a height of –1.0 or –1.5 V, also with a resistance of the devices in a gradual way between the HRS and the LRS and vice versa. The transition from the HRS to the LRS in pulse length of 100 ms. Again, both the RPM and SPM curve characteristics are non-linear. The SPM curves saturate after10 the SPM and the transition from the LRS to the HRS in the RPM pulses, similar to the Al/Al O , but the maximum resistance happen stepwise. With these measurements, we can show that it 2 3 reached is different. The larger negative voltage leads to a lower is possible to write different resistance states into the investigated resistance value, compared with the smaller negative voltages. devices, resembling the LTP/LTD behavior of biological synapses. For the RPM curves, the resistance increases less with each pulse In Figure 4A, the SPM and RPM measurements of PCMO for the 0.8 V pulses as for the 1.4 V pulses. Furthermore, the with the Al/Al O interlayer are depicted. The chosen voltage 2 3 obtained saturation resistance is also smaller and therefore the for the SPM and RPM are 1.8 V/2.0 V and –1.5 V/ 2.0V, ON/OFF ratio is smaller. Beside the smaller ON/OFF ratio, the respectively, at a pulse length of 100 ms. Every pulse was applied 50 times without going back to the initial state. The largest curve shows a more linear increase in resistance during the pulse measurement with the smaller SET voltage compared with the resistance change for the SPMs and RPMs of the Al/Al O 2 3 interface device occurs during the first few pulses of a cycle. larger SET voltage. Frontiers in Neuroscience | www.frontiersin.org 5 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 6 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 4 | Results of the SPM and RPM for the different devices. The RPM curve (red) and the SPM curve are plotted for the same material into the same coordinate system. The SPM curve starts at higher pulse numbers and goes down to lower pulse numbers; the RPM curve can be read as normal. The pulse voltages for the different stacks can also be seen in Table 1. (A) Al O , (B) Ta O , and (C) WO . Every pulse has a length of 100 ms. For the shown curves, the fits 2 3 2 5 3 are shown in the plots. In Figure 4C, measurements for the WO stack with The fit function is used to determine the change of the 1.2 V/1.4 V and –2 V/–2.6 V for RPM and SPM, respectively, resistance of a memristive device upon the application of either are shown. All measurements are performed with a pulse length a SET or RESET pulse. For a SET pulse, the fit function for of 100 ms. The RPM curves of WO are more linear compared the respective SPM curve y .n/ is inverted and the current 3 SPM with the RPM curves of the Al and the Ta O devices, and no conductance (before the update pulse) of the device is used 2 5 clear saturation can be seen for the shown RPM curves. The to determine the pulse number n , which resembles this current SPM curves show a saturation after 20 pulses with a slight conductance. Following to this, y .n/ is evaluated at n C SPM current resistance decrease afterward. Here, a clear separation between 1 to yield the conductance of the device after the SET update the saturation levels for the different voltages can also be seen. pulse. For a RESET pulse, the inverse of the RPM curve’s fit function y .n/ and the current resistance yield n and RPM current y .n C 1/ gives the resistance value of the device after RPM current Behavioral Modeling of the Resistance the update pulse. Changes of the PCMO Devices Another common approach for a behavioral model in the To better analyze the impact of the material stack and applied literature is fitting the resistance change [e.g., (Suri et al., 2015)] voltages on the shape of the SPM and RPM pulse measurements instead of the actual resistance as it is proposed in this work. and to use these measurements in the ANN simulations in However, this approach showed a similar fitting accuracy for the Section “Perceptron Learning of Mnist Dataset,” the evolution data used here but a lower computational performance in the of the resistance for the devices with an Al, Ta O , and WO 2 5 3 TensorFlow environment. interlayer is mathematically fitted. Similar to other approaches in The fit parameters ý and ý correspond to the SPM RPM literature (Suri et al., 2015), a logistic function saturation value of the resistance in the SPM and RPM, respectively. Therefore, the maximum ON/OFF ratio of a pair of yK SPM and RPM curves can be calculated from these parameters. y.n/ D (1) 1C exp.a  n c/ The ON/OFF ratio strongly depends on the used material stack. With the Ta O interlayer samples, the lowest ON/OFF ratios 2 5 is employed, where y is the fitted resistance for the RPM curve can be reached, while the WO samples show the highest values and conductance for the SPM curve. ý is the maximum value at and with an Al interlayer, intermediate values can be reached. which the function saturates, and a and c determine the steepness Additionally, higher SET and RESET voltages lead to a higher of the increasing swing. In the following, parameters concerning maximum ON/OFF ratio in the pulse measurements, except for the SPM fit are equipped with the index SPM and parameters the SPM with –2.6 V for the WO interlayer samples and the 1.6- V RPM for the Ta O interlayer samples. It can be found that an concerning the RPM fit with the index RPM. This formula shows 2 5 a strong saturation for high values of the pulse number n as increase of the SET voltage leads to a lower saturation resistance in the SPM and a higher RESET voltage to a higher saturation observed in our experiments and reasonably good fitting of the measured resistance values. A fitted resistance value can therefore resistance of the RPM. For the a parameter of the fit function (1), which resembles be attributed to each measured resistance, leading to a total of 50 different resistance values for every SPM and RPM curves and the steepness of the initial change of the resistance, no clear therefore in total 100 resistance levels for every material stack and dependency on the applied voltage can be found, neither for the pair of SPM and RPM voltages. The different resistance levels are SPM nor the RPM. Concerning the a parameter for the fit SPM not evenly spaced. of the SPM, a clear separation between the different material Frontiers in Neuroscience | www.frontiersin.org 6 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 7 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 5 | (A) RPM of the Ta O devices with 1.6 V per pulse and different pulse lengths –1, 10 and 100 ms. (B) Resistance of the RPM plotted against the total 2 5 applied pulse length for three different pulse lengths with a pulse voltage of 1.6 V. (C) SPM curves of the Ta O devices with the resistance plotted against the total applied pulse length, with a pulse voltage of –2.5 V. stacks can be observed, where the Al interlayer samples show that can be used and the resistance change that one pulse triggers. the highest, Ta O intermediate and WO the lowest values. In In addition to the former described current adaptability of the 2 5 3 the fit of the RPM curves, a trend to increasing a parameters devices by their sizes, this gives the opportunity to adapt the pulse RPM with an increasing reset pulse voltage can be observed. However, length to the network requirements. the height of the increase is the largest for the Ta O interlayer 2 5 samples and comparably small for the WO interlayer samples. Discussion of the Impact of Pulse Height In conclusion for the fit parameters, in the most cases, a higher and Length on SPM/RPM SET voltage leads to a lower saturation resistance in the SPM As described in the previous section, the final ON/OFF ratio that and a higher RESET voltage to a higher saturation resistance in can be reached after saturation increases with the pulse height. the RPM. a , corresponding with the steepness of the initial SPM The tradeoff is to find a voltage that allows a high saturation increase of the SPM, depends mostly on the material and not on ON/OFF ratio but without reaching the saturation immediately, the SET voltage, while a , for the RPM steepness, depends on RPM which would lead to a binary synapse. With respect to the pulse the RESET voltage. length dependence (see Figure 5A), the reduction of the pulse length results in a smaller increase in resistance and the final Impact of Pulse Length on SET and ON/OFF ratio that can be reached after saturation increases only RESET Pulse Measurements with the pulse height and not with the pulse length. To avoid In order to study the impact of the pulse length on the shape switching the device completely with only one pulse, the voltage of the SPM and RPM curves, we vary the pulse length between and the applied pulse time has to be reduced. This way we can 1 ms and 100 ms. Figure 5A shows that using shorter pulses reach a high number of intermediate resistance states. for the RPM, the maximum reached resistance after 50 pulses In filamentary systems, the switching current is confined to the is smaller, similar to what we have observed for smaller pulse filament resulting in high current densities and self-heating up to amplitudes (see Figure 4B). Moreover, Figure 5A implies that 800 K (Menzel et al., 2011). The increase in temperature leads to a the total change of the resistance only depends on the total time to a self-enhanced, abrupt SET process. In area type switching, the of the applied pulses irrespective of the length of a single pulse. current is distributed over the whole area resulting in low current For example, for the application of a single pulse of 100 ms or 10 densities and a large dissipation area. Simulations confirm that pulses of 10 ms pulse length, the total applied time is the same self-heating is not important in area type devices, and we neglect and the observed resistance change is the same. Figure 5B shows its influence on the switching process (Menzel et al., 2019). As a the measured device resistance at –0.3 V, plotted against the total result, the velocity of the oxygen ions only depends on the force applied pulse length for the Ta O RPM curve starting in the LRS. of the applied electric field and the diffusion force. It does not 2 5 A clear trend of increasing resistance with increasing total applied depend on the length of the applied voltage pulse. pulse time can be seen. Also the measurements with the different We assume that the resistance changes with the amount of pulse lengths show a continuous behavior. For the Ta O devices, oxygen in the tunnel oxide and in the PCMO. An oxygen ion 2 5 this behavior is also depicted for the SPM curve in Figure 4C. transfers from the tunnel oxide into the PCMO (or vice versa) This proves that the total change in resistance is indeed only if it overcomes the distance to the interface between the two dependent on the total applied time for Ta O . This behavior is materials. If we apply an electric field, oxygen ions begin to move. 2 5 also observed for the Al and the WO devices (not shown here). The higher the distance each oxygen ion can travel, the more ions In summary, the combination of different pulse lengths and can move in total from the tunnel barrier to the PCMO (or vice pulse voltages makes the devices flexible in the resistance range versa) if the drift process takes place via vacancy sites. Therefore, Frontiers in Neuroscience | www.frontiersin.org 7 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 8 Gutsche et al. Neuronal Networks With PCMO Devices the change in resistance is directly related to the total distance an In order to determine whether a SET or RESET pulse must be oxygen ion has moved. applied to a ReRAM device, a form of gradient descent learning This distance is the time integral of the velocity that only using the backpropagation algorithm is employed. depends on the total time a voltage is applied as long as the In the forward pass, a sample image is presented to the input velocity itself is not a function of the pulse length. This should be of the network with the grayscale values of each pixel being the case if no Joule heating takes place. Therefore, the distance converted to an input voltage. By Ohm’s and Kirchhoff ’s Law, the oxygen can be moved does not depend on the number of this vector of input voltages is transformed to a vector of output pulses and their length. For example, one 100 ms pulse has the currents by the memristive weight matrix. A Rectified Linear Unit same effect as ten 10 ms pulses (see Figures 4B,C). (ReLU) function determines the input voltage to the next weight layer from these currents. The gradient for the gradient-descent algorithm is calculated PERCEPTRON LEARNING OF MNIST in the backward pass of the total network error E for every @E DATASET weight as . This calculation is executed within the TensorFlow @g i;j framework. The calculated gradients for every weight are The presented devices allow for usage with two different accumulated within each epoch. In every training epoch, a subset learning rules. The measurements presented in Section “Spike of 60,000 samples from the MNIST dataset is shown to the Timing Dependent Plasticity” resemble a synaptic STDP behavior network, resulting in a batch size of 60,000. Previous works by and therefore suggest the use of the proposed devices for Gao et al. (2020) showed that larger batches can lead to a better Hebbian style learning in a spiking neural network. The second recognition performance. With this large batch size, only one approach uses the stepwise resistance change in the pulse update cycle per epoch is performed. measurements presented in Section “Stepwise SET and RESET In the update operation of the PCMO devices we propose Pulse Measurements”. The gradual nature of the resistance here, either a gradual SET, RESET, or no pulse can be applied change can be exploited in an artificial neural network, which is to a device. The pulse height for SPM and RPM are fixed. trained by a gradient descent learning rule. This duality shows Therefore, only the sign of the accumulated gradient would the wide range of applications for the proposed devices in determine whether a device receives a SET or RESET pulse neuromorphic systems, as these learning rules differ significantly for update. With this sign update rule, an infinitesimal small in the scope of processed information (local comparison of gradient would have the same effect as a large gradient, what pre- and post-synaptic activity for Hebbian learning and global can be expected problematic for the training of the network. error minimisation for gradient descent) and the initial point Therefore, the set of updated conductances is restricted to only of their derivation (neurophysiology for Hebbian learning and the largest positive gradient and the smallest negative gradient in mathematical optimisation theory for gradient descent). In the every layer. Another benefit of such a very sparse update matrix following, an exemplary ANN trained by a gradient descent in a matrix-shaped weight layer is that it is much more time learning rule is shown. consuming to update a large number of conductances than to To investigate the use of PCMO resistive switching devices as infer the whole network. Using a matrix structure of the PCMO presented above as weights in an ANN, we conduct simulations devices, the inference of a complete layer takes one step, whereas of multilayer perceptrons in a TensorFlow (Abadi et al., 2016) the update of device is performed sequentially. The very low environment in Python. Furthermore, the impact of the different number of updated devices on the other hand leads to a large material stacks and the hyperparameters SET/RESET pulse number of learning epochs necessary to reach the maximum voltage and pulse length on the learning and recognition accuracy recognition accuracy. are analyzed. To compare and benchmark the results of our After each learning epoch, a validation subset of 10,000 network, the common dataset of hand-written digits MNIST samples from the MNIST dataset, which is different than the is used for training of the network and validation of the learning set is shown to the network, and the fraction of correctly recognition performance. recognized numbers is calculated as the recognition accuracy after this epoch. To investigate the impact of the PCMO-based Gradient-Descent Learning of the MNIST ReRAM devices in this network, we also performed a benchmark Dataset test with the described network structure and learning rule, but The investigated perceptron network consists of four layers of floating point weights instead of resistive switching weights. In neurons with the second and third being hidden. The input layer this benchmark, the network showed a maximum recognition has 784 neurons, the first hidden layer 250, the second hidden accuracy of 96.3%. layer 125 and the output layer 10 neurons. This structure is chosen to make the network comparable with similar memristive Results of Multilayer Perceptron networks in the literature. Similar to previous works, a matrix Simulations With PCMO-Based structure of PCMO-based ReRAM devices is assumed as the Memristive Switching Weights weight layer between two neuron layers (Xia and Yang, 2019). A more detailed description of the network can be found in the For all combinations of measured SPMs and RPMs, multilayer Supplementary Material. The weights are initialized randomly perceptron simulations are conducted for 6,000 epochs. from the range the employed SPM and RPM curves provide. As described in Section “Stepwise SET and RESET Pulse Frontiers in Neuroscience | www.frontiersin.org 8 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 9 Gutsche et al. Neuronal Networks With PCMO Devices FIGURE 6 | Results of the neural network simulations. (A) Recognition accuracy of the simulated network depending on the fitted ON/OFF ratio and the steepness of the SPM fit function. (B) Network recognition accuracy on a subset of the MNIST dataset. Red: Al interlayer. Blue: Ta O interlayer. Yellow: WO interlayer. (C) 2 5 3 Recognition accuracies of similar networks with ReRAM devices from literature depending on the ON-OFF ratio of the used ReRAM devices (Moon et al., 2015; Suri et al., 2015; Ambrogio et al., 2016; Babu et al., 2018; Fumarola et al., 2018; Go et al., 2019; Wu et al., 2020; Yin et al., 2020). (D) Learning curves for PCMO devices with Al and Ta O interlayers with 100 ms (red) and 1 ms update pulses. 2 5 Measurements,” the material stack of the PCMO devices and RPM. On the y-axis, the steepness value from the fit of the SPM choice of the voltage for the SET and RESET pulses in the pulse a is shown. This parameter is mostly dependent on the used SPM measurements lead to significantly different evolutions of the material stack (see Section “Stepwise SET and RESET Pulse resistance in these measurements. This also has an influence Measurements”). on the maximum accuracies that can be achieved using these The lowest accuracies resulting from the training simulations resistance curves for ANNs. The maximum accuracies of all can be found in the lower left corner for low ON/OFF ratio and simulations are plotted in Figure 6A. The x-axis shows the fitted low steepness of the SPM curve. Many of these simulations yield maximum resistance in the SPM and RPM curves. The higher accuracies below 70%, and for the data points in black no learning this value, the higher also the measured ON/OFF ratio in the at all with accuracies around 10% is even reached, which would pulse measurements is. As discussed in Section “Spike Timing be like a random drawing. However, for higher values of the a SPM Dependent Plasticity,” in most cases, a higher ON/OFF ratio parameter around 0.25, the maximum accuracy increases to 75% can be reached by choosing higher voltages for the SPM and to 80%, even for comparably low ON/OFF ratios. This means Frontiers in Neuroscience | www.frontiersin.org 9 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 10 Gutsche et al. Neuronal Networks With PCMO Devices that an initially steeper increase of the conductivity of a weight software. To achieve higher recognition accuracies, the ON/OFF leads to an increase in recognition accuracy. This observation ratio must be increased further. With an ON/OFF ratio of around appears contrary to previous observations that a more linear 30, Wu et al. (2020) showed an accuracy of about 95%. Even update behavior of the resistance is beneficial for the recognition higher accuracies can be reached using memristive devices as accuracy (Cai et al., 2020). Burr et al. (2017) attribute this to the storage for weights in a fashion of digital numbers instead of non-reversibility of a weight update that follows from a strong analog weights (Moon et al., 2015) (Babu et al., 2018). non-linearity. However, the difference here can be explained by As described in Section “Stepwise SET and RESET Pulse the different update rules. With the large batch size and sparse Measurements,” by varying the length of the applied pulses in update matrix used in this work, updates on the same device the SPM and RPM, short pulses lead to slower and longer mostly happen in the same direction and update pulses with pulses to a faster progression on the SPM or RPM curve, different polarities on the same device rarely occur. Therefore, respectively. Figure 6D shows a comparison of the learning the non-reversibility of a weight update is not an issue. of the MNIST dataset with long pulses of 100 ms (red) and Another path to higher accuracies is a higher ON/OFF ratio. short pulses of 1 ms (blue) using PCMO devices with Al and For a low a of around 0.1, the accuracy increases from low Ta O interlayers. Initially, for both interlayers, the accuracy SPM 2 5 to higher ON/OFF ratios. The same trend can also be found for for the network using the long pulses increases faster. In the high values of the RPM steepness factor a of about 0.4. The end, both saturate at about the same values, 82% for the Al SPM ON/OFF ratio has two effects here. Since all pulse measurements interlayer devices and 78% for the Ta O interlayer devices. 2 5 consist of an equal number of pulses, a higher ON/OFF ratio In the simulation with the latter devices, for the long pulses, means that the difference between the resistance steps is larger, as the recognition accuracy oscillates at high pulse numbers. This long as the curve reaches the saturation within the same number is not the case for the short pulses. In conclusion, the use of of pulses. On the other hand, a low ON/OFF ratio of, e.g., 3 means shorter update pulses can lead to a more stable, but slower that three OFF switched devices contribute the same activation to learning process. A gain in accuracy is not reached. However, a neuron in the subsequent layer as one ON switched device. For in conventional perceptron networks, the learning rate is an higher ON/OFF ratios, e.g., of 10, 10 devices can be in the OFF important factor for a successful learning and has a large impact state with one ON switched device still having a relatively high on the convergence and accuracy of the network. In this work, impact on the activation of the next neuron layer. The ability of we present one approach to implement a variable learning rate a device to differentiate the activation of the next neuron layer for resistive switching ReRAM devices by changing the pulse decreases with decreasing ON/OFF ratio. length. Such adaptive learning rates are not only beneficial for As described in Section 2.4, a parameter strongly depends artificial neural networks like perceptrons but can also be used in SPM on the material stack. Therefore, a separation between the brain-like learning systems to realize more biologically plausible materials can also be observed in Figure 6A, with the WO learning rules from neuroscience. interlayer samples for low, the Ta O interlayer samples for In conclusion for the neural network simulations, PCMO 2 5 intermediate and the Al interlayer samples for high values of ReRAM devices with Al, Ta O and WO interlayers can be used 2 5 3 a . For each material, the complete learning curves for those as weights in ANN learning to MNIST dataset. For devices with pot pulse measurement voltages resulting in the highest maximum an Al interlayer, the highest recognition accuracy of about 82% recognition accuracy are displayed in Figure 6B. After 6,000 could be achieved. A parameter optimisation showed how the training epochs, the network using the PCMO devices with the shape of the resistance evolution curve of pulse measurements Al interlayer reaches the highest recognition accuracy with 82%. affects the maximum accuracy. A high steepness of the SPM and The network using the Ta O interlayer devices, which initially the maximum ON/OFF ratio were identified as most important to 2 5 shows a faster increase of the recognition accuracy, exhibits a reach the highest accuracy values. While the steepness of the SPM saturation at a lower level of 76%. The WO interlayer devices depends mostly on the material stack used, the ON/OFF ratio can lead to a much lower learning speed but a similar recognition be maximized by choosing greater voltages for the SPM and RPM. accuracy of about 76%. Finally, the concept of a variable learning rate was implemented To benchmark the performance of the proposed network using different pulse lengths and the effect on the learning speed and memristive devices, the maximum accuracy reached can be and accuracy investigated. compared with similar networks. In Figure 6C, a comparison of previous works on ANNs with memristive devices training MNIST is provided separated by the maximum ON-OFF ratio of CONCLUSION the used devices on the x-axis. Most networks here have a similar layer and neuron-per-layer count as the network proposed in this In this work, we compared the performance of area-dependent work. The maximum accuracy of 82% reached in our simulations memristive PCMO devices with Al interlayer, where Al O is 2 3 can be found in the lower left corner of Figure 6C, which means formed naturally at the interface with devices where Ta O 2 3 a 14% higher error rate compared with the same network with and WO have been deposited intentionally. All investigated ideal floating point weights, as described above. It can be seen that devices show area-dependent switching and exhibit a STDP- this value is comparable with other networks using memristive like behavior. devices with a similar ON-OFF ratio, which is still lower than Furthermore, for all three types of devices, we performed accuracy values one would expect from conventional ANNs in SPMs and RPMs. The shape of the SPM and RPM curves differs Frontiers in Neuroscience | www.frontiersin.org 10 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 11 Gutsche et al. Neuronal Networks With PCMO Devices significantly for the different materials. In particular, the WO DATA AVAILABILITY STATEMENT stack showed a better linearity than the other two types of devices. Moreover, we showed that we can adapt the SPM and The raw data supporting the conclusions of this article will be RPM curves with the pulse parameters. By reducing the pulse made available by the authors, without undue reservation. height and the pulse length, we could adapt the step width of the resistance change and the ON/OFF ratio. Additionally, we showed that the amount of resistance change during the SPM or AUTHOR CONTRIBUTIONS RPM depends on the total time a voltage is applied irrespective of the number of pulses. All authors listed have made a substantial, direct and intellectual For the neural network simulations, the application of directly contribution to the work, and approved it for publication. deposited Ta O and WO layers does not lead to an increase 2 5 3 in recognition accuracy or increased learning speed compared with the Al interlayer devices despite of the better linearity FUNDING of the SPM and RPM curves of the WO . A hyperparameter optimisation shows the influence of the pulse lenght and height This work was supported by the DFG (German Science on the SPM and RPM curves and the influence of their shape Foundation) within the collaborative research center SFB on the maximum accuracy. The ON/OFF ratio and the SPM 917 “Nanoswitches” and by the Helmholtz Association Initiative and Networking Fund under project number SO-092 steepness are identified as the most crucial for high accuracies. Furthermore, it was shown that using shorter update pulses leads [Advanced Computing Architectures (ACA)] and the Federal to a slower initial increase of the recognition accuracy but a more Ministry of Education and Research (project NEUROTEC stable learning process, with less oscillations. Based on this, we grant no. 16ES1133K). propose a new approach to implement a variable learning rate for resistive switching ReRAM devices by changing the pulse length that might be interesting for perceptron networks in the future. SUPPLEMENTARY MATERIAL In conclusion, we demonstrated how two fundamentally different learning rules for neural networks, STDP in SNN and The Supplementary Material for this article can be found gradient descent learning in ANN, could be realized in the same online at: https://www.frontiersin.org/articles/10.3389/fnins. memristive devices. 2021.661261/full#supplementary-material Bagdzevicius, S., Maas, K., and Boudard, M. (2017). Interface-type resistive REFERENCES switching in perovskite materials. J. Electroceram. 39, 157–184. doi: 10.1007/ Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., et al. (2016). s10832- 017- 0087- 9 “TensorFlow: a system for large-scale machine learning,” in Proceedings of the Burr, G. W., Shelbya, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., et al. (2017). 12th USENIX Symposium on Operating Systems Design and Implementation, Neuromorphic computing using non-volatile memory. Adv. Phys. 2, 89–124. (Savannah, GA). Cai, F., Correll, J. M., Hwan Lee, S., Lim, Y., Bothra, V., Zhang, Z., Alibart, F., Zamanidoost, E., and Strukov, D. B. (2013). Pattern classification by et al. (2019). A fully integrated reprogrammable memristor– CMOS memristive crossbar circuits using ex situ and in situ training. Nat. Commun. system for efficient multiply–accumulate operations. Nat. Electron. 2, 4:2072. 290–299 Ambrogio, S., Balatti, S., and Milo, V. R Carboni, Z-Q Wang, A Calderoni, Cai, F., Correll, J. M., Lee, S. H., Lim, Y., Bothra, V., Zhang, Z., et al. (2020). A fully et al. (2016). Neuromorphic learning and recognition with one-transistor-one- integrated reprogrammable memristor- CMOS system for efficient multiply– resistor synapses and bistable metal oxide RRAM. IEEE Trans. Electron Devices accumulate operation. Nat. Electron. 2, 290–299. doi: 10.1038/s41928- 019- 63, 1508–1515. doi: 10.1109/ted.2016.2526647 0270- x Arndt, B., Borgatti, F., Offi, F., Phillips, M., Parreira, P., Meiners, T., et al. (2017). Chowdhury, A. P., Kulkarni, P., and Bojnordi, M. N. (2018). MB-CNN: memristive Spectroscopic indication of tunnel barrier charging as the switching mechanims binary convolutional neural networks for embedded mobile devices. J. Low in memristive devices. Adv. Func. Mater. 27:1702282. doi: 10.1002/adfm. Power Electron. Appl. 8:38. doi: 10.3390/jlpea8040038 201702282 Fumarola, A., Sidler, S., Moon, K., Jang, J., Shelby, R. M., Narayanan, P., et al. Asamitsu, A., Tomioka, Y., Kuwahara, H., and Tokura, Y. (1997). Current (2018). Bidirectional non-filamentary RRAM as an analog neuromorphic switching of resistive states in mgnetoresistive manganites. Nature 388, 50–52. synapse, part II: impact of Al/Mo/Pr0.7Ca0.3 MO device characteristics on doi: 10.1038/40363 neural network training accuracy. IEEE J. Electron Devices Soc. 6, 169–178. Asanuma, S., Akoh, H., Yamada, H., and Sawa, S. (2009). Relationship between doi: 10.1109/jeds.2017.2782184 resistive switching characteristics and band diagrams of TiÕPr1–xCaxMnO3 Gao, Y., Wu, S., and Adam, G. C. (2020). “Batch training for neuromorphic systems junctions. Phys. Rev. B 80:235113. with device non-idealities,” in Paper Presented at the International Conference on Babu, V., Lashkare, S., Ganguly, U., and Rajendran, B. (2018). Stochastic learning Neuromorphic Systems, Washington: DC. in deep neural networks based on nanoscale PCMO device characteristics. Go, Y., Kim, M., Kwak, J., Song, S., Chekol, A., Kwon, J.-D., et al. (2019). W/WO3– Neurocomputing 321, 227–236. doi: 10.1016/j.neucom.2018.09.019 x based three-terminal synapse device with linear conductance change and Backus, J. (1978). Can programming be liberated from the von neumann style? high on/off ratio for neuromorphic application. Appl. Phys. Express 12:026503. Commun. ACM 21, 613–641. doi: 10.1145/359576.359579 doi: 10.7567/1882- 0786/aafc74 Baek, K., Park, S., Park, J., and Hwang, H. (2017). In situ TEM observation on Herpers, A. (2014). Electrical Characterization of Manganite and Titanate the interface-type resistive switching by electrochemical redox reactions at a Heterostructures, Jülich: Forschungszentrum Jülich GmbH Zentalbibliothek. TiN/PCMO interface. Nanoscale 9, 582–593. doi: 10.1039/c6nr06293h Ph.D. thesis, Aachen: Verlag. Frontiers in Neuroscience | www.frontiersin.org 11 July 2021 | Volume 15 | Article 661261 fnins-15-661261 June 26, 2021 Time: 19:12 # 12 Gutsche et al. Neuronal Networks With PCMO Devices Jooss, L., Wu, T., Beetz, R., Klie, F., Beleggia, M., Schofield, M. A., et al. (2007). reactive metal/polycrystalline Pr0.7Ca0.3MnO3 device for nonvolatile memory Polaron melting and ordering as key mechanism for colossal resistance effects applications,” in Proceedings of the IEEE International Electron Devices Meeting, in manganites. Proc. Natl. Acad. Sci. U.S.A. 104, 13597–13602. doi: 10.1073/ (Baltimore, MD: IEEE). pnas.0702748104 Seong, D.-J., Hassan, M., Choi, H., Lee, J., Yoon, J., Park, J.-B., et al. (2009a). Krestinskaya, O., Choubey, B., and James, A. P. (2020). Memristive GAN in analog. Resistive-switching characteristics of Al/Pr0.7Ca0.3MnO3 for nonvolatile Sci. Rep. 10:5838. memory applications. IEEE Electron. Device Lett. 30:919–921. Liu, X., Kim, I., Siddik, M., Sadaf, S. M., Biju, K. P., Park, S., et al. (2011). Simmons, J. G., and Verderber, R. R. (1967). New conduction and reversible Resistive switching mechanism of a Pr0.7Ca0.3MnO3-based memory device memory phenomena in thin insulating films. Proc. R. Soc. Lond. A Math. Phys. and assessment of its suitability for nano-scale applications. J. Korean Phys. Soc. Sci. 301, 77–102. doi: 10.1098/rspa.1967.0191 59, 497–500. doi: 10.3938/jkps.59.497 Suri, M., Parmar, V., Kumar, A., Querlioz, D., and Alibart, F. (2015). Menzel, S., von Witzleben, M., Havel, V., and Böttger, U. (2019). The ultimate “Neuromorphic hybrid RRAM_CMOS RBM architecture,” in Proceedings of the switching speed limit of redox-based resistive switching devices. Faraday 15th Non-Volatile Memory Technology Symposium, Beijing. Discuss. 213, 197–213. doi: 10.1039/c8fd00117k Tsymbal, E. Y., and Kohlsted, H. (2006). Tunneling across a ferroelectric. Science Menzel, S., Waters, M., Marchewka, A., Böttger, U., Dittmann, R., and Waser, R. 5784, 181–183. doi: 10.1126/science.1126230 (2011). Origin of the ultra-nonlinear switching kinetics in oxide-based resistive Waser, R., Dittmann, R., Staikov, G., and Szot, K. (2009). Redox-based resistive switches. Adv. Func. Mater. 21, 4487–4492. doi: 10.1002/adfm.201101117 switching memories - nanoionic mechanims, prospects, and challenges. Adv. Moon, K., Cha, E., Park, J., Chu, S. M., Gi, H., Wang, H., et al. (2015). “High Mater. 21, 2632–2663. doi: 10.1002/adma.200900375 density neuromorphic system with Mo/Pr0.7Ca0.3MnO3 synapse and NbO2 Wolf, W. A., and McKee, S. A. (1994). Hitting the memory wall: implications IMT oscillator neuron,” in Proceedings of the IEEE International Electron Devices of the obvious. Dep. Comput. Sci. 23, 20–24. doi: 10.1145/216585.21 Meeting, (Washington, DC: IEEE), 463–466. 6588 Moon, K., Park, S., Lee, D., Woo, J., Cha, E., Lee, S., et al. (2014). “Resistive- Wu, F., Mo, T., Saraya, T., Hiramoto, T., and Kobayashi, M. (2020). A monolithic switchiing analogue memory device for neuromorphic applications,” in 3-D integration of RRAM array and oxide semiconductor FET for in-memory Proceedings of the 2014 Silicon Nanoelectronics Workshop, (Honolulu, HI: computing in 3-D neural network. IEEE Tran. Electron Devices 67, 5322–5328. IEEE). doi: 10.1109/ted.2020.3033831 Park, S., Chu, M., Kim, J., Noh, J., Jeon, M., Lee, B. H., et al. (2015). Electronic Xia, Q., and Yang, J. J. (2019). Memristive crossbar arrays for brain-inspired. Nat. system with memristive synapses for pattern recognition. Sci. Rep. 5:10123. Mater. 18, 309–323. doi: 10.1038/s41563- 019- 0291- x Park, S., Kim, H., Choo, M., Noh, J., Sheri, A., Jung, S., et al. (2012). “RRAM- Yakopcic, C., Alom, M. Z., and Taha, T. M. (2017). “Extremely parallel memristor based synapses for neuromorphic system with pattern recognition function,” in crossbar architecture,” in Proceedings of the International Joint Conference on Proceedings of the 2012 International Electron Devices Meeting, (San Francisco, Neural Networks, (Anchorage, AK: IEEE). CA: IEEE), 231–234. Yao, P., Wu, H., Gao, B., Tang, J., Zhang, Q., Zhang, W., et al. (2020). Fully Park, S., Sheri, A., Kim, J., Noh, J., Jang, J., Jeon, M., et al. (2013). “Neuromorphic hardware-implemented memristor convolutional neual network. Nature 577, Speech systems using advanced ReRAM-based synapses,” in Proceedings of the 641–646. doi: 10.1038/s41586- 020- 1942- 4 2013 IEEE International Electron Devices Meeting, (Washington, DC: IEEE), Yin, L., Cheng, R., Wang, Z., Wang, F., Sendeku, M. G., Wen, Y., et al. (2020). Two- 625–628. dimensional unipolar memristors with logic and memory functions. Nano Lett. Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., and 20, 4144–4152. doi: 10.1021/acs.nanolett.0c00002 Strukov, D. B. (2015). Training andoperation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64. doi: 10.1038/ Conflict of Interest: The authors declare that the research was conducted in the nature14441 absence of any commercial or financial relationships that could be construed as a Sawa, A. (2006). Interface resistance switching at a few nanometer thick perovskite potential conflict of interest. manganite active layers. Appl. Phys. Lett. 88:232112. doi: 10.1063/1.2211147 Sawa, A. (2008). Resistive switching in transition metal oxides. Mater. Today 11, Copyright © 2021 Gutsche, Siegel, Zhang, Hambsch and Dittmann. This is an open- 28–36. doi: 10.1016/s1369- 7021(08)70119- 6 access article distributed under the terms of the Creative Commons Attribution Sawa, A., Fujii, T., Kawasaki, M., and Tokura, Y. (2004). Hysteretic current–voltage License (CC BY). The use, distribution or reproduction in other forums is permitted, characteristics and resistance switching at a rectifying Ti/Pr0.7Ca0.3MnO3 provided the original author(s) and the copyright owner(s) are credited and that the interface. Appl. Phys. Lett. 85:4073. doi: 10.1063/1.1812580 original publication in this journal is cited, in accordance with accepted academic Seong, D.-J, Park, J., Lee, N., Hasan, M., Jung, S., Choi, H., et al. (2009b). “Effect of practice. No use, distribution or reproduction is permitted which does not comply oxygen migration and interface engineering on resistance switching behavior of with these terms. Frontiers in Neuroscience | www.frontiersin.org 12 July 2021 | Volume 15 | Article 661261

Journal

Frontiers in NeurosciencePubmed Central

Published: Jul 2, 2021

References