12. Requesting and receiving data from Arduino

---

import time

import numpy as np

import serial
import serial.tools.list_ports

import bokeh.plotting
import bokeh.io
bokeh.io.output_notebook()

Loading BokehJS ...

---

In the previous lessons, we learned how to send a signal to Arduino from Python to ask it to do something, in that case to turn on an LED. Now, we will ask Arduino to send data back. We will store the data in RAM (or we could write it to disk) for later use.

We will read in a voltage that we control with a potentiometer. The idea is that we send a request from Python to Arduino, and then it reads the voltage and sends it back, with all communication over USB. You should work in your own Jupyter notebook for this.

To begin, wire up the following schematic.

Upload the following sketch.

const int voltagePin = A0;

const int HANDSHAKE = 0;
const int VOLTAGE_REQUEST = 1;

int inByte;


void printVoltage() {
  // Read value from analog pin
  int value = analogRead(voltagePin);

  // Get the time point
  unsigned long time_ms = millis();

  // Write the result
  if (Serial.availableForWrite()) {
    String outstr = String(String(time_ms, DEC) + "," + String(value, DEC));
    Serial.println(outstr);
  }
}


void setup() {
  // Initialize serial communication
  Serial.begin(115200);
}


void loop() {
  // Check if data has been sent to Arduino and respond accordingly
  if (Serial.available() > 0) {
    // Read in request
    inByte = Serial.read();

    // If data is requested, fetch it and write it, or handshake
    switch(inByte) {
      case VOLTAGE_REQUEST:
        printVoltage();
        break;
      case HANDSHAKE:
        if (Serial.availableForWrite()) {
          Serial.println("Message received.");
        }
        break;
    }
  }
}

This sketch warrants some explanation.

• I have written a function printVoltage() that handles reading in the voltage on the appropriate pin, adding a time stamp, and then printing the result to the serial connection. Note that I have used String objects to write a comma-delimited string, separating the time and the voltage. I have chosen not to convert the time to seconds nor the 10-bit integer from the analogRead() to volts on Arduino. I usually make this choice; conversions on the Python side are faster, and it is easier to send integers.

• The DEC argument in String() specifies the format of the number as decimal, as opposed to, e.g., HEX for hexidecimal. It is not necessary (DEC is the default), but I left it in there because it is better to be explicit than implicit.

• The structure of the loop() function is much the same as we have seen, except that we conveniently call the printVoltage() function. Writing modular code is always a good idea.

We’ll use the same global variable names on the Python side.

HANDSHAKE = 0
VOLTAGE_REQUEST = 1

And, of course, we have our usual connectivity functions.

def find_arduino(port=None):
    """Get the name of the port that is connected to Arduino."""
    if port is None:
        ports = serial.tools.list_ports.comports()
        for p in ports:
            if p.manufacturer is not None and "Arduino" in p.manufacturer:
                port = p.device
    return port


def handshake_arduino(
    arduino, sleep_time=1, print_handshake_message=False, handshake_code=0
):
    """Make sure connection is established by sending
    and receiving bytes."""
    # Close and reopen
    arduino.close()
    arduino.open()

    # Chill out while everything gets set
    time.sleep(sleep_time)

    # Set a long timeout to complete handshake
    timeout = arduino.timeout
    arduino.timeout = 2

    # Read and discard everything that may be in the input buffer
    _ = arduino.read_all()

    # Send request to Arduino
    arduino.write(bytes([handshake_code]))

    # Read in what Arduino sent
    handshake_message = arduino.read_until()

    # Send and receive request again
    arduino.write(bytes([handshake_code]))
    handshake_message = arduino.read_until()

    # Print the handshake message, if desired
    if print_handshake_message:
        print("Handshake message: " + handshake_message.decode())

    # Reset the timeout
    arduino.timeout = timeout

And now, let’s make a connection. We’ll leave it open through the exercise and close it at the end.

port = find_arduino()
arduino = serial.Serial(port, baudrate=115200)
handshake_arduino(arduino, handshake_code=HANDSHAKE, print_handshake_message=True)

Handshake message: Message received.

Let’s ask for data from Arduino and receive it. We will again use arduino.read_until(), forcing reading until a newline character is encountered. This makes Python wait until Arduino has finished sending the message (or until the timeout, which is one second by default).

# Ask Arduino for data
arduino.write(bytes([VOLTAGE_REQUEST]))

# Receive data
raw = arduino.read_until()

# Look at what we got
raw

b'20423,0\r\n'

The b in front of the string signifies that the string is a bytes literal. This is not a Python string, but a representation of an ASCII encoded set of characters as binary. To convert it to a string, we need to use the decode() method.

raw_str = raw.decode()

# Take a look
raw_str

'20423,0\r\n'

Now it is a Python string that we can convert to two integers. The carriage return (\r) and the newline (\n) should be stripped off the end, and we split the string at the comma.

t, V = raw_str.rstrip().split(",")

# Take a look
t, V

('20423', '0')

We now have two strings, the time and the voltage. We can convert these to integers and then to whatever units we like. I’ll leave time in milliseconds and convert the voltage to volts.

t = int(t)
V = int(V) * 5 / 1023

# Take a look
t, V

(20423, 0.0)

Since modular programming is a good idea, let’s write this parsing into a function.

def parse_raw(raw):
    """Parse bytes output from Arduino."""
    raw = raw.decode()
    if raw[-1] != "\n":
        raise ValueError(
            "Input must end with newline, otherwise message is incomplete."
        )

    t, V = raw.rstrip().split(",")

    return int(t), int(V) * 5 / 1023

Finally, we can write a function to acquire a data point from the Arduino.

def request_single_voltage(arduino):
    """Ask Arduino for a single data point"""
    # Ask Arduino for data
    arduino.write(bytes([VOLTAGE_REQUEST]))

    # Read in the data
    raw = arduino.read_until()

    # Parse and return
    return parse_raw(raw)

We can call the function to get single time, voltage pairs.

request_single_voltage(arduino)

(23638, 0.0)

We could set up lists containing the time and voltages we acquire and use the functions to get them programmatically, say every 20 ms. I’ll run the following code cell while twisting the potentiometer knob back and forth.

time_ms = []
voltage = []

for i in range(400):
    # Request and append
    t, V = request_single_voltage(arduino)
    time_ms.append(t)
    voltage.append(V)

    # Wait 20 ms
    time.sleep(0.02)

We can make a plot of the results.

time_s = np.array(time_ms) / 1000

p = bokeh.plotting.figure(
    x_axis_label='time (s)',
    y_axis_label='voltage (V)',
    frame_height=175,
    frame_width=500,
    x_range=[time_s[0], time_s[-1]],
)
p.line(time_s, voltage)

bokeh.io.show(p)

As it turns out, this is not the preferred method to stream data from Arduino, though it works fine if you are sampling at much longer time intervals. The call-and-response and the use of Python’s time.sleep() to do the delays leads to inconsistent timing. We can see this by computing the time difference between subsequent data points.

np.diff(time_ms)

array([27, 28, 24, 25, 25, 23, 24, 29, 29, 27, 30, 26, 26, 24, 26, 28, 29,
       28, 23, 28, 27, 22, 28, 28, 26, 25, 25, 28, 27, 27, 25, 25, 26, 25,
       29, 29, 25, 27, 24, 27, 28, 27, 31, 29, 24, 29, 26, 27, 29, 30, 26,
       30, 25, 26, 24, 26, 30, 24, 29, 30, 25, 28, 26, 26, 30, 28, 26, 25,
       27, 27, 31, 29, 27, 26, 26, 31, 29, 27, 27, 25, 24, 28, 25, 24, 26,
       27, 25, 29, 27, 29, 29, 29, 29, 24, 29, 26, 25, 27, 29, 26, 23, 24,
       28, 26, 25, 24, 24, 25, 28, 27, 26, 28, 28, 26, 28, 31, 28, 27, 31,
       26, 24, 28, 27, 28, 28, 28, 28, 29, 28, 27, 26, 29, 27, 31, 29, 27,
       30, 29, 25, 28, 28, 24, 28, 26, 29, 29, 28, 27, 27, 24, 28, 25, 28,
       26, 25, 26, 24, 27, 26, 23, 28, 29, 26, 27, 30, 29, 27, 27, 26, 26,
       29, 27, 30, 29, 26, 23, 29, 28, 29, 29, 29, 28, 27, 23, 27, 30, 29,
       27, 30, 26, 25, 30, 27, 23, 27, 26, 26, 28, 28, 29, 26, 25, 25, 23,
       28, 25, 24, 30, 27, 30, 25, 28, 24, 30, 23, 29, 29, 25, 28, 25, 28,
       26, 26, 29, 25, 30, 28, 28, 27, 27, 29, 27, 28, 23, 29, 24, 29, 28,
       25, 28, 30, 24, 27, 29, 27, 28, 26, 27, 29, 27, 26, 25, 28, 28, 31,
       25, 26, 30, 28, 30, 29, 26, 31, 29, 27, 28, 28, 22, 29, 25, 29, 29,
       28, 27, 27, 28, 29, 26, 23, 27, 31, 26, 30, 28, 28, 29, 22, 25, 24,
       28, 29, 28, 25, 28, 27, 26, 31, 28, 29, 25, 28, 25, 25, 28, 25, 28,
       30, 24, 24, 29, 27, 31, 29, 24, 27, 31, 25, 28, 25, 27, 30, 24, 29,
       28, 26, 23, 27, 27, 24, 29, 29, 24, 27, 29, 24, 23, 28, 25, 29, 26,
       26, 30, 25, 28, 29, 29, 24, 29, 29, 28, 29, 28, 29, 27, 26, 26, 29,
       23, 29, 30, 24, 28, 24, 30, 29, 23, 29, 28, 28, 28, 26, 27, 28, 29,
       23, 25, 25, 25, 25, 24, 27, 30, 29, 25, 24, 27, 23, 25, 24, 26, 27,
       24, 26, 26, 30, 28, 30, 25, 26])

This is not the 20 ms we wanted, and the time between acquisitions jumps around a lot. In future lessons, we will cover how to get better streaming data from Arduino. For now, you will code up a dashboard for a more commonly used mode of data-upon-request, where you hit a button and get data back.

---

Do-it-yourself exercise 5: Asking for data with buttons

Using the same setup as we have so far in this lesson, make a small dashboard with two buttons. One of them blanks out lists time_ms and voltage, while the other one asks for a time, voltage pair from Arduino and appends them to the lists. You can embed them in the notebook with either Panel or base Bokeh. Since you would presumably be using the data right away, leave them in the notebook and not as a stand along Bokeh app.

Use your button to ask for ten or more data points while messing around with the potentiometer knob and then make a plot of the result.

Hint: Check out Panel’s Reference Gallery. A Button is different that a Toggle. If you use a button, there is no on_change method, but rather an on_click method.

Another hint: Say I have two widgets, widget_1 and widget_2. To lay them out, you can use, e.g., pn.Row(widget_1, pn.Spacer(width=15), widget_2).

---

arduino.close()

Computing environment

%load_ext watermark
%watermark -v -p serial,bokeh,jupyterlab

CPython 3.8.5
IPython 7.18.1

serial 3.4
bokeh 2.2.1
jupyterlab 2.2.6