Home / Remote Arduino Deployments via Raspberry Pi / Wiring the Arduino and Raspberry Pi

Setting Up Deployments

With both a means to communicate data and reliably reset the Arduino, we are ready to actually push firmware (a sketch) to the Arduino from the Raspberry Pi.

The initial deployment will be a manual process to ensure everything works before automating the process with Python.

Create A Test Sketch

Create a new test sketch or copy the testSketch.ino file from the following GitHub repository:
github.com/chriscummings/remote-arduino-deployment

arduino-cli sketch new testSketch
cd testSketch
nano testSketch.ino

Paste the following C code into that file.

void setup()
{
	Serial.begin(9600);
	pinMode(13, OUTPUT);
}

void loop()
{
	Serial.println("Hello");
	digitalWrite(13, HIGH); // LED on
	delay(500);
	digitalWrite(13, LOW); // LED off
	delay(500);
}

In an Arduino sketch, before any other code executes, the setup() function is called once. Documentation for setup()

In this example the setup() function:

Sets the baud rate of the serial connection to 9600. Documenation for Serial.begin().
Configures pin 13 to be an output. Documentation for pinMode().

After the setup() function runs, the loop() function is called and continues to repeat until the program is explicitly exited or crashes. Documentation for loop.

In this example the loop() function repeats the following actions:

Prints Hello to the serial connection. Documentation for Serial.println()
Turns pin 13 "on" applying a 3.3V current. Documentation for digitalWrite()
Waits 500 milliseconds. Documentation for delay()
Turns pin 13 "off" removing current.
Waits 500 milliseconds.

What's special about pin 13? There is an onboard LED connected to it so that whenever 13 is "on" or "high" that LED lights up. It is commonly used for testing purposes because of this. Thus, every run of the loop() function causes this LED to blink on and off.

Manually Deploy the Test Sketch

Compile the Sketch

The entire sketch folder (YOUR_SKETCH_NAME).ino that contains all the source code must be compiled into a single .hex file using the compile command.

arduino-cli compile -b arduino:avr:uno testSketch.ino -v

Documentation for the compile command.

The -b flag (alternatively the --fqbn flag) defines the fully qualified board name of the attached microcontroller.

The -v flag enables verbose output. You will need to scan this output for the name of the generated destination folder containing the compiled testSketch.ino.hex file. The path will look like:
/tmp/arduino/sketches/(SOME_ARBITRARY_HASH)/testSketch.ino.hex

Compiled .hex File Found in Verbose Output

Flash the Arduino

Most guides online will suggest using arduino-cli to flash the Arduino (arduino-cli just calls AVRDUDE under the hood after all). However, this is apt to fail because arduino-cli times out very quickly. To avoid this, just call AVRDUDE yourself.

Note: Command line parameter flags are case-sensitive and the same character is often re-used for unrelated functions. For example the AVRDUDE -c and -C flags.

avrdude -e -F -c arduino -p m328p -U flash:w:(YOUR_PATH_HERE)/testSketch.ino.hex

AVRDUDE documentation
AVRDUDE (6.4) parameter options-specific documentation

The -e flag ensures that the chip memory is erased prior to flashing. Not every microcontroller requires this but explicitly erasing the memory will also side-step complications related to a previously botched flash attempt.

The -F flag skips a device signiture verification step. This parameter may be optional for you but I found it necessary.

The -c flag sets the programmer-id to be used. This is board-dependant.

The -p flag defines the part number of the destination board.

The -U flag defines what memory operation is to be performed by AVRDUDE. The format of this value must be:
(MEMORY_TYPE):(OPERATION):(FILENAME)
Our MEMORY_TYPE is flash, OPERATION is w for write, and FILENAME is the path to your .hex file.

Possible AVRDUDE Config File Error

You might get this command to work as described but I have found that, for now, an additional config file argument must also be passed. Otherwise the following error occurs:

avrdude: can't open config file […]

I believe this is error with either the current AVRDUDE or arduino-cli package that will eventually be fixed. For now, you'll need to download this file yourself and reference it in your call to AVRDUDE. The file can be found on the official Arduino GitHub repository for arduino-flash-tools. I've also provided a (possibly deprecated) copy in the remote-arduino-deployment GitHub repo.

If needed, add the config-file paramter -C with the path to avrdude.conf.

avrdude -c arduino -p m328p -U flash:w:(YOUR_PATH_HERE)/testSketch.ino.hex -C ~/avrdude.conf

Possible Port Error

If you run into the error, avrdude: ser_open(): can't open device you need to specify your desired port with the -P argument.

GPIO Pin Port: /dev/ttyACM0
USB Port: /dev/ttyS0

avrdude -c arduino -p m328p -U flash:w:(YOUR_PATH_HERE)/testSketch.ino.hex -C ~/avrdude.conf -P /dev/ttyACM0

An Expected Sync Error

This failure is expected. For the Arduino's boot loader to accept an upload, it must be physically reset. You can hold down the Arduino’s reset button, submit the command in the terminal and immediately release the reset button to achieve this manually.

You have to get the timing right because the boot loader only accepts uploads for a short duration after a reset.

Now that we've proven everything works, we can automate this whole process.

Automating the Deployment Process with Python

Note: You can download the following files from
github.com/chriscummings/remote-arduino-deployment

Shared Files

.env

On GitHub: example.env

RPI_IP='192.168.0.42'
RPI_PORT=22
RPI_USER='YOUR_USER'
RPI_PASS='YOUR_PASS'

constants.py

On GitHub: constants.py

import os

# Shared constants
# ==============================================================================
# These constants are used by both deploy.py and flash.py

# User directory on RPi
RPI_USER_DIR: str = '/home/YOUR_USER'

# Project directory on RPi
PROJECT_DIR: str = '/home/YOUR_USER/demeter'

# Path to arduino-cli on RPi
RPI_ARDUINO_CLI: str = os.path.join(RPI_USER_DIR, 'bin/arduino-cli')

# Deploy script constants
# ==============================================================================
# These constants are only used by deploy.py

# RPi directory for source files
SKETCH_DESTINATION_DIRECTORY: str = PROJECT_DIR

# Arduino libaries to download to RPi
SKETCH_LIBS: list[str] = [
	'WatchDog',
	# etc...
]

# Sketch source files.
# Feel free to change this to a glob command or something similar so long as it
# return a list or strings. However, be sure to include constants.py or migrate
# the constants over some how.
SKETCH_FILES: list[str] = [
	'contants.py' # Copy this constants file to RPi for ease of updates
	'my_sketch.ino',
	'my_sketch.h',
	# etc...
]

# Path flash.py on RPi
RPI_FLASH_SCRIPT: str = os.path.join(RPI_USER_DIR, 'flash.py')

# Local (deploy) SSH log file
LOCAL_SSH_LOG: str = 'ssh.log'

# RPi-side-only flash script constants
# ==============================================================================
# These constants are only used byt flash.py

# Arduino-specific params for arduino-cli.
# See: https://avrdude.readthedocs.io/en/latest/2-Command_Line_Options.html
ARDUINO_CORE = 'arduino:avr:uno'
AVRDUDE_PROGRAMMER_ID = 'arduino'
AVRDUDE_PART_NO = 'm328p'

# Default output for arduino-cli compiled .hex files on RPi
COMPILED_SKETCH_PATH = '/tmp/arduino/sketches/'

# Path to avrdude on RPi
RPI_AVRDUDE = os.path.join(
	RPI_USER_DIR,
	'.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/bin/avrdude')

# Path to avrdude.conf on RPi
AVRDUDE_CONF = os.path.join(RPI_USER_DIR, 'avrdude.conf')

# RPi GPIO pin used to pull Arduino's reset pin
GPIO_RESET_PIN = 4

# Duration to 'hold' Arduino's reset
ARDUINO_RESET_INTERVAL = 1

Getting Code on to the Pi

deploy.py

On GitHub: deploy.py

import os
from dotenv import load_dotenv
import paramiko
import constants


conn: paramiko.client.SSHClient

def setup_connection() -> paramiko.client.SSHClient:
	'''Configures SSH connection'''
	paramiko.util.log_to_file(constants.LOCAL_SSH_LOG)
	ssh: paramiko.client.SSHClient = paramiko.client.SSHClient()
	ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
	ssh.connect(
		os.getenv('RPI_IP'),
		port=int(os.getenv('RPI_PORT', '')),
		username=os.getenv('RPI_USER'),
		password=os.getenv('RPI_PASS'))
	return ssh

def run_rpi_cmd(cmd: str) -> str:
	'''Executes cmd on RPi'''
	stdin, stdout, stderr = conn.exec_command(cmd)
	stdout.channel.set_combine_stderr(True)
	output = '\n'.join(stdout.readlines()) # Waits for command to finish
	return output

def teardown_connection() -> None:
	'''Cleans up open SSH connections'''
	if conn:
		conn.close()

def transfer_src() -> None:
	'''Copies source code to RPi'''
	# Setup SFTP conn
	transport: paramiko.transport.Transport = paramiko.Transport(
		os.getenv('RPI_IP'),
		int(os.getenv('RPI_PORT', '')))
	transport.connect(None,
		os.getenv('RPI_USER'),
		os.getenv('RPI_PASS'))
	sftp: paramiko.sftp_client.SFTPClient = paramiko.SFTPClient.from_transport(transport)
	try:
		# Ensure directory
		run_rpi_cmd(f'mkdir {constants.SKETCH_DESTINATION_DIRECTORY}')
		# Copy files
		for sketch_file in constants.SKETCH_FILES:
			sftp.put(
				os.path.abspath(sketch_file),
				os.path.join(constants.SKETCH_DESTINATION_DIRECTORY, sketch_file))
	except Exception as err:
		print(err) # TODO: log
	finally:
		if sftp: sftp.close()
		if transport: transport.close()

def fetch_sketch_libs() -> None:
	'''Downloads sketch libraries to RPi'''
	output = run_rpi_cmd(f'{constants.RPI_ARDUINO_CLI} lib list')
	for lib in constants.SKETCH_LIBS:
			print(f'Checking for required sketch library {lib}')
			if lib not in output:
				print(f'Installing {lib}')
				output = run_rpi_cmd(f'{constants.RPI_ARDUINO_CLI} lib install "{lib}"')
				print(output)

def compile_and_flash() -> None:
	'''Compiles sketch on RPi and flashes Arduino'''
	output = run_rpi_cmd(f'python3 {constants.RPI_FLASH_SCRIPT} {constants.SKETCH_DESTINATION_DIRECTORY}')
	print(output)

def deploy() -> None:
	conn = setup_connection()
	try:
		transfer_src()
		fetch_sketch_libs()
		compile_and_flash()
	except Exception as err:
		print(err)
	finally:
		teardown_connection()

if __name__ == '__main__':
	load_dotenv()
	deploy()

Compiling and Flashing Firmware

flash.py

On GitHub: flash.py

import os
import sys
import time
import subprocess
from glob import glob
import RPi.GPIO as GPIO
import contants


def configure_gpio_reset_pin() -> None:
	'''Sets up RPi GPIO pin for Arduino reset'''
	GPIO.setwarnings(False)
	GPIO.setmode(GPIO.BCM)
	GPIO.setup(contants.GPIO_RESET_PIN, GPIO.OUT)

def delete_sketches() -> None:
	'''Delete old compiled sketches'''
	subprocess.run(['rm', '-rf', os.path.join(contants.COMPILED_SKETCH_PATH, '*')])

def get_ino_path() -> str:
	'''Returns first .ino found. Assumes only one .ino!'''
	# First param is src folder
	target_dir: str = sys.argv[1]
	ino_filepath = glob(os.path.join(target_dir, '*.ino'))[0]
	if not ino_filepath: raise Exception(f'No .ino file found in {target_dir}')
	return ino_filepath

def compile(ino_filepath: str) -> None:
	'''Runs arduino-cli compilation'''
	subprocess.run([
		contants.RPI_ARDUINO_CLI,
		'compile',
		'-b', contants.ARDUINO_CORE,
		ino_filepath])

def get_compiled_sketch_path() -> str:
	'''Returns first compiled .hex found.'''
	# Assumes previous copilations have been deleted!
	path: str = ''
	for filepath in glob(os.path.join(contants.COMPILED_SKETCH_PATH, '*', '*')):
			root, ext = os.path.splitext(filepath)
			if ext == '.hex' and 'bootloader' not in root:
					path = filepath
					break
	if path == '': raise Exception(f'No .hex found in {contants.COMPILED_SKETCH_PATH}')
	return path

def reset_arduino() -> None:
	'''Hard resets Arduino'''
	GPIO.output(contants.GPIO_RESET_PIN, GPIO.LOW)
	time.sleep(contants.ARDUINO_RESET_INTERVAL)
	GPIO.output(contants.GPIO_RESET_PIN, GPIO.HIGH)

def flash(hex_file: str) -> None:
	'''Runs AVRDUDE cmd to flash Arduino'''
	subprocess.run([
		contants.RPI_AVRDUDE,
		'-c', contants.AVRDUDE_PROGRAMMER_ID,
		'-p', contants.AVRDUDE_PART_NO,
		'-C', contants.AVRDUDE_CONF,
		'-U', f'flash:w:{hex_file}'])

def run():
	configure_gpio_reset_pin()
	delete_sketches()
	compile(get_ino_path())
	flash(get_compiled_sketch_path())
	reset_arduino()

if __name__ == "__main__":
	run()