Bald Wisdom

I’m a big fan of Scratch, the educational programming language created at MIT. International Scratch Day is coming up, and when we went two years ago, we had a blast. Last time, I added a distance sensor to an Arduino, made it emulate a Scratch Sensor Board, and  put it under a trampolene. This year, I decided to document the general process of making your own scratch sensors and bring a compact sensor module for the kids to play with.

At one point during our research at work, we needed a way to measure the movement and relative distance of a camera platform, so I designed, and my tech put together a combination of a Teensy (Arduino Compatible microcontroller), an Adafruit ADXL335 Accelerometer breakout, and a Sharp IR distance sensor.

I dug up my old code for the trampolene hack, which was based on some code I got off the Scratch forums. It didn’t work!

What we are actually doing is emulating what was originally called a “Scratchboard” and now produced by a company called the Playful invention company and called a “PicoBoard”.

There’s a spec (and schematics) at http://info.scratch.mit.edu/sites/infoscratch.media.mit.edu/files/file/ScratchBoard_Tech_InfoR2.pdf

Turns out, the platform used to be more forgiving, and now it expects a report of all the sensors (whether we are using them or not.)

The IR distance Sensor outputs an analog voltage proportional to the distance to the nearest thing in it’s path, and likewise the accelerometer outputs a voltage proportional to the acceleration (including gravity). I mapped the distance sensor to the “slider” sensor, and the X, Y and Z outputs to the Resistance A, B and C sensors.

Here’s a demo video:

I’m planning on re-working the interface code into an Arduino Library, and I’ll post that here when I get it done.

In the meantime here’s the Arduino code:

// Scratchboard emulation
// by Kevin Osborn
// http://baldwisdom.com
// Feel free to use this code in any way you wish! (Though if you want to link
// to my blog, that would be cool)
// The Scratchmote has a sharp ir distance sensor
// and a 3 axis accellerometer

#define DISTSENSE A0
#define ACCELX    A1
#define ACCELY    A2
#define ACCELZ    A3

#define TESTMODE  0

// Format output for ScratchBoard emulation
// sensor=0-7, value=0-1023 (rescaled by Scratch to 0-100)
// 0="A", 1="B", 2="C", 3="D",
// 4="Slider", 5="Light", 6="Sound", 7="Button"
void ScratchBoardSensorReport(int sensor, int value)
{
  Serial.write( B10000000
                 | ((sensor & B1111)<>7) & B111));
  delayMicroseconds(400);
  Serial.write( value & B1111111);
}
int flatx,flaty,flatz; // might be used to set midpoint
int maxX,minX; //testing purposes
int lastx,lasty,lastz;

void setup()
{
  pinMode(DISTSENSE, INPUT);
  pinMode(ACCELX,INPUT);
  pinMode(ACCELY,INPUT);
  pinMode(ACCELZ,INPUT);

  lastx =maxX= minX = flatx=analogRead(ACCELX);
  lasty = flaty=analogRead(ACCELY);
  lastz = flatz=analogRead(ACCELZ);

  Serial.begin(38400);
}

void loop()
{
  int distance = abs(map(analogRead(DISTSENSE),0,650,0,1023)-1023);
  //experimenting with the ADXL335 gives us readings between low single digits
  // and < 700 for human powered movements   int x = map(analogRead(ACCELX),0,700,0,1023);   int y = map(analogRead(ACCELY),0,700,0,1023);   int z = map(analogRead(ACCELZ),0,700,0,1023);      int rx = (x +lastx)/2;   int ry = (y +lasty)/2;   int rz = (z +lastz)/2;      lastx = rx;   lasty = ry;   lastz = rz; #if TESTMODE Serial.println(distance); /* Serial.print("x,y,z= "); Serial.print(x);Serial.print(","); Serial.print(y);Serial.print(","); Serial.println(z); */ if (x > maxX) maxX=x;
if (x< minX) minX=x;
Serial.print("maxX = ");Serial.println(maxX);
Serial.print("minX = ");Serial.println(minX);
delay(500);

#else
if (Serial.available())
{
  if (Serial.read() == 0x01){
  ScratchBoardSensorReport(0,rx);
  ScratchBoardSensorReport(1,ry);
  ScratchBoardSensorReport(2,rz);
  ScratchBoardSensorReport(3,0);
  ScratchBoardSensorReport(4,distance);
  ScratchBoardSensorReport(5,0);
  ScratchBoardSensorReport(6,0);
  ScratchBoardSensorReport(7,0);
  }
 //ScratchBoardSensorReport(1,analogRead(POT2));
 // ScratchBoardSensorReport(7,digitalRead(BUTTON1)?0:1023);
  delay(30);
}
#endif

}

The Scratch examples are on the scratch site:

• Accelerometer demo: http://scratch.mit.edu/projects/osbock/2542866
• Distance sensor theremin: http://scratch.mit.edu/projects/osbock/2542860

Note you can’t play them directly from the site, as you need the sensors!

I love google+ it often exposes me to really amazing people and projects. Even better, sometimes when you comment, people are very generous with their help. I recently saw a post on Dangerous Prototypes blog about a group build of one of their the Bus Pirate V4. They laid out the parts and did two shifts to build 30 copies of the board.

It was done at Dorkbot PDX (People Doing Strange Things with Electricity in Portland Oregon), and I noticed that they were using a really cool PID controlled hotplate for surfacemount soldering, using a hacked Harbor Freight IR thermometer, and an optimized for the task Arduino clone.

I commented (Google +) on one of the pictures taken by Scott Dixon, one of the organizers, and developers of the soldering system with some questions.  The conversation that ensued was very informative and I wanted to share it with you:

Kevin: I’m newish to SMD, but I’m finding in some ways it’s easier than through hole. No more trying to get things flat, etc.

Scott: Definitely.  Also no turning the board over all the time to solder the leads, etc…

Kevin: Someone showed me how to use a toaster oven, but a lot of folks I know use the hotplate. Is one better than another?

Scott: For boards with parts primarily on one side, the hot plate is simpler and works fine.  I haven’t used a toaster oven but I hear some models work quite well particularly when you want to reflow both sides at once.  That was one of the reasons I put the thermocouple interface on the board but so far we haven’t used it for that.  Also, some day I might try sous vide.

Kevin: I recently built my first all SMD board. I did it by hand, with a little guidance from an experienced technician. Is there a time when it’s better to use one technique or the other?

There is an interesting cross over point between hand soldering and hot plate reflow depending on the complexity of the board (and how good you are at hand soldering).  As an experiment, three of us hand soldered the Bus Pirate.  We all have pretty good equipment, including stereo microscopes and we are all pretty good at hand soldering (although nowhere in the league of a tech who does it all the time).  It took each of us about 2 hours to do the BPV4 by hand.  By comparison, the first time we did the BP on the hotplate it took about 45-50 minutes or so (including getting the parts off the reels, etc).  We had a solder paste stencil so that made it somewhat faster.  So for something with that many parts it was over twice as long to hand solder.  A simpler project would be closer and a really simple project would be faster by hand, compared to setting up the hotplate and cleaning the stencil, etc.  If you don’t have a stencil and have to put the solder paste on by hand, that is somewhat slower.

I guess the only other thing I’d add is that we have done a number of SMD workshops and training sessions and our success rate for people walking out with a working project is often 100% and never very much below that.  So I would encourage people to give it a try.  It is almost certainly easier than you think.  It helps if you can find a workshop or class the first time but you can certainly learn on your own.

As a new member of Wyolum.com, I’ve been pitching in on their (our?) I2GPS project (Now available for pre-order!).

Wyolum has a terrific LED array board, originally intended to build word clocks, but Justin and others have also built a large format digital clock/timer display out of several of these boards, and the Super Accurate GPS clock will make it a terrific Race Clock.

I wanted to test out some GPS modules from an Asian source, and I felt that whatever Arduino code I had time to write was bound to be incomplete, so I decided to use the manufacturer’s PC progam.

We’re using the Fastrax UPS501, and the software is called Fastrax Workbench. The UPS501 is a terrific module, very sensitive (gets a good fix and tracks 9-10 satellites from within my house!) The reason Justin and Anool picked it, though, is that it has a 1 PPS output. This one Pulse Per Second can be used to get microsecond accuracy by lining this up with the GPS time as well.

Although it’s trivial, I thought I’d share how I used Dangerous Prototype’s Bus Pirate to hook up the GPS module to a PC for testing.

Since the UP501 GPS module communicates via TTL Serial, I wanted to use the Bus Pirate in “Transparent UART Bridge” mode.

I used this pinout chart to determine what was needed: http://dangerousprototypes.com/docs/Common_Bus_Pirate_cable_pinouts

• Pin 1 is RXD on the GPS and this goes to TXD on the Bus Pirate: Grey wire or MOSI
• Pin 2 is TXD on the GPS and this goes to RXD on the Bus Pirate:  Black wire or MISO
• Pin 3 is GND on the GPS to GND on the Bus Pirate:  Brown Wire
• Pin 5 and 6 are power and backup power, hook both those up to 3.3v Red wire on Bus Pirate

HiZ>m    ; Mode
1. HiZ
2. 1-WIRE
3. UART
4. I2C
5. SPI
6. 2WIRE
7. 3WIRE
8. LCD
x. exit(without change)

(1)>3     ;UART
Set serial port speed: (bps)  
 1. 300
 2. 1200
 3. 2400
 4. 4800
 5. 9600            ; Default baud for the GPS
 6. 19200
 7. 38400
 8. 57600
 9. 115200
10. BRG raw value

(1)>5      ;9600
Data bits and parity:
 1. 8, NONE *default
 2. 8, EVEN
 3. 8, ODD
 4. 9, NONE
(1)>      ;default - 1-8-none
Stop bits:
 1. 1 *default
 2. 2
(1)>              ; default 1
Receive polarity:
 1. Idle 1 *default
 2. Idle 0
(1)>           ;default 1
Select output type:
 1. Open drain (H=Hi-Z, L=GND)
 2. Normal (H=3.3V, L=GND)

(1)>2      ;Normal
Ready
UART>W          ; Turn on the power supplies
POWER SUPPLIES ON
UART>(1)    ; Select the transparent UART Bridge. If you forget this, you can type (0) for the options
UART bridge
Reset to exit
Are you sure? Y

Then it will start spitting out NMEA sentences

$GPGGA,235957.037,,,,,0,0,,,M,,M,,*43
$GPGSA,A,1,,,,,,,,,,,,,,,*1E
$GPGSV,1,1,00*79

Exit the program and fire up FastTrax Workbench. The only tricky part here is to select the correct baud rate. Since the Bus Pirate talks to the host at 115200 Baud, and the GPS at 9600 baud, we use 115200.

Charlotte, Mason and I hopped into Cambridge last Friday for the Mini Maker Faire at the Cambridge Science Festival. They helped me set up a table to introduce DuckyBots! to the attendees. We had probably close to 500 kids (and a few adults) making ducky bots in the 4 hours of the Faire, and almost got into trouble because people didn’t want to stop. We were the last ones to shut down. An unqualified success! (more details after the picture)

Photo by Chris Conners

The idea for duckybots started from a conversation at the Boston Robotics Meetup with Meredith Garniss.  In addition to trying to involve kids in robotics, we thought it would be good to inspire the adult robotics crew to create for kids.

Programs like FIRST robotics, and school programs are already full of people interested in STEM and robotics, and we wanted to find something that would turn kids on to the joys of Science and Engineering.

The basic idea is to use turning rubber ducks into robots as a platform for experimentation, creativity and learning.
We thought we could apply this at many ages and levels. Really young kids can exercise their creativity by decorating ducks (drawing, gluing on.)

The middle level would concentrate on mechanical physics, making ducks move, maybe racing (but also have to offer non-competitive challenges for kids scared of  competition.)

To keep the ideas coming in, we can also have an advanced level, perhaps Sumo ducks (on water!)

For their first appearance, I needed an activity that could be be done by hundreds of kids in a short period of time, so I focused on “making them Move”

Rather than presenting them with a blank slate, I put together some modules that represent different propulsion types.

I created a fan unit from a toy motor, a AA battery box (with switch) and a fan propeller sold as a spare for snap circuits.

I also had some playmobil underwater motors, and lots of Duck(!) tape (especially yellow).

As kids strapped on the motors, and found the ducks tipping over,  presenting an engineering challenge! I also had some closed cell foam (from packing materials) that they could use for floats, and outriggers.

There was lots of inventing, trying, doing things “wrong” (turns out those air fans work fine underwater!)

I was thinking that this was a good test to see if I should spend any more time on this, and I think the answer is yes!

These ducks have legs!

Photo by Chris Conners

About a week ago, Sparkfun posted a great prank in their weekly new product video. As April 1 was coming up, and I thought the kids would enjoy it, I ordered the bits that I didn’t have, and managed to pull it off.  Sparkfun didn’t give many details, so I thought I’d share my implementation.First here’s a video showing my wife getting pranked. She was a super good sport, (even not minding posting a video of her in her bathrobe!)

NOTE: Someone on google+ pointed out that at 8 amps, these horns could seriously heat up the li-po, and a hot li-po is not a good thing (fires/explosions!) so do this at your own risk. I only pulse the horns for 1 second at a time (twice per activation) so the battery never gets hot, but if you had a programming error, and it was stuck on, look out! Another good reason for the cutout switch and testing without the horns connected.

Here’s what you’ll need for this prank:

1. Arduino
2. Accelerometer: I used this one from ModernDevices
3. Digitally controlled relay. I used the same one they used in the Sparkfun prank
4. The Horns!
5. A (nearly) 12V lightweight battery. Sparkfun has this nice 1500 MAH 11.1 LIPO for a good price
6. A compatible charger. Sparkfun recommends the IMAX B6 This can also be found on Amazon
7. Wires, connectors, a good way to connect all the parts together

.

Here I wired the battery to power the Arduino, one leg of power went directly to both horns, and the other switched through the relay. I also put a toggle switch inline  with the power to the horns so I could shut off the horns for testing (The relay has an led and also clicks, so you can tell if it’s operating).

I had a proto-screw-shield on hand which was convenient for hooking up both the accelerometer and connecting to the relays.

Here it’s almost all hooked up. The relay connections are made to the analog side of the arduino to simplify wire routing. (Both 5V ground and io pins together.)

I use the button and LED on the protoshield to arm the device and indicate it’s status.

The bolts on the horns were too short to go through 1/4 ” plywood, so I used the supplied mounting plates to offset mount the horns.

Here it is all put together.  Here’s the Arduino sketch, but I recommend you try to program this yourself!

New toy (o-o-o)

After I showed the BootDrive on the Adafruit Show and Tell, I watched their weekly “Ask an Engineer” show where they showcased a new product, an LCD shield with an RGB backlight. The really cool thing is that the LCD AND 5 buttons only need 2 IO pins on the Arduino, because the shield is designed with an I2C expander. I ordered it the next day (close to 5pm) selected UPS ground shipping and it was at my house the next day!

Here’s a quick video before I go through the blow-by-blow:

I soldered the micro-sd breakout onto an Adafruit SD Shield, and a 6 pin right angle header to connect an FTDI cable. Adafruit also had a pre-made 6pin-6pin female cable that was perfect for this project!

I wired it with fine wire-wrap wire on the bottom

The wiring is almost identical to the original BootDrive, except I changed the reset pin assignment:

Digital Pin5  –> FTDI Pin 6 (CTS)
GND                –> FTDI Pin 1 (GND)
Digital Pin1 (RX) –>FTDI Pin 4 (TX)
Digital Pin2 (TX)–>FTDI Pin 5 (RX)

SD Card Interface:
Digital Pin 10 –> SD-breakout CS
Digital Pin 11  –>  SD breakout DI
Digital Pin 12  –> SD breakout DO
Digital Pin 13  –> SD breakout CLK
GND                    –> SD breakout GND
5V                       –> SD breakout 5V

Debugging

I incorporated the directory listing code from the SD library examples, and instead of listing all the files, I did an openNextFile() each time the “down” button was pressed. It would be nice to be able to scroll up and down, but I’m pretty tight on memory as it is, so I didn’t want to cache the directory, or do some kludgy re-reading the whole directory to get to the file we are on.

The first problem I ran into was it worked fine until you scrolled through what turned out to be 25 names (actually the same 3 names 8+ times) and then it started repeating the same name over and over. When I realized it was always 25, I knew it wasn’t some overflow, and I finally figured out the example code had a bug in it! They list the directory by opening each file (with openNextFile() to get the name, but they never close it! If you try the listfiles example with more than 25 files on the card, you’ll see only the first 25. Easy enough to fix, as I’m scrolling I close the file after I get it’s name. I reopen it if you press the “select” or program button.

The next problem was a bit harder. The hex programs seemed to work on some boards but not others, even though there were apparently no errors. I created new test programs and suddenly they didn’t work either. Since the writing to flash seemed to be working, I figured that there must be some corruption of data. I read back the programmed AVR code from the target arduino (with avrdude  <programmer parameters….> -U f:r:readfile.hex:i) and compared it with the file I was sending to be programmed. It turns out I was advancing to the next “line” of 16 bytes even when I read less. Files that ended in shorter lines were getting placed ahead of where they needed to be and so the program was corrupt. Probably on some, the data that was already there was harmless, or the program didn’t actually execute that part of it’s memory, but in some cases it clearly was. Fixed, putback to github!

Have fun and let me know if you try it!

https://github.com/osbock/Baldwisdom/tree/master/LCDBootDrive

There are two kinds of memory(well three including EEPROM, but we won’t deal with that here) you can run out of when doing an Arduino project (or any embedded project):

1. Flash Memory: For a lot of applications, the Arduino has a fairly big flash memory (the UNO and all atmega328 based Arduinos give you 32k flash.) The Arduino IDE will also tell you if your program size exceeds the amount of flash in the particular Arduino you have selected.
2. Static RAM: Atmega328 in the UNO gives you 2k of RAM. This is the most precious commodity in Arduino programming, and it’s often surprising how fast you can run out. The bad thing is,  the IDE won’t warn you, and your program can go from working to behaving really weirdly by doing something as simple as adding an additional debug print statement. This post will attempt to explain this, and give you a few tips for understanding your RAM use.

Where Did it All Go?
When you run out of SRAM, your program can just stop working, for reasons ranging from one data structure overwriting another (so your program does something based on the wrong value) to overflowing into the stack. When that happens, generally, your program dies a horrible death, because returning from subroutines can cause the processor to jump to random places in memory. If you make an innocuous change to your code (like adding another debug statement) and your program suddenly behaves very differently, chances are you have run out of SRAM.

It’s obvious if you declare large variable arrays like:

int valueArray[1024];

would instantly overflow memory because int’s occupy 2 bytes in atmega land. You also need to leave room for stack, and any other variables your libraries, and the Arduino software itself uses.

What’s Harvard got to do with it?
Not so obvious is the fact that:

Serial.println("Hello");
const char constantString[] = {"Hello"};

both  occupy 6 bytes in both flash memory and in SRAM (c-strings are terminated with a Null, or 0×00 byte).  The reason is AVR 8 bit microcontrollers use a Harvard architecture addressing scheme. This means that when an instruction addresses memory, there are two versions, one that addresses program memory (flash), and another to address RAM. Most routines like Serial.println use regular RAM, as they don’t know if you’ll be passing in constant info or stuff that you’ve constructed. As a result, even static strings (with a const declaration) are copied from flash to SRAM at system startup. It’s possible to write routines to only use flash (using the PROGMEM attribute) and it’s definitely useful, but that’s covered elsewhere.

I ran into this porting the avrdude stk500 routines to Arduino, blindly converting the printfs into Serial.print statements. In order to get enough RAM I ended up using numeric codes for errors (I’ll probably eventually get around to PROGMEM-ing them).

So, how do you tell if you are nearly running out, or have already? The IDE won’t tell you, so you have to dig in a little.

The build process for arduino produces an “ELF” file or Executable and Linkable File. This is used to create the actual HEX file for programming, but still has memory segment information embedded in it.

First find your build directory. On Windows 7 this is \users\<youruser>\AppData\Temp\build<a bunch of numbers>.tmp

Now, assuming you have the avrdude files somewhere in your path, use the command:

avr-size -C --mcu=atmega328p <yourelf-file.elf>

For example, the current version of Bootdrive gives me:

avr-size -C --mcu=atmega328p BootDrive.cpp.elf
 AVR Memory Usage
 ----------------
 Device: atmega328p

Program: 19272 bytes (58.8% Full)
 (.text + .data + .bootloader)

Data: 1340 bytes (65.4% Full)
 (.data + .bss + .noinit)

It’s probably best not to go beyond 95% full for your Data (that’s the SRAM part) to save room for your stack, and even less if you pass big variables on the stack or have deep call chains.

Announcing the formal release of BootDrive for Arduino. If you’ve been reading the blog, you know that I’ve been working with Justin Shaw of Wyolum labs to enable their I2SD (Arduino clone with a micro-sd card) to bootload a program onto another Arduino. After battling several issues, mostly related to porting avrdude code to the Arduino (assumptions about infinite memory, etc!) it’s working well enough people can fool around with it.

Right now the code looks for a single file (“program.hex”) and waits 5 sec and then blasts it into the target arduino. Details about setting up to try this yourself follow the video.

Here’s a demo of it in action:

If you’d like to try this with a couple of Arduinos, you’ll also need an SD card (or micro-sd) interface such as the Adafruit Micro-SD breakout. This board is great because it has onboard 3.3v conversion so it’s safe to use with 5V (stock) Arduinos. Download the code from github: https://github.com/osbock/Baldwisdom/tree/master/BootDrive
or a zipped version here

Hook up your Arduinos like this:

I

The basic setup is:

Arduino Control:
Master Digital Pin6  –> Target reset
GND                                –> GND
MASTER Digital Pin1 –>Target Digital Pin2
MASTER Digital Pin2 –>Target Digital Pin3

SD Card Interface:
 Master Digital Pin 10 –> SD-breakout CS
Master Digital Pin 11  –>  SD breakout DI
Master Digital Pin 12  –> SD breakout DO
Master Digital Pin 13  –> SD breakout CLK
Master GND                    –> SD breakout GND
Master 5V                       –> SD breakout 5V

Optional debugging serial interface (This lets you see error messages and the like)
Master GND    –> FTDI cable Pin 1 (GND)
Master Digital Pin 4 –> FTDI cable Pin 5 (diagram is wrong, but you can change it in the code if you need to)

Here are the defines you can change to swap the pins around:

#define BOOT_BAUD 115200 // This is the Arduino UNO’s bootloader baud rate. Other boards are different!
#define DEBUG_BAUD 19200 // for software serial debugging
#define txPin 4
#define rxPin 5 // not really used…
#define rstPin 6

Also if you want to put your SD card on a different line (or are using a shield that uses a different chip select) change this:

const int chipSelect = 10;

Action!

Right now, 5 seconds after initialization, the program blasts a file called “program.hex”. You can add your own user interface to select amongst multiple programs.
Just call:

void programArduino(char *filename)

with the filename, e.g.”

programArduino("program2.hex");

You can build new hex files by grabbing them from the Arduino build directory. Make sure you have the IDE set up for your target board and press the “compile/verify” button.

From http://arduino.cc/en/Hacking/BuildProcess:

The .hex file is the final output of the compilation which is then uploaded to the board. During a “Verify” the .hex file is written to /tmp (on Mac and Linux) or \Documents and Settings\<USER>\Local Settings\Temp (on Windows). During upload, it’s written to the applet sub-directory of the sketch directory (which you can open with the “Show Sketch Folder” item in the Sketch menu)

Note on Windows7, the base directory is \users\<USER> \appdata\local\Temp\build<some bunch of numbers>.tmp\<sketchname>.cpp.hex

Note that if you want to load an Arduino other than UNO, you’ll need to change the baud rate. You can find that in “hardware/arduino/boards.txt” in the arduino directory. This won’t work on MEGA based Arduinos as they use the stk500v2 protocol (thanks westfw!)

Breaking through!

It’s been a while since the last update, but some progress has been made. I discovered that SoftwareSerial library doesn’t work really well at 115200 baud, so I switched the wiring around to use software serial for debugging, and the hardware uart pins (Arduino pins 1 and 2) to talk to the bootloader on the target. This is a little bit of a pain, as you have to unplug the connection to the target to download code to the “programmer” but it does work now.

I had a little trouble getting sensible responses from the bootloader until I put in a delay after resetting. In retrospect this makes sense as the AVRdude code reminded me that there are capacitors on the Arduino reset line. I was probably trying to talk too soon.

What bootloader, speed, protocol?

There have been several different bootloaders in the history of Arduino, and the source code for the current “canonical” loaders is included in the Arduino software distribution.  The bootloader is a little piece of code that allows you to program the flash memory of the Arduino’s atmega328p via serial or USB instead of using an ICSP programmer.  These bootloaders implement a subset of the STK500 8bit protocol (not to be confused with STK500v2) which is documented at www.atmel.com/atmel/acrobat/doc2525.pdf

Arduino MEGA (1280 and 2560) use yet another bootloader, one that uses the stk500v2 protocol instead.

The source code for the bootloader used by the diecimila, duemilanove, and a few other clones  is in …\arduino-1.0\hardware\arduino\bootloaders\atmega. Many clones use this, or a variant hacked by Lady Ada (Limor Fried).

The UNO and other later Arduino variants use a much smaller bootloader called Optiboot (more room for your code!). A little secret is that you can actually burn this bootloader into your older arduinos (at least at atmega168 and greater) and then just refer to the board as an UNO.

Adafruit Forum user westfw (one of /the?) author of  optiboot points out:

Optiboot also supports the mega8. However, you can only call a board an Uno if it at has a m328 cpu in it. For m8 and m168s with optiboot, you’ll need to make a new entry in boards.txt that copies the Uno bootloader parameters, but has the correct “build.mcu” specification (and fuses, if you want to program the bootloader using the IDE.)

The code for this is in …\arduino-1.0\hardware\arduino\bootloaders\optiboot. If you need to change the processor or baud rate, the settings are in the Makefile. This version is very stripped down, and doesn’t support, for example, writing to EEPROM. I’m choosing to use this as my initial target. Most of the code will still work on the older bootloader, but may require a slightly different command set.

The latest and greatest optiboot can be found at it’s Google code site: http://code.google.com/p/optiboot/

Which is also the preferred place for bug reports and to check for newer versions (atmega1284 is now supported!)

Because of varying clock rates, and different bootloader code, the various boards use different baud rates to talk to the IDE. This information is found in the …\arduino-1.0\hardware\arduino\boards.txt file. The UNO programs at 115200 baud, the Duemilanove (with ATmega328p) programs at 57600 baud. The Diecemila or Duemilanove with ATmega168 programs at 19200 baud.

Looking at the actual protocol

After the Arduino IDE compiles everything down to a intel hex file for downloading to the target Arduino, it uses the avrdude command to program the Arduino. The upload protocol is also specified in the boards.txt file and is called arduino, which is a slight variant of avrdude’s stk500 protocol. Most of the commands are from the stk500 module, except the signature reading is slightly different. For now, I’m adapting the avrdude code directly for the Arduino. This requires some minor porting (eliminating fprintfs, removing some of the indirection that links it to the avrdude programmer framework). I may simplify some of this code later as I’m a little nervous about running out of room.

As I mentioned in my last post, I downloaded a trial version of AGG Software’s Advanced Serial Port Monitor, and recorded the conversation between avrdude (controlled by the Arduino IDE) and the UNO target board. I stayed up late last night annotating the conversation:

DTR and RTS are both toggled from High to low several times

Avrdude then sends the GET_SYNCH command, thows away whatever comes back and sends it again a couple of times
Avrdude/Arduino IDE:#30#20 STK_GET_SYNCH, Sync_CRC_EOP
Target/Arduino UNO:  #14#10 STK_INSYNC, STK_OK
Avrdude/Arduino IDE: #30#20 STK_GET_SYNCH, Sync_CRC_EOP
Target/Arduino UNO:  #14#10 STK_INSYNC, STK_OK
Avrdude/Arduino IDE: #30#20 STK_GET_SYNCH, Sync_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK

Get the version number of the bootloader (in this case it’s 4.4) we could use this to branch to different variants of protocol.
Avrdude/Arduino IDE: #41#81#20 STK_GET_PARAMETER, STK_SW_MAJOR, SYNC,CRC_EOP
Target/Arduino UNO: #14#04#10 STK_INSYNC, 0×04, STK_OK
Avrdude/Arduino IDE: #41#82#20 STK_GET_PARAMETER, STK_SW_MINOR, SYNC_CRC_EOP
Target/Arduino UNO: #14#04#10 STK_INSYNC, 0×04, STK_OK

The next two commands are consumed but ignored (probably used by a real programmer to retrieve device specific stuff.)
Avrdude/Arduino IDE: #42#86#00#00#01#01#01#01#03#FF#FF#FF#FF#00#80#04#00#00#00#80#00#20(This is a STK_SET_DEVICE Command that is consumed but ignored by the bootloader)
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK
Avrdude/Arduino IDE: #45#05#04#D7#C2#00#20 STK_SET_DEVICE_EXT (also ignored)
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK

Enter Programming mode.
Avrdude/Arduino IDE: #50#20 STK_ENTER_PROGMODE, SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK
Avrdude/Arduino IDE: #75#20 STK_READ_SIGN, SYNC_CRC_EOP (read device signature)
Target/Arduino UNO: #14#1E#95#0F#10 STK_INSYNC, (three byte signature), STK_OK

The next four commands are consumed but ignored. I won’t bother to reproduce them in my code.
Avrdude/Arduino IDE: #56#A0#03#FC#00#20 STK_UNIVERSAL,0xA0,0×03,0xFC,0×00 SYNC_CRC_EOP (This is a pass through command that is supposed to be used to perform arbitrary native SPI commands. In this case, and the next 3, they are Read EEPROM byte commands They are consumed but ignored by the bootloader)
Target/Arduino UNO: #14#00#10 STK_INSYNC, (last byte written by the Universal command, STK_OK
Avrdude/Arduino IDE: #56#A0#03#FD#00#20 STK_UNIVERSAL,0xA0,0×03,0xFD,0×00,SYNC_CRC_EOP (Ignored by the bootloader)
Target/Arduino UNO: #14#00#10 STK_INSYNC, (last byte written by the Universal command, STK_OK
Avrdude/Arduino IDE: #56#A0#03#FE#00#20 STK_UNIVERSAL, 0xA0,0×03,0xFE,0×00,SYNC_CRC_EOP (Ignored by the bootloader)
Target/Arduino UNO: #14#00#10 STK_INSYNC, (last byte written by the Universal command, STK_OK
Avrdude/Arduino IDE: #56#A0#03#FF#00#20 STK_UNIVERSAL, 0xA0,0×03,0xFF,0×00,SYNC_CRC_EOP (Ignored by the bootloader)
Target/Arduino UNO: #14#00#10 STK_INSYNC, (last byte written by the Universal command, STK_OK

Here’s where we actually get started. This specifies the address in Flash where the follwoing PROGRAM_PAGE data goes.
Avrdude/Arduino IDE: #55#00#00#20 STK_LOAD_ADDRESS, 0×0000, SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK
And then the actual data. Note that the data is in the same order as the bytes in the Intel Hex file
Avrdude/Arduino IDE:
#64#00#80#46#0C#94#61#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C
#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94
#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E
#00#0C#94#7E#00#0C#94#9A#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00
#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C
#94#7E#00#00#00#00#00#24#00#27#00#2A#00#00#00#00#00#25#00#28#00#2B
#00#00#00#00#00#20
STK_PROGRAM_PAGE, 0×0080 (page size), ‘F’(flash memory), data bytes…,SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK

And then we repeat – set address, program page, etc.
Avrdude/Arduino IDE: #55#40#00#20 STK_LOAD_ADDRESS, 0×0040, SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK
Avrdude/Arduino IDE: STK_PROGRAM_PAGE, 0×0080 (page size), ‘F’(flash memory), data bytes….,SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK
.
.
.
Then it reads several pages (probably to verify what it wrote)

Avrdude/Arduino IDE: #55#00#00#20 STK_LOAD_ADDRESS, 0×0000,SYNC_CRC_EOP
Target/Arduino UNO: #14#10 STK_INSYNC, STK_OK 

Avrdude/Arduino IDE: #74#00#80#46#20 STK_READ_PAGE,0×0080 bytes, ‘F’ for flash, SYNC_CRC_EOP
 Target/Arduino UNO #14 STK_INSYNC
Target/Arduino UNO #14#10 STK_INSYNC, STK_OK
#0C#94#61#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E
#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94
#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#9A#00#0C
#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00
#0C#94#7E#00#0C#94#7E#00#0C#94#7E#00#00#00#00#00#24#00#27#00#2A#00#00
#00#00#00#25#00#28#00#2B#00#00#00#00#00 

#10 STK_OK

repeat several times at different addresses

.
.

And then leaves programming mode
Avrdude/Arduino IDE: #51#20 STK_LEAVE_PROGMODE, SYNC_CRC_EOP
Target/Arduino UNO #14#10 STK_INSYNC, STK_OK

Then we’re done! Avrdude then toggles DTR/RTS to reset

When I originally posted about the Auto-programmer I was pleased to see several people on my Google+ stream and in the Adafruit forums were interested. Justin Shaw of Wyolum labs was interested in the idea of reading the program off the SD card, but instead of programming a microcontroller chip directly, program another Arduino compatible (specifically one of his gorgeous clocks!)

He’s already made an Arduino compatible SD card reader board he calls I2SD (it speaks I2C) that also speaks TTL serial, so it would be perfect. I’m putting the Auto-programmer on hold to make …. drum roll please…. The Boot Drive for Arduino!

I promised a blow by blow, and I’ve been working on this for a couple of sessions. I’ve read through all the docs (8 bit stk500 protocol, optiboot source code, AVRDUDE source.) and It looks fairly straight forward. Essentially the “programming arduino” stands in for the IDE.

The optiboot source code is pretty cleanly documented (used in most newer arduinos, like the UNO). It’s pretty tiny and does only the bare minimum to get the job done. Since it’s fairly stateless (not completely, as there are timeouts that cause you to jump to the user program), I needed to get an idea of the actual sequence.

I downloaded a trial version of AGG Software’s Advanced Serial Port Monitor which lets you “listen in” on the conversation between a computer (i.e. the Arduino IDE) and a device (the Arduino.) I collected sample program downloads for the UNO and for the Duemilanove (they have different bootloaders). There were no surprises, but it was nice to confirm the sequence. If people are interested, I could post about this later. Drop me a line on my Google+ stream.

Here’s how I have it wired (using regular Arduinos until I can get my hands on an I2SD).  Note I haven’t set up the Adafruit MicroSD breakout board yet.

I’ve got pins 2 and 3 configured as software serial on the “Master” Arduino hooked in a null modem configuration with pins 0 and 1 of the “Target” Arduino. I chose to do this, so I can send debugging info down the USB serial channel. I’m doing this at 115200, because that’s what the UNO bootloader operates at. I also hooked a digital pin from the Master to the Reset of the Target. If I read the code correctly, you have a very short period of time to start sending STK500 commands before the bootloader times out and jumps to user code after RESET. The Arduino IDE asserts reset by toggling the RTS line (well, actually a software call on the USB-serial driver). The RTS output of the FTDI chip (or equivalent on the UNO) is tied to RESET.

Now comes the frustrating part, I haven’t been able to get them to talk. I get bytes back but they aren’t what I expect, and I’m not sure where they are coming from if I’m not properly entering the bootloader.

I also tried setting up dummy programs, the Target sending “Hello” in a loop, and the “Master just trying to receive it and print it out. That doesn’t work either.

I’ll keep you posted. I’m also going to try talking to a TTL serial Arduino clone (one of the WIBLOCKS Pico boards).