It’s all wrapped up into a new “Sleepy” class:

See the powerdown_demo.pde and radioBlip.pde sketches for examples of use.

This class is documented in the Café:

FWIW, I’m also evaluating the Redmine system as a way to bring the code repository, Café docs, issue tracker, and Talk forums all into one context:

That site is still experimental, so I’m not making it public yet. The one missing feature holding me back is that Redmine does not have a good spam prevention mechanism, such as Akismet. At least last time I looked. But all in all, this would be a great way to provide a place to describe projects, fill in the documentation, and track all code changes and bugs collaboratively. If you’d like to have a sneak preview, or want to have a place to describe your project, or perhaps would like to help with the fairly gigantic task of getting a good documentation section going, please email me.

I’ve started copying over some content, but it’s going to take a while before everything has been brought over and adjusted. Both the old and the new system use Markdown, but there are always them pesky little details…

Anyway, back to the topic at hand – enjoy the low-power options, and please consider sharing your explorations, findings, and inventions in the Talk discussion forums.

After yesterday’s improved power-down current results, I wanted to find out how much power it really takes to send out a bunch of small packets. No acks, just periodically sending out a packet and sleeping.

Trouble is, I didn’t want to wait for a battery to run down, since that would take months or even years. Bit long to get results for this weblog, eh?

So instead, I used this little chap:

It’s a super-capacitor which can handle up to 5.5V and has a capacity of 0.47 Farad! That’s like putting a thousand 470 µF caps in parallel. Amazing stuff, in an even more amazingly small package.

Here’s a JeeNode, fitted with this new power source, to give you an idea of just how small this thing is:

The next step was to design a small sketch to test this. Here’s what I came up with:

Some of the code left out for brevity. Full source code can be found here.

What this demo does is send out a packet with a 2-byte payload, then sleep for approximately 1 second, then send again, etc. Until power runs out.

Sure enough, packets started coming in every second:

    OK 4 0 0
    OK 4 1 0
    OK 4 2 0
    OK 4 3 0
    [...]

I expected it to send out say 100 packets or so, before the charge in the 0.47 F supercap would run out.

Guess how far it went…

    [...]
    OK 4 178 28
    OK 4 179 28
    OK 4 180 28
    OK 4 181 28

That’s packet number … clickety, clickety … 28 * 256 + 181 = 7349 !!!

In other words, the JeeNode was able to send out well over 7000 packets, i.e. two hours of packets sent once a second.

This is fantastic. I think the secret – apart from the 3 µA powerdown mode – is that the RFM12B + RF12 driver require very little time to start up, send off a packet, and go back to sleep again. There is no ACK involved, the RFM12B is hardly ever in reception mode.

With real-world use, I expect power requirements to be considerably higher. First of all, a Room Board will draw around 50 µA, due to the on-board PIR sensor. And second, I’m going to want to use ACKs for at least the motion detector reports, so that the system has a robust mechanism for reporting motion whenever it is detected. This means putting the RFM12B in receive mode for a few milliseconds, waiting for the ACK. And repeating this process a few times if that ACK isn’t immediately received.

But still … over 7000 packets without a battery!

Update – with one packet per 5 seconds, the charge lasted 4523 packets, i.e. just over 6:15.

Update #2 – with one packet per 60 seconds, 771 packets got sent out, that’s 12:51 hours … looks like the self-discharge of the supercap is eating up the remaining energy.

A while back, I got it down to a fraction of that, by turning off the brown-out detector (BOD) through a fuse bit on the ATmega. That’s a little circuit which prevents the ATmega from coming out of reset and doing anything if the voltage is too low.

As it turns out, you can turn off the BOD in software, but only for sleep mode. The reasoning being that there’s no point in protecting the processor from going berserk while it’s not doing anything in the first place…

Well, my code to turn off the BOD was wrong. You have to do this right before going to sleep. Here’s the updated powerdown_demo.pde sketch:

(correction – I mixed up my bits: change “PINB |= …” to “PINB = …” or “PORTD ^= …”)

The result?

Now we’re cookin’ again!

Yeah, right… there is no such thing, of course – unless you’re into probabilities or fuzzy logic, perhaps.

What I actually want to do, is describe “analog output” on the ATmega, using the Arduino library’s analogWrite() function. And throw in some bit manipulations along the way, to stay somewhat on topic.

The little secret with analogWrite() is that it doesn’t do what its name suggests. The ATmega has no way of generating an analog signal, i.e. a voltage level between 0 and VCC.

Instead, a pulse is generated, with a varying duty cycle. I.e. the “on” and the “off” times of the pulse will be different, the ratio of these times being proprtional to the 0..255 value passed as argument to analogWrite(). With “0″, the signal will be 100% off, with “255″ the signal will be 100% on. With “128″ the pulse will be on the same amount of time as off. With “1″, it will be on very briefly, and then off most of the time, and so on…

The neat thing is that you connect it to an incandescent lamp, or a motor, then the effect will be that these will light/turn at less than their full power, due to the time it takes for these devices to try and follow the pulse. So the effect is similar to a fractional adjustment: you can dim / slow down these devices by using analogWrite().

It even works with LEDs, although these turn on and off very fast. In this case, the reason is that our eyes can’t follow such fast changes, and so we perceive the result as dimmed as well. A whole industry was once created around this “persistence of vision” property of our eyes – it’s called TV…

Here’s a sketch which uses this pulsed output to control the brightness of a LED connected to DIO3 (i.e. D6):

Note that I didn’t have to define pin 6 as an output, analogWrite() does that.

What the above does, is ramp up gradually from 0 to 255, and then repeat:

Suppose we want it to fade in and out instead:

Try implementing this yourself.

Note that you’re going to need at least 9 bits of information to do this: 8 for the brightness level and 1 to keep track of whether you’re currently in the up ramp or in the down ramp:

Here’s one way to do it, using some bit trickery:

A few notes:

• I’ve changed the “level” variable from an 8-bit byte to a 16-bit word
• bit 8 toggles from 0 to 1 and back every 256 level counts
• it’ll be 1 when level is 256..511, 768..1023, etc
• when it’s 1, we flip the bits, i.e. 0 becomes 255, 1 becomes 254, etc
• the analogWrite() function ignores all upper bits

If you think that was an obscure call to analogWrite(), try this one:

    analogWrite(6, level ^ -((level >> 8) & 1));

Maybe you can decypher it when written slightly differently?

    analogWrite(6, level ^ -bitRead(level, 8));

(hint: bitRead() always returns either 0 or 1)

It’s all pretty geeky stuff, and let’s hope you’ll never have to deal with code such as this again, but the point of this story is that there’s no magic. You just have to know what each operator does, and how to translate an integer from decimal to binary notation and back.

I’ll summarize my intuitive interpretation of bit operators below:

• “X | Y” = take X and copy all the 1′s of Y into it
• “X & Y” = take X and copy all the 0′s of Y into it
• “X ^ Y” = take X and flip all the bits where Y has 1′s
• “~ X” = flip all the bits of X
• “- X” = arithmetic minus (same as “(~X) + 1″ !)
• “! X” = 1 if X is zero, 0 otherwise
• “X << N” = multiply X by 2, N times
• “X >> N” = divide X by 2, N times

Some tricks based on this:

• “~ 0” = all bits set to 1 (same as “-1″ !)
• “~ 0 << N” = all bits 1, but N lowest bits set to 0
• “bit(N) – 1” = a constant with N lowest bits set to 1
• “X & (bit(N) – 1)” = the N lowest bits of X, the rest is 0
• “X & ~ (bit(N) – 1)” = X, but with the N lowest bits set to 0
• “!! X” = 0 if X is zero, 1 otherwise

An useful rule when writing logical expressions is: when in doubt, parenthesize! – see C operator precedence.

Sooo… use bit(), bitRead(), bitWrite(), bitSet(), and bitClear() wherever you can, since it usually makes the code easier to read. But there’s no need to get lost if you see ^&|~!’s in your expression – just slow down and decode such expressions step by step!

The “if (onOff = 0)” etc is the logic that toggles onOff between 0 and 1 on each pass through the loop:

    if (onOff == 0) onOff = 1; else onOff = 0;

But there are lots of ways to do the same thing, coded differently:

    if (onOff == 0) onOff = 1; else onOff = 0;
    if (onOff == 0) onOff = bit(0); else onOff = 0;
    if (onOff == 0) bitSet(onOff, 0); else bitClear(onOff, 0);
    onOff = onOff ? 0 : 1;
    onOff = (~ onOff) & 1;
    onOff = (onOff + 1) & 1;
    onOff = ! onOff;
    onOff = 1 - onOff;
    onOff = onOff ^ 1;
    onOff ^= 1;

See if you can figure out all of these.

Take your pick. Those last two use C’s XOR operator. I tend to prefer shorter source code, so I’d use that last notation (note that the resulting compiled code is not necessarily shorter than the other examples).

Now suppose you have a byte value “X” and you want to flip the 4th bit in it, while not changing anything else. That’s a bit more work. It could be done like this, for example:

    if (bitRead(X, 4) == 0) bitSet(X, 4); else bitClear(X, 4);

Or like either of these:

    X = X ^ bit(4);
    X ^= bit(4);

This shows clearly that the “^” XOR operator does exactly what we need: flip bits.

Back to blinking an actual LED, as done with the above sketch. Here’s a little mind bender – another sketch, doing the same using raw ports and the XOR operator:

The first example was doing things “the Arduino way”, using pinMode() and digitalWrite(). It compiles to 890 bytes of code. This second example goes straight to the hardware and uses 554 bytes of code:

• Arduino digital pin 4 is bit 4 on the “D port” of an ATmega
• “DDRD” is the “Data Direction Register”, where we set up pin 4 as an output
• “PORTD” is the out “Port Register”, which controls the actual output signal

You can see the XOR in action in that last example. It takes all the output bits of port D (Arduino pins 0 .. 7), and flips just a single bit, i.e. bit 4.

Just for kicks, I’ll show you one more way to blink the LED:

This uses a relatively little-known feature of the hardware, which actually has “bit flipping” built-in. The “PIND” register is normally used for input, i.e. for reading the state of a pin as an input signal. But you can also write to that register. When you do, it will be used to flip output pins, but only for the bits which were set to 1. It’s essentially a built-in XOR.

That last example uses 550 bytes of code, most of which is overhead from the Arduino run-time library (setting up the milliseconds timer, etc). So what’s in a measly 4 bytes, right? Wrong. There is a minute, but sometimes important difference: the other approaches all had to read the register value first, flip the bit, and then write the value back. This last version only writes a (constant) value to a register. With interrupts, that can be very important: this last version can’t ever go wrong, it will always flip the requested bit. The other version could have an interrupt occur between the read and the write. It’s a known issue for the Arduino Mega. It can lead to code which runs for a week, and then fails mysteriously. Bugs like these are fiendishly hard to properly diagnose.

Bit-flipping can be quite useful for physical computing. Not only does it let you easily toggle specific bits, and change the state of some output pins, it can also be a way to clear a bit. Let’s say you need to generate a (very) quick pulse. Here are four ways to accomplish the same thing:

    bitSet(PORTD, 4); bitClear(PORTD, 4);
    PORTD |= bit(4); PORTD ^= bit(4);
    PORTD |= bit(4); PIND = bit(4);
    PIND = bit(4); PIND = bit(4);

That second one based on XOR works, because bit 4 is known to be one, so setting it to zero is always the same as flipping it. That’s also why the third PORTD/PIND example works, with PIND doing the XOR in hardware. Lastly, the fourth approach will only work if bit 4 was initially zero. It’s the fastest one, and does not suffer from the interrupt race condition mentioned above.

Ok, that’s enough flippin’ for one day!

Tomorrow, I’m going to go into, ehm… “fractional bits” (haha!) ;)

Update – see comment below on why “bitSet(PORTD, 4); bitClear(PORTD, 4);” are also interrupt-safe (mostly – but not on every pin of an Arduino Mega!).

You need bit manipulation when you’re dealing with the individual bits in a byte, such as on the I/O ports of an ATmega, for example.

First the easy approach – use these predefined macros from the Arduino library:

• bit(N) returns an integer with the N’th bit set to 1
• bitRead(X,N) – returns the N-th bit of X as 0 or 1
• bitWrite(X,N,B) – sets Nth bit of X to B (0 or 1)
• bitSet(X,N) – sets the Nth bit of X to 1
• bitClear(X,N) – sets the Nth bit of X to 0

This is why you might see code such as the following:

    bitSet(WDTCSR, WDIE);

This means: “set the Watchdog Interrupt Enable to 1 in the Watchdog Timer Control Register”. The WDTCSR and WDIE terms are predefined constants. WDIE is 6, for example.

Note that some of these routines can be written in terms of the others, i.e.

• bitSet(X,N) is the same as bitWrite(X,N,1)
• bitClear(X,N) is the same as bitWrite(X,N,0)

But what does it all mean?

Well, let’s dive in. First make sure that you are comfortable with “bit shifting”. The expression “bit(3)” is the same as “1 << 3″, which in turn is the same as doubling the value 1 three times, i.e. the value eight. So “bit(0)” is 1 doubled zero times (i.e. 1) and “bit(7)” is 1 doubled 7 times, i.e. 128. Bytes have 8 bits numbered 0 to 7, so all you need for (byte-sized) hardware registers is to remember that bits 0..7 map to (specific!) integers with values 1 to 128.

Setting a bit, is like OR-ing the bit with the rest of the value. The following statements are all identical:

    WDTCSR = WDTCSR | bit(WDIE);
    WDTCSR = WDTCSR | (1 << WDIE);
    WDTCSR = WDTCSR | (1 << 6);
    WDTCSR = WDTCSR | 0b1000000;
    WDTCSR = WDTCSR | 0x40;
    WDTCSR = WDTCSR | 64;

This, in turn, can be abbreviated in C as:

    WDTCSR |= bit(WDIE);
    WDTCSR |= (1 << WDIE);
    WDTCSR |= (1 << 6);
    etc...

Or you could write:

    bitSet(WDTCSR, WDIE);

It’s all the same. So OR-ing is about setting bits (to 1).

Likewise, AND-ing is about not clearing bits (to 0). Whoa, that’s confusing. This expression returns a value which is what X was, but only for bit N:

    X & bit(N);

So this will change X to a value with all bits except bit N set to zero:

    X = X & bit(N);

To put it differently: X will lose its original bits, except bit N, which will be left alone. All the bits are set to zero, except bit N.

Usually, you want the opposite, setting only bit N to zero. That too is accomplished with AND-ing, but you have to flip all the 0′s to 1 and all the 1′s to 0 first. Hang in there, it’s a slightly longer story. This sets bit N to zero:

    X = X & ~ bit(N);

Let’s examine what’s going on here. First, “bit(N)” is a value with only the Nth bit set. Now, “~ bit(N)” is a value with all the bits flipped around (“~” is called the complement operator in C), so that’s a value with all but the Nth bit set. Everything is 1, but bit N is 0.

Now we can tackle the expression “X & ~ bit(N)”. Since AND-ing is about “not clearing bits”, that means that the result of this expression is all the bits of X unchanged where “~ bit(N)” was one, which is almost everywhere. The only bit that differs is bit N – it is zero in “~ bit(N)”, therefore that particular bit will “not not clear …” (a double negation!): it will be cleared (to 0) in the result.

Finally, we replace X by that result. So X will change in precisely one bit: bit N. That bit will be cleared to zero, the rest is not affected. In short: we’ve cleared bit N.

Confused?

Well, that’s why the bit/bitSet/etc macro definitions were introduced. These expressions are all identical:

    X = X & ~ bit(N);
    X = X & ~ (1 << N);
    if (X & bit(N)) X = X - bit(N);
    bitClear(X, N);

That last one is clearest by far, because it conveys the actual operation with a well-chosen name: clear bit N, leave the rest alone.

So why would anyone ever choose to use anything but the bit/bitRead/etc routines?

Many reasons. Habit, perhaps. Coming from another environment which doesn’t have these macros. Being so used to this bit-manipulation that the use of words doesn’t really look any clearer. Whatever…

But another more important reason is that you can’t do everything with these bit/bitSet/bitClear routines. Sometimes you just have to go to the raw code. Such as when you need to set multiple bits at once, or flip bits. That’s why the ATmega datasheet has examples like these:

    WDTCSR |= (1<<WDCE) | (1<<WDE);

By now, you should be able to decode such a statement. It’s the same as:

    WDTCSR |= bit(WDCE) | bit(WDE);

In other words:

    WDTCSR = WDTCSR | bit(WDCE) | bit(WDE);

Which in turn is almost the same as these two statements together:

    WDTCSR |= bit(WDCE);
    WDTCSR |= bit(WDE);

That in turn, can be written as these two lines:

    bitSet(WDTCSR, WDCE);
    bitSet(WDTCSR, WDE);

Much clearer, right? Except… it’s not 100% identical!

The problem is a hardware issue: timing. The above two statements will set both bits, but not at the same time! For hardware register settings, that difference can be important. There is a fraction of a microsecond between the WDCE bit being set and the WDE bit being set. Unfortunately, in some cases that causes real problems – your code won’t work as expected.

Tomorrow, I’ll continue on this topic, but it’ll be a bit more fun, because there will be LEDs involved!

(Please ignore the cable on the left, I snatched the above picture from this post)

Update – see this excellent Wikipedia article for more details about bitwise operations.

The Ports library has had a BlinkPlug class for some time now, including a “pushed()” function to do all the debouncing. Unfortunately, that function turned out to be a bit harder to use than I originally intended.

Time to add some more code to the BlinkPlug class!

I’ve added a new “buttonCheck()” member, which returns events instead of state. That makes it a lot easier to detect when people press a button, which is usually all you’re after anyway.

Here’s a new button_demo.pde example sketch, which illustrates the new functionality:

Sample output:

As you can see, it’s now a lot simpler to detect when people press or release one of the two buttons on a Blink Plug. Each time you call buttonCheck(), you’ll get one of the following events:

ALL_OFF, ON1, OFF1, ON2, OFF2, SOME_ON.

You have to keep calling “buttonCheck()” reasonably often, at least 10 times per second, if you don’t want to miss any events. Calling it all the time in the main loop is fine. Keep in mind that ON1, etc. will be returned only once for each actual button press.

You can still call “state()” whenever you want, to check the position of either button. But when you use buttonCheck(), you should not call the old – now deprecated – “pushed()” function, as these two will interfere with each other.

This code is now part of the Ports library (subversion and ZIP). Gory details are in Ports.cpp, near line 230.

The last step is to program a sketch into the ATmega. This isn’t as straightforwards as with a JeeNode, because there’s no on-board FTDI or USB serial port hookup.

It’s fairly easy to create an FTDI connection, but even if you do, you’ll still need an ISP programmer to install a boot loader (see this recent post for some background).

So let’s hook up an ISP programmer first:

I’m using a somewhat weird setup: first of all, my cable connector was attached the wrong way around, so I always have to use this one in that weird folded-over position.

But a more important issue is that the ISP connection needs to use pins 1..6 of the 2×4-pin SPI/ISP connector on the JeeSMD. That doesn’t work with normal flat cable connectors, which assume 2×3 pins and are too wide to fit in a 2×4-pin header. My solution is to insert wire-wrap pins the wrong way around into the cable header. This effectively extends the connector, but now it won’t be as wide and it’ll fit just fine. Another solution would be to only solder 2×3 pins in the SPI/ISP position – you can always add two more pins later.

Once you’ve passed that hurdle, you can use any ISP programmer you like. There have been several posts about this on the weblog, as listed here.

Now, if you want to use FTDI, then presumably you just uploaded a bootloader into the ATmega, with all the proper fuse settings, etc. The next step then, is to somehow connect to a 6-pin FDTI header.

There are several ways to do this. The one I use nowadays, is through a Carrier Board, which includes the 6-pin FTDI connector:

The point about the FTDI connector, is that it’s almost trivial. All you need is 4 wires to GND, PWR, TX, and RX – plus a way to reset the board from the RTS signal. The clever way to generate a reset is to insert a 0.1 µF capacitor between the serial side RTS and the ATmega’s reset pin. Tiny trick, huge implications (does the name “Arduino” ring a bell?).

So how does the Carrier Board implement FTDI? Easy: it adds the capacitor. And you can easily do that yourself without a Carrier Board. Here’s how:

Note that what FTDI calls “RX” is connected to what the ATmega calls “TXD”, and vice versa. It’s all a matter of perspective… Once you have the FTDI connection set up, you can upload sketches using the Arduino IDE just as with any other board. All you need is a USB-BUB or some other equivalent USB-to-FTDI interface.

Congratulations: that’s all it takes to build and use the Arduino-compatible JeeSMD!

You won’t be able to do this without at least a fine-tipped (0.4..0.6mm) soldering iron plus the following tools:

A magnifier lamp also helps, I know I couldn’t do this without one anymore!

This is a step-by-step guide on how to assemble the SMD kit, which consists of these parts:

The tiny ones (don’t sneeze!) are hardest to tell apart:

(thanks to Steve E for the macro shot – I just added some labels)

There are 2 10 kΩ resistors in there, although you only need one. That lets you get started without having to worry too much – if you mess up completely, just remove it and start over with the other one.

For a fantastic resource with detailed videos about hand-soldering SMD, see this Curious Inventor page.

So let’s get started. First thing to do is to apply flux wherever you’re going to solder things. The flux is essential because the flux in your solder wire will have evaporated longe before you can move your tip from the wire to the part.

I’m right-handed, so that’s how I hold my soldering iron. That leaves only my left hand for the tweezers – and no hand at all for the soldering wire:

Use your tweezers for all these parts, and don’t let go – once a 0603 part flies off or drops on the floor, your chances of finding it again are next to zero. Best is to work on a clean flat surface with everything around you removed.

The trick is to place the part and then push it down while you touch it with your iron with solder already on it. The moment a part is soldered down on at least one pin, it becomes a lot easier:

The matte “stain” you see around these pads is the flux, which has dried up but is still active.

(Remainder continued after the break…)

Ok, apply solder and add the other three 0603-sized caps. Don’t forget to add solder to both sides, of course:

It’s not hard, it just takes some patience.

Try to get better at it, and don’t touch the parts longer than necessary. These ones will take a lot of beating, but the next ones are less happy with too much heat.

Next is the voltage regulator, it’s in a 3-pin SOT-23 package. Again, once the first pin is soldered on, the rest is relatively easy:

Now a tricky one: the 16 MHz resonator. This is a three-pad package. I tend to put it slightly off-center, and always start with the middle pad:

Make sure it’s properly aligned, and make sure it’s still flush against the PCB. Otherwise, re-heat from the side until everything is right. Be patient – you can fix it with just one pad soldered, you can’t fix it later.

I use the middle pad, because it’s easy to check whether the connection is good: just try to lift the part with the tweezers. If well-attached, the board will lift as well.

Now carefully apply a little solder to the other two pads. The top is a metal can, you don’t want any of the solder joints to connect to that:

Now comes the pièce de résistance – the ATmega328 as a 32-TQFP package. Start by soldering only 2 pins on opposite sides of the chip:

Whoops – I used too much solder and created a bridge. Help!

No problem. Ignore it. This happens all the time. Just make sure the chip is well-aligned on the pads, and don’t proceed before it is. Fix the problem now, because there will be no way to do so later. Once you’re satisfied, do all the remaining pins:

Yep, two more solder bridges. Again, no problem.

Once all the pins have been soldered, make sure each one of them really has a solder joint to the pad. If not: add flux, and touch the pin with a drop of solder again. All that matters here is that every single pin is connected to its pad.

Now we’re going to fix the solder bridges. That’s where the solder wick comes in. Solder wick is nothing but thin copper braid, which clings to solder like crazy once hot enough. So the idea is to put a fresh & unused bit of wick on top of the pins with too much solder, and then heat the whole sandwich with the iron. At one point, the wick wil start to soak itself with solder, removing it from the pins and pads below.

This process will make the pads look completely different, almost as if there is no longer any solder, even:

Not to worry. If the ATmega pins are flat and flush against the pads, then there wail always remain a bit of solder between them (due to capillary forces) – exactly where it matters.

Every time you use solder wick, you have to cut off the bit soaked in solder, to have a new piece ready for the next repair. Note that copper is a very good conductor – also of heat – so the wick can get very hot, everywhere.

Ok, if you’ve got this far: you’re done! Congratulations. Your board will now look something like this:

Tomorrow, the easy final part: adding headers, programming it, and trying it out.

(Reminder: the Jee Labs shop will be closed starting July 14th. All orders before today will be sent out before then – I hope! – the rest as well, but I might run out of stock on certain items. Please get in touch to check if you don’t want to risk having to wait until the end of August)

I’d like to go through some terms and buzzwords to try and clarify how to get your Arduino or JeeNode to do the thing you want it to do. I’m going to assume that you are familiar with the process of writing software (“code”), compiling it, and running it – at least on a Windows, Mac, or Linux computer (let’s call them all PC’s for now). With software for the ATmega, there are two approaches, depending on whether you use the Arduino IDE or not:

Both lead to the same result: a “hex” file with code that needs to be transferred from the PC to the ATmega.

The step which can be puzzling when starting out with physical computing and embedded hardware is how to get things across from your PC to that little Arduino or Arduino-like system you’re holding in your hand. And vice versa, since we often want to get results back or see some confirmation that things are working properly.

The confusion comes from the different conceptual levels involved, and things like:

• do you mean how to connnect? – plug in a USB cable
• do you mean how does the ATmega change its own code? – through a boot loader
• do you mean via ISP (In System Programming)? – no, that’s normally not needed
• do you mean via FTDI? – yes, that’s the name of the chip which is hooked up to USB
• isn’t FTDI a connector? – yeah, that too, sort of… i.e. a 6-pin convention
• isn’t a power + serial cable enough? – no, resets also play a key role
• do I have to use the Arduino IDE? – no, you can also use a program called “avrdude”
• what’s avrdude? – a program which can upload to a boot loader or an ISP programmer
• do I need an ISP progranmmer? – nope, the boot loader does essentially the same thing
• so why not just get rid of ISP? – because you need ISP to install the boot loader

Confused? Welcome to the club…

In case you’re wondering… the process is called “uploading” because the PC initiates this as outbound transfer:

To get your code (“sketch” in Arduino-speak) into an ATmega, you need three things, working together to make uploading happen under all possible conditions:

1. An electrical connection – to power the ATmega, to communicate with it, and to restart the ATmega when it is stuck or busy doing something else.

2. A common language / protocol – the communication must be understood by both sides, i.e. PC and ATmega.

3. Software on both sides of the connection – Sending something when the other side isn’t listening, or listening while no-one is sending will not have the desired outcome.

Let’s go through each of these separately.

1. An electrical connection

There are several ways to make the connection. With an Arduino, or any similar board which has a USB connector, you can simply plug in the USB cable:

Some boards use a separate USB interface (“FTDI adapter”), allowing reuse of that interface for multiple boards:

The end effect is the same: a connection which powers the ATmega and allows communicating with it using a simple serial protocol. There’s also a trick in this hookup to let the PC reset the ATmega whenever it wants.

2. A common language / protocol

Ah, now it gets interesting. First thing to note is that there is no single common language / protocol!

That’s right. It all depends on what you want to do. Here we want to upload code from the PC to the ATmega. That requires exchanging “ISP commands” over the connection. But once uploading is done, we really want to reuse the connection as a regular two-way serial link.

The way it works is that the PC will reset the ATmega just before uploading new code. This activates a “boot loader” on the ATmega. Now both sides will be in sync (briefly) so they can exchange the necessary information to make an upload happen. IOW, right after reset the protocol is “ISP commands”. Once the upload is done, the connection can be reused for any protocol you like – as determined by the code that was uploaded.

3. Software on both sides of the connection

Back to the software now. We need to send ISP commands over the connection.

As you may have guessed, that’s exactly what the Arduino boot loader on the ATmega understands. When reset, the boot loader in the ATmega gets control. It waits and listens for incoming STK500 ISP commands. If none come in within a second or so, it relinquishes control to whatever was previously uploaded to the ATmega.

On the PC side, we need software which resets the ATmega and then immediately sends all the ISP commands needed to transfer and program the contents of a hex file.

This is what “avrdude” does. You can either use it implicitly from the Arduino IDE by starting an “Upload” from the menu, or you can launch it manually from the command line – avrdude needs a few options to tell it where the USB port is, what baudrate to use, the type of ATmega, the protocol to use (i.e. STK500), etc.

There’s more…

The above describes the different pieces and concepts involved in getting code into an ATmega. The beauty of it is that once it works, it really works well. Supplying power, uploading, communication, control, debugging … all with one simple USB cable. You only need to go back a few years to appreciate just how much this approach simplifies embedded development.

But there’s one important detail: the ATmega has to have a functioning boot loader. Placing a boot loader into an ATmega is a bit more complicated (and involves other things such as “fuses”). It’s a chicken-and-egg problem.

This is where the ISP programmer comes in. An ISP programmer is a fairly simple piece of hardware. In fact, you can make your own, as I’ve described in several posts on this weblog. See this and this post for some quick solutions which require nothing more than a working Arduino or JeeNode.

The good news, is that you usually don’t have to worry about installing the boot loader – it’s all done for you. Once. For the mechanism described above, you’ll never need an ISP programmer.

Some people actually prefer to use the ISP technique for uploading their sketches. In fact, sometimes you have no choice, i.e. when you need the serial port at all times, or when you can’t spare the 1..4 Kb required by the boot loader code, or when working with ATtiny chips which don’t support bootloaders.

In thoses cases, you’ll need a setup with an ISP programmer. But for most people doodling around with ATmega’s and the Arduino IDE, the above boot loader mechanism is usually very convenient and the easiest to get going.

Either way, it helps to understand the process. I hope the above was helpful in that direction.