Computing stuff tied to the physical world

Archive for 2010

,

FritzBox call log

In Software on Mar 12, 2010 at 00:01

I’ve been using the FRITZ!Box 7170 as ADSL modem and telephone interface for some time now:

Screen shot 2010-03-08 at 12.50.41.png

It has a nifty feature: when you enable the built-in call monitor by typing in “#96*5*”, you can connect to TCP/IP port 1012 to get real-time messages about all phone calls.

This is easy to hook up to JeeMon. I added a new “FritzBox.tcl” file with the following code:

Screen shot 2010-03-08 at 13.15.39.png

(Some experimental code was omitted from the above listing)

To enable this, the following line was added to application.tcl:

Screen shot 2010-03-08 at 13.14.42.png

Here’s an example of an incoming call, as shown in the JeeMon log:

Screen shot 2010-03-09 at 13.51.12.png

And here’s an outgoing call:

Screen shot 2010-03-08 at 12.55.09.png

Neat! I’ll hook this up as events once the JeeMon event system is ready.

These experiments are very useful to see how the current design is working out. The above code depends on the “MessageStream” class in the “Serial” rig. It doesn’t really add much value in this case, but some class hierarchies seem to be emerging. This socket connection is an example of a similar-but-not-quite-the same case as serial connections, requiring its own class. A socket connection is very similar to a serial connection once initialized – but this doesn’t show as clearly in the code as I would like and there’s already a bit of code duplication.

Will probably need some re-shuffling later, to better match concepts / design and implementation.


Outbound X10

In Software on Mar 11, 2010 at 00:01

I’ve added support for controlling X10 devices via the CTX15 module described in the past two days:

Screen shot 2010-03-09 at 20.31.20.png

First thing was to extend the ookRemote GUI, by adding these definitions to ookRemote.tcl:

Screen shot 2010-03-09 at 20.34.10.png

With this change, the configuration section in “config.txt” supports CTX15 buttons:

Screen shot 2010-03-09 at 20.34.56.png

These new buttons will send X10 commands to the CTX15 module. But this needs a bit of wrapping to turn them into frames, including the proper checksums. So I added two methods to “sketches/ctx15try/host.tcl”:

Screen shot 2010-03-09 at 20.37.16.png

The “send” method overrides the existing one and sets up a frame before calling the original method with it.

That’s it. I can now make the Marmitek AM12 switch “clunk” with my mouse – audible feedback! :)


JeeMon interface for X10

In Software on Mar 10, 2010 at 00:01

Now that the CTX15 has been connected via a JeeNode, it’s fairly straightforward to tie it into JeeMon.

First I created a new folder called “sketches/ctx15try/” with a file “host.tcl” in it, using this code as quick test:

Screen shot 2010-03-08 at 14.37.36.png

The reason there is nothing more, is that I changed the ctx15try.pde sketch a little to return lines starting with “CTX15″ – since these are the easiest to tie into JeeMon. The change affects these lines in yesterday’s demo:

Screen shot 2010-03-08 at 14.33.08.png

I’ve also wrapped the received frame in {}’s, because that simplifies processing in JeeMon (Tcl interprets $’s and []’s unless escaped).

To activate everything, I added an extra line to the config.txt configuration file, so that the output from the JeeNode USB (serial# A900adwo) automatically gets picked up as sketch:

Screen shot 2010-03-08 at 15.01.22.png

Here is an example of the above code running:

Screen shot 2010-03-08 at 14.36.50.png

Good. CTX15 events are coming in. They are easily recognizable as (unit#, command) pairs.

But as you can see, these pairs don’t always arrive in the same frame. So we have to add a bit of logic to get things right. Here’s an updated version of host.tcl:

Screen shot 2010-03-08 at 14.51.50.png

The solution used above is as follows: if it looks like a unit code (A..P + 1..16), then save that as last one seen. If it’s anything else, use the last saved location. I’m not sure this covers all possible scenarios, but for this very simple test it seems to work. Here’s a screen shot of the real-time status window:

Screen shot 2010-03-08 at 14.50.48.png

As you can see, the CTX15 event was turned into an “AOFF” command for the “ctx_A01″ device, which was created on-the-fly by this code.

Hmm, looking further, I think the “A” prefix needs to be stripped from “AOFF”. Oh well, it’s all scripted, I’ll adjust that when I get to using this stuff for real.

So that’s it. A few dozen lines of C code in the JeeNode to act as pass-through and do the polling, and a few dozen lines of Tcl code to tell JeeMon how to interpret the incoming data. Now we get notifications whenever any X10 activity is detected on the mains power line.

Onwards!

,

Xanura CTX15

In Hardware, Software on Mar 9, 2010 at 00:01

Another popular home automation module is the Xanura CTX15:

DSC_1228.jpg

That’s a live 220V power-line connection in the top right corner!

I’ve connected the CTX15 module through a UART Plug, with a 4.7 kΩ resistor in series with the RX signal (yellow from CTX15). This is because the signal swings up to 5V, whereas the UART plug only accepts 3.3V voltage levels. The module is powered from the PWR pin, which on a JeeNode USB carries 5V.

The CTX15 is a bi-directional interface, it can send as well as receive A10-type power-line commands (a superset of X10). The trouble is that it needs to be polled to read out what has been received and buffered so far.

Here’s a sketch which takes care of that:

Screen shot 2010-03-08 at 12.08.53.png

And here’s some sample output:

Screen shot 2010-03-08 at 12.08.40.png

As a test, I powered up yesterday’s XM10E test setup as well, to send out on and off commands to unit A.1 every 3 seconds. As you can see with the CTX being read out every 5 seconds, multiple received packets will sometimes be combined and returned as one reply.

,

X10 control

In Uncategorized on Mar 8, 2010 at 00:01

Let’s go some more into home automation – the X10 power-line system in this case.

Here’s is a sketch which turns a remote appliance on and off every 3 seconds:

Screen shot 2010-03-07 at 23.06.56.png

I used the Arduino X10 library (had to mess around with the use of headers to get rid of compile errors, and enable the pull-up on the zero-crossing input).

This sketch runs @ 3.3V on a JeeNode with some stuff attached to port 1:

DSC_1224.jpg

It’s just a simple re-wiring to an RJ11 connector. This in turn, plugs into the XM10E opto-isolated interface:

DSC_1225.jpg

There’s also a X10 receiver built into the XM10E, which I’m going to ignore for now.

X10, in its simplest and oldest form, is a power-line transmission system, i.e. the signals to control a switch are sent over the same wires as the AC power itself (using a 120 KHz signal injected at the zero crossings). Proper mains isolation is essential, of course – as built into the XM10E unit.

The result is that you can plug this device into any outlet in the house and it’ll switch on and off as defined in the above sketch:

DSC_1226.jpg

It’s hard to miss – the relay built into that thing switches on and off with a very loud “clunk”!

The power consumption of this switch is 0.6 W off and 0.9 W when on, according to the Cost Control. Not bad, until you start installing many dozens of these switches around the house … then it will add up!

,

New board summary

In Hardware on Mar 7, 2010 at 00:01

Here is a summary of the boards which are currently being, eh … prepared? etched? sprayed? drilled?

First of all, a new JeeNode USB with improved voltage regulator setup:

Screen shot 2010-03-07 at 00.53.02.png

The Dimmer Plug contains a 16-channel LED dimmer with independent PWM brightness control, using I2C:

Screen shot 2010-03-07 at 00.53.25.png

The Lux Plug contains a sensitive light meter with high dynamic range, using I2C:

Screen shot 2010-03-07 at 00.53.36.png

The Gravity Plug contains a 3-axis accelerometer with 2..8G range, using I2C:

Screen shot 2010-03-07 at 00.53.45.png

The Proximity Plug connects up to 8 contacts as capacitive sensing switches, using I2C:

Screen shot 2010-03-07 at 00.53.58.png

The Input Plug supports 16 inputs, as either analog or digital inputs (not I2C):

Screen shot 2010-03-07 at 00.54.09.png

And three more boards. One of them is a mystery sensor – I’ve masked out its name for now :)

Screen shot 2010-03-07 at 00.56.00.png

The other two boards are experimental boards of a different kind – I’ll describe ‘em when they work.

I’ve started setting up documentation pages for these new PCBs, but so far that’s just the EAGLE files. Will add more info in the coming week – with luck they’ll be back here for testing within the next 10 days.

I guess the best tactic for me now, is to just forget about this and work on some other stuff for a while. Drat.

Patience, patience, patience. I hope it pays off!


Catching sun rays

In Hardware on Mar 6, 2010 at 00:01

A few days ago, this country went through a curious ritual…

In the Netherlands, there is a subsidy as encouragement for people to place solar panels on their roof tops.

Screen shot 2010-03-05 at 23.03.57.png

The system works as follows:

You’re considering setting up a couple of photovoltaic solar panels.
You quickly find out that the government is handing out money for this.
Yippie … and then you get into the rules and regulations bit:

  • You can’t apply for the subsidy before March 1st.
  • There is a limited amound of funding.
  • On the day when submissions exceed funding … it’s roulette day!
  • … the remaining subsidies are assigned at random.

The result?

On March 1st – everyone submits their proposal for solar energy.

On March 2nd – sorry folks, we’re all sold out, try again next year!

It turns out that some 17,000 people signed up for 2010. And about 5,000 of them will win that lottery ticket.

Tell me, please, what’s the meaning of the word “encouragement” again?

Anyway, so much for silly incentives. The reason I’m posting this is that we’ve been looking into solar energy and have decided to install 18..20 solar panels on our roof. It’s relatively well-positioned, and even the cabling in the house turns out to be very easily adapted for it. And yes, I’m one of the 17,000 who signed up for the Dutch Solar Casino. Except that we’re going to go ahead regardless of the outcome (3 months from now!).

The numbers are much better than I expected:

  • With 18..20 panels, we can fill the smallest half of our roof
  • Modern panels can produce about 3600 “Watt peak” total
  • Heading and tilt are such that we’re still at ≈ 90% of optimal
  • Estimated yearly output will be 2500 kWh, 20% more than we need
  • Unused power is fed to the grid and billed at the same rate, as credit
  • In other words, the electricity grid will become a – very – big capacitor for us :)

All in all, I think it’s a good deal. There will always be better deals in the future, but this is fine.

As for buying more stuff to save and join the green hype: panels also need to be produced and shipped. The energy needed to do so appears to be less than what those panels produce in their first 18 months. So the electricity in the next 20..30 years after that is free, green, and effortless.

I was very surprised to find out that solar energy is practical at 52° latitude, but it really is.

Sooo… apart from seasonal swings and energy storage, Jee Labs will be “slightly off the grid” later in 2010!


JeeMon goes embedded

In Hardware, Software on Mar 5, 2010 at 00:01

Testing out a new setup … I started by launching the JeeMon runtime executable as follows:

Screen shot 2010-03-02 at 15.41.09.png

Then I went back to my Mac and opened a browser window:

Screen shot 2010-03-02 at 15.56.12.png

It works! You’re looking at a little web server running on a tiny Technologic TS-7500 board:

DSC_1213.jpg

ARM based, 64 Mb RAM, µSD card socket on board, running Debian (lenny). It’s not very high powered, but in return it only uses 400 mA @ 5V, i.e. it can run off USB power.

I haven’t yet figured out how to get the FTDI USB serial driver on there, but once that’s solved this little unit will be able to act as host for USB-connected JeeNodes or JeeLinks.

The RAM and storage is more than sufficient to run even a very elaborate JeeMon setup, I expect.

This might be an interesting low-end always-on home sensor & automation system!


Packaging madness

In Musings on Mar 4, 2010 at 00:01

I ran out of little zip-lock bags. The ones I put small 6-pin headers in, and such.

So I ordered a few more by direct mail, from a Dutch office supplies shop on the web.

Sure enough, one day later – a big courier delivery truck from DHL stops by and delivers a 25×35 cm box with the requested goods:

DSC_1212.jpg

They could have dropped it in a padded envelope, added €1.60 postage, and dropped it in the mail. Which – in the Netherlands – is guaranteed to reach me in 24 hours, IOW just as quickly, along with all my other mail. Delivered to my doorstep by our friendly mailman, who services the whole neighborhood … on bike.

Instead, I’ve caused this ridiculous packaging and delivery nonsense. Yuck.

Update – I’ve re-ordered some more (slightly larger) bags from another Dutch supplier suggested by someone after reading this post. Not only did that supplier do the right thing – they even alerted me to the fact that another item would have pushed postage up to an unreasonable level, and proposed to omit that extra item. Hurray for vendors who use their common sense when serving their customers!

,

Pushing data as web client

In Software on Mar 3, 2010 at 00:01

Another way to get data to another place is via push HTTP requests, i.e. acting as a client for another web server somewhere on internet.

The Pachube site is one place to send measurement results. I’ve set up a test feed to use with JeeMon, with two data streams. One of the nice features is that you can easily produce and embed graphs from that system:

Screen shot 2010-02-26 at 02.45.13.png

Here’s how I implemented it in JeeMon. First, I created a configuration section:

Screen shot 2010-02-25 at 22.37.12.png

(I’ve omitted most of my private API key, but anyone can sign up and get their own …)

Next, the code, in the form of a “pachube.tcl” file / module / rig:

Screen shot 2010-03-02 at 03.11.12.png

Note how the configuration file in fact contains “settings” which are simply evaluated as a script: the two lines starting with “Fetch …” are handled by the Fetch proc in pachube.tcl, using the JeeMon notification mechanism to get called back whenever either of the “meter1″ or “meter2″ readings changes. The Update proc then updates the “latest” variable accordingly. Lastly, SendToWebSite periodically does an HTTP PUT request in the background, supplying all the info needed to successfully submit to the Pachube website.

Btw, this simple example is flawed, in that it does not calculate averages – it just sends the last reading. But things like averaging require a sense of history, and persistence. Haven’t added that to JeeMon yet…

The missing link, as usual, is a line in the application file to start the ball rolling:

Screen shot 2010-03-02 at 03.03.46.png

There’s a lot of flexibility at the protocol and network levels. Such as creating an XML request via templates, if the remote server needs XML. This isn’t limited to HTTP requests, or to using port 80 – send emails, FTP files, etc.

There are quite a few details in the above code which I won’t go into. Again, I’m doing this mostly to show how little code it takes to initiate periodic HTTP client requests to an outside website.

Apart from sites such as Pachube, this could also be used from a tiny embedded Linux running JeeMon, to submit incoming data to a different setup elsewhere on the LAN, for example. IOW, JeeMon can be a front-end for other systems. It’s not about lock-in. It’s a switchboard. It can glue systems together. It bridges gaps.

,

JeeMon as web server

In Software on Mar 2, 2010 at 00:01

JeeMon comes with a nifty built-in HTTP/1.1 web server (using coroutines to handle requests in parallel).

So let’s create another real-time status display, but as web application this time:

Screen shot 2010-02-25 at 17.56.25.png

As before, I’m leaving out the lipstick – just showing the core functionality.

Here’s what I did to generate this page. First of all, I’m re-using some functionality which is already in the GUI version, to dynamically construct a matrix with the proper columns and rows, so the “statusWindow” code has to be running for the above to work. For real use, that code should probably be reorganized into its own rig.

The main task was to create a HTML template in a file called “statusPage.tmpl”:

Screen shot 2010-02-26 at 00.41.23.png

This uses a simple templating mechanism based on Tcl’s “subst” command. The meta tag sets up the page to self-refresh every 5 seconds. The last line tells the Wibble web server to deliver the page as HTML.

To launch the server, this line was added to the main application file (port 8080, current dir is doc root):

Screen shot 2010-02-25 at 18.00.15.png

That’s it – there’s very little to activating a web server in JeeMon, as you can see. Which is important, because on tiny embedded Linux systems, an HTTP server will probably be the only option to present information.

Creating a full-blown site with CSS, JavaScript, and Ajax is a matter of adding more files – the usual stuff…

Did I mention that it’s all 100% open source, so you can browse / extend / change all of this? – I did? Oh, ok ;)

,

Node discovery

In Software on Mar 1, 2010 at 00:01

This post isn’t about GUIs, but about the dynamic behavior behind them.

And about wireless. Let’s take the voltmeter setup I described recently, and make it send its readings over the air – it’s a JeeNode after all, so it has wireless connectivity built in.

I extended the original sketch to send readings out roughly once per second:

Screen shot 2010-02-25 at 15.57.03.png

Extending the host side to also work from received packets is trivial:

Screen shot 2010-02-25 at 16.49.26.png

The problem now, is that this node will start sending out packets, but JeeMon has no idea what to do with them. One day, automatic node type registration would be great to add, but for now I’ll just use the configuration file to associate nodes with sketches. Here’s an extract of my current “config.txt” file:

Screen shot 2010-02-25 at 16.00.30.png

For this example, I added the “3/ { type analogPlug }” line to the “868:5/” entry (i.e. 868 MHz, netgroup 5). And sure enough, JeeMon will now launch the GUI for this node, displaying results coming in over the air in real-time:

Screen shot 2010-02-25 at 16.03.00.png

Here’s the relevant part of the log output:

Screen shot 2010-02-25 at 16.04.34.png

The fun part is that a timeout mechanism has also been implemented. If no packets come in from the analog node for 5 seconds, the connection is torn down and the window is closed again. The timeout value is sketch-specific and is included in the above configuration info.

In other words: node is ON => window appears, node is OFF => window goes away. Look ma, no hands!

Note: it’s not good UI practice to open and close windows like this. In a more refined setup, these windows would be panels in a larger window, or they could just be grayed-out to signal their disconnected state, for example.

The analog node is surprisingly convenient this way. I can use it anywhere, turn it on, look on the screen, and turn it off when I’m done. It wouldn’t be hard to add another wireless node to show the results on an LCD screen, btw. If this is extended with a way to assign node IDs ad-hoc, and a way for a sketch to identify itself over the air, then a whole family of wireless “instruments” could be created – using nothing but a JeeNode and some plugs.

This illustrates a guiding principle in JeeMon: don’t ask, just do it! – JeeMon tries to be as dynamic as possible, adapting to changes in status and configuration by adjusting everything it does accordingly. In the case of nodes, new ones are discovered and stale ones are weeded out, in real-time. With objects getting created and destroyed automatically. What these objects do, as side-effect, is up to each one of them,

Right now, this dynamism in JeeMon is still fairly messy. There’s too much state in too many places which needs to be managed carefully. But with a bit of luck, this can eventually be simplified further to create a very consistent and predictable behavior.


A real-time status GUI

In Software on Feb 28, 2010 at 00:01

One more post about setting up a GUI with JeeMon…

I wanted to have a basic real-time display of all the readings coming in, sorted by source. Like this:

Screen shot 2010-02-25 at 15.48.08.png

(the size of this screen shot was reduced to fit this post)

Everything in this window is dynamic: rows get added when new sources start reporting their data, and columns get added when new parameter types are reported. I’m not storing any information persistently yet, which is why this status window looks as follows right after starting up JeeMon:

Screen shot 2010-02-23 at 21.31.15.png

Not very exciting, but all you have to do is wait …

Let’s see what it took to create this GUI. First, I added a generic “Notifier” rig to JeeMon, and a “Reading” method for all sketches to use. The rooms sketch, for example, now reports its incoming results as follows:

Screen shot 2010-02-23 at 22.06.04.png

Similar changes were made to the ookRelay and RF12demo “host.tcl” sources.

The rest of the GUI code is in “statusWindow.tcl”:

Screen shot 2010-02-24 at 01.16.42.png

The tabular display is based on a very nice Tcl/Tk open source package called Tablelist by Csaba Nemethi, which has been added to the JeeMon core library.

The coupling with the rest of the system is accomplished by the “Notifier attach readings * …” call at the end of the setup code. This registers a callback which will be invoked when anything in the “readings” notification group changes. Notifiers are going to be used a lot in JeeMon because they provide such a manageable way to glue independent parts of an application together.

And lastly, adding the following line to the main application starts the ball rolling:

Screen shot 2010-02-23 at 21.42.16.png

There’s quite a bit going on here which I won’t go into. I just wanted to illustrate how little code it takes to create a basic, fully dynamic, cross-platform, self-updating status display window. There – how’s that for adjectives!

,

Voltmeter with GUI

In Software on Feb 27, 2010 at 00:01

Let’s make a 5-digit voltmeter with a JeeNode, an Analog plug, and JeeMon. But instead of a readout on an LCD screen, I’m going to show the results on the desktop. To start with that last part, here’s my little display on a Mac:

Screen shot 2010-02-23 at 20.23.25.png

You’re looking at my feeble attempt to create a good-looking / souped-up GUI, btw :)

Anyway. Here’s the sketch I’m running on the JeeNode, with an Analog plug inserted into port 3:

Screen shot 2010-02-23 at 11.01.16.png

Trivial stuff. It sends out lines with “VOLT <value>” readings twice per second.

The important part is that this sketch starts up with “[analogPlug]” as greeting. To hook up to this sketch, all we need to do is create a directory called “analogPlug”, with a file called “host.tcl” in it.

This new “analogPlug” directory is a sub-directory of “JeeMon-sketches”. This directory, in turn, has been registered as being used for sketches – with this command in my main application code:

Screen shot 2010-02-23 at 12.21.51.png

Back to the “JeeMon-sketches/analogPlug/host.tcl” file. Here’s its contents:

Screen shot 2010-02-23 at 11.40.34.png

That’s the whole kaboodle.

Now, whenever I plug in a JeeNode running the above “analogPlug.pde” sketch, a window pops up showing readings in real-time. When I unplug the JeeNode, the window is closed again. It’s all automatic, as long as JeeMon is kept running.

This mechanism is not limited to JeeNodes or Analog plugs, of course.


OOK remote control

In Software on Feb 26, 2010 at 00:01

The JeeMon story continues. Here’s a fun little panel for controlling some switches around the house:

Screen shot 2010-02-22 at 21.42.14.png Screen shot 2010-02-22 at 23.31.35.png

Screen shot 2010-02-23 at 20.19.28.png Screen shot 2010-02-22 at 21.51.23.png

These screen shots show Windows (7 and 2K), Mac OS X, and Linux Ubuntu, respectively – long live diversity!

This GUI was created with these configuration settings:

Screen shot 2010-02-26 at 02.09.33.png

Those settings were loaded by adding this line to the main application code:

Screen shot 2010-02-26 at 02.09.16.png

Which in turn loaded the following “ookRemote.tcl” source file I wrote, as module / rig:

Screen shot 2010-02-26 at 02.08.48.png

That’s the entire app. Adding lipstick (i.e. a fancier visual design) is left as exercise for the reader.

Note how the design is split in a configuration section and a source code file – I’m probably going to use this approach often in JeeMon. The idea is that people who don’t care about the code behind all this stuff can just adjust the configuration file (using a visual interface, hopefully, one day). While those who like to tinker can view the full source code, and explore and extend it at will, and more importantly: ad infinitum.

It’s called open source for a reason!

PS. There’s another major reason for the split between configuration settings and source code files: turnkey deployment. You can put all the source code files in a ZIP archive called “JeeMon-rigs.zip”, so that all you need for a new installation with JeeMon, is: 1) your archive, 2) the proper JeeMon runtime, and 3) a config file to match the target setup. The config file is optional, btw – more on that another time…


Output and Expander plug fixes

In Software on Feb 25, 2010 at 00:01

While testing an Output Plug, I found a little mess-up: the documentation pinout is wrong, all even and odd I/O pins on the 2×6 output connector were swapped. Here is the correct pinout:

op1-pinout.png

I’ve updated the documentation.

The “expander” sketch is also very confusing, it was still for the PCA8574A, whereas the current Expander Plug uses an MCP23008.

Here’s an improved sketch, with a little running-light demo:

Screen shot 2010-02-22 at 12.38.18.png

This code has been checked in to replace the “expander.pde” sketch in the Ports examples.

,

Fabrications

In Hardware on Feb 24, 2010 at 00:01

Several hours of work, and this sort of stuff is born… 16 new JeeNode USB’s:

DSC_1203.jpg

It looks pretty, but I’m not going to show you the high resolution picture… even on this one you can probably spot one or two problems :)

The fact is that hand-assembly of SMD takes a lot of patience. By now, I think it’s absolutely doable for one-offs, such as to assemble one board with some parts. But larger quantities do tend to bring out the trouble spots.

Usually, on a run this size, I’ll expect to see 3 or 4 boards not working right away, and ending up with perhaps one board which I can’t fix quickly. I’ve got a box with a bunch of units by now, which don’t seem to want to be cured. Oh well, their loss – that means they get to wither away in a dark box, waiting for better times :)

,

Secure transmissions

In Software on Feb 23, 2010 at 00:01

For some time, I’ve been thinking about adding optional encryption to the RF12 wireless driver. I’ll leave it to the cryptography experts to give hard guarantees about the resulting security of such a system…

The basic goal is to provide a mechanism which lets me get messages across with a high level of confidence that an outsider cannot successfully do the same.

The main weakness which most home automation systems such as FS20 and KAKU completely fail to address is message replay. The “house code” used by FS20 has 16 bits and the address has 8 bits, with the naive conclusion being that it takes millions of attempts to send out messages to which my devices respond. Unfortunately, that’s a huge fallacy: all you have to do is sit outside the house for a while, eaves-dropping on the radio packets, and once you’ve got them, you’ve figured out the house code(s) and adresses in use…

I don’t care too much about people picking up signals which turn the lights on or close the curtains. You don’t need technology to see those events from outside the house anyway. I do care about controlling more important things, such as a server or a door opener.

Here are my design choices for optional encryption in the RF12 driver:

  • The cipher used is David Wheeler’s XXTEA, which takes under 2 Kb of code.
  • The keys are 128 bits, they have to be stored in EEPROM on all nodes involved.
  • All nodes in the same net group will use either no encryption or a shared encryption key.
  • A sequence number of 6, 14, 22, or 30 bits is added to each packet.

To start with the latter: XXTEA requires padding of packets to a multiple of 4 bytes. What I’ve done is add the sequence number at the end, using as many bytes as needed to pad to the proper length, with 2 bits to indicate the sequence number size. Encrypted packets must be 4..62 bytes long. It’s up to the sender to decide what size packets to send out, and implicitly how many bits of the sequence number to include. Each new transmission bumps the sequence number.

To enable encryption, call the new rf12_encrypt() function with a pointer to the 16-byte key (in EEPROM):

Screen shot 2010-02-21 at 18.37.36.png

Encryption will then be transparently applied to both sending and receiving sides. This mechanism also works in combination with the easy transmission functions. To disable encryption, pass a null pointer instead.

The received sequence number is available as a new “rf12_seq” global variable. It is up to the receiver (or in the case of acks: the originator) to ascertain that the sequence number is in the proper range. Bogus transmissions will decrypt to an inappropriate sequence number. To make absolutely certain that the packet is from a trusted source, include some known / fixed bytes – these will only be correct if the proper encryption key was used.

This new functionality has been implemented in such a way that the extra code is only included in your sketch if you actually have a call to rf12_encrypt(). Without it, the RF12 driver still adds less than 3 Kb overhead.

I’ve added two sample sketches called “crypSend” and “crypRecv” to the RF12 library. The test code sends packets with 4..14 bytes of data, containing “ABC 0123456789″ (truncated to the proper length). The receiving end alternates between receiving in encrypted mode for 10 packets, then plaintext for another 10, etc:

Screen shot 2010-02-21 at 22.42.27.png

As expected, the encrypted packets look like gibberish and are always padded to multiples of 4 bytes. Note also that the received sequence number is only 6 bits on every 4th packet, when the packet size allows for only one byte padding. The strongest protection against replay attacks will be obtained by sending packets which are precisely a multiple of 4 bytes (with a 30-bit sequence number included in the 4 bytes added for padding).

So this should provide a fair amount of protection for scenarios that need it. Onwards!

,

Declarative configuration

In Software on Feb 22, 2010 at 00:01

Things are starting to come together quite nicely with JeeMon – all in less than 1000 lines of Tcl code so far. That’s not to say that it’s perfect yet – I’m still massively rearranging some of these code structures – but there is already some functionality to play with, which helps expose the strengths and weaknesses so far.

The basic approach is declarative in nature: specifying what should happen, but not necessarily in the same place as when or how all the work should be done.

Here’s my first tentative configuration file:

Screen shot 2010-02-21 at 14.31.44.png

For each serial interface, an action is specified when that interface comes on-line. For example I can now plug in my A8007Up6 device (a USB-BUB with a JeeNode) and JeeMon will open a terminal window showing the output stream. Unplugging closes the window again (and automatically cleans everything up).

Right now, I’m editing this config file by hand, but at some point that could be augmented with a set of GUI or web-based configuration panels (even remotely).

There are a lot of challenges ahead. One of them is decoupling things so that code and data end up organized in the most convenient, logical, and flexible way. I’m trying very hard to avoid any calls to the “Config” module from generic JeeMon code. This allows you to set up the configuration file structure and hierarchy in any way you like.

My application file currently looks as follows:

Screen shot 2010-02-21 at 15.42.58.png

As you can see, I’m setting up a periodic port scanner and supplying a “SerialEvent” callback to decide what to do when serial ports get added or removed. In those decisions, most of the logic is driven from the “config.txt” configuration file. I’m about to implement a similar callback for radio events, i.e. RF12 nodes coming online and dropping off again.

In a way, the above two files are the entire application.

The rest is utility code to make the above convenient and concise. That rest might well be 99% of the code, but it’s all structured as loosely-coupled “rigs” (modules), which can be used / overridden / ignored at will.

The way I see it, there can be three sources for such utility code: your own code, plug-ins from others, and a couple of rigs included in the JeeMon core itself. The above two files require just the core, using the built-in rigs (Config, Gui, JeeSketch, Log, Serial, and SysDep).

,

Serial USB devices

In Software on Feb 21, 2010 at 00:01

For JeeMon, I’d like to be able to auto-detect USB device insertions and to identify FTDI serial numbers (with names like “FTE54GKC” and “A9007CNE”).

Cross-platform… i.e. on Windows, Mac, and Linux!

It looks like it can be done.

On Windows, we can browse the good ol’ registry, as follows:

Screen shot 2010-02-20 at 15.24.16.png

Sample output:

Screen shot 2010-02-20 at 15.24.41.png

This was tested on Win2000 and Windows 7, in the hope that everything in between works the same.

On Mac OS X, it’s easiest (for a change):

Screen shot 2010-02-20 at 15.26.02.png

Sample output:

Screen shot 2010-02-20 at 15.26.37.png

On Linux, it’s a bit messy because there are so many different distributions:

Screen shot 2010-02-20 at 15.27.43.png

Sample output:

Screen shot 2010-02-20 at 15.28.01.png

Tested on Debian 5 (Lenny), Ubuntu 9.10 (Karmic), and Gentoo.

The above code can be found in the subversion repository, i.e. here.

I’m testing this all at once on a single machine btw, courtesy of VMware Fusion. And using JeeMon’s built-in self-update mechanism to quickly get new versions across while debugging and tweaking things.

By calling “SysDep listSerialPorts” periodically, we can automatically detect a change and see which plug was inserted (FTDI only for now). Without depending on any external libs or executables.

Onwards!

,

RF transport independence

In Software on Feb 20, 2010 at 00:01

With the basic serial interface access and dispatch in place in JeeMon, it’s time to move on to JeeNode / JeeLink packet processing.

What I want is to be able to forget all about how readings got to JeeMon. It shouldn’t make a difference whether I’m using a directly connected Room Node and grabbing the readings off its serial USB connection, or whether they came in through the air via the RF12 wireless driver – or by any other means. Take a snapshot with your cell phone, send the picture to EverNote, have it OCR’d, and then grab the readings off the web … whatever!

The way I’ve tied this into JeeMon, is to let the interface to RF12DEMO act as de-multiplexer. This is purely a decision at the RF12DEMO listener level. Each incoming packet is examined to determine which node it came from. Then we need a way to map from nodes to listener class – i.e. find out what sketch is running on the remote node. This is hard-coded for now:

Screen shot 2010-02-18 at 22.57.41.png

What this does is actually a bit more elaborate: a RF12DEMO listener will set up and manage a bunch of extra “RF12_PacketStream” listeners, one for each node. When packets come in, they will simply be dispatched to the corresponding stream. Each packet stream can process its packets in a different way.

The fun part is that these packet streams can use the same listener classes as with direct serial interfaces. All they need is an extra “decode_RF12″ method:

Screen shot 2010-02-18 at 23.02.04.png

The task of decode_RF12 is to re-cast the incoming packet as messages of the same structure as what would come in over a serial connection.

Here’s the “rooms” listener as example:

Screen shot 2010-02-18 at 23.04.34.png

This one class encapsulates all the protocol details of room nodes, both serial and via RF12. When a 4-byte data packet comes in via RF12 (as $a..$d), the bits are decoded and an “onMessage” call is generated with a “ROOM” message id and the 5 decoded readings.

Here is a log of this code in action, one message per line:

Screen shot 2010-02-18 at 22.37.15.png

The way to read these messages is as key-value pairs, i.e. id = OK, type = RF12, name = usb-A900ad5m, etc.

The first two lines show an incoming OK message from node 21 (53=21+32), which is then turned into a ROOM message, tagged as coming from the “rf12-868.5.23″ packet listener.

The next 3 lines are more involved: first an EM10 message came in over USB, then an OK message came in which got dispatched again, as the same EM10 message. That’s because I’m running JeeMon with a direct connection to the ookRelay board, even though it transmits all its information over RF12. So everything from the ookRelay comes in twice (great for debugging).

The point is that the two EM10 messages have the same info. It no longer matters how the message got here (but it is traceable, using the remaining details). And all the code to accomplish this is in a single source file, right next to the sketch running on the ookRelay board.

This design makes it possible to develop an application using only the serial USB connection, and then later add logic to send the information via RF12 (or not). Infrared, XBee, Twitter, anything: transport independence!

Note that nothing is done with these decoded messages at this stage. This messaging framework is independent of higher-level application decisions, such as where to store / send / show msgs, or even when to process them.

,

Serial port encapsulation

In Software on Feb 19, 2010 at 00:01

This post continues to look a bit into the new JeeMon design.

Let’s focus on serial interfaces first, mostly USB. There’s a “Serial” module which does all the basics. On the Mac, if I want to open device /dev/tty.usbserial-A900ad5l, then the following call with do everything:

Serial connect usb-A900ad5l 57600

The name of the device would be different on Windows and Linux (COM5, or USB1), but that’s all.

By default, this creates a new Serial object, which logs all incoming text to stdout. To send a command out, we need to keep a handle to this object:

set conn [Serial connect usb-A900ad5l 57600]
$conn send "some text"

Nice, but not very exciting…

Let’s take it one step further. The “JeeSketch” module does the automation described in the previous post, i.e. detect the running sketch, associate it with a class, instantiate an object for it, and call the methods of that object whenever text lines come in over that serial port. Here’s a complete JeeMon custom “application.tcl” program:

Screen shot 2010-02-18 at 14.44.59.png

First, all the sketch drivers are made available with one or more “register” calls. This lets the appropriate classes take over for each new serial connection – depending on what sketch is running. That’s all it takes. Servicing such serial ports now becomes an event-driven background activity.

The listeners are defined in separate files, one for each type of sketch:

Screen shot 2010-02-18 at 15.05.58.png

The ookRelay/host.tcl file looks like this, for example:

Screen shot 2010-02-18 at 14.51.58.png

This structure makes it easy to manage stuff that belongs together. Projects can be exchanged (or archived, or revision-controlled), with all the pieces needed to use them in one place. And as far as I’m concerned, it won’t be limited to JeeNodes etc, or Tcl, or a specific platform. This has to remain general enough to hook up to any hardware and use with any language (via networking, files, and direct launching of executables/scripts). My goal for JeeMon is not to limit anyone’s options, but to create a simple switchboard between whatever is needed.

(I’m still mucking around with the organization and naming of code and files, as you can see)

Nice, but still not very practical…

The problem with the above is that it doesn’t deal with devices getting plugged in or unplugged. Well, unplugging is the easy bit – the above code will automatically clean up after itself on connection loss, so that part is covered.

Wouldn’t it be nice if we could just plug in new devices and get them to automatically start doing something?

I implemented such a mechanism in a recent revision of the code, but I’m hesitant to add it again – because it was Mac OS X specific, where USB devices connected via the FTDI driver include a unique code. On Windows and Linux, you just get COM<n> and USB<n> devices, where “n” seems to be related to the order and number of device insertions.

I haven’t looked into “libusb” yet. Should I? Will it help for Windows too? Do I need to start learning about USB enumeration? What OSS-compatible tools and libs are there?

Update – on Linux, it looks like /sys/bus/usb/devices/* has all the info needed to identify USB devices. So that only leaves Windows – good: at most one lib or dll to deal with.

,

In the kitchen

In Software on Feb 18, 2010 at 00:01

Ok, so maybe it’s time to start describing some of the new stuff cooking in the kitchen lab.

I’ve been exploring software designs to use as basis for JeeMon, that new switchboard-type application for physical computing devices. The idea is to treat the system as one big message-passing machine – a bit like Message-Oriented-Middleware (MOM), which has been around for ages, but without getting sucked into any heavy-weight conventions.

Messages can be passed around, queued, stored, duplicated, filtered, transmitted, returned, ignored, sorted, etc. After all, living organisms do nothing but send chemical messages around, so why not do the same for an infra-structure focused on physical computing (sensors, displays, actuators) and home automation?

If you’ve been following this weblog for a while, you’ll have noticed that almost all the output I generate in sketches is of the form “identifier arg1 arg2 …”. So for example, packets received by RF12demo look like:

OK 61 9 2 8 79 243 87 13 0 15 0

A barometric reading from the BMP085 sensor on the pressure plug may get reported as:

BMP 233 10019

And so on. One or more upper case letters, optionally followed by digits. Then the payload (which may also include floating point and string values, not just ints).

Another convention I’ve been sticking with is to report the name of the sketch on the serial port during startup:

[ookRelay]

Or an identifier plus the current configuration settings:

[RF12DEMO] W i23 g5 @ 868 MHz

These two conventions can be used for an object-oriented design. With a bit of preparation, the name of such sketches can be automatically associated with a class, and each line treated as a method call.

Here’s the skeleton of the first level of code I use for decoding RF12demo.pde output:

Screen shot 2010-02-16 at 00.15.27.png

Another class definition, for the ookRelay.pde sketch:

Screen shot 2010-02-16 at 00.16.11.png

Or to put it differently: with JeeMon running, you can hook up a device (JeeLink, JeeNode, Arduino, etc) containing some sketch and the matching class will automatically be associated with that serial port, once the sketch has been identified. A new object is then created, and its methods will be called whenever the device sends new output (messages?) over the serial line.

Simple!

What I would like to do, is manage the sketch (C/C++) and the class (Tcl) together, perhaps as two files in a common development directory for that project. That way the interface between the two pieces of hardware essentially vanishes into the background. The point being that each class can then do all sorts of things, such as storing results in a database, sending it to another system over the network, updating web server pages, popping up a GUI window to show incoming data in real-time, etc.

This mechanism is very simple, even under the hood. This matters, because it has to work even when JeeMon is running on low-end embedded Linux hardware. But at the same time, such a MOM + OO design will allow considerable flexibility for abstract / high-end use later.

PS. If you’re familiar with Tcl, you might be surprised to see all this “oo” stuff in here. That’s because JeeMon uses Tcl 8.6 with built-in object system. Multiple inheritance, mixins, filters, dynamic class change support, delegation, prototypes / per-object methods, it’s all in there. I’m also using ensembles and there’s an interesting coroutine-based web server waiting in the wings (called wibble).

For the record: nothing ever gets added just to be buzz-word compliant. If a feature simplifies application-level concepts and leads to a cleaner implementation in JeeMon, it’ll be used, else I’ll pass. Life’s too short to jump on bandwagons.


New Dimmer Plug

In Hardware on Feb 17, 2010 at 00:01

One last plug… who knows, with a bit of luck, some of them might just work on first try?

The Dimmer Plug controls up to 16 LEDs or other lamps, and is based on the PCA9635 chip:

Screen shot 2010-02-16 at 21.29.21.png

It’s driven from the I2C bus, with a couple of jumpers to allow daisy-chaining up to 8 plugs on a single port:

Screen shot 2010-02-16 at 21.46.19.png

Here’s an extract of the datasheet, showing the different ways to hook up LEDs:

Screen shot 2010-02-16 at 02.04.23.png

Basically, you can drive one LED directly per pin, or add a transistor to drive many more in parallel or in series. The chip just does the PWM part, with various blinking options and more, all from an I2C bus.

To drive high-power displays, the outputs can be tied to darlington arrays such as the ULN2803, or to MOSFETs. There are no drivers on the plug because there are too many different usage scenarios. It all depends!

Originally, I wanted to make this plug also fit on a the back of a display such as this one:

DSC_1205.jpg

That pressure plug on the photo was added for size comparison. It’s the same size as this Dimmer Plug, which happens to allow daisy-chaining two such displays tightly next to each other when you include the headers.

But I can’t make it fit without widening the plug and mounting components on both sides. So I’ll leave it as is – it could still be mounted on the back by adding lots of little wires and 16 resistors, but it will take some more work. Or one could add a little board to sit in between, with 16 resistors and pins to stick into the 3×6 headers.

Now I’ll stop with this plug madness. Need to finalize all the layouts and traces, send the designs off, and then the waiting starts. Luckily, there’s still enough other stuff left to do around here :)

Update – there are some errors in the schematic. I’ll fix them before sending off this board.


New Input Plug

In Hardware on Feb 16, 2010 at 00:01

The plug rage continues. Sixteen inputs, analog and digital in any mix:

Screen shot 2010-02-15 at 12.58.04.png

I haven’t routed the connections yet so the layout and dimensions might still need to change.

This plug is an experiment. It does not use I2C, so it will need a dedicated port. The trick is that an on-board ATTiny45 is used to decode a pulse train of four pin selection bits from just the DIO pin.

Here’s the schematic:

Screen shot 2010-02-15 at 13.08.55.png

The 16 inputs are multiplexed into the AIO pin. By defining this as an analog input, you can have up to 16 analog pins, using the 10-bit ADC built into the ATmega. But defining AIO as a digital input, you can use it as a digital pin. Even works with pull-up, but the pull-up will only be active as long as the same input remains selected. Unselected inputs return to a high-impedance state.

Needs some more work though, including some firmware for the ATtiny.

,

New Proximity Plug

In Hardware on Feb 15, 2010 at 00:01

Yet another plug! This time it’s a proximity capacitive sensor based on the MPR084:

Screen shot 2010-02-14 at 22.01.08.png

This one has 8 inputs which can be connected to various surfaces to create capacitive touch sensors. There’s also a pin for a piezo sound unit to produce key clicks.

Slightly more elaborate schematic this time, with an extra 8-resistor array for pull-up:

Screen shot 2010-02-14 at 12.05.27.png

Yet again: connected via I2C, and supports no more than 3.6V as power source. There’s a solder jumper to choose between two I2C addresses, so you can use two of these for a total of 16 inputs on one port (or Plug Shield).

Haven’t tried this chip out yet, but it will be fun to see what it can do – once I get some prototype boards ready.

,

New Gravity Plug

In Hardware on Feb 14, 2010 at 00:01

Another plug coming up – based on the BMA020 (BMA120?) 3-axis accelerometer:

Screen shot 2010-02-12 at 23.07.28.png

Again an awfully tiny chip, with even smaller pads. This chip has a programmable ±2g/4g/8g range with 10 bit resolution. As with yesterday’s plug, this chip communicates over I2C and supports up to 3.6V.

The schematic is obviously very simple again:

Screen shot 2010-02-12 at 23.07.59.png

The Gravity Plug will also work with an Arduino, through the Plug Shield.

It’ll take at least a few weeks before I can test these new plugs and make them available.

Onwards!

,

New Lux Plug

In Hardware on Feb 13, 2010 at 00:01

There’s a new plug in the works, with a really neat TSL2561 light sensor:

Screen shot 2010-02-12 at 21.06.49.png

The chip is even smaller than what I’ve been working with until now. The nice thing about the TSL2561 is that it has six orders of magnitude dynamic range and that it’s interfaced via I2C.

The JeePlug port headers are in fact perfect for sensor chips like these, which often only tolerate up to 3.6V:

Screen shot 2010-02-12 at 21.08.28.png

The Lux Plug will also work with an Arduino, through the Plug Shield.

Stay tuned. There’s more coming…


Improved bracket

In Hardware on Feb 12, 2010 at 00:01

Encouraged by some comments on yesterday’s post, I printed the same bracket again, laying on its side:

DSC_1188.jpg

The horizontal base comes out ok (I cleaned up the hole):

DSC_1189.jpg

Interestingly, the top of the bracket isn’t filled in, which is actually better:

DSC_1190.jpg

The sides look much better as well:

DSC_1191.jpg

(the Skeinforge settings aren’t perfect yet, layers could be mashed smoother together)

But wow – what a difference this makes! Thanks for the tip!

Here’s another object, a Geneva Wheel from Thingiverse:

DSC_1192.jpg

Turning is not very smooth, had to cut off some plastic from the round shafts, but hey: it works!

PS. Don’t worry, I’m working most of my time on JeeNode stuff, hardware and software that is… ;)


JeeNode bracket experiment

In Hardware on Feb 11, 2010 at 00:01

I just had to try making a little mounting bracket for JeeNodes…

Here’s the design:

Screen shot 2010-02-08 at 11.13.51.png

The design dimensions are 25 x 12 x 10.5 mm. It was created with OpenSCAD, which takes a description of the object and generates the solid model shown above:

Screen shot 2010-02-08 at 11.14.06.png

Does it work? Can it be printed? Can it be used? Yeah, sort of…

DSC_1180.jpg

Works for JeePlugs too, since all the boards have the same 21.1 mm width:

DSC_1182.jpg

Took me three tries to get the sizes and the little ridge right (the middle one):

DSC_1179.jpg

Pretty ugly stuff, when seen up close. But hey, that’s what pioneering looks like!

The ridge which holds the board doesn’t come out very clearly – the features are not yet a match for what the JeeCake 3D printer can do (or should it be the other way around?):

DSC_1181.jpg

But despite appearances, this bracket is already surprisingly functional. Apart from the awful surface and rough shape, it’s a springy and very robust little piece of ABS plastic. It could easily be screwed down, and it would hold the board quite well.

Fascinating, it’ll be fun to see how this evolves over the next few years.

,

Wireless works, sort of…

In Software on Feb 10, 2010 at 00:01

Ah, now we’re getting somewhere!

This is my current test setup:

Screen shot 2010-02-07 at 21.23.55.png

The JeeNode on the printer side implements a packet pass-through system, receiving command packets from the JeeLink and sending back response packets. Here is the sketch which does all the work:

Screen shot 2010-02-08 at 01.02.16.png

On the Mac/PC side, some Tcl code was added to go through the RF12demo text-mode protocol to send and receive arbitrary data, using the “a” command. Still work in progress, but the basic transport encapsulation works.

The tricky part is timing … it always is with this sort of real-time control stuff. Unfortunately, the current G3 software on the CupCake isn’t quite as responsive as defined in the specs. Some responses take way over 80 milliseconds to come back from the motherboard. This is the case when scanning the SD card, as well as when stopping the extruder motor.

So what this sketch does is wait up to 500 ms for a reply to come in. Even if there isn’t one, an acknowledgement packet will be sent back. The new code on the Mac in turn waits up to 1 second for that ack to come back.

If no ack came back, then there was an error in the wireless connection (this can happen either during the request or during the ack, there is no way to tell!). Probably best thing to do would be to resend the command.

If an empty ack came back, then the response packet did not arrive within 500 ms. In this case, we could send an empty command and wait for its ack. This hasn’t been implemented yet, but it will allow dealing with even the slowest responses, simply by polling a few more times with an “empty command”.

But hey – it works, and the output is the same as before:

Screen shot 2010-02-07 at 21.26.39.png

This is probably the first wirelessly controllable CupCake in the world :)

What I should mention though is that this doesn’t yet work reliably due to those very loose timing behaviors and the fact that packet errors are not yet dealt with. Test runs fail occasionally – mostly in the SD card access code, i.e. while grabbing all the filenames with NEXT_FILENAME.

,

Connecting to a CupCake

In Software on Feb 9, 2010 at 00:01

To follow up on yesterday’s post, I wrote some test code to request the extruder nozzle temperature from the JeeCake and send the results out over wireless. Here is the full “jeeStatus.pde” sketch:

Screen shot 2010-02-07 at 16.19.43.png

And sure enough, it works:

Screen shot 2010-02-07 at 16.08.55.png

You’re seeing the nozzle cool down, after heating it up via ReplicatorG. The connected JeeNode has been given node ID 19, and it’s transmitting in group 5 of the 868 MHz band, so I can simply track these incoming packets through the JeeLink which is already collecting all sorts of data anyway.

So much for the easy part – the real software will be more work!

,

Wireless CupCake

In Hardware on Feb 8, 2010 at 00:01

JeeCake, the CupCake 3D printer here, is an interesting mix of machine, electronics, and software.

The RepRap Motherboard is the main on-board controller, based on an ATmega644. It drives 3 stepper motors, communicates with the Extruder board, and talks to a desktop computer via its FTDI interface and a USB cable.

On the PC side (which can be Windows, Mac, or Linux), there is a Processing-/Arduino-like package called ReplicatorG to control the machine. It takes G-code, which is not related to Google in any way, but rather an ancient CNC control language from the 60’s.

ReplicatorG then converts this to a binary RepRap 3G protocol, as documented here and then uses that to drive the CupCake. Everything from moving the axes, adjusting the nozzle temperature, controlling the extruder motor, to writing settings in the machine’s EEPROM memory.

The neat part is that the v1.2 motherboard has an SD card adapter on board, and that the CupCake can run unattended by getting its detailed build instructions from files stored on an SD card.

Except for one little detail: there are no controls or displays on the CupCake, other than a few LEDs. This is not enough to select which file to print, adjust the zero position, or start a build. It looks like a new “Generation 4″ design is on the way, including an interface board with an LCD screen and some push buttons.

Right now, you have to connect the machine via USB, start everything up via ReplicatorG, and then you can yank the USB plug and it’ll happily continue printing what it started, right to the end (well, except that on Mac OS X 10.6, yanking USB cables can lead to kernel panics – looks like a serious bug in the FTDI USB driver).

Anyway, I don’t really care for controls, or even displays on the CupCake. All I want is some way to control a unit sitting on the other side of the room, or in a nearby room. It can be fairly noisy due to some sort of occasional wood panel resonance, and having it printing right next to me is not really my idea of fun.

Which is where JeeNodes come in: wouldn’t it be nice to be able to control the CupCake via wireless? The protocol is already perfectly suited for it, since it uses packets of max 32 bytes – well within the 66-byte limit of the RF12 driver. And if the object being printed is already on the SD card, then only a few packets need to be exchanged to get going. During printing, some status info could be sent back – again very low rate stuff, easily within the JeeNode’s wireless constraints.

Here’s the idea:

DSC_1177.jpg

Hook up a JeeNode to take the place of the USB cable, and let it behave as a RecpicatorG control program.

The interface needs to swap RX and TX for this, and because the CupCake’s signal levels are at 5V, two 1 kΩ resistors need to be inserted to prevent excessive current. One more detail is that the power pin on the FTDI connector is not connected, since the motherboard is powered off its own PC supply. So a separate wire needs to be added to power the JeeNode off the ISP connector right under the FTDI connector:

DSC_1178.jpg

The one wire not shown here is the ground wire, running from top right to bottom left on the other side of this little custom interface board.

It turns out that the motherboard is actually powered from the 5V standby supply pin, so it’s always powered, even when the power supply is in standby mode.

I’ve started writing some code on the PC/Mac side to control the CupCake without using ReplicatorG. Here’s some sample code in Tcl which works when connected directly through USB:

Screen shot 2010-02-07 at 15.05.08.png

And here’s the corresponding output:

Screen shot 2010-02-07 at 15.04.35.png

As you can see, it can access all sorts of status info, read the file names on the SD card, and control the machine. This is part of a larger project, the beginnings of which are now in the subversion code repository.

It takes a lot of work to make hookups like these work, because there are so many different bits and pieces (literally) involved. The next step is to see if the JeeNode can indeed communicate with – and control – the JeeCake, and then code needs to be written to replace the current direct-USB connection by a packet-based wireless hookup through a JeeLink + JeeNode.

Oh, and then I need to create some sort of little on-screen control panel to adjust the nozzle temperature, jog the Z axis up and down, pick a file to print, and start the print job! Not to mention making it robust and secure…

,

Plug shield on Arduino Mega

In AVR, Hardware on Feb 7, 2010 at 00:01

The Plug Shield can also be used on an Arduino Mega:

DSC_1175.jpg

Note the two extra wire jumpers, since the I2C interface is on pins 20 and 21 on the Mega board.

The above has an RTC plug and an LCD plug hooked up, so let’s to set up a simple clock with this – and use it to demonstrate the brand-new RTClib along the way:

Screen shot 2010-02-05 at 17.22.01.png

I’ve omitted the details of the Wire coding, but you can get the full sketch here.


Custom object

In Hardware on Feb 6, 2010 at 00:01

This is not what you might think it is:

DSC_1169.jpg

I’ve been placing my headphone next to me on my desk for ages. There is of course never enough room on top, but plenty of spare room underneath:

DSC_1170.jpg

So here’s my home-made headphone bracket:

DSC_1172.jpg

Here’s the design, made with Sketchup:

Screen shot 2010-02-03 at 14.22.47.png

I just picked a “2″ in a suitable font, re-sized it, “drilled” some holes in the base, and then “printed” it.

Piece of JeeCake!

PS. Given that CNC and 3D printing is not really the main focus of this weblog, I’ve set up a page on the wiki.

Update – design files added to Thingiverse.


New date / time / RTC library

In AVR, Software on Feb 5, 2010 at 00:01

Not so long ago, I had the opportunity to work a bit on something which has bugged me for a long time – the lack of date and time handling in connection with RTC chips. There are a few libraries out there, but I think I could do better – i.e. make it simpler, smaller, yet sufficiently powerful for real day-to-day use.

Seeing where this was going on the Arduino developer mailing list (and disagreeing with just about everything that happens over there), I decided to put my money time where my mouth is, and build my own library.

Here’s the header file of the new RTClib Arduino-compatible library:

Screen shot 2010-02-04 at 13.52.13.png

This lets you do date / time calculations, and it provides two different ways to implement a clock: via a hardware chip or using the built-in millis() timer.

RTClib has been checked into subversion, see the CODE page for details on how to get it.

It includes four example sketches:

  • datecalc illustrates how to do calculate with dates and times
  • ds1307 interfaces with a DS1307 RTC chip, connected via the Wire library
  • plugrtc interfaces with the RTC Plug, connected via the Ports library
  • softrtc demonstrates how to do the same with just software

One fun trick I added, inspired by a comment from Limor Fried, is to allow initializing a DateTime object from the __DATE__ and __TIME__ strings generated by the C compiler. That means you can run that “softrtc” sketch without hardware support, and it’ll automatically have its clock set to the compilation date of the sketch, i.e. fairly close to correct. Not good enough for general use, but great during quick debug cycles when you’re re-compiling your sketch all the time anyway.

Note that to use RTClib, you need to include the “Wire.h” library – even if you don’t use it!

The inability to properly deal with libraries, particularly in a resource-constrained embedded processor context, is one of the aspects of the current Arduino direction which irritates me – see an older post for more details.

,

Fascinating concurrency

In AVR, Software on Feb 4, 2010 at 00:01

There is a new language for the Arduino / JeeNode / ATmega328, called Occam-π.

I found out about it yesterday, at http://concurrency.cc/ – it’s high level, and it supports parallel programming. The current development environment release is for Mac OS X, with Windows and Linux coming soon.

Here is a complete program with 4 blinking LED’s, one on each DIO pin of the JeeNode ports:

Screen shot 2010-02-03 at 01.13.19.png

That’s it. Compiles to roughly 2 Kb. Each extra blink adds just 20 bytes, btw.

And yes, it really makes four LEDs blink at an independent rate:

DSC_1167.jpg

There is slightly more to it than that, but this is mind-blowing stuff. The “parallelism” is simulated, of course. Looks like the ATmega can do around 6000 context switches per second (i.e. parallel task switches).

There is a roughly 20 Kb interpreter part that needs to be uploaded once (which is why this requires at least an ATmega328). After that, the IDE will upload just the bytecode for your program, i.e. 2 Kb in the above case.

B R I L L I A N T .

Imagine hooking up the RF12 driver to this – there’s plenty of room for the extra 3 Kb or so. And for doing all sorts of things… in parallel! My earlier complaint post about how awful it is to do several things at once on an ATmega board might just have been wiped off the table.

Looks like I’ve got some very serious learning ahead of me to try and get to grips with all this.


JeeNode USB v2 error!

In Hardware on Feb 3, 2010 at 00:01

Yikes… there is a major bug in the current JeeNode USB v2 board!

The on-board MCP1703 3.3V has been connected with Vin and Vout reversed!

The effect is that the +3V pins on all the port headers will have approx. 4.7V instead of the intended 3.3V. This in turn means that all plugs connected to these headers will have a higher voltage than should be.

The on-board logic is not powered by this regulator but gets its power from the 3.3V pin on the FTDI chip, so the JeeNode USB v2 itself works just fine (my weak excuse for why I didn’t catch this problem earlier…).

Here is the schematic – with the problem staring me in the face, now that I know about it:

Screen shot 2010-02-02 at 13.51.39.png

Here are the printed circuit board traces:

Screen shot 2010-02-02 at 14.39.16.png

And here is the board with the voltage regulator connected the wrong way around, as it has been shipped until yesterday, i.e. Feb 1st 2010:

DSC_0737.jpg

If you have a faulty unit, there are a number of ways to fix this:

  1. Send it back and I’ll fix the voltage regulator for you.
  2. Remove the voltage regulator and add your own from a PWR pin to a +3V pin.
  3. Unsolder and re-solder the VR flipped, as described below.

To reiterate: this bug only affects plugs connected to the JeeNode USB, not the JeeNode USB itself. If you don’t use the ports, or don’t mind having a higher voltage on the +3V pins, then you could just ignore the issue.

Here’s how I’ve fixed this on existing JeeNode USB’s, including ones still in stock. First, I used solder wick to remove the regulator (careful not to lift the pads):

JU2 no VR.png

The next step is to mount the VR back on its side. The pins which need to be exchanged are:

MCP flip.png

One way to do this, is to turn the VR on its side and solder pins 1 and 2 back on pads 1 and 3:

DSC_1164.jpg

As last step, I added a thin wire from what used to be pad 2, to the pin now sticking out on top (was pad 3):

DSC_1162.jpg

As I said, I’ll be happy to do all this fixing for you if you send me your JeeNode USB v2.

It ain’t pretty (real bugs never are) – but it’ll at least let you continue to use the JeeNode USB v2 as intended.

All JeeNode USB v2’s shipped from now on will have this fix, until an improved “v3″ is ready. As always, “new copper” (i.e. a new PCB) will take a few weeks – it’s at the top of my list to fix and resolve this for good.

With many thanks to castello for drawing my attention to this bug on the forum.

,

Room Board assembly

In Hardware on Feb 2, 2010 at 00:01

As announced recently, I’ve now added a full kit for the Room Board in the shop, containing:

  • The room board PCB.
  • SHT11 temperature + humidity sensor.
  • An LDR light sensor.
  • The ELV PIR motion sensor.
  • Two 6-pin male headers.

Here is the kit (oops, forgot the two 6-pin headers):

DSC_0980.jpg

From top left to bottom right: PIR lens, PIR sensor, 3-pin header, SHT11, LDR, Room Board PCB, pre-assembled PIR sensor board.

The hardest part to solder is the SHT11 sensor. It has 10 gold pads on the side, of which only 8 are connected. Make sure you place the sensor in the right orientation. With the text on the pcb readable, the round part is pointing down:

DSC_0981.jpg

Next, solder in the LDR light sensor. Be sure to place it in the LDR1 position:

DSC_0982.jpg

The wires of the LDR may be a bit more difficult to solder than ordinary components, so be sure to use enough flux:

DSC_0983.jpg

Next, we need to prepare the ELV PIR sensor board – the board itse;f already has all the SMD’s soldered on, just the PIR sensor itself.

Careful – do NOT touch the surface of the PIR sensor, as any traces of oil will reduce its sensitivity:

DSC_0984.jpg

The sensor has to be pushed through the bottom side of the lens holder, into the board – from the non-component side:

DSC_0985.jpg

Solder the three wires and snip them off. Don’t use too much solder or you’ll risk shorting out the pads with the nearby components:

DSC_0986.jpg

Then turn the board around and click the fresnel lens onto the base (it has three different-sized notches):

DSC_0987.jpg

Next, we need to connect the boards together:

DSC_0988.jpg

The pins go through the room board and should be snipped off once soldered:

DSC_0990.jpg

Almost there – the last step is to solder the two 6-pin male headers onto the room board. To hold them in the right position, I just push them into a JeeNode:

DSC_0991.jpg

… then place the room board on top and solder both headers, all 12 pins. Leave the 8 holes on the side of the PCB free (these are for another type of PIR sensor):

DSC_0992.jpg

That’s it – you’re done!

For more info on uploading the “rooms.pde” sketch firmware to the JeeNode, go back to the docs page.


RTC battery fix

In Hardware on Feb 1, 2010 at 00:01

I’ve been getting some email about the RTC Plugs not retaining their clock. It turns out that there is a problem:

DSC_1154.jpg

The center contact of the battery “holder” is the PCB itself, and being gold-plated, I thought it’d be best to keep it as is.

The problem is that the solder mask layer can be slightly thicker than the gold-plated pad, causing the battery to not make contact – despite the pressure of the metal clip.

This can be fixed by adding a small dab of solder:

DSC_1155.jpg

This will cause the coin cell to press more against both contacts, so a good connection is virtually guaranteed.

Needless to say, I’ve updated all RTC plugs here and will ship this improved version from now on.

,

Room node – last

In Hardware on Jan 31, 2010 at 00:01

Here’s a last post about mounting a room node in a ceiling corner.

The enclosure and support struts were cut out of an A4 sheet of foam board, according to the template PDF on the wiki:

DSC_0993.jpg

Unfortunately, the JeeNode doesn’t quite fit with the FTDI pins sticking out, so I had to cut away a bit of the support material. As you can see, everything is glued together with hot glue, even the room board. The battery pack is stuck on the top part with double-side tape.

The motion sensor sticks out in the middle, at the cross-point of two diagonals on that slanted surface, to be precise. The holes for the LDR and SHT11 have not yet been made:

DSC_0994.jpg

Here’s how everything sits in the completed unit:

DSC_0995.jpg

There’s still some leeway to move the JeeNode up a notch, but I’ll leave it at that.

Enough of all this mechanical tinkering!

,

Room node mount

In Hardware on Jan 30, 2010 at 00:01

It’s time to finish that room node ceiling mount, at least the prototype.

At first, I thought I’d use the new 3D printer for some sort of nifty mounting bracket:

Screen shot 2010-01-29 at 20.48.04.png

… but it turns out that there’ a much simpler solution:

DSC_0977.jpg

This pushes the top of the mount against the ceiling, right under the battery pack, which is by far the heaviest part anyway:

DSC_0978.jpg

I don’t like the look of the paper sheet wrapped around the foamboard, but whatever – it works!

There’s a new wiki page with the description and a PDF with the outline of the pieces required to make a ceiling mount. Haven’t quite accounted for the thickness of the foam, since it depends a bit on which way it’s cut and folded, but the PDF should be usable as starting point.


One more 3D post

In Hardware on Jan 29, 2010 at 00:01

Ok, one last post on the CupCake CNC trials before I get back to JeeNode stuff.

The raft is now very solidly stuck to the base:

DSC_0968.jpg

This thing really sticks well, with plastic pushed down into the criss-cross grooves of the acrylic base. The trick is to place a plain paper sheet between the extruder head and the base, and to lower the Z axis until the paper slides away with just a tad of friction.

Unfortunately, the next problem comes up – as seen in this first Geneva Wheel trial:

DSC_0967.jpg

It turns out that the next layer of the raft doesn’t stick well to the first one. I solved this by raising the default 230° setting to 240° for the raft.

Another serious problem was that there is a lot of plastic on the right end of the raft as the print nozzle paused (!) on each turn. And the second layer of the raft doesn’t extend all the way to the end. This was solved by adding three noise-suppressing 0.1 µF caps to the extruder motor and disconnecting all the end-stop optocouplers.

The object itself seems to come out fairly well, though there were some blobs of plastic around the nozzle during printing which clearly interfered with the X-Y motion at times.

Here are over a dozen more parts, printed using black ABS for a change:

DSC_0975.jpg

These all came out ok, but note that even these small parts take 10..15 minutes to print – each.

The biggest remaining problem is “warping” – it’s causing over half my trials to fail:

DSC_0976.jpg

This part failed to build (should have been 20 mm high), it slowly came off the base and started interfering more and more with the extruder nozzle. At some point the nozzle got stuck in fact. Note how the thickness of the part is completely uneven. This is caused by the ABS cooling down and shrinking a bit – slowly pulling itself off the raft (with considerable force). A heated baseplate seems to be the hack to solve this – but that will require a lot more tinkering.

Oh well. We’ll get there… eventually :)

Update – upgraded to ReplicatorG 0012 with Extruder firmware v1.8 and switched to Zach Hoeken’s thermistor settings (raft setting reverted to original 230°). The last change was to shine a halogen light on the platform and object, to heat it up a bit – this seems to get rid of just about all the warping, yeay!


More 3D trials

In Hardware on Jan 28, 2010 at 00:01

It’s not as simple as it seems…

Well, the CupCake worked straight out of the box, which is of course fantastic. But printing does seem to have two tricky bits, called “extrusion” and “rafts”.

Here’s the printer making its third part:

DSC_0963.jpg

What’s about to happen, is that the “raft” (that woven mat laid out on the bottom to hold the rest of the construction) is about to come off the base. In the above picture you can see two other problems: the raft itself coming apart, and a blob of plastic (front right) when the printer decided to pause for a few seconds, while the extruder kept pushing out plastic…

The problem seems to be that when the plastic cools down fully it becomes quite springy and rigid. Which is great for the final object, but causes trouble in keeping it stuck to the acrylic base. I suspect that the heated platform which people are experimenting with is to overcome this problem, by keeping the base slightly sticky for the duration of the build.

Or maybe a few wires extending further out from the raft would help, by “tacking down” the whole thing.

The other purpose of the raft is to even out the surface. In my case there is a slight difference in height across the base. I should probably try to get rid of that first.

Anyway, here are the objects made so far:

DSC_0964.jpg

The meandering plastic retains its shape – that’s probably part of the magic behind all this.

PS. The stepper motors are noisy. I don’t think the software I’m using is micro-stepping right now.


CupCake #361 is alive!

In Hardware on Jan 27, 2010 at 00:01

Just got the missing part for the CupCake CNC 3D-printer in from MakerBot – partly their mistake, partly my mistake, long story… anyway – in the end they were extremely helpful in coming up with a good solution.

Which I had to try right away, of course ;)

After a bit of head scratching w.r.t. firmware versions and uploading, everything came together at last.

Here’s the first object getting printed at Jee Labs – whee this is fun:

DSC_0959.jpg

After 8 minutes, it’s done and steps aside:

DSC_0960.jpg

Mystery object…

DSC_0961.jpg

After removing the “raft” and cutting off some loose ends:

DSC_0962.jpg

So what is it? Oh, nothing spectacular really – it’s a Z-endstop for the machine itself. I figured that after us people doing so much to get it going, this machine might as well start working on improving itself right away…

I’m astonished by how well this first part came out. Very clean, very plastic’y, much sturdier than I expected.

All that’s left is to give this new beast a name – Jee3D? JeeCup? JeeCake? CuppaJeeno?


Ceiling mount – sliding part

In Hardware on Jan 26, 2010 at 00:01

Ok, here’s the sliding part of the room node mount:

DSC_0953.jpg

Totally hacky for now, the reinforcement bits are all glued in and cut to shape.

What totally surprised me, is that the JeeNode itself barely fits, even though there seems to be so much empty space in there. The tricky bit is that it can’t stick out on either side – in the end I could only make it fit by turning it a little, and moving the battery pack out of the way to make room for that orientation:

DSC_0955.jpg

This thing is larger than I expected it to be – the big triangle against the ceiling is 14 cm on both sides and ≈ 20 cm on the long diagonal side.

Last step will to to figure out a way to mount this concoction. Oh, and then redo it all from scratch to create a decent printable DIY template.


Ceiling mount – first step

In Hardware on Jan 25, 2010 at 00:01

As promised, the foam board construction – well, a first step anyway:

DSC_0950.jpg

The “folds” are a bit ugly this way, I’ll hide them on the inside using V-cuts in the next version.

Room node and battery pack included for size comparison. You’d think they fit easily inside, but the battery pack is surprisingly tricky, given that the room node needs to stick through the slanted side facing outwards.

The “walls” are also a foam mockup I’m using to avoid having to use an actual ceiling corner :)

DSC_0951.jpg

Here’s how I intend to to place everything inside:

DSC_0952.jpg

The battery pack will be attached to the top, i.e. flat against the ceiling. The room node will be mounted such that the PIR sensor sits in the middle, sticking out.

Onwards!


Room node ceiling mount

In Hardware on Jan 24, 2010 at 00:01

Been pondering a lot lately on how to mount all those room nodes around the house…

Without an enclosure the room nodes look very “hacky”. And what’s even worse: there is no way to mount them! The problem is that you want to point the PIR sensors at least approximately into the center of the rooms they are overseeing. The least conspicuous place for them is probably in the corner, near the ceiling.

Our house has white walls, so naturally I’d like to have something white up there, not some black or metal box!

Here’s an idea:

Screen shot 2010-01-22 at 14.12.54.png

To be made of – wait for it – foam board!

It’s probably a bit hard to visualize, but the idea is to attach a triangular “corner plate” to the wall (using glue / nails / screws / telekinesis / whatever). Then you slide this assembly with JeeNode and battery pack onto it.

Here’s a paper mockup of the sliding part:

DSC_0946.jpg

With one or two “struts” to reinforce it, this ought to be fairly sturdy and easy to slide in and out (will probably have to do that often in the beginning!). Perhaps a few needle pins will be enough to keep it fixed in place.

Laid flat, the sliding part looks like this:

DSC_0947.jpg

Needs some holes in the middle section, for the PIR, SHT11, and LDR sensors.

The geometry of this thing is quite simple, once you wrap your mind around what it really is – an equilateral triangle placed against the top corner with horizontal cross sections on top and cutting off the bottom part:

Screen shot 2010-01-23 at 13.46.52.png

The trick will be to get the sizes right. The paper mockup shown here is about 12 cm on each side, enough for a JeeNode, but too small to also hold a 3x AA battery pack behind it. It also shouldn’t be too hard to make some extra templates pointing off-center – so that you’d essentially aim the PIR by picking the most suitable enclosure variant.

I’m going to try a version using foam board next. Never a dull moment :)

P.S. If you qualify for a 10% discount in the shop, please note that there are 7 days left.

,

Lithium node

In Hardware on Jan 23, 2010 at 00:01

The 3.6 long-life / low-current OmniCel lithium batteries just came in.

Say hello to the latest room node at Jee Labs:

DSC_0944.jpg

Colorful, eh? ;)

The battery is connected directly to +3V and GND, i.e. after the voltage regulator, since its operating voltage is an excellent match for the ATmega and RFM12B.

No motion sensor was needed for where this node is going (and I didn’t have any left anyway), so that reduces the current draw by some 40 µA. Average should be somewhere around 200 µA – as mentioned before, this can no doubt be optimized further.

Since there are no motion triggers, this node will only send one or two packets per minute – the battery should last well into 2011, according to the specs.

Unless the receiver is turned off and this thing starts re-transmitting all the time. I really need to fix that flaw…

For a fun enclosure, check out the Airwick post by Szymon Kobalczyk on the Jee Labs discussion forum.

,

Temperature plots

In Software on Jan 22, 2010 at 00:01

With over a dozen nodes now active, JeeMon is finally starting to collect a bit more useful data to work with:

Screen shot 2010-01-22 at 12.03.23.png

Looks like some temperatures have a granularity of 1°C – might be a few cheap DS1820 units I had…

The Mason-like templating code in “try-temps.html” to generate this plot is still far from obvious:

Screen shot 2010-01-20 at 01.52.42.png

Once I get to spend some real time on all this, it’ll look totally different. JeeMon as a whole is going to change completely in fact – until then, you can find some technical background information in an older post.

Let’s just say that it gets the job done right now…


Single AA room node

In AVR, Hardware on Jan 21, 2010 at 00:01

The range of things to try is just endless…

Here is a (v2) JeeNode with a (prototype) room board and a 1-cell battery:

DSC_0942.jpg

Note that this prototype board is turned 180° w.r.t. the official Room Board, to match the Rooms sketch.

This node uses a 1.5 => 5V converter from SparkFun, which I had lying around. It’s far from optimal, with one third of the energy eaten up by the 3.3V voltage regulator, but it’ll have to do for now. Let’s see how long it lasts…

The whole thing is mounted on a little piece of foam board with dual-sided adhesive tape. Just to try it out.

To give you an idea of what’s flying around over the 868 MHz airwaves around here:

Screen shot 2010-01-20 at 00.18.35.png

That’s some 10 room nodes, the OOK relay with a bunch of sensors, and the energy/gas metering packets. Note how the Linux system time is exactly in sync with the DCF77 atomic clock receiver (using UCT i.s.o. CET). Note also that the OOK relay data is currently reported twice: once through a direct USB connection, once over the air.

Lots of fun! And that’s just the beginning, as far as I’m concerned :)

On a related note: I’m investigating whether it would be feasible to bring out a complete kit for the Room Board – with motion, light, temperature, and humidity sensors all included. If I can find the right distributors and order in sufficient quantity, it might be possible to get such a kit in the shop for €39, incl. VAT. Let me just say that if it can be done, you’ll be the first to know!

,

Sleep mode fix

In AVR, Software on Jan 20, 2010 at 00:01

The Rooms sketch (latest code here) had problems with the sleep mode added about a month ago:

Screen shot 2010-01-17 at 14.05.55.png

The first value is the current consumption in µA.

For some reason or other, the node would stop working and enter a permanent-on mode, drawing over 7 mA and draining the battery in a matter of days. Not good.

It seems to be related to the way the power down mode was implemented. To get absolutely rock bottom power draw, I was using the RF12’s watchdog timer. The ATmega watchdog time draws slightly more current and isn’t quite as configurable.

I’ve now reverted to using the ATmega watchdog anyway. Here is the modified logic:

Screen shot 2010-01-18 at 12.09.39.png

The watchdog is set to interrupt every 16 milliseconds, constantly. When the node is powered down, this will wake it up again. What the new loseSomeTime() code does, is simply to power down a couple of times, until the target delay time has been reached. There is some inaccuracy in these timings since the watchdog timer is free-running, but this should not matter too much when waiting for a second or so.

The new code has been running fine for over a day on six nodes. Here’s a sample from my power tracker:

Screen shot 2010-01-19 at 16.56.22.png

That’s 110 µA right now, 265 µA in the last minute, and 230 µA in the last hour.

At around 250 µA average, the power consumption is a bit higher than before, but my main concern is first to get the nodes running reliably. Even @ 250 µA, the AA batteries should last several months.

The average power down current draw is 110 µA/sec, 40 µA of which is due to the PIR sensor. The total current draw while transmitting is around 29 mA – during reception it’s about half, i.e. 14 mA. Still, due to the very brief on-times, this current consumption averages to only about 400 µA during 1 second for a normal send + ack sequence. Under optimal RF conditions, the long-term average consumption of a node will be under 150 µA. I’m confident that further optimizations could reduce this to well under 100 µA.

But there is one known flaw, which can be observed to happen once in a while: the nodes always wait for an ack in full-power mode, i.e. with the receiver on. This will normally be within milliseconds, but if the connection is flakey or if the central node is unresponsive, then nodes can spend a great deal of time in full-power mode. This needs to be fixed one day.

I’ve started re-flashing all the room nodes and replacing their dead batteries, here at Jee Labs – let’s see how it goes this time around.


Interesting battery option

In Hardware on Jan 19, 2010 at 00:01

Found an interesting (one-time, i.e. disposable) battery cell the other day at Farnell:

Screen shot 2010-01-17 at 21.03.13.png

These have the size of an AA cell, but they deliver 3.6V, so they can effectively replace 3 AA cells at once. The voltage is just right – this battery could be used with or without the JeeNode’s 3.3V voltage regulator.

The discharge graph is as follows:

Screen shot 2010-01-19 at 16.47.29.png

I’ve ordered a few with wires attached to them, so they could be tied directly to the PWR & GND pins of the FTDI connector. Will report back once some “field tests” have been done.

Update – better discharge graph, see comments below.


More C++ template trials

In AVR, Software on Jan 18, 2010 at 00:01

Here are some more results in trying to use templates for embedded software on JeeNodes. This time, it’s about using Jee Plugs with bit-banged I2C on one of the 4 ports and with built-in TWI hardware on pins A4/A5.

Let’s start with an extract of the Latest JeeLib.h header:

Screen shot 2010-01-17 at 11.50.27.png

I’ve omitted all the actual code, for brevity in this example. The full code is here. The previous version of JeeLib has been moved to the avr-jeelib-first branch in subversion.

The Port<N> class is essentially the same as in an earlier post. It generates efficient code for raw pin access, given fixed pin assignments known at compile time.

The above code adds a templatized PortI2C<N,W> class, where N is again the port number (1..4) and W is a constant which determines the bus speed. As with Port<N> class, this leads to a class definition which requires no state and which can therefore consist entirely of inlined static members.

A HardwareI2C<W> is also defined, with the same API as PortI2C<N,W>, but using the hardware TWI logic in the ATmega. The point is that in use, PortI2C<N,W> and HardwareI2C<W> objects are interchangeable.

You can see how all this templating stuff complicates the naming of even simple classes such as these.

The final class implemented above is DeviceI2C<P,A> – it does the same as DeviceI2C in the original Ports library, but again using templates to “bring in” the underlying port classes and the device I2C address.

Here is an example sketch built with all this new code:

Screen shot 2010-01-17 at 12.48.05.png

It supports two bit-banged I2C devices on ports 1 and 2, respectively, as well as a third I2C device driven by the built-in TWI hardware.

This compiles to 980 bytes (using Arduino IDE 0017).

The good news is that this generates pretty efficient code. It’s not 100% inlined – but quite a bit is, especially at the lower levels, so the result looks like a pretty good implementation of a high-level I2C driver which can be used for both bit-banged and hardware-supported I2C, all by changing one declaration at the top of the sketch.

But there are quite a few inconveniences with this approach…

First of all, note that the declarations at the top are fairly obscure. I did my best to simplify, but all this template mumbo-jumbo means you have to understand pretty well how to declare a port, and how to declare an I2C device object for that port. The “typeof” keyword in there is a GCC extension, without which these declarations would have looked even more complex.

The major trade-off is that the above example essentially generates separate code for each of these three I2C devices. There is virtually no code sharing. This can lead to code bloat, despite the fact that each version generates pretty good code. In practice this might not be so important – it is not likely after all that you’ll need all three types of I2C buses in the same sketch. Just keep in mind that you’re trading off efficient hard-wired coding against the need to generate such code for each type of I2C access you might need.

So would this be a good foundation to build on? I don’t know yet…

C++ templates do seem to get a lot more of the logic “into” the compiler. Instead of passing say an I2C device address as constant to an object, we generate a highly customized class which is special-cased to implement just one device at just one address. With the result that the compiler can perform quite impressive optimizations. In the above example, there are lots of cbi and sbi instructions in the generated code, just as if you were to dig in and hand-craft an optimized implementation for exactly what you need. All from a small general-purpose library!

But it comes at a price. There is no (non-template) “DeviceI2C” class anymore. Writing a class on top to implement access to the UART Plug for example, means this class has to use templates as well. It’s a bit like “const” – once you start on the path of templates, they will start to permeate all your your code. Yikes!

The other “cost” is that all templates have to end up in header files. The size and complexity of the “JeeLib.h” header file is going to increase immensely. Not so great if you want to get to grips with it and just use the darn stuff. On the plus side, I was pleasantly surprised that error messages are fairly good now.

These drawbacks may be acceptable if all the template code can indeed remain inside the library – I sure wouldn’t want to impose the need for all library users to learn the intricacies of templates. So maybe it’s all doable – but this approach has major implications.

Is it all worth it? Hm, big decision. I do not like big decisions, not at this stage…


Fun board

In Hardware on Jan 17, 2010 at 00:01

Here’s a fun project:

DSC_0938.jpg

This is an A4-sized foam board with four “indicators”. Each of the colored vane pointers can rotate 90°. The idea is to place a sheet of A4 paper underneath them with some sort of scale, so that you can see 4 status values at a glance. Energy consumption, number of pending emails, whatever.

Here’s the back side:

DSC_0939.jpg

That’s an old v2 JeeNode with an ATmega168 and 4 cheap micro-servos strapped to it. The extra capacitors reduce the voltage ripple while these servos move, since they seem to generate quite a lot of noise.

The whole thing is held together by zip-lock straps:

DSC_0940.jpg

The servo axes pass through holes in the bottom, which is in the fact the front plate.

Here is a test sketch to move each of the servos via commands given to the serial port:

Screen shot 2010-01-16 at 10.55.46.png

The basic range for each servo is 45 to 135 degrees, and the h/j/k/l keys control each individual servo, so this lets me position each one with commands such as “45h”, “90j”, “135k”, “75l” etc. There are also commands to move all servos, or to return each one of them to the middle position. These positions are only approximate, each servo will need slightly different values for their end ranges.

It turns out that these servos can’t all be moved at the same time reliably. I’m guessing that they generate too much noise or pull down the 3.3V regulator too far. So all actions involving multiple servos need to be staggered. Another problem is that these low-cost servos sometimes get “stuck” and keep jittering around the target position – this can be prevented by detaching them, since each servo will stay at its last position.

Next step will be to drive these indicators from wireless. I’m going to postpone that for now, because this status display really needs a “switchboard” type application running on the central PC to manage real-time updates.

This thing can’t run off batteries, unfortunately, because the servos draw too much current. I’d be interested to make a display in the future which does run wirelessly – maybe something with solenoids or tiny DC motors.


Sheeva – powering up

In Uncategorized on Jan 16, 2010 at 00:01

Yesterday, I mentioned the SheevaPlug. Today, I tried it out:

  • Plug in Ethernet, plug in power, done.
  • The box registers via DHCP.
  • Figure out its IP address, and follow the New Plugger instructions.
  • That’s it. Ubuntu 9.04, running on a pretty amazing little box. Effortless!

Here are some system details, once you log into it via SSH:

Screen shot 2010-01-15 at 01.23.35.png

Upgrading to the latest software packages went flawlessly, using the normal Debian/Ubuntu apt-get update/dist-upgrade/autoremove commands. Once done, there is still plenty of room left on the internal flash ROM:

Screen shot 2010-01-15 at 01.50.09.png

According to this post, power consumption is really low:

Screen shot 2010-01-15 at 01.38.07.png

I’m seeing about 3.5 W, doing nothing and hooked up to 100 Mbit ethernet. Close enough.

Mighty impressive for a 1.2 GHz processor with 512 Mb RAM. Here’s a brief review of it.

This could make a pretty nifty “home monitoring and control center”. I’m not going to limit my options to this, the plan is to support a wide range of platforms – but still, this ShevaPlug sure is a comfy unit, almost like a desktop!

,

One of the JeeHubs?

In Hardware on Jan 15, 2010 at 00:01

Just got this thing in:

DSC_0936.jpg

It’s a SheevaPlug with 512 Mb RAM, 512 Mb flash, Gb ethernet, and a USB host port.

As you can see, I plugged a JeeLink into it, along with a 2 Gb memory card.

This is just one of several configurations I am going to look into. One of the design criteria for the server side is that it needs to be portable to a fairly wide range of hardware, from desktop PC’s, to some really small embedded Linux boards. This SheevaPlug sits somewhere in between, with a lot more oomph than needed just for JeeMon, yet an idle consumptions of only a few watts.

Haven’t turned it on, let alone adapted the JeeMon software for it, but that’s only a matter of time.

Time? Man, that’s such a scarce resource around here!


New library experiments

In AVR, Software on Jan 14, 2010 at 00:01

Encouraged by the previous post, I started writing a bit more code based on C++ features such as templates and namespaces, in the form of an Arduino library (compatible with it, not depending on it):

Screen shot 2010-01-09 at 02.08.27.png

Things to point out:

Less #defines (none so far), I turned bitRead() and bitWrite() into template functions (getBit and setBit), so that they can be used with 1..4 byte ints, just as the macros.

The Port class is inside a new “Jee” namespace, so there is no conflict with the existing Ports library.

Atomic access hasn’t been solved. Some of this won’t work if I/O pins get changed by interrupt code. I’m hoping to solve that at a higher level.

There are compatibility definitions for pinMode(), digitalRead(), and digitalWrite() using the normal Arduino pin numbering conventions (only Duemilanove so far). These are in their own namespace, but can also be used without any qualifiers by adding “using namespace Jee::Arduino;” to the sketch.

Totally incomplete stuff, untested, and not 100% compatible with Arduino in fact.

The other thing I want to explore, is to treat the choice of what a pin does as a pin initialization mode. Hence the enum with Mode_IN .. Mode_PWM definitions. The underlying code is PortBase::modeSetter(), but it hasn’t been written yet. It’s all steamy hot vapor for now.

Update – I’ve placed the code in subversion, but the API is going to be in flux for a long time.

Update #2 – Atomic access is actually better than I thought. With constant pin numbers, setBit() will use the cbi/sbi instructions, which are atomic.

,

Meet Roomie

In Hardware on Jan 13, 2010 at 00:01

It may not be the prettiest gadget in the home …

DSC_0931.jpg

… but I find it mighty convenient. I think I’m going to call it a “Roomie”.

The battery pack shown here is an older green one, the ones currently in the shop are black.

It’s nothing but a JeeNode with a battery pack and a Room Board on top. This example is not optimal, it uses a power hungry ePIR. But you get the idea.

The battery pack is tied to the dedicated connection for it – going through the strain relief holes and around the other side to the solder pads:

DSC_0932.jpg

And the whole thing is glued together with hot glue:

DSC_0935.jpg

Nice thing is that this doesn’t restrict the uses of a JeeNode. When switching the battery off, you can hook it up to a USB interface via the FTDI header.

In fact this will even work upside down on a foam board, as used with POFs:

DSC_0934.jpg

Just think of it as a JeeNode with a power pack sitting on its back :)


C++ templates

In AVR, Software on Jan 12, 2010 at 00:01

A recent post described the performance loss in the Arduino’s digitalRead() and digitalWrite() functions, compared to raw pin access.

Can we do better – i.e. hide the details, yet still get the benefits of raw I/O? Sure.

If you’ve used JeeNodes and in particular the “Ports” library, you’ll have noticed that there is a C++ class which hides the details of each port (i.e. JeeNode port, not ATmega port). Let’s look at that first:

Screen shot 2010-01-06 at 12.40.09.png

I’ve omitted the implementation, but there are still lots of secondary details.

The main point is that this is now C++, and uses a “Port” object as central mechanism. Each object has one byte of data, containing the port number (1..4).

Due to heavy inlining, there is almost no additional overhead for using the Port class over using digitalRead() and digitalWrite(), on which they are based. I verified it by running similar tests as in the recent post about pin I/O:

Screen shot 2010-01-06 at 12.48.50.png

Using the definition “Port orig (1);” – and sure enough the results are nearly the same.

There are two issues which make this approach sub-optimal: using the slow digital read/write calls, and storing the port number in a memory location which needs to be accessed at run time. There is no way for the compiler to optimize such calls, even “orig.digiRead()” should be the same as writing “bitRead(PORTD, 4)” in this example.

That’s where C++ templates come in. Check out this definition of a new “XPort” class (named that way to avoid a name conflict) and an example of use for port 1:

Screen shot 2010-01-06 at 12.54.02.png

(As you can see, I’m switching to a different, and hopefully clearer, API along the way)

There’s some funky <…> stuff going on. We’re in fact not declaring one class, but a whole family of classes, parametrized by the integer included in the <…> notation on the last line.

The big difference, is that each class now has that integer value “built-in”, so to speak. So we can define member functions which directly pass that value on to the corresponding bitRead() and bitWrite() macros. And then all of a sudden, all the overhead vanishes: since the member needs no access to object state, it can be made static, and since all the info is known in the header, it can be made inline as well.

So the above template is C++’s modern way of doing far more at compile time, allowing the optimizer to generate much better code.

Note that templates come with some pitfalls: first of all, it’s very easy to inadvertently generate huge amounts of code, so very careful inlining and base class derivation is essential. The second problem is that templates tend to be “instantiated” as late as possible by the compiler, which can lead to confusing error messages when the templates are wrong or used wrongly.

I’m still just exploring this approach for embedded use. The potential performance gains are substantial enough to give it a serious try. My hope is that the hard work can be done in a library, so that everyone else can just use it and benefit from these gains without having to think much about templates, let alone implement new ones. The “one” object declared above acts like any other C++ object, so using it will be just as easy as non-template objects.

Does the above lead to fast code? You bet. Here’s a test sketch:

Screen shot 2010-01-06 at 13.05.29.png

And here’s some sample output:

Screen shot 2010-01-06 at 13.06.14.png

As you can see, values 5 and 6 are virtually the same as values 7 and 8. We’ve obtained the performance of direct pin access while using a high-level port-style notation to access those pins. This is why templates are so attractive for embedded use.

The timings are different from the previous post because the loops are coded differently. In this case, only the relative comparisons are relevant.

Open – Some notes (5/5)

In Musings on Jan 11, 2010 at 00:01

(This is the last part of a 5-part series on open source – parts 1, 2, 3, 4, 5)

JeeLabs134x55.png

Jee Labs is an open source shop – both open source hardware and open source software.

Nothing earth-shattering: I’m exploring what’s possible in the domain of physical computing, and have been focusing on simple ATmega-based wireless sensor networks for in and around the house. A simple variation on the Arduino theme, really. In fact, it’s pretty hard not to be compatible with an Arduino when your stuff is based on the same Atmel AVR chips. Because in terms of basic functions and software, an Arduino is (at least right now) essentially simply an ATmega on a board. Plus a serial port boot loader.

This “computing stuff tied to the physical world” is a lot of fun. And a lot of work.

There’s a substantial amount of software required for all this, from dealing with wireless to collecting, storing, and presenting measurement data. And that’s just going one way – to also robustly (and securely) control things in the house, there’s also this whole field of domotics to go into.

I love doing software development. I spend most of my time on it.

And with Jee Labs involved in both OSH and OSS, there is finally a way to dedicate all my time to this stuff.

It’s a very simple mechanism: the hardware side funds the software side.

Does that make Jee Labs a commercial undertaking? Yep. Small shop, but it’s about as direct as it gets:

Stuff sold => food on the table => hunger gone => happy coding.

So I’m hired by the shop to do the hardware and software side of things. As long as there is funding, I get to spend all my time on this stuff. All of it in the open, described on a public weblog, and with all designs usable by anyone to do whatever they like with it. Try it out, hack it, extend it, rip it apart, get rich with it, or ignore it – it’s all up to you. There’s nothing to steal, because your gain is not my loss. On the contrary.

Quite a few people have already done things like take the RF12 driver (software) and use it with their own hardware designs. Many others are taking some sensors (hardware) and tying it into their stuff with their own software. Cool – way cool, in fact. Hardly a day goes by without something encouraging or rewarding happening on some front.

The trick is sustainable funding. Which means I need to stay on the ball, and figure out what others would like to see from me. That’s good – normal market economics, really. There’s a lot of uncharted terrain, still waiting to be discovered, explored, and turned into projects and products. Just gotta look, listen, and keep moving. Which is a lot easier when everything is out in the “open” – hardware, software, … and ideas.

This concludes my mini-series on Open Source. It’s time to return to the techy stuff again!

Open? Really? (4/5)

In Musings on Jan 10, 2010 at 00:01

(This is part 4 of a 5-part series on open source – parts 1, 2, 3, 4, 5)

Screen shot 2010-01-06 at 00.06.41.png

For OSS (S as in software), “open” is all about collaboration. Yeah, sure, there’s a lot of re-invention and not everyone is willing to share when it matters (i.e. real innovation), but even just the collaboration of developers (producers) and users (consumers) is fruitful in the sense that bugs get exposed, investigated, and fixed. Even if asymmetric, this leads to faster evolution and better software.

And even if not always, OSS projects do tend to attract a lot of talent, and talent breeds creative solutions and excellence. Because of the low barrier to entry, the potential talent pool is huge. It doesn’t take a lot of talented / motivated people to make exceptional software. Collaboration among developers is the norm, not the exception – and the visibility helps, even if only by attributing status to the team participants.

With OSH (H as in hardware), things are a bit different. Sure, there too the exchange between producers and consumers helps create better products, no matter how asymmetric the groups are. Got a problem? Submit it to the forum, and it’ll be out for all to see so the producer(s) better get his/her/their act together.

But the numbers are different. Very different. The barriers to entry are a lot higher – designing hardware and actually building working “stuff” is hard and requires specific skills. It’s error prone, time-consuming, and above all it costs real money. You need equipment and inventory, the prototype turnaround times take ages when you’re doing this on a budget, and serious testing can be extremely difficult. Not to mention getting production QA up to scratch and the QC yield high enough.

Want some examples? Try “Chumby” or “Adafruit”, each driven by amazingly talented and dedicated people. Exercise for the reader: find out how many people are behind each of these OSH shops…

What about the Arduino boards? Well, to be honest, I have no idea who designed them, nor what discussion is going on about them for future developments. The Arduino “Mega” sort of dropped out of the sky once it was on the shelf.

Because, you see, selling hardware generates income. And any income, say N dollars, divided by anything but 1 would be less than N. What seems to be happening right now, is that OSH producers are indeed “open”, but only after the designs have been created, tested, and manufactured. All the hardware work has been done, all the choices and decisions have been made, and all the time-consuming steps have already been taken.

So OSH “shops” are in fact putting a fairly large distance between themselves and any potential contributors (and competitors). Want to see, use, or even alter what they do? Fine – but not before they’ve got their revenue streams set up.

Don’t get me wrong. This is absolutely everybody’s full right. To be an entrepreneur means you get to take the initiative and you get to make all the choices you want. That’s the whole point. It’s just not really open.

I’m not even sure it could be done any differently. At some point, as producer, you’ve got to have an advantage over others to be able to justify the time, effort, and money spent on whatever it is you do. And given that most supermarkets want money to hand over the ingredients for your meals, incentives tend to work best if they too are in terms of money…

But there’s preciously little collaboration going on. Ideas aren’t flowing freely, far from it. And as with OSS, some people are waiting on the sidelines, trying to soak up as much info as they can, and keeping ideas to themselves to try and create their own revenue source when the time is ripe.

There’s nothing inherently “wrong” with that.

I’m the same – ask me about what my plans are, how I solved some tricky issues, and I might give you an evasive answer… because I’m afraid someone might “beat me to the punch”, or become a competitor.

It looks like OSH is being treated as a zero-sum game, i.e. “your gain is my loss” and vice-versa. I’m not sure it actually is, even w.r.t. money. In fact, as far as ideas go, I’m convinced that it definitely isn’t – ideas breed more ideas, infinitely so. We just need to figure out and implement the mechanisms which encourage the flow of ideas. Right now, neither OSS nor – especially – OSH are quite there yet.

In the first part of this series I started out by saying that there is hypocrisy involved. Whoa, that’s a big word. Let me explain…

First, allow me to summarize: with OSS there is usually no money involved – it all runs on voluntary contributions and team participation. With OSH, on the other hand, there is always money in the equation, because “stuff” is involved, i.e. real atoms, not just bits. And once money flows, normal / healthy economics will lead to revenues, profits, and income, no matter how modest or at what scale. Fine. Cool, even.

Except… with physical computing, there is always both hardware and software (call it “firmware” if you like). The hardware makes things possible, the software makes things happen. Either one is useless without the other. Yin and yang. Yet hardware and software are quite different beasts, and require quite distinct skill sets, so the people doing (“good at”) the hardware are often not the same as the people doing (“good at”) the software.

If I were involved in the hardware side of physical computing, and wanted to get rich (a real possibility, since there is money involved), I’d work like crazy on the hardware, and then kick it out into the world (i.e. sell it) with the minimum amount of software I can get away with. I wouldn’t put it that way of course, but I’d say “hey, I did my part, now someone else kick in to finish the other part”.

Then I’d sit back, or rather: participate just enough to keep up appearances. Build a community, take the lead, create the infrastructure for (software-only!) collaboration, and – pardon the bluntness – rake in the money…

If I were involved in the software side of physical computing, I’d design my own hardware and set up a shop to generate a revenue stream, so that I’d have at least a chance of funding some of my software efforts. Because I wouldn’t know how to sustain the necessary Open Source Software development for it in any other way.

I’ll leave it to the reader to be the judge as to what extent the above is happening today, and I urge you to look well beyond what I am doing with my little Jee Labs setup…

All I’ll add to conclude with, is that Open Source Hardware is not nearly as open and collaborative on the hardware side and in the design phase at it could be, and that the Open Source Software efforts which need to take place to make such hardware tick are not funded nearly as well as they should be to become truly attractive / sustainable.

And that’s a crying shame. Because somewhere along the way, we lost the ability to freely exchange ideas and benefit as a whole community from the potential of absolutely explosive creative growth. As it stands today, OSS and OSH are fine at spreading and even democratizing technology, but in effect stifling innovation. Open? How?

In tomorrow’s last episode – my feeble attempt to come clean, with Jee Labs. No climax. No grand finale.

Open Source Hardware (3/5)

In Musings on Jan 9, 2010 at 00:01

(This is part 3 of a 5-part series on open source – parts 1, 2, 3, 4, 5)

Screen shot 2010-01-06 at 00.04.22.png

On the side of open source hardware (OSH), the situation is less rosy.

The tricky part with OSH is that it’s about “stuff”. And stuff costs money right from the outset – to collect, to manufacture, to move around, to keep in stock, to use, to dispose of (OSS also has costs, btw – but indirectly, in the form of invested time, effort, and expertise).

There’s absolutely nothing wrong with money. Once money enters the picture, you can do things like add a margin to cover the costs. Or add a fat margin to cover more than the costs. Or pay others to get involved and bring in specific expertise. Which is way cool. Now we’re talking economy. And sustainability. And incentive. And of course: bread-on-the-table, i.e. income.

Oh yes, this is potent stuff. Markets. Resources. Competition. Profits. Investments. Capitalism!

The reason I’m pointing it out is that it’s a fundamental difference between OSH and OSS. And it’s obvious – everyone knows that to use software, you just gotta go to the proper site and download it. And to use stuff, you gotta purchase it and wait for it to be delivered on your doorstep (or go shopping and take it home, whatever).

We’ve all just been through the Christmas season. Shop ’til you drop. Pay and get.

But wait a minute… that’s for any kind of stuff. What’s “open source” hardware got to do with this? Well, you see, that’s where it gets a bit unusual.

Open source hardware means that the producers “open up” about what they are doing. In the electronics / physical computing / kits domain, the idea is to make the designs for the products openly available, often in the form of schematics and printed circuit board designs.

What’s the point? Well, you’re not locked-in, presumably. If you don’t like the design, you could take it and modify it according to your own wishes. Amazingly, this doesn’t seem to happen a lot. I can think of three reasons: economies of scale, interface conventions, and tools/skills.

The production of printed circuits is very much driven by economies of scale. It’s relatively expensive to create all the masks to produce good 2-layer PCB’s, with silk screens, via’s, solder masks, etc. So it makes a lot of sense for a supplier to have a certain number of PCBs made at once, and then sell individual boards at a cost lower than one-off production could. This is the traditional manufacturer’s sweet spot – it just happens to apply to tiny shops as well now, instead of just car manufacturers.

And hey, you know what? It’s great: all these manufacturers do us a service, by producing tons of goods of which we only need one, at fantastically attractive prices.

Another reason for not just starting from what there is and inventing tons of variations, is that an important aspect of a design is its interface with the rest of the world. This is particularly obvious with physical computing. Once you change interfaces (whether electrical, mechanical, or logical), you risk reducing inter-operability. This doesn’t apply just to OSH – think of all the “after-market” accessories in numerous domains. Many product choices end up affecting a lot more than that product itself over time.

The third reason why OSH isn’t spurring as much collaboration and innovation is that there’s often a fairly steep learning curve involved. It took me quite a lot of time and effort to learn how to design printed circuits, and how to have them manufactured. In fact, I made a ridiculous number of mistakes along the way, and had to throw away lots of failed trials. That’s not just wasted time and effort – that’s wasted “stuff”, and real money.

So it’s not surprising that not everyone grabs OSH designs and starts tinkering with them and producing their own boards. The process is non-trivial, takes time, and wastes money. We’d much rather buy ready-made products, or at least partially-made (including kits), with all the errors resolved so we can avoid them.

Again, this is great. Some people out there are willing to do the hard work, and others pick the fruits of their labor and compensate them for it.

Just keep in mind that here too, OSH is totally different from OSS. In a way, OSH solved the problem which OSS was never able to: to create a natural mechanism for rewarding time and effort spent.

So in a way, OSH could be called a success story. OSH seems to be picking up steam like crazy these days. Need an example? Try “Arduino”.

But wait, not so fast…

To be continued tomorrow – Open? Really?

Open Source Software (2/5)

In Musings on Jan 8, 2010 at 00:01

(This is part 2 of a 5-part series on open source – parts 1, 2, 3, 4, 5)

Screen shot 2010-01-06 at 00.05.56.png

With open source software (OSS), the immediate costs are generally very low. Get a computer and an internet connection (which you probably would anyway), and you’ve got all the expenses covered to benefit from OSS – and participate in its development. Infinite audience, negligible variable costs, very low barrier to entry. Other than your own time, skills, and effort.

The result? Explosive growth – SourceForge lists some 160,000 projects (or 380,000, depending where you look). Its two top projects (file sharing), have been downloaded half a billion times each.

Is it good? Sure, let a thousand OSS projects bloom, or more, why not.

But there’s also a lot of wheel re-invention going on (I should know, I’m an expert on that!). Some of it, and I fear quite a bit more than most people would be inclined to acknowledge, is extremely unfortunate: “hey, feature/app X in language Y is neat, let’s redo it in language Z!”

Is OSS leading to innovation? I’m not so sure. I suspect that when people stumble upon a potentially truly great idea, they will be tempted to re-consider whether they really want to share that idea with the world, and risk diluting it – by others adding more ideas (in the best scenario), or by others adopting it and succeeding at drawing more attention and claiming credit for it.

Another major hurdle is that turning an innovative idea into an innovative solution requires a lot of hard work. For “big” innovations, you’ll need to get all the right volunteers involved and motivated, have excellent people skills, and show true leadership. It’s easy to hit a wall somewhere along the road and lose interest before the product is finished (I should know, I’ve been there far too often).

It wouldn’t surprise me if for all the people sharing their ideas and working on them in public, there were at least as many others soaking up everything they can find and thinking “hey, maybe I can do something with this and become famous” – or even rich, by switching to a closed source software model. Even the GPL can’t fully prevent that, as long as you keep your software secret (and don’t get caught by having “similar bugs”!). The mere prospect of that happening can drain all motivation from an open source developer (see also: Prisoner’s Dilemma).

So all in all, my impression is that OSS isn’t all that innovative and collaborative as it’s often cracked up to be. Do you need an example? How about “Google”, which was built on the shoulders of open source. Sure, they do give back a lot, also in the form of open source. In fact, they provide what is arguably the greatest free service on the internet. But genuine collaboration w.r.t. their core innovations? No way.

There’s nothing “wrong” at all with this, btw. It’s just not really open.

To be continued tomorrow… Open Source Hardware

Open Source (1/5)

In Musings on Jan 7, 2010 at 00:01

(This is the first part of a 5-part series on open source – parts 1, 2, 3, 4, 5)

Jee Labs is an open source shop – both open source hardware and open source software.

There is money involved in open source. And there is hypocrisy. Let me explain…

The basic idea of open source, is literally to make the source of things freely available – and more or less allow reuse. With software, it usually comes down to making the source code publicly available to anyone. With hardware, the common approach is to make all design files publicly available to anyone.

40bsd.png 88x31.png 40gnugpl.gif

Depending on the model, you’re either allowed to do anything you like with it (all the way to hiding it inside your own for-sale stuff), or you have to at least maintain the proper attribution to the original author(s), or you restrict the use to projects which accept to be bound by the same rules as yours – preventing anyone from running away (and perhaps getting rich) with the original stuff and not letting anyone know they did so.

The “do anything you like with it” approach basically says: you can’t really steal it, because the author treats his/her work more as a form of self-expression than as promotion of something which deserves to be rewarded (whether monetary of in terms of fame). The latter “only on my terms” approach says “hey, as author I’m entitled to benefit at least as much as anyone else if there is any commercial value”.

I am (thank goodness) not a lawyer, so please treat all of the above as no more than my personal attempt to summarize what this is all about.

Newsgroups have overflowed and probably even been shut down because of debates on the differences between all the “licensing” approaches – (flame) wars have been waged about all this, and probably still do. Been there, done that. Yawn.

But in my view, all the open source approaches are minor variations of the same basic concept. The term “open source” itself is actually an amazing success, in that it manages to capture the essence despite all the factions and disagreements. It’s about volunteer effort, the power of gifts, competence, creativity, and the freedom of choice and re-use this approach offers.

Above all, open source is about enabling collaboration. Because, you see, the big differentiator of open source is that all ideas can be out in the open. When there is no need to protect ideas, there is also no need to restrict the flow of information. And, well… we’ve got this thing called the internet which happens to be friggin’ good at shuttling information (and ideas) around the globe.

So far so good. But there are a couple of weird things going on…

To be continued tomorrow… Open Source Software

,

Pin I/O performance

In AVR, Hardware, Software on Jan 6, 2010 at 00:01

There was a discussion on the Arduino developer’s mailing list about the impact of a small change to the digitalWrite() function, and for some time I’ve been hearing that digitalWrite() has a huge amount of overhead.

Time to find out.

Here is the sketch I used to measure how often a pin I/O command can be issued using various mechanisms:

Screen shot 2010-01-05 at 11.42.53.png

The logic is that I’m counting how often the same command can be called between timer overflows, i.e. every 1024 µs (one byte, incrementing @ 16 MHz / 64), before the timer tick count changes again.

And here’s the sample output:

Screen shot 2010-01-05 at 11.42.14.png

There’s a small amount of jitter, which tells me the loops are syncing up almost exactly on the timer ticks. Interrupts have not been disabled, so the timer interrupt is indeed being serviced – once for each loop.

What these values tell me, is that we can do about:

  • 10 analog 10-bit readings per millisecond with analogRead()
  • 128 pwm settings per millisecond with analogWrite()
  • 220 pin reads per millisecond with digitalRead()
  • 224 pin writes per millisecond with digitalWrite()
  • 1056 pin reads per millisecond with direct port reads
  • 1059 pin writes per millisecond with direct port writes

(I’ve corrected the counts by 1000/1024 to arrive at these millisecond values)

So the Arduino’s digital I/O in IDE version 0017 can do roughly 1/5th the speed of direct port access on a 16 MHz ATmega328.

But WAIT! – There’s a large systematic error in the above calculations, due to the loop overhead. It looks like the loop takes 1024000/1251 = 819 ns overhead, so the actual values are quite different: digitalRead() -> 3712 ns, direct port read -> 151 ns. Now the values are more like 1/25th!

So let’s redo this with more I/O in each loop iteration (all 4 ports):

Screen shot 2010-01-05 at 11.55.31.png

The sample output now becomes:

Screen shot 2010-01-05 at 11.56.59.png

With these results we get: one digitalRead() takes 4134 ns, one direct port read takes 83 ns (again correcting for 819 ns loop overhead). The conclusion being that digitalRead() is 50x as slow as direct port reads.

Which one is correct? I don’t know for sure. I retried the direct port read with 16 entries per loop, and got 67 ns, which seems to indicate that a direct port read takes one processor cycle (62.5 ns), as I would indeed expect.

Conclusion: if performance is the goal, then we may need to ditch the Arduino approach.

Update – Based on JimS’s timing code (see comments): digitalRead() = 58 cycles and direct pin read = 1 cycle.

Update #2 – The “1 cycle” mentioned above is indeed what I measured, but incorrect. The bit extraction was probably optimized away. So it looks like direct pin access can’t be more than 29x faster than digitalRead(). As pointed out by WestfW in the comments, digitalRead() and digitalWrite() have predictable performance across all use cases, including when the pin number is variable. In some cases that may matter more than raw speed.

Update #3 – Another caveat – Lies, damn lies, and statistics! – is that the register allocations for the above loops make it extremely difficult to draw exact conclusions. Let me just conclude with: there are order-of-magnitude performance implications, depending on how you do things. As long as you keep that in mind, you’ll be fine.

Merci, Paris

In News on Jan 5, 2010 at 00:01

Just got back from a couple of very relaxing days in Paris.

Here’s a quick impression of the 1680’s Comédie-Française (on the right):

IMG_5375.jpg

IMG_5384.jpg

IMG_5396.jpg

We went there to see a nice French theater piece. And although I’m not even remotely as culturally enlightened as such a remark might suggest, I did have a great time.

Normal transmissions to resume tomorrow…


JeeMon – one year later

In Software on Jan 4, 2010 at 00:01

JeeMon is a little web server / database / reporting tool I wrote a while back. It has been in use at Jee Labs for over a year now. Here is the electricity use for 2009:

Screen shot 2009-12-31 at 17.27.15.png

Couple of glitches, such as incorrect readouts when power failed and the sending node got reset to zero counts.

Here’s the gas consumption for 2009:

Screen shot 2009-12-31 at 17.28.06.png

Gas consumption (heating and hot water) is relatively high for this house, which is an open split-level design with lots of windows. Well isolated, but there’s simply no way to keep heat from rising up through the open stairways.

Here are the temperature and humidity readings for the past month:

Screen shot 2009-12-31 at 17.38.39.png

These are commercial S300 and KS300 sensors, and they seem to have frequent glitches. It looks like those could easily be filtered out, though.

Still, we managed to get €600 back on the 2009 energy bill, and with about 3000 kWh and 2000 m3 we’re 30% below the average consumption in this residential neighborhood, for both electricity and gas.

Being aware of what’s going on makes a difference, IMO. It has become a habit to turn off all the lights when I leave a room, and closing curtains right away when it gets dark. When we go out, we turn down the thermostat. Who needs high tech, when common sense is all you need? It’s so obvious, yet so effective …

The JeeMon database for all of 2009 is 26 Mb, i.e. tiny when considering that this includes every reading and some aggregated series as well. In fact, it contains a lot more data than what’s shown in the above graphs.

I’ve got several ideas and plans for JeeMon in 2010. I want to make it far more modular, so that nearly all its current functionality becomes available as easy-to-extend plug-ins. And it needs to be fullly configurable – as it is, JeeMon is still little more than a one-off implementation. But it has served its purpose very nicely already.


Better UART code and GPS

In Software on Jan 3, 2010 at 00:01

The UartPlug class which was recently implemented used per-byte access via I2C to retrieve each received character.

This isn’t terribly efficient, since it requires sending several bytes back and forth for each received character: first we address the UART to check its status, then we access it again to get one character from the FIFO. That’s six bytes of data for each received character.

It didn’t work well enough at 38400 baud, so I added a small buffer as optimization. The UartClass API hasn’t changed. The code which does this is:

Screen shot 2009-12-29 at 15.36.49.png

Nothing fancy, just a small buffer (10 bytes currently) to speed things up. The essence is that when lots of data is coming in, we get 10 bytes at a time, for a total I2C overhead of 15 bytes. That’s a 4-fold reduction in bandwidth.

The setup I’m experimenting with is this FV-M8 GPS receiver:

DSC_0928.jpg

The hookup looks daunting, but that’s because the connector is a bit small so I made a little breakout board for it:

Screen shot 2009-12-29 at 15.44.56.png

Here’s some sample output:

Screen shot 2009-12-29 at 16.05.09.png

The “gpsdemo” sketch is simply a serial pass-through, converting 38400 baud to 57600 along the way:

Screen shot 2009-12-29 at 15.46.40.png

But it still loses character along the way, as seen in this snippet:

Screen shot 2009-12-29 at 15.17.26.png

The RMC line has a partial line at the end, after the “*7B” checksum.

My guess is that this isn’t the UARTs fault, nor the I2C bus. What seems to be happening is that the Serial class does not buffer its output, so all output causes the sketch to become unresponsive for input. In theory the 57600 baud link should be able to get data out fast enough, but I suspect that in practice there is an occasional hickup.

If this is indeed the cause, then it’s another example of how wasting time can lead to problems.

Back to the GPS. After a few minutes it finds its position in 3D. The following info comes in 5 x per second:

Screen shot 2009-12-30 at 01.54.08.png

Some additional details are reported once a second:

Screen shot 2009-12-30 at 01.53.35.png

(ignore the last few garbled lines, I really need to lower that baud rate a bit)

Here’s an even more accurate fix I found in the data stream:

$GPGSA,A,3,12,22,09,28,18,26,17,27,,,,,1.32,1.02,0.85*02

So that’s up to 8 satellites – indoors, with stone/concrete walls and coated double-glazing windows. Impressive… older units I tried never even got a fix indoors!

To interpret this GPS data, check out Mikal Hart’s TinyGPS library, it can easily be hooked up to the UART plug – just use this code at the top of his test_with_gps example:

Screen shot 2009-12-31 at 00.58.36.png

… and init nss to match the baud rate of your GPS unit.


Wasting time

In Software on Jan 2, 2010 at 00:01

No, not your time or mine… the topic I want to go into here is how to let a micro-controller deal with things that happen over time.

The way an Arduino sketch works is as follows:

Screen shot 2009-12-28 at 14.59.53.png

The little slashes represent real code, where “stuff happens” so to speak. You can see that there is a clear flow of control, from the start of the code to the end, with delays and calls to other bits of code.

The trouble with this is that it can’t deal with multiple events. You can’t count pulses on an input pin and keep a LED blinking at the same time, for example.

In the old days, interactive computer interfaces were written in this same way. You’d enter a couple of values and then you got some results back. If you made a mistake halfway in the sequence, you had to restart and enter everything all over again.

Then “command menus” were invented: elaborate decision trees of the form “do you want to do X, Y, or Z?” – now at least it was possible to go back just one step. Today, user interfaces are event driven, responding to whatever interaction the user initiates instead of presenting choices. The user is in control, not the computer.

The traditional event-driven logic uses a dispatch / switchboard approach:

Screen shot 2009-12-28 at 15.00.03.png

The more modern approach is to use callbacks and closures. Fancier still is to use coroutines or continuations.

Could we use something similar for micro-controllers? Sure.

Callbacks are not that convenient in C, since it does not support the closures concept which would make it convenient (nor coroutines or continuations). Also, callbacks would have to be represented by pointers to C functions (2 bytes), whereas dispatch codes can probably be represented as a single byte (as long as we don’t have more than 256 of them). Efficiency and memory space matters on small 8-bit chips such as the ATmega.

This requires a change in mindset when writing sketches. A good way to get into such an event-oriented style is to imagine that there is no delay() function. In event-oriented code, you’re not allowed to wait for time to pass.

So how can we blink a LED if we can’t wait to switch it on and off? Well, instead of delays, we have timers. We’re still not allowed to loop until the timer expires, but we can tell this new type of timer to generate an event when its time has come.

For a low-end implementation, an event can simply be a small integer dispatch code.

So instead of waiting until we can turn the LED on or off again, we tell the system to wake us up again at the right time. This approach is also a perfect match for low-power nodes, by the way: sleep and wake-up on events can be taken literally to enter power-down mode and start up again.

The code might look something like this:

Screen shot 2009-12-28 at 16.00.55.png

Sure, it takes some more work than this delay-based code:

Screen shot 2009-12-28 at 16.03.06.png

… but it has the huge benefit that it’s now fairly simple to deal with other activities at the same time. What you need to do is turn each of those activities into one ore more events as well. They can then be added right next to the blink event code, as additional case(s) in the above “switch” statement.

An event-based dispatch mechanism adds a lot of flexibility. Tasks such as counting pulses, blinking LEDs, serial communication, wireless packet reception and transmission – these can all be processed as they occur. With as nice bonus that low-power sleep modes can become fully automatic: when there are no events, go to sleep!

The trick is to ban all uses of “delay()” and “delayMicroseconds()” and to never use “busy loops” to wait for something to happen. This has far-reaching implications, because all libraries must also play by these rules.

I’m going to explore this approach further. Maybe one of the existing nanokernels could be used as foundation. To qualify, it must be truly minimal (and I’m picky!) – i.e. it has to fit into an ATmega328 without claiming too much flash or RAM memory space.

Update – here’s a web page by “inthebitz” which illustrates the problems described above. It’s an absolutely genuine attempt to help people get started, and it no doubt does a very fine job at that, but it also shows how everything gets serialized time-wise. For slow things, it’ll be just fine of course.

Happy New Year!

In News on Jan 1, 2010 at 00:01

Here’s my little thank you for everyone who has supported Jee Labs in 2009 – in one way or another …

discount2010.png

No need to hurry, January has 31 days. I’m away now, will reply and send out discount codes on Monday.