Saturday, November 12, 2011

Build Your Own Davis Console Datalogger!!!

It all started off with my seminal post, Davis Weatherlink Software Not Required, to which I added not one, not two, but three exclamation points to get across the enormity of the discovery.  With a $15 USB to Serial Adapter, you no longer had to pay the exorbitant sum charged by Davis to get a computer connection to your Davis weather station console.  Enthusiastic enthusiasts then charged ahead with several wireless solutions.  How about a wireless XBee link as written up by af4ex on this wxforum post?   Or what about gwynethh's Wi-Fi interface via the almost perpetually out of stock Roving Networks RN-XV (only $35!) as written up on this wxforum post?  If you don't think a weather station with its own TCP/IP stack is cool, then you have no soul.  Or you don't care.  One or the other.

But there were still murmurs of discontent in wx-land, and they centered on two bits of criticism.  The first, of course, was this was no plug and play solution.  Wires would need to be cut and soldering irons plugged in to get anywhere.  That was until SLOweather pitched in and started selling ready to go serial adapters based on the console logger pinout.  It is nice to know that this discovery enabled a cottage industry to spring up, though it must be said that it is an extremely small cottage.

The second criticism was that my hack did not have a data logging feature.  Indeed, this did suck.  I knew the pinout at the back of the console, but this wasn't enough for me to figure out how the logging function worked.
The Pinout That Started It All

It wasn't until I tripped over this post by wxforum member wxtech that things got really interesting.  This is a guy who obviously isn't afraid of letting the magic smoke out of his electronic devices.  He actually unpotted the Davis dongle to see what made it tick.  He wrote "U1 is an Atmel 45DB memory", and it was already well known that the logger had one megabit of memory.  That was all we really needed to figure out the datasheet for the Atmel 45DB011 Dataflash chip.

Technically, it all became clear with the datasheet in hand.  The console talks to the flash chip via I2C, and the remote PC talks to the console.  The remote PC is not directly connected to the flash chip.  That's why my homebrew serial interface works: the datalogger and interface functions are completely separate.  And that's why Davis brought out the I2C lines to the rear connector.

It gets better.  The 1Mbit version of the 45DB011 will set you back less than a dollar.  So you know what that means...
It Means A Cheap Excuse To Embed A Song I Really Like

After a summer spent preparing to feed the rabbits this winter, I got off my butt and ordered up some parts to see if I could make a DIY version of the Datalogger.  I ended up putting my order in to Mouser Electronics,  rather than the usual suspect.  The reason was quite simple really.  They were the only outfit I could find that both stocked 1mm pitch ribbon cable by the foot rather than the roll and also had an SOIC breakout board.  The flash chips can be found just about anywhere.  Here is what I ordered up.
  • A couple Atmel 4Mbit Flash Chips and a couple 1MBit Flash Chips to store my data.  I wanted to try an experiment here to see if the console could actually read chips of the larger capacity.  I bought a couple of each just in case I wrecked one. 
  • A 20 pin, dual row, 2mm pitch IDC header to plug into the back of the console.  I bought four of these to stock up.
  • A 16 pin SOIC to DIP adapter so I could prototype this on a breadboard.  Why 16 pins for 8 pin parts?  My plan was to solder on one chip of each capacity, and try the bigger one first to see if it would work.  This way I wouldn't have to mess with desoldering one to try the other.
  • Some 1mm pitch ribbon cable to connect it all together.  I bought three feet, so I could make a couple cables if need be.  Note that the pitch of the ribbon cable is twice as fine as the header, because it has to pack twice as many conductors in to the same width.
You'll need solder and a fine-tipped soldering iron, of course.  While I intend to buy myself a Hakko FX-888 at some point this winter, I continue to get by against all odds with a pencil-type soldering iron I bought at Radio Shack sometime in the 80's.  You'll also want some kind of solder flux.  I have a flux pen (looks kind of like a highlighter) and it works great.

This first shot at things was going to take a prototype approach in case I had any wiring incorrect.  A 1Mbit and a 4 Mbit flash chip were soldered to the SOIC to DIP adapter.  I followed the Curious Inventor's most excellent instructions for soldering these fine pitch components like I did when I built my GoodFET.

And here is how it came out.  The stuff that looks like water droplets on there is just some flux that didn't burn off.  This is way bigger than actual size, BTW.  4 Mbit on the left, 1 Mbit on the right.
Not Bad, Given My Crappy Eyesight
I would figure out only later that I screwed up the chip on the right: the 1MBit part.  See how this is a 16 DIP breakout and yet there are 24 connections to solder to?  The rightmost pads don't actually connect to anything.  They are apparently there just to mess with people's heads.

Undaunted, I next built up my cable to go from the console to the breadboard.  It is easy to build up your own insulation-displacement flat ribbon cables as you can see here.  All you need is a vise or some kind of flat clamp to apply even pressure across the connector as the insulation is displaced in each contact (hence the name, Insulation Displacement Connector).  It actually takes a decent amount of force to press these on, so don't think you can do it with your fingers or mash it on with a screwdriver.  That will just ruin the connector and make you angry.

Of course, Davis would throw a wrench into things.  The connection on the back of the console doesn't leave room for the top clip on the IDC connector or the key that makes sure you get the orientation right.  I had to pull the top clip off and use an Exacto knife to shave the extraneous plastic bits off.  That got the console side of the cable done.  On the other side, I tinned the wires and soldered them to some 0.1" breakout headers stuck through a piece of perfboard.  This gave me something that was fairly robust with lots of flexibility for future prototyping.
Turned Out Pretty Well, I Must Say

Next I had to wire things up.  Based on the datasheet and the expansion port pinout, I came up with this pinout to connect the console to the flash memory chip.
Console Pin   45DB0x1 Pin
1 - SS        4 - CS*
2 - SCLK      2 - SCK
3 - MOSI      1 - SI
4 - MISO      8 - SO
7 - RST       3 - RESET*
14 - VCC      6 - VCC
14 - VCC      5 - WP*
16 - GND      7 - GND
And now a word of caution:
Before you get all excited and start wiring something yourself, be aware of this: in the pinout diagram above, Pin 1 is shown on the bottom right.  If you use an IDC connector like I did, Pin 1 will be on the top right.  So the blue edge of the cable that normally represents Pin 1 will actually be Pin 2, the second wire over will be Pin 1, etc.  Now don't go blaming me for the confusion.  Way back when I was determining the pinout, I picked Pin 1 as I did because that one has the square pad on the circuit board where the rest are round (proof here).  So I called that pin to be Pin 1 because Davis did.  Now we are stuck with that.  Just remember that the console pin numbers shown above correspond to the diagram at the top of this post and you should be fine.
I wired the cable connecting to the breadboard to the 4Mbit part for my first attempt.  I then powered off the console and plugged in the cable.  Note that the console only looks for the logger on powerup.  If you plug it in after that, it won't be recognized.  Note also that Davis recommends that you enter setup mode before popping the batteries on the console (press and hold Done, then press the '-' down arrow button).

Here is what everything looked like plugged into my breadboard.
L-R: USB to Serial Converter, Cable to Console, SOIC to DIP Breakout with Two Flash Chips

So once I hooked everything up, I powered up the console and none of the magic smoke escaped from the station or my circuit.  A good sign.  I then fired up Cumulus, told it to use the datalogger, and set the archive interval.  I then shut Cumulus down and left it for 10 minutes, hoping to see archived data when I fired it back up.

Nada.

I did't get it.  When I fired up Cumulus, I saw the LED activity lights on my USB to Serial interface flashing like crazy for some length of time, but there was no data?  It didn't make sense.  I thought it might have been because I started with the 4Mbit part, but it did seem to recognize the chip or the converter LED's wouldn't have been flashing like they were.  I tried a bunch of things until I came across this post on the Sandaysoft forums.  Just because you specify to use the datalogger on the GUI and are able to set the archive interval, that doesn't mean that it actually configures the console to use the datalogger or sets the archive interval.

Argh.

That post said you "have to" use Davis' Weatherlink software to actually activate archiving and set the interval, but that isn't exactly true.  You just need the Davis serial protocol document I linked back in the original post.

Here is the magic incantations I entered. 
"SETPER 1": Sets archive period to 1 minute. 1, 5, 10, 15, 30, 60, and 120 are OK too
"START": Enables the creation of archive records.
Remember you need these commands to be entered using a terminal program set to 19,200 8N1.  Set half duplex to see what you are typing and make sure all commands ARE ALL IN CAPS.  Like this:
Activating The Logger... Hopefully

So I gave that a shot and fired up Cumulus again.  This time it spent less time looking for the data but actually showed a progress bar during the download - something it didn't do before.  And then I got this.
Six Records Is More Than Zero
Success!  Now ask Cumulus to show me some graphs:
First Third Party Archived Data Ever
w00t!  We are Free At Last!  And all it took was a few bucks in parts.  You can actually build something up even  cheaper than the route I took here.  You could use this SOIC breakout for less than two bucks, and you might even be able to take an Exacto knife to a 2.5" laptop IDE cable you have lying around that has the correct 2mm pin spacing.  Either way, you are in business for around $5 - $10.  That plus the $15 Sparkfun converter, and you've got a bulletproof console interface with data logging capability.  But of course it goes without saying that if you try to build one of these things and break your console in the process, then you get to keep both pieces and I will disavow all knowledge of your mission.

By the way, this is the capacity of the standard 1MBit data logger.
1 Minute Archive Interval . . . . . . . . .  . . . . . . . . . . . 42 hours
5 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . . 8 days
10 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 17 days
15 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 26 days
30 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 53 days
60 Minute Archive Interval . . . . . . . . . . . . . . . . . . . . 106 days
120 Minute Archive Interval . . . . . . . . . . . . . . . . . . . 213 days
Will I get more capacity because I'm using a 4Mbit part?  I guess I'll find out in a few days.  I'll let you know in a future post.

So like the post that started all this off, the title of this blog also has not one, not two, but three exclamation points.  I thought it rated that.  Good luck to anyone else that gives this a shot.  I'll keep an eye on the comments for anybody who needs a hand.

27 comments:

  1. Some months ago I read the first article on building my own cable with the FTDI convertor. While browsing sparkfun I found a usb FTDI cable for about $2 more than the convertor. This meant I only had to wire the Davis end. I have been using that cable since. I expect that I may give in and build this data logger but I think using a netbook instead of a pc would give enough logging time for most power loses. a netbook will run for hours on the battery and uses about 1/10 of the power of a desktop pc. I am looking for a super cheap deal on one now. Thanks for all of your articles. I try to read everyone.

    ReplyDelete
    Replies
    1. Is this the cable you used? FTDI Cable 5V VCC-3.3V I/O Found Here http://www.sparkfun.com/products/9717

      Thanks, Glenn

      Delete
  2. Thanks DeKay, i have now produce a schematic from your description, and then layed out a PCB (all in my lunch hour!!)

    ReplyDelete
  3. Yeah, I saw that on wxforum. I'll try to find time to look your design over tonight. I was a little surprised at first that there were very few comments on my post over there - I didn't realize that people would be busy designing a PCB within a day or two!!!

    ReplyDelete
  4. You are a real inspiration, I hadn't realised how simple the logger actually was, otherwise I would have done this years ago!!

    ReplyDelete
  5. Did the 4mbit get you more capacity?

    ReplyDelete
  6. Sadly, no. Unfortunately I have been very busy and I haven't had time to blog an update. But I will, along with some other interesting stuff.

    ReplyDelete
  7. Awesome. I love your blog. Your writing style is hysterical. And the content is priceless.

    BTW, I'm considering either the Pro 2 or the Vue... is there any reason I should not get the Vue? It looks like the same hacks seem to work in the Vue, and the $170 price savings would be nice.

    Thanks again!

    ReplyDelete
  8. Ben, thanks. Glad you enjoy it.

    The same hacks do work on the Vue. I don't have one so can't comment otherwise on the Vue vs VP2 tradeoffs. Just go here where this is endlessly debated.

    ReplyDelete
  9. Thanks! I poked around here and at the wxforum to feel confident enough the hacks would work, but wanted to make sure before pulling the big "buy it now" switch.

    And thanks for the reminder to check over there... I found a really good deal on the Pro 2... so I went for that instead!

    ReplyDelete
  10. Does this also work for the Weather Wizard III? Or has anyone tried it out?

    ReplyDelete
  11. Does this also work for the Weather Wizard III? Or has anyone tried it out?

    ReplyDelete
  12. @Anonymous / Glenn: I believe the connector is different and don't know of anyone who has worked out the appropriate pinout. You could post your question here: http://www.wxforum.net/index.php?board=59.0

    ReplyDelete
  13. Hi DeKay, i need to have com port rs-232 instead of usb, can i use that http://www.sparkfun.com/products/133 instead the sparkfun FTDI basic ?

    ReplyDelete
  14. @Météo: Maybe, maybe not. Lots of people have had problems with this thing. Like this guy.

    ReplyDelete
  15. Are you going to make some of these connectors and sale them? I would like to purchase one.

    ReplyDelete
    Replies
    1. No, not my thing. But if you sign up to wxforum.net and send a message to a guy who goes by belfryboy, he'll set you up. He is currently designing an interface that includes a logger. See here: http://www.wxforum.net/index.php?topic=14687.0

      Delete
  16. It looks like all the different components exist to put together a wireless data logger interface to the console. Do you know if anyone has pulled this together, documented it in one place and tested it? I'd really like to be able to use the XBee based wireless solution with the data logger here.

    ReplyDelete
  17. I have a Raspberry Pi on order. Should be here this week. I am thinking that I can probably wire the Pi directly to Tx/Rx/Gnd using the UART 3v3 connections an the riser (pi side). No need for the voltage converter. I will post back if I decide to try this....

    ReplyDelete
  18. There is a much easier way to perform this hack; You can actually solder the AT45DB011D-SH-B (wide body) Flash right into the Davis Vantage Pro II. All but one pin is correct. Here's an image:
    http://ingineerix.com/pic/?davishack1

    Before soldering the flash, bend up pin 4 (CS) and don't solder that to the board pad. Then, after soldering run a jumper from pin 4 to the SS (pin 1) header, or you can "cheat" as shown above if you have good soldering skills. Add a .1uf and a 4.7uF cap in the 2 locations next to the flash. It will probably be fine with just the .1uf actually.

    Then to round out an elegant hack, instead of the wired serial, simply purchase a bluetooth serial module (available cheap on ebay) and install it inside. Pic:
    http://ingineerix.com/pic/?davishack2

    Voila! Wireless link to your PC or server!

    Be sure to get the widebody variant of the flash:
    http://www.digikey.com/product-detail/en/AT45DB011D-SH-B/AT45DB011D-SH-B-ND/1886154

    I have mine up and running with WEEWX and it works great!

    ReplyDelete
    Replies
    1. Hi i have just got my self a davis vantage vue 6351uk/6250uk.
      i would like to connect it to my pc to monitor via software and the internet. so do all i need is a widebody flash chip and a bluetooth serial module ? do you get anything like a wifi module instead of bluetooth ?

      Delete
  19. @DeKay:
    I have the Vantage Vue ISS sensors but no console.
    It turns out this is the US version which is forbidden in my country (as it's in the 900mhz range).

    Do you think it's possible to remove the transmitter entirely (or otherwise make sure it's not sending out any 900mhz signal) and hook the sensors to the computer with a cable?

    What do you think?

    ReplyDelete
  20. Hi, when I send the TEST\r command, Davis responds \nNO\n\r\r, just as with any command. Could you help me with this?

    ReplyDelete
    Replies
    1. See this post: http://madscientistlabs.blogspot.ca/2014/02/build-your-own-davis-weather-station_17.html

      Delete
  21. Buy Electronic Components in Online for affordable prices at Component Buy. Get 25% discount on Semiconductor micro controller relays Connectors and many more. visit http://www.componentbuy.com/

    ReplyDelete