Overo + Controller Area Network (Revisited)
A while back I posted about my work to enable support for the Microchip MCP2515 CAN controller with my Gumstix Overo. At the time I was confident of success because I was successfully receiving messages from my AVR setup. It was only recently that I discovered that in fact although reception was ok, transmission was not actually working.
Debugging the mcp251x Linux driver was an interesting exercise. Google wasn’t much help, as it seemed the few others attempting this same feat were either successful or completely unsuccessful. My situation suggested I didn’t have any electrical problems nor did I have any compatibility issues. Though resolving my problem I learned a whole lot of stuff I didn’t expect to, but it was really fun and rather educational. Maybe by sharing my diagnostics process others can learn something too.
Since I had a working receiver I was working under the impression there was some sort of issue with the way I configured the mcp251x driver. I tried upgrading to the latest version of the kernel (2.6.34) but no joy. I also looked at the latest development version of the mcp251x driver from the SocketCAN project. No new insight there at all.
From building a simple driver for the AVR chip I had a good working knowledge about how the MCP2515 works, so I figured it was really something to do with how the driver was written. It obviously wouldn’t be a generalized problem, but I had no idea what could be wrong. I started by adding a whole bunch of printk() debugging statements to better trace how the driver was called from userspace. Something odd was happening where the driver was repeatedly polling the MCP2515 chip for status. I traced what it was doing in the code and also followed the SPI conversation using my Saleae Logic logic analyzer. Nifty device by the way, highly recommend it. Turns out it didn’t really help solve the problem, but it did manage to help eliminate various false leads.
The SPI conversation and responses were perfectly normal. More printk() statements later and I was rather stumped. Then it occurred to me (at a time I wasn’t actually in front of my project) that there really should not be any polling. The driver configured itself with an interrupt line to know exactly when an event was pending. The polling I could observe over the SPI bus suggested this mechanism was not working (the status returned by the chip was correctly suggesting there was no pending events most of the time).
Turns out the default Overo build from OpenEmbedded sets up the GPIO pin for the interrupt line with a pull-down resistor. The interrupt line should normally be high then pulled low to signal a pending event. As far as I could tell, the pin was never actually being pulled high enough to be sensed by the Overo processor. I expect that the voltage level converters I’m using are to blame, failing to overcome the resistance of the pull-down resistor. This meant the Overo was seeing the GPIO pin as a low logic level all the time, leading to a constant interrupt. When this happens, the interrupt handler will fire and poll for the reason for the interrupt. Inspection of the code suggests this could starve the transmit code path from ever getting enough time to send messages. Indeed this would explain the situation I was observing. I adjusted the pin configuration in the u-boot recipe (now in my user.collection) to use a pull-up resistor instead, and now everything is fine. Because of the way that the mcp251x driver is written this should be ok.
I also discovered my previously published user.collection tarball was not matching my actual configuration. For reasons that are (again) likely due to the voltage level converters I lowered the maximum SPI rate to 500kHz. I’m using a CAN bus speed of 125kHz with a very low message frequency, so this doesn’t seem to be a problem. I was also annoyed by several of the useless messages printed to the console from the SPI driver and the mcp251x driver, so I commented them out with my own patch. Both issues are now fixed in the tarball. Download, roll into your own user.collection directory, and enjoy.
Thank you again for sharing your knowledge. This will surely fix my transmit problem and hopefully give better performance. I will need to be running at 1MHz and utilize at least 50% of the bus. Do you think this is possible with the current driver? I am also starting my own driver for the mcp2515 which will use two more pins for the rxb0 and rxb1 pins of the 2515.
Thanks again.
Comment by Dat — August 13, 2010 @ 7:05 pmHi Dat,
The existing mcp251x driver is pretty efficient as far as what is needed for communications with the MCP2515 chip. Separate interrupt lines for the RXB0 and RXB1 might give you a small win, you’d save the initial poll (about three bytes over the SPI bus) for the interrupt reason. You’ll still need the general interrupts for the transmit buffers and error conditions though.
I have no idea whether the general structure of the driver is efficient or not – probably you know more than I do about this.
I’d guess that you’ll want to figure out a way to increase the SPI bus speed beyond the 500kHz max I’m using. Theoretically the BSS138 mosfets used in the Sparkfun logic level converters should be able to handle a few hundred MHz, but suggest you need to put it onto an oscilloscope to know for sure. There are dedicated chips out there that do this sort of thing too, and probably do it better.
Each standard CAN message is a minimum of 44 bits plus data, and an extended frame is 64 bits plus data. I suppose that means its feasible to expect an upper limit on a very busy 1MHz bus might be ten thousand messages per second. The work to read each receive buffer on the MCP2515 chip will require SPI transfers of 4 bytes for standard addressing (6 bytes for extended) plus 3 to read the DLC. Additional transfers are required for any data bytes. Resetting the receive buffer requires 3 additional SPI transfers. If you use the existing driver that polls for status based on a single interrupt there will be 3 bytes to transfer to learn the interrupt reason. Add it all up (factoring in your expected size of data bytes) and see what SPI speed will be needed to keep up.
Another concern: depending on what you are doing for each message, the mcp251x driver might not be your biggest worry!
Comment by Bob — August 14, 2010 @ 12:25 pmJust wanted to thank you again for posting the fix. I solved my problems with the transmit and performs much much better.
Thank you.
By the way, would you mind if I post this site on the gumstix nabble forum? I think everyone should know about this if they are working with the mcp2515 chip.
Comment by Dat — August 16, 2010 @ 7:16 amBy the way, I am using a level translator from Maxim-ic model number MAX3002. It seems to be performing well. You can get free samples from maxim-ic.com
They are in 20 pin tssop package. You can find the breakout boards here: http://www.futurlec.com/SMD_Adapters.shtml
Comment by Dat — August 16, 2010 @ 7:19 amHi Dat,
No problem with posting this site or information elsewhere. Definitely happy to share the small information I’ve learned. And thanks for the tip for the logic level translator chip I’ll give it a try.
Comment by Bob — August 16, 2010 @ 7:28 amHi,
Thanks for sharing the info. This info were what I needed to get started with the MCP2515 on the Overo.
I still have RX problems (lost CAN frames) when multiple frames are send without space between them so I’m really interested in the perfermance. At an early point I changed to the driver from kernel 2.6.35, but that version did have problems (a lot of lost frames, out of order frames etc.)
I did post a patch to the socket can list, but have not received any comments on that one yet.
You can check it here where I also describe my problems:
http://old.nabble.com/MCP-2515-RX-problems-td29365471.html
It could be really interesting to hear if the old driver performs better than the modified 2.6.35 version
Comment by Benny — August 17, 2010 @ 1:59 pmHi Benny,
I looked briefly at your patch, but didn’t study it in depth – been really busy at work, no time for my hobby! At first glance it seems to look pretty good. I’ve been thinking about the challenge of insuring you don’t miss messages but also getting them in the correct order.
I hope to find time this weekend to review it in depth. Hopefully the experts will also give you feedback.
Comment by Bob — August 18, 2010 @ 7:28 pmBen,
I haven’t tried your patch but I can tell you that for the 2.6.34 version, I can stable frames at 2 messages per millisecond. I use a pic32 to generate can messages with the first byte being a sequence number. If I go any more than 2 per ms then I start losing frames badly.
Comment by dat — August 19, 2010 @ 7:56 pmDat,
Ok as you can see if you follow the link to socket can list I can receive ~3 frames per ms almost without loosing frames and at least I’m now told when I loose frames. I did use EXT ID and one data byte. Guess the worst case is Norm. ID and no data (shortest frame).
Comment by Benny — August 20, 2010 @ 1:47 amI run SPI @ 10Mbit.
It would be great if you have time to test/check/improve the patch.
Here is another patch someone posted: http://old.nabble.com/-RFC—PATCH–MCP251x-driver-using-Async-SPI-Interface-ts29267338.html
I will try to test yours and his patch soon.
Thanks.
Comment by Dat — August 20, 2010 @ 6:05 am[…] and the motor controller and sensor boards. I’m using CAN as the communication mechanism. My other post sort of explained how that all […]
Pingback by random scribblings»Blog Archive » Outdoor Robot Project Update — May 2, 2011 @ 7:11 pm