Applied Logic Engineering, Inc.

Technology and Management Solutions

Avoiding Bad Embedded System Designs

Kelly Nehowig

Copyright 2009 Applied Logic Engineering, Inc.

 

Maybe it’s because I’ve been around a while, but I’ve seen more and more cases of truly bad embedded system design lately.  For those of you not familiar with the term – an embedded system is any computing platform that is “embedded” as part of a product.  Examples are all around you; from microwave ovens to your kid’s toys to cell phones to your car (did you know the typical car now has well over 20 microprocessors on board?).  All of these devices have processing power and associated software that provides its unique functionality.

 

So why have these systems gotten worse?  It is a fact that these devices certainly have gotten more complicated – yesterday’s simple car radio with volume and tuning knobs now have a dizzying array of buttons that most people cannot figure out even if they tried.  But beyond the typical (crappy) user interfaces that seem to be the norm today, I think the problem is much deeper than that.  There seems to be a fundamental lack of understanding of the unique and challenging environment in which embedded systems are deployed.

 

One theory I hold is that part of the problem is that more and more systems / software developers come into the embedded space from the desktop space.  While the tools and languages may be the same in both environments, there are huge differences in how a software designer needs to approach the different platforms – some things can be pretty much assumed on the desktop platform and if you make the same assumptions on an embedded system, you will be doomed to a life of agony – as least as far as your project is concerned.

 

Let me give you some examples:

 

Input Power

If there is one piece of advice I could give designers of embedded systems, it would be to insure that your system has clean, stable, and consistent input power and to also be paranoid about losing it.  Without good power and a design that accommodates occasional power issues, you will constantly be confronted by weird system resets, lockups, data loss, and other unexplainable problems that apparently come and go at random.  Make sure that you know that power is good, especially during system power on and system power down.  Also make provisions in your design to make sure your system protects itself during power up/down – no memory writes, etc. during those time periods.

 

Processing Capability

Here is one of those points of differentiation between the desktop world and embedded systems.  While it is easy to forget that your desktop PC screams along with dual core/quad-core gigahertz processors, most embedded systems have much less raw processing power.  Make sure your software design accounts for this.  I can’t tell you the number of times that developers, happy with the performance of their code running  on a PC simulator, are shocked when they actually run their code on the target system only to see it plod along unacceptably.

 

Memory

Same issue with memory – while desktops have “virtually” unlimited memory, embedded systems typically have very defined boundaries around limited amounts of ROM and RAM.  Back in the old days of eight-bit processors, we would count every byte and use every bit.   Now with development environments that allocate and deallocate their own memory, it is much harder to keep track of your memory usage – but it can be the difference between a successful implementation and a failed one.  Be stingy with memory – think about every byte in every data structure and make sure it is necessary.

 

Fault Tolerance

A few years ago, I was chatting with a young software developer involved with writing a serial driver for the project we were working on.  He was mentioning to me that the data coming to our system from the host system was exactly 32 bytes and then he went on to tell me how his code processed the data.  I asked him what would happen if only 31 bytes arrived in the data packet from the host instead of 32.  He got this quizzical look on his face and said “Well I don’t account for that because it won’t happen”.

The funny thing is that it did happen.  And things like that happen all the time in every system out there.  Stuff happens.  You can’t always explain it, but you better account for it in your design.  Make sure you have redundancy and recovery functions in your code in case things don’t go according to plan, especially when interfacing with “outside” systems over communication lines, networks, etc.  This stuff always breaks down and fails to deliver – don’t let your system lock up because it is looking for that 32^nd byte that never comes.

 

Tightly Coupled vs. Loosely Coupled Systems

Another trend I have seen lately is that more and more embedded systems are interconnected as part of a larger network, passing data back and forth to accomplish sophisticated control, data collection, or other types of functionality.  The danger here is that it is easy to fall into the trap of a dependency on other systems for data, commands, or other things.  In this type of “tightly coupled” system, your embedded system can be subject to hang ups, freeze ups, or poor performance if it is dependent on this type of interaction.  Make sure that your system can function, even at a low level, if the interaction with other devices on the network stops working.

 

Don’t think of your Embedded System as an Extension of your Server

As an extension to the item above, another dangerous practice is to essentially use your embedded system as a “terminal” to a larger server that houses a database that is important to the operation of the embedded system.  Again, communication links can fail – you want to insure that your system continues to function.  It may be operating with restrictions or some sort of limited capacity, but you do not want to turn your device into a brick because it cannot access some remote database on some other computer.  Can you keep a copy of important data in a local data store as a fallback?  Is there another way to keep the system running?

 

Keep the User informed

Lastly, keep your user informed in a meaningful way.  If you’ve detected a problem that affects your unit’s operation, performance or capability, let the user know – don’t keep them guessing.  And don’t just flash a LED – if you can, give them meaningful information that conveys the situation.  Put messages up on your display,  have your system send an email or message to an operator, or at a minimum keep an internal log of each important event that can be accessed by the user at a late time.   There is nothing more frustrating than not being able to access the functionality of your device and not know why.

 

I’ve really just touched on a few of the important factors to consider when doing embedded system designs – as you can imagine, there are tons of other things to consider as well.  But keeping these basic items in mind will go a long way in insuring that your design is robust and able to operate in the way in which the user expects it to operate.

————————————————————————————————————————-- 

Kelly Nehowig is President of Applied Logic Engineering, Inc., a Minneapolis-based consulting company specializing in software and embedded system design.  A 28-year veteran of the software industry, he has worked with companies from startups to large corporate clients.  Kelly holds a MS degree in Software Engineering from the University of St. Thomas and a BS degree in Electronic Engineering from Minnesota State University – Mankato.  He can be reached at kellyn@appliedlogiceng.com.