To make sure we’re on the same page, we’re NOT talking about test system automation.  If you’re interested in that topic, please see here.

Use cases for embedded systems in industrial automation can be divided into two main classes: machine control and machine monitoring.

Embedded machine/equipment control – for this use case, the embedded system is controlling some aspect of the industrial machine/equipment.  It might be controlling the tight tolerance timing of a particular manufacturing process, it might dynamically adapt production of the part to improve product quality, or it might control the operation of a piece of industrial equipment out in the field.  Some examples of machine control applications include:

Machine/equipment monitoring – this could include generalized monitoring of a machine or it could be more focused specifically on condition monitoring, which generally has the objective of improving machine up-time/reliability and/or reducing maintenance costs and production loses.  Some examples of machine/equipment monitoring applications include:

If you’re looking for help with using embedded systems for your industrial application, there are two things you’ll want to do to get started: (1) develop a good set of requirements (see here for an industrial embedded requirements template) and (2) find a vendor capable of helping you (check out this Outsourcing Industrial Embedded System Development Guide).  If you want to chat about your application with us, you can reach out here.

The Big Hurdles

When choosing to develop a new product or platform based on an embedded system, companies commit to spending time, money, and resources. There’s an enormous benefit to finishing this development as quickly as possible so that the product or platform can be released to the customers or users as quickly as possible. Fortunately, by using certain tools and methods, companies can in fact shorten the development cycle, and improve their time to market.

I want to make a small distinction between a product and platform. The distinction surrounds the type of user. Embedded systems developed for external use by general customers are labeled as a product. When developed for internal use by specific end-users, it is labeled as a platform.

Don’t minimize the prevalence of these internally-used platforms. I’m including controllers that are used inside a product sold to a customer (e.g., a controller that operates a forklift would qualify, or a condition monitoring device for a gas compressor would qualify).

The key point to be made about these two classes of users is that both need tech support, repair services, upgrades, and so on. So, I’m going to refer to a platform as a product too.

Here is a breakdown of the steps needed to bring a product to market:

I want to describe each of these items a bit in this post and will do a few “deep dives” in future posts.

As you might imagine, some of these groups associated with these steps are more amenable than others to doing things that improve time to market (TTM). I want to touch on these things quickly and leave some details to those future posts.

The design activities are critical to choosing a platform that will achieve the requirements as quickly as is reasonable. The outcome of this group of steps has a big impact on the TTM.

The prototype activities are also critical to achieving the requirements, but the way they impact the project is in identifying any changes to requirements and design based on the outcome of some initial proof-of-concept-level development and testing. This group is all about failing fast, quickly identifying weaknesses in the component and design choices.

The last group of development activities is less critical to achieving the requirements, and is instead mainly focused on completing the development. There are some tools and techniques that can help speed TTM, but I think these tools have less effect (or maybe a better term is “smaller levers”) than the effect that the other groups have.

So, with that setup, look forward to future posts on design, prototyping, and development when I give you some ideas to speed TTM for embedded systems.

Want more information on developing embedded systems? Read our white paper “Top Five Embedded System Design Fails.”

Tip 1 for Reducing Embedded Prototyping Costs – Reuse. Reuse. Reuse.

In the world of software (embedded or not), coders do their best to prevent copy and pasting code. We do this by creating reusable code blocks (commonly referred to as functions, but come in a variety of different names) that can be called upon to perform tasks by various different parts of the system. Examples would include converting between data types, performing complicated file IO, or manipulating data structures.

This reuse in code provides a few different things that make the coders’ lives easier:

• If the functionality of the code snippet has to change, you only have to change it in one spot.
• You don’t risk “copy pasting” errors by having very similar code all over with just slight differences.
• Testing complexity is significantly reduced.

Reusing code is usually a pretty easily understood concept, and thus is implemented most computing projects.

The other aspect of reusability to consider is the hardware perspective. This is commonly accomplished with libraries of components (Integrated Circuits, passives, interconnects, stencil artwork, etc.). Most embedded engineering teams have amassed quite the library of hundreds, thousands, or even tens of thousands of components that they have used over the years in their projects.

Organizing that many components can really be a challenge. There are, luckily, a number of software providers that assist with managing this sea of schematic, layout, and mechanical drawings. Some even allow for parametric search and meta-data assignment for internal use. The *really* good ones help with BOM export/management so buyers aren’t flying blind when they go to actually purchase the components that designers put on the boards.

Wait, aren’t we talking about saving money? Those tools sound expensive …

Tip 2 for Reducing Embedded Prototyping Costs – Yea. Tools cost money. So does not buying them.

 I think my biggest trick for saving money in embedded designs is not cutting corners with tools for the design team. Too many times I’ve heard things like:

“ … we didn’t buy the debugger because it was $1,100, and we didn’t have that in the budget”

or

“ … we rented the cheapest scope we could for 6 months, because the engineers insisted we get one.”

Meanwhile they’re 6 weeks past their final milestone because there was a null pointer reference deep in their SPI Flash library, or they couldn’t see the overshoot on their clock because the scope only had 500 MHz of bandwidth. Sure, Michelangelo probably would have been able to paint the Sistine Chapel with a toothbrush, but for the rest of the artists out there, spending the money on high-quality brushes helps them produce a higher-quality output product. Engineers and their tools are no different.

The next level of reuse is entire hardware designs, not just the single components within them. Although it does happen that hardware designs do not use software, with the increased complexity of ICs on the market hardware designs are becoming more and more depending on software to work correctly. This pairing of hardware and software in reusable, modular packages allows for massive reductions in cost and risk.

Pulling from a library of proven hardware designs is not uncommon when moving between versions of products, or producing tech refreshes for existing product lines. This, of course, requires the team to first create the product from scratch, and then build off of it. Sounds great, but building up that library to pull from can be hundreds of thousands to millions of dollars – even for a small library.

To address some of these challenges, we’ve developed an ecosystem for embedded development called VERDI, to help reduce engineering costs and development risk. Check it out and let us know what you think:

In part 1 of this article, we introduced some well-suited applications for FPGAs, and highlighted some strengths and weaknesses of FPGAs. Now we transition over to some cautionary elements of utilizing FPGAs.

Some Things to be Mindful Of

Traditional development environments that tend to utilize text-based languages such as VHDL and Verilog are NOT for the casual user. The languages and the tool chains have a steep learning curve and the tool chains can be expensive.

Debugging tools –

The disadvantage of a run-of-the-mill sequential processor is that only one operation is executing at any point in time. The advantage of a sequential processor is that only one operation is executing at any point in time. When it comes to debugging, this sequencing makes life easier than debugging an FPGA. Simulation environments for FPGAs are generally pretty solid, but you eventually have to move into hardware, where your visibility into what’s going on decreases significantly. You can view outputs of course, and you can create test points, but you have to be able to probe all of those points, so you’ll need a logic analyzer, which can get very pricey. You may be able to get away with embedding test resources into your device (e.g. Xilinx has the Integrated Logic Analyzer), but this will utilize FPGA logic and memory resources, and is often challenging to see enough breadth or depth about what’s going on inside your FPGA. However, generally these tools are better for augmenting a true logic analyzer, as opposed to replacing them outright.

Cyber security –

A lot of FPGAs are embedding Ethernet cores, common processor cores, and some are even running an OS, making it so FPGA-based (sometimes referred to as SoC (system on a chip)) solutions look like another computer on the network. This opens up their vulnerability to more traditional attack methods. Take the time to understand your risks and mitigate them. Obscurity is generally NOT a solid security approach (check out The Great Debate: Security by Obscurity). Here are two articles to get you thinking:

• How to build an FPGA-based I&C
• Microsemi Steps Up Its Cyber Security Leadership in FPGAs: SmartFusion2 SoC FPGAs and IGLOO2 FPGAs Enhanced with Physically Unclonable Function Technology

Safety –

If you’re considering an FPGA-based system for a safety-related function, you need to understand the risks that you’re incurring. Here are a few articles to get you thinking:

• Safe FPGA Design Practices for Instrumentation and Control in Nuclear Plants
• SRAM-Based FPGA Systems for Safety-Critical Applications: A Survey on Design Standards and Proposed Methodologies
• Xilinx Reduces Risk and Increases Efficiency for IEC61508 and ISO26262 Certified Safety Applications

Where You Might Head From Here:

Hopefully these thoughts have given you some things to chew on. If you decide it might make sense to proceed with an FPGA-based embedded system and don’t have the time or manpower to create your own solution, check out VERDI.  If you’re interested in other industrial embedded system info, check out our resources page.

So you keep hearing about FPGAs being utilized in more and more applications, but aren’t sure whether it makes sense to switch to a new technology. Or maybe you’re just getting into the embedded world and want to figure out if an FPGA-based system makes sense for you or not.

One of the first questions you should be asking yourself on this topic is: relative to what? What are we comparing an FPGA-based solution to? Mostly we’re comparing to scenarios where general-purpose microprocessors or microcontrollers are being considered as the alternative technology.

Well-Suited Applications

Historically, FPGAs were VERY expensive (like up to thousands-of-dollars-per-chip, expensive), which limited the applications that utilized FPGAs to VERY expensive systems, such as military and aerospace systems. See Wikipedia – FPGA Applications for more use cases.

As FPGA prices continue to drop, I speculate we’ll see more and more advanced industrial equipment and machines (with particular needs) taking advantage of FPGA-based systems. Here are a few that we’ve seen:

• Industrial machine – Gear lapping
• Electrical Energy – VAR compensator
• Structural health monitoring – Railroad viaduct
• Electrical Energy – Remote monitoring
• Electrical Energy – Generator Condition Monitoring
• Industrial Equipment – Compressor Condition Monitoring

FPGA Strengths/Best Suited:

Much of what will make it worthwhile to utilize an FPGA comes down to the low-level functions being performed within the device. There are four processing/algorithm attributes defined below that FPGAs are generally well-suited for. While just one of these needs may drive you toward an FPGA, the more of these your application has, the more an FPGA-based solution will appeal.

1. Parallel processes – if you need to process several input channels of information (e.g. many simultaneous A/D channels) or control several channels at once (e.g. several PID loops).
2. High data-to-clock-rate-ratio – if you’ve got lots of calculations that need to be executed over and over and over again, essentially continuously. The advantage is that you’re not tying up a centralized processor. Each function can operate on its own.
3. Large quantities of deterministic I/O – the amount of determinism that you can achieve with an FPGA will usually far surpass that of a typical sequential processor. If there are too many operations within your required loop rate on a sequential processor, you may not even have enough time to close the loop to update all of the I/O within the allotted time.
4. Signal processing – includes algorithms such as digital filtering, demodulation, detection algorithms, frequency domain processing, image processing, or control algorithms.

Weaknesses/Non-Optimal:

With any significant benefit, there’s often times a corresponding cost. In the case of FPGAs, the following are generally the main disadvantages of FPGA-based solutions.

1. Complex calculations infrequently – If the majority of your algorithms only need to make a computation less than 1% of the time, you’ve generally still allocated those logic resources for a particular function (there are exceptions to this), so they’re still sitting there on your FPGA, not doing anything useful for a significant amount of time.
2. Sorting/searching – this really falls into the category of a sequential process. There are algorithms that attempt to reduce the number of computations involved, but in general, this is a sequential process that doesn’t easily lend itself to efficient use of parallel logical resources. Check out the sorting section here and check out this article here for some more info.
3. Floating point arithmetic – historically, the basic arithmetic elements within an FPGA have been fixed-point binary elements at their core. In some cases, floating point math can be achieved (see Xilinx FP Operator and Altera FP White Paper ), but it will chew up a lot of logical resources. Be mindful of single-precision vs double-precision, as well as deviations from standards. However, this FPGA weakness appears to be starting to fade, as hardened floating-point DSP blocks are starting to be embedded within some FPGAs (see Altera Arria 10 Hard Floating Point DSP Block).
4. Very low power – Some FPGAs have low power modes (hibernate and/or suspend) to help reduce current consumption, and some may require external mode control ICs to get the most out of this. Check out an example low power mode FPGA here. There are both static and dynamic aspects to power consumption. Check out these power estimation spreadsheets to start to get a sense of power utilization under various conditions. However, if low power is critical, you can generally do better power-wise with low-power architected microprocessors or microcontrollers.
5. Very low cost – while FPGA costs have come down drastically over the last decade or so, they are still generally more expensive than sequential processors.

Some Things to be Mindful Of

Stay tuned for part 2 of this article, where I’ll throw out the caution flag and highlight some areas to be mindful of, revolving around debugging tools, cyber security, and safety.

Where You Might Head From Here:

Hopefully these thoughts have given you some things to chew on. If you decide it might make sense to proceed with an FPGA-based embedded system and don’t have the time or manpower to create your own solution, check out VERDI.  If you’d like more useful info on industrial embedded systems, check out our resources page here.  You can check out part 2 of this article here.

Beginning a new embedded design is both exciting and daunting. There are many aspects of starting a new design that are fun and engaging, however it’s important to step back and make sure you’ve made the right decisions and assumptions up-front, to reduce headaches later. Below is a simple checklist you can use when starting a new design.

Does everyone have a firm grasp of the scope of the project?

Most embedded projects seem simple when designing on the white board, but it’s often the case that large parts of the design aren’t yet known, or are not well communicated to the system architect. Make sure you work with your external customer(s) as well as your internal team to ensure all stakeholders have a firm grasp on the scope of the system. One of the worst phrases you can hear at demo time is “well, where is feature X?”.

Does everyone know their role?

It’s easy to just assign tasks to the team based on approximate time-loading, however this can be troublesome in a number of ways. Since each member of your team probably doesn’t have the exact same skill sets, the team should work together to best understand which tasks are the best fit for each member. It is, of course, important to continue to challenge yourself and team members, however if the project isn’t going to say on budget because a member needs to learn too many skills to complete a task, that doesn’t make sense. This touches on the next checklist item …

Does everyone have the training they need?

Not every new design is going to be “cookie cutter”. It’s very possible that a customer ( internal or external ) requires your new design to use a technology that no one on your team has experience in. This could be as simple as a transmission protocol, or as complicated as a brand new processor, DSP, or FPGA platform/architecture. It’s important to understand what training may be needed, build that cost into the project, and set expectations appropriately with all stakeholders associated with the project on how long it may take to bring everyone up to speed on the new technology.

Has someone done a ‘back of the napkin timeline’?

A big part of engineering is sanity-checking your decisions ( calculations and assumptions ) with estimation. You should always be able to look at a design or problem and get a rough order of magnitude of how large of a problem this is ( this statement obviously falls apart for very large programs such as the space shuttle, but for the argument of this blog post let’s assume this is just for embedded devices and smaller ). Being able to understand what the goal timeline, and what a realistic timeline ( and hopefully those match up closely ) is, will help you continue to set expectations with stakeholders, and better work with your team to meet milestones. There will always be projects that end up behind schedule and will be stressful. However, by having an understanding of what a possible timeline is before starting the project can greatly help reduce stress going in.

Do you have a block diagram?

Spend some time with a whiteboard and your team. Get everyone in the same room with a large, blank whiteboard and go to town. Start with the central processing IC ( Processor, DSP, FPGA, SoC, etc. ), and move outward. Most embedded systems need Memory ( both volatile and non-volatile ), power management, and communications ports. Additional components might include Analog to Digital and Digital to Analog converters. Get as much of the system down onto the Whiteboard as you can, and then start taking pictures. It is also a good idea to have a dedicated note taker for each meeting, and have those meeting notes posted for all team members to see ( it may also be appropriate to send these notes and the block diagram to other stakeholders ).

Has the team settled on what Software Tools will be used?

More than likely everyone on your team is going to be using a computer. And on those computers will be software that will help team members complete the tasks that are assigned to them. Projects live and die based on what software packages and technologies they use. In some cases the software you will be using will be picked for you by the silicon vendor you are using ( you’re going to be working with Vivado if you use Xilinx, or LabVIEW if you use National Instruments ). Other cases you may have a choice. It’s important to have your team spend time between projects working with different software packages so they know which ones work well and which ones are just garbage. By encouraging team members to always be trying new software packages, you are more likely to settle on one that is the right fit for your needs when a new project comes in the door. Additionally, if you have been using the same software package for 5 years or more, it’s probably time to revisit the marketplace to see what innovations have been made in the last half decade.

Have we bitten off too much to chew?

“I know it’s only two months, but do you think you can get this done by October?”. This is the kind of question that often comes from stakeholders ( internal and external ) from management all of the time. Those inside and outside of your team often are going to be pushing you to be a little bit smarter, a little bit faster, and a little bit cheaper. That’s their job. It also can be super stressful and frustrating (especially when they know darn well that it’s an unrealistic request). Sometimes there is just too much work to be done in the timeframe that you’re given (that’s why “Has someone done a ‘back of the napkin timeline’?” is so important). You’re options are usually:

1. Push back right away rather than let stakeholders think that their request was reasonable
2. Fold, and just try your best to get it done ( this is, obviously, the least desirable since it results in long days and a grumpy team )
3. Outsource/subcontract a portion of the project that you know is an easy bite-sized chunk that your team can quickly define.

The third option does not come without headaches – so don’t think it’s always the answer. It can, however, allow work to be done by a contractor in parallel with your team, thus making the unrealistic deadline a little bit more realistic.

So above is my quick New Embedded Design Checklist. The above items, along with reading datasheets, talking with suppliers, and ( sometimes far too many ) team meetings is how I start each new embedded design. If you’d like to chat more about starting a new design, and how I or Viewpoint Systems might be able to help, feel free to drop us a line.  If you’d like to learn more about embedded design considerations, check out this white paper on the Top 5 Embedded System Design Fails.  If you’d like more industrial embedded info, check out our resources page.

Recall from the initial introductory post on the topic of improving Time to Market, I introduced three stages in a typical product development process: Design, Prototyping, and Development.

This post will focus on the Prototyping stage for industrial embedded systems.

I broke down this Prototyping stage into two major components. See the chart below for reference to all the stages. Two stages are grayed out to highlight the one covered by this post.

Variations of this chart have been included in all the posts so far.

Recall from the initial introductory post on the topic of improving Time to Market, I introduced three stages in a typical product development process: Design, Prototyping, and Development.

This post will focus on the Development stage of an industrial embedded system.

I broke down this Development stage into two major components. See the chart below for reference to all the stages. Two stages are grayed out to highlight the one covered by this post.

Setup

Recall from a prior post, about the Design stage, that the hypothetical company already has a product out in the market that needs to be upgraded. The product is electromechanical in nature (it interfaces to stuff like valves, motors, sensors, …). About a couple 100 of these units are sold per year.

Also, remember that this series of blog posts has been discussing these stages relative to comparing various hardware and software platforms. Since we are going to replace the existing obsolete controller, I’m considering 3 approaches to replace this old controller.

The first approach starts with a completely COTS hardware approach and ends with a custom hardware (single board) solution. The second approach starts with completely custom (single board) solution and uses that hardware through the whole product development cycle. Finally, the third approach starts with a COTS controller and combines that controller with both COTS and, if needed, custom I/O hardware. I label this approach as “Hybrid” since it considers both COTS and custom hardware.

IMPORTANT CHANGE OF PLANS: Note that in prior posts, I’ve only compared the Hybrid approach with a sort of merged COTS & Custom approach. But, while writing this post, it became clear to me that I really needed to differentiate the purely COTS and Custom approaches since they are so different when comparing the amount of time each spends in the PROTO and DEV steps. So, I’ve expanded the comparison bar chart below to make these differences clearer by reviewing 3 distinct approaches.

You might remember from the design post that options for this hybrid approach are many. Back in that earlier post, I recommended that you look for a platform that uses components close to the same physical form of your end goal. One example is the PC/104 standard. Utilized for decades (i.e., since 1992), it has now evolved to support the PCIe bus. Another option worthy of consideration is the VERDI platform, based on the NI SoM RIO controller and several pre-developed I/O hardware modules. Finally, on the low end, the Raspberry Pi and Beagle Boards offer some pretty amazing capability for a very low price – the downside being that this hardware is not as industrially robust as the others (e.g., the BeagleBone and RasPi have no heat spreader whereas the NI SoM RIO does).

OK, now that I’ve done this “overview”, let’s talk about the Development stage.

Development

After completing the Prototyping stage, the Development stage picks up after the hardware and software designs for this updated product have been vetted and the team is ready to begin creating the actual product. The Development stage contains the DEV and DEBUG steps which are intended to assure that the product is functioning as desired and doing so robustly. Also, very importantly, this stage includes the REL step for assuring that the product documentation is completed and archived and the product is ready for manufacturing. This REL step is boring but saves you from the embarrassment and frustration when you need to make a revision in the future and can’t find the layout files! Such a simple step but so important too.

Check out the bar chart below. The biggest point I want to make in this chart (and the reason I explicitly describe the 3 hardware approaches) is the inherent tradeoff between prototyping and development. Specifically, when the hardware used to prototype is different than used to develop, a lot of time is spent in either PROTO or DEV, but not both.

For example, if following the Custom approach, then the effort in the PROTO step is much more than if I were using COTS hardware. And, vice versa for the DEV step with COTS since there is a large effort to convert the functionality of the COTS hardware into custom hardware.

For the platform that starts with a COTS controller and as much pre-developed I/O hardware as possible, much if not all of this same hardware can be used in both PROTO and DEV steps. Consequently, I can choose to split the effort across those two categories, as illustrated in the bar chart. (Remember the sum of all the blue Hybrid bars totals 100%).

Finally, by way of comparison, experience shows that developing a Custom hardware solution from scratch takes something like 4X the time that is takes to combine pieces of pre-developed hardware modules from the Hybrid platform into a custom single board.

All these observations are illustrated in the bar chart which shows radically different bar lengths in PROTO and DEV for the three different approaches.

Interestingly, when following the Custom approach, the DEV step effort might be reduced relative to the Hybrid level. Why? Because all that hardware developed in the PROTO step would have been used exactly as is in the DEV step without any redesign, whereas the Hybrid approach would need to take the pre-developed I/O from each module and layout all that modular HW onto a single board.

CAVEAT: This benefit of Custom over Hybrid in the DEV step assumes no rework is needed on the Custom approach after the PROTO step is completed, a potentially unlikely scenario since so much information is discovered during the PROTO step. Rather, it is often the case that the DEV step with Custom is larger than the Hybrid case due to unexpected hardware redesign.

Even with the Custom approach, the blue bar for DEV would still not be zero length because the software created during the Prototype stage is not product-worthy, being temporary and somewhat hacked together to show functionality as quickly as possible.

In summary, the three scenarios considered are 1) Hybrid platform (COTS controller, pre-built custom I/O), 2) completely Custom, and 3) completely COTS. Then, the combination of time spent in the PROTO and DEV steps might be as shown in the table below.

The key takeaway in this table is that you can save time (and cost) by passing thorough the Prototyping stage as quickly as possible by utilizing previously developed hardware. And, if that same hardware is also used for the Development stage, significant time (and money) is saved.

One downside of a Hybrid platform appears in the PROTO step. If the COTS controller and pre-developed hardware does not have the type of I/O needed for the PROTO step, then the COTS approach will certainly be best for PROTO since that platform likely has all the I/O you need.

Another way to describe the lesson in this table is that designing custom hardware takes effort and cost. When a hybrid COTS and pre-built I/O can be used both for prototyping and development, you win.

Note that these numbers don’t include any costs associated with the actual hardware, which would be substantially larger in the ‘All COTS’ scenario than the ‘All Custom’ scenario.

Debug and Beta

When the DEV step is done, the application is ready for debug and testing. No matter which approach you use in the COTS vs Custom spectrum, you will need debugging. Consequently, as shown in the bar chart above, I’ve given each of the 3 approaches roughly the same schedule, with a slight advantage to the approach that uses pre-developed hardware since there will likely be fewer last-minute tweaks to the hardware I/O. Check out some suggestions for debugging techniques.

After you’ve debugged the system, you should have various users try your system to see if they can break it. Good beta testers take pleasure in breaking your stuff.

In the end, there is just no way around this DEBUG step of debugging and beta testing and hence not much can be done to shorten the time.

Release

After the testing steps, you are ready to release the product to the manufacturer. This is the time to complete the configuration management of your system, such as labeling or tagging the version of your software, assuring that all the documentation (e.g., BOM and drawing package) is complete, before releasing your product to production.

At this REL step, the actual hardware platform selected does not affect the effort much, and I’ve given each of the 3 approaches the same time to complete.

As with the DEBUG step, not much can be done to reduce the amount of time on this step.  If you’re interested in learning more about industrial embedded systems, check out our resources page.

Recall from the previous post on the topic of improving Time to Market, I introduced three stages in a typical product development process: Design, Prototyping, and Development.

This post will focus on the Design stage.

I broke down this Embedded Design stage into three major components. See the chart below for reference to all the stages. Two stages are grayed out to highlight the one covered by this post.

This chart was taken from the previous post and modified to add some labels, such as “[REQ]”, for quick reference to the specific items. (Also, I changed “language” to “software”.)

Setup

To make this post (and subsequent posts on this topic) more relevant, I have in mind a hypothetical project for a company that already has a product out in the market. This embedded product is electro-mechanical in nature, meaning that it needs to interface to stuff like various sensors (voltage, temperature, …), valves, motors, and so on. The company expects to ship a couple 100 of these units per year.

The existing product needs to be upgraded due to obsolete or hard-to-get components. In addition, the product needs a refresh. For example, the present model has no Ethernet port for data downloads or remote operation. Some other user interface and operational features need to be added as well.

The point is that the company already has a good starting point for requirements, based on the existing model, for the upgraded product. Contrast this situation with a completely new concept which has a blank whiteboard just waiting for requirements. Clearly the time expended on requirements gathering on the upgrade will be less than for a brand-new product.

With this hypothetical project in mind, refer to the chart below, which shows time spent on the project relative to the time reduced by using ideas and tools I will recommend in this post.

The “%Effort” bars show the estimated time using the recommendations in this blog series. The “%Reduction” bars show estimates for the additional effort it would take if the often-traditional approach is used. The “%Effort” bars sum to 100%. The “%Reduction” bars sum to an additional roughly 90% more work.

Requirements

The project engineers need to know what they are building. The sales and marketing team needs to know what they are trying to sell. So, the company needs to document the product requirements. I can’t think of any good tips on reducing this effort – it just needs to happen. Hence, the REQ step has no bar for reducing the effort.

Processor & Peripherals

Proper choice of processor and peripherals can certainly save time and effort. Familiarity of a processor and all its support tools has a large effect on making the product development cycle shorter. This choice can be very complex and time-consuming because there are so very many options. The choice of peripherals is large too. Deliberations about mixing and matching all these items with all the pros and cons takes considerable time.

However, if you consider a solution based on a processor board with many peripherals already designed in, much time can be saved. One such solution is the sbRIO board from NI.

Almost certainly, the unit cost price point would be higher for this “pre-developed” approach, than a board you spin completely anew. However, our hypothetical product contains so many other costly components that it makes sense to emphasize getting the upgraded product to the market sooner than to shave say 50% off the price of the controller alone (i.e., which contributes maybe a couple % of overall system cost).

Find a platform that is closest to your end goal in form and function that also requires the smallest amount of NRE development effort.

The same is true for analog, digital, and other I/O boards. Using “pre-developed” I/O hardware will speed development.

Going down this “pre-developed” path is tried and true. One that addresses enclosure and connectivity needs, is the Compact RIO from NI. This platform is robust, industrial, and solid. But note that this hardware is typically the most expensive option.

Among the maker-movement, which certainly has strong market-direction influence, are the Arduino and the Raspberry Pi platforms each with pre-developed controller boards and I/O hardware. Unfortunately, these “maker” solutions may not be robust enough for industrial applications.

My recommendation? Look for a platform that uses components close to the same physical form of your end goal. One example is the PC/104 standard. Utilized for decades (i.e., since 1992), it has now evolved to support the PCIe bus. Another option worthy of consideration is the VERDI platform. VERDI offers pre-developed I/O hardware and uses the NI SoM RIO for the controller.

Obviously, when choosing a “pre-developed” controller and associated I/O hardware based on off-the-shelf components, you trade off higher unit costs for lower non-recurring engineering (NRE) development costs. So, the best tip I can make is to find a platform that is closest to your end goal in form and function that also requires the smallest amount of NRE development effort.

In the bar chart above, I’ve chosen an NI processor (SoM RIO) which has a lot of the connectivity I need. By using this off-the-shelf product, I don’t have to spend a lot of time searching for all the parts. I’ve spent an estimated 30% of the effort that I would have had to otherwise spend researching all the options and support component parts.

(When I wrap-up this blog series, I’ll show you an illuminating diagram that summarizes platform choices in a way that nicely captures the evolution of a unit costs as each year (hopefully) brings higher sales volumes which allow unit costs to be driven lower. This chart really drives home the need to start with components close to the end goal. Stay tuned.)

Development Software

Obviously, the OS and development software have to be compatible and well-supported with the controller and I/O you select. Yes, there are lots of options here too, but most of the choices quickly settle into Windows vs. Linux, for the OS, and text-based vs. graphical programming, for the development software.

Linux is more prevalent in the embedded market place, but interestingly Microsoft is making significant progress their Windows IoT Core aimed at Arduino, ARM, RasPi, and other “tiny” platforms. This effort is an outgrowth of the Windows Embedded platform Microsoft has had for many years. Microsoft also has Azure, a full-featured cloud platform. So, you have choices.

Much of programming software choice is seemingly based on almost religious reasons (i.e., either you believe or you don’t), and this preference may so strong as to even drive your hardware selections.

…the graphical approach is generally faster to develop and easier to debug.

Nevertheless, based on experience using both text and graphical languages, my recommended tip is that the graphical approach is generally faster to develop and easier to debug. Hence, I generally prefer LabVIEW. I recognize that this choice drives me toward NI hardware solutions, but the tight integration between the software and hardware is also a bonus in reducing Time to Market, as compared with combining disparate hardware and software solutions offered by separate companies.

In the bar chart above, I’ve chosen LabVIEW which provides the development tools that I need. Also, since LabVIEW works with the SoM RIO hardware platform, I don’t have to spend a lot of time making sure that my software tools will work with my hardware. I’ve spent an estimated 50% of the effort that I would have otherwise spent researching OS and software development SDKs.   If you’re interested in learning more about industrial embedded systems, check out our resources page.

Technical Marketing Person:
“How’s the turbo-encabulator demo coming along? Are you going to be demoing the inverse reactive current for the unilateral phase detractors?”
Engineer:
“Um, no … I don’t remember seeing inverse reactive current for unilateral phase detractors in the requirements document …”

Some of us have been there (hopefully you have not!) the moment when someone from marketing asks if a feature has been included in the demo spec, and it hasn’t. Even worse, the question comes at nearly the end of the project. And the first thing you think is: “How did this happen?”
An engineer or a project lead has a seemingly endless number of things to keep track of and develop when starting a new project or program. A big part of engineering is solving technical problems, but there is a whole other dimension of less technical things that you should consider.
Requirements. Requirements, requirements, requirements! Getting specifications and project details from customers and stakeholders can be a really painful and stressful part of any project – technical or not. BUT, once those requirements are documented and agreed upon (even if they evolve over time), it helps reduce situations like the one above with our turbo-encabulator.

People. They’re everywhere. You work for some of them, and perhaps some of them work for you. Either way, you are all working with each other to get to the solution to a specific problem (or a large number of problems). Communication between team members really is super important. If you’re a leader, make sure you work to identify what your team members’ strengths and weaknesses are, and suggest different tools and methods to better themselves – especially if it’s communications skills. If you are working with the team, always work on bettering yourself as an engineer and as a communicator – take classes, solicit feedback at review time, and review your previous correspondences to make sure that what you thought you communicated actually was received that way.

Resources. Although you work with people, in some cases, it’s important to look at people like they are resources. You would not, of course, pick a 110V relay for your 280V design, or use twine to attach your communications satellite wire harness to the frame. In engineering we are presented (sometimes relentlessly) with different tools to pick from when solving problems. We spend quite a bit time researching and learning those new tools to help us pick the right ones for our particular project. We should apply this same level of rigor to picking people when assigning tasks on projects. You wouldn’t put your material scientists in charge of picking what embedded software framework to use, so don’t pick people to solve problems that they aren’t good at solving. Work within your team to decide which tasks are going to be taken on by which members.

I really enjoy working in teams, and learning to better understand what my team is and isn’t good at. As an engineer, I love problem solving. And if I’m given a puzzle of different personalities, skills, abilities, and passions, I look forward to the optimization problem ahead of me at the beginning of each new project!

Want to read more about this topic? Check out this white paper I put together on the “Top 5 Embedded System Design Fails – Common Pitfalls and How to Avoid Them”.

Developing embedded systems can be a ton of fun.  However, the debugging process often induces a lot of unneeded stress.  With the right resources, time, and techniques, the debugging process can actually be a lot of fun as well.  We recently published an article in Control Engineering which discusses some helpful techniques:  Six debugging techniques for embedded system development.  If you like what you see, sign up to receive more helpful information from Viewpoint.

You’ve got a great idea for a new smart industrial machine, instrument, or piece of equipment that you think has a good chance of doing well in your industry (okay, maybe it’s not as sexy as designing the Model S or the new iPhone, but in your world, this is a pretty cool idea you’ve got). You know you’re on to something because you’ve lived in your domain long enough to recognize real pain when you see it.

Now you need to actually make the thing. You’ve talked to some people you know that you think fit your target customer profile wonderfully. You get them all excited, a few of them excited enough to actually give you some money to be early adopters and try the thing out. Now the race is on. You know it’s important to be successful out of the gate. You don’t have a ton of time, and you sure didn’t get enough money to plan for 10 years down the road. You need to bootstrap yourself and get the thing out the door yesterday!

Maybe you have a pretty good handle on the mechanical aspects, the algorithms, and the sensors, but you either don’t have the background, or maybe you just don’t have the manpower, to tackle the embedded electronics hardware and software.

Read on to learn how you can reduce engineering costs (AKA non-recurring engineering or NRE) and development risks with a SOM-based approach for your industrial embedded system.

Embedded System Value Layers

One of the biggest ways to reduce engineering development costs and risk is by utilizing existing hardware and software whenever possible.

Of course if some performance criteria can’t be met with something off-the-shelf, you have no choice but to utilize a custom component to satisfy that requirement. But whenever possible, you want to re-use what’s already been developed because that cost is likely being amortized, and existing components have already been validated.

The main potential candidates for re-use in a scenario like this are:

1. Processor – absolutely. Many processors are architected for a variety of applications. In addition, if you can re-use many of the support components you put down onto a circuit board, then you get that re-use as well.
2. Memory – absolutely. A similar story applies as for the processor.
3. Communication Interfaces – in many cases, yes. While different applications do call for different communication interfaces, there are a handful of very popular interfaces for different applications, including Ethernet, USB, CAN, I2C, SPI, and RS-232/485/422.
4. I/O Hardware – Many industrial applications follow the 80-20 rule whereby 80% of applications are satisfied by some common I/O, such as analog outputs (e.g. +/- 10 V or 4-20 mA) with a maximum of few 10s of kHz sampling rate, thermocouples, digital I/O (e.g. 3.3V or 5V), and some sensors needing excitation (e.g. bridge-style or IEPE). After a few projects, a set of I/O circuitry built for other projects can be reused for new projects.
5. I/O Firmware – this is generally going to be tied pretty directly to your chosen I/O hardware, so depending on how much you can re-use your I/O hardware will mostly drive your ability to re-use your I/O firmware. On top of that, some serial interfaces are pretty prevalent, for example SPI, so that firmware has a high probability of re-use across different hardware.
6. Power Regulation – this is very dependent on the various ICs that you need for your application, but there are certainly ways to increase re-use here.
7. Application Software – much of this is going to be specific to your application, essentially by design. There still may be some potential areas for re-use with core functional blocks for things like PID loops and FFTs.
8. Packaging – this is another area that will mostly be unique to your design.

Based on this assessment, we’ll leave packaging and application software alone, since they aren’t usually the best candidates for re-use, and group the remaining items into two categories:

• Core Hardware – memory, processor, and some of the communication interface elements.
• I/O – we’re going to include I/O hardware and firmware in this category, as well as some additional communication interface elements. We’ll also lump power regulation into this category for now, even though it doesn’t quite fit, since our I/O (and core HW) will require power supply regulation.

Development Cost and Risk Reduction Category 1 – Core Hardware

The controller market offers a core hardware sub-assembly concept known as a System On Module (or SoM). There are several SoMs out there from various vendors. One of them is made by National Instruments. The NI SOM hardware (the NI sbRIO-9651 SOM) consists of the bare necessities to act as the core hardware element, including a processor, memory, some communication interface foundational elements, and an expansion connector to be able to connect your I/O to. The processor is a Xilinx Zynq SoC, which is a pretty awesome little chip, incorporating both an FPGA and a dual-core ARM-based processor in a single silicon package. Check out the Zynq-7000 devices here.

For more details on the NI SOM, check this out.

Development Cost and Risk Reduction Category 2 – I/O

Since you’re developing an industrial embedded machine, instrument, or piece of equipment, then chances are, you need to interface to something, so you’re going to need I/O. You might need analog inputs to monitor temperature, pressure, or vibration, or you may need 5V outputs or a serial interface to control a valve or actuator. Maybe you need GPS for position or timing information, or maybe you need to send data off to the cloud via Ethernet.

At Viewpoint we’ve taken some of the more common I/O and created modules out of them (some hardware, some firmware, and some both), and created a platform called VERDI to help with your I/O needs. Check out the VERDI modules here. If you need a module that we don’t have in our holster, we can design it for you.

Next Steps

Hopefully this article has provided a path for you to continue to make progress with your industrial embedded system. If you want to see if VERDI is a good fit for your application, request a demo.  If you’re interested in more useful info on industrial embedded systems, check out our resources page.

FPGA: to many developers and integrators this is a four-letter word for a scary, dark place where monsters lurk. I have been given an opportunity to learn FPGA, but not in the way that most FPGA developers use it. I am from the school of LabVIEW. This allowed me to add a whole new embedded tool (and sometimes a huge one) to my toolbox with very little brainpower investment. This has drastically increased the range of applications I can tackle. LabVIEW FPGA is a great tool, but not without its drawbacks, so let’s have a look at what some of the benefits, drawbacks, and just plain black holes there are.

LabVIEW FPGA Pros – the good

Reduced project completion time

While LabVIEW has a barrier to entry, like any other language, the bottom line is that one of the things that it excels at is being able to generate useful code to acquire and analyze real-world data quickly. This applies to LabVIEW FPGA too as it is just an extension of the base G language (LabVIEW graphical programming language). National Instruments (NI) does a wonderful job of abstracting hardware description language (HDL) from the G developer so one can write G code very similar to regular LabVIEW and have a working FPGA application in minutes.

Of course, a larger, more complex application still requires experience and good techniques to make a LABVIEW FPGA application meet size and timing requirements. This is still significantly less than most FPGA developers invest in learning HDL.

Simple simulation

LABVIEW FPGA has several simulation tools that allow you to quickly simulate the operation of the FPGA design. One very powerful tool is the ability to directly copy and paste LabVIEW FPGA code into a Windows context and execute the code with little or no changes. This will not preserve timing, but it is incredibly useful when you need to test the behavior of a state machine or other timing-independent code. There are also some useful techniques that can allow you to simulate timing if it is critical.

You get to leverage the existing NI ecosystem

This is another great benefit of LabVIEW FPGA. NI has a large (internal and external) ecosystem of code behind LabVIEW and that is extended to LabVIEW FPGA. Many libraries that are available for use in Windows are also available in LabVIEW FPGA.

There is also a large offering of common communication protocols that are available on the NI Tools Network, such as SPI, I2C, and RS-232 as well as some encryption algorithms. These allow a developer to implement one of these in a matter of minutes and are right at your fingertips via VI Package Manager.

For every I/O module they sell, NI also provides lots of example code that can be easily replicated and modified to your needs.

Large algorithms are much easier to understand in dataflow, and far quicker to implement in LV FPGA (an FFT is an example of this)

As an extension of the ecosystem discussion, some code is just much easier to understand in LabVIEW FPGA. Here is an example of a loop that implements an FFT.

Most developers, G or not, could look at this code and tell what it is doing. It does require opening a configuration dialog, but that is also very easy to follow.

LabVIEW FPGA Cons – the bad

Notable overhead on all operations

Because LabVIEW FPGA is an abstraction of HDL, it adds a fair amount of overhead to the resource usage of any FPGA design. As FPGAs get larger, this becomes less of an issue, but it can still have a very real impact on time to completion. LabVIEW FPGA still uses the same Xilinx compilation tools (ISE and Vivado) that any other Xilinx FPGA developer uses, an equivalent design done in LabVIEW FPGA will take longer to compile every time. Once or twice an extra 10-20 minutes does not present a problem, but a complex design that must be compiled 20-30+ times at the additional cost of 30 minutes per compile can add up to days of compiler time. While most of this time can usually be offset by doing other work during the compile time, if you (or a whole team) are waiting for a compile to finish to begin testing the fix on hardware, it can be a significant cost. Most of the time, it is not worth worrying too much about, but it is something that everyone involved should be aware of.

The other problem that this causes is that it allows you to get less done in a LabVIEW FPGA design vs. an equivalent HDL design when trying to pack more in to a design.

An example of this overhead would be in the implementation of a state machine, which is very common in LabVIEW code. A state machine in HDL can be recognized by the compiler and synthesized to a very specific set of flip-flops. In LabVIEW, a state machine is represented by a loop with a case structure that is evaluating an enum. The compiler is not going to recognize this as a state machine, but rather a collection of higher level objects that will use more resources in synthesis.

Dataflow might be challenging for those coming from a non-dataflow world

Of course, being a LabVIEW developer, I think in dataflow. That may not be so easy for those coming from the text-based world. If your code needs to be understood by a team of non-dataflow developers LabVIEW FPGA may (or may not) be harder for them to read than HDL.

You are a level removed from the Xilinx tools

Again, thinking of LabVIEW FPGA as an abstraction, NI does not provide nearly as much control of the FPGA design as when you are using the Xilinx tools directly. This means that many of the knobs that you might turn to constrain or optimize a design are just simply not available in LabVIEW FPGA. For 95% of LabVIEW FPGA developers this will not be a problem, but when you need those tools, it can be a significant hurdle. Notice that I used the word hurdle. In none of our designs was this a show-stopper, but in a couple of cases, we needed to engage NI applications engineers directly to help us solve layout issues.

Difficult to source control

All good programmers worth their salt use source control to protect and version their code. LabVIEW source files (VIs) are proprietary and binary and thus more difficult to source control. Viewpoint has created an add-on tool for LabVIEW that allows you to use SVN right in the LabVIEW IDE, but distributed version control (DVCS) tools like Git or Hg are very difficult to use with LabVIEW.

Some complex applications still require writing HDL

In some cases, you can’t get away without writing some amount of HDL. Viewpoint has completed some applications where the customer needed a custom physical connector or a specific LVDS driver on a FlexRIO Adapter Module (FAM), a unique Aurora configuration, or some combination of those. These specialized cases require the ability to write HDL for the FAM Component-Level IP (CLIP), which creates the interface between LabVIEW FPGA and the physical interface, or for the link layer protocol. It helps to have experienced developers in both LabVIEW FPGA and HDL.

The Ugly

Can only view within the LabVIEW tools

One major drawback to working with NI tools is that the file format is proprietary and can only be viewed and edited with LabVIEW, which has a non-negligible entry price in the thousands of dollars to start working with LabVIEW FPGA.

Next Steps

We’ve got over 700 LabVIEW FPGA projects under our belt. If you’d like to chat about your LabVIEW FPGA needs, you can reach out to chat here.

If you want to know more about how we can help with test system development, see here.  If you want to see how we can help with your industrial embedded needs, see here. Also, here’s some additional resources to help you on your journey:

• Need LabVIEW FPGA programming help? | LabVIEW FPGA Developers
• Outsourcing Industrial Embedded System Development Guide
• HDL FPGA Development – The Good, the Bad, and the Ugly
• Industrial Embedded Design Resources –  White Papers, Case Studies, Articles

We had fantastic attendance at our NIWeek presentation titled ‘cRIO Takes Flight – Creating a Helicopter Autopilot’. This presentation covered our development of a cRIO-based embedded controller that could control the orientation and position of an experimental helicopter — an actual autopilot!

The cRIO was connected to several of the helicopter sensors, including GPS, and flight-control actuators which enabled the real-time application control loops to keep the helicopter on a pre-defined flight path.

For safety reasons, the pilot had complete manual override capability and at any time could take command of the machine. However, with the rapid development and deployment tools offered by LabVIEW coupled with the flexible COTS cSeries modules HW, the cRIO embedded development effort was less than 3 weeks. Furthermore, initial autopilot flight goals were met in 2 days.

We congratulate both our client and our engineers for making this very cool project a huge success!  You can check out the complete presentation here: Creating a Helicopter Autopilot.  If you want more useful info on industrial embedded systems, check out our resources page here.

As Smart Grid investment grows, two important premises for Smart Grid design are:

1. Access to local power sources
2. An understanding of loads and disturbances on the grid at various locations

These local power sources are typically alternative, such as solar and wind, which have intermittent power levels.

Since the levels fluctuate, an important feature of proper Smart Grid operation is handling these erratic supplies. Optimal understanding of these disturbances and load changes increasingly requires measurements on individual AC power cycles.

Local power analysis systems typically have constraints in equipment cost, size, and power usage, balanced against the need for simultaneous sampling front-end circuitry and back-end custom data processing algorithms. Furthermore, many of these systems are presently deployed as prototypes or short-run productions, requiring a combination of off-the-shelf and custom-designed components.

Viewpoint Systems recently worked with a designer and manufacturer of leading-edge electrical power monitoring equipment to provide access to simultaneously sampled signals from the 3-phase and neutral lines of an AC power source.

Using a custom RIO Mezzanine card (RMC) designed and built for the National Instruments SingleBoardRIO platform, Viewpoint Systems delivered a powerful, cost-effective, and configurable solution for AC power signals measurement.

Furthermore, through the GPS component on the RMC, measurement units can be placed at dispersed locations while still providing adequate synchronization of acquired waveforms. This aids in localizing and understanding disturbances in power transmission and distribution, irrespective of any specific application.

Read the full case study to learn more about the application and view full specifications of Viewpoint Systems’ remote monitoring system for electrical power. If you have an embedded monitoring application that you’d like help with, you can reach out to chat here.  If you’d like to learn more about our circuit board design capabilities, go here.

The emphasis of this article is on the prototyping of the embedded controller.  When I say “embedded controller” in this context, it doesn’t mean that it actually has to control anything, but it will most likely have outputs. It could simply be monitoring signals to convey useful information to another piece of the system or system of systems.

What’s out of scope for this discussion?

• Manufacturing costs: while production volume should absolutely be taken into consideration for off-the-shelf vs custom component selection, I won’t focus on the manufacturing end of the product development process; rather I’ll focus on the prototype development costs. Of course, designing with production in mind (not just volumes, but DFA& DFM as applicable as well) is still critical.
• The post-production maintenance, obsolescence, and customer support aspects are not part of this discussion either.
• The front end research: in other words, an idea exists, but hasn’t been prototyped or proven out yet, or if it has, it’s been a lab-equipment-based proof-of-concept hack job.

Example Embedded Controller Costs:

I’m going to throw the caution flag here.  Examples are tricky. We like them because they are very tangible. They’re also very dangerous because they are so situational and make many assumptions. Your particular scenario may vary significantly, but we’ve got to start somewhere, right?  It’s worth noting that the labor costs below are fully loaded (salary, benefits, overhead, etc).  So with those caveats, I offer you the following two examples of embedded controller design costs.

Example 1 (circa 2017): Relatively Straightforward Embedded Controller

Example 2 (circa 2017): Relatively Complex Embedded Controller

Key Cost Drivers:

• Re-use libraries: these include both hardware components (e.g. a particular signal conditioning sub-circuit) and software components (e.g. a particular communication protocol interface).
• Off-the-shelf hardware availability: generally when a component can be purchased (that meets the designs requirements), it will be cheaper to buy than to design from scratch.

Cost Breakdown:

Estimates for engineering costs are challenging. The main reason is because something is being created from nothing, and there are a lot of components (software and hardware) that have to work together in harmony (protocols, algorithms, physical interfaces, power consumption, heat dissipation) in order for the embedded controller to perform as desired.

Much of the ability to generate reasonable cost estimates comes from leveraging past experience solving similar problems. Being able to recognize the differences in hardware and software platforms and requirements, allows seasoned engineers to quickly estimate the amount of work required.

The other cost estimation technique often utilized is bottom-up cost estimation. This is where you start from extremely low-level components that can be estimated, and sum the parts to come up with a final tally. When I say low-level components, I mean down to the level of something like:

• An algorithm, like an FFT, or maybe even lower level like a butterfly element
• GUI elements, like a plotting function
• A circuit board sub-circuit, like a particular SMPS

Combining past experience with bottom-up cost estimation is pretty typical in the engineering world.

The biggest challenge/unknown from a costing standpoint is generally estimation of integration and debug of the prototype. This is where the various components that were developed and simulated all have to start working together.

The costs that people don’t like to think about:

Embedded designers are more apt to want to focus on the creation aspect of the core design (e.g. coding on the software/firmware side, and circuit design on the hardware side), and less likely to want to pay as much attention to costs associated with tasks such as:

• Requirements – developing and managing requirements isn’t free, but it can cost you even more by not paying attention to them.
• Project management – hardcore developers often hate project management, often seeing it as superfluous. On the smallest of projects, project management can be handled individually. Otherwise, at a minimum, it’s a necessary evil, and if done well, it can make the development team’s life better by helping them not get overloaded, helping clear roadblocks, and providing clear goals.
• Risk mitigation – If nothing ever goes wrong with any of your projects, I’d be grateful if you’d get in touch with me and explain your magic.
• Documentation – This can get out of hand for sure. Keep it to a minimum. The three best reasons to document in my mind are: (1) to help you design in the first place (2) to help teach someone else whatever they need to know (3) to help you remember what you did a year or more later. If none of these criteria are met, you may want to second-guess your effort.
• Integration – Integration/interfacing of things is becoming more and more prevalent. Just always remember that whenever you create a new component, you need to interface with it somehow, and you should allocate time for this.
• Debugging – I wrote an article on this topic to help with the debugging process.
• Hardware re-spins – It is possible to design a custom circuit board well enough on the first shot that a re-spin is not needed;it’s just highly unlikely. I wouldn’t recommend planning for it.
• Certifications – You may not require certification for your prototype, but you probably want to start thinking about it so you don’t design yourself into a corner.

Next Steps:

Presumably if you’ve made it to this point in the article, you’re likely either:

• Overloaded and just need some additional resources to get the job done.
• Relatively new to the embedded world and are considering pulling in some outside resources to help.
• Curious about alternative ways of doing things.

If you’re interested in scoping the cost of an embedded controller for your application, you can reach out to us here.  If you’re interested in understanding how we can help, go here.  If you want to see the sorts of requirements you should be thinking about, go here.  If you’d like to get a feel for the engineering labor costs to develop the prototype embedded controller for scenarios that are able to utilize off-the-shelf hardware, see our engineering prototype cost calculator.

Background:

As electronic sensing and processing power increases, more and more companies are coming out with great ideas that are functionally and financially feasible. The problem shifts more from “if we can implement these ideas” to “how we can implement these ideas.” Large companies have had the resources for years, and in some cases decades, to embed electronic intelligence into their products. The threshold for feasibility is decreasing, making it easier for smaller companies to enter the market. This is mainly due to:

• Lower cost per processing operation
• Lower cost per bit of information stored
• Lower cost per bit of information communicated
• Continued miniaturization of electronics
• Lower power consumption per processing operation

To be clear, when I say “small companies,” I mean those with fewer than 100 employees. I’m talking about companies in the industrial space (v.s. consumer), that manufacture sub-systems or systems that are generally more expensive (~$1,000s – $100,000s per unit), and have a relatively low production volume (10s to 100s of units per year).

The differences between embedded design for small and large companies matter because:

1. They can help small companies become aware of potential pitfalls before they get in too deep.
2. They can level the playing field for small companies.
   

Differences:

The main differences between embedded design for small companies versus large companies can be grouped into four main categories: capital, people, process, and organization.

Capital

Smaller companies generally have less working capital. They may have limited funds for:

• Development and debugging tools (e.g. Xilinx Vivado, Intel Quartus Prime, OrCAD, Eclipse) for doing schematic capture, board layout, simulations, and software/firmware development can range from free to sometimes costing thousands to tens of thousands of dollars to purchase, with an additional annual maintenance cost for each tool.
• Lab equipment, such as logic analyzers and oscilloscopes (e.g. Tektronix Logic Analyzers,  Tektronix Oscilloscopes), can cost thousands to several tens of thousands of dollars per piece of equipment. There are less expensive routes for initial cost, but keep in mind that generally speaking (in steady state), you get what you pay for, so a piece of equipment that has a lower purchase price may have lower performance (so make sure you understand your needs), lower reliability, or take more time to use. This last point is one that people often don’t weigh strongly enough in their decision-making. Time is often one of the engineering community’s most valuable resources. Another option is renting equipment. This comes with the obvious benefit of lower cost, but the risk is the possibility of out-of-calibration or misuse from the previous renter.
• Lab space – make sure to have enough room for development hardware, lab equipment, and computers (as well as enough power to handle all this gear). In addition, ESD is an important aspect of dealing with sensitive electronic components (check out ANSI/ESD standards). Components can be fried immediately, or sometimes worse, set up for a latent failure.

People

• Team sizes are often much smaller. Whereas a larger company may have several people that only do embedded software or only do embedded hardware, a smaller company might have an entire product development team that is just a handful of people, limiting the expertise to what is truly core for the company. An embedded developer in a small company might wear many hats, or there may not be an embedded developer at all and the company may rely on external resources to augment the team.
• Focus: smaller companies often rely on particular domain expertise. This is the core of why they (the companies) exist, and may be what keeps them afloat. They may be scientists with a really cool new idea, or they may be a more mechanically oriented company that builds electro-mechanical machines or equipment. Because of this, they may not have the ability or inclination to become experts with embedded systems. The algorithm may be very important to the smaller company, but the way that it is implemented may not be. Having some capability to sense a particular property or measure a particular parameter may be very relevant, but how it gets done may not be.

Organization

• Agility: Small companies are able to respond/react to market feedback faster. This Tech Crunch article discusses some ways in which bigger companies are trying to behave more like smaller companies: How Big Companies Are Becoming Entrepreneurial
• Small teams feel ripples from small events more than a larger company would. For example, if someone gets pulled off onto another project or leaves the company, a larger company may have the resources to plug in a new person, but a smaller company may not have this ability. Even losing a 25 percent of a full-time person on a small team that can’t be backfilled can be detrimental to a project.
• Small companies generally have less red tape to go through in order to get approvals faster and try things outside the scope of what is considered normal. For example, if a new embedded system has been released to a customer, and that customer needs a feature that isn’t available in the release, the smaller company will likely have a much easier time getting approval to add the new feature if it makes sense than if the same request is made to a larger company.

Process

• A large company may have a very well-defined, rigorous process for development, such as various design review gates, release processes, and documentation, whereas smaller companies may just focus on getting stuff done. The benefit is that the small company may get through development faster, however the risk is that if there is a flaw in the design, or bugs aren’t caught, it gets more and more expensive to fix later on (check this out: NASA – Error Cost Escalation Through the Project Life Cycle ; even though it’s old and based on developing systems that are very complex, the general trend likely holds, although the multipliers likely go down significantly).
• Smaller companies are less likely to develop a rigorous set of requirements and specifications, and the embedded developer will likely be involved throughout the project, whereas in a larger company they may not get involved until well into the project, where they are handed a set of derived requirements for their portion of the design.
• Larger companies may be more mature from a configuration management standpoint, including source code control, release process and version control.

So What Now?

Hopefully this article improved or solidified your mental model in some way and you feel more confident to take the next steps either on your own or engaging with an external company.  If you’d like to chat about your embedded needs, you can reach out to us here. If you’d like to learn more about how we can help with your industrial embedded needs, see here.  Or, if you’d like more information on industrial embedded systems, check out our resources page.

Intro

Each year the various journals and publishers of embedded systems design material are seeing more and more talk of field-programmable gate arrays (FPGAs). FPGA technology is often talked about as one of the most powerful, but also one of the most frustrating parts of the embedded designer’s arsenal. With each generation of FPGA technology being faster, having smaller geometry and lower power, and with more logic gates, all for less money per operation, the possibilities seem endless. However, with great power (and re-configurability) comes, at least with FPGAs, some serious learning curves and frustrating days (weeks) at the office. Let’s talk about where FPGAs are today, where they were just a few years ago, and some pain points that still exist.

The Good

Large, Reconfigurable, Sea of Logic Gates

FPGA technology allows a designer to implement nearly any algorithm and/or control that she or he likes. Modern-day FPGA fabric is made up of tens to hundreds of thousands of logic gates and flip flops that can be combined together to implement everything from a button denounce circuit to an x16 PCIe gen-3 host controller. Using FPGAs within your design gives you the near infinite freedom to implement whatever functionality you need.

Abstraction Layers

It sounds terrifying to design a system using 74LS logic ICs, and the folks who were working on the first FPGAs had to do nearly that. Luckily today we don’t have to think on such a low level. There are languages, called hardware description languages (HDLs), which help designers describe their design to a synthesizer that then creates net lists of logic equations and registers. Additionally, there is a wide range of higher-level abstraction technologies and languages. Xilinx provides a tool called Vivado HLS (HLS stands for high-level synthesis) that allows a designer to write C, C++, or System C and have it generate HDL in the form of VHDL or Verilog. Here at Viewpoint Systems we often use a technology called LabVIEW FPGA, which allows you to describe FPGA designs using a graphical programming environment that leverages dataflow paradigms.

Resources and Ecosystem

Commercially available FPGAs have been around since the mid-1980s. That’s a long time for a technology, and its ecosystem, to mature. There are LOTS of resources available for everything from understanding how the silicon is laid out, to how the synthesizer generates net lists, to the best methods to write HDL to generate the logic structures you’re looking for. You can go onto Amazon and search FPGA and find hundreds of available books. If you google FPGA, you can find tens of thousands of answered questions available for reference. Because HDL is text-based, these question and answers are indexed by search engines, making finding information simple and fast.

Also, in the last several years more resources have come out about the higher-level synthesis tools such as Vivado HLS, Calypto, and LabVIEW FPGA.

Speed and Size

Modern FPGAs can run really fast – like hundreds of megahertz fast. You may be thinking: “But Tim, my Intel processor runs at gigahertz fast!” That is true! However, your Intel processor is a “general-purpose processing unit” – it does a large number of things pretty well, not a small number of things really well. FPGAs allow you to write massively parallel implementations of algorithms producing throughputs that can be 10x, 100x, or 1000x higher than any CPU on the market today.

Although high-level abstraction is great for faster time-to-market (TTM), you simply will never be able to get your design to run as fast, or be as small as writing the code by hand in a traditional HDL such as VHDL or Verilog. The abstraction languages and technologies can be great for quickly putting something together; however, the inevitable overhead will always make your design larger, and thus slower (lower maximum clock speed).

The Bad

Learning Curves

As anyone working with FPGAs on a regular basis will tell you, the learning curve is pretty steep. This rather unfortunate aspect of working with FPGAs means that someone looking to enter the world of FPGA design needs to either undergo extensive training in the work place (which can be quite costly at up to $1500/class), or engage in intense self-motivated learning.

“It Works In Simulation”

HDLs inherently suffer from needing to be written and tested on a different platform than they will be “run” on. In the case of VHDL, it is very easy to write VHDL code that works perfectly fine in simulation but would never work in hardware (loaded into the FPGA). This includes trying to do too much between clock cycles (like trying to perform dozens or hundreds of multiply-and-accumulate (MAC) operations for a finite impulse response (FIR) filter).

The flow of writing and validating HDL is usually something like this:

• Identify the inputs and outputs of the entity
• Identify any algorithms needed to be implemented within the entity
• Produce block diagrams for the different parts of the design
• If applicable, draw out the state machine states and transition diagram
• Write code
• Check syntax
• Simulate
• Verify operation against requirements and datasheets
• Check if it works
  1. If no, go back to #5
• Load into hardware to test
• Check if it works
  1. If no, go back to #5
• Design Review(s)

The problem with this flow is you can get stuck in a vicious cycle of 11a -> 5 over and over until you’ve driven yourself mad. A very big part of being a proficient FPGA designer and programmer is knowing what the code you are writing is actually creating. This means consistently checking the schematic that is generated by your code, and keeping your code as small as possible (this also makes testing easier!).

Additionally, it’s easy to write code that produce massive amounts of logic with a single line. Here is an example:

The above code, generates this schematic:

Vivado created a divider, as requested, but it took almost 1800 cells to do it. Meeting timing on this operation may prove to be very difficult, especially if it is a part of other logic operations that have to happen that clock cycle.

The Ugly

The Tools Aren’t Great

There is nothing quite like the frustration that comes from software tools that don’t do what you expect them to, or produce inconsistent results. Although commercial FPGAs have been around for quite some time, some aspects of the software ecosystems still are lacking. With the latest version of the Xilinx tools, Vivado, the industry has seen a large jump in functionality and features. However, since this tool is brand new and a near-complete rewrite of their legacy tools. It has its own set of challenges and bugs. Altera, the number two FPGA manufacturer, is still working with the same software package that they have been using for over a decade. To put that in perspective, that’s like running Windows ME today thinking that it was appropriate.

Debugging Timing Violations

Timing. Ugh. I’ve probably spent the most time being frustrated with FPGA design when it comes to some rough little bit of a netlist that just refuses to meet timing, no matter what I do to it. These little “show stoppers” come about all of the time. And the best way to squash them is simply with experience – there really isn’t much else that a designer can do. Xilinx has their UltraFast Design Methodology guide, and Altera copious amounts of documentation for “best practices,” but really when it comes down to it, it’s all about writing good, small, synthesizable HDL code. But no matter what, you’re going to find yourself in a situation where you need to instantiate 32 of something, and timing is going to be a nightmare.  If you’d like to chat about your embedded needs, you can reach out to us here.  Or, if you’d like more information on industrial embedded systems, check out our resources page.

Next Steps

If you’d like to chat about your industrial embedded needs, you can reach out to us here.  Also, here’s some additional resources to help you on your journey:

• Custom Automated Test System Buyers Guide
• LabVIEW FPGA: The Good, the Bad, and the Ugly
• Outsourcing Industrial Embedded System Development Guide
• Need LabVIEW FPGA programming help? | LabVIEW FPGA Developers
• LabVIEW Industrial Automation Guide
• Industrial Embedded Design Resources –  White Papers, Case Studies, Articles
• Automated Test System Resources – Webinars, White Papers, Case Studies, Articles

Bevel and cylindrical gears can be found virtually everywhere – from automobiles and airplanes to trucks and tractors, and from giant wind turbines that can power a thousand homes to lawn mowers and power tools.

Gear tooth surfaces and spacing are never perfectly machined, and consequently, noise and vibration are often present in applications where the gears are later used. Gears, after the typical heat treatment process, are commonly lapped or ground to smooth the gear teeth surfaces and improve operational characteristics. This should reduce the surface and tooth spacing deviations that produce noisy gear sets.

The Gleason Corporation and The Gleason Works, a global gears solution provider, enlisted Viewpoint Systems to help create a torque-controlled lapping solution with responsive, real-time feedback to create better quality gears and reduce cycle time for its gear lapping machines.

The result – Viewpoint and Gleason created a patent-pending system that produces higher quality gears in 30 percent less time. The new system, founded on embedded control and dynamic real-time process monitoring technologies, creates the unprecedented ability to improve gearset quality during lapping, and to do so at higher speeds.

Download the case study to learn how Viewpoint Systems developed the solution using NI LabVIEW and NI CompactRIO hardware.

Contact Viewpoint to discuss your application needs.

Getting Data In and Out of the NI-9651 System on Module (SoM)

National Instruments released their latest technology in their line of single-board reconfigurable input/output (sbRIO) products: the NI-9651. The NI-9651 System on Module ( SoM ) is unlike any RIO product that NI has released, and is by far the most powerful.

The NI-9651 takes the Xilinx Zynq-7020 System on Chip ( SoC ) and pairs it with 512MB of memory and 1GB of non-volatile flash, and brings out over 160 I/O to two connectors. The SoM also has a feature that other sbRIO products haven’t had: it runs the Linux kernel.

SoM vs sbRIO

Previous generations of the sbRIO products from NI have included digital and analog inputs, as well as a large amount of the support electronics needed to run the module. I/O such as UART, Ethernet, and USB were included on the module. The NI-9651 is different in that it needs quite a bit more support circuitry to operate. The SoM does not have any connections to it other than two Samtec connectors on the bottom of it; thus, the carrier circuit board that it is mated with must include support circuitry such as:

• Power conditioning and voltage generation
  • The SoM needs a number of different voltages to work correctly.

• IO connectors
  • Ethernet, USB, CAN and UART are all supported by the SoM; however, the various circuits needed to take advantage of these I/O are not included and must be supplied.

• Analog-to-digital converter to get analog signals into LabVIEW
  • You may also need analog front-end circuitry to condition the signal.

• Digital input signal conditioning
  • Even digital inputs usually need to be conditioned in some way, if anything over/under voltage protection should be used.

The Socketed Component-Level Intellectual Property (CLIP)

Once the SoM is in its carrier board or a custom circuit board, the various I/O needed by the system need to be identified and those circuits connected or created. Once that is done, the signals from those circuits and devices must be brought into LabVIEW FPGA and possibly LabVIEW RT. The bridge between the outside world and LabVIEW FPGA is the Socketed CLIP.

Reference carrier board for NI-9651

The Socketed CLIP is written in a hardware description language (HDL), usually VHDL or Verilog. These languages are inferred languages. The interpreted language tells a Xilinx synthesizer information about the logic that it needs to accomplish a task, and then the synthesizer produces the technology-specific configuration to implement the solutions to those tasks. We use VHDL here, as it is very verbose and strongly typed, and thus results in clear, concise code.

Example SoM Block Diagram

Within the Socketed CLIP usually exists the logic and state machines to initialize the various circuits and components that exist on the circuit board attached to the SoM. The Socketed CLIP may also include translations between I/O types such as LVDS and single-ended signals. Additionally, any device-specific IO features, such as delays or serializers, must be implemented in the Socketed CLIP, since LabVIEW FPGA does not know how to work with those technologies.

Finally, once the CLIP has been written and all of the signals have been converted to work with LabVIEW FPGA, the Socketed CLIP can be instantiated into the design, and its I/O can be accessed via LabVIEW FPGA via FPGA I/O node blocks.

Example LabVIEW FPGA code, where data is moving from the CLIP to a target-to-host FIFO.

LabVIEW FPGA is an extremely powerful tool for a LabVIEW programmer to have in their tool belt. One struggle, however, can come when the need arises to interface with a technology that LabVIEW FPGA does not speak natively. This is where using the SoM can be advantageous, and leveraging the power of a traditional HDL such as VHDL can help. The SoM has many great uses. Adding good analog and digital circuit design and a proficiency in HDL can elevate the usefulness of the SoM by quite a bit.  If you’re considering utilizing the NI SOM, start here first. Or, if you’d like more information on industrial embedded systems, check out our resources page.

This article is a continuation of part 1. If you’ve not already read part 1, please start here. As a re-cap, last time we covered the following costs:

1. Requirements
2. Project management
3. Risk mitigation
4. Documentation

The emphasis of this article is on the prototyping of the embedded controller.

Let’s jump right back into it:

Cost 5: Interfacing –

Just always remember that whenever you create a new component, you need to interface with it somehow, and you should allocate time and effort for this. Consider low coupling and open standards whenever feasible.

Here’s why you should care: Interfacing with other components, sub-systems, or systems is becoming more and more prevalent. It offers benefits along the lines of the whole being greater than the sum of the parts. We’ve got a webcast on interfacing to help.

Cost 6: Debugging –

Debugging often has negative connotations associated with it because engineers might think that it means they messed something up that they shouldn’t have, and now they have to fix it in a hurry. This is the wrong way to think about it. While it does generally mean that something was coded, designed, or interfaced incorrectly, these imperfections should be expected.

Here’s why you should care: debugging doesn’t have to be a stressful thing if you are prepared. It can actually be one of the more fun parts of the development process. It’s both a puzzle to solve and a learning experience. I wrote an article to help with the debugging process.

Cost 7: Hardware re-spins –

It is possible to design a custom circuit board well enough on the first shot that a re-spin is not needed; it’s just highly unlikely. I wouldn’t recommend planning for it.

Here’s why you should care: if you plan for a re-spin (or maybe even more depending on your scenario) and you don’t need it, you come out ahead of the game from a cost and schedule standpoint. If you don’t plan on it and you need it, life gets very unhappy very quickly.

Cost 8: Employee performance variation –

People don’t like to talk about this, but not everyone works at the same speed, and sometimes at quite sizable deltas. If you take the average engineer at a given level of skill, I’d say most people work within about +-25%’ish of that range, but I’ve seen people that I’d say performed as much as +-50% faster/slower than that average.

Here’s why you should care: If the project team is large enough (say with ~10 people or more), these variations tend to average themselves out, but with small teams of 2-3 people, the variations can make or break a project. I’d recommend either being aware of who will be on the project team in the planning phase, or if that’s not possible, be conservative with your estimates.

Cost 9: People/team issues –

Many engineers thrive off of solving technical and analytical problems, and are more apt to view people-oriented problems as superfluous. The challenge here is that for problems of any significant complexity, people need to organize into teams, which creates a necessary new class of problems to address.

Here’s why you should care: Friction between team members that don’t get along can create project inefficiencies (generally stemming from a lack of communication) or can even de-rail high-stress moments such as demonstrations.

So what now?

If you’d like more useful info on industrial embedded systems, check out our resources page.  If you’d like help scoping the cost of your industrial embedded controller, you can reach out to us here.  If you don’t have the time or the manpower to develop your own embedded solution, check out VERDI.

The emphasis of this article is on the prototyping of the industrial embedded controller.

When I say “embedded controller” in this context, it doesn’t mean that it actually has to control anything (it could, but isn’t required), but it will most likely have outputs. It could simply be monitoring signals to convey useful information to another piece of the system or system of systems.

This article is geared toward companies in the industrial world (vs consumer), that manufacture systems or sub-systems (machines, equipment) that are generally on the more expensive side (~$4,000 – $200,000 per system), and have lower production volume (10 – 1,000 units per year).

What is out of scope for this discussion?

• The front end research: in other words, an idea already exists, but it hasn’t been prototyped or proven out yet, or if it has, it’s been a lab-equipment-based proof-of-concept hack job.
• Manufacturing costs: while production volume should absolutely be taken into consideration for off-the-shelf vs custom component selection, I won’t focus on the manufacturing end of the product development process; rather I’ll focus on the development costs. Of course, designing with production in mind (not just volumes, but DFA& DFM as well) is still critical.
• The post-production maintenance, obsolescence, and customer support aspects are not part of this discussion.

The industrial embedded development costs that people don’t like to think about:

Estimates for engineering development costs are challenging, and I mean really challenging. The main reason is because something is being created from nothing, and there are a lot of components (software and hardware) that have to work together in harmony (protocols, algorithms, physical interfaces, power consumption, heat dissipation) in order for the embedded controller to perform as desired.

I think the majority of costs that engineers don’t like to think about are areas that are generally considered less fun, but that doesn’t make them any less valuable.

Embedded designers are more apt to want to focus on the creation aspect of the core design (e.g. coding on the software/firmware side, and circuit design on the hardware side), and less likely to want to pay as much attention to costs associated with tasks such as these 9 that I’ve identified below.

Cost 1: Requirements –

Many of the more formal methods for requirements generation and management may be overkill for some industries, and sometimes people get out of hand with requirements, but that’s not a good reason not to do anything at all. Consider developing a minimum set of requirements for your scenario.

Here’s why you should care: Requirements are the core of communicating what your industrial embedded system will become. Often the objective with requirements is getting one person to convey and organize information that’s in their head, with others. Developing and managing requirements isn’t free, but it can cost you even more by not paying attention to them. Not having requirements could create the need for a board re-spin, or could cause processor overloading. Requirements ultimately transform into what you design and test against. What you test against ultimately is what you judge success or failure against.

Cost 2: Project management –

Hardcore developers often hate project management, seeing it as superfluous. On the smallest of projects, project management can be handled individually. Otherwise, at a minimum, it’s a necessary evil.

Here’s why you should care: Project management can make the development team’s life better by helping you:

• not get overloaded
• clear roadblocks
• by providing clear goals.

Cost 3: Risk mitigation –

If nothing ever goes wrong with any of your projects, I’d be grateful if you’d get in touch with me and explain your magic. Otherwise, you should understand the likelihood and severity of your main risk items.

Here’s why you should care: Understanding your main risk items can help you come up with backup plans if a risk item occurs, and at a minimum, can be used as a tool to set expectations with the team, management, and your customer.

Cost 4: Documentation –

There are a few people out there that enjoy documenting. For the rest of us that don’t enjoy it, it’s worth noting that it can get out of hand. Just documenting because you’re supposed to, is not a very good reason in my mind. Keep it to a minimum.

Here’s why you should care: The three most helpful reasons to document in my mind are:

• to help you design in the first place
• to help teach someone else whatever they need to know
• to help you remember what you did a year or more later.

If none of these criteria are met, second-guess your effort.

Where you might head from here:

Check out part 2 of this article where we cover the other 5 costs that people don’t like to think about when designing an embedded controller. If you’d like more useful info on industrial embedded systems, check out our resources page.  If you’d like help scoping the cost of your industrial embedded controller, you can reach out to us here.

The Dilemma

Dare I say that every product designer today stumbles on a single big step when building a new product, refurbishing an obsolete product, or enhancing an existing product? Here’s the big step: should the controller be constructed from off-the-shelf (OTS) components or from a custom design?

Finding the answer is complex and no one path will apply to everyone. But reviewing some specific questions will alleviate some of the confusion in making a choice. These are the questions I ask our clients when they are looking for guidance on possible directions.

Preparation

Before being able to answer these path-defining questions, prepare yourself by knowing the following items. Don’t spend more than a few days or maybe a week to get this information. If you do, you will be feeding precise information into fuzzy considerations, which will likely cause you to rework that precise information. Here’s the list.

1. Is the controller for a new device or an upgrade to an existing device?
2. Annual quantities. I want to see quantities for each specific product model, even if, for example, model X is only a slight variation of model Y. Better yet if you know projected annual for the next three to five years.
3. A list of required I/O for each model. I want to see the following:
   • For each input, sensor types or levels and type (DC or AC) for voltage and current.
   • For each output, actuator types or levels and type (DC or AC) for voltage and current.
   • For each I/O, need for isolation.
   • For each I/O, connectivity requirements.
   • For each I/O, sample data rates and resolutions.
4. Operating environment, such as temperature range and shock and vibe levels.
5. Restrictions on physical size and weight. I would want to know if the controller needs to fit in a tight location or be less than a certain weight.
6. Certifications required, such as UL, CSA, and CE Mark. Obtaining each one of these certifications costs money, so if you don’t plan to sell to Europe (yet), then I’d like to know.
7. Software development environment. I want to understand which SW dev tools you plan to use, and how familiar you are with them.

Important Considerations

With the design preparations in the prior section, you can start on some cost comparisons.

Note that commonalities exist within each industry. For example, in the energy power monitoring sector, the I/O is often tailored around 3-phase synchronous digitizing. In industrial machines used in manufacturing and testing, force and proximity sensors are often used, sometimes with synchronous digitizing as well (such as might occur in monitoring a CNC metal cutting machine or a cold head force machine).

When commonalities exist in an industry, OTS solutions can be more available than niche or new markets, for the simple reason that a marketplace already exists.

So, here are the considerations I look at when helping clients make decisions about the OTS versus custom, also called the buy versus build choice. These considerations aim at the custom choice, since you can simply ask the vendor of the OTS product for the comparable information.

• Build a cost model for the cost per unit.
• If there are multiple similar models, compare the differences in I/O, because it may be cheaper to build a unit to handle the “max” configuration in higher quantities than multiple “cheaper” but slightly different units.
• Review warranty handling to understand repair versus replace costs.
• Understand enclosure, mounting, and connection needs.

And, of course, you need to determine if an OTS solution even exists with the specifications you need at a price that is reasonable. Even if no OTS path exists, reviewing the considerations above will help you refine the custom path.

For more in-depth information about COTS vs. custom designs for embedded systems, check out our white paper: Comparing Off-The-Shelf to Custom Designs for Industrial Embedded Systems.

Gotcha #1:  Thinking that you’re writing software

Get out of the software mindset – You’re not writing software.

Let me say that again because this is the single most important point if you’re working with FPGAs.  You-are-NOT-writing-software.  You’re designing a digital circuit.  You’re using code to tell the chip how to configure itself.  Now, before someone says it, yes, when you’re coding up one of the microprocessor cores within the FPGA, then of course you’re writing software, but that’s not what we’re talking about here.  We’re talking about when you’re coding the digital logic.

Gotcha #9: Trying to use all of the FPGA!

In general, you want to use less than 100% of the part.  The amount that you’ll want to keep in reserve depends on several factors, with significant factors including:

• The generation/architecture of FPGA being utilized
• Your code quality
• The FPGA clock rate

The fuller your part is, the longer the build process (compile, synthesis, place, route, etc.) will take.  The reason this is a useful note, is that build times generally don’t take seconds.  Depending on many factors, they can take 20 minutes, or four hours.  In some cases it may fail to build at all, or worse yet, build, but be unstable, wasting countless hours in the lab.

Gotcha #10: Not planning for Enough Bugs

Something about that feels wrong on first read, because who actually plans for bugs?  You should.  If you do it will be one of the most stress-relieving things you do, and you’ll thank me later.  You’ll get many of the most basic level 1 bugs out during simulation, but at some point there will be a driving reason where you’ll have to move out to the lab and integrate your part of the world with the rest of the system being developed.  This is where those famous words “But it worked fine in simulation…” come into play.

The disadvantage of a run-of-the-mill basic sequential processor is that only one operation is executing at any point in time. The advantage of a sequential processor is that only one operation is executing at any point in time. When it comes to debugging, the sequential nature of the basic processor makes life easier than debugging the parallel nature of the FPGA.  While the simulation environment offered impressive visibility, now that you’re moving into hardware, your visibility into what’s going on decreases significantly.

Plan for a lot of bugs.  Take a couple hours and think through other bugs you’ve dealt with as an engineer.  Now triple that number (I’m making this multiplier up of course).  If you’re a newbie developer, you need to pull in someone that has experience with FPGA development to help with this estimate.  You’ll be wrong, but you’ll be better off than if you hadn’t thought this through.  Here are a few tips on debugging to help you along the way:  Six debugging techniques for embedded system development.

Intermittent bugs are common within FPGAs.  They can be affected by temperature, humidity, the phase of the moon (I may be exaggerating a bit there, but sometimes I wonder).

The logic analyzer is your eyes into the inner workings of the FPGA. You can view outputs of course, and you can create test points to go to a testpoint header, but you have to be able to probe all of those points, so you’ll probably need a logic analyzer, which can get very pricey (if you’re looking for an inexpensive logic analyzer, check this out: https://www.saleae.com/ , and similar).  A logic analyzer is a very important tool for FPGA-based development.

An internal logic analyzer can be helpful as well, at least in some scenarios.  You may be able to get away with embedding test resources into your device (e.g. Xilinx has the Integrated Logic Analyzer), but this will utilize FPGA logic and memory resources, and is often challenging to see enough breadth or depth about what’s going on inside your FPGA. However, generally these tools are better for augmenting a true logic analyzer, as opposed to replacing them outright.

Gotcha #11: Making fun of the software guy!

The software guy (or girl of course; the term “guy” is used here androgynously) is your best friend – If your FPGA interfaces to a higher level supervisory sort of processor and/or it provides the UI, the software guy can do things to help make your life a lot easier.  A couple of the more major categories include:

• Engineering debug panels – chances are the information you need to view is not the same as what the end-user needs to view. Having a debug panel can save you significant time, effort, and frustration.
• Special debug functions and modes – maybe there is some routine that software normally runs through with steps 1-12, but maybe you want to be able to run just step 3 and 4 repeatedly, or maybe just step 7 once. Or perhaps software can add a special function to keep track of the content or frequency of messages that you’re sending to it and set a trigger when something unexpected happens.

Work with the software developer early on to see how you can work together to facilitate the integration and debug process, and remember that it goes both ways.  Chances are there is additional functionality that you could add to aid in their debug process as well.

Next Steps:

If you like these gotchas and want to see the others, read the white paper here:

David LaVine

David is the Marketing Manager & Solutions Architect at Viewpoint. David has a BS in Electrical Engineering from Rensselaer Polytechnic Institute and an MS in Electrical and Computer Engineering from Georgia Tech. He’s been at Viewpoint Systems since 2013.

6 Considerations when Choosing an NI RIO for Embedded Systems

I’d like to share with you the results of a conversation I had at a recent meeting with a long-time client. They wanted to talk about developing a controller for measuring and controlling some aspects of their product. For me, it was a great example of the struggles in development teams when starting a new project using modern tools and components for an embedded system.

Here is some background.

This client has a good, but small, team of engineers who have been supporting various data collection and control systems for many years for support of their (somewhat expensive) machines in the field. (Let’s give this machine a name so we don’t get confused about all the talk of machinery. Let’s use BEM for big expensive machine.)

These engineers have many years of experience putting subsystems in place to monitor the BEMs after delivery to their customers in the field. Some of these subsystems are getting old and obsolete and need to be upgraded, while some are new ideas. The issue is that they, like many other companies, are trying to make their BEMs smarter by attaching new and better sensors to the BEMs and giving the subsystems that use these sensors the ability to control and/or sense the monitoring conditions. The overall goal is to get better information about the BEM performance. It’s actually fancy condition monitoring.

The buzz about the Industrial Internet of Things (IIoT) has these engineers wondering about all those small and cheap embedded processors available today. Heck, you can buy a wireless sensor and cloud data aggregator to monitor your home for water heater leakage and air temperatures for cheap (say about $200 upfront and $100/year)! These engineers are wondering how close they can get to that cost.

So, I visited them to discuss the upgrade of an existing PLC-based subsystem. Let’s call this the OES for Old Existing Subsystem.

They like the NI (National Instruments) platform, and want to start using NI hardware to replace the several types of OESs as well as design new controllers.

During the discussions, we talked about the tradeoffs between the various NI HW RIO options, CompactRIO (cRIO), the sbRIO, and the SoM RIO (the most recent platform), as well as other non-NI options, such as PLCs, Arduino, and so on.

I realized that many other companies must be in the same situation, so I wanted to share some of the important issues that came out of the conversation.

1. Don’t go too cheap
2. Only need small volumes of about 100 units per year
3. Need to balance prototype costs with deployment costs
4. Want the units to be supportable for at least 10 years
5. Engineering resources are hard to find to design and build the new controller
6. Need to support the units in the field for a long time

Some details …

Too Cheap is Not Good

Despite all you hear about cloud-based devices, in an industrial situation, you don’t want to get too cheap on the embedded system. Usually performance specs drive costs and typical “consumer grade” devices won’t have the required operating specs for temperature, measurement noise, shock & vib, and so on.

In this example, the BEM operates in an environment MUCH hotter than the usual office and surrounding equipment can cause vibrations. Furthermore, the measurements need to be accurate to 1% or better, so good quality data acquisition hardware is needed. The NI RIO HW is often a better choice.

Engineering Resources Are Crunched

In the past two years, I’ve seen engineering staffs being increasingly slammed with work. Furthermore, a lot of engineering talent is retiring every year. Companies are turning to outside engineering firms for design and build while maintaining their core capabilities in-house. If you do plan on using an outside firm, do your homework and find a good fit. Bad results can occur if the project is not defined clearly up-front, so be prepared to do some work. Check out this Dilbert comic:

Small Production Volumes

The processors available today for embedded systems are way better than ever before in price per performance. But, often these processors are aimed at consumer markets which sell millions of units per year. For small volumes of a 100s to 1000s per year, a typical (say with 80% coverage) embedded system using NI RIO technology will cost between $500 and $2000 per unit. It’s not yet possible to reach that 100 $/unit mark.

Even with retrofitting all the existing BEMs, the client could not build more than about a couple 100 units per year between new BEMs and retrofitting old ones (they just don’t have enough people to retrofit every BEM in one year).

Support for How Many Years?

Often, the new embedded system is replacing that OES (Old Existing Subsystem) that has been around for way over a decade, perhaps 20 or 25 years. That duration is unlikely now without extraordinary measures to counteract today’s typical subcomponent life cycle of around a maximum of 10 years. I think a lot of this shorter duration is due to the rapid design cycles in electronics. I’m guessing that the photolithography technology used to make electronic components 10 years ago is just not easily accessible anymore!

Tradeoff of Prototyping versus Production Costs

If you are unsure how many units you can sell, then prototype the first units with as much off the shelf stuff as possible. This approach keeps NRE (non-recurring engineering) costs as low as possible but unit costs will be higher than possible. If you know that you can sell at least say 25 units, then consider a using as much custom stuff as possible. This approach has higher NRE but unit costs can be reduced.

The graph below illustrates the two approaches, with the solid line being the off-the-shelf approach and the dotted line being the custom approach.

The solid line would be representative of a cRIO approach, while the dotted line would describe the sbRIO or SoM RIO.

Note that hybrid approaches exist. For example, using an already-designed analog input (AI) card designed to work with the SoM RIO can get you closer to the end system than having to wait for the design and build of custom AI circuitry. This approach enables you to prototype your embedded system with modular pre-built parts.

We Have 100s of Units Deployed. Help!!

Often, our clients are very focused on those first few units which will be deployed in a sort of “beta test” program. The thought of having to support 100s in the field is far off in the future. Yet, this support is often the hardest part of an embedded project.

In the case of our BEM manufacturing client, they already know how to support equipment in the field. But with this new embedded controller, they want to use the Ethernet connectivity for remote activities such as controller diagnostics, data downloads, application upgrades, and so on. These activities take time away from future development work and interfere with typical engineering work.

Next Steps

Many companies are looking at ways to embed smarts into their machines. When the total number of units is expected to approach around at least 25, it makes sense to look at a custom-built embedded controller based on the sbRIO or SoM RIO (check out our SOM development platform). I recommend that you prototype with hardware as close as possible to that used in the finished controller while your custom circuit board is developed in parallel. If you are not able to justify the extra NRE (non-recurring engineering) for 25 units, then use off-the-shelf hardware, such as the cRIO (check out our cRIO expertise).  If you’d like our thoughts on your scenario, you can reach out to us here.  If you’d like more info on the world of industrial embedded, check out our resources page.