The Importance of Using Structure Over Prompts when using LLMs in Embedded Development.
AI in software development is a hot topic in today’s media. Most of this buzz is written from the perspective of web design or application teams, not embedded design. This makes it hard to gauge the usefulness of AI in other fields, where mistakes can be more costly, like they are in embedded development.
In embedded development we deal with hardware constraints and real-time behavior, along with fragile toolchains and delicate systems that may fail in such a way as to be hard to observe directly. In that design environment, large language models can be very helpful and a useful tool. However, they can also be genuinely dangerous. It all depends on how we use them.
Over the past year, I have used LLMs in embedded development in several real projects. I had to meet deadlines and work with real customers buying the finished products. These aren’t experiments or demos; they are actual firmware that must build, boot, and behave correctly. In this post I will cover what I have learned about using LLMs in embedded design; what has worked for me, as well as what hasn’t worked. I want to focus specifically on why the difference most often came down to structure and discipline rather than clever prompts.
The Baseline: AI is a Tool, not a Replacement for Actual Engineering
My primary experience is with the LLM plugin that integrates with VS Code. This model can read and operate directly on my active project. The integration of that plugin is solid, as it is on most modern ESP-class platforms, which means the LLM can stay directly anchored to the system I am working in and still build the code I need or help reason through compiler outputs. There is no confusion from a “middleman” program or loss in translation between systems.
Unfortunately, such strong integration is not always possible, and in those instances, I have had to use AI-assisted editors alongside the vendor IDE I am using. While this can and does work, it is much more fragile, because the further the model is from the real build system, the more it guesses.
This leads me to my first hard rule of working with AI in Embedded design:
An LLM is only useful in embedded work if it is constrained within the same process discipline you would enforce on a junior engineer.
Without enforcing such constraints, a LLM will happily generate lots of plausible, but ultimately useless, outputs that do not survive contact with the toolchain.
LLMs Always Need Explicit Ground Rules
I had to make some big changes to how I prompted AI in order to make the AI work for me how I wanted it to. One of the biggest changes I made was to create a formula for starting each AI session.
Opening a blank chat and asking for code does not provide the LLM enough structure to properly assist me on a project. I had to adapt, and now I begin each session with a structured set of instructions. The model needs to know:
How and what prior context must be reviewed and considered
Which documents are authoritative
The fixed platforms, SDK versions, and targets
What actions are mandatory after every code change, and which are not
Where in the project changes are not allowed without approval
These strict rules and structure aren’t there simply to micromanage the model. They do help remove ambiguity and focus the answers the AI gives. With the number of moving parts already inherent to Embedded systems, we need to eliminate as many options as possible to prevent the AI from inventing APIs or skipping builds.
Making this shift in the way I prompt an AI to code for me led to a very simple, yet very critical, insight.
Prompts are not the unit of control. Contracts are.
By forcing the AI to agree to operate within the strict boundaries and control parameters I have given it, I am eliminating the ambiguity and uncertainty of how the model will interpret each prompt I give it in a session.
Planning First, Code Second, Always
I almost never ask an LLM to implement a feature directly.
The first step is always a planning artifact.
To facilitate this, I describe the problem we are solving at a functional level, then ask for a structured plan from the AI model. After it produces a plan for me, I carefully review that plan, and iterate if necessary, until it matches the way the project system actually works, rather than how the model wishes it worked.
When we move on to execution and implementation, I make sure to break everything down into distinct, testable steps. The criteria I look for when creating these steps are as follows.
Each step must:
Compile on its own
Be verifiable without side effects
Have clear acceptance criteria
This process closely mirrors the process I use to approach complex embedded design challenges without an LLM. The only difference is the speed at which the AI can do everything. By rapidly drafting, revising, and tracking plans and details across files, AI increases efficiency, but only if the necessary structure is enforced properly.
Build Checks Are Mandatory
One of the earliest difficulties I came across when working with AI was fairly predictable, but no less aggravating.
The model would tell me a task was complete, but then the code would not compile. There could be any number of reasons why, including incorrect prototypes or subtle API mismatches, but the result was the same. the model was confidently wrong about the viability of its code.
Luckily, the fix was simple and straightforward:
After every code change, a build must be run before the model is allowed to claim success.
This simple check drastically improved output quality. The model must be able to run the build itself, and if it can’t, it must explain exactly why, and which command needs to be run instead. While this rule did not eliminate the actual logic bugs, it easily filtered out many of the careless errors the LLM was making. In embedded work, that is more than worth the overhead.
Using Instrumentation as a First-Class Tool
While LLMs are useful in creating code, where I have found their greatest utility to lie is in the realm of instrumentation and debugging.
By adding logs, counters, state trackers, and health reporting systems to my embedded systems, AI can quickly and easily remove a time consuming but essential part of my design process workload.
Rather than having to manually code those solutions and tools myself, I can instead describe the failure mode to the model and ask it to instrument a system that exposes the signals I need. These can include:
Platform-specific diagnostics
Tracking resource and memory usage
Creating structured logs designed for later analysis
This functionality is especially effective for issues that involve memory behavior, task health, or communication failures. The model doesn’t have to invent a new debugging method for any of these systems, it just automates and speeds up the system I already trust.
Self-Analysis of Its Own Evidence
Instrumentation only helps if you can interpret the output.
Embedded logs are long and repetitive. This makes spotting data trends difficult, even with filtering.
Luckily, LLMs are very good at spotting trends. I have had consistent success feeding logs back into the model and asking for analysis. The model produced the instrumentation, so it understands what each signal means and how it relates to everything else in the system.
I recently wrote a follow-up post focused specifically on debugging, because this is one of the areas where LLMs have been most useful in practice. If you want a deeper look at how I use them to add instrumentation, analyze long logs, and tighten the path to root cause, read Using LLMs for Embedded Debugging to Shorten the Debug Loop.
I typically ask things like:
Is this resource steadily leaking or oscillating?
Did behavior change after a specific event?
How failures correlate with timing, load, or state transitions?
All of these questions are what I would normally ask myself when checking logs. The difference is speed. An LLM can scan tens of thousands of log lines quickly and efficiently, much faster than any human.
External Documents Are the Real Memory
A recurring pattern in my workflow is heavy reliance on external documents.
I use architecture summaries, troubleshooting notes, interface descriptions, and rolling session logs to preserve context between sessions. By requiring the model to reference the same external, unchanging documents each time I prompt it, we maintain consistency.
One particularly important artifact is a session handoff document. It captures:
Where it made changes
Which commands we ran
What worked and what did not
Any next steps that need to be taken
I require the model to review that handoff at the start of a session, before doing anything else.
This compensates for the lack of long-term chat memory and enforces a habit that benefits human engineers just as much as it benefits the LLM.
Code Review, Documentation, and Sanity Checks
While LLMs are effective assistants, I never treat them as authoritative reviewers. The use cases for an AI model are usually fairly narrow.
They are good at:
Flagging missing cases or unchecked errors
Spotting inconsistent naming or formatting
Identifying comments that no longer match the code
Producing plain-language summaries of complex functions
It is fairly common that I ask the LLM to explain a block of code back to me. I know I have to dig deeper if the model can’t describe the code properly, or if the description doesn’t match my intent.
I also regularly have the model update architecture or design notes after changes. This ensures the documentation stays up to date with the design process.
Testing Beyond the Device Itself
There are many uses for on-device unit testing, and LLMs are good at generating tests quickly. However, the larger payoff often occurs when we add host-side tooling to the mix.
Embedded systems almost always need external tools to help with configuration, command injection, and automated validation. These extremely important tools are often underdeveloped, which is where AI comes in.
LLMs are very good at accelerating this kind of work. Simply give the model a command definition or protocol description, and it will generate useable host-side testing tools. These tools enable faster iteration and validation. I have even had the model derive a protocol description from embedded code and then implement the corresponding host logic from that description alone.
Using AI in this capacity removes a common bottleneck and keeps focus on the embedded logic itself.
How I Actually Got AI to Work for Me
After a lot of trial and error, a few principles stand out:
While LLMs are powerful, they must be tightly constrained
Providing structure is better than clever prompts, every time
Step-by-step plans are critical
Mandatory builds and tests are essential
Referencing external documents helps maintain consistency.
When used casually, AI is a decent tool, and it can produce a lot of output, but that output is usually of little value. However, when used deliberately, AI can significantly improve productivity and throughput within the narrow applications it excels at. In my own personal experience, overall productivity has at least doubled, and code production has increased even more.
The important thing to note is that these gains do not come from replacing engineering judgment. The benefits come from removing the friction of repetitive, mechanical work. The kind of work that seems mind-numbing and tedious to a human engineer does not have the same effect on an LLM.
Why This Matters for Embedded Teams
When you are building embedded products, mistakes can be costly. You have to worry about deadlines, cost, reliability, and compliance, and the margin for error is thin. The workflow I have described here is not about chasing AI trends, but rather about augmenting existing engineering discipline with new tools, used properly.
This is the same mindset I bring to client work at Embedded Engineering Solutions, whether that is firmware development, system architecture, or test strategy for embedded firmware development services.
If you are evaluating how AI-assisted workflows might fit into your embedded development process, or where they might introduce risk, feel free to reach out or schedule a call. These are decisions worth making carefully, and we are happy to help.
