Do You Need a File System?

Designing an embedded device at the IoT’s edge? If you want to store data or have the option of updating the device firmware in the field, you may be asking yourself, “do we need a file system?”

For this article, I am going to focus on flash media, as that is the most common storage for embedded systems. This sort of media can be in a raw format (NAND or NOR flash) or packaged in some form (eMMC, SD media or CompactFlash). Flash media drivers will need to take care of error detection, correction and bad block management, resulting in a simple block-based storage.

As a start, this storage media can be thought of as a memory pool. The flash media driver or firmware will handle the wear leveling, meaning the media can be addressed directly (write to block 1, read from block 2, etc.) without exceeding the erase limits in a particular section.

Figure 1:  A boundary you don’t want to cross (or be pushed over) is the one keeping BOM expenses in line. These could rise if you are constrained by a file system package you cannot alter or tailor to fit your design requirements.

Figure 1: A boundary you don’t want to cross (or be pushed over) is the one keeping BOM expenses in line. These could rise if you are constrained by a file system package you cannot alter or tailor to fit your design requirements.

Early implementations of PC BIOS actually used CMOS SRAM to store the BIOS settings, though it was not directly accessible as a memory pool. Later BIOS implementations utilized flash media as a standard storage for both the BIOS settings and the BIOS code itself.

Message logging, either debug or runtime, is a common use for storage media. An application can write to a log area by keeping track of a current log pointer and wrapping when it reaches the end. These routines should be as solidly tested as the rest of the embedded device, of course.

Complexity Considerations

If more than one log area needs to be written, two pointers could be maintained, with a corresponding increase in testing complexity. If each log area will be written from a different thread, even more overhead code will have to be added, because most flash memory doesn’t allow for two simultaneous writes.

Another use for storage media in an embedded device is to keep configuration settings. Assuming these can be changed at runtime, the added complexity of handling a power interruption is introduced. Two copies of the settings can be kept, with a runtime update overwriting the older copy. This code also needs to be tested thoroughly—a failure here can result in lost settings or even a bricked device.

The Case for a File System

Anything more complex than the examples cited earlier are probably best handled through a file system. File systems can handle multiple files, multiple threads and power interruption. And using a file system gives access to added information from metadata—file creation and/or access date and time, file size and attributes.

File attributes allow some control over who has access to a file, allowing a device to have multiple users and some basic security. Beyond the basic FAT file system are options such as Datalight’s Reliance Nitro, with full OEM attributes and support for Linux style groups and permissions.

Finally, a file system is pretty much required if your embedded device can be connected to a desktop or workstation. These larger machines access all their data through file systems or abstractions. The alternative is a custom driver to both copy down a media image and parse it into meaningful data packets. These could subsequently be stored in files or inserted into a local database.

Avoiding Higher BOM Costs

More important than the addition of all these features, however, is the ability to offload the additional testing to a third party. The file system developer can test more thoroughly and provide a solid base, allowing your testing crew to stick to what they know—testing your particular design and application.

Of course, file system functionality also takes additional resources from your design. Working with a file system package that can’t be modified may push your resources over a boundary and increase your bill of materials cost (Figure 1). A better alternative is a file system that can be easily modified or adapted to provide just what you need for the design, and no more.

If you are going to store data on the media, you either have to create elements from a file system (and tests!) or use one that is available off-the-shelf. Choosing the latter option allows you to reduce your QA needs, with the possible cost of unused functionality. The ability to modify a file system to suit your needs (for size, performance, and reliability) is very important. That alone could save you more time and effort than even writing a basic logging alternative to a file system.
Untitled-1Thom Denholm is a technical product manager at Datalight. He is an embedded software engineer with more than 20 years of experience, combining a strong focus on operating system and fie system internals with a knowledge of modern flash devices. Thom holds a BS in Mathematics and Computer Science from Gonzaga University. His love for solving difficult technical problems has served him well in his fifteen years with Datalight. In his spare time, he works as a professional baseball umpire and an Internet librarian. Though he has lived in and around Seattle all his life, he has never had a cup of coffee.