--- /dev/null
+# Copyright (c) 2016 Google, Inc
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+
+Introduction
+------------
+
+Firmware often consists of several components which must be packaged together.
+For example, we may have SPL, U-Boot, a device tree and an environment area
+grouped together and placed in MMC flash. When the system starts, it must be
+able to find these pieces.
+
+So far U-Boot has not provided a way to handle creating such images in a
+general way. Each SoC does what it needs to build an image, often packing or
+concatenating images in the U-Boot build system.
+
+Binman aims to provide a mechanism for building images, from simple
+SPL + U-Boot combinations, to more complex arrangements with many parts.
+
+
+What it does
+------------
+
+Binman reads your board's device tree and finds a node which describes the
+required image layout. It uses this to work out what to place where. The
+output file normally contains the device tree, so it is in principle possible
+to read an image and extract its constituent parts.
+
+
+Features
+--------
+
+So far binman is pretty simple. It supports binary blobs, such as 'u-boot',
+'spl' and 'fdt'. It supports empty entries (such as setting to 0xff). It can
+place entries at a fixed location in the image, or fit them together with
+suitable padding and alignment. It provides a way to process binaries before
+they are included, by adding a Python plug-in. The device tree is available
+to U-Boot at run-time so that the images can be interpreted.
+
+Binman does not yet update the device tree with the final location of
+everything when it is done. A simple C structure could be generated for
+constrained environments like SPL (using dtoc) but this is also not
+implemented.
+
+Binman can also support incorporating filesystems in the image if required.
+For example x86 platforms may use CBFS in some cases.
+
+Binman is intended for use with U-Boot but is designed to be general enough
+to be useful in other image-packaging situations.
+
+
+Motivation
+----------
+
+Packaging of firmware is quite a different task from building the various
+parts. In many cases the various binaries which go into the image come from
+separate build systems. For example, ARM Trusted Firmware is used on ARMv8
+devices but is not built in the U-Boot tree. If a Linux kernel is included
+in the firmware image, it is built elsewhere.
+
+It is of course possible to add more and more build rules to the U-Boot
+build system to cover these cases. It can shell out to other Makefiles and
+build scripts. But it seems better to create a clear divide between building
+software and packaging it.
+
+At present this is handled by manual instructions, different for each board,
+on how to create images that will boot. By turning these instructions into a
+standard format, we can support making valid images for any board without
+manual effort, lots of READMEs, etc.
+
+Benefits:
+- Each binary can have its own build system and tool chain without creating
+any dependencies between them
+- Avoids the need for a single-shot build: individual parts can be updated
+and brought in as needed
+- Provides for a standard image description available in the build and at
+run-time
+- SoC-specific image-signing tools can be accomodated
+- Avoids cluttering the U-Boot build system with image-building code
+- The image description is automatically available at run-time in U-Boot,
+SPL. It can be made available to other software also
+- The image description is easily readable (it's a text file in device-tree
+format) and permits flexible packing of binaries
+
+
+Terminology
+-----------
+
+Binman uses the following terms:
+
+- image - an output file containing a firmware image
+- binary - an input binary that goes into the image
+
+
+Relationship to FIT
+-------------------
+
+FIT is U-Boot's official image format. It supports multiple binaries with
+load / execution addresses, compression. It also supports verification
+through hashing and RSA signatures.
+
+FIT was originally designed to support booting a Linux kernel (with an
+optional ramdisk) and device tree chosen from various options in the FIT.
+Now that U-Boot supports configuration via device tree, it is possible to
+load U-Boot from a FIT, with the device tree chosen by SPL.
+
+Binman considers FIT to be one of the binaries it can place in the image.
+
+Where possible it is best to put as much as possible in the FIT, with binman
+used to deal with cases not covered by FIT. Examples include initial
+execution (since FIT itself does not have an executable header) and dealing
+with device boundaries, such as the read-only/read-write separation in SPI
+flash.
+
+For U-Boot, binman should not be used to create ad-hoc images in place of
+FIT.
+
+
+Relationship to mkimage
+-----------------------
+
+The mkimage tool provides a means to create a FIT. Traditionally it has
+needed an image description file: a device tree, like binman, but in a
+different format. More recently it has started to support a '-f auto' mode
+which can generate that automatically.
+
+More relevant to binman, mkimage also permits creation of many SoC-specific
+image types. These can be listed by running 'mkimage -T list'. Examples
+include 'rksd', the Rockchip SD/MMC boot format. The mkimage tool is often
+called from the U-Boot build system for this reason.
+
+Binman considers the output files created by mkimage to be binary blobs
+which it can place in an image. Binman does not replace the mkimage tool or
+this purpose. It would be possible in some situtions to create a new entry
+type for the images in mkimage, but this would not add functionality. It
+seems better to use the mkiamge tool to generate binaries and avoid blurring
+the boundaries between building input files (mkimage) and packaging then
+into a final image (binman).
+
+
+Example use of binman in U-Boot
+-------------------------------
+
+Binman aims to replace some of the ad-hoc image creation in the U-Boot
+build system.
+
+Consider sunxi. It has the following steps:
+
+1. It uses a custom mksunxiboot tool to build an SPL image called
+sunxi-spl.bin. This should probably move into mkimage.
+
+2. It uses mkimage to package U-Boot into a legacy image file (so that it can
+hold the load and execution address) called u-boot.img.
+
+3. It builds a final output image called u-boot-sunxi-with-spl.bin which
+consists of sunxi-spl.bin, some padding and u-boot.img.
+
+Binman is intended to replace the last step. The U-Boot build system builds
+u-boot.bin and sunxi-spl.bin. Binman can then take over creation of
+sunxi-spl.bin (by calling mksunxiboot, or hopefully one day mkimage). In any
+case, it would then create the image from the component parts.
+
+This simplifies the U-Boot Makefile somewhat, since various pieces of logic
+can be replaced by a call to binman.
+
+
+Example use of binman for x86
+-----------------------------
+
+In most cases x86 images have a lot of binary blobs, 'black-box' code
+provided by Intel which must be run for the platform to work. Typically
+these blobs are not relocatable and must be placed at fixed areas in the
+firmare image.
+
+Currently this is handled by ifdtool, which places microcode, FSP, MRC, VGA
+BIOS, reference code and Intel ME binaries into a u-boot.rom file.
+
+Binman is intended to replace all of this, with ifdtool left to handle only
+the configuration of the Intel-format descriptor.
+
+
+Running binman
+--------------
+
+Type:
+
+ binman -b <board_name>
+
+to build an image for a board. The board name is the same name used when
+configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox').
+Binman assumes that the input files for the build are in ../b/<board_name>.
+
+Or you can specify this explicitly:
+
+ binman -I <build_path>
+
+where <build_path> is the build directory containing the output of the U-Boot
+build.
+
+(Future work will make this more configurable)
+
+In either case, binman picks up the device tree file (u-boot.dtb) and looks
+for its instructions in the 'binman' node.
+
+Binman has a few other options which you can see by running 'binman -h'.
+
+
+Image description format
+------------------------
+
+The binman node is called 'binman'. An example image description is shown
+below:
+
+ binman {
+ filename = "u-boot-sunxi-with-spl.bin";
+ pad-byte = <0xff>;
+ blob {
+ filename = "spl/sunxi-spl.bin";
+ };
+ u-boot {
+ pos = <CONFIG_SPL_PAD_TO>;
+ };
+ };
+
+
+This requests binman to create an image file called u-boot-sunxi-with-spl.bin
+consisting of a specially formatted SPL (spl/sunxi-spl.bin, built by the
+normal U-Boot Makefile), some 0xff padding, and a U-Boot legacy image. The
+padding comes from the fact that the second binary is placed at
+CONFIG_SPL_PAD_TO. If that line were omitted then the U-Boot binary would
+immediately follow the SPL binary.
+
+The binman node describes an image. The sub-nodes describe entries in the
+image. Each entry represents a region within the overall image. The name of
+the entry (blob, u-boot) tells binman what to put there. For 'blob' we must
+provide a filename. For 'u-boot', binman knows that this means 'u-boot.bin'.
+
+Entries are normally placed into the image sequentially, one after the other.
+The image size is the total size of all entries. As you can see, you can
+specify the start position of an entry using the 'pos' property.
+
+Note that due to a device tree requirement, all entries must have a unique
+name. If you want to put the same binary in the image multiple times, you can
+use any unique name, with the 'type' property providing the type.
+
+The attributes supported for entries are described below.
+
+pos:
+ This sets the position of an entry within the image. The first byte
+ of the image is normally at position 0. If 'pos' is not provided,
+ binman sets it to the end of the previous region, or the start of
+ the image's entry area (normally 0) if there is no previous region.
+
+align:
+ This sets the alignment of the entry. The entry position is adjusted
+ so that the entry starts on an aligned boundary within the image. For
+ example 'align = <16>' means that the entry will start on a 16-byte
+ boundary. Alignment shold be a power of 2. If 'align' is not
+ provided, no alignment is performed.
+
+size:
+ This sets the size of the entry. The contents will be padded out to
+ this size. If this is not provided, it will be set to the size of the
+ contents.
+
+pad-before:
+ Padding before the contents of the entry. Normally this is 0, meaning
+ that the contents start at the beginning of the entry. This can be
+ offset the entry contents a little. Defaults to 0.
+
+pad-after:
+ Padding after the contents of the entry. Normally this is 0, meaning
+ that the entry ends at the last byte of content (unless adjusted by
+ other properties). This allows room to be created in the image for
+ this entry to expand later. Defaults to 0.
+
+align-size:
+ This sets the alignment of the entry size. For example, to ensure
+ that the size of an entry is a multiple of 64 bytes, set this to 64.
+ If 'align-size' is not provided, no alignment is performed.
+
+align-end:
+ This sets the alignment of the end of an entry. Some entries require
+ that they end on an alignment boundary, regardless of where they
+ start. If 'align-end' is not provided, no alignment is performed.
+
+ Note: This is not yet implemented in binman.
+
+filename:
+ For 'blob' types this provides the filename containing the binary to
+ put into the entry. If binman knows about the entry type (like
+ u-boot-bin), then there is no need to specify this.
+
+type:
+ Sets the type of an entry. This defaults to the entry name, but it is
+ possible to use any name, and then add (for example) 'type = "u-boot"'
+ to specify the type.
+
+
+The attributes supported for images are described below. Several are similar
+to those for entries.
+
+size:
+ Sets the image size in bytes, for example 'size = <0x100000>' for a
+ 1MB image.
+
+align-size:
+ This sets the alignment of the image size. For example, to ensure
+ that the image ends on a 512-byte boundary, use 'align-size = <512>'.
+ If 'align-size' is not provided, no alignment is performed.
+
+pad-before:
+ This sets the padding before the image entries. The first entry will
+ be positionad after the padding. This defaults to 0.
+
+pad-after:
+ This sets the padding after the image entries. The padding will be
+ placed after the last entry. This defaults to 0.
+
+pad-byte:
+ This specifies the pad byte to use when padding in the image. It
+ defaults to 0. To use 0xff, you would add 'pad-byte = <0xff>'.
+
+filename:
+ This specifies the image filename. It defaults to 'image.bin'.
+
+sort-by-pos:
+ This causes binman to reorder the entries as needed to make sure they
+ are in increasing positional order. This can be used when your entry
+ order may not match the positional order. A common situation is where
+ the 'pos' properties are set by CONFIG options, so their ordering is
+ not known a priori.
+
+ This is a boolean property so needs no value. To enable it, add a
+ line 'sort-by-pos;' to your description.
+
+multiple-images:
+ Normally only a single image is generated. To create more than one
+ image, put this property in the binman node. For example, this will
+ create image1.bin containing u-boot.bin, and image2.bin containing
+ both spl/u-boot-spl.bin and u-boot.bin:
+
+ binman {
+ multiple-images;
+ image1 {
+ u-boot {
+ };
+ };
+
+ image2 {
+ spl {
+ };
+ u-boot {
+ };
+ };
+ };
+
+end-at-4gb:
+ For x86 machines the ROM positions start just before 4GB and extend
+ up so that the image finished at the 4GB boundary. This boolean
+ option can be enabled to support this. The image size must be
+ provided so that binman knows when the image should start. For an
+ 8MB ROM, the position of the first entry would be 0xfff80000 with
+ this option, instead of 0 without this option.
+
+
+Examples of the above options can be found in the tests. See the
+tools/binman/test directory.
+
+
+Order of image creation
+-----------------------
+
+Image creation proceeds in the following order, for each entry in the image.
+
+1. GetEntryContents() - the contents of each entry are obtained, normally by
+reading from a file. This calls the Entry.ObtainContents() to read the
+contents. The default version of Entry.ObtainContents() calls
+Entry.GetDefaultFilename() and then reads that file. So a common mechanism
+to select a file to read is to override that function in the subclass. The
+functions must return True when they have read the contents. Binman will
+retry calling the functions a few times if False is returned, allowing
+dependencies between the contents of different entries.
+
+2. GetEntryPositions() - calls Entry.GetPositions() for each entry. This can
+return a dict containing entries that need updating. The key should be the
+entry name and the value is a tuple (pos, size). This allows an entry to
+provide the position and size for other entries. The default implementation
+of GetEntryPositions() returns {}.
+
+3. PackEntries() - calls Entry.Pack() which figures out the position and
+size of an entry. The 'current' image position is passed in, and the function
+returns the position immediately after the entry being packed. The default
+implementation of Pack() is usually sufficient.
+
+4. CheckSize() - checks that the contents of all the entries fits within
+the image size. If the image does not have a defined size, the size is set
+large enough to hold all the entries.
+
+5. CheckEntries() - checks that the entries do not overlap, nor extend
+outside the image.
+
+6. ProcessEntryContents() - this calls Entry.ProcessContents() on each entry.
+The default implementatoin does nothing. This can be overriden to adjust the
+contents of an entry in some way. For example, it would be possible to create
+an entry containing a hash of the contents of some other entries. At this
+stage the position and size of entries should not be adjusted.
+
+7. BuildImage() - builds the image and writes it to a file. This is the final
+step.
+
+
+Advanced Features / Technical docs
+----------------------------------
+
+The behaviour of entries is defined by the Entry class. All other entries are
+a subclass of this. An important subclass is Entry_blob which takes binary
+data from a file and places it in the entry. In fact most entry types are
+subclasses of Entry_blob.
+
+Each entry type is a separate file in the tools/binman/etype directory. Each
+file contains a class called Entry_<type> where <type> is the entry type.
+New entry types can be supported by adding new files in that directory.
+These will automatically be detected by binman when needed.
+
+Entry properties are documented in entry.py. The entry subclasses are free
+to change the values of properties to support special behaviour. For example,
+when Entry_blob loads a file, it sets content_size to the size of the file.
+Entry classes can adjust other entries. For example, an entry that knows
+where other entries should be positioned can set up those entries' positions
+so they don't need to be set in the binman decription. It can also adjust
+entry contents.
+
+Most of the time such essoteric behaviour is not needed, but it can be
+essential for complex images.
+
+
+History / Credits
+-----------------
+
+Binman takes a lot of inspiration from a Chrome OS tool called
+'cros_bundle_firmware', which I wrote some years ago. That tool was based on
+a reasonably simple and sound design but has expanded greatly over the
+years. In particular its handling of x86 images is convoluted.
+
+Quite a few lessons have been learned which are hopefully be applied here.
+
+
+Design notes
+------------
+
+On the face of it, a tool to create firmware images should be fairly simple:
+just find all the input binaries and place them at the right place in the
+image. The difficulty comes from the wide variety of input types (simple
+flat binaries containing code, packaged data with various headers), packing
+requirments (alignment, spacing, device boundaries) and other required
+features such as hierarchical images.
+
+The design challenge is to make it easy to create simple images, while
+allowing the more complex cases to be supported. For example, for most
+images we don't much care exactly where each binary ends up, so we should
+not have to specify that unnecessarily.
+
+New entry types should aim to provide simple usage where possible. If new
+core features are needed, they can be added in the Entry base class.
+
+
+To do
+-----
+
+Some ideas:
+- Fill out the device tree to include the final position and size of each
+ entry (since the input file may not always specify these)
+- Use of-platdata to make the information available to code that is unable
+ to use device tree (such as a very small SPL image)
+- Write an image map to a text file
+- Allow easy building of images by specifying just the board name
+- Produce a full Python binding for libfdt (for upstream)
+- Add an option to decode an image into the constituent binaries
+- Suppoort hierarchical images (packing of binaries into another binary
+ which is then placed in the image)
+- Support building an image for a board (-b) more completely, with a
+ configurable build directory
+- Consider making binman work with buildman, although if it is used in the
+ Makefile, this will be automatic
+- Implement align-end
+
+--
+Simon Glass <sjg@chromium.org>
+7/7/2016
--- /dev/null
+binman.py
\ No newline at end of file
--- /dev/null
+#!/usr/bin/python
+
+# Copyright (c) 2016 Google, Inc
+# Written by Simon Glass <sjg@chromium.org>
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Creates binary images from input files controlled by a description
+#
+
+"""See README for more information"""
+
+import os
+import sys
+import traceback
+import unittest
+
+# Bring in the patman and dtoc libraries
+our_path = os.path.dirname(os.path.realpath(__file__))
+sys.path.append(os.path.join(our_path, '../patman'))
+sys.path.append(os.path.join(our_path, '../dtoc'))
+
+# Also allow entry-type modules to be brought in from the etype directory.
+sys.path.append(os.path.join(our_path, 'etype'))
+
+import cmdline
+import command
+import control
+
+def RunTests():
+ """Run the functional tests and any embedded doctests"""
+ import entry_test
+ import fdt_test
+ import func_test
+ import test
+ import doctest
+
+ result = unittest.TestResult()
+ for module in []:
+ suite = doctest.DocTestSuite(module)
+ suite.run(result)
+
+ sys.argv = [sys.argv[0]]
+ for module in (func_test.TestFunctional, fdt_test.TestFdt,
+ entry_test.TestEntry):
+ suite = unittest.TestLoader().loadTestsFromTestCase(module)
+ suite.run(result)
+
+ print result
+ for test, err in result.errors:
+ print test.id(), err
+ for test, err in result.failures:
+ print err
+
+def RunTestCoverage():
+ """Run the tests and check that we get 100% coverage"""
+ # This uses the build output from sandbox_spl to get _libfdt.so
+ cmd = ('PYTHONPATH=%s/sandbox_spl/tools coverage run '
+ '--include "tools/binman/*.py" --omit "*test*,*binman.py" '
+ 'tools/binman/binman.py -t' % options.build_dir)
+ os.system(cmd)
+ stdout = command.Output('coverage', 'report')
+ coverage = stdout.splitlines()[-1].split(' ')[-1]
+ if coverage != '100%':
+ print stdout
+ print "Type 'coverage html' to get a report in htmlcov/index.html"
+ raise ValueError('Coverage error: %s, but should be 100%%' % coverage)
+
+
+def RunBinman(options, args):
+ """Main entry point to binman once arguments are parsed
+
+ Args:
+ options: Command-line options
+ args: Non-option arguments
+ """
+ ret_code = 0
+
+ # For testing: This enables full exception traces.
+ #options.debug = True
+
+ if not options.debug:
+ sys.tracebacklimit = 0
+
+ if options.test:
+ RunTests()
+
+ elif options.test_coverage:
+ RunTestCoverage()
+
+ elif options.full_help:
+ pager = os.getenv('PAGER')
+ if not pager:
+ pager = 'more'
+ fname = os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])),
+ 'README')
+ command.Run(pager, fname)
+
+ else:
+ try:
+ ret_code = control.Binman(options, args)
+ except Exception as e:
+ print 'binman: %s' % e
+ if options.debug:
+ print
+ traceback.print_exc()
+ ret_code = 1
+ return ret_code
+
+
+if __name__ == "__main__":
+ (options, args) = cmdline.ParseArgs(sys.argv)
+ ret_code = RunBinman(options, args)
+ sys.exit(ret_code)
--- /dev/null
+# Copyright (c) 2016 Google, Inc
+# Written by Simon Glass <sjg@chromium.org>
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Command-line parser for binman
+#
+
+from optparse import OptionParser
+
+def ParseArgs(argv):
+ """Parse the binman command-line arguments
+
+ Args:
+ argv: List of string arguments
+ Returns:
+ Tuple (options, args) with the command-line options and arugments.
+ options provides access to the options (e.g. option.debug)
+ args is a list of string arguments
+ """
+ parser = OptionParser()
+ parser.add_option('-b', '--board', type='string',
+ help='Board name to build')
+ parser.add_option('-B', '--build-dir', type='string', default='b',
+ help='Directory containing the build output')
+ parser.add_option('-d', '--dt', type='string',
+ help='Configuration file (.dtb) to use')
+ parser.add_option('-D', '--debug', action='store_true',
+ help='Enabling debugging (provides a full traceback on error)')
+ parser.add_option('-I', '--indir', action='append',
+ help='Add a path to a directory to use for input files')
+ parser.add_option('-H', '--full-help', action='store_true',
+ default=False, help='Display the README file')
+ parser.add_option('-O', '--outdir', type='string',
+ action='store', help='Path to directory to use for intermediate and '
+ 'output files')
+ parser.add_option('-p', '--preserve', action='store_true',\
+ help='Preserve temporary output directory even if option -O is not '
+ 'given')
+ parser.add_option('-t', '--test', action='store_true',
+ default=False, help='run tests')
+ parser.add_option('-T', '--test-coverage', action='store_true',
+ default=False, help='run tests and check for 100% coverage')
+ parser.add_option('-v', '--verbosity', default=1,
+ type='int', help='Control verbosity: 0=silent, 1=progress, 3=full, '
+ '4=debug')
+
+ parser.usage += """
+
+Create images for a board from a set of binaries. It is controlled by a
+description in the board device tree."""
+
+ return parser.parse_args(argv)
--- /dev/null
+# Copyright (c) 2016 Google, Inc
+# Written by Simon Glass <sjg@chromium.org>
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Creates binary images from input files controlled by a description
+#
+
+from collections import OrderedDict
+import os
+import sys
+import tools
+
+import command
+import fdt_select
+import fdt_util
+from image import Image
+import tout
+
+# List of images we plan to create
+# Make this global so that it can be referenced from tests
+images = OrderedDict()
+
+def _ReadImageDesc(binman_node):
+ """Read the image descriptions from the /binman node
+
+ This normally produces a single Image object called 'image'. But if
+ multiple images are present, they will all be returned.
+
+ Args:
+ binman_node: Node object of the /binman node
+ Returns:
+ OrderedDict of Image objects, each of which describes an image
+ """
+ images = OrderedDict()
+ if 'multiple-images' in binman_node.props:
+ for node in binman_node.subnodes:
+ images[node.name] = Image(node.name, node)
+ else:
+ images['image'] = Image('image', binman_node)
+ return images
+
+def _FindBinmanNode(fdt):
+ """Find the 'binman' node in the device tree
+
+ Args:
+ fdt: Fdt object to scan
+ Returns:
+ Node object of /binman node, or None if not found
+ """
+ for node in fdt.GetRoot().subnodes:
+ if node.name == 'binman':
+ return node
+ return None
+
+def Binman(options, args):
+ """The main control code for binman
+
+ This assumes that help and test options have already been dealt with. It
+ deals with the core task of building images.
+
+ Args:
+ options: Command line options object
+ args: Command line arguments (list of strings)
+ """
+ global images
+
+ if options.full_help:
+ pager = os.getenv('PAGER')
+ if not pager:
+ pager = 'more'
+ fname = os.path.join(os.path.dirname(os.path.realpath(sys.argv[0])),
+ 'README')
+ command.Run(pager, fname)
+ return 0
+
+ # Try to figure out which device tree contains our image description
+ if options.dt:
+ dtb_fname = options.dt
+ else:
+ board = options.board
+ if not board:
+ raise ValueError('Must provide a board to process (use -b <board>)')
+ board_pathname = os.path.join(options.build_dir, board)
+ dtb_fname = os.path.join(board_pathname, 'u-boot.dtb')
+ if not options.indir:
+ options.indir = ['.']
+ options.indir.append(board_pathname)
+
+ try:
+ tout.Init(options.verbosity)
+ try:
+ tools.SetInputDirs(options.indir)
+ tools.PrepareOutputDir(options.outdir, options.preserve)
+ fdt = fdt_select.FdtScan(dtb_fname)
+ node = _FindBinmanNode(fdt)
+ if not node:
+ raise ValueError("Device tree '%s' does not have a 'binman' "
+ "node" % dtb_fname)
+ images = _ReadImageDesc(node)
+ for image in images.values():
+ # Perform all steps for this image, including checking and
+ # writing it. This means that errors found with a later
+ # image will be reported after earlier images are already
+ # completed and written, but that does not seem important.
+ image.GetEntryContents()
+ image.GetEntryPositions()
+ image.PackEntries()
+ image.CheckSize()
+ image.CheckEntries()
+ image.ProcessEntryContents()
+ image.BuildImage()
+ finally:
+ tools.FinaliseOutputDir()
+ finally:
+ tout.Uninit()
+
+ return 0
--- /dev/null
+# Copyright (c) 2016 Google, Inc
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Base class for all entries
+#
+
+# importlib was introduced in Python 2.7 but there was a report of it not
+# working in 2.7.12, so we work around this:
+# http://lists.denx.de/pipermail/u-boot/2016-October/269729.html
+try:
+ import importlib
+ have_importlib = True
+except:
+ have_importlib = False
+
+import fdt_util
+import tools
+
+modules = {}
+
+class Entry(object):
+ """An Entry in the image
+
+ An entry corresponds to a single node in the device-tree description
+ of the image. Each entry ends up being a part of the final image.
+ Entries can be placed either right next to each other, or with padding
+ between them. The type of the entry determines the data that is in it.
+
+ This class is not used by itself. All entry objects are subclasses of
+ Entry.
+
+ Attributes:
+ image: The image containing this entry
+ node: The node that created this entry
+ pos: Absolute position of entry within the image, None if not known
+ size: Entry size in bytes, None if not known
+ contents_size: Size of contents in bytes, 0 by default
+ align: Entry start position alignment, or None
+ align_size: Entry size alignment, or None
+ align_end: Entry end position alignment, or None
+ pad_before: Number of pad bytes before the contents, 0 if none
+ pad_after: Number of pad bytes after the contents, 0 if none
+ data: Contents of entry (string of bytes)
+ """
+ def __init__(self, image, etype, node, read_node=True):
+ self.image = image
+ self.etype = etype
+ self._node = node
+ self.pos = None
+ self.size = None
+ self.contents_size = 0
+ self.align = None
+ self.align_size = None
+ self.align_end = None
+ self.pad_before = 0
+ self.pad_after = 0
+ self.pos_unset = False
+ if read_node:
+ self.ReadNode()
+
+ @staticmethod
+ def Create(image, node, etype=None):
+ """Create a new entry for a node.
+
+ Args:
+ image: Image object containing this node
+ node: Node object containing information about the entry to create
+ etype: Entry type to use, or None to work it out (used for tests)
+
+ Returns:
+ A new Entry object of the correct type (a subclass of Entry)
+ """
+ if not etype:
+ etype = fdt_util.GetString(node, 'type', node.name)
+ module_name = etype.replace('-', '_')
+ module = modules.get(module_name)
+
+ # Import the module if we have not already done so.
+ if not module:
+ try:
+ if have_importlib:
+ module = importlib.import_module(module_name)
+ else:
+ module = __import__(module_name)
+ except ImportError:
+ raise ValueError("Unknown entry type '%s' in node '%s'" %
+ (etype, node.path))
+ modules[module_name] = module
+
+ # Call its constructor to get the object we want.
+ obj = getattr(module, 'Entry_%s' % module_name)
+ return obj(image, etype, node)
+
+ def ReadNode(self):
+ """Read entry information from the node
+
+ This reads all the fields we recognise from the node, ready for use.
+ """
+ self.pos = fdt_util.GetInt(self._node, 'pos')
+ self.size = fdt_util.GetInt(self._node, 'size')
+ self.align = fdt_util.GetInt(self._node, 'align')
+ if tools.NotPowerOfTwo(self.align):
+ raise ValueError("Node '%s': Alignment %s must be a power of two" %
+ (self._node.path, self.align))
+ self.pad_before = fdt_util.GetInt(self._node, 'pad-before', 0)
+ self.pad_after = fdt_util.GetInt(self._node, 'pad-after', 0)
+ self.align_size = fdt_util.GetInt(self._node, 'align-size')
+ if tools.NotPowerOfTwo(self.align_size):
+ raise ValueError("Node '%s': Alignment size %s must be a power "
+ "of two" % (self._node.path, self.align_size))
+ self.align_end = fdt_util.GetInt(self._node, 'align-end')
+ self.pos_unset = fdt_util.GetBool(self._node, 'pos-unset')
+
+ def ObtainContents(self):
+ """Figure out the contents of an entry.
+
+ Returns:
+ True if the contents were found, False if another call is needed
+ after the other entries are processed.
+ """
+ # No contents by default: subclasses can implement this
+ return True
+
+ def Pack(self, pos):
+ """Figure out how to pack the entry into the image
+
+ Most of the time the entries are not fully specified. There may be
+ an alignment but no size. In that case we take the size from the
+ contents of the entry.
+
+ If an entry has no hard-coded position, it will be placed at @pos.
+
+ Once this function is complete, both the position and size of the
+ entry will be know.
+
+ Args:
+ Current image position pointer
+
+ Returns:
+ New image position pointer (after this entry)
+ """
+ if self.pos is None:
+ if self.pos_unset:
+ self.Raise('No position set with pos-unset: should another '
+ 'entry provide this correct position?')
+ self.pos = tools.Align(pos, self.align)
+ needed = self.pad_before + self.contents_size + self.pad_after
+ needed = tools.Align(needed, self.align_size)
+ size = self.size
+ if not size:
+ size = needed
+ new_pos = self.pos + size
+ aligned_pos = tools.Align(new_pos, self.align_end)
+ if aligned_pos != new_pos:
+ size = aligned_pos - self.pos
+ new_pos = aligned_pos
+
+ if not self.size:
+ self.size = size
+
+ if self.size < needed:
+ self.Raise("Entry contents size is %#x (%d) but entry size is "
+ "%#x (%d)" % (needed, needed, self.size, self.size))
+ # Check that the alignment is correct. It could be wrong if the
+ # and pos or size values were provided (i.e. not calculated), but
+ # conflict with the provided alignment values
+ if self.size != tools.Align(self.size, self.align_size):
+ self.Raise("Size %#x (%d) does not match align-size %#x (%d)" %
+ (self.size, self.size, self.align_size, self.align_size))
+ if self.pos != tools.Align(self.pos, self.align):
+ self.Raise("Position %#x (%d) does not match align %#x (%d)" %
+ (self.pos, self.pos, self.align, self.align))
+
+ return new_pos
+
+ def Raise(self, msg):
+ """Convenience function to raise an error referencing a node"""
+ raise ValueError("Node '%s': %s" % (self._node.path, msg))
+
+ def GetPath(self):
+ """Get the path of a node
+
+ Returns:
+ Full path of the node for this entry
+ """
+ return self._node.path
+
+ def GetData(self):
+ return self.data
+
+ def GetPositions(self):
+ return {}
+
+ def SetPositionSize(self, pos, size):
+ self.pos = pos
+ self.size = size
+
+ def ProcessContents(self):
+ pass
--- /dev/null
+#
+# Copyright (c) 2016 Google, Inc
+# Written by Simon Glass <sjg@chromium.org>
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Test for the fdt modules
+
+import os
+import sys
+import tempfile
+import unittest
+
+from fdt_select import FdtScan
+import fdt_util
+import tools
+
+class TestFdt(unittest.TestCase):
+ @classmethod
+ def setUpClass(self):
+ self._binman_dir = os.path.dirname(os.path.realpath(sys.argv[0]))
+ self._indir = tempfile.mkdtemp(prefix='binmant.')
+ tools.PrepareOutputDir(self._indir, True)
+
+ def TestFile(self, fname):
+ return os.path.join(self._binman_dir, 'test', fname)
+
+ def GetCompiled(self, fname):
+ return fdt_util.EnsureCompiled(self.TestFile(fname))
+
+ def _DeleteProp(self, fdt):
+ node = fdt.GetNode('/microcode/update@0')
+ node.DeleteProp('data')
+
+ def testFdtNormal(self):
+ fname = self.GetCompiled('34_x86_ucode.dts')
+ fdt = FdtScan(fname)
+ self._DeleteProp(fdt)
+
+ def testFdtFallback(self):
+ fname = self.GetCompiled('34_x86_ucode.dts')
+ fdt = FdtScan(fname, True)
+ fdt.GetProp('/microcode/update@0', 'data')
+ self.assertEqual('fred',
+ fdt.GetProp('/microcode/update@0', 'none', default='fred'))
+ self.assertEqual('12345678 12345679',
+ fdt.GetProp('/microcode/update@0', 'data', typespec='x'))
+ self._DeleteProp(fdt)
--- /dev/null
+# Copyright (c) 2016 Google, Inc
+# Written by Simon Glass <sjg@chromium.org>
+#
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Class for an image, the output of binman
+#
+
+from collections import OrderedDict
+from operator import attrgetter
+
+import entry
+from entry import Entry
+import fdt_util
+import tools
+
+class Image:
+ """A Image, representing an output from binman
+
+ An image is comprised of a collection of entries each containing binary
+ data. The image size must be large enough to hold all of this data.
+
+ This class implements the various operations needed for images.
+
+ Atrtributes:
+ _node: Node object that contains the image definition in device tree
+ _name: Image name
+ _size: Image size in bytes, or None if not known yet
+ _align_size: Image size alignment, or None
+ _pad_before: Number of bytes before the first entry starts. This
+ effectively changes the place where entry position 0 starts
+ _pad_after: Number of bytes after the last entry ends. The last
+ entry will finish on or before this boundary
+ _pad_byte: Byte to use to pad the image where there is no entry
+ _filename: Output filename for image
+ _sort: True if entries should be sorted by position, False if they
+ must be in-order in the device tree description
+ _skip_at_start: Number of bytes before the first entry starts. These
+ effecively adjust the starting position of entries. For example,
+ if _pad_before is 16, then the first entry would start at 16.
+ An entry with pos = 20 would in fact be written at position 4
+ in the image file.
+ _end_4gb: Indicates that the image ends at the 4GB boundary. This is
+ used for x86 images, which want to use positions such that a
+ memory address (like 0xff800000) is the first entry position.
+ This causes _skip_at_start to be set to the starting memory
+ address.
+ _entries: OrderedDict() of entries
+ """
+ def __init__(self, name, node):
+ self._node = node
+ self._name = name
+ self._size = None
+ self._align_size = None
+ self._pad_before = 0
+ self._pad_after = 0
+ self._pad_byte = 0
+ self._filename = '%s.bin' % self._name
+ self._sort = False
+ self._skip_at_start = 0
+ self._end_4gb = False
+ self._entries = OrderedDict()
+
+ self._ReadNode()
+ self._ReadEntries()
+
+ def _ReadNode(self):
+ """Read properties from the image node"""
+ self._size = fdt_util.GetInt(self._node, 'size')
+ self._align_size = fdt_util.GetInt(self._node, 'align-size')
+ if tools.NotPowerOfTwo(self._align_size):
+ self._Raise("Alignment size %s must be a power of two" %
+ self._align_size)
+ self._pad_before = fdt_util.GetInt(self._node, 'pad-before', 0)
+ self._pad_after = fdt_util.GetInt(self._node, 'pad-after', 0)
+ self._pad_byte = fdt_util.GetInt(self._node, 'pad-byte', 0)
+ filename = fdt_util.GetString(self._node, 'filename')
+ if filename:
+ self._filename = filename
+ self._sort = fdt_util.GetBool(self._node, 'sort-by-pos')
+ self._end_4gb = fdt_util.GetBool(self._node, 'end-at-4gb')
+ if self._end_4gb and not self._size:
+ self._Raise("Image size must be provided when using end-at-4gb")
+ if self._end_4gb:
+ self._skip_at_start = 0x100000000 - self._size
+
+ def CheckSize(self):
+ """Check that the image contents does not exceed its size, etc."""
+ contents_size = 0
+ for entry in self._entries.values():
+ contents_size = max(contents_size, entry.pos + entry.size)
+
+ contents_size -= self._skip_at_start
+
+ size = self._size
+ if not size:
+ size = self._pad_before + contents_size + self._pad_after
+ size = tools.Align(size, self._align_size)
+
+ if self._size and contents_size > self._size:
+ self._Raise("contents size %#x (%d) exceeds image size %#x (%d)" %
+ (contents_size, contents_size, self._size, self._size))
+ if not self._size:
+ self._size = size
+ if self._size != tools.Align(self._size, self._align_size):
+ self._Raise("Size %#x (%d) does not match align-size %#x (%d)" %
+ (self._size, self._size, self._align_size, self._align_size))
+
+ def _Raise(self, msg):
+ """Raises an error for this image
+
+ Args:
+ msg: Error message to use in the raise string
+ Raises:
+ ValueError()
+ """
+ raise ValueError("Image '%s': %s" % (self._node.path, msg))
+
+ def _ReadEntries(self):
+ for node in self._node.subnodes:
+ self._entries[node.name] = Entry.Create(self, node)
+
+ def FindEntryType(self, etype):
+ """Find an entry type in the image
+
+ Args:
+ etype: Entry type to find
+ Returns:
+ entry matching that type, or None if not found
+ """
+ for entry in self._entries.values():
+ if entry.etype == etype:
+ return entry
+ return None
+
+ def GetEntryContents(self):
+ """Call ObtainContents() for each entry
+
+ This calls each entry's ObtainContents() a few times until they all
+ return True. We stop calling an entry's function once it returns
+ True. This allows the contents of one entry to depend on another.
+
+ After 3 rounds we give up since it's likely an error.
+ """
+ todo = self._entries.values()
+ for passnum in range(3):
+ next_todo = []
+ for entry in todo:
+ if not entry.ObtainContents():
+ next_todo.append(entry)
+ todo = next_todo
+ if not todo:
+ break
+
+ def _SetEntryPosSize(self, name, pos, size):
+ """Set the position and size of an entry
+
+ Args:
+ name: Entry name to update
+ pos: New position
+ size: New size
+ """
+ entry = self._entries.get(name)
+ if not entry:
+ self._Raise("Unable to set pos/size for unknown entry '%s'" % name)
+ entry.SetPositionSize(self._skip_at_start + pos, size)
+
+ def GetEntryPositions(self):
+ """Handle entries that want to set the position/size of other entries
+
+ This calls each entry's GetPositions() method. If it returns a list
+ of entries to update, it updates them.
+ """
+ for entry in self._entries.values():
+ pos_dict = entry.GetPositions()
+ for name, info in pos_dict.iteritems():
+ self._SetEntryPosSize(name, *info)
+
+ def PackEntries(self):
+ """Pack all entries into the image"""
+ pos = self._skip_at_start
+ for entry in self._entries.values():
+ pos = entry.Pack(pos)
+
+ def _SortEntries(self):
+ """Sort entries by position"""
+ entries = sorted(self._entries.values(), key=lambda entry: entry.pos)
+ self._entries.clear()
+ for entry in entries:
+ self._entries[entry._node.name] = entry
+
+ def CheckEntries(self):
+ """Check that entries do not overlap or extend outside the image"""
+ if self._sort:
+ self._SortEntries()
+ pos = 0
+ prev_name = 'None'
+ for entry in self._entries.values():
+ if (entry.pos < self._skip_at_start or
+ entry.pos >= self._skip_at_start + self._size):
+ entry.Raise("Position %#x (%d) is outside the image starting "
+ "at %#x (%d)" %
+ (entry.pos, entry.pos, self._skip_at_start,
+ self._skip_at_start))
+ if entry.pos < pos:
+ entry.Raise("Position %#x (%d) overlaps with previous entry '%s' "
+ "ending at %#x (%d)" %
+ (entry.pos, entry.pos, prev_name, pos, pos))
+ pos = entry.pos + entry.size
+ prev_name = entry.GetPath()
+
+ def ProcessEntryContents(self):
+ """Call the ProcessContents() method for each entry
+
+ This is intended to adjust the contents as needed by the entry type.
+ """
+ for entry in self._entries.values():
+ entry.ProcessContents()
+
+ def BuildImage(self):
+ """Write the image to a file"""
+ fname = tools.GetOutputFilename(self._filename)
+ with open(fname, 'wb') as fd:
+ fd.write(chr(self._pad_byte) * self._size)
+
+ for entry in self._entries.values():
+ data = entry.GetData()
+ fd.seek(self._pad_before + entry.pos - self._skip_at_start)
+ fd.write(data)
projects / u-boot / commitdiff