Index: head/usr.bin/dtc/Makefile =================================================================== --- head/usr.bin/dtc/Makefile (revision 328172) +++ head/usr.bin/dtc/Makefile (revision 328173) @@ -1,13 +1,11 @@ # $FreeBSD$ PROG_CXX=dtc SRCS= dtc.cc input_buffer.cc string.cc dtb.cc fdt.cc checking.cc MAN= dtc.1 -WARNS?= 3 - CXXFLAGS+= -std=c++11 -fno-rtti -fno-exceptions NO_SHARED?=NO .include Index: head/usr.bin/dtc/dtc.1 =================================================================== --- head/usr.bin/dtc/dtc.1 (revision 328172) +++ head/usr.bin/dtc/dtc.1 (revision 328173) @@ -1,338 +1,412 @@ .\"- .\" Copyright (c) 2013 David Chisnall .\" All rights reserved. .\" .\" This software was developed by SRI International and the University of .\" Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) .\" ("CTSRD"), as part of the DARPA CRASH research programme. .\" .\" This software was developed by SRI International and the University of .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\"/ -.Dd January 1, 2013 +.Dd January 17, 2018 .Dt DTC 1 .Os .Sh NAME .Nm dtc .Nd device tree compiler .Sh SYNOPSIS .Nm .Op Fl @fhsv .Op Fl b Ar boot_cpu_id .Op Fl d Ar dependency_file .Op Fl E Ar [no-]checker_name .Op Fl H Ar phandle_format .Op Fl I Ar input_format .Op Fl O Ar output_format .Op Fl o Ar output_file .Op Fl R Ar entries .Op Fl S Ar bytes .Op Fl p Ar bytes .Op Fl V Ar blob_version .Op Fl W Ar [no-]checker_name .Op Fl P Ar predefined_properties .Ar input_file .Sh DESCRIPTION The .Nm utility converts flattened device tree (FDT) representations. - It is most commonly used to generate device tree blobs (DTB), the binary +It is most commonly used to generate device tree blobs (DTB), the binary representation of an FDT, from device tree sources (DTS), the ASCII text source representation. .Pp The binary can be written in two formats, binary and assembly. The binary is identical to the in-memory representation and can be used directly by firmware, loaders, and so on. The assembly format, documented in .Sx "ASM FORMAT" , will produce the same binary format when assembled, but also includes some global variables that refer to parts of the table. This format is most commonly used to produce a kernel specific to a device, with the device tree blob compiled in. .Pp The options are as follows: .Bl -tag -width indent .It Fl d Ar dependency_file Writes a dependency file understandable by make to the specified file. This file can be included in a Makefile and will ensure that the output file depends on the input file and any files that it includes. This argument is only useful when the input is DTS, as only the source format has a notion of inclusions. .It Fl E Ar [no-]checker_name Enable or disable a specified checker. The argument is the name of the checker. The full list of checkers is given in .Sx CHECKERS . .It Fl @ Emit a __symbols__ node to allow plugins to be loaded. .It Fl f Force the tool to attempt to generate the output, even if the input had errors. .It Fl h Display the help text and exit. .It Fl H Ar phandle_format Specifies the type of phandle nodes to generate in the output. Valid values are: .Pp .Bl -tag -width indent -compact .It Ar linux Generate the legacy linux,phandle nodes expected by older systems. .It Ar epapr Generate the phandle nodes, as described in the ePAPR specification. This is the most sensible option for device trees being used with .Fx . .It Ar both Generate both, for maximum compatibility. .El .It Fl I Ar input_format Specifies the input format. Valid values are: .Pp .Bl -tag -width indent -compact .It Ar dtb Device tree blob. The binary representation of the FDT. .It Ar dts Device tree source. The ASCII representation of the FDT. This is the default if the input format is not explicitly stated. .El .It Fl O Ar output_format Specifies the output format. Valid values are: .Pp .Bl -tag -width indent -compact .It Ar asm Assembler source for generating a device tree blob, as described in .Sx "ASM FORMAT" . .It Ar dtb Device tree blob. The binary representation of the FDT. This is the default if the output format is not explicitly stated. .It Ar dts Device tree source. The ASCII representation of the FDT. .El .It Fl o Ar output_file The file to which to write the output. .It Fl P Ar predefined_macro Defines a macro, in the form .Ar name=value or .Ar name to be used for device tree source files that contain conditional components. This tool supports two extensions to the standard to support conditional compilation of device trees. The first is an .Ar /include/if [property]/ "file.dts" directive that is allowed at the start of a file and which will only include the specified file if it the specified property is passed with this flag. The second is the .Ar $NAME format for property values. These allow property value to be specified on the command line. .It Fl R Ar entries The number of empty reservation table entries to pad the table with. -This is -useful if you are generating a device tree blob for bootloader or similar that -needs to reserve some memory before passing control to the operating system. +This is useful if you are generating a device tree blob for bootloader or +similar that needs to reserve some memory before passing control to the +operating system. .It Fl S Ar bytes The minimum size in bytes of the blob. The blob will be padded after the strings table to ensure that it is the correct size. This is useful for environments where the device tree blob must be modified in place. .It Fl p Ar bytes The number of bytes of padding to add to the blob. The blob will be padded after the strings table to ensure that it is the correct size. This is useful for environments where the device tree blob must be modified in place. .It Fl W Ar [no-]checker_name Enable or disable a specified checker. This is an alias for .Fl E . .It Fl s Sorts the properties and nodes in the tree. This is mainly useful when using tools like .Xr diff 1 to compare two device tree sources. .It Fl V Ar output_version The version of the format to output. This is only relevant for binary outputs, and only a value of 17 is currently supported. .It Fl v Display the tool version and exit. .It Ar input_file The source file. .El .Sh "ASM FORMAT" The assembly format defines several globals that can be referred to from other compilation units, in addition to any labels specified in the source. These are: .Pp .Bl -tag -width "dt_strings_start" -compact -offset indent .It dt_blob_start start of the device tree blob. .It dt_header start of the header, usually identical to the start of the blob. .It dt_reserve_map start of the reservation map. .It dt_struct_start start of the structure table. .It dt_struct_end end of the structure table. .It dt_strings_start start of the strings table. .It dt_strings_end end of the strings table. .It dt_blob_end end of the device tree blob. .El .Sh CHECKERS The utility provides a number of semantic checks on the correctness of the tree. These can be disabled with the .Fl W flag. For example, .Fl W Ar no-type-phandle will disable the phandle type check. The supported checks are: .Pp .Bl -tag -width "no-type-phandle" -compact -offset indent .It type-compatible Checks the type of the .Va compatible property. .It type-model Checks the type of the .Va model property. .It type-compatible Checks the type of the .Va compatible property. .It cells-attributes Checks that all nodes with children have both .Va #address-cells and .Va #size-cells properties. .It deleted-nodes Checks that all .Va /delete-node/ statements refer to nodes that are merged. .El +.Sh OVERLAYS +The utility provides support for generating overlays, also known as plugins. +Overlays are a method of patching a base device tree that has been compiled with +the +.Fl @ +flag, with some limited support for patching device trees that were not compiled +with the +.Fl @ +flag. +.Pp +To denote that a DTS is intended to be used as an overlay, +.Va /plugin/; +should be included in the header, following any applicable +.Va /dts-v1/; +tag. +.Pp +Conventional overlays are crafted by creating +.Va fragment +nodes in a root. +Each fragment node must have either a +.Va target +property set to a label reference, or a +.Va target-path +string property set to a path. +It must then have an +.Va __overlay__ +child node, whose properties and child nodes are merged into the base device +tree when the overlay is applied. +.Pp +Much simpler syntactic sugar was later invented to simplify generating overlays. +Instead of creating targetted fragments manually, one can instead create a root +node that targets a label in the base node using the +.Va &label +syntax supported in conventional DTS. +This will indicate that a fragment should be generated for the node, with the +given +.Va label +being the target, and the properties and child nodes will be used as the +__overlay__. +.Pp +Both conventional overlays and the later-added syntactic sugar are supported. +.Pp +Overlay blobs can be applied at boot time by setting +.Va fdt_overlays +in +.Xr loader.conf 5 . +Multiple overlays may be specified, and they will be applied in the order given. +.El .Sh EXAMPLES The command: .Pp .Dl "dtc -o blob.S -O asm device.dts" .Pp will generate a .Pa blob.S file from the device tree source .Pa device.dts and print errors if any occur during parsing or property checking. -The -resulting file can be assembled and linked into a binary. +The resulting file can be assembled and linked into a binary. .Pp The command: .Pp .Dl "dtc -o - -O dts -I dtb device.dtb" .Pp will write the device tree source for the device tree blob .Pa device.dtb to the standard output. This is useful when debugging device trees. +.Pp +The command: +.Pp +.Dl "dtc -@ -O dtb -I dts -o device.dtb device.dts" +.Pp +will generate a +.Pa device.dtb +file from the device tree source +.Pa device.dts +with a __symbols__ node included so that overlays may be applied to it. +.Pp +The command: +.Pp +.Dl "dtc -@ -O dtb -I dts -o device_overlay.dtbo device_overlay.dts" +.Pp +will generate a +.Pa device_overlay.dtbo +file, using the standard extension for a device tree overlay, from the device +tree source +.Pa device_overlay.dts . +A __symbols__ node will be included so that overlays may be applied to it. +The presence of a +.Va /plugin/; +directive in +.Pa device_overlay.dts +will indicate to the utility that it should also generate the underlying +metadata required in overlays. .Sh COMPATIBILITY This utility is intended to be compatible with the device tree compiler provided by elinux.org. Currently, it implements the subset of features required to build FreeBSD and others that have been requested by FreeBSD developers. .Pp The .Ar fs input format is not supported. This builds a tree from a Linux .Pa /proc/device-tree , a file system hierarchy not found in FreeBSD, which instead exposes the DTB directly via a sysctl. .Pp The warnings and errors supported by the elinux.org tool are not documented. This tool supports the warnings described in the .Sx CHECKERS section. .Sh SEE ALSO .Xr fdt 4 .Sh STANDARDS The device tree formats understood by this tool conform to the Power.org Standard for Embedded Power Architecture Platform Requirements .Pq Vt ePAPR , except as noted in the .Sx BUGS section and with the following exceptions for compatibility with the elinux.org tool: .Pp .Bl -bullet -compact .It The target of cross references is defined to be a node name in the specification, but is in fact a label. .El .Pp The /include/ directive is not part of the standard, however it is implemented with the semantics compatible with the elinux.org tool. It must appear in the top level of a file, and imports a new root definition. If a file, plus all of its inclusions, contains multiple roots then they are merged. All nodes that are present in the second but not the first are imported. Any that appear in both are recursively merged, with properties from the second replacing those from the first and properties child nodes being recursively merged. .Sh HISTORY A dtc tool first appeared in .Fx 9.0 . This version of the tool first appeared in .Fx 10.0 . .Sh AUTHORS .An David T. Chisnall .Pp Note: The fact that the tool and the author share the same initials is entirely coincidental. .Sh BUGS The device tree compiler does not yet support the following features: .Pp .Bl -bullet -compact .It Labels in the middle of property values. This is only useful in the assembly output, and only vaguely useful there, so is unlikely to be added soon. .It Full paths, rather than labels, as the targets for phandles. This is not very hard to add, but will probably not be added until something actually needs it. .El .Pp The current version performs a very limited set of semantic checks on the tree. This will be improved in future versions. Index: head/usr.bin/dtc/dtc.cc =================================================================== --- head/usr.bin/dtc/dtc.cc (revision 328172) +++ head/usr.bin/dtc/dtc.cc (revision 328173) @@ -1,355 +1,359 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 David Chisnall * All rights reserved. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #include #include #include #include #include #include #include #include #include "fdt.hh" #include "checking.hh" #include "util.hh" using namespace dtc; using std::string; +namespace { + /** * The current major version of the tool. */ int version_major = 0; int version_major_compatible = 1; /** * The current minor version of the tool. */ int version_minor = 5; int version_minor_compatible = 4; /** * The current patch level of the tool. */ int version_patch = 0; int version_patch_compatible = 0; -static void usage(const string &argv0) +void usage(const string &argv0) { fprintf(stderr, "Usage:\n" "\t%s\t[-fhsv@] [-b boot_cpu_id] [-d dependency_file]" "[-E [no-]checker_name]\n" "\t\t[-H phandle_format] [-I input_format]" "[-O output_format]\n" "\t\t[-o output_file] [-R entries] [-S bytes] [-p bytes]" "[-V blob_version]\n" "\t\t-W [no-]checker_name] input_file\n", basename(argv0).c_str()); } /** * Prints the current version of this program.. */ -static void version(const char* progname) +void version(const char* progname) { fprintf(stdout, "Version: %s %d.%d.%d compatible with gpl dtc %d.%d.%d\n", progname, version_major, version_minor, version_patch, version_major_compatible, version_minor_compatible, version_patch_compatible); } + +} // Anonymous namespace using fdt::device_tree; int main(int argc, char **argv) { int ch; int outfile = fileno(stdout); const char *outfile_name = "-"; const char *in_file = "-"; FILE *depfile = 0; bool debug_mode = false; auto write_fn = &device_tree::write_binary; auto read_fn = &device_tree::parse_dts; uint32_t boot_cpu; bool boot_cpu_specified = false; bool keep_going = false; bool sort = false; clock_t c0 = clock(); class device_tree tree; fdt::checking::check_manager checks; const char *options = "@hqI:O:o:V:d:R:S:p:b:fi:svH:W:E:DP:"; // Don't forget to update the man page if any more options are added. while ((ch = getopt(argc, argv, options)) != -1) { switch (ch) { case 'h': usage(argv[0]); return EXIT_SUCCESS; case 'v': version(argv[0]); return EXIT_SUCCESS; case '@': tree.write_symbols = true; break; case 'I': { string arg(optarg); if (arg == "dtb") { read_fn = &device_tree::parse_dtb; } else if (arg == "dts") { read_fn = &device_tree::parse_dts; } else { fprintf(stderr, "Unknown input format: %s\n", optarg); return EXIT_FAILURE; } break; } case 'O': { string arg(optarg); if (arg == "dtb") { write_fn = &device_tree::write_binary; } else if (arg == "asm") { write_fn = &device_tree::write_asm; } else if (arg == "dts") { write_fn = &device_tree::write_dts; } else { fprintf(stderr, "Unknown output format: %s\n", optarg); return EXIT_FAILURE; } break; } case 'o': { outfile_name = optarg; outfile = open(optarg, O_CREAT | O_TRUNC | O_WRONLY, 0666); if (outfile == -1) { perror("Unable to open output file"); return EXIT_FAILURE; } break; } case 'D': debug_mode = true; break; case 'V': if (string(optarg) != "17") { fprintf(stderr, "Unknown output format version: %s\n", optarg); return EXIT_FAILURE; } break; case 'd': { if (depfile != 0) { fclose(depfile); } if (string(optarg) == "-") { depfile = stdout; } else { depfile = fdopen(open(optarg, O_CREAT | O_TRUNC | O_WRONLY, 0666), "w"); if (depfile == 0) { perror("Unable to open dependency file"); return EXIT_FAILURE; } } break; } case 'H': { string arg(optarg); if (arg == "both") { tree.set_phandle_format(device_tree::BOTH); } else if (arg == "epapr") { tree.set_phandle_format(device_tree::EPAPR); } else if (arg == "linux") { tree.set_phandle_format(device_tree::LINUX); } else { fprintf(stderr, "Unknown phandle format: %s\n", optarg); return EXIT_FAILURE; } break; } case 'b': // Don't bother to check if strtoll fails, just // use the 0 it returns. boot_cpu = (uint32_t)strtoll(optarg, 0, 10); boot_cpu_specified = true; break; case 'f': keep_going = true; break; case 'W': case 'E': { string arg(optarg); if ((arg.size() > 3) && (strncmp(optarg, "no-", 3) == 0)) { arg = string(optarg+3); if (!checks.disable_checker(arg)) { fprintf(stderr, "Checker %s either does not exist or is already disabled\n", optarg+3); } break; } if (!checks.enable_checker(arg)) { fprintf(stderr, "Checker %s either does not exist or is already enabled\n", optarg); } break; } case 's': { sort = true; break; } case 'i': { tree.add_include_path(optarg); break; } // Should quiet warnings, but for now is silently ignored. case 'q': break; case 'R': tree.set_empty_reserve_map_entries(strtoll(optarg, 0, 10)); break; case 'S': tree.set_blob_minimum_size(strtoll(optarg, 0, 10)); break; case 'p': tree.set_blob_padding(strtoll(optarg, 0, 10)); break; case 'P': if (!tree.parse_define(optarg)) { fprintf(stderr, "Invalid predefine value %s\n", optarg); } break; default: fprintf(stderr, "Unknown option %c\n", ch); return EXIT_FAILURE; } } if (optind < argc) { in_file = argv[optind]; } if (depfile != 0) { fputs(outfile_name, depfile); fputs(": ", depfile); fputs(in_file, depfile); } clock_t c1 = clock(); (tree.*read_fn)(in_file, depfile); // Override the boot CPU found in the header, if we're loading from dtb if (boot_cpu_specified) { tree.set_boot_cpu(boot_cpu); } if (sort) { tree.sort(); } if (depfile != 0) { putc('\n', depfile); fclose(depfile); } if (!(tree.is_valid() || keep_going)) { fprintf(stderr, "Failed to parse tree.\n"); return EXIT_FAILURE; } clock_t c2 = clock(); if (!(checks.run_checks(&tree, true) || keep_going)) { return EXIT_FAILURE; } clock_t c3 = clock(); (tree.*write_fn)(outfile); close(outfile); clock_t c4 = clock(); if (debug_mode) { struct rusage r; getrusage(RUSAGE_SELF, &r); fprintf(stderr, "Peak memory usage: %ld bytes\n", r.ru_maxrss); fprintf(stderr, "Setup and option parsing took %f seconds\n", ((double)(c1-c0))/CLOCKS_PER_SEC); fprintf(stderr, "Parsing took %f seconds\n", ((double)(c2-c1))/CLOCKS_PER_SEC); fprintf(stderr, "Checking took %f seconds\n", ((double)(c3-c2))/CLOCKS_PER_SEC); fprintf(stderr, "Generating output took %f seconds\n", ((double)(c4-c3))/CLOCKS_PER_SEC); fprintf(stderr, "Total time: %f seconds\n", ((double)(c4-c0))/CLOCKS_PER_SEC); // This is not needed, but keeps valgrind quiet. fclose(stdin); } return EXIT_SUCCESS; } Index: head/usr.bin/dtc/fdt.cc =================================================================== --- head/usr.bin/dtc/fdt.cc (revision 328172) +++ head/usr.bin/dtc/fdt.cc (revision 328173) @@ -1,1823 +1,2020 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 David Chisnall * All rights reserved. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #define __STDC_LIMIT_MACROS 1 #include "fdt.hh" #include "dtb.hh" #include #include #include #include #include #include #include #include #include #include #include #include using std::string; namespace dtc { namespace fdt { uint32_t property_value::get_as_uint32() { if (byte_data.size() != 4) { return 0; } uint32_t v = 0; v &= byte_data[0] << 24; v &= byte_data[1] << 16; v &= byte_data[2] << 8; v &= byte_data[3] << 0; return v; } void property_value::push_to_buffer(byte_buffer &buffer) { if (!byte_data.empty()) { buffer.insert(buffer.end(), byte_data.begin(), byte_data.end()); } else { push_string(buffer, string_data, true); // Trailing nul buffer.push_back(0); } } void property_value::write_dts(FILE *file) { resolve_type(); switch (type) { default: assert(0 && "Invalid type"); case STRING: case STRING_LIST: case CROSS_REFERENCE: write_as_string(file); break; case PHANDLE: write_as_cells(file); break; case BINARY: if (byte_data.size() % 4 == 0) { write_as_cells(file); break; } write_as_bytes(file); break; } } void property_value::resolve_type() { if (type != UNKNOWN) { return; } if (byte_data.empty()) { type = STRING; return; } if (byte_data.back() == 0) { bool is_all_printable = true; int nuls = 0; int bytes = 0; bool lastWasNull = false; for (auto i : byte_data) { bytes++; is_all_printable &= (i == '\0') || isprint(i); if (i == '\0') { // If there are two nulls in a row, then we're probably binary. if (lastWasNull) { type = BINARY; return; } nuls++; lastWasNull = true; } else { lastWasNull = false; } if (!is_all_printable) { break; } } if ((is_all_printable && (bytes > nuls)) || bytes == 0) { type = STRING; if (nuls > 1) { type = STRING_LIST; } return; } } type = BINARY; } size_t property_value::size() { if (!byte_data.empty()) { return byte_data.size(); } return string_data.size() + 1; } void property_value::write_as_string(FILE *file) { putc('"', file); if (byte_data.empty()) { fputs(string_data.c_str(), file); } else { bool hasNull = (byte_data.back() == '\0'); // Remove trailing null bytes from the string before printing as dts. if (hasNull) { byte_data.pop_back(); } for (auto i : byte_data) { // FIXME Escape tabs, newlines, and so on. if (i == '\0') { fputs("\", \"", file); continue; } putc(i, file); } if (hasNull) { byte_data.push_back('\0'); } } putc('"', file); } void property_value::write_as_cells(FILE *file) { putc('<', file); assert((byte_data.size() % 4) == 0); for (auto i=byte_data.begin(), e=byte_data.end(); i!=e ; ++i) { uint32_t v = 0; v = (v << 8) | *i; ++i; v = (v << 8) | *i; ++i; v = (v << 8) | *i; ++i; v = (v << 8) | *i; fprintf(file, "0x%" PRIx32, v); if (i+1 != e) { putc(' ', file); } } putc('>', file); } void property_value::write_as_bytes(FILE *file) { putc('[', file); for (auto i=byte_data.begin(), e=byte_data.end(); i!=e ; i++) { fprintf(file, "%02hhx", *i); if (i+1 != e) { putc(' ', file); } } putc(']', file); } void property::parse_string(text_input_buffer &input) { property_value v; assert(*input == '"'); ++input; std::vector bytes; bool isEscaped = false; while (char c = *input) { if (c == '"' && !isEscaped) { input.consume('"'); break; } isEscaped = (c == '\\'); bytes.push_back(c); ++input; } v.string_data = string(bytes.begin(), bytes.end()); values.push_back(v); } void property::parse_cells(text_input_buffer &input, int cell_size) { assert(*input == '<'); ++input; property_value v; input.next_token(); while (!input.consume('>')) { input.next_token(); // If this is a phandle then we need to get the name of the // referenced node if (input.consume('&')) { if (cell_size != 32) { input.parse_error("reference only permitted in 32-bit arrays"); valid = false; return; } input.next_token(); string referenced; if (!input.consume('{')) { referenced = input.parse_node_name(); } else { referenced = input.parse_to('}'); input.consume('}'); } if (referenced.empty()) { input.parse_error("Expected node name"); valid = false; return; } input.next_token(); // If we already have some bytes, make the phandle a // separate component. if (!v.byte_data.empty()) { values.push_back(v); v = property_value(); } v.string_data = referenced; v.type = property_value::PHANDLE; values.push_back(v); v = property_value(); } else { //FIXME: We should support labels in the middle //of these, but we don't. unsigned long long val; if (!input.consume_integer_expression(val)) { input.parse_error("Expected numbers in array of cells"); valid = false; return; } switch (cell_size) { case 8: v.byte_data.push_back(val); break; case 16: push_big_endian(v.byte_data, (uint16_t)val); break; case 32: push_big_endian(v.byte_data, (uint32_t)val); break; case 64: push_big_endian(v.byte_data, (uint64_t)val); break; default: assert(0 && "Invalid cell size!"); } input.next_token(); } } // Don't store an empty string value here. if (v.byte_data.size() > 0) { values.push_back(v); } } void property::parse_bytes(text_input_buffer &input) { assert(*input == '['); ++input; property_value v; input.next_token(); while (!input.consume(']')) { { //FIXME: We should support //labels in the middle of //these, but we don't. uint8_t val; if (!input.consume_hex_byte(val)) { input.parse_error("Expected hex bytes in array of bytes"); valid = false; return; } v.byte_data.push_back(val); input.next_token(); } } values.push_back(v); } void property::parse_reference(text_input_buffer &input) { assert(*input == '&'); ++input; input.next_token(); property_value v; v.string_data = input.parse_node_name(); if (v.string_data.empty()) { input.parse_error("Expected node name"); valid = false; return; } v.type = property_value::CROSS_REFERENCE; values.push_back(v); } property::property(input_buffer &structs, input_buffer &strings) { uint32_t name_offset; uint32_t length; valid = structs.consume_binary(length) && structs.consume_binary(name_offset); if (!valid) { fprintf(stderr, "Failed to read property\n"); return; } // Find the name input_buffer name_buffer = strings.buffer_from_offset(name_offset); if (name_buffer.finished()) { fprintf(stderr, "Property name offset %" PRIu32 " is past the end of the strings table\n", name_offset); valid = false; return; } key = name_buffer.parse_to(0); // If we're empty, do not push anything as value. if (!length) return; // Read the value uint8_t byte; property_value v; for (uint32_t i=0 ; ifind(name)) == defines->end())) { input.parse_error("Undefined property name\n"); valid = false; return; } values.push_back((*found).second->values[0]); } property::property(text_input_buffer &input, string &&k, string_set &&l, bool semicolonTerminated, define_map *defines) : key(k), labels(l), valid(true) { do { input.next_token(); switch (*input) { case '$': { parse_define(input, defines); if (valid) { break; } } default: input.parse_error("Invalid property value."); valid = false; return; case '/': { + if (input.consume("/incbin/(\"")) + { + auto loc = input.location(); + std::string filename = input.parse_to('"'); + if (!(valid = input.consume('"'))) + { + loc.report_error("Syntax error, expected '\"' to terminate /incbin/("); + return; + } + property_value v; + if (!(valid = input.read_binary_file(filename, v.byte_data))) + { + input.parse_error("Cannot open binary include file"); + return; + } + if (!(valid &= input.consume(')'))) + { + input.parse_error("Syntax error, expected ')' to terminate /incbin/("); + return; + } + values.push_back(v); + break; + } unsigned long long bits = 0; valid = input.consume("/bits/"); input.next_token(); valid &= input.consume_integer(bits); if ((bits != 8) && (bits != 16) && (bits != 32) && (bits != 64)) { input.parse_error("Invalid size for elements"); valid = false; } if (!valid) return; input.next_token(); if (*input != '<') { input.parse_error("/bits/ directive is only valid on arrays"); valid = false; return; } parse_cells(input, bits); break; } case '"': parse_string(input); break; case '<': parse_cells(input, 32); break; case '[': parse_bytes(input); break; case '&': parse_reference(input); break; case ';': { break; } } input.next_token(); } while (input.consume(',')); if (semicolonTerminated && !input.consume(';')) { input.parse_error("Expected ; at end of property"); valid = false; } } property_ptr property::parse_dtb(input_buffer &structs, input_buffer &strings) { property_ptr p(new property(structs, strings)); if (!p->valid) { p = nullptr; } return p; } property_ptr property::parse(text_input_buffer &input, string &&key, string_set &&label, bool semicolonTerminated, define_map *defines) { property_ptr p(new property(input, std::move(key), std::move(label), semicolonTerminated, defines)); if (!p->valid) { p = nullptr; } return p; } void property::write(dtb::output_writer &writer, dtb::string_table &strings) { writer.write_token(dtb::FDT_PROP); byte_buffer value_buffer; for (value_iterator i=begin(), e=end() ; i!=e ; ++i) { i->push_to_buffer(value_buffer); } writer.write_data((uint32_t)value_buffer.size()); writer.write_comment(key); writer.write_data(strings.add_string(key)); writer.write_data(value_buffer); } bool property_value::try_to_merge(property_value &other) { resolve_type(); switch (type) { case UNKNOWN: __builtin_unreachable(); assert(0); return false; case EMPTY: *this = other; case STRING: case STRING_LIST: case CROSS_REFERENCE: return false; case PHANDLE: case BINARY: if (other.type == PHANDLE || other.type == BINARY) { type = BINARY; byte_data.insert(byte_data.end(), other.byte_data.begin(), other.byte_data.end()); return true; } } return false; } void property::write_dts(FILE *file, int indent) { for (int i=0 ; i *vals = &values; std::vector v; // If we've got multiple values then try to merge them all together. if (values.size() > 1) { vals = &v; v.push_back(values.front()); for (auto i=(++begin()), e=end() ; i!=e ; ++i) { if (!v.back().try_to_merge(*i)) { v.push_back(*i); } } } fputs(" = ", file); for (auto i=vals->begin(), e=vals->end() ; i!=e ; ++i) { i->write_dts(file); if (i+1 != e) { putc(',', file); putc(' ', file); } } } fputs(";\n", file); } size_t property::offset_of_value(property_value &val) { size_t off = 0; for (auto &v : values) { if (&v == &val) { return off; } off += v.size(); } return -1; } string node::parse_name(text_input_buffer &input, bool &is_property, const char *error) { if (!valid) { return string(); } input.next_token(); if (is_property) { return input.parse_property_name(); } string n = input.parse_node_or_property_name(is_property); if (n.empty()) { if (n.empty()) { input.parse_error(error); valid = false; } } return n; } void node::visit(std::function fn) { fn(*this); for (auto &&c : children) { c->visit(fn); } } node::node(input_buffer &structs, input_buffer &strings) : valid(true) { std::vector bytes; while (structs[0] != '\0' && structs[0] != '@') { bytes.push_back(structs[0]); ++structs; } name = string(bytes.begin(), bytes.end()); bytes.clear(); if (structs[0] == '@') { ++structs; while (structs[0] != '\0') { bytes.push_back(structs[0]); ++structs; } unit_address = string(bytes.begin(), bytes.end()); } ++structs; uint32_t token; while (structs.consume_binary(token)) { switch (token) { default: fprintf(stderr, "Unexpected token 0x%" PRIx32 " while parsing node.\n", token); valid = false; return; // Child node, parse it. case dtb::FDT_BEGIN_NODE: { node_ptr child = node::parse_dtb(structs, strings); if (child == 0) { valid = false; return; } children.push_back(std::move(child)); break; } // End of this node, no errors. case dtb::FDT_END_NODE: return; // Property, parse it. case dtb::FDT_PROP: { property_ptr prop = property::parse_dtb(structs, strings); if (prop == 0) { valid = false; return; } props.push_back(prop); break; } break; // End of structs table. Should appear after // the end of the last node. case dtb::FDT_END: fprintf(stderr, "Unexpected FDT_END token while parsing node.\n"); valid = false; return; // NOPs are padding. Ignore them. case dtb::FDT_NOP: break; } } fprintf(stderr, "Failed to read token from structs table while parsing node.\n"); valid = false; return; } node::node(const string &n, const std::vector &p) : name(n) { props.insert(props.begin(), p.begin(), p.end()); } node_ptr node::create_special_node(const string &name, const std::vector &props) { node_ptr n(new node(name, props)); return n; } node::node(text_input_buffer &input, string &&n, std::unordered_set &&l, string &&a, define_map *defines) : labels(l), name(n), unit_address(a), valid(true) { if (!input.consume('{')) { input.parse_error("Expected { to start new device tree node.\n"); } input.next_token(); while (valid && !input.consume('}')) { // flag set if we find any characters that are only in // the property name character set, not the node bool is_property = false; string child_name, child_address; std::unordered_set child_labels; auto parse_delete = [&](const char *expected, bool at) { if (child_name == string()) { input.parse_error(expected); valid = false; return; } input.next_token(); if (at && input.consume('@')) { child_name += '@'; child_name += parse_name(input, is_property, "Expected unit address"); } if (!input.consume(';')) { input.parse_error("Expected semicolon"); valid = false; return; } input.next_token(); }; if (input.consume("/delete-node/")) { input.next_token(); child_name = input.parse_node_name(); parse_delete("Expected node name", true); if (valid) { deleted_children.insert(child_name); } continue; } if (input.consume("/delete-property/")) { input.next_token(); child_name = input.parse_property_name(); parse_delete("Expected property name", false); if (valid) { deleted_props.insert(child_name); } continue; } child_name = parse_name(input, is_property, "Expected property or node name"); while (input.consume(':')) { // Node labels can contain any characters? The // spec doesn't say, so we guess so... is_property = false; child_labels.insert(std::move(child_name)); child_name = parse_name(input, is_property, "Expected property or node name"); } if (input.consume('@')) { child_address = parse_name(input, is_property, "Expected unit address"); } if (!valid) { return; } input.next_token(); // If we're parsing a property, then we must actually do that. if (input.consume('=')) { property_ptr p = property::parse(input, std::move(child_name), std::move(child_labels), true, defines); if (p == 0) { valid = false; } else { props.push_back(p); } } else if (!is_property && *input == ('{')) { node_ptr child = node::parse(input, std::move(child_name), std::move(child_labels), std::move(child_address), defines); if (child) { children.push_back(std::move(child)); } else { valid = false; } } else if (input.consume(';')) { props.push_back(property_ptr(new property(std::move(child_name), std::move(child_labels)))); } else { input.parse_error("Error parsing property. Expected property value"); valid = false; } input.next_token(); } input.next_token(); input.consume(';'); } bool node::cmp_properties(property_ptr &p1, property_ptr &p2) { return p1->get_key() < p2->get_key(); } bool node::cmp_children(node_ptr &c1, node_ptr &c2) { if (c1->name == c2->name) { return c1->unit_address < c2->unit_address; } return c1->name < c2->name; } void node::sort() { std::sort(property_begin(), property_end(), cmp_properties); std::sort(child_begin(), child_end(), cmp_children); for (auto &c : child_nodes()) { c->sort(); } } node_ptr node::parse(text_input_buffer &input, string &&name, string_set &&label, string &&address, define_map *defines) { node_ptr n(new node(input, std::move(name), std::move(label), std::move(address), defines)); if (!n->valid) { n = 0; } return n; } node_ptr node::parse_dtb(input_buffer &structs, input_buffer &strings) { node_ptr n(new node(structs, strings)); if (!n->valid) { n = 0; } return n; } property_ptr node::get_property(const string &key) { for (auto &i : props) { if (i->get_key() == key) { return i; } } return 0; } void -node::merge_node(node_ptr other) +node::merge_node(node_ptr &other) { for (auto &l : other->labels) { labels.insert(l); } // Note: this is an O(n*m) operation. It might be sensible to // optimise this if we find that there are nodes with very // large numbers of properties, but for typical usage the // entire vector will fit (easily) into cache, so iterating // over it repeatedly isn't that expensive. for (auto &p : other->properties()) { bool found = false; for (auto &mp : properties()) { if (mp->get_key() == p->get_key()) { mp = p; found = true; break; } } if (!found) { add_property(p); } } for (auto &c : other->children) { bool found = false; for (auto &i : children) { if (i->name == c->name && i->unit_address == c->unit_address) { - i->merge_node(std::move(c)); + i->merge_node(c); found = true; break; } } if (!found) { children.push_back(std::move(c)); } } children.erase(std::remove_if(children.begin(), children.end(), [&](const node_ptr &p) { string full_name = p->name; if (p->unit_address != string()) { full_name += '@'; full_name += p->unit_address; } if (other->deleted_children.count(full_name) > 0) { other->deleted_children.erase(full_name); return true; } return false; }), children.end()); props.erase(std::remove_if(props.begin(), props.end(), [&](const property_ptr &p) { if (other->deleted_props.count(p->get_key()) > 0) { other->deleted_props.erase(p->get_key()); return true; } return false; }), props.end()); } void node::write(dtb::output_writer &writer, dtb::string_table &strings) { writer.write_token(dtb::FDT_BEGIN_NODE); byte_buffer name_buffer; push_string(name_buffer, name); if (unit_address != string()) { name_buffer.push_back('@'); push_string(name_buffer, unit_address); } writer.write_comment(name); writer.write_data(name_buffer); writer.write_data((uint8_t)0); for (auto p : properties()) { p->write(writer, strings); } for (auto &c : child_nodes()) { c->write(writer, strings); } writer.write_token(dtb::FDT_END_NODE); } void node::write_dts(FILE *file, int indent) { for (int i=0 ; iwrite_dts(file, indent+1); } for (auto &c : child_nodes()) { c->write_dts(file, indent+1); } for (int i=0 ; iname, n->unit_address)); for (const string &name : n->labels) { if (name != string()) { auto iter = node_names.find(name); if (iter == node_names.end()) { node_names.insert(std::make_pair(name, n.get())); node_paths.insert(std::make_pair(name, path)); } else { node_names.erase(iter); auto i = node_paths.find(name); if (i != node_paths.end()) { node_paths.erase(name); } fprintf(stderr, "Label not unique: %s. References to this label will not be resolved.\n", name.c_str()); } } } for (auto &c : n->child_nodes()) { collect_names_recursive(c, path); } // Now we collect the phandles and properties that reference // other nodes. for (auto &p : n->properties()) { for (auto &v : *p) { if (v.is_phandle()) { fixups.push_back({path, p, v}); } if (v.is_cross_reference()) { cross_references.push_back(&v); } } if ((p->get_key() == "phandle") || (p->get_key() == "linux,phandle")) { if (p->begin()->byte_data.size() != 4) { fprintf(stderr, "Invalid phandle value for node %s. Should be a 4-byte value.\n", n->name.c_str()); valid = false; } else { uint32_t phandle = p->begin()->get_as_uint32(); used_phandles.insert(std::make_pair(phandle, n.get())); } } } path.pop_back(); } void device_tree::collect_names() { node_path p; node_names.clear(); node_paths.clear(); cross_references.clear(); fixups.clear(); collect_names_recursive(root, p); } +property_ptr +device_tree::assign_phandle(node *n, uint32_t &phandle) +{ + // If there is an existing phandle, use it + property_ptr p = n->get_property("phandle"); + if (p == 0) + { + p = n->get_property("linux,phandle"); + } + if (p == 0) + { + // Otherwise insert a new phandle node + property_value v; + while (used_phandles.find(phandle) != used_phandles.end()) + { + // Note that we only don't need to + // store this phandle in the set, + // because we are monotonically + // increasing the value of phandle and + // so will only ever revisit this value + // if we have used 2^32 phandles, at + // which point our blob won't fit in + // any 32-bit system and we've done + // something badly wrong elsewhere + // already. + phandle++; + } + push_big_endian(v.byte_data, phandle++); + if (phandle_node_name == BOTH || phandle_node_name == LINUX) + { + p.reset(new property("linux,phandle")); + p->add_value(v); + n->add_property(p); + } + if (phandle_node_name == BOTH || phandle_node_name == EPAPR) + { + p.reset(new property("phandle")); + p->add_value(v); + n->add_property(p); + } + } + + return (p); +} + void -device_tree::resolve_cross_references() +device_tree::assign_phandles(node_ptr &n, uint32_t &next) { + if (!n->labels.empty()) + { + assign_phandle(n.get(), next); + } + + for (auto &c : n->child_nodes()) + { + assign_phandles(c, next); + } +} + +void +device_tree::resolve_cross_references(uint32_t &phandle) +{ for (auto *pv : cross_references) { node_path path = node_paths[pv->string_data]; auto p = path.begin(); auto pe = path.end(); if (p != pe) { // Skip the first name in the path. It's always "", and implicitly / for (++p ; p!=pe ; ++p) { pv->byte_data.push_back('/'); push_string(pv->byte_data, p->first); if (!(p->second.empty())) { pv->byte_data.push_back('@'); push_string(pv->byte_data, p->second); } } pv->byte_data.push_back(0); } } std::unordered_map phandle_set; for (auto &i : fixups) { phandle_set.insert({&i.val, i}); } std::vector> sorted_phandles; root->visit([&](node &n) { for (auto &p : n.properties()) { for (auto &v : *p) { auto i = phandle_set.find(&v); if (i != phandle_set.end()) { sorted_phandles.push_back(i->second); } } } }); assert(sorted_phandles.size() == fixups.size()); - uint32_t phandle = 1; for (auto &i : sorted_phandles) { string target_name = i.get().val.string_data; node *target = nullptr; string possible; // If the node name is a path, then look it up by following the path, // otherwise jump directly to the named node. if (target_name[0] == '/') { string path; target = root.get(); std::istringstream ss(target_name); string path_element; // Read the leading / std::getline(ss, path_element, '/'); // Iterate over path elements while (!ss.eof()) { path += '/'; std::getline(ss, path_element, '/'); std::istringstream nss(path_element); string node_name, node_address; std::getline(nss, node_name, '@'); std::getline(nss, node_address, '@'); node *next = nullptr; for (auto &c : target->child_nodes()) { if (c->name == node_name) { if (c->unit_address == node_address) { next = c.get(); break; } else { possible = path + c->name; if (c->unit_address != string()) { possible += '@'; possible += c->unit_address; } } } } path += node_name; if (node_address != string()) { path += '@'; path += node_address; } target = next; if (target == nullptr) { break; } } } else { target = node_names[target_name]; } if (target == nullptr) { if (is_plugin) { unresolved_fixups.push_back(i); continue; } else { fprintf(stderr, "Failed to find node with label: %s\n", target_name.c_str()); if (possible != string()) { fprintf(stderr, "Possible intended match: %s\n", possible.c_str()); } valid = 0; return; } } // If there is an existing phandle, use it - property_ptr p = target->get_property("phandle"); - if (p == 0) - { - p = target->get_property("linux,phandle"); - } - if (p == 0) - { - // Otherwise insert a new phandle node - property_value v; - while (used_phandles.find(phandle) != used_phandles.end()) - { - // Note that we only don't need to - // store this phandle in the set, - // because we are monotonically - // increasing the value of phandle and - // so will only ever revisit this value - // if we have used 2^32 phandles, at - // which point our blob won't fit in - // any 32-bit system and we've done - // something badly wrong elsewhere - // already. - phandle++; - } - push_big_endian(v.byte_data, phandle++); - if (phandle_node_name == BOTH || phandle_node_name == LINUX) - { - p.reset(new property("linux,phandle")); - p->add_value(v); - target->add_property(p); - } - if (phandle_node_name == BOTH || phandle_node_name == EPAPR) - { - p.reset(new property("phandle")); - p->add_value(v); - target->add_property(p); - } - } + property_ptr p = assign_phandle(target, phandle); p->begin()->push_to_buffer(i.get().val.byte_data); assert(i.get().val.byte_data.size() == 4); } } void device_tree::parse_file(text_input_buffer &input, std::vector &roots, bool &read_header) { input.next_token(); // Read the header if (input.consume("/dts-v1/;")) { read_header = true; } input.next_token(); if (input.consume("/plugin/;")) { is_plugin = true; } input.next_token(); if (!read_header) { input.parse_error("Expected /dts-v1/; version string"); } // Read any memory reservations while (input.consume("/memreserve/")) { unsigned long long start, len; input.next_token(); // Read the start and length. if (!(input.consume_integer_expression(start) && (input.next_token(), input.consume_integer_expression(len)))) { input.parse_error("Expected size on /memreserve/ node."); } input.next_token(); input.consume(';'); reservations.push_back(reservation(start, len)); input.next_token(); } while (valid && !input.finished()) { node_ptr n; if (input.consume('/')) { input.next_token(); n = node::parse(input, string(), string_set(), string(), &defines); } else if (input.consume('&')) { input.next_token(); string name = input.parse_node_name(); input.next_token(); n = node::parse(input, std::move(name), string_set(), string(), &defines); } else { input.parse_error("Failed to find root node /."); } if (n) { roots.push_back(std::move(n)); } else { valid = false; } input.next_token(); } } template void device_tree::write(int fd) { dtb::string_table st; dtb::header head; writer head_writer; writer reservation_writer; writer struct_writer; writer strings_writer; // Build the reservation table reservation_writer.write_comment(string("Memory reservations")); reservation_writer.write_label(string("dt_reserve_map")); for (auto &i : reservations) { reservation_writer.write_comment(string("Reservation start")); reservation_writer.write_data(i.first); reservation_writer.write_comment(string("Reservation length")); reservation_writer.write_data(i.first); } // Write n spare reserve map entries, plus the trailing 0. for (uint32_t i=0 ; i<=spare_reserve_map_entries ; i++) { reservation_writer.write_data((uint64_t)0); reservation_writer.write_data((uint64_t)0); } struct_writer.write_comment(string("Device tree")); struct_writer.write_label(string("dt_struct_start")); root->write(struct_writer, st); struct_writer.write_token(dtb::FDT_END); struct_writer.write_label(string("dt_struct_end")); st.write(strings_writer); // Find the strings size before we stick padding on the end. // Note: We should possibly use a new writer for the padding. head.size_dt_strings = strings_writer.size(); // Stick the padding in the strings writer, but after the // marker indicating that it's the end. // Note: We probably should add a padding call to the writer so // that the asm back end can write padding directives instead // of a load of 0 bytes. for (uint32_t i=0 ; i(fd); } void device_tree::write_asm(int fd) { write(fd); } void device_tree::write_dts(int fd) { FILE *file = fdopen(fd, "w"); fputs("/dts-v1/;\n\n", file); if (!reservations.empty()) { const char msg[] = "/memreserve/"; fwrite(msg, sizeof(msg), 1, file); for (auto &i : reservations) { fprintf(file, " %" PRIx64 " %" PRIx64, i.first, i.second); } fputs(";\n\n", file); } putc('/', file); putc(' ', file); root->write_dts(file, 0); fclose(file); } void device_tree::parse_dtb(const string &fn, FILE *) { auto in = input_buffer::buffer_for_file(fn); if (in == 0) { valid = false; return; } input_buffer &input = *in; dtb::header h; valid = h.read_dtb(input); boot_cpu = h.boot_cpuid_phys; if (h.last_comp_version > 17) { fprintf(stderr, "Don't know how to read this version of the device tree blob"); valid = false; } if (!valid) { return; } input_buffer reservation_map = input.buffer_from_offset(h.off_mem_rsvmap, 0); uint64_t start, length; do { if (!(reservation_map.consume_binary(start) && reservation_map.consume_binary(length))) { fprintf(stderr, "Failed to read memory reservation table\n"); valid = false; return; } } while (!((start == 0) && (length == 0))); input_buffer struct_table = input.buffer_from_offset(h.off_dt_struct, h.size_dt_struct); input_buffer strings_table = input.buffer_from_offset(h.off_dt_strings, h.size_dt_strings); uint32_t token; if (!(struct_table.consume_binary(token) && (token == dtb::FDT_BEGIN_NODE))) { fprintf(stderr, "Expected FDT_BEGIN_NODE token.\n"); valid = false; return; } root = node::parse_dtb(struct_table, strings_table); if (!(struct_table.consume_binary(token) && (token == dtb::FDT_END))) { fprintf(stderr, "Expected FDT_END token after parsing root node.\n"); valid = false; return; } valid = (root != 0); } string device_tree::node_path::to_string() const { string path; auto p = begin(); auto pe = end(); if ((p == pe) || (p+1 == pe)) { return string("/"); } // Skip the first name in the path. It's always "", and implicitly / for (++p ; p!=pe ; ++p) { path += '/'; path += p->first; if (!(p->second.empty())) { path += '@'; path += p->second; } } return path; } +node_ptr +device_tree::create_fragment_wrapper(node_ptr &node, int &fragnum) +{ + // In a plugin, we can massage these non-/ root nodes into into a fragment + std::string fragment_address = "fragment@" + std::to_string(fragnum); + ++fragnum; + + std::vector symbols; + + // Intentionally left empty + node_ptr newroot = node::create_special_node("", symbols); + node_ptr wrapper = node::create_special_node("__overlay__", symbols); + + // Generate the fragment with target = <&name> + property_value v; + v.string_data = node->name; + v.type = property_value::PHANDLE; + auto prop = std::make_shared(std::string("target")); + prop->add_value(v); + symbols.push_back(prop); + + node_ptr fragment = node::create_special_node(fragment_address, symbols); + + wrapper->merge_node(node); + fragment->add_child(std::move(wrapper)); + newroot->add_child(std::move(fragment)); + return newroot; +} + +node_ptr +device_tree::generate_root(node_ptr &node, int &fragnum) +{ + + string name = node->name; + if (name == string()) + { + return std::move(node); + } + else if (!is_plugin) + { + return nullptr; + } + + return create_fragment_wrapper(node, fragnum); +} + void +device_tree::reassign_fragment_numbers(node_ptr &node, int &delta) +{ + + for (auto &c : node->child_nodes()) + { + if (c->name == std::string("fragment")) + { + int current_address = std::stoi(c->unit_address, nullptr, 16); + std::ostringstream new_address; + current_address += delta; + // It's possible that we hopped more than one somewhere, so just reset + // delta to the next in sequence. + delta = current_address + 1; + new_address << std::hex << current_address; + c->unit_address = new_address.str(); + } + } +} + +void device_tree::parse_dts(const string &fn, FILE *depfile) { auto in = input_buffer::buffer_for_file(fn); if (!in) { valid = false; return; } std::vector roots; std::unordered_set defnames; for (auto &i : defines) { defnames.insert(i.first); } text_input_buffer input(std::move(in), std::move(defnames), std::vector(include_paths), dirname(fn), depfile); bool read_header = false; + int fragnum = 0; parse_file(input, roots, read_header); switch (roots.size()) { case 0: valid = false; input.parse_error("Failed to find root node /."); return; case 1: - root = std::move(roots[0]); + root = generate_root(roots[0], fragnum); + if (!root) + { + valid = false; + input.parse_error("Failed to find root node /."); + return; + } break; default: { - root = std::move(roots[0]); + root = generate_root(roots[0], fragnum); + if (!root) + { + valid = false; + input.parse_error("Failed to find root node /."); + return; + } for (auto i=++(roots.begin()), e=roots.end() ; i!=e ; ++i) { auto &node = *i; string name = node->name; if (name == string()) { - root->merge_node(std::move(node)); + if (is_plugin) + { + // Re-assign any fragment numbers based on a delta of + // fragnum before we merge it + reassign_fragment_numbers(node, fragnum); + } + root->merge_node(node); } else { auto existing = node_names.find(name); if (existing == node_names.end()) { collect_names(); existing = node_names.find(name); } if (existing == node_names.end()) { - fprintf(stderr, "Unable to merge node: %s\n", name.c_str()); + if (is_plugin) + { + auto fragment = create_fragment_wrapper(node, fragnum); + root->merge_node(fragment); + } + else + { + fprintf(stderr, "Unable to merge node: %s\n", name.c_str()); + } } else { - existing->second->merge_node(std::move(node)); + existing->second->merge_node(node); } } } } } collect_names(); - resolve_cross_references(); + uint32_t phandle = 1; + // If we're writing symbols, go ahead and assign phandles to the entire + // tree. We'll do this before we resolve cross references, just to keep + // order semi-predictable and stable. if (write_symbols) { + assign_phandles(root, phandle); + } + resolve_cross_references(phandle); + if (write_symbols) + { std::vector symbols; // Create a symbol table. Each label in this device tree may be // referenced by other plugins, so we create a __symbols__ node inside // the root that contains mappings (properties) from label names to // paths. for (auto &s : node_paths) { property_value v; v.string_data = s.second.to_string(); v.type = property_value::STRING; string name = s.first; auto prop = std::make_shared(std::move(name)); prop->add_value(v); symbols.push_back(prop); } root->add_child(node::create_special_node("__symbols__", symbols)); // If this is a plugin, then we also need to create two extra nodes. // Internal phandles will need to be renumbered to avoid conflicts with // already-loaded nodes and external references will need to be // resolved. if (is_plugin) { // Create the fixups entry. This is of the form: // {target} = {path}:{property name}:{offset} auto create_fixup_entry = [&](fixup &i, string target) { string value = i.path.to_string(); value += ':'; value += i.prop->get_key(); value += ':'; value += std::to_string(i.prop->offset_of_value(i.val)); property_value v; v.string_data = value; v.type = property_value::STRING; auto prop = std::make_shared(std::move(target)); prop->add_value(v); return prop; }; // If we have any unresolved phandle references in this plugin, // then we must update them to 0xdeadbeef and leave a property in // the /__fixups__ node whose key is the label and whose value is // as described above. if (!unresolved_fixups.empty()) { symbols.clear(); for (auto &i : unresolved_fixups) { auto &val = i.get().val; symbols.push_back(create_fixup_entry(i, val.string_data)); val.byte_data.push_back(0xde); val.byte_data.push_back(0xad); val.byte_data.push_back(0xbe); val.byte_data.push_back(0xef); val.type = property_value::BINARY; } root->add_child(node::create_special_node("__fixups__", symbols)); } symbols.clear(); // If we have any resolved phandle references in this plugin, then - // we must leave a property in the /__local_fixups__ node whose key - // is 'fixup' and whose value is as described above. + // we must create a child in the __local_fixups__ node whose path + // matches the node path from the root and whose value contains the + // location of the reference within a property. + + // Create a local_fixups node that is initially empty. + node_ptr local_fixups = node::create_special_node("__local_fixups__", symbols); for (auto &i : fixups) { if (!i.val.is_phandle()) { continue; } - symbols.push_back(create_fixup_entry(i, "fixup")); + node *n = local_fixups.get(); + for (auto &p : i.path) + { + // Skip the implicit root + if (p.first.empty()) + { + continue; + } + bool found = false; + for (auto &c : n->child_nodes()) + { + if (c->name == p.first) + { + n = c.get(); + found = true; + break; + } + } + if (!found) + { + n->add_child(node::create_special_node(p.first, symbols)); + n = (--n->child_end())->get(); + } + } + assert(n); + property_value pv; + push_big_endian(pv.byte_data, static_cast(i.prop->offset_of_value(i.val))); + pv.type = property_value::BINARY; + auto key = i.prop->get_key(); + property_ptr prop = n->get_property(key); + // If we don't have an existing property then create one and + // use this property value + if (!prop) + { + prop = std::make_shared(std::move(key)); + n->add_property(prop); + prop->add_value(pv); + } + else + { + // If we do have an existing property value, try to append + // this value. + property_value &old_val = *(--prop->end()); + if (!old_val.try_to_merge(pv)) + { + prop->add_value(pv); + } + } } // We've iterated over all fixups, but only emit the // __local_fixups__ if we found some that were resolved internally. - if (!symbols.empty()) + if (local_fixups->child_begin() != local_fixups->child_end()) { - root->add_child(node::create_special_node("__local_fixups__", symbols)); + root->add_child(std::move(local_fixups)); } } } } bool device_tree::parse_define(const char *def) { const char *val = strchr(def, '='); if (!val) { if (strlen(def) != 0) { string name(def); defines[name]; return true; } return false; } string name(def, val-def); string name_copy = name; val++; std::unique_ptr raw(new input_buffer(val, strlen(val))); text_input_buffer in(std::move(raw), std::unordered_set(), std::vector(), string(), nullptr); property_ptr p = property::parse(in, std::move(name_copy), string_set(), false); if (p) defines[name] = p; return (bool)p; } } // namespace fdt } // namespace dtc Index: head/usr.bin/dtc/fdt.hh =================================================================== --- head/usr.bin/dtc/fdt.hh (revision 328172) +++ head/usr.bin/dtc/fdt.hh (revision 328173) @@ -1,953 +1,982 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 David Chisnall * All rights reserved. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _FDT_HH_ #define _FDT_HH_ #include #include #include #include #include #include "util.hh" #include "input_buffer.hh" namespace dtc { namespace dtb { struct output_writer; class string_table; } namespace fdt { class property; class node; /** * Type for (owned) pointers to properties. */ typedef std::shared_ptr property_ptr; /** * Owning pointer to a node. */ typedef std::unique_ptr node_ptr; /** * Map from macros to property pointers. */ typedef std::unordered_map define_map; /** * Set of strings used for label names. */ typedef std::unordered_set string_set; /** * Properties may contain a number of different value, each with a different * label. This class encapsulates a single value. */ struct property_value { /** * The label for this data. This is usually empty. */ std::string label; /** * If this value is a string, or something resolved from a string (a * reference) then this contains the source string. */ std::string string_data; /** * The data that should be written to the final output. */ byte_buffer byte_data; /** * Enumeration describing the possible types of a value. Note that * property-coded arrays will appear simply as binary (or possibly * string, if they happen to be nul-terminated and printable), and must * be checked separately. */ enum value_type { /** * This is a list of strings. When read from source, string * lists become one property value for each string, however * when read from binary we have a single property value * incorporating the entire text, with nul bytes separating the * strings. */ STRING_LIST, /** * This property contains a single string. */ STRING, /** * This is a binary value. Check the size of byte_data to * determine how many bytes this contains. */ BINARY, /** This contains a short-form address that should be replaced * by a fully-qualified version. This will only appear when * the input is a device tree source. When parsed from a * device tree blob, the cross reference will have already been * resolved and the property value will be a string containing * the full path of the target node. */ CROSS_REFERENCE, /** * This is a phandle reference. When parsed from source, the * string_data will contain the node label for the target and, * after cross references have been resolved, the binary data * will contain a 32-bit integer that should match the phandle * property of the target node. */ PHANDLE, /** * An empty property value. This will never appear on a real * property value, it is used by checkers to indicate that no * property values should exist for a property. */ EMPTY, /** * The type of this property has not yet been determined. */ UNKNOWN }; /** * The type of this property. */ value_type type; /** * Returns true if this value is a cross reference, false otherwise. */ inline bool is_cross_reference() { return is_type(CROSS_REFERENCE); } /** * Returns true if this value is a phandle reference, false otherwise. */ inline bool is_phandle() { return is_type(PHANDLE); } /** * Returns true if this value is a string, false otherwise. */ inline bool is_string() { return is_type(STRING); } /** * Returns true if this value is a string list (a nul-separated * sequence of strings), false otherwise. */ inline bool is_string_list() { return is_type(STRING_LIST); } /** * Returns true if this value is binary, false otherwise. */ inline bool is_binary() { return is_type(BINARY); } /** * Returns this property value as a 32-bit integer. Returns 0 if this * property value is not 32 bits long. The bytes in the property value * are assumed to be in big-endian format, but the return value is in * the host native endian. */ uint32_t get_as_uint32(); /** * Default constructor, specifying the label of the value. */ property_value(std::string l=std::string()) : label(l), type(UNKNOWN) {} /** * Writes the data for this value into an output buffer. */ void push_to_buffer(byte_buffer &buffer); /** * Writes the property value to the standard output. This uses the * following heuristics for deciding how to print the output: * * - If the value is nul-terminated and only contains printable * characters, it is written as a string. * - If it is a multiple of 4 bytes long, then it is printed as cells. * - Otherwise, it is printed as a byte buffer. */ void write_dts(FILE *file); /** * Tries to merge adjacent property values, returns true if it succeeds and * false otherwise. */ bool try_to_merge(property_value &other); /** * Returns the size (in bytes) of this property value. */ size_t size(); private: /** * Returns whether the value is of the specified type. If the type of * the value has not yet been determined, then this calculates it. */ inline bool is_type(value_type v) { if (type == UNKNOWN) { resolve_type(); } return type == v; } /** * Determines the type of the value based on its contents. */ void resolve_type(); /** * Writes the property value to the specified file as a quoted string. * This is used when generating DTS. */ void write_as_string(FILE *file); /** * Writes the property value to the specified file as a sequence of * 32-bit big-endian cells. This is used when generating DTS. */ void write_as_cells(FILE *file); /** * Writes the property value to the specified file as a sequence of * bytes. This is used when generating DTS. */ void write_as_bytes(FILE *file); }; /** * A value encapsulating a single property. This contains a key, optionally a * label, and optionally one or more values. */ class property { /** * The name of this property. */ std::string key; /** * Zero or more labels. */ string_set labels; /** * The values in this property. */ std::vector values; /** * Value indicating that this is a valid property. If a parse error * occurs, then this value is false. */ bool valid; /** * Parses a string property value, i.e. a value enclosed in double quotes. */ void parse_string(text_input_buffer &input); /** * Parses one or more 32-bit values enclosed in angle brackets. */ void parse_cells(text_input_buffer &input, int cell_size); /** * Parses an array of bytes, contained within square brackets. */ void parse_bytes(text_input_buffer &input); /** * Parses a reference. This is a node label preceded by an ampersand * symbol, which should expand to the full path to that node. * * Note: The specification says that the target of such a reference is * a node name, however dtc assumes that it is a label, and so we * follow their interpretation for compatibility. */ void parse_reference(text_input_buffer &input); /** * Parse a predefined macro definition for a property. */ void parse_define(text_input_buffer &input, define_map *defines); /** * Constructs a new property from two input buffers, pointing to the * struct and strings tables in the device tree blob, respectively. * The structs input buffer is assumed to have just consumed the * FDT_PROP token. */ property(input_buffer &structs, input_buffer &strings); /** * Parses a new property from the input buffer. */ property(text_input_buffer &input, std::string &&k, string_set &&l, bool terminated, define_map *defines); public: /** * Creates an empty property. */ property(std::string &&k, string_set &&l=string_set()) : key(k), labels(l), valid(true) {} /** * Copy constructor. */ property(property &p) : key(p.key), labels(p.labels), values(p.values), valid(p.valid) {} /** * Factory method for constructing a new property. Attempts to parse a * property from the input, and returns it on success. On any parse * error, this will return 0. */ static property_ptr parse_dtb(input_buffer &structs, input_buffer &strings); /** * Factory method for constructing a new property. Attempts to parse a * property from the input, and returns it on success. On any parse * error, this will return 0. */ static property_ptr parse(text_input_buffer &input, std::string &&key, string_set &&labels=string_set(), bool semicolonTerminated=true, define_map *defines=0); /** * Iterator type used for accessing the values of a property. */ typedef std::vector::iterator value_iterator; /** * Returns an iterator referring to the first value in this property. */ inline value_iterator begin() { return values.begin(); } /** * Returns an iterator referring to the last value in this property. */ inline value_iterator end() { return values.end(); } /** * Adds a new value to an existing property. */ inline void add_value(property_value v) { values.push_back(v); } /** * Returns the key for this property. */ - inline std::string get_key() + inline const std::string &get_key() { return key; } /** * Writes the property to the specified writer. The property name is a * reference into the strings table. */ void write(dtb::output_writer &writer, dtb::string_table &strings); /** * Writes in DTS format to the specified file, at the given indent * level. This will begin the line with the number of tabs specified * as the indent level and then write the property in the most * applicable way that it can determine. */ void write_dts(FILE *file, int indent); /** * Returns the byte offset of the specified property value. */ size_t offset_of_value(property_value &val); }; /** * Class encapsulating a device tree node. Nodes may contain properties and * other nodes. */ class node { public: /** * The labels for this node, if any. Node labels are used as the * targets for cross references. */ std::unordered_set labels; /** * The name of the node. */ std::string name; /** * The unit address of the node, which is optionally written after the * name followed by an at symbol. */ std::string unit_address; /** * The type for the property vector. */ typedef std::vector property_vector; /** * Iterator type for child nodes. */ typedef std::vector::iterator child_iterator; private: /** * Adaptor to use children in range-based for loops. */ struct child_range { child_range(node &nd) : n(nd) {} child_iterator begin() { return n.child_begin(); } child_iterator end() { return n.child_end(); } private: node &n; }; /** * Adaptor to use properties in range-based for loops. */ struct property_range { property_range(node &nd) : n(nd) {} property_vector::iterator begin() { return n.property_begin(); } property_vector::iterator end() { return n.property_end(); } private: node &n; }; /** * The properties contained within this node. */ property_vector props; /** * The children of this node. */ std::vector children; /** * Children that should be deleted from this node when merging. */ std::unordered_set deleted_children; /** * Properties that should be deleted from this node when merging. */ std::unordered_set deleted_props; /** * A flag indicating whether this node is valid. This is set to false * if an error occurs during parsing. */ bool valid; /** * Parses a name inside a node, writing the string passed as the last * argument as an error if it fails. */ std::string parse_name(text_input_buffer &input, bool &is_property, const char *error); /** * Constructs a new node from two input buffers, pointing to the struct * and strings tables in the device tree blob, respectively. */ node(input_buffer &structs, input_buffer &strings); /** * Parses a new node from the specified input buffer. This is called * when the input cursor is on the open brace for the start of the * node. The name, and optionally label and unit address, should have * already been parsed. */ node(text_input_buffer &input, std::string &&n, std::unordered_set &&l, std::string &&a, define_map*); /** * Creates a special node with the specified name and properties. */ node(const std::string &n, const std::vector &p); /** * Comparison function for properties, used when sorting the properties * vector. Orders the properties based on their names. */ static inline bool cmp_properties(property_ptr &p1, property_ptr &p2); /* { return p1->get_key() < p2->get_key(); } */ /** * Comparison function for nodes, used when sorting the children * vector. Orders the nodes based on their names or, if the names are * the same, by the unit addresses. */ static inline bool cmp_children(node_ptr &c1, node_ptr &c2); public: /** * Sorts the node's properties and children into alphabetical order and * recursively sorts the children. */ void sort(); /** * Returns an iterator for the first child of this node. */ inline child_iterator child_begin() { return children.begin(); } /** * Returns an iterator after the last child of this node. */ inline child_iterator child_end() { return children.end(); } /** * Returns a range suitable for use in a range-based for loop describing * the children of this node. */ inline child_range child_nodes() { return child_range(*this); } /** * Accessor for the deleted children. */ inline const std::unordered_set &deleted_child_nodes() { return deleted_children; } /** * Accessor for the deleted properties */ inline const std::unordered_set &deleted_properties() { return deleted_props; } /** * Returns a range suitable for use in a range-based for loop describing * the properties of this node. */ inline property_range properties() { return property_range(*this); } /** * Returns an iterator after the last property of this node. */ inline property_vector::iterator property_begin() { return props.begin(); } /** * Returns an iterator for the first property of this node. */ inline property_vector::iterator property_end() { return props.end(); } /** * Factory method for constructing a new node. Attempts to parse a * node in DTS format from the input, and returns it on success. On * any parse error, this will return 0. This should be called with the * cursor on the open brace of the property, after the name and so on * have been parsed. */ static node_ptr parse(text_input_buffer &input, std::string &&name, std::unordered_set &&label=std::unordered_set(), std::string &&address=std::string(), define_map *defines=0); /** * Factory method for constructing a new node. Attempts to parse a * node in DTB format from the input, and returns it on success. On * any parse error, this will return 0. This should be called with the * cursor on the open brace of the property, after the name and so on * have been parsed. */ static node_ptr parse_dtb(input_buffer &structs, input_buffer &strings); /** * Construct a new special node from a name and set of properties. */ static node_ptr create_special_node(const std::string &name, const std::vector &props); /** * Returns a property corresponding to the specified key, or 0 if this * node does not contain a property of that name. */ property_ptr get_property(const std::string &key); /** * Adds a new property to this node. */ inline void add_property(property_ptr &p) { props.push_back(p); } /** * Adds a new child to this node. */ inline void add_child(node_ptr &&n) { children.push_back(std::move(n)); } /** * Merges a node into this one. Any properties present in both are * overridden, any properties present in only one are preserved. */ - void merge_node(node_ptr other); + void merge_node(node_ptr &other); /** * Write this node to the specified output. Although nodes do not * refer to a string table directly, their properties do. The string * table passed as the second argument is used for the names of * properties within this node and its children. */ void write(dtb::output_writer &writer, dtb::string_table &strings); /** * Writes the current node as DTS to the specified file. The second * parameter is the indent level. This function will start every line * with this number of tabs. */ void write_dts(FILE *file, int indent); /** * Recursively visit this node and then its children. */ void visit(std::function); }; /** * Class encapsulating the entire parsed FDT. This is the top-level class, * which parses the entire DTS representation and write out the finished * version. */ class device_tree { public: /** * Type used for node paths. A node path is sequence of names and unit * addresses. */ class node_path : public std::vector> { public: /** * Converts this to a string representation. */ std::string to_string() const; }; /** * Name that we should use for phandle nodes. */ enum phandle_format { /** linux,phandle */ LINUX, /** phandle */ EPAPR, /** Create both nodes. */ BOTH }; private: /** * The format that we should use for writing phandles. */ - phandle_format phandle_node_name; + phandle_format phandle_node_name = EPAPR; /** * Flag indicating that this tree is valid. This will be set to false * on parse errors. */ - bool valid; + bool valid = true; /** * Type used for memory reservations. A reservation is two 64-bit * values indicating a base address and length in memory that the * kernel should not use. The high 32 bits are ignored on 32-bit * platforms. */ typedef std::pair reservation; /** * The memory reserves table. */ std::vector reservations; /** * Root node. All other nodes are children of this node. */ node_ptr root; /** * Mapping from names to nodes. Only unambiguous names are recorded, * duplicate names are stored as (node*)-1. */ std::unordered_map node_names; /** * A map from labels to node paths. When resolving cross references, * we look up referenced nodes in this and replace the cross reference * with the full path to its target. */ std::unordered_map node_paths; /** * A collection of property values that are references to other nodes. * These should be expanded to the full path of their targets. */ std::vector cross_references; /** * The location of something requiring a fixup entry. */ struct fixup { /** * The path to the node. */ node_path path; /** * The property containing the reference. */ property_ptr prop; /** * The property value that contains the reference. */ property_value &val; }; /** * A collection of property values that refer to phandles. These will * be replaced by the value of the phandle property in their * destination. */ std::vector fixups; /** * The locations of all of the values that are supposed to become phandle * references, but refer to things outside of this file. */ std::vector> unresolved_fixups; /** * The names of nodes that target phandles. */ std::unordered_set phandle_targets; /** * A collection of input buffers that we are using. These input * buffers are the ones that own their memory, and so we must preserve * them for the lifetime of the device tree. */ std::vector> buffers; /** * A map of used phandle values to nodes. All phandles must be unique, * so we keep a set of ones that the user explicitly provides in the * input to ensure that we don't reuse them. * * This is a map, rather than a set, because we also want to be able to * find phandles that were provided by the user explicitly when we are * doing checking. */ std::unordered_map used_phandles; /** * Paths to search for include files. This contains a set of * nul-terminated strings, which are not owned by this class and so * must be freed separately. */ std::vector include_paths; /** * Dictionary of predefined macros provided on the command line. */ define_map defines; /** * The default boot CPU, specified in the device tree header. */ - uint32_t boot_cpu; + uint32_t boot_cpu = 0; /** * The number of empty reserve map entries to generate in the blob. */ - uint32_t spare_reserve_map_entries; + uint32_t spare_reserve_map_entries = 0; /** * The minimum size in bytes of the blob. */ - uint32_t minimum_blob_size; + uint32_t minimum_blob_size = 0; /** * The number of bytes of padding to add to the end of the blob. */ - uint32_t blob_padding; + uint32_t blob_padding = 0; /** * Is this tree a plugin? */ - bool is_plugin; + bool is_plugin = false; /** * Visit all of the nodes recursively, and if they have labels then add * them to the node_paths and node_names vectors so that they can be * used in resolving cross references. Also collects phandle * properties that have been explicitly added. */ void collect_names_recursive(node_ptr &n, node_path &path); /** + * Assign a phandle property to a single node. The next parameter + * holds the phandle to be assigned, and will be incremented upon + * assignment. + */ + property_ptr assign_phandle(node *n, uint32_t &next); + /** * Assign phandle properties to all nodes that have been referenced and * require one. This method will recursively visit the tree starting at * the node that it is passed. */ void assign_phandles(node_ptr &n, uint32_t &next); /** * Calls the recursive version of this method on every root node. */ void collect_names(); /** * Resolves all cross references. Any properties that refer to another * node must have their values replaced by either the node path or - * phandle value. + * phandle value. The phandle parameter holds the next phandle to be + * assigned, should the need arise. It will be incremented upon each + * assignment of a phandle. */ - void resolve_cross_references(); + void resolve_cross_references(uint32_t &phandle); /** * Parses a dts file in the given buffer and adds the roots to the parsed * set. The `read_header` argument indicates whether the header has * already been read. Some dts files place the header in an include, * rather than in the top-level file. */ void parse_file(text_input_buffer &input, std::vector &roots, bool &read_header); /** * Template function that writes a dtb blob using the specified writer. * The writer defines the output format (assembly, blob). */ template void write(int fd); public: /** * Should we write the __symbols__ node (to allow overlays to be linked * against this blob)? */ bool write_symbols = false; /** * Returns the node referenced by the property. If this is a tree that * is in source form, then we have a string that we can use to index * the cross_references array and so we can just look that up. */ node *referenced_node(property_value &v); /** * Writes this FDT as a DTB to the specified output. */ void write_binary(int fd); /** * Writes this FDT as an assembly representation of the DTB to the * specified output. The result can then be assembled and linked into * a program. */ void write_asm(int fd); /** * Writes the tree in DTS (source) format. */ void write_dts(int fd); /** * Default constructor. Creates a valid, but empty FDT. */ - device_tree() : phandle_node_name(EPAPR), valid(true), - boot_cpu(0), spare_reserve_map_entries(0), - minimum_blob_size(0), blob_padding(0) {} + device_tree() {} /** * Constructs a device tree from the specified file name, referring to * a file that contains a device tree blob. */ void parse_dtb(const std::string &fn, FILE *depfile); /** + * Construct a fragment wrapper around node. This will assume that node's + * name may be used as the target of the fragment, and the contents are to + * be wrapped in an __overlay__ node. The fragment wrapper will be assigned + * fragnumas its fragment number, and fragment number will be incremented. + */ + node_ptr create_fragment_wrapper(node_ptr &node, int &fragnum); + /** + * Generate a root node from the node passed in. This is sensitive to + * whether we're in a plugin context or not, so that if we're in a plugin we + * can circumvent any errors that might normally arise from a non-/ root. + * fragnum will be assigned to any fragment wrapper generated as a result + * of the call, and fragnum will be incremented. + */ + node_ptr generate_root(node_ptr &node, int &fragnum); + /** + * Reassign any fragment numbers from this new node, based on the given + * delta. + */ + void reassign_fragment_numbers(node_ptr &node, int &delta); + /* * Constructs a device tree from the specified file name, referring to * a file that contains device tree source. */ void parse_dts(const std::string &fn, FILE *depfile); /** * Returns whether this tree is valid. */ inline bool is_valid() { return valid; } /** * Sets the format for writing phandle properties. */ inline void set_phandle_format(phandle_format f) { phandle_node_name = f; } /** * Returns a pointer to the root node of this tree. No ownership * transfer. */ inline const node_ptr &get_root() const { return root; } /** * Sets the physical boot CPU. */ void set_boot_cpu(uint32_t cpu) { boot_cpu = cpu; } /** * Sorts the tree. Useful for debugging device trees. */ void sort() { - root->sort(); + if (root) + { + root->sort(); + } } /** * Adds a path to search for include files. The argument must be a * nul-terminated string representing the path. The device tree keeps * a pointer to this string, but does not own it: the caller is * responsible for freeing it if required. */ void add_include_path(const char *path) { std::string p(path); include_paths.push_back(std::move(p)); } /** * Sets the number of empty reserve map entries to add. */ void set_empty_reserve_map_entries(uint32_t e) { spare_reserve_map_entries = e; } /** * Sets the minimum size, in bytes, of the blob. */ void set_blob_minimum_size(uint32_t s) { minimum_blob_size = s; } /** * Sets the amount of padding to add to the blob. */ void set_blob_padding(uint32_t p) { blob_padding = p; } /** * Parses a predefined macro value. */ bool parse_define(const char *def); }; } // namespace fdt } // namespace dtc #endif // !_FDT_HH_ Index: head/usr.bin/dtc/input_buffer.cc =================================================================== --- head/usr.bin/dtc/input_buffer.cc (revision 328172) +++ head/usr.bin/dtc/input_buffer.cc (revision 328173) @@ -1,1226 +1,1268 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 David Chisnall * All rights reserved. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #include "input_buffer.hh" #include #include #include #include #include #include #include #include #ifndef NDEBUG #include #endif #include #include #include #include #include #ifndef MAP_PREFAULT_READ #define MAP_PREFAULT_READ 0 #endif using std::string; namespace { /** * Subclass of input_buffer that mmap()s a file and owns the resulting memory. * When this object is destroyed, the memory is unmapped. */ struct mmap_input_buffer : public dtc::input_buffer { string fn; const string &filename() const override { return fn; } /** * Constructs a new buffer from the file passed in as a file * descriptor. */ mmap_input_buffer(int fd, string &&filename); /** * Unmaps the buffer, if one exists. */ virtual ~mmap_input_buffer(); }; /** * Input buffer read from standard input. This is used for reading device tree * blobs and source from standard input. It reads the entire input into * malloc'd memory, so will be very slow for large inputs. DTS and DTB files * are very rarely more than 10KB though, so this is probably not a problem. */ struct stream_input_buffer : public dtc::input_buffer { const string &filename() const override { static string n = ""; return n; } /** * The buffer that will store the data read from the standard input. */ std::vector b; /** * Constructs a new buffer from the standard input. */ stream_input_buffer(); }; mmap_input_buffer::mmap_input_buffer(int fd, string &&filename) : input_buffer(0, 0), fn(filename) { struct stat sb; if (fstat(fd, &sb)) { perror("Failed to stat file"); } size = sb.st_size; buffer = (const char*)mmap(0, size, PROT_READ, MAP_PRIVATE | MAP_PREFAULT_READ, fd, 0); if (buffer == MAP_FAILED) { perror("Failed to mmap file"); exit(EXIT_FAILURE); } } mmap_input_buffer::~mmap_input_buffer() { if (buffer != 0) { munmap((void*)buffer, size); } } stream_input_buffer::stream_input_buffer() : input_buffer(0, 0) { int c; while ((c = fgetc(stdin)) != EOF) { b.push_back(c); } buffer = b.data(); size = b.size(); } } // Anonymous namespace namespace dtc { void input_buffer::skip_to(char c) { while ((cursor < size) && (buffer[cursor] != c)) { cursor++; } } void text_input_buffer::skip_to(char c) { while (!finished() && (*(*this) != c)) { ++(*this); } } void text_input_buffer::skip_spaces() { if (finished()) { return; } char c = *(*this); bool last_nl = false; while ((c == ' ') || (c == '\t') || (c == '\n') || (c == '\f') || (c == '\v') || (c == '\r')) { last_nl = ((c == '\n') || (c == '\r')); ++(*this); if (finished()) { c = '\0'; } else { c = *(*this); } } // Skip C preprocessor leftovers if ((c == '#') && ((cursor == 0) || last_nl)) { skip_to('\n'); skip_spaces(); } if (consume("/include/")) { handle_include(); skip_spaces(); } } void text_input_buffer::handle_include() { bool reallyInclude = true; if (consume("if ")) { next_token(); string name = parse_property_name(); - if (defines.count(name) > 0) + if (defines.count(name) == 0) { - reallyInclude = true; + reallyInclude = false; } consume('/'); } next_token(); if (!consume('"')) { parse_error("Expected quoted filename"); return; } auto loc = location(); string file = parse_to('"'); consume('"'); if (!reallyInclude) { return; } string include_file = dir + '/' + file; auto include_buffer = input_buffer::buffer_for_file(include_file, false); if (include_buffer == 0) { for (auto i : include_paths) { include_file = i + '/' + file; include_buffer = input_buffer::buffer_for_file(include_file, false); if (include_buffer != 0) { break; } } } if (depfile) { putc(' ', depfile); fputs(include_file.c_str(), depfile); } if (!include_buffer) { loc.report_error("Unable to locate input file"); return; } input_stack.push(std::move(include_buffer)); +} + +bool text_input_buffer::read_binary_file(const std::string &filename, byte_buffer &b) +{ + bool try_include_paths = true; + string include_file; + if (filename[0] == '/') + { + include_file = filename; + // Don't try include paths if we're given an absolute path. + // Failing is better so that we don't accidentally do the wrong thing, + // but make it seem like everything is alright. + try_include_paths = false; + } + else + { + include_file = dir + '/' + filename; + } + auto include_buffer = input_buffer::buffer_for_file(include_file, false); + if (include_buffer == 0 && try_include_paths) + { + for (auto i : include_paths) + { + include_file = i + '/' + filename; + include_buffer = input_buffer::buffer_for_file(include_file, false); + if (include_buffer != 0) + { + break; + } + } + } + if (!include_buffer) + { + return false; + } + if (depfile) + { + putc(' ', depfile); + fputs(include_file.c_str(), depfile); + } + b.insert(b.begin(), include_buffer->begin(), include_buffer->end()); + return true; } input_buffer input_buffer::buffer_from_offset(int offset, int s) { if (offset < 0) { return input_buffer(); } if (s == 0) { s = size - offset; } if (offset > size) { return input_buffer(); } if (s > (size-offset)) { return input_buffer(); } return input_buffer(&buffer[offset], s); } bool input_buffer::consume(const char *str) { int len = strlen(str); if (len > size - cursor) { return false; } else { for (int i=0 ; i(&buffer[size]); outInt = strtoull(&buffer[cursor], &end, 0); if (end == &buffer[cursor]) { return false; } cursor = end - buffer; return true; } namespace { /** * Convenience typedef for the type that we use for all values. */ typedef unsigned long long valty; /** * Expression tree currently being parsed. */ struct expression { typedef text_input_buffer::source_location source_location; /** * The type that is returned when computing the result. The boolean value * indicates whether this is a valid expression. * * FIXME: Once we can use C++17, this should be `std::optional`. */ typedef std::pair result; /** * Evaluate this node, taking into account operator precedence. */ virtual result operator()() = 0; /** * Returns the precedence of this node. Lower values indicate higher * precedence. */ virtual int precedence() = 0; /** * Constructs an expression, storing the location where it was created. */ expression(source_location l) : loc(l) {} virtual ~expression() {} #ifndef NDEBUG /** * Dumps this expression to `std::cerr`, appending a newline if `nl` is * `true`. */ void dump(bool nl=false) { void *ptr = this; if (ptr == nullptr) { std::cerr << "{nullptr}\n"; return; } dump_impl(); if (nl) { std::cerr << '\n'; } } private: /** * Method that sublcasses override to implement the behaviour of `dump()`. */ virtual void dump_impl() = 0; #endif protected: source_location loc; }; /** * Expression wrapping a single integer. Leaf nodes in the expression tree. */ class terminal_expr : public expression { /** * The value that this wraps. */ valty val; /** * Evaluate. Trivially returns the value that this class wraps. */ result operator()() override { return {val, true}; } int precedence() override { return 0; } public: /** * Constructor. */ terminal_expr(source_location l, valty v) : expression(l), val(v) {} #ifndef NDEBUG void dump_impl() override { std::cerr << val; } #endif }; /** * Parenthetical expression. Exists to make the contents opaque. */ struct paren_expression : public expression { /** * The expression within the parentheses. */ expression_ptr subexpr; /** * Constructor. Takes the child expression as the only argument. */ paren_expression(source_location l, expression_ptr p) : expression(l), subexpr(std::move(p)) {} int precedence() override { return 0; } /** * Evaluate - just forwards to the underlying expression. */ result operator()() override { return (*subexpr)(); } #ifndef NDEBUG void dump_impl() override { std::cerr << " ("; subexpr->dump(); std::cerr << ") "; } #endif }; /** * Template class for unary operators. The `OpChar` template parameter is * solely for debugging and makes it easy to print the expression. The `Op` * template parameter is a function object that implements the operator that * this class provides. Most of these are provided by the `` * header. */ template class unary_operator : public expression { /** * The subexpression for this unary operator. */ expression_ptr subexpr; result operator()() override { Op op; result s = (*subexpr)(); if (!s.second) { return s; } return {op(s.first), true}; } /** * All unary operators have the same precedence. They are all evaluated * before binary expressions, but after parentheses. */ int precedence() override { return 3; } public: unary_operator(source_location l, expression_ptr p) : expression(l), subexpr(std::move(p)) {} #ifndef NDEBUG void dump_impl() override { std::cerr << OpChar; subexpr->dump(); } #endif }; /** * Abstract base class for binary operators. Allows the tree to be modified * without knowing what the operations actually are. */ struct binary_operator_base : public expression { using expression::expression; /** * The left side of the expression. */ expression_ptr lhs; /** * The right side of the expression. */ expression_ptr rhs; /** * Insert a node somewhere down the path of left children, until it would * be preempting something that should execute first. */ void insert_left(binary_operator_base *new_left) { if (lhs->precedence() < new_left->precedence()) { new_left->rhs = std::move(lhs); lhs.reset(new_left); } else { static_cast(lhs.get())->insert_left(new_left); } } }; /** * Template class for binary operators. The precedence and the operation are * provided as template parameters. */ template struct binary_operator : public binary_operator_base { result operator()() override { Op op; result l = (*lhs)(); result r = (*rhs)(); if (!(l.second && r.second)) { return {0, false}; } return {op(l.first, r.first), true}; } int precedence() override { return Precedence; } #ifdef NDEBUG /** * Constructor. Takes the name of the operator as an argument, for * debugging. Only stores it in debug mode. */ binary_operator(source_location l, const char *) : binary_operator_base(l) {} #else const char *opName; binary_operator(source_location l, const char *o) : binary_operator_base(l), opName(o) {} void dump_impl() override { lhs->dump(); std::cerr << opName; rhs->dump(); } #endif }; /** * Ternary conditional operators (`cond ? true : false`) are a special case - * there are no other ternary operators. */ class ternary_conditional_operator : public expression { /** * The condition for the clause. */ expression_ptr cond; /** * The expression that this evaluates to if the condition is true. */ expression_ptr lhs; /** * The expression that this evaluates to if the condition is false. */ expression_ptr rhs; result operator()() override { result c = (*cond)(); result l = (*lhs)(); result r = (*rhs)(); if (!(l.second && r.second && c.second)) { return {0, false}; } return c.first ? l : r; } int precedence() override { // The actual precedence of a ternary conditional operator is 15, but // its associativity is the opposite way around to the other operators, // so we fudge it slightly. return 3; } #ifndef NDEBUG void dump_impl() override { cond->dump(); std::cerr << " ? "; lhs->dump(); std::cerr << " : "; rhs->dump(); } #endif public: ternary_conditional_operator(source_location sl, expression_ptr c, expression_ptr l, expression_ptr r) : expression(sl), cond(std::move(c)), lhs(std::move(l)), rhs(std::move(r)) {} }; template struct lshift { constexpr T operator()(const T &lhs, const T &rhs) const { return lhs << rhs; } }; template struct rshift { constexpr T operator()(const T &lhs, const T &rhs) const { return lhs >> rhs; } }; template struct unary_plus { constexpr T operator()(const T &val) const { return +val; } }; // TODO: Replace with std::bit_not once we can guarantee C++14 as a baseline. template struct bit_not { constexpr T operator()(const T &val) const { return ~val; } }; template struct divmod : public binary_operator<5, T> { using binary_operator<5, T>::binary_operator; using typename binary_operator_base::result; result operator()() override { result r = (*binary_operator_base::rhs)(); if (r.second && (r.first == 0)) { expression::loc.report_error("Division by zero"); return {0, false}; } return binary_operator<5, T>::operator()(); } }; } // anonymous namespace expression_ptr text_input_buffer::parse_binary_expression(expression_ptr lhs) { next_token(); binary_operator_base *expr = nullptr; char op = *(*this); source_location l = location(); switch (op) { default: return lhs; case '+': expr = new binary_operator<6, std::plus>(l, "+"); break; case '-': expr = new binary_operator<6, std::minus>(l, "-"); break; case '%': expr = new divmod>(l, "/"); break; case '*': expr = new binary_operator<5, std::multiplies>(l, "*"); break; case '/': expr = new divmod>(l, "/"); break; case '<': switch (peek()) { default: parse_error("Invalid operator"); return nullptr; case ' ': case '(': case '0'...'9': expr = new binary_operator<8, std::less>(l, "<"); break; case '=': ++(*this); expr = new binary_operator<8, std::less_equal>(l, "<="); break; case '<': ++(*this); expr = new binary_operator<7, lshift>(l, "<<"); break; } break; case '>': switch (peek()) { default: parse_error("Invalid operator"); return nullptr; case '(': case ' ': case '0'...'9': expr = new binary_operator<8, std::greater>(l, ">"); break; case '=': ++(*this); expr = new binary_operator<8, std::greater_equal>(l, ">="); break; case '>': ++(*this); expr = new binary_operator<7, rshift>(l, ">>"); break; return lhs; } break; case '=': if (peek() != '=') { parse_error("Invalid operator"); return nullptr; } expr = new binary_operator<9, std::equal_to>(l, "=="); break; case '!': if (peek() != '=') { parse_error("Invalid operator"); return nullptr; } cursor++; expr = new binary_operator<9, std::not_equal_to>(l, "!="); break; case '&': if (peek() == '&') { expr = new binary_operator<13, std::logical_and>(l, "&&"); } else { expr = new binary_operator<10, std::bit_and>(l, "&"); } break; case '|': if (peek() == '|') { expr = new binary_operator<12, std::logical_or>(l, "||"); } else { expr = new binary_operator<14, std::bit_or>(l, "|"); } break; case '?': { consume('?'); expression_ptr true_case = parse_expression(); next_token(); if (!true_case || !consume(':')) { parse_error("Expected : in ternary conditional operator"); return nullptr; } expression_ptr false_case = parse_expression(); if (!false_case) { parse_error("Expected false condition for ternary operator"); return nullptr; } return expression_ptr(new ternary_conditional_operator(l, std::move(lhs), std::move(true_case), std::move(false_case))); } } ++(*this); next_token(); expression_ptr e(expr); expression_ptr rhs(parse_expression()); if (!rhs) { return nullptr; } expr->lhs = std::move(lhs); if (rhs->precedence() < expr->precedence()) { expr->rhs = std::move(rhs); } else { // If we're a normal left-to-right expression, then we need to insert // this as the far-left child node of the rhs expression binary_operator_base *rhs_op = static_cast(rhs.get()); rhs_op->insert_left(expr); e.release(); return rhs; } return e; } expression_ptr text_input_buffer::parse_expression(bool stopAtParen) { next_token(); unsigned long long leftVal; expression_ptr lhs; source_location l = location(); switch (*(*this)) { case '0'...'9': if (!consume_integer(leftVal)) { return nullptr; } lhs.reset(new terminal_expr(l, leftVal)); break; case '(': { consume('('); expression_ptr &&subexpr = parse_expression(); if (!subexpr) { return nullptr; } lhs.reset(new paren_expression(l, std::move(subexpr))); if (!consume(')')) { return nullptr; } if (stopAtParen) { return lhs; } break; } case '+': { consume('+'); expression_ptr &&subexpr = parse_expression(); if (!subexpr) { return nullptr; } lhs.reset(new unary_operator<'+', unary_plus>(l, std::move(subexpr))); break; } case '-': { consume('-'); expression_ptr &&subexpr = parse_expression(); if (!subexpr) { return nullptr; } lhs.reset(new unary_operator<'-', std::negate>(l, std::move(subexpr))); break; } case '!': { consume('!'); expression_ptr &&subexpr = parse_expression(); if (!subexpr) { return nullptr; } lhs.reset(new unary_operator<'!', std::logical_not>(l, std::move(subexpr))); break; } case '~': { consume('~'); expression_ptr &&subexpr = parse_expression(); if (!subexpr) { return nullptr; } lhs.reset(new unary_operator<'~', bit_not>(l, std::move(subexpr))); break; } } if (!lhs) { return nullptr; } return parse_binary_expression(std::move(lhs)); } bool text_input_buffer::consume_integer_expression(unsigned long long &outInt) { switch (*(*this)) { case '(': { expression_ptr e(parse_expression(true)); if (!e) { return false; } auto r = (*e)(); if (r.second) { outInt = r.first; return true; } return false; } case '0'...'9': return consume_integer(outInt); default: return false; } } bool input_buffer::consume_hex_byte(uint8_t &outByte) { if (!ishexdigit((*this)[0]) && !ishexdigit((*this)[1])) { return false; } outByte = (digittoint((*this)[0]) << 4) | digittoint((*this)[1]); cursor += 2; return true; } text_input_buffer& text_input_buffer::next_token() { auto &self = *this; int start; do { start = cursor; skip_spaces(); if (finished()) { return self; } // Parse /* comments if (*self == '/' && peek() == '*') { // eat the start of the comment ++self; ++self; do { // Find the ending * of */ while ((*self != '\0') && (*self != '*') && !finished()) { ++self; } // Eat the * ++self; } while ((*self != '\0') && (*self != '/') && !finished()); // Eat the / ++self; } // Parse // comments if ((*self == '/' && peek() == '/')) { // eat the start of the comment ++self; ++self; // Find the ending of the line while (*self != '\n' && !finished()) { ++self; } // Eat the \n ++self; } } while (start != cursor); return self; } void text_input_buffer::parse_error(const char *msg) { if (input_stack.empty()) { fprintf(stderr, "Error: %s\n", msg); return; } input_buffer &b = *input_stack.top(); parse_error(msg, b, b.cursor); } void text_input_buffer::parse_error(const char *msg, input_buffer &b, int loc) { int line_count = 1; int line_start = 0; int line_end = loc; if (loc < 0 || loc > b.size) { return; } for (int i=loc ; i>0 ; --i) { if (b.buffer[i] == '\n') { line_count++; if (line_start == 0) { line_start = i+1; } } } for (int i=loc+1 ; i= 'a') && (c <= 'z')) || ((c >= 'A') && (c <= 'Z')); } }; /** * Check whether a character is in the set allowed for node names. This is a * class so that it can be used with a template function for parsing strings. */ struct is_node_name_character { static inline bool check(const char c) { switch(c) { default: return false; case 'a'...'z': case 'A'...'Z': case '0'...'9': case ',': case '.': case '+': case '-': case '_': return true; } } }; /** * Check whether a character is in the set allowed for property names. This is * a class so that it can be used with a template function for parsing strings. */ struct is_property_name_character { static inline bool check(const char c) { switch(c) { default: return false; case 'a'...'z': case 'A'...'Z': case '0'...'9': case ',': case '.': case '+': case '-': case '_': case '#': return true; } } }; template string parse(text_input_buffer &s) { std::vector bytes; for (char c=*s ; T::check(c) ; c=*(++s)) { bytes.push_back(c); } return string(bytes.begin(), bytes.end()); } } string text_input_buffer::parse_node_name() { return parse(*this); } string text_input_buffer::parse_property_name() { return parse(*this); } string text_input_buffer::parse_node_or_property_name(bool &is_property) { if (is_property) { return parse_property_name(); } std::vector bytes; for (char c=*(*this) ; is_node_name_character::check(c) ; c=*(++(*this))) { bytes.push_back(c); } for (char c=*(*this) ; is_property_name_character::check(c) ; c=*(++(*this))) { bytes.push_back(c); is_property = true; } return string(bytes.begin(), bytes.end()); } string input_buffer::parse_to(char stop) { std::vector bytes; for (char c=*(*this) ; c != stop ; c=*(++(*this))) { bytes.push_back(c); } return string(bytes.begin(), bytes.end()); } string text_input_buffer::parse_to(char stop) { std::vector bytes; for (char c=*(*this) ; c != stop ; c=*(++(*this))) { if (finished()) { break; } bytes.push_back(c); } return string(bytes.begin(), bytes.end()); } char text_input_buffer::peek() { return (*input_stack.top())[1]; } std::unique_ptr input_buffer::buffer_for_file(const string &path, bool warn) { if (path == "-") { std::unique_ptr b(new stream_input_buffer()); return b; } int source = open(path.c_str(), O_RDONLY); if (source == -1) { if (warn) { fprintf(stderr, "Unable to open file '%s'. %s\n", path.c_str(), strerror(errno)); } return 0; } struct stat st; if (fstat(source, &st) == 0 && S_ISDIR(st.st_mode)) { if (warn) { fprintf(stderr, "File %s is a directory\n", path.c_str()); } close(source); return 0; } std::unique_ptr b(new mmap_input_buffer(source, string(path))); close(source); return b; } } // namespace dtc Index: head/usr.bin/dtc/input_buffer.hh =================================================================== --- head/usr.bin/dtc/input_buffer.hh (revision 328172) +++ head/usr.bin/dtc/input_buffer.hh (revision 328173) @@ -1,539 +1,554 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) 2013 David Chisnall * All rights reserved. * * This software was developed by SRI International and the University of * Cambridge Computer Laboratory under DARPA/AFRL contract (FA8750-10-C-0237) * ("CTSRD"), as part of the DARPA CRASH research programme. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _INPUT_BUFFER_HH_ #define _INPUT_BUFFER_HH_ #include "util.hh" #include #include #include #include namespace dtc { namespace { struct expression; typedef std::unique_ptr expression_ptr; } /** * Class encapsulating the input file. Can be used as a const char*, but has * range checking. Attempting to access anything out of range will return a 0 * byte. The input buffer can be cheaply copied, without copying the * underlying memory, however it is the user's responsibility to ensure that * such copies do not persist beyond the lifetime of the underlying memory. * * This also contains methods for reporting errors and for consuming the token * stream. */ class input_buffer { friend class text_input_buffer; protected: /** * The buffer. This class doesn't own the buffer, but the * mmap_input_buffer subclass does. */ const char* buffer; /** * The size of the buffer. */ int size; private: /** * The current place in the buffer where we are reading. This class * keeps a separate size, pointer, and cursor so that we can move * forwards and backwards and still have checks that we haven't fallen * off either end. */ int cursor; /** * Private constructor. This is used to create input buffers that * refer to the same memory, but have different cursors. */ input_buffer(const char* b, int s, int c) : buffer(b), size(s), cursor(c) {} public: /** * Returns the file name associated with this buffer. */ virtual const std::string &filename() const { static std::string s; return s; } static std::unique_ptr buffer_for_file(const std::string &path, bool warn=true); /** * Skips all characters in the input until the specified character is * encountered. */ void skip_to(char); /** * Parses up to a specified character and returns the intervening * characters as a string. */ std::string parse_to(char); /** * Return whether all input has been consumed. */ bool finished() { return cursor >= size; } /** * Virtual destructor. Does nothing, but exists so that subclasses * that own the memory can run cleanup code for deallocating it. */ virtual ~input_buffer() {}; /** * Constructs an empty buffer. */ input_buffer() : buffer(0), size(0), cursor(0) {} /** * Constructs a new buffer with a specified memory region and size. */ input_buffer(const char* b, int s) : buffer(b), size(s), cursor(0){} /** * Returns a new input buffer referring into this input, clamped to the * specified size. If the requested buffer would fall outside the * range of this one, then it returns an empty buffer. * * The returned buffer shares the same underlying storage as the * original. This is intended to be used for splitting up the various * sections of a device tree blob. Requesting a size of 0 will give a * buffer that extends to the end of the available memory. */ input_buffer buffer_from_offset(int offset, int s=0); /** * Dereferencing operator, allows the buffer to be treated as a char* * and dereferenced to give a character. This returns a null byte if * the cursor is out of range. */ inline char operator*() { if (cursor >= size) { return '\0'; } if (cursor < 0) { return '\0'; } return buffer[cursor]; } /** * Array subscripting operator, returns a character at the specified * index offset from the current cursor. The offset may be negative, * to reread characters that have already been read. If the current * cursor plus offset is outside of the range, this returns a nul * byte. */ inline char operator[](int offset) { if (cursor + offset >= size) { return '\0'; } if (cursor + offset < 0) { return '\0'; } return buffer[cursor + offset]; } /** * Increments the cursor, iterating forward in the buffer. */ inline input_buffer &operator++() { cursor++; return *this; } + const char *begin() + { + return buffer; + } + const char *end() + { + return buffer + size; + } /** * Consumes a character. Moves the cursor one character forward if the * next character matches the argument, returning true. If the current * character does not match the argument, returns false. */ inline bool consume(char c) { if (*(*this) == c) { ++(*this); return true; } return false; } /** * Consumes a string. If the (null-terminated) string passed as the * argument appears in the input, advances the cursor to the end and * returns true. Returns false if the string does not appear at the * current point in the input. */ bool consume(const char *str); /** * Reads an integer in base 8, 10, or 16. Returns true and advances * the cursor to the end of the integer if the cursor points to an * integer, returns false and does not move the cursor otherwise. * * The parsed value is returned via the argument. */ bool consume_integer(unsigned long long &outInt); /** * Reads an arithmetic expression (containing any of the normal C * operators), evaluates it, and returns the result. */ bool consume_integer_expression(unsigned long long &outInt); /** * Consumes two hex digits and return the resulting byte via the first * argument. If the next two characters are hex digits, returns true * and advances the cursor. If not, then returns false and leaves the * cursor in place. */ bool consume_hex_byte(uint8_t &outByte); /** * Template function that consumes a binary value in big-endian format * from the input stream. Returns true and advances the cursor if * there is a value of the correct size. This function assumes that * all values must be natively aligned, and so advances the cursor to * the correct alignment before reading. */ template bool consume_binary(T &out) { int align = 0; int type_size = sizeof(T); if (cursor % type_size != 0) { align = type_size - (cursor % type_size); } if (size < cursor + align + type_size) { return false; } cursor += align; assert(cursor % type_size == 0); out = 0; for (int i=0 ; i inline bool input_buffer::consume_binary(uint8_t &out) { if (size < cursor + 1) { return false; } out = buffer[cursor++]; return true; } /** * An input buffer subclass used for parsing DTS files. This manages a stack * of input buffers to handle /input/ operations. */ class text_input_buffer { std::unordered_set defines; /** * The cursor is the input into the input stream where we are currently reading. */ int cursor = 0; /** * The current stack of includes. The current input is always from the top * of the stack. */ std::stack> input_stack; /** * */ const std::vector include_paths; /** * Reads forward past any spaces. The DTS format is not whitespace * sensitive and so we want to scan past whitespace when reading it. */ void skip_spaces(); /** * Returns the character immediately after the current one. * * This method does not look between files. */ char peek(); /** * If a /include/ token is encountered, then look up the corresponding * input file, push it onto the input stack, and continue. */ void handle_include(); /** * The base directory for this file. */ const std::string dir; /** * The file where dependencies should be output. */ FILE *depfile; public: /** * Construct a new text input buffer with the specified buffer as the start * of parsing and the specified set of input paths for handling new * inclusions. */ text_input_buffer(std::unique_ptr &&b, std::unordered_set &&d, std::vector &&i, const std::string directory, FILE *deps) : defines(d), include_paths(i), dir(directory), depfile(deps) { input_stack.push(std::move(b)); } /** * Skips all characters in the input until the specified character is * encountered. */ void skip_to(char); /** * Parse an expression. If `stopAtParen` is set, then only parse a number * or a parenthetical expression, otherwise assume that either is the * left-hand side of a binary expression and try to parse the right-hand * side. */ expression_ptr parse_expression(bool stopAtParen=false); /** * Parse a binary expression, having already parsed the right-hand side. */ expression_ptr parse_binary_expression(expression_ptr lhs); /** * Return whether all input has been consumed. */ bool finished() { return input_stack.empty() || ((input_stack.size() == 1) && input_stack.top()->finished()); } /** * Dereferencing operator. Returns the current character in the top input buffer. */ inline char operator*() { if (input_stack.empty()) { return 0; } return *(*input_stack.top()); } /** * Increments the cursor, iterating forward in the buffer. */ inline text_input_buffer &operator++() { if (input_stack.empty()) { return *this; } cursor++; auto &top = *input_stack.top(); ++top; if (top.finished()) { input_stack.pop(); } return *this; } /** * Consumes a character. Moves the cursor one character forward if the * next character matches the argument, returning true. If the current * character does not match the argument, returns false. */ inline bool consume(char c) { if (*(*this) == c) { ++(*this); return true; } return false; } /** * Consumes a string. If the (null-terminated) string passed as the * argument appears in the input, advances the cursor to the end and * returns true. Returns false if the string does not appear at the * current point in the input. * * This method does not scan between files. */ bool consume(const char *str) { if (input_stack.empty()) { return false; } return input_stack.top()->consume(str); } /** * Reads an integer in base 8, 10, or 16. Returns true and advances * the cursor to the end of the integer if the cursor points to an * integer, returns false and does not move the cursor otherwise. * * The parsed value is returned via the argument. * * This method does not scan between files. */ bool consume_integer(unsigned long long &outInt) { if (input_stack.empty()) { return false; } return input_stack.top()->consume_integer(outInt); } /** * Reads an arithmetic expression (containing any of the normal C * operators), evaluates it, and returns the result. */ bool consume_integer_expression(unsigned long long &outInt); /** * Consumes two hex digits and return the resulting byte via the first * argument. If the next two characters are hex digits, returns true * and advances the cursor. If not, then returns false and leaves the * cursor in place. * * This method does not scan between files. */ bool consume_hex_byte(uint8_t &outByte) { if (input_stack.empty()) { return false; } return input_stack.top()->consume_hex_byte(outByte); } /** * Returns the longest string in the input buffer starting at the * current cursor and composed entirely of characters that are valid in * node names. */ std::string parse_node_name(); /** * Returns the longest string in the input buffer starting at the * current cursor and composed entirely of characters that are valid in * property names. */ std::string parse_property_name(); /** * Parses either a node or a property name. If is_property is true on * entry, then only property names are parsed. If it is false, then it * will be set, on return, to indicate whether the parsed name is only * valid as a property. */ std::string parse_node_or_property_name(bool &is_property); /** * Parses up to a specified character and returns the intervening * characters as a string. */ std::string parse_to(char); /** * Advances the cursor to the start of the next token, skipping * comments and whitespace. If the cursor already points to the start * of a token, then this function does nothing. */ text_input_buffer &next_token(); /** * Location in the source file. This should never be interpreted by * anything other than error reporting functions of this class. It will * eventually become something more complex than an `int`. */ class source_location { friend class text_input_buffer; /** * The text buffer object that included `b`. */ text_input_buffer &buffer; /** * The underlying buffer that contains this location. */ std::shared_ptr b; /** * The offset within the current buffer of the source location. */ int cursor; source_location(text_input_buffer &buf) : buffer(buf), b(buf.input_stack.empty() ? nullptr : buf.input_stack.top()), cursor(b ? b->cursor : 0) {} public: /** * Report an error at this location. */ void report_error(const char *msg) { if (b) { buffer.parse_error(msg, *b, cursor); } else { buffer.parse_error(msg); } } }; /** * Returns the current source location. */ source_location location() { return { *this }; } /** * Prints a message indicating the location of a parse error. */ void parse_error(const char *msg); + /** + * Reads the contents of a binary file into `b`. The file name is assumed + * to be relative to one of the include paths. + * + * Returns true if the file exists and can be read, false otherwise. + */ + bool read_binary_file(const std::string &filename, byte_buffer &b); private: /** * Prints a message indicating the location of a parse error, given a * specified location. This is used when input has already moved beyond * the location that caused the failure. */ void parse_error(const char *msg, input_buffer &b, int loc); }; } // namespace dtc #endif // !_INPUT_BUFFER_HH_