linaro-image-tools-2016.05.orig/0000755000175000017500000000000012724030055015722 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/org.linaro.linaro-image-tools.policy0000644000175000017500000000143712724020110024712 0ustar voipiovoipio Linaro http://www.linaro.org Create a disk image by combining an OS image and a hardware pack. Authentication is required to run linaro-media-create as $(user) no auth_admin /usr/bin/linaro-media-create linaro-image-tools-2016.05.orig/do-release0000755000175000017500000000356712724020110017672 0ustar voipiovoipio#!/bin/sh # tag tree, generate ChangeLog and roll a release tarball set -e self="$(basename "$0")" usage() { echo "Usage: $self []" } log() { echo "$*" >&2 } log_i() { log "I:" "$@" } die() { log "E:" "$@" exit 1 } set_version() { log_i "Setting version to $1 in __version__.py" sed -i "s/^__version__ =.*/__version__ = \"$1\"/" linaro_image_tools/__version__.py } version="$1" old_version="$2" if [ -z "$version" ]; then usage >&2 exit 1 fi log_i "Checking tree status" status=`git status --short` if [ -n "$status" ]; then die "Tree is dirty according to git status" fi log_i "Running tests" if ! python -m testtools.run linaro_image_tools.tests.test_suite; then die "Testsuite doesn't pass" fi log_i "Removing test repository data" rm -rf .testrepository/ if git tag | awk '{print $1}' | grep -qFx "$version"; then die "Tag $version already exists" fi set_version $version log_i "Committing $version" git commit -a -m "Release $version." log_i "Creating tag $version" git tag "$version" log_i "Pushing changes and tag" git push origin master git push --tags log_i "Generating ChangeLog" if [ -z "$old_version"]; then git log --date=short --no-merges --format=format:"%ad %aN <%aE>%n%n %s%n" >ChangeLog else # If we have also the old revision, we can generate changelog only for # those tags, instead of a full changelog since the beginning of time. git log --date=short --no-merges --format=format:"%ad %aN <%aE>%n%n %s%n" >ChangeLog "$old_version".."$version" fi log_i "Creating release tarball in parent directory" ./setup.py sdist -d .. log_i "Cleaning up" rm -f ChangeLog MANIFEST log_i "Signing tarball" gpg --armor --sign --detach-sig "../linaro-image-tools-$version.tar.gz" set_version "$version.1" log_i "Committing $version.1" git commit -a -m "Post-release version bump to $version.1." linaro-image-tools-2016.05.orig/.testr.conf0000644000175000017500000000015612724020110020001 0ustar voipiovoipio[DEFAULT] test_command=python -m subunit.run $IDLIST test_id_list_default=linaro_image_tools.tests.test_suite linaro-image-tools-2016.05.orig/COPYING.GPL-30000644000175000017500000010451312724020110017531 0ustar voipiovoipio GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . linaro-image-tools-2016.05.orig/linaro-hwpack-create0000755000175000017500000000437112724020110021644 0ustar voipiovoipio#!/usr/bin/python # Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import argparse import sys from linaro_image_tools.hwpack.builder import ( ConfigFileMissing, HardwarePackBuilder) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ if __name__ == '__main__': parser = argparse.ArgumentParser(version='%(prog)s ' + __version__) parser.add_argument( "CONFIG_FILE", help="The configuration file to take the hardware pack information " "from.") parser.add_argument( "VERSION", help="The version of the hardware pack to create.") parser.add_argument( "--local-deb", action="append", dest="local_debs", default=[], metavar="LOCAL_DEB", help=("Include LOCAL_DEB in the hardware pack, even if it's an older " "version than a package that would be otherwise installed. " "Can be used more than once.")) parser.add_argument("--debug", action="store_true") parser.add_argument("--backports", action="store_true", help="Level the pin priority for the backports repositories.") args = parser.parse_args() logger = get_logger(debug=args.debug) try: builder = HardwarePackBuilder(args.CONFIG_FILE, args.VERSION, args.local_debs, args.backports) except ConfigFileMissing, e: logger.error(str(e)) sys.exit(1) builder.build() linaro-image-tools-2016.05.orig/linaro-hwpack-append0000755000175000017500000002035312724020110021646 0ustar voipiovoipio#!/usr/bin/env python # Copyright (C) 2013 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. It adds the feature # to append a debian package into the given hwpack. # We might need to change the manifest and Packages file in the # future to match the hardware pack v2 changes when available. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . # import argparse import atexit import os import sys import tarfile import tempfile import shutil from debian.arfile import ArError from linaro_image_tools.hwpack.packages import ( get_packages_file, FetchedPackage ) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ logger = None def setup_args_parser(): """Setup the argument parsing. :return The parsed arguments. """ description = "Adds new packages inside a hardware pack." parser = argparse.ArgumentParser(version=__version__, description=description) parser.add_argument("-d", "--debug", action="store_true") parser.add_argument("-t", "--hwpack", required=True, help="The hardware pack to modify") parser.add_argument("-p", "--package", action="append", required=True, help="The debian package to append. Can be repeated " "multiple times.") parser.add_argument("-i", "--inplace", action="store_true", help="Add the packages in place, without creating a " "new hardware pack.") return parser.parse_args() def validate_args(args): """Verify the arguments passed on the command line. :param args: The arguments passed. """ hwpack_path = os.path.abspath(args.hwpack) if not os.path.isfile(hwpack_path): logger.error("Error: provided hardware pack file does not exists, or " "is not a file: {0}.".format(args.hwpack)) sys.exit(1) if not tarfile.is_tarfile(hwpack_path): logger.error("Error: cannot read hardware pack file. Make sure it " "is a supported tar archive.") sys.exit(1) for package in args.package: if not os.path.isfile(os.path.abspath(package)): logger.error("Error: provided package to add does not exists, or " "is not a file: {0}.".format(package)) sys.exit(1) def modify_manifest_file(debpackage_info, hwpack_dir): """Modify the manifest file to include a new package entry. :param debpackage_info: The info to write. :param hwpack_dir: Where the manifest file is located. """ debpack_manifest = os.path.join(hwpack_dir, "manifest") new_debpack_line = '{0}={1}\n'.format(debpackage_info.name, debpackage_info.version) logger.debug("Manifest line: {0}".format(new_debpack_line)) with open(debpack_manifest, "a") as manifest: manifest.write(new_debpack_line) def modify_packages_file(debpack_info, pkgs_dir): """Modify the Packages file to include a new package entry. :param debpack_info: The info to be written. :param pkgs_dir: The directory with the Packages file. """ debpack_Packages_fname = os.path.join(pkgs_dir, "Packages") package_info = get_packages_file([debpack_info]).strip() logger.debug("Packages line:\n{0}".format(package_info)) with open(debpack_Packages_fname, "a") as packages_file: packages_file.write("{0}\n\n".format(package_info)) def has_matching_package(pkg_to_search, dir_to_search): """Search for a matching file name in the provided directory. :param pkg_to_search: The package whose name will be matched. :param dir_to_search: Where to search for a matching name. """ logger.debug("Searching matching packages") package_found = False for pkg in os.listdir(dir_to_search): if os.path.basename(pkg_to_search) == os.path.basename(pkg): package_found = True break return package_found def add_packages_to_hwpack(hwpack, packages_to_add, inplace): """Add the provided packages to the hardware pack. Each package to add will be checked against the already available packages: if a similar one is found (it just need to have the same name), it will be skipped. :param hwpack: The hardware pack where to add the new files. :param packagess_to_add: List of package to add. """ hwpack = os.path.abspath(hwpack) tempdir = tempfile.mkdtemp() pkgs_dir = os.path.join(tempdir, 'pkgs') # Cleanup once done. atexit.register(shutil.rmtree, tempdir) # Unfortunately we cannot operate in memory, Python tar library does not # allow adding files with compressed tarballs. We have to extract it. logger.info("Opening hardware pack {0}...".format(hwpack)) logger.debug("Extracting hardware pack in {0}".format(tempdir)) with tarfile.open(hwpack, "r|gz") as tar_file: tar_file.extractall(tempdir) if not os.path.isdir(pkgs_dir): logger.error("Error: tar file does not include packages directory.") sys.exit(1) # Flag to check if we really need to save the new hwpack. save_hwpack = False for debpackage in packages_to_add: debpackage_path = os.path.abspath(debpackage) if has_matching_package(debpackage_path, pkgs_dir): logger.warning("Found similar package in the tar archive: file " "will not be added.") continue if os.path.isfile(debpackage_path): logger.info("Adding file {0}...".format(debpackage)) try: debpackage_info = FetchedPackage.from_deb(debpackage_path) except ArError: logger.warning("File {0} is invalid, skipping " "it.".format(debpackage)) continue if debpackage_info: logger.debug("Package info data:\n{0}".format(debpackage_info)) modify_manifest_file(debpackage_info, tempdir) modify_packages_file(debpackage_info, pkgs_dir) shutil.copy2(debpackage_path, pkgs_dir) save_hwpack |= True else: logger.warning("Unable to find valid info for package " "{0}.".format(debpackage)) else: logger.warning("File {0} does not exists, skipping " "it.".format(debpackage)) if save_hwpack: if inplace: logger.info("Saving hardware pack {0}...".format(hwpack)) with tarfile.open(hwpack, "w|gz") as tar_file: tar_file.add(tempdir, arcname="") else: save_dir = os.path.dirname(hwpack) # Retrieve the file name without the extensions, and create a new # file name. root_ext, ext1 = os.path.splitext(os.path.basename(hwpack)) root, ext2 = os.path.splitext(root_ext) root += "_new" new_file_name = root + ext2 + ext1 save_file = os.path.join(save_dir, new_file_name) logger.info("Saving new hardware pack {0}...".format(save_file)) with tarfile.open(save_file, "w|gz") as tar_file: tar_file.add(tempdir, arcname="") logger.info("New packages added successfully.") else: logger.info("No packages added. Exiting.") def hwpack_append(): args = setup_args_parser() global logger logger = get_logger(debug=args.debug) validate_args(args) add_packages_to_hwpack(args.hwpack, args.package, args.inplace) if __name__ == '__main__': hwpack_append() linaro-image-tools-2016.05.orig/.gitignore0000644000175000017500000000003412724020110017676 0ustar voipiovoipio.testrepository _trial_temp linaro-image-tools-2016.05.orig/linaro-android-media-create0000755000175000017500000001570012724020110023062 0ustar voipiovoipio#!/usr/bin/env python # Copyright (C) 2011 Linaro # # Author: Jeremy Chang # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import atexit import os import sys import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.android_boards import ( get_board_config, ) from linaro_image_tools.media_create.check_device import ( confirm_device_selection_and_ensure_it_is_ready) from linaro_image_tools.media_create.partitions import ( Media, setup_android_partitions, partition_mounted, ) from linaro_image_tools.media_create.rootfs import populate_partition from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_android_binary_tarball ) from linaro_image_tools.media_create import get_android_args_parser from linaro_image_tools.utils import ( additional_android_option_checks, android_hwpack_in_boot_tarball, ensure_command, get_logger, disable_automount, enable_automount, ) # Just define the global variables TMP_DIR = None BOOT_DISK = None CACHE_DISK = None SDCARD_DISK = None # Registered as the first atexit handler as we want this to be the last # handler to execute. @atexit.register def cleanup_tempdir(): """Remove TEMP_DIR with all its contents. Before doing so, make sure DISKs are not mounted. """ devnull = open('/dev/null', 'w') # ignore non-zero return codes for disk in BOOT_DISK, CACHE_DISK, \ SDCARD_DISK: if disk is not None: try: cmd_runner.run(['umount', disk], stdout=devnull, stderr=devnull, as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass # Remove TMP_DIR as root because some files written there are # owned by root. if TMP_DIR is not None: cmd_runner.run(['rm', '-rf', TMP_DIR], as_root=True).wait() def ensure_required_commands(args): """Ensure we have the commands that we know are going to be used.""" required_commands = [ 'mkfs.vfat', 'sfdisk', 'mkimage', 'parted'] for command in required_commands: ensure_command(command) if __name__ == '__main__': parser = get_android_args_parser() args = parser.parse_args() logger = get_logger(debug=args.debug) additional_android_option_checks(args) # If --help was specified this won't execute. # Create temp dir and initialize rest of path vars. TMP_DIR = tempfile.mkdtemp() BOOT_DIR = os.path.join(TMP_DIR, 'boot') SYSTEM_DIR = os.path.join(TMP_DIR, 'system') DATA_DIR = os.path.join(TMP_DIR, 'data') BOOT_DISK = os.path.join(TMP_DIR, 'boot-disc') CACHE_DISK = os.path.join(TMP_DIR, 'cache-disc') SDCARD_DISK = os.path.join(TMP_DIR, 'sdcard-disc') if args.dev == 'iMX53': # XXX: remove this and the corresponding entry in android_board_configs logger.warning("DEPRECATION WARNING: iMX53 is deprecated, please " "use mx53loco.") ensure_required_commands(args) # Do this by default, disable automount options and re-enable them at exit. disable_automount() atexit.register(enable_automount) media = Media(args.device) if media.is_block_device: if not confirm_device_selection_and_ensure_it_is_ready(args.device): sys.exit(1) elif not args.should_create_partitions: logger.error("Do not use --no-part in conjunction with --image_file.") sys.exit(1) cmd_runner.run(['mkdir', '-p', BOOT_DIR]).wait() cmd_runner.run(['mkdir', '-p', SYSTEM_DIR]).wait() cmd_runner.run(['mkdir', '-p', DATA_DIR]).wait() unpack_android_binary_tarball(args.boot, BOOT_DIR) board_config = get_board_config(args.dev) hwpack_exists, config_file = android_hwpack_in_boot_tarball(BOOT_DIR) if not args.hwpack and not hwpack_exists: # No hwpack in the boot tarball nor provided on the command line. logger.warning("No hwpack found in the boot tarball nor passed on " "the command line. Default values will be used.") elif not args.hwpack and hwpack_exists: board_config.from_file(config_file) elif args.hwpack: logger.warning("Values from the hwpack provided on the command line " "will be used.") board_config.from_file(args.hwpack) board_config.add_boot_args(args.extra_boot_args) board_config.add_boot_args_from_file(args.extra_boot_args_file) # Create partitions boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition = setup_android_partitions( \ board_config, media, args.image_size, args.boot_label, args.should_create_partitions, args.should_align_boot_part) board_config.populate_raw_partition(args.device, BOOT_DIR) populate_partition(BOOT_DIR + "/boot", BOOT_DISK, boot_partition) board_config.populate_boot_script(boot_partition, BOOT_DISK, args.consoles) with partition_mounted(boot_partition, BOOT_DISK): board_config.install_boot_loader(args.device, BOOT_DISK) if args.system: with partition_mounted(system_partition, SYSTEM_DIR): unpack_android_binary_tarball(args.system, TMP_DIR) elif args.systemimage : cmd_runner.run( [ 'e2label', args.systemimage, "system"], stderr=open('/dev/null', 'w'), as_root=True).wait() cmd_runner.run( [ 'dd', 'if=%s' % args.systemimage, 'of=%s' % system_partition], stderr=open('/dev/null', 'w'), as_root=True).wait() else: #should not reach here pass if args.userdata: with partition_mounted(data_partition, DATA_DIR): unpack_android_binary_tarball(args.userdata, TMP_DIR) elif args.userdataimage: cmd_runner.run( [ 'e2label', args.userdataimage, "userdata"], stderr=open('/dev/null', 'w'), as_root=True).wait() cmd_runner.run( [ 'dd', 'if=%s' % args.userdataimage, 'of=%s' % data_partition], stderr=open('/dev/null', 'w'), as_root=True).wait() else: #should not reach here pass print "Done creating Linaro Android image on %s" % args.device linaro-image-tools-2016.05.orig/linaro-hwpack-convert0000755000175000017500000000413612724020110022060 0ustar voipiovoipio#!/usr/bin/python # Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import argparse import sys from linaro_image_tools.hwpack.hwpack_convert import ( HwpackConverter, HwpackConverterException, check_and_validate_args, ) from linaro_image_tools.utils import get_logger from linaro_image_tools.__version__ import __version__ if __name__ == '__main__': parser = argparse.ArgumentParser(version='%(prog)s ' + __version__) parser.add_argument("CONFIG_FILE", help="The configuration file to convert.") parser.add_argument("--out", help="The output file name to write. If none is " "given, the input file name (and path) will be " "used with the '.yaml' suffix.") parser.add_argument("--debug", action="store_true") args = parser.parse_args() logger = get_logger(debug=args.debug) try: input_file, output_file = check_and_validate_args(args) logger.info("Converting '%s' into new YAML format..." % input_file) converter = HwpackConverter(input_file, output_file) except HwpackConverterException, e: logger.error(str(e)) sys.exit(1) converter.convert() logger.info("File '%s' converted in '%s'." % (input_file, output_file)) linaro-image-tools-2016.05.orig/COPYING0000644000175000017500000000073112724020110016745 0ustar voipiovoipioFormat-Specification: http://svn.debian.org/wsvn/dep/web/deps/dep5.mdwn?op=file&rev=135 Name: Linaro Image Tools Maintainer: Linaro Infrastructure team Source: https://launchpad.net/linaro-image-tools Files: linaro-hwpack-create linaro-hwpack-install linaro_image_tools/hwpack/* Copyright: 2010, 2011 Linaro Limited License: GPL-2+ Files: linaro-media-create linaro_image_tools/media_create/* Copyright: 2010, 2011 Linaro Limited License: GPL-3+ linaro-image-tools-2016.05.orig/COPYING.GPL-20000644000175000017500000004325412724020110017534 0ustar voipiovoipio GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. linaro-image-tools-2016.05.orig/linaro-hwpack-install0000755000175000017500000002613312724020110022047 0ustar voipiovoipio#!/bin/sh # linaro-hwpack-install - Install a Linaro Hardware Pack. # This script is meant to run inside a chroot. It must not depend on anything # that's not in there. # TODO: When upgrading to a newer hwpack, make sure packages and apt sources # that are no longer needed are removed. # Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. set -e if [ -n "${TMPDIR+x}" ]; then echo -e "\nWARNING: TMPDIR variable is set. It will be unset because in chroot environment it likely doesn't exist and can lead to error messages.\n" unset TMPDIR fi LOCKFILE="/var/lock/hwpack" TEMP_DIR=$(mktemp -d) HWPACK_DIR="${TEMP_DIR}/unpacked" INSTALL_LATEST="no" FORCE_YES="no" SOURCES_LIST_FILE="${TEMP_DIR}/sources.list" APT_GET_OPTIONS="Dir::Etc::SourceList=${SOURCES_LIST_FILE}" SUPPORTED_FORMATS="1.0 2.0 3.0" # A space-separated list of hwpack formats. FLASH_KERNEL_SKIP="true" export FLASH_KERNEL_SKIP # skip attempting to run flash-kernel-hooks DISTRIBUTION=`grep '^ID=' /etc/os-release | sed 's/ID=//'` sudo="sudo" if [ $(id -u) -eq 0 ]; then sudo="" fi die() { echo -e "$@" exit 1 } usage_msg="Usage: $(basename $0) [--install-latest] [--force-yes] [--extract-kernel-only] --hwpack-version --hwpack-arch --hwpack-name HWPACK_TARBALL" if [ $# -eq 0 ]; then die $usage_msg fi HWPACK_TARBALL_FOUND="no" HWPACK_VERSION="" HWPACK_ARCH="" HWPACK_NAME="" EXTRACT_KERNEL_ONLY="no" while [ $# -gt 0 ]; do case "$1" in --install-latest) INSTALL_LATEST="yes" shift;; --force-yes) FORCE_YES="yes" shift;; --hwpack-version) HWPACK_VERSION=$2 shift; shift;; --hwpack-arch) HWPACK_ARCH=$2 shift; shift;; --hwpack-name) HWPACK_NAME=$2 shift; shift;; --extract-kernel-only) EXTRACT_KERNEL_ONLY="yes" shift;; --*) die $usage_msg "\nUnrecognized option: \"$1\"";; *) [ "$HWPACK_TARBALL_FOUND" = "yes" ] && die $usage_msg HWPACK_TARBALL="$1" HWPACK_TARBALL_FOUND="yes" shift;; esac done [ "$HWPACK_TARBALL_FOUND" = "no" ] && die $usage_msg [ "$HWPACK_VERSION" = "" ] && die $usage_msg [ "$HWPACK_ARCH" = "" ] && die $usage_msg [ "$HWPACK_NAME" = "" ] && die $usage_msg setup_hwpack() { # This creates all the directories we need. mkdir -p "$HWPACK_DIR" # Unpack the hwpack tarball. We don't download it here because the chroot may # not contain any tools that would allow us to do that. echo -n "Unpacking hardware pack ..." tar zxf "$HWPACK_TARBALL" -C "$HWPACK_DIR" echo "Done" # Check the format of the hwpack is supported. hwpack_format=$(cat ${HWPACK_DIR}/FORMAT) supported="false" for format in $SUPPORTED_FORMATS; do if [ "x$hwpack_format" = "x$format" ]; then supported="true" break fi done [ $supported = "true" ] || \ die "Unsupported hwpack format: $hwpack_format. "\ "Try using a newer version of $(basename $0)." # Check the architecture of the hwpack matches that of the host system. if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then # TODO: create a generic way to identify the architecture, without depending on dpkg [ "$HWPACK_ARCH" = `dpkg --print-architecture` ] || \ die "Hardware pack architecture ($HWPACK_ARCH) does not match the host's architecture" fi } setup_apt_sources() { # Install the apt sources that contain the packages we need. for filename in $(ls "${HWPACK_DIR}"/sources.list.d/); do file="${HWPACK_DIR}"/sources.list.d/$filename should_install=0 stripped_file=${TEMP_DIR}/$filename grep -v "\(^#\|^\s*$\)" $file > $stripped_file while read line; do # Only install files that have at least one line not present in the # existing sources lists. grep -qF "$line" $(find /etc/apt/sources.list.d/ -name '*.list') /etc/apt/sources.list \ || should_install=1 done < $stripped_file if [ $should_install -eq 1 ]; then $sudo cp $file /etc/apt/sources.list.d/hwpack.$filename fi done # Import the OpenPGP keys for the files installed above. for filename in $(ls "${HWPACK_DIR}"/sources.list.d.gpg/); do file="${HWPACK_DIR}"/sources.list.d.gpg/$filename $sudo apt-key add $file done # Add one extra apt source for the packages included in the hwpack and make # sure it's the first on the list of sources so that it gets precedence over # the others. echo "deb file:${HWPACK_DIR}/pkgs ./" > "$SOURCES_LIST_FILE" cat /etc/apt/sources.list >> "$SOURCES_LIST_FILE" if [ "$FORCE_YES" = "yes" ]; then FORCE_OPTIONS="--yes --force-yes" else FORCE_OPTIONS="" fi # Do two updates. The first doesn't try to download package lists: # * First update doesn't access net # - not allowed to fail. Image file + hwpack should contain all packages # needed to create image. If this update fails we have problems. # * Second update may fail # - If can't download package updates (the only difference between the two # commands), we should still be OK. echo "Updating apt package lists ..." $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" update -q --no-download --ignore-missing $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" update -q || true } setup_ubuntu_rootfs() { # Prevent daemons to start in the chroot echo "exit 101" > /usr/sbin/policy-rc.d chmod a+x /usr/sbin/policy-rc.d mv -f /sbin/start-stop-daemon /sbin/start-stop-daemon.REAL cat > /sbin/start-stop-daemon << EOF #!/bin/sh echo "Warning: Fake start-stop-daemon called, doing nothing" EOF chmod 755 /sbin/start-stop-daemon if [ -x /sbin/initctl ]; then mv -f /sbin/initctl /sbin/initctl.REAL cat > /sbin/initctl << EOF #!/bin/sh echo "Warning: Fake initctl called, doing nothing" EOF chmod 755 /sbin/initctl fi # Create dummy fstab. fsck initramfs-tools hook is reading fstab entries # in order to copy e2fsck into initramfs image [ -f /etc/fstab ] && mv -f /etc/fstab /etc/fstab.REAL echo "UUID=00000000-0000-0000-0000-000000000000 / ext4 defaults 0 1" > /etc/fstab } install_deb_packages() { echo -n "Installing packages ..." # "newer" hwpacks contain a dependency package whose Depends is the # same as the packages config setting from the file the hwpack was # build from. But if we just installed that, a newer version of a # package than that in the hwpack might have made it to the main # archive and apt-get would install that instead. So we install the # specific package versions that make up the hwpack. /That/ however # would leave all the packages from the hwpack marked as manually # installed, so if a newer hwpack was installed over the top which no # longer depended on one of the packages the older one did, the # package would not be eligible for autoremoval. So we mark the all # packages newly installed as part of hwpack installed (apart from the # dependency package) as automatically installed with apt-get # markauto. # # For "older" hwpacks that don't have a dependency package, we just # manually install the contents of the hwpack. dependency_package="hwpack-${HWPACK_NAME}" if grep -q "^${dependency_package}=${HWPACK_VERSION}\$" "${HWPACK_DIR}"/manifest; then DEP_PACKAGE_PRESENT="yes" else DEP_PACKAGE_PRESENT="no" fi packages_without_versions=`sed 's/=.*//' "${HWPACK_DIR}"/manifest` packages_with_versions=`cat "${HWPACK_DIR}"/manifest` if [ "$INSTALL_LATEST" = "yes" ]; then packages="${packages_without_versions}" else packages="${packages_with_versions}" fi if [ "$DEP_PACKAGE_PRESENT" = "yes" ]; then to_be_installed= for package in $packages_without_versions; do if [ "${package}" != "${dependency_package}" ]; then { dpkg --get-selections $package 2>/dev/null| grep -qw 'install$'; } || to_be_installed="$to_be_installed $package" fi done fi $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" install ${packages} if [ "$DEP_PACKAGE_PRESENT" = "yes" ]; then if [ -n "${to_be_installed}" ]; then $sudo apt-get $FORCE_OPTIONS -o "$APT_GET_OPTIONS" markauto ${to_be_installed} fi fi } extract_kernel_packages() { echo "Extracting all kernel packages ..." # We assume the hwpack is always available at the rootfs ROOTFS_DIR=$(dirname $HWPACK_TARBALL) ls ${HWPACK_DIR}/pkgs/linux-[ih]*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done ls ${HWPACK_DIR}/pkgs/*-pre-boot*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done ls ${HWPACK_DIR}/pkgs/uefi-image-*.deb | while read pkg; do echo "Extracting package `basename $pkg`" dpkg-deb -x ${pkg} $ROOTFS_DIR done # manually generate modules.dep ls $ROOTFS_DIR/lib/modules | while read kernel; do depmod -b $ROOTFS_DIR ${kernel} || true done; } cleanup() { # Ensure our temp dir and apt sources are removed. echo -n "Cleaning up ..." rm -rf $TEMP_DIR if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then rm -f /usr/sbin/policy-rc.d mv -f /sbin/start-stop-daemon.REAL /sbin/start-stop-daemon if [ -x /sbin/initctl.REAL ]; then mv -f /sbin/initctl.REAL /sbin/initctl fi rm -f /etc/fstab [ -f /etc/fstab.REAL ] && mv -f /etc/fstab.REAL /etc/fstab # Do two updates. The first doesn't try to download package lists: # * First update doesn't access net # - not allowed to fail. Image file + hwpack should contain all packages # needed to create image. If this update fails we have problems. # * Second update may fail # - If can't download package updates (the only difference between the two # commands), we should still be OK. $sudo apt-get update -qq --no-download --ignore-missing $sudo apt-get update -qq || true fi echo "Done" } ## main # Try to acquire fd #9 (i.e. /var/lock/hwpack) # Using 9 as the file descriptor because of https://launchpad.net/bugs/249620 exec 9>$LOCKFILE flock -n 9 || die "Could not acquire lock: $LOCKFILE" # From now on we'll be making changes to the system, so we need to clean # things up when the script exits. trap cleanup EXIT # Extract and set up the hwpack at the rootfs setup_hwpack # In case we only care about the kernel, don't mess up with the system if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then setup_apt_sources setup_ubuntu_rootfs install_deb_packages else extract_kernel_packages fi echo "Done" linaro-image-tools-2016.05.orig/.gitreview0000644000175000017500000000011112724020110017710 0ustar voipiovoipio[gerrit] host=review.linaro.org port=29418 project=ci/linaro-image-tools linaro-image-tools-2016.05.orig/linaro_image_tools/0000755000175000017500000000000012724027364021601 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/utils.py0000644000175000017500000003571212724020110023301 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import platform import subprocess import re import logging import tempfile import tarfile import sys from linaro_image_tools import cmd_runner DEFAULT_LOGGER_NAME = 'linaro_image_tools' # The boot path in the boot tarball. BOOT_DIR_IN_TARBALL = "boot" # The name of the hwpack file found in the boot tarball. HWPACK_NAME = "config" # dconf keys to disable automount options. AUTOMOUNT_DCONF_KEY = '/org/gnome/desktop/media-handling/automount' AUTOMOUNT_OPEN_DCONF_KEYU = '/org/gnome/desktop/media-handling/automount-open' # try_import was copied from python-testtools 0.9.12 and was originally # licensed under a MIT-style license but relicensed under the GPL in Linaro # Image Tools. # Copyright (c) 2011 Jonathan M. Lange . def try_import(name, alternative=None, error_callback=None): """Attempt to import ``name``. If it fails, return ``alternative``. When supporting multiple versions of Python or optional dependencies, it is useful to be able to try to import a module. :param name: The name of the object to import, e.g. ``os.path`` or ``os.path.join``. :param alternative: The value to return if no module can be imported. Defaults to None. :param error_callback: If non-None, a callable that is passed the ImportError when the module cannot be loaded. """ module_segments = name.split('.') last_error = None while module_segments: module_name = '.'.join(module_segments) try: module = __import__(module_name) except ImportError: last_error = sys.exc_info()[1] module_segments.pop() continue else: break else: if last_error is not None and error_callback is not None: error_callback(last_error) return alternative nonexistent = object() for segment in name.split('.')[1:]: module = getattr(module, segment, nonexistent) if module is nonexistent: if last_error is not None and error_callback is not None: error_callback(last_error) return alternative return module CommandNotFound = try_import('CommandNotFound.CommandNotFound') def path_in_tarfile_exists(path, tar_file): exists = True try: tarinfo = tarfile.open(tar_file, 'r:*') tarinfo.getmember(path) tarinfo.close() except KeyError: exists = False except (tarfile.ReadError, tarfile.CompressionError): exists = False # Fallback to tar command cmd = ['tar', '-tf', tar_file, '--wildcards', '*' + path] proc = cmd_runner.run(cmd, stdout=open('/dev/null', 'w'), stderr=open('/dev/null', 'w')) proc.wait() if proc.returncode == 0: exists = True finally: return exists def verify_file_integrity(sig_file_list): """Verify a list of signature files. The parameter is a list of filenames of gpg signature files which will be verified using gpg. For each of the files it is assumed that there is an sha1 hash file with the same file name minus the '.asc' extension. Each of the sha1 files will be checked using sha1sums. All files listed in the sha1 hash file must be found in the same directory as the hash file. """ gpg_sig_ok = True gpg_out = "" verified_files = [] for sig_file in sig_file_list: hash_file = sig_file[0:-len('.asc')] tmp = tempfile.NamedTemporaryFile() try: cmd_runner.run(['gpg', '--status-file={0}'.format(tmp.name), '--verify', sig_file]).wait() except cmd_runner.SubcommandNonZeroReturnValue: gpg_sig_ok = False gpg_out = gpg_out + tmp.read() tmp.close() if os.path.dirname(hash_file) == '': sha_cwd = None else: sha_cwd = os.path.dirname(hash_file) try: sha1sums_out, _ = cmd_runner.Popen( ['sha1sum', '-c', hash_file], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=sha_cwd ).communicate() except cmd_runner.SubcommandNonZeroReturnValue as inst: sha1sums_out = inst.stdout for line in sha1sums_out.splitlines(): sha1_check = re.search(r'^(.*):\s+OK', line) if sha1_check: verified_files.append(sha1_check.group(1)) return verified_files, gpg_sig_ok, gpg_out def check_file_integrity_and_log_errors(sig_file_list, binary, hwpacks): """ Wrapper around verify_file_integrity that prints error messages to stderr if verify_file_integrity finds any problems. """ verified_files, gpg_sig_pass, _ = verify_file_integrity(sig_file_list) # Check the outputs from verify_file_integrity # Abort if anything fails. logger = logging.getLogger(__name__) if len(sig_file_list): if not gpg_sig_pass: logger.error("GPG signature verification failed.") return False, [] if not os.path.basename(binary) in verified_files: logger.error("OS Binary verification failed") return False, [] for hwpack in hwpacks: if not os.path.basename(hwpack) in verified_files: logger.error("Hwpack {0} verification failed".format(hwpack)) return False, [] for verified_file in verified_files: logger.info('Hash verification of file {0} OK.'.format( verified_file)) return True, verified_files def install_package_providing(command): """Install a package which provides the given command. If we can't find any package which provides it, raise UnableToFindPackageProvidingCommand. If the user denies installing the package, the program exits. """ if CommandNotFound is None: raise UnableToFindPackageProvidingCommand( "CommandNotFound python module does not exist.") packages = CommandNotFound().getPackages(command) if len(packages) == 0: raise UnableToFindPackageProvidingCommand( "Unable to find any package providing %s" % command) # TODO: Ask the user to pick a package when there's more than one that # provides the given command. package, _ = packages[0] output, _ = cmd_runner.run(['apt-get', '-s', 'install', package], stdout=subprocess.PIPE).communicate() to_install = [] for line in output.splitlines(): if line.startswith("Inst"): to_install.append(line.split()[1]) if not to_install: raise UnableToFindPackageProvidingCommand( "Unable to find any package to be installed.") try: print ("In order to use the '%s' command, the following package/s " "have to be installed: %s" % (command, " ".join(to_install))) resp = raw_input("Install? (Y/n) ") if resp.lower() != 'y': print "Package installation is necessary to continue. Exiting." sys.exit(1) print ("Installing required command '%s' from package '%s'..." % (command, package)) cmd_runner.run(['apt-get', '--yes', 'install', package], as_root=True).wait() except EOFError: raise PackageInstallationRefused( "Package installation interrupted: input error.") except KeyboardInterrupt: raise PackageInstallationRefused( "Package installation interrupted by the user.") def has_command(command): """Check the given command is available.""" try: cmd_runner.run( ['which', command], stdout=open('/dev/null', 'w')).wait() return True except cmd_runner.SubcommandNonZeroReturnValue: return False def ensure_command(command): """Ensure the given command is available. If it's not, look up a package that provides it and install that. """ if not has_command(command): install_package_providing(command) def find_command(name, prefer_dir=None): """Finds a linaro-image-tools command. Prefers specified directory, otherwise searches only the current directory when running from a checkout, or only PATH when running from an installed version. """ assert name != "" assert os.path.dirname(name) == "" cmd_runner.sanitize_path(os.environ) # default to searching in current directory when running from a bzr # checkout dirs = [os.getcwd(), ] if os.path.isabs(__file__): dirs = os.environ["PATH"].split(os.pathsep) # empty dir in PATH means current directory dirs = map(lambda x: x == '' and '.' or x, dirs) if prefer_dir is not None: dirs.insert(0, prefer_dir) for dir in dirs: path = os.path.join(dir, name) if os.path.exists(path) and os.access(path, os.X_OK): return path return None def is_arm_host(): return platform.machine().startswith('arm') def preferred_tools_dir(): prefer_dir = None # running from bzr checkout? if not os.path.isabs(__file__): prefer_dir = os.getcwd() return prefer_dir def prep_media_path(args): if args.directory is not None: loc = os.path.abspath(args.directory) try: os.makedirs(loc) except OSError: # Directory exists. pass path = os.path.join(loc, args.device) else: path = args.device return path class UnableToFindPackageProvidingCommand(Exception): """We can't find a package which provides the given command.""" class PackageInstallationRefused(Exception): """User has chosen not to install a package.""" class InvalidHwpackFile(Exception): """The hwpack parameter is not a regular file.""" class MissingRequiredOption(Exception): """A required option from the command line is missing.""" def __init__(self, value): self.value = value def __str__(self): return repr(self.value) class IncompatibleOptions(Exception): def __init__(self, value): self.value = value def __str__(self): return repr(self.value) def additional_option_checks(args): if args.directory is not None: # If args.device is a path to a device (/dev/) then this is an error if "--mmc" in sys.argv: raise IncompatibleOptions("--directory option incompatible with " "option --mmc") # If directory is used as well as having a full path (rather than just # a file name or relative path) in args.device, this is an error. if re.search(r"^/", args.device): raise IncompatibleOptions("--directory option incompatible with " "a full path in --image-file") for hwpack in args.hwpacks: if not os.path.isfile(hwpack): raise InvalidHwpackFile( "--hwpack argument (%s) is not a regular file" % hwpack) def additional_android_option_checks(args): """Checks that some of the args passed to l-a-m-c are valid.""" if args.hwpack: if not os.path.isfile(args.hwpack): raise InvalidHwpackFile( "--hwpack argument (%s) is not a regular file" % args.hwpack) def android_hwpack_in_boot_tarball(boot_dir): """Simple check for existence of a path. Needed to make cli command testable in some way. :param boot_dir: The path where the boot tarball has been extracted. :type str :return A tuple with a bool if the path exists, and the path to the config file. """ conf_file = os.path.join(boot_dir, BOOT_DIR_IN_TARBALL, HWPACK_NAME) return os.path.exists(conf_file), conf_file def check_required_args(args): """Check that the required args are passed.""" if args.dev is None: raise MissingRequiredOption("--dev option is required") if args.binary is None: raise MissingRequiredOption("--binary option is required") def get_logger(name=DEFAULT_LOGGER_NAME, debug=False): """ Retrieves a named logger. Default name is set in the variable DEFAULT_LOG_NAME. Debug is set to False by default. :param name: The name of the logger. :param debug: If debug level should be turned on :return: A logger instance. """ logger = logging.getLogger(name) ch = logging.StreamHandler() if debug: ch.setLevel(logging.DEBUG) formatter = logging.Formatter( "%(asctime)s - %(name)s - %(levelname)s - %(message)s") ch.setFormatter(formatter) logger.setLevel(logging.DEBUG) else: ch.setLevel(logging.INFO) formatter = logging.Formatter("%(message)s") ch.setFormatter(formatter) logger.setLevel(logging.INFO) logger.addHandler(ch) return logger def disable_automount(): """Disables the desktop environment automount option. This will work only under GNOME with dconf installed. """ logger = logging.getLogger(DEFAULT_LOGGER_NAME) if has_command('dconf'): logger.info("Disabling desktop environment automount option.") try: cmd_runner.run( ['dconf', 'write', AUTOMOUNT_DCONF_KEY, 'false'], stdout=open('/dev/null', 'w')).wait() cmd_runner.run( ['dconf', 'write', AUTOMOUNT_OPEN_DCONF_KEYU, 'false'], stdout=open('/dev/null', 'w')).wait() except cmd_runner.SubcommandNonZeroReturnValue: logger.error("Error disabling desktop environemnt automount.") def enable_automount(): """Re-enables back the desktop environment automount option. This will work only under GNOME with dconf installed. It should be run as an atexit function. """ logger = logging.getLogger(DEFAULT_LOGGER_NAME) if has_command('dconf'): try: cmd_runner.run( ['dconf', 'write', AUTOMOUNT_DCONF_KEY, 'true'], stdout=open('/dev/null', 'w')).wait() cmd_runner.run( ['dconf', 'write', AUTOMOUNT_OPEN_DCONF_KEYU, 'true'], stdout=open('/dev/null', 'w')).wait() except cmd_runner.SubcommandNonZeroReturnValue: logger.error("Error enabling back desktop environemnt automount.") linaro-image-tools-2016.05.orig/linaro_image_tools/__version__.py0000644000175000017500000000003012724027364024425 0ustar voipiovoipio__version__ = "2016.05" linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/0000755000175000017500000000000012724020110024161 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/media_create/check_device.py0000644000175000017500000001077012724020110027134 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import glob import dbus from linaro_image_tools.media_create import partitions def _get_system_bus_and_udisks_iface(): """Return the system bus and the UDisks interface. :return: System bus and UDisks inteface tuple. """ bus = dbus.SystemBus() udisks = dbus.Interface( bus.get_object("org.freedesktop.UDisks", "/org/freedesktop/UDisks"), 'org.freedesktop.UDisks') return (bus, udisks) def _get_dbus_property(prop, device, path): """ Return a named property for a specific device. :param prop: Named property. :param device: Device object. :param path: Device path. :return: Device property. """ return device.Get( path, prop, dbus_interface='org.freedesktop.DBus.Properties') def _does_device_exist(path): """Checks if the provided path is an existing device. :param path: Disk device path. :return: True if the device exist, else False. """ bus, udisks = _get_system_bus_and_udisks_iface() try: udisks.get_dbus_method('FindDeviceByDeviceFile')(path) except dbus.exceptions.DBusException: # TODO: Check that this exception isn't hiding other errors. return False return True def _print_devices(): """Print disk devices found on the system.""" bus, udisks = _get_system_bus_and_udisks_iface() print '%-16s %-16s %s' % ('Device', 'Mount point', 'Size') devices = udisks.get_dbus_method('EnumerateDevices')() devices.sort() for path in devices: device = bus.get_object("org.freedesktop.UDisks", path) device_file = _get_dbus_property('DeviceFile', device, path) mount_paths = _get_dbus_property('device-mount-paths', device, path) mount_point = ''.join(b for b in mount_paths) if mount_point == '': mount_point = 'none' if _get_dbus_property('DeviceIsPartition', device, path): part_size = _get_dbus_property('partition-size', device, path) print '%-16s %-16s %dMB' % ( device_file, mount_point, part_size / 1024 ** 2) else: device_size = _get_dbus_property('device-size', device, path) print '%-16s %-16s %dMB' % ( device_file, mount_point, device_size / 1024 ** 2) def _select_device(device): """Ask the user to confirm the selected device. :param device: Device path. :return: True if the user confirms the selection, else False. """ resp = raw_input('Are you 100%% sure, on selecting [%s] (y/n)? ' % device) if resp.lower() != 'y': return False return True def _ensure_device_partitions_not_mounted(device): """Ensure all partitions of the given device are not mounted.""" # Use '%s?*' as we only want the device files representing # partitions and not the one representing the device itself. for part in glob.glob('%s?*' % device): partitions.ensure_partition_is_not_mounted(part) def confirm_device_selection_and_ensure_it_is_ready( device, yes_to_mmc_selection=False): """Confirm this is the device to use and ensure it's ready. If the device exists, the user is asked to confirm that this is the device to use. Upon confirmation we ensure all partitions of the device are umounted. :param device: The path to the device. :return: True if the device exist and is selected, else False. """ if _does_device_exist(device): print '\nI see...' _print_devices() if yes_to_mmc_selection or _select_device(device): _ensure_device_partitions_not_mounted(device) return True else: print '\nAre you sure? I do not see [%s].' % device print 'Here is what I see...' _print_devices() return False linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/boards.py0000644000175000017500000026500712724020110026017 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . """Configuration for boards supported by linaro-media-create. To add support for a new board, you need to create a subclass of BoardConfig, set appropriate values for its variables and add it to board_configs at the bottom of this file. """ from binascii import crc32 from parted import Device import atexit import glob import logging import os import re import string import struct import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.media_create.partitions import ( SECTOR_SIZE, partition_mounted, register_loopback, ) from linaro_image_tools.hwpack.hwpack_fields import ( BOOTFS, BOOTFS16, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, DTB_ADDR_FIELD, DTB_FILES_FIELD, DTB_FILE_FIELD, ENV_DD_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, INITRD_ADDR_FIELD, KERNEL_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, LOAD_ADDR_FIELD, MMC_ID_FIELD, PARTITION_LAYOUT_FIELD, RESERVED_BOOTFS, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SPL_DD_FIELD, SPL_IN_BOOT_PART_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, ) logger = logging.getLogger(__name__) KERNEL_GLOB = 'vmlinuz-*-%(kernel_flavor)s' INITRD_GLOB = 'initrd.img-*-%(kernel_flavor)s' DTB_GLOB = 'dt-*-%(kernel_flavor)s/%(dtb_name)s' # Notes: # * since we align partitions on 4 MiB by default, geometry is currently 128 # heads and 32 sectors (2 MiB) as to have CHS-aligned partition start/end # offsets most of the time and hence avoid some warnings with disk # partitioning tools # * apparently some OMAP3 ROMs don't tolerate vfat length of an odd number of # sectors (only sizes rounded to 1 KiB seem to boot) # * we want partitions aligned on 4 MiB as to get the best performance and # limit wear-leveling # * image_size is passed on the command-line and should preferably be a power # of 2; it should be used as a "don't go over this size" information for a # real device, and a "give me a file exactly this big" requirement for an # image file. Having exactly a power of 2 helps with QEMU; there seem to be # some truncating issues otherwise. XXX to be researched # align on 4 MiB PART_ALIGN_S = 4 * 1024 * 1024 / SECTOR_SIZE def align_up(value, align): """Round value to the next multiple of align.""" return (value + align - 1) / align * align def align_partition(min_start, min_length, start_alignment, end_alignment): """Compute partition start and end offsets based on specified constraints. :param min_start: Minimal start offset of partition :param min_lengh: Minimal length of partition :param start_alignment: Alignment of this partition :param end_alignment: Alignment of the data following this partition :return: start offset, end offset (inclusive), length """ start = align_up(min_start, start_alignment) # end offset is inclusive, so substact one end = align_up(start + min_length, end_alignment) - 1 # and add one to length length = end - start + 1 return start, end, length def copy_drop(src, dest_dir): """Copy a file from src to destdir, dropping root ownership on the way. """ cmd = ["cp", "-v", src, dest_dir] cmd_runner.run(cmd, as_root=True).wait() final = os.path.join(dest_dir, os.path.basename(src)) cmd = ["chown", "%s:%s" % (os.getuid(), os.getgid()), final] cmd_runner.run(cmd, as_root=True).wait() os.chmod(final, 0644) class BoardException(Exception): """Class for board related exceptions.""" class BoardConfig(object): """The configuration used when building an image for a board.""" LOADER_MIN_SIZE_S = align_up(1 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE BOOT_MIN_SIZE_S = align_up(50 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE ROOT_MIN_SIZE_S = align_up(50 * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE def __init__(self): super(BoardConfig, self).__init__() # XXX: when killing v1 hwpack, we might rename these two without the # leading underscore. It is done in this way since sublcasses use # placeholders in the string for dinamically change values. But this # is done only for hwpack v1. self._extra_serial_options = '' self._live_serial_options = '' self.board = None self.boot_script = None self.bootloader_dd = False self.bootloader_file_in_boot_part = False self.bootloader_flavor = None self.dtb_addr = None self.dtb_file = None self.dtb_files = None self.dtb_name = None self.env_dd = False self.extra_boot_args_options = None self.fat_size = 32 self.fatload_command = 'fatload' self.load_interface = 'mmc' self.bootfs_type = 'vfat' self.fdt_high = '0xffffffff' self.hardwarepack_handler = None self.hwpack_format = None self.initrd_addr = None self.initrd_high = '0xffffffff' self.kernel_addr = None self.kernel_flavors = None self.load_addr = None self.loader_start_s = 1 self.mmc_device_id = 0 self.mmc_id = None self.mmc_option = '0:1' self.mmc_part_offset = 0 self.partition_layout = None self.serial_tty = None self.spl_dd = False self.spl_in_boot_part = False self.supports_writing_to_mmc = True self.uimage_path = '' self.wired_interfaces = None self.wireless_interfaces = None # Samsung Boot-loader implementation notes and terminology # # BL0, BL1, BL2, BL3 are the boot loader stages in order of execution # # BL0 - embedded boot loader on the internal ROM # BL1 - chip-specific boot loader provided by Samsung # BL2 - tiny boot loader; SPL (Second Program Loader) # BL3 - customized boot loader; U-Boot # # In linaro-image-tools, variables have been named samsung_blN-1 # e.g BL1 is samsung_bl0, BL2 is samsung_bl1, BL3 is samsung_bl2 # # samsung_bl0_{start,len}: Offset and maximum size for BL1 # samsung_bl1_{start,len}: Offset and maximum size for BL2 # samsung_bl2_{start,len}: Offset and maximum size for BL3 # samsung_env_{start,len}: Offset and maximum size for settings # self.samsung_bl1_start = 1 self.samsung_bl1_len = 32 self.samsung_bl2_start = 65 self.samsung_bl2_len = 1024 self.samsung_env_start = 33 self.samsung_env_len = 32 # XXX: attributes that are not listed in hwpackV3, should be removed? self.vmlinuz = None self.initrd = None # XXX: can be removed when killing v1 hwpack. def _get_live_serial_options(self): live_serial = self._live_serial_options if live_serial: if isinstance(live_serial, list): live_serial = ' '.join(live_serial) if self._check_placeholder_presence(live_serial, r'%s'): live_serial = live_serial % self.serial_tty return live_serial def _set_live_serial_options(self, value): self._live_serial_options = value live_serial_options = property(_get_live_serial_options, _set_live_serial_options) # XXX: can be removed when killing v1 hwpack. def _get_extra_serial_options(self): extra_serial = self._extra_serial_options if extra_serial: if isinstance(extra_serial, list): extra_serial = ' '.join(extra_serial) if self._check_placeholder_presence(extra_serial, r'%s'): extra_serial = extra_serial % self.serial_tty return extra_serial def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def get_metadata_field(self, field_name): """ Return the metadata value for field_name if it can be found. """ data, _ = self.hardwarepack_handler.get_field(field_name) return data def set_metadata(self, hwpacks, bootloader=None, board=None, dtb_file=None): self.hardwarepack_handler = HardwarepackHandler(hwpacks, bootloader, board) with self.hardwarepack_handler: self.hwpack_format = self.hardwarepack_handler.get_format() if (self.hwpack_format == self.hardwarepack_handler.FORMAT_1): self.bootloader_copy_files = None return if (self.hwpack_format != self.hardwarepack_handler.FORMAT_1): # Clear V1 defaults. # TODO When removing v1 support, remove also default values # in the constructor and avoid all this. self.kernel_addr = None self.initrd_addr = None self.load_addr = None self.serial_tty = None self.fat_size = None self.dtb_name = None self.dtb_addr = None self.extra_boot_args_options = None self.boot_script = None self.kernel_flavors = None self.mmc_option = None self.mmc_part_offset = None self.samsung_bl1_start = None self.samsung_bl1_len = None self.samsung_env_len = None self.samsung_bl2_len = None # self.samsung_bl2_start and self.samsung_env_start should # be initialized to default value for backward compatibility. self.board = board # Set new values from metadata. self.kernel_addr = self.get_metadata_field(KERNEL_ADDR_FIELD) self.initrd_addr = self.get_metadata_field(INITRD_ADDR_FIELD) self.load_addr = self.get_metadata_field(LOAD_ADDR_FIELD) self.dtb_addr = self.get_metadata_field(DTB_ADDR_FIELD) self.serial_tty = self.get_metadata_field(SERIAL_TTY_FIELD) wired_interfaces = self.get_metadata_field(WIRED_INTERFACES_FIELD) if wired_interfaces: self.wired_interfaces = wired_interfaces wireless_interfaces = self.get_metadata_field( WIRELESS_INTERFACES_FIELD) if wireless_interfaces: self.wireless_interfaces = wireless_interfaces self.dtb_file = self.get_metadata_field(DTB_FILE_FIELD) # XXX: need to deprecate dtb_file field and use only dtb_files # for multiple entries. if self.dtb_file: logger.warn("Deprecation warning: use the 'dtb_files' field " "instead of 'dtb_file'.") self.dtb_files = self.get_metadata_field(DTB_FILES_FIELD) if dtb_file: dtb_dict = self._find_dtb_dict(dtb_file) if dtb_dict: self.dtb_files = [] self.dtb_files.append(dtb_dict) self.extra_boot_args_options = self.get_metadata_field( EXTRA_BOOT_OPTIONS_FIELD) self.boot_script = self.get_metadata_field(BOOT_SCRIPT_FIELD) self.extra_serial_options = self.get_metadata_field( EXTRA_SERIAL_OPTIONS_FIELD) self.snowball_startup_files_config = self.get_metadata_field( SNOWBALL_STARTUP_FILES_CONFIG_FIELD) self.partition_layout = self.get_metadata_field( PARTITION_LAYOUT_FIELD) if self.partition_layout in [BOOTFS, RESERVED_BOOTFS, None]: self.fat_size = 32 elif self.partition_layout == BOOTFS16: self.fat_size = 16 else: raise AssertionError("Unknown partition layout '%s'." % self.partition_layout) self.mmc_option = self.get_metadata_field(MMC_ID_FIELD) if self.mmc_option: self.mmc_device_id = int(self.mmc_option.split(':')[0]) self.mmc_part_offset = int(self.mmc_option.split(':')[1]) - 1 # XXX: need to fix these values. boot_min_size = self.get_metadata_field(BOOT_MIN_SIZE_FIELD) if boot_min_size: self.BOOT_MIN_SIZE_S = align_up(int(boot_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE root_min_size = self.get_metadata_field(ROOT_MIN_SIZE_FIELD) if root_min_size: self.ROOT_MIN_SIZE_S = align_up(int(root_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE loader_min_size = self.get_metadata_field(LOADER_MIN_SIZE_FIELD) if loader_min_size: self.LOADER_MIN_SIZE_S = ( align_up(int(loader_min_size) * 1024 ** 2, SECTOR_SIZE) / SECTOR_SIZE) spl_in_boot_part = self.get_metadata_field(SPL_IN_BOOT_PART_FIELD) if spl_in_boot_part is None: self.spl_in_boot_part = False elif string.lower(spl_in_boot_part) == 'yes': self.spl_in_boot_part = True elif string.lower(spl_in_boot_part) == 'no': self.spl_in_boot_part = False env_dd = self.get_metadata_field(ENV_DD_FIELD) if env_dd is None: self.env_dd = False elif string.lower(env_dd) == 'yes': self.env_dd = True elif string.lower(env_dd) == 'no': self.env_dd = False # XXX: in hwpack v3 this field is just called 'dd'. # Need to check its use. bootloader_dd = self.get_metadata_field('bootloader_dd') # Either bootloader_dd is not specified, or it contains the dd # offset. if bootloader_dd is None: self.bootloader_dd = False else: self.bootloader_dd = int(bootloader_dd) spl_dd = self.get_metadata_field(SPL_DD_FIELD) # Either spl_dd is not specified, or it contains the dd offset. if spl_dd is None: self.spl_dd = False else: self.spl_dd = int(spl_dd) loader_start = self.get_metadata_field(LOADER_START_FIELD) if loader_start: self.loader_start_s = int(loader_start) samsung_bl1_start = self.get_metadata_field( SAMSUNG_BL1_START_FIELD) if samsung_bl1_start: self.samsung_bl1_start = int(samsung_bl1_start) samsung_bl1_len = self.get_metadata_field( SAMSUNG_BL1_LEN_FIELD) if samsung_bl1_len: self.samsung_bl1_len = int(samsung_bl1_len) samsung_bl2_start = self.get_metadata_field( SAMSUNG_BL2_START_FIELD) if samsung_bl2_start: self.samsung_bl2_start = int(samsung_bl2_start) samsung_bl2_len = self.get_metadata_field( SAMSUNG_BL2_LEN_FIELD) if samsung_bl2_len: self.samsung_bl2_len = int(samsung_bl2_len) samsung_env_start = self.get_metadata_field( SAMSUNG_ENV_START_FIELD) if samsung_env_start is not None: self.samsung_env_start = int(samsung_env_start) samsung_env_len = self.get_metadata_field( SAMSUNG_ENV_LEN_FIELD) if samsung_env_len: self.samsung_env_len = int(samsung_env_len) self.bootloader_copy_files = self.hardwarepack_handler.get_field( "bootloader_copy_files")[0] # XXX: no reference in hwpackV3 format of these fields, double # check if they can be dropped when killing v1. self.bootloader = self.hardwarepack_handler.get_field( "bootloader") self.vmlinuz = self.get_metadata_field('vmlinuz') self.initrd = self.get_metadata_field('initrd') bootloader_file_in_boot_part = self.get_metadata_field( 'bootloader_file_in_boot_part') if bootloader_file_in_boot_part is None: self.bootloader_file_in_boot_part = False elif string.lower(bootloader_file_in_boot_part) == 'yes': self.bootloader_file_in_boot_part = True elif string.lower(bootloader_file_in_boot_part) == 'no': self.bootloader_file_in_boot_part = False def get_file(self, file_alias, default=None): # XXX remove the 'default' parameter when V1 support is removed! file_in_hwpack = self.hardwarepack_handler.get_file(file_alias) if file_in_hwpack is not None: return file_in_hwpack else: return default def get_v1_sfdisk_cmd(self, should_align_boot_part=False): # XXX: This default implementation and all overrides are left for V1 # compatibility only. They should be removed as part of the work to # kill off hwpacks V1. return self.get_normal_sfdisk_cmd(should_align_boot_part) def get_normal_params(self, should_align_boot_part=False): if self.bootfs_type == 'vfat': if self.fat_size == 32: partition_type = '0x0C' else: partition_type = '0x0E' else: partition_type = '0x83' # align on sector 63 for compatibility with broken versions of x-loader # unless align_boot_part is set # XXX OMAP specific, might break other boards? boot_align = 63 if should_align_boot_part: boot_align = PART_ALIGN_S # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( 1, self.BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room # XXX OMAP specific, might break other boards? boot_len = boot_len - boot_len % 2 boot_end = boot_start + boot_len - 1 # we ignore _root_end / _root_len and return a sfdisk command to # instruct the use of all remaining space; XXX we now have root size # config, so we can do something more sensible root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return (boot_start, boot_len, partition_type, root_start) def get_reserved_params(self, should_align_boot_part=None): loader_start, loader_end, loader_len = align_partition( self.loader_start_s, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) boot_start, boot_end, boot_len = align_partition( loader_end + 1, self.BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return (loader_start, loader_len, boot_start, boot_len, root_start) def get_normal_sfdisk_cmd(self, should_align_boot_part=False): """Return the sfdisk command to partition the media. :param should_align_boot_part: Whether to align the boot partition too. This returns a boot partition of type FAT16 or FAT32 or Linux, followed by a root partition. """ (boot_start, boot_len, partition_type, root_start) = self.get_normal_params(should_align_boot_part) return '%s,%s,%s,*\n%s,,,-' % ( boot_start, boot_len, partition_type, root_start) def get_reserved_sfdisk_cmd(self, should_align_boot_part=None): """Return the sfdisk command to partition the media. :param should_align_boot_part: Ignored. This returns a loader partition, then a boot vfat partition of type FAT16 or FAT32, followed by a root partition. """ (loader_start, loader_len, boot_start, boot_len, root_start) = self.get_reserved_params(should_align_boot_part) return '%s,%s,0xDA\n%s,%s,0x0C,*\n%s,,,-' % ( loader_start, loader_len, boot_start, boot_len, root_start) def get_normal_sgdisk_cmd(self, should_align_boot_part=None): (boot_start, boot_len, partition_type, root_start) = self.get_normal_params(should_align_boot_part) # Ignoring partition type because we need 0xEF for GPT return '-n 1:%s:%s -t 1:EF00 ' \ '-n 2:%s:- -t 2:8300' % (boot_start, boot_len, root_start) def get_reserved_sgdisk_cmd(self, should_align_boot_part=None): """Return the sgdisk command to partition the media. :param should_align_boot_part: Ignored. """ (loader_start, loader_len, boot_start, boot_len, root_start) = self.get_reserved_params(should_align_boot_part) return '-n 1:%s:%s -t 1:DA00 ' \ '-n 2:%s:%s -t 2:EF00 ' \ '-n 3:%s:- -t 3:8300' % ( loader_start, loader_len, boot_start, boot_len, root_start) def get_sfdisk_cmd(self, should_align_boot_part=False): if (self.partition_layout in ['bootfs_rootfs', 'bootfs16_rootfs'] or self.board == 'snowball_sd'): return self.get_normal_sfdisk_cmd(should_align_boot_part) elif self.partition_layout in ['reserved_bootfs_rootfs']: return self.get_reserved_sfdisk_cmd(should_align_boot_part) else: assert (self.hwpack_format == HardwarepackHandler.FORMAT_1), ( "Hwpack format is not 1.0 but " "partition_layout is unspecified.") return self.get_v1_sfdisk_cmd(should_align_boot_part) def get_sgdisk_cmd(self, should_align_boot_part=False): if (self.partition_layout in ['bootfs_rootfs', 'bootfs16_rootfs'] or self.board == 'snowball_sd'): return self.get_normal_sgdisk_cmd(should_align_boot_part) elif self.partition_layout in ['reserved_bootfs_rootfs']: return self.get_reserved_sgdisk_cmd(should_align_boot_part) else: assert (self.hwpack_format == HardwarepackHandler.FORMAT_1), ( "Hwpack format is not 1.0 but " "partition_layout is unspecified.") return self.get_normal_sgdisk_cmd(should_align_boot_part) def _get_bootcmd(self, i_img_data, d_img_data): """Get the bootcmd for this board. In general subclasses should not have to override this. """ replacements = dict( fatload_command=self.fatload_command, uimage_path=self.uimage_path, mmc_option=self.mmc_option, kernel_addr=self.kernel_addr, initrd_addr=self.initrd_addr, dtb_addr=self.dtb_addr, load_interface=self.load_interface) boot_script = ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(kernel_addr)s %(uimage_path)suImage; ")) % replacements boot_script_bootm = (("bootm %(kernel_addr)s")) % replacements if i_img_data is not None and d_img_data is not None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(initrd_addr)s %(uimage_path)suInitrd; " "%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(dtb_addr)s board.dtb; ")) % replacements boot_script_bootm += ( (" %(initrd_addr)s %(dtb_addr)s")) % replacements elif i_img_data is None and d_img_data is not None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(dtb_addr)s board.dtb; ")) % replacements boot_script_bootm += ((" - %(dtb_addr)s")) % replacements elif i_img_data is not None and d_img_data is None: boot_script += ( ("%(fatload_command)s %(load_interface)s %(mmc_option)s " "%(initrd_addr)s %(uimage_path)suInitrd; ")) % replacements boot_script_bootm += ((" %(initrd_addr)s")) % replacements boot_script += boot_script_bootm return boot_script def add_boot_args(self, extra_args): if extra_args is not None: if self.extra_boot_args_options is None: self.extra_boot_args_options = extra_args else: self.extra_boot_args_options += ' %s' % extra_args def add_boot_args_from_file(self, path): if path is not None: with open(path, 'r') as boot_args_file: self.add_boot_args(boot_args_file.read().strip()) def _get_bootargs(self, is_live, is_lowmem, consoles, rootfs_id): """Get the bootargs for this board. In general subclasses should not have to override this. """ boot_args_options = 'rootwait ro' serial_options = '' if self.extra_boot_args_options: boot_args_options += ' %s' % self.extra_boot_args_options.strip() if self.extra_serial_options: serial_options = self.extra_serial_options.strip() for console in consoles: serial_options += ' console=%s' % console.strip() lowmem_opt = '' boot_snippet = 'root=%s' % rootfs_id.strip() if is_live: serial_options += ' %s' % self.live_serial_options boot_snippet = 'boot=casper' if is_lowmem: lowmem_opt = 'only-ubiquity' replacements = dict(serial_options=serial_options.strip(), lowmem_opt=lowmem_opt, boot_snippet=boot_snippet, boot_args_options=boot_args_options) boot_args = ("%(serial_options)s %(lowmem_opt)s %(boot_snippet)s" " %(boot_args_options)s" % replacements).strip() return boot_args def _get_boot_env(self, is_live, is_lowmem, consoles, rootfs_id, i_img_data, d_img_data): """Get the boot environment for this board. In general subclasses should not have to override this. """ boot_env = {} boot_env["bootargs"] = self._get_bootargs( is_live, is_lowmem, consoles, rootfs_id) boot_env["bootcmd"] = self._get_bootcmd(i_img_data, d_img_data) boot_env["initrd_high"] = self.initrd_high boot_env["fdt_high"] = self.fdt_high return boot_env def make_boot_files(self, bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file): if self.hwpack_format == HardwarepackHandler.FORMAT_1: parts_dir = bootloader_parts_dir else: parts_dir = chroot_dir (k_img_data, i_img_data, d_img_data) = self._get_kflavor_files( parts_dir) boot_env = self._get_boot_env(is_live, is_lowmem, consoles, rootfs_id, i_img_data, d_img_data) if self.hwpack_format == HardwarepackHandler.FORMAT_1: self._make_boot_files( boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data) else: self._make_boot_files_v2( boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data) def _copy_dtb_files(self, dtb_files, dest_dir, search_dir): """Copy the files defined in dtb_files into the boot directory. :param dtb_files: The list of dtb files :param dest_dir: The directory where to copy each dtb file. :param search_dir: The directory where to search for the real file. """ logger.info("Copying dtb files") for dtb_file in dtb_files: if dtb_file: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file in the new position. to_file = os.path.basename(key) # The directory where to copy the dtb file. to_dir = os.path.join(dest_dir, os.path.dirname(key)) from_file = value # User specified only the directory, without renaming # the file. if not to_file: to_file = os.path.basename(from_file) if not os.path.exists(to_dir): cmd_runner.run(["mkdir", "-p", to_dir], as_root=True).wait() dtb = _get_file_matching(os.path.join(search_dir, from_file)) if not dtb: logger.warn('Could not find a valid dtb file, ' 'skipping it.') continue else: dest = os.path.join(to_dir, to_file) logger.debug('Copying %s into %s' % (dtb, dest)) cmd_runner.run(['cp', dtb, dest], as_root=True).wait() else: # Hopefully we should never get here. # This should only happen if the hwpack config YAML file is # wrong. logger.warn('WARNING: Wrong syntax in metadata file. ' 'Check the hwpack configuration file used to ' 'generate the hwpack archive.') def _dd_file(self, from_file, to_file, seek, max_size=None): assert from_file is not None, "No source file name given." if max_size is not None: assert os.path.getsize(from_file) <= max_size, ( "'%s' is larger than %s" % (from_file, max_size)) logger.info("Writing '%s' to '%s' at %s." % (from_file, to_file, seek)) _dd(from_file, to_file, seek=seek) def install_samsung_boot_loader(self, samsung_spl_file, bootloader_file, boot_device_or_file): self._dd_file(samsung_spl_file, boot_device_or_file, self.samsung_bl1_start, self.samsung_bl1_len * SECTOR_SIZE) self._dd_file(bootloader_file, boot_device_or_file, self.samsung_bl2_start, self.samsung_bl2_len * SECTOR_SIZE) def _make_boot_files_v2(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): with self.hardwarepack_handler: spl_file = self.get_file('spl_file') if self.spl_in_boot_part: assert spl_file is not None, ( "SPL binary could not be found") logger.info( "Copying spl '%s' to boot partition." % spl_file) cmd_runner.run(["cp", "-v", spl_file, boot_dir], as_root=True).wait() if self.spl_dd: self._dd_file(spl_file, boot_device_or_file, self.spl_dd) bootloader_file = self.get_file('bootloader_file') if self.bootloader_dd: self._dd_file(bootloader_file, boot_device_or_file, self.bootloader_dd) make_uImage(self.load_addr, k_img_data, boot_dir) if i_img_data is not None: make_uInitrd(i_img_data, boot_dir) if d_img_data is not None: make_dtb(d_img_data, boot_dir) if self.boot_script is not None: boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) # Only used for Omap, will this be bad for the other boards? make_boot_ini(boot_script_path, boot_dir) if (self.snowball_startup_files_config is not None and self.board != 'snowball_sd'): self.populate_raw_partition(boot_device_or_file, chroot_dir) if self.env_dd: # Do we need to zero out the env before flashing it? _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) env_size = self.samsung_env_len * SECTOR_SIZE env_file = make_flashable_env(boot_env, env_size) self._dd_file(env_file, boot_device_or_file, self.samsung_env_start) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): """Make the necessary boot files for this board. This is usually board-specific so ought to be defined in every subclass. """ raise NotImplementedError() def populate_boot(self, chroot_dir, rootfs_id, boot_partition, boot_disk, boot_device_or_file, is_live, is_lowmem, consoles): parts_dir = 'boot' if is_live: parts_dir = 'casper' bootloader_parts_dir = os.path.join(chroot_dir, parts_dir) cmd_runner.run(['mkdir', '-p', boot_disk]).wait() with partition_mounted(boot_partition, boot_disk): with self.hardwarepack_handler: if self.bootloader_file_in_boot_part: # if self.bootloader_flavor is not None: default = os.path.join( chroot_dir, 'usr', 'lib', 'u-boot', self.bootloader_flavor, 'u-boot.img') if not os.path.exists(default): default = os.path.join( chroot_dir, 'usr', 'lib', 'u-boot', self.bootloader_flavor, 'u-boot.bin') else: default = None # bootloader_bin = self.get_file('bootloader_file', default=default) assert bootloader_bin is not None, ( "bootloader binary could not be found") proc = cmd_runner.run( ['cp', '-v', bootloader_bin, boot_disk], as_root=True) proc.wait() # Handle copy_files field. self.copy_files(boot_disk) # Handle dtb_files field. if self.dtb_files: self._copy_dtb_files(self.dtb_files, boot_disk, chroot_dir) self.make_boot_files( bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_disk, boot_device_or_file) def copy_files(self, boot_disk): """Handle the copy_files metadata field.""" # Extract anything specified by copy_files sections # self.bootloader_copy_files is always of the form: # {'source_package': # [ # {'source_path': 'dest_path'} # ] # } if self.bootloader_copy_files is None: return for source_package, file_list in \ self.bootloader_copy_files.iteritems(): for file_info in file_list: for source_path, dest_path in file_info.iteritems(): source = self.hardwarepack_handler.get_file_from_package( source_path, source_package) dest_path = dest_path.lstrip("/\\") dirname = os.path.dirname(dest_path) dirname = os.path.join(boot_disk, dirname) if not os.path.exists(dirname): cmd_runner.run(["mkdir", "-p", dirname], as_root=True).wait() proc = cmd_runner.run( ['cp', '-v', source, os.path.join(boot_disk, dest_path)], as_root=True) proc.wait() def _get_kflavor_files(self, path): """Search for kernel, initrd and optional dtb in path.""" if self.kernel_flavors is None: # V2 metadata specifies each glob, not flavors. # XXX This duplication is temporary until V1 dies. return self._get_kflavor_files_v2(path) for flavor in self.kernel_flavors: kregex = KERNEL_GLOB % {'kernel_flavor': flavor} iregex = INITRD_GLOB % {'kernel_flavor': flavor} dregex = DTB_GLOB % {'kernel_flavor': flavor, 'dtb_name': self.dtb_name} kernel = _get_file_matching(os.path.join(path, kregex)) if kernel is not None: initrd = _get_file_matching(os.path.join(path, iregex)) if initrd is not None: dtb = None if self.dtb_name is not None: dtb = _get_file_matching(os.path.join(path, dregex)) return (kernel, initrd, dtb) raise ValueError( "Found kernel for flavor %s but no initrd matching %s" % ( flavor, iregex)) raise ValueError( "No kernel found matching %s for flavors %s" % ( KERNEL_GLOB, " ".join(self.kernel_flavors))) def _get_kflavor_files_v2(self, path): kernel = initrd = dtb = None if self.vmlinuz: kernel = _get_file_matching(os.path.join(path, self.vmlinuz)) if not self.vmlinuz or not kernel: raise ValueError("Unable to find a valid kernel image.") if self.initrd: initrd = _get_file_matching(os.path.join(path, self.initrd)) if not self.initrd or not initrd: logger.warn("Could not find a valid initrd, skipping uInitrd.") if self.dtb_file: dtb = _get_file_matching(os.path.join(path, self.dtb_file)) if not self.dtb_file or not dtb: logger.warn("Could not find a valid dtb file from dtb_file, " "trying dtb_files...") if self.dtb_files: # Use first file from list as a default dtb file. dtb_file = self.dtb_files[0] if dtb_file: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file. to_file = os.path.basename(key) from_file = value # User specified only the directory, without renaming # the file. if not to_file: to_file = os.path.basename(from_file) dtb = _get_file_matching(os.path.join(path, from_file)) if not self.dtb_files and not dtb: logger.warn("Could not find a valid dtb file, skipping it.") logger.info("Will use kernel=%s, initrd=%s, dtb=%s." % (kernel, initrd, dtb)) return (kernel, initrd, dtb) def populate_raw_partition(self, media, boot_dir): # Override in subclass if needed pass def snowball_config(self, chroot_dir): # Override in subclasses where applicable raise NotImplementedError( "snowball_config() must only be called on BoardConfigs that " "use the Snowball startupfiles.") # XXX: can be removed when killing v1 hwpack and updating the attributes # that use it. @staticmethod def _check_placeholder_presence(string, placeholder): """Checks if the passed string contains the particular placeholder.""" # Very simple way of achieving that. presence = False if string and placeholder in string: presence = True return presence def _find_dtb_dict(self, dtb): """Returns dictionary entry from dt_files containing dtb file.""" for dtb_file in self.dtb_files: if isinstance(dtb_file, dict): for key, value in dtb_file.iteritems(): # The name of the dtb file. if dtb in key: return dtb_file return None class OmapConfig(BoardConfig): def __init__(self): super(OmapConfig, self).__init__() self.kernel_flavors = ['linaro-omap4', 'linaro-lt-omap', 'linaro-omap', 'omap4'] self.bootloader_file_in_boot_part = True # XXX: Here we define these things as dynamic properties because our # temporary hack to fix bug 697824 relies on changing the board's # serial_tty at run time. self._serial_tty = None # XXX: when killing v1 hwpack this should be safely removed. def _get_serial_tty(self): return self._serial_tty def _set_serial_tty(self, value): self._serial_tty = value serial_tty = property(_get_serial_tty, _set_serial_tty) def set_appropriate_serial_tty(self, chroot_dir): """Set the appropriate serial_tty depending on the kernel used. If the kernel found in the chroot dir is << 2.6.36 we use tyyS2, else we use the default value (_serial_tty). """ # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 # XXX: This is also part of our temporary hack to fix bug 697824. # cls.serial_tty = classproperty(lambda cls: cls._serial_tty) vmlinuz = _get_file_matching( os.path.join(chroot_dir, 'boot', 'vmlinuz*')) basename = os.path.basename(vmlinuz) match = re.match('.*2\.6\.([0-9]{2}).*', basename) # Assume if it doesn't match that it is 3.0 or later. if match is not None: minor_version = match.group(1) if int(minor_version) < 36: self.serial_tty = 'ttyS2' def make_boot_files(self, bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file): # XXX: This is also part of our temporary hack to fix bug 697824; we # need to call set_appropriate_serial_tty() before doing anything that # may use self.serial_tty. if self.hwpack_format == HardwarepackHandler.FORMAT_1: self.set_appropriate_serial_tty(chroot_dir) super(OmapConfig, self).make_boot_files( bootloader_parts_dir, is_live, is_lowmem, consoles, chroot_dir, rootfs_id, boot_dir, boot_device_or_file) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 install_omap_boot_loader(chroot_dir, boot_dir, self) make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) make_dtb(d_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) make_boot_ini(boot_script_path, boot_dir) class BeagleConfig(OmapConfig): def __init__(self): super(BeagleConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap3_beagle' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap3-beagle.dtb' self.extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80000000' self.load_addr = '0x80008000' self._serial_tty = 'ttyO2' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' class OveroConfig(OmapConfig): def __init__(self): super(OveroConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap3_overo' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap3-overo.dtb' self.extra_boot_args_options = ( 'earlyprintk mpurate=${mpurate} vram=12M ' 'omapdss.def_disp=${defaultdisplay} omapfb.mode=dvi:${dvimode}') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80000000' self.load_addr = '0x80008000' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._serial_tty = 'ttyO2' class PandaConfig(OmapConfig): def __init__(self): super(PandaConfig, self).__init__() self._serial_tty = 'ttyO2' self.boot_script = 'boot.scr' self.bootloader_flavor = 'omap4_panda' self.dtb_addr = '0x815f0000' self.dtb_name = 'omap4-panda.dtb' self.extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=48M ' 'omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000') self.initrd_addr = '0x81600000' self.kernel_addr = '0x80200000' self.load_addr = '0x80008000' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' class BeagleBoneConfig(OmapConfig): def __init__(self): super(BeagleBoneConfig, self).__init__() self.boot_script = 'boot.scr' self.bootloader_flavor = 'am335x_evm' self.kernel_flavors = ['am335x'] self._serial_tty = 'ttyO0' self.dtb_addr = '0x815f0000' self.initrd_addr = '0x81600000' self.kernel_addr = '0x80200000' self.load_addr = '0x80008000' self.extra_boot_args_options = ('fixrtc') self._extra_serial_options = 'console=ttyO0,115200n8' class IgepConfig(BeagleConfig): def __init__(self): super(IgepConfig, self).__init__() self.bootloader_file_in_boot_part = False self.bootloader_flavor = None self.dtb_name = 'isee-igep-v2.dtb' def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) make_dtb(d_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) make_boot_ini(boot_script_path, boot_dir) class Ux500Config(BoardConfig): def __init__(self): super(Ux500Config, self).__init__() self.boot_script = 'flash.scr' self.extra_boot_args_options = ( 'earlyprintk rootdelay=1 fixrtc nocompcache ' 'mem=96M@0 mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M') self.initrd_addr = '0x08000000' self.kernel_addr = '0x00100000' self.kernel_flavors = ['u8500', 'ux500'] self.load_addr = '0x00008000' self.mmc_option = '1:1' self.serial_tty = 'ttyAMA2' self._extra_serial_options = 'console=tty0 console=%s,115200n8' self._live_serial_options = 'serialtty=%s' def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) make_uInitrd(i_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) class SnowballSdConfig(Ux500Config): '''Use only with --mmc option. Creates the standard vfat and ext2 partitions for kernel and rootfs on an SD card. Note that the Snowball board needs a loader partition on the internal eMMC flash to boot. That partition is created with the SnowballConfigImage configuration.''' def __init__(self): super(SnowballSdConfig, self).__init__() def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) class SnowballEmmcConfig(SnowballSdConfig): '''Use only with --image option. Creates a raw image which contains an additional (raw) loader partition, containing some boot stages and u-boot.''' SNOWBALL_LOADER_START_S = (128 * 1024) / SECTOR_SIZE TOC_SIZE = 512 def __init__(self): super(SnowballEmmcConfig, self).__init__() # Boot ROM looks for a boot table of contents (TOC) at 0x20000 # Actually, it first looks at address 0, but that's where l-m-c # puts the MBR, so the boot loader skips that address. self.supports_writing_to_mmc = False self.snowball_startup_files_config = 'startfiles.cfg' self.mmc_option = '0:2' def get_v1_sfdisk_cmd(self, should_align_boot_part=None): """Return the sfdisk command to partition the media. :param should_align_boot_part: Ignored. The Snowball partitioning scheme depends on whether the target is a raw image or an SD card. Both targets have the normal FAT 32 boot partition and EXT? root partition. The raw image prepends these two partitions with a raw loader partition, containing HW-dependent boot stages up to and including u-boot. This is done since the boot rom always boots off the internal memory; there simply is no point to having a loader partition on SD card. """ # boot ROM expects bootloader at 0x20000, which is sector 0x100 # with the usual SECTOR_SIZE of 0x200. # (sector 0 is MBR / partition table) loader_start, loader_end, loader_len = align_partition( self.SNOWBALL_LOADER_START_S, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) boot_start, boot_end, boot_len = align_partition( loader_end + 1, self.BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) # we ignore _root_end / _root_len and return an sfdisk command to # instruct the use of all remaining space; XXX if we had some root size # config, we could do something more sensible root_start, _root_end, _root_len = align_partition( boot_end + 1, self.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return '%s,%s,0xDA\n%s,%s,0x0C,*\n%s,,,-' % ( loader_start, loader_len, boot_start, boot_len, root_start) def _make_boot_files(self, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, d_img_data): # XXX: delete this method when hwpacks V1 can die assert self.hwpack_format == HardwarepackHandler.FORMAT_1 make_uImage(self.load_addr, k_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) self.populate_raw_partition(boot_device_or_file, chroot_dir) def populate_raw_partition(self, boot_device_or_file, chroot_dir): # Populate created raw partition with TOC and startup files. _, toc_filename = tempfile.mkstemp() config_files_dir = self.snowball_config(chroot_dir) new_files = self.get_file_info(chroot_dir, config_files_dir) with open(toc_filename, 'wb') as toc: self.create_toc(toc, new_files) self.install_snowball_boot_loader(toc_filename, new_files, boot_device_or_file, self.SNOWBALL_LOADER_START_S, self.delete_startupfiles) self.delete_file(toc_filename) if self.delete_startupfiles: self.delete_file(os.path.join(config_files_dir, self.snowball_startup_files_config)) def snowball_config(self, chroot_dir): # We will find the startupfiles in the target boot partition. return os.path.join(chroot_dir, 'boot') @property def delete_startupfiles(self): # The startupfiles will have been installed to the target boot # partition by the hwpack, and should be deleted so we don't leave # them on the target system. return True def install_snowball_boot_loader(self, toc_file_name, files, boot_device_or_file, start_sector, delete_startupfiles=False): ''' Copies TOC and boot files into the boot partition. A sector size of 1 is used for some files, as they do not necessarily start on an even address. ''' assert os.path.getsize(toc_file_name) <= self.TOC_SIZE _dd(toc_file_name, boot_device_or_file, seek=start_sector) for file in files: # XXX We need checks that these files do not overwrite each # other. This code assumes that offset and file sizes are ok. filename = file['filename'] if (file['offset'] % SECTOR_SIZE) != 0: seek_bytes = start_sector * SECTOR_SIZE + file['offset'] _dd(filename, boot_device_or_file, block_size=1, seek=seek_bytes) else: seek_sectors = start_sector + file['offset'] / SECTOR_SIZE _dd(filename, boot_device_or_file, seek=seek_sectors) if delete_startupfiles: self.delete_file(filename) def delete_file(self, file_path): cmd = ["rm", "%s" % file_path] proc = cmd_runner.run(cmd, as_root=True) proc.wait() def create_toc(self, f, files): ''' Writes a table of contents of the boot binaries. Boot rom searches this table to find the binaries.''' # Format string means: < little endian, # I; unsigned int; offset, # I; unsigned int; size, # I; unsigned int; flags, # i; int; align, # i; int; load_address, # 12s; string of char; name # http://igloocommunity.org/support/index.php/ConfigPartitionOverview toc_format = '/MLO) files = glob.glob( os.path.join(chroot_dir, 'usr', 'lib', '*', '*', 'MLO')) if len(files) == 0: # This one matches the path of MLO files installed by older # x-loader-omap package (e.g. /usr/lib/x-loader-omap[34]/MLO) files = glob.glob( os.path.join(chroot_dir, 'usr', 'lib', '*', 'MLO')) if len(files) == 1: return files[0] elif len(files) > 1: raise AssertionError( "More than one MLO file found on %s" % chroot_dir) else: raise AssertionError("No MLO files found on %s" % chroot_dir) def install_omap_boot_loader(chroot_dir, boot_disk, cls): with cls.hardwarepack_handler: try: default = _get_mlo_file(chroot_dir) except AssertionError: default = None mlo_file = cls.get_file('spl_file', default=default) cmd_runner.run(["cp", "-v", mlo_file, boot_disk], as_root=True).wait() def make_boot_ini(boot_script_path, boot_disk): proc = cmd_runner.run( ["cp", "-v", boot_script_path, "%s/boot.ini" % boot_disk], as_root=True) proc.wait() linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/chroot_utils.py0000644000175000017500000002273612724020110027263 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import sys from linaro_image_tools import cmd_runner from linaro_image_tools.utils import ( is_arm_host, find_command, ) from linaro_image_tools.hwpack.handler import HardwarepackHandler # It'd be nice if we could use atexit here, but all the things we need to undo # have to happen right after install_hwpacks completes and the atexit # functions would only be called after l-m-c.py exits. local_atexit = [] class ChrootException(Exception): """Base class for chroot exceptions.""" def prepare_chroot(chroot_dir, tmp_dir): """Prepares a chroot to run commands in it (networking and QEMU setup).""" chroot_etc = os.path.join(chroot_dir, 'etc') temporarily_overwrite_file_on_dir('/etc/resolv.conf', chroot_etc, tmp_dir) temporarily_overwrite_file_on_dir('/etc/hosts', chroot_etc, tmp_dir) if not is_arm_host(): for root, dirs, files in os.walk('/usr/bin'): for file in files: # Copy all the QEMU ARM binaries if file.startswith('qemu-arm') or \ file.startswith('qemu-aarch64'): file_name = os.path.join(root, file) copy_file(file_name, os.path.join(chroot_dir, 'usr', 'bin')) def install_hwpacks( rootfs_dir, tmp_dir, tools_dir, hwpack_force_yes, verified_files, extract_kpkgs=False, *hwpack_files): """Install the given hwpacks onto the given rootfs.""" install_command = 'linaro-hwpack-install' linaro_hwpack_install_path = find_command( install_command, prefer_dir=tools_dir) if not linaro_hwpack_install_path: raise ChrootException("The program linaro-hwpack-install could not " "be found found: cannot proceed.") else: linaro_hwpack_install_path = os.path.abspath( linaro_hwpack_install_path) # In case we just want to extract the kernel packages, don't force qemu # with chroot, as we could have archs without qemu support if not extract_kpkgs: prepare_chroot(rootfs_dir, tmp_dir) # FIXME: shouldn't use chroot/usr/bin as this might conflict with # installed packages; would be best to use some custom directory like # chroot/linaro-image-tools/bin copy_file(linaro_hwpack_install_path, os.path.join(rootfs_dir, 'usr', 'bin')) mount_chroot_proc(rootfs_dir) try: # Sometimes the host will have qemu-user-static installed but # another package (i.e. scratchbox) will have mangled its config # and thus we won't be able to chroot and install the hwpack, so # we fail here and tell the user to ensure qemu-arm-static is # setup before trying again. cmd_runner.run(['true'], as_root=True, chroot=rootfs_dir).wait() except: print ("Cannot proceed with hwpack installation because " "there doesn't seem to be a binfmt interpreter registered " "to execute arm binaries in the chroot. Please check " "that qemu-user-static is installed and properly " "configured before trying again.") raise else: # We are not in the chroot, we do not copy the linaro-hwpack-install # file, but we might not have l-i-t installed, so we need the full path # of the linaro-hwpack-install program to run. install_command = linaro_hwpack_install_path try: for hwpack_file in hwpack_files: hwpack_verified = False if os.path.basename(hwpack_file) in verified_files: hwpack_verified = True install_hwpack(rootfs_dir, hwpack_file, extract_kpkgs, hwpack_force_yes or hwpack_verified, install_command) finally: run_local_atexit_funcs() def install_hwpack(rootfs_dir, hwpack_file, extract_kpkgs, hwpack_force_yes, install_command): """Install an hwpack on the given rootfs. Copy the hwpack file to the rootfs and run linaro-hwpack-install passing that hwpack file to it. If hwpack_force_yes is True, also pass --force-yes to linaro-hwpack-install. In case extract_kpkgs is True, it will not install all the packages, but just extract the kernel ones. """ hwpack_basename = os.path.basename(hwpack_file) copy_file(hwpack_file, rootfs_dir) print "-" * 60 print "Installing (linaro-hwpack-install) %s in target rootfs." % ( hwpack_basename) # Get information required by linaro-hwpack-install with HardwarepackHandler([hwpack_file]) as hwpack: version, _ = hwpack.get_field("version") architecture, _ = hwpack.get_field("architecture") name, _ = hwpack.get_field("name") args = [install_command, '--hwpack-version', version, '--hwpack-arch', architecture, '--hwpack-name', name] if hwpack_force_yes: args.append('--force-yes') if extract_kpkgs: args.append('--extract-kernel-only') args.append(os.path.join(rootfs_dir, hwpack_basename)) chroot_dir = None else: args.append('/%s' % hwpack_basename) chroot_dir = rootfs_dir cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "-" * 60 def install_packages(chroot_dir, tmp_dir, *packages): """Install packages in the given chroot. This does not run apt-get update before hand.""" prepare_chroot(chroot_dir, tmp_dir) try: # TODO: Use the partition_mounted() contextmanager here and get rid of # mount_chroot_proc() altogether. mount_chroot_proc(chroot_dir) print "-" * 60 print "Installing (apt-get) %s in target rootfs." % " ".join(packages) args = ("apt-get", "--yes", "install") + packages cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "Cleaning up downloaded packages." args = ("apt-get", "clean") cmd_runner.run(args, as_root=True, chroot=chroot_dir).wait() print "-" * 60 finally: run_local_atexit_funcs() def mount_chroot_proc(chroot_dir): """Mount a /proc filesystem on the given chroot. Also register a function in local_atexit to unmount that /proc filesystem. """ chroot_proc = os.path.join(chroot_dir, 'proc') def umount_chroot_proc(): cmd_runner.run(['umount', '-v', chroot_proc], as_root=True).wait() local_atexit.append(umount_chroot_proc) proc = cmd_runner.run( ['mount', 'proc', chroot_proc, '-t', 'proc'], as_root=True) proc.wait() def copy_file(filepath, directory): """Copy the given file to the given directory. The copying of the file is done in a subprocess and run using sudo. We also register a function in local_atexit to remove the file from the given directory. """ cmd_runner.run(['cp', filepath, directory], as_root=True).wait() def undo(): new_path = os.path.join(directory, os.path.basename(filepath)) cmd_runner.run(['rm', '-f', new_path], as_root=True).wait() local_atexit.append(undo) def temporarily_overwrite_file_on_dir(filepath, directory, tmp_dir): """Temporarily replace a file on the given directory. We'll move the existing file on the given directory to a temp dir, then copy over the given file to that directory and register a function in local_atexit to move the orig file back to the given directory. """ basename = os.path.basename(filepath) path_to_orig = os.path.join(tmp_dir, basename) # Move the existing file from the given directory to the temp dir. oldpath = os.path.join(directory, basename) if os.path.lexists(oldpath): cmd_runner.run( ['mv', '-f', oldpath, path_to_orig], as_root=True).wait() # Now copy the given file onto the given directory. cmd_runner.run(['cp', '-a', filepath, directory], as_root=True).wait() def undo(): if os.path.lexists(path_to_orig): cmd_runner.run( ['mv', '-f', path_to_orig, directory], as_root=True).wait() else: cmd_runner.run( ['rm', '-f', oldpath], as_root=True).wait() local_atexit.append(undo) def run_local_atexit_funcs(): # Run the funcs in LIFO order, just like atexit does. exc_info = None while len(local_atexit) > 0: func = local_atexit.pop() try: func() except SystemExit: exc_info = sys.exc_info() except: import traceback print >> sys.stderr, "Error in local_atexit:" traceback.print_exc() exc_info = sys.exc_info() if exc_info is not None: raise exc_info[0], exc_info[1], exc_info[2] linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/tests/0000755000175000017500000000000012724020110025323 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/media_create/tests/fixtures.py0000644000175000017500000000462312724020110027553 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import subprocess from linaro_image_tools.media_create import partitions from linaro_image_tools.tests.fixtures import MockSomethingFixture class CreateTarballFixture(object): def __init__(self, basedir, reldir='tarball', filename='tarball.tar.gz'): self.basedir = basedir self.reldir = reldir self.tarball = os.path.join(self.basedir, filename) def setUp(self): # Create gzipped tar archive. os.mkdir(os.path.join(self.basedir, self.reldir)) args = ['tar', '-C', self.basedir, '-czf', self.tarball, self.reldir] proc = subprocess.Popen(args) proc.wait() def tearDown(self): if os.path.exists(self.tarball): os.remove(self.tarball) dir = os.path.join(self.basedir, self.reldir) if os.path.exists(dir): os.rmdir(dir) def get_tarball(self): return self.tarball class MockCallableWithPositionalArgs(object): """A callable mock which just stores the positional args given to it. Every time an instance of this is "called", it will append a tuple containing the positional arguments given to it to self.calls. """ calls = None return_value = None def __call__(self, *args): if self.calls is None: self.calls = [] self.calls.append(args) return self.return_value class MockRunSfdiskCommandsFixture(MockSomethingFixture): def __init__(self): mock = MockCallableWithPositionalArgs() mock.return_value = ('', '') super(MockRunSfdiskCommandsFixture, self).__init__( partitions, 'run_sfdisk_commands', mock) linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/tests/test_android_boards.py0000644000175000017500000005622212724020110031715 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from testtools import TestCase from linaro_image_tools.media_create.boards import ( BoardConfigException, ) from linaro_image_tools.media_create.android_boards import ( AndroidBeagleConfig, get_board_config, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import CreateTempFileFixture class TestAndroidBoards(TestCase): """Class to test small things in android_boards.""" def test_get_board_config(self): instance = get_board_config('beagle') self.assertIsInstance(instance, AndroidBeagleConfig) def test_get_board_config_wrong(self): self.assertRaises(BoardConfigException, get_board_config, 'notadevice') def test_hwpack_not_exists(self): instance = get_board_config('beagle') self.assertRaises(BoardConfigException, instance.from_file, 'a_file') class TestAndroidBoardsHwpack(TestCaseWithFixtures): """Class to test the new Android hwpack configuration file.""" # All the necessary Android hwpack fields for the tests. hwpack_format = 'format: 3.0\n' hwpack_dtb_name = 'dtb_name: %(dtb_name)s\n' hwpack_fdt_high = "fdt_high: '%(fdt_high)s'\n" hwpack_fat_size = 'fat_size: 16\n' hwpack_android_args = 'android_specific_args: %(android_specific_args)s\n' hwpack_extra_serial = 'extra_serial_options: %(extra_serial_options)s\n' hwpack_extra_boot = ('extra_boot_args_options: ' '%(extra_boot_args_options)s\n') hwpack_bootloader_flavor = 'bootloader_flavor: %(bootloader_flavor)s\n' hwpack_initrd_addr = 'initrd_addr: \'%(initrd_addr)s\'\n' hwpack_initrd_high = 'initrd_high: \'%(initrd_high)s\'\n' hwpack_kernel_addr = 'kernel_addr: \'%(kernel_addr)s\'\n' hwpack_load_addr = 'load_addr: \'%(load_addr)s\'\n' hwpack_dtb_addr = 'dtb_addr: \'%(dtb_addr)s\'\n' hwpack_boot_script = 'boot_script: %(boot_script)s\n' hwpack_mmc_option = 'mmc_option: \'%(mmc_option)s\'\n' # Some defaults YAML-like strings to use for the tests. android_hwpack_base = (hwpack_format + hwpack_dtb_name) android_hwpack_simple = (android_hwpack_base + hwpack_fdt_high + hwpack_fat_size) android_hwpack_android_args = (android_hwpack_base + hwpack_android_args) android_hwpack_extra_serial = (android_hwpack_base + hwpack_extra_serial) android_hwpack_extra_boot = (android_hwpack_base + hwpack_extra_boot) android_hwpack_panda = ( hwpack_format + hwpack_android_args + hwpack_bootloader_flavor + hwpack_dtb_addr + hwpack_dtb_name + hwpack_extra_boot + hwpack_extra_serial ) android_hwpack_mx6 = ( android_hwpack_panda + hwpack_initrd_addr + hwpack_kernel_addr + hwpack_load_addr ) android_hwpack_snowball_sd = ( hwpack_android_args + hwpack_boot_script + hwpack_dtb_addr + hwpack_dtb_name + hwpack_extra_boot + hwpack_extra_serial + hwpack_fdt_high + hwpack_format + hwpack_initrd_addr ) android_hwpack_snowball_emmc = ( android_hwpack_snowball_sd + hwpack_initrd_high + hwpack_mmc_option) def setUp(self): super(TestAndroidBoardsHwpack, self).setUp() # Pick a default board. self.config = get_board_config('beagle') def _get_tmp_file_name(self, content=None): name = self.useFixture(CreateTempFileFixture(content)).get_file_name() return name def assertBootEnv(self, expected, config=None, board='beagle'): """Helper function the boot env parameters. :param config: The string containing the YAML configuration. :type str :param expected: The expected configuration. :type dict :param board: The name of the board to test. Defaults to beagle. :type str """ board_conf = get_board_config(board) if config: name = self.useFixture(CreateTempFileFixture(config)).\ get_file_name() board_conf.from_file(name) self.assertEqual(expected, board_conf._get_boot_env(consoles=[])) def test_read_from_file(self): values = {'fdt_high': '0xFFFFFFFF', 'dtb_name': 'a_name'} expected = {'format': 3.0, 'dtb_name': 'a_name', 'fdt_high': '0xFFFFFFFF', 'fat_size': 16} yaml_conf = self.android_hwpack_simple % values name = self._get_tmp_file_name(yaml_conf) conf = self.config.from_file(name) self.assertEqual(expected, conf) def test_android_specific_args(self): """The field android_specific_args should be a concatenated string.""" values = {'android_specific_args': ['init=/init', 'androidboot.console=ttyO2'], 'dtb_name': 'a_name'} yaml_conf = self.android_hwpack_android_args % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'init=/init androidboot.console=ttyO2' self.assertEqual(expected, self.config.android_specific_args) def test_extra_serial_options(self): """The field extra_serial_options should be a concatenated string.""" values = {'dtb_name': 'a_name', 'extra_serial_options': ['console=tty0', 'console=ttyO2,115200n8']} yaml_conf = self.android_hwpack_extra_serial % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'console=tty0 console=ttyO2,115200n8' self.assertEqual(expected, self.config.extra_serial_options) def test_extra_boot_args_options(self): """The field extra_boot_args_options should be a concatenated string. Testing presence of a field defined in the parent class.""" values = { 'dtb_name': 'a_name', 'extra_boot_args_options': ['earlyprintk', 'mem=128M@0', 'mali.mali_mem=64M@128M'] } yaml_conf = self.android_hwpack_extra_boot % values name = self._get_tmp_file_name(yaml_conf) self.config.from_file(name) expected = 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M' self.assertEqual(expected, self.config.extra_boot_args_options) def test_android_mx6(self): values = { "android_specific_args": ["init=/init", "androidboot.console=%s"], "bootloader_flavor": "mx6qsabrelite", "dtb_addr": '0x11ff0000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "rootdelay=1", "fixrtc", "nocompcache", "di1_primary", "tve"], "extra_serial_options": ["console=%s,115200n8"], "initrd_addr": '0x12000000', "kernel_addr": '0x10000000', "load_addr": '0x10008000', } expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x10000000 uImage; ' 'fatload mmc 0:2 0x12000000 uInitrd; ' 'fatload mmc 0:2 0x11ff0000 board.dtb; ' 'bootm 0x10000000 0x12000000 0x11ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = self.android_hwpack_mx6 % values self.assertBootEnv(expected, config=config, board='mx6qsabrelite') def test_android_mx6_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x10000000 uImage; ' 'fatload mmc 0:2 0x12000000 uInitrd; ' 'fatload mmc 0:2 0x11ff0000 board.dtb; ' 'bootm 0x10000000 0x12000000 0x11ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='mx6qsabrelite') def test_panda(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyO2"], "bootloader_flavor": "omap4_panda", "dtb_addr": '0x815f0000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "fixrtc", "nocompcache", "vram=48M", "omapfb.vram=0:24M,1:24M", "mem=456M@0x80000000", "mem=512M@0xA0000000"], "extra_serial_options": ["console=ttyO2,115200n8"], } expected = { 'bootargs': 'console=ttyO2,115200n8 ' 'rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M,1:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000 ' 'init=/init androidboot.console=ttyO2', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = self.android_hwpack_panda % values self.assertBootEnv(expected, config=config, board='panda') def test_panda_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyO2,115200n8 ' 'rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M,1:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000 ' 'init=/init androidboot.console=ttyO2', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='panda') def test_android_snowball_sd(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyAMA2"], "boot_script": "boot.scr", "dtb_addr": '0x8000000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "mem=128M@0", "mali.mali_mem=64M@128M", "hwmem=168M@192M", "mem=22M@360M", "mem_issw=1M@383M", "mem=640M@384M", "vmalloc=500M"], "extra_serial_options": ["console=ttyAMA2,115200n8"], "fdt_high": '0x05000000', "initrd_addr": '0x05000000', "initrd_high": '0x06000000', } expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x05000000 uInitrd; ' 'fatload mmc 1:1 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} config = self.android_hwpack_snowball_sd % values self.assertBootEnv(expected, config=config, board='snowball_sd') def test_android_snowball_sd_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x05000000 uInitrd; ' 'fatload mmc 1:1 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} self.assertBootEnv(expected, board='snowball_sd') def test_android_snowball_emmc(self): values = { "android_specific_args": ["init=/init", "androidboot.console=ttyAMA2"], "boot_script": "boot.scr", "dtb_addr": '0x8000000', "dtb_name": "board.dtb", "extra_boot_args_options": ["earlyprintk", "mem=128M@0", "mali.mali_mem=64M@128M", "hwmem=168M@192M", "mem=22M@360M", "mem_issw=1M@383M", "mem=640M@384M", "vmalloc=500M"], "extra_serial_options": ["console=ttyAMA2,115200n8"], "fdt_high": '0x05000000', "initrd_addr": '0x05000000', "initrd_high": '0x06000000', "mmc_option": '0:2' } expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x05000000 uInitrd; ' 'fatload mmc 0:2 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} config = self.android_hwpack_snowball_emmc % values self.assertBootEnv(expected, config, board='snowball_emmc') def test_android_snowball_emmc_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttyAMA2,115200n8 ' 'rootwait ro earlyprintk ' 'mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M ' 'vmalloc=500M init=/init androidboot.console=ttyAMA2', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x05000000 uInitrd; ' 'fatload mmc 0:2 0x8000000 board.dtb; ' 'bootm 0x00100000 0x05000000 0x8000000', 'fdt_high': '0x05000000', 'initrd_high': '0x06000000'} self.assertBootEnv(expected, board='snowball_emmc') def test_android_origen(self): values = { "extra_serial_options": ["console=tty0", "console=ttySAC2,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=ttySAC2"] } expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='origen') def test_android_origen_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='origen') def test_android_origen_quad(self): values = { "extra_serial_options": ["console=tty0", "console=ttySAC2,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=ttySAC2"] } expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='origen_quad') def test_android_origen_quad_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttySAC2,115200n8 ' 'rootwait ro init=/init androidboot.console=ttySAC2', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='origen_quad') def test_android_vexpress(self): values = { "extra_serial_options": ["console=tty0", "console=ttyAMA0,38400n8"], "android_specific_args": ["init=/init", "androidboot.console=ttyAMA0"] } expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'rootwait ro init=/init androidboot.console=ttyAMA0', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='vexpress') def test_android_vexpress_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'rootwait ro init=/init androidboot.console=ttyAMA0', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='vexpress') def test_android_mx5(self): values = { "extra_boot_args_options": ["earlyprintk", "rootdelay=1", "fixrtc", "nocompcache", "di1_primary", "tve"], "extra_serial_options": ["console=%s,115200n8"], "android_specific_args": ["init=/init", "androidboot.console=%s"] } expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x70000000 uImage; ' 'fatload mmc 0:2 0x72000000 uInitrd; ' 'bootm 0x70000000 0x72000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} config = ((self.hwpack_format + self.hwpack_extra_boot + self.hwpack_extra_serial + self.hwpack_android_args) % values) self.assertBootEnv(expected, config=config, board='mx53loco') def test_android_mx5_old(self): # Old test: use the values from the class, instead of passing them. expected = { 'bootargs': 'console=ttymxc0,115200n8 ' 'rootwait ro earlyprintk rootdelay=1 fixrtc ' 'nocompcache di1_primary tve init=/init ' 'androidboot.console=ttymxc0', 'bootcmd': 'fatload mmc 0:2 0x70000000 uImage; ' 'fatload mmc 0:2 0x72000000 uInitrd; ' 'bootm 0x70000000 0x72000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertBootEnv(expected, board='mx53loco') def test_android_arndale_old(self): """Test that uses values taken directly from the class. """ expected = { 'bootargs': 'ttySAC2,115200n8 rootwait ro rootdelay=3 ' 'init=/init androidboot.console=ttySAC2 ' 'console=ttySAC2 initrd=0x41000000', 'bootcmd': 'fatload mmc 0:1 0x40007000 uImage; fatload mmc 0:1 ' '0x41000000 uInitrd; fatload mmc 0:1 0x41f00000 ' 'exynos5250-arndale.dtb; bootm 0x40007000 0x41000000 ' '0x41f00000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff', } self.assertBootEnv(expected, board='arndale') linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/tests/__init__.py0000644000175000017500000000201512724020110027432 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest def test_suite(): module_names = [ 'linaro_image_tools.media_create.tests.test_media_create', 'linaro_image_tools.media_create.tests.test_android_boards', ] loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) return suite linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/tests/test_media_create.py0000644000175000017500000054166512724020110031357 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import atexit import glob import os import random import string import subprocess import sys import tempfile import textwrap import time import types import struct import tarfile import dbus import shutil from mock import MagicMock from StringIO import StringIO from testtools import TestCase from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.hwpack.packages import PackageMaker import linaro_image_tools.media_create from linaro_image_tools.media_create import ( android_boards, boards, check_device, partitions, rootfs, ) from linaro_image_tools.media_create.boards import ( SECTOR_SIZE, align_up, align_partition, get_plain_boot_script_contents, make_flashable_env, install_mx5_boot_loader, install_omap_boot_loader, make_boot_script, make_uImage, make_uInitrd, make_dtb, _get_file_matching, _get_mlo_file, _run_mkimage, BoardConfig, get_board_config, ) from linaro_image_tools.media_create.android_boards import ( AndroidSnowballEmmcConfig, ) from linaro_image_tools.media_create.chroot_utils import ( copy_file, install_hwpack, install_hwpacks, install_packages, mount_chroot_proc, prepare_chroot, run_local_atexit_funcs, temporarily_overwrite_file_on_dir, ) from linaro_image_tools.media_create.partitions import ( MIN_IMAGE_SIZE, Media, _check_min_size, _get_device_file_for_partition_number, _parse_blkid_output, calculate_android_partition_size_and_offset, calculate_partition_size_and_offset, create_partitions, ensure_partition_is_not_mounted, get_android_loopback_devices, get_boot_and_root_loopback_devices, get_boot_and_root_partitions_for_media, get_partition_size_in_bytes, get_uuid, partition_mounted, run_sfdisk_commands, setup_partitions, wait_partition_to_settle, ) from linaro_image_tools.media_create.rootfs import ( append_to_fstab, create_flash_kernel_config, has_space_left_for_swap, move_contents, populate_rootfs, rootfs_mount_options, update_network_interfaces, write_data_to_protected_file, ) from linaro_image_tools.media_create.tests.fixtures import ( CreateTarballFixture, MockRunSfdiskCommandsFixture, ) from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_binary_tarball, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, MockCmdRunnerPopenFixture, MockSomethingFixture, ) from linaro_image_tools.utils import find_command, preferred_tools_dir from linaro_image_tools.hwpack.testing import ContextManagerFixture chroot_args = " ".join(cmd_runner.CHROOT_ARGS) sudo_args = " ".join(cmd_runner.SUDO_ARGS) class TestHardwarepackHandler(TestCaseWithFixtures): def setUp(self): super(TestHardwarepackHandler, self).setUp() self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) self.metadata = ( "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\nORIGIN=linaro\n") def test_hardwarepack_bootloaders(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], bootloader='u_boot') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'a_file') def test_hardwarepack_boards(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") metadata += ("boards:\n panda:\n bootloaders:\n u_boot:\n " "file: panda_file") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], board='panda') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'panda_file') def test_hardwarepack_boards_and_bootloaders(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") metadata += ("bootloaders:\n u_boot:\n file: a_file\n uefi:\n file: " "b_file\n") metadata += ("boards:\n panda:\n bootloaders:\n u_boot:\n " "file: panda_file\n uefi:\n file: " "uefi_panda_file\n") metadata += (" panda-lt:\n bootloaders:\n u_boot:\n " "file: panda_lt_file") data = '3.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', metadata)]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: self.assertEquals(hp.get_field('bootloader_file')[0], 'uefi_panda_file') def add_to_tarball(self, files, tarball=None): if tarball is None: tarball = self.tarball_fixture.get_tarball() tar_file = tarfile.open(tarball, mode='w:gz') for filename, data in files: tarinfo = tarfile.TarInfo(filename) tarinfo.size = len(data) tar_file.addfile(tarinfo, StringIO(data)) tar_file.close() return tarball def test_get_format_1(self): data = HardwarepackHandler.FORMAT_1 format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertEquals(hp.get_format(), data) def test_get_format_2(self): data = '2.0' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertEquals(hp.get_format(), data) def test_get_unknown_format_raises(self): data = '9.9' format = "%s\n" % data tarball = self.add_to_tarball( [('FORMAT', format), ('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertRaises(AssertionError, hp.get_format) def test_mixed_formats(self): format1 = "%s\n" % HardwarepackHandler.FORMAT_1 format2 = "%s\n" % HardwarepackHandler.FORMAT_2 tarball1 = self.add_to_tarball( [('FORMAT', format1), ('metadata', self.metadata)], tarball=self.tarball_fixture.get_tarball()) tarball_fixture2 = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', filename='secondtarball.tar.gz') self.useFixture(tarball_fixture2) tarball2 = self.add_to_tarball( [('FORMAT', format2), ('metadata', self.metadata)], tarball=tarball_fixture2.get_tarball()) hp = HardwarepackHandler([tarball2, tarball1]) with hp: self.assertEquals(hp.get_format(), '1.0and2.0') def test_identical_formats_ok(self): format1 = "%s\n" % HardwarepackHandler.FORMAT_2 format2 = "%s\n" % HardwarepackHandler.FORMAT_2 tarball1 = self.add_to_tarball( [('FORMAT', format1), ('metadata', self.metadata)], tarball=self.tarball_fixture.get_tarball()) tarball_fixture2 = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', filename='secondtarball.tar.gz') self.useFixture(tarball_fixture2) tarball2 = self.add_to_tarball( [('FORMAT', format2), ('metadata', self.metadata)], tarball=tarball_fixture2.get_tarball()) hp = HardwarepackHandler([tarball1, tarball2]) with hp: self.assertEquals(hp.get_format(), '2.0') def test_get_metadata(self): data = 'data to test' metadata = self.metadata + "U_BOOT=%s\n" % data tarball = self.add_to_tarball( [('metadata', metadata)]) hp = HardwarepackHandler([tarball]) with hp: test_data, _ = hp.get_field('bootloader_file') self.assertEqual(test_data, data) def test_preserves_formatters(self): data = '%s%d' metadata = self.metadata + "U_BOOT=%s\n" % data tarball = self.add_to_tarball( [('metadata', metadata)]) hp = HardwarepackHandler([tarball]) with hp: test_data, _ = hp.get_field('bootloader_file') self.assertEqual(test_data, data) def test_creates_tempdir(self): tarball = self.add_to_tarball( [('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: self.assertTrue(os.path.exists(hp.tempdir)) def test_tempfiles_are_removed(self): tempdir = None tarball = self.add_to_tarball( [('metadata', self.metadata)]) hp = HardwarepackHandler([tarball]) with hp: tempdir = hp.tempdir self.assertFalse(os.path.exists(tempdir)) def test_get_file(self): data = 'test file contents\n' file_in_archive = 'testfile' metadata = self.metadata + "%s=%s\n" % ('U_BOOT', file_in_archive) tarball = self.add_to_tarball( [('metadata', metadata), (file_in_archive, data)]) hp = HardwarepackHandler([tarball]) with hp: test_file = hp.get_file('bootloader_file') self.assertEquals(data, open(test_file, 'r').read()) def test_list_packages(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" tarball = self.add_to_tarball([ ("FORMAT", format), ("metadata", metadata), ("pkgs/foo_1-1_all.deb", ''), ("pkgs/bar_1-1_all.deb", ''), ]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: packages = hp.list_packages() names = [p[1] for p in packages] self.assertIn('pkgs/foo_1-1_all.deb', names) self.assertIn('pkgs/bar_1-1_all.deb', names) self.assertEqual(len(packages), 2) def test_find_package_for(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" tarball = self.add_to_tarball([ ("FORMAT", format), ("metadata", metadata), ("pkgs/foo_1-3_all.deb", ''), ("pkgs/foo_2-5_arm.deb", ''), ("pkgs/bar_1-3_arm.deb", ''), ]) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: self.assertEqual(hp.find_package_for("foo")[1], "pkgs/foo_1-3_all.deb") self.assertEqual(hp.find_package_for("bar")[1], "pkgs/bar_1-3_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2)[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2, revision=5)[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", version=2, revision=5, architecture="arm")[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", architecture="arm")[1], "pkgs/foo_2-5_arm.deb") self.assertEqual(hp.find_package_for("foo", architecture="all")[1], "pkgs/foo_1-3_all.deb") def test_get_file_from_package(self): metadata = ("format: 3.0\nname: ahwpack\nversion: 4\narchitecture: " "armel\norigin: linaro\n") format = "3.0\n" names = ['package0', 'package1', 'package2'] files = { names[0]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "usr/share/doc/u-boot-linaro-omap4-panda/copyright"], names[1]: ["usr/lib/u-boot/omap4_panda/u-boot2.img", "foo/bar", "flim/flam"], names[2]: ["some/path/config"]} # Generate some test packages maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) tarball_content = [("FORMAT", format), ("metadata", metadata)] package_names = [] for package_name in names: # The files parameter to make_package is a list of files to create. # These files are text files containing package_name and their # path. Since package_name is different for each package, this # gives each file a unique content. deb_file_path = maker.make_package(package_name, '1.0', {}, files=files[package_name]) name = os.path.basename(deb_file_path) tarball_content.append((os.path.join("pkgs", name), open(deb_file_path).read())) package_names.append(name) tarball = self.add_to_tarball(tarball_content) hp = HardwarepackHandler([tarball], board='panda', bootloader='uefi') with hp: path = hp.get_file_from_package("some/path/config", "package2") self.assertTrue(path.endswith("some/path/config")) class TestSetMetadata(TestCaseWithFixtures): class MockHardwarepackHandler(HardwarepackHandler): metadata_dict = {} def __enter__(self): return self def get_field(self, field): try: return self.metadata_dict[field], None except: return None, None def get_format(self): return '2.0' def get_file(self, file_alias): return None def test_does_not_set_if_old_format(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(None, board_conf.kernel_addr) def test_sets_kernel_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'kernel_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.kernel_addr) def test_sets_initrd_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'initrd_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.initrd_addr) def test_sets_load_addr(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'load_addr' data_to_set = '0x8123ABCD' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.load_addr) def test_sets_serial_tty(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'serial_tty' data_to_set = 'ttyAA' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.serial_tty) def test_sets_wired_interfaces(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'wired_interfaces' data_to_set = 'eth0 eth1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.wired_interfaces) def test_sets_wireless_interfaces(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'wireless_interfaces' data_to_set = 'wlan0 wl1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.wireless_interfaces) def test_sets_mmc_id(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'mmc_id' data_to_set = '0:1' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.mmc_option) self.assertEquals(0, board_conf.mmc_device_id) self.assertEquals(0, board_conf.mmc_part_offset) def test_sets_boot_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'boot_min_size' data_to_set = '100' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.BOOT_MIN_SIZE_S) def test_sets_root_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'root_min_size' data_to_set = '3' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.ROOT_MIN_SIZE_S) def test_sets_loader_min_size(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'loader_min_size' data_to_set = '2' expected = align_up(int(data_to_set) * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(expected, board_conf.LOADER_MIN_SIZE_S) def test_sets_partition_layout_32(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(32, board_conf.fat_size) def test_sets_partition_layout_16(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs16_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(16, board_conf.fat_size) def test_sets_partition_layout_raises(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'partition_layout' data_to_set = 'bootfs_bogus_rootfs' self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() self.assertRaises( AssertionError, board_conf.set_metadata, 'ahwpack.tar.gz') def test_sets_copy_files(self): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, 'HardwarepackHandler', self.MockHardwarepackHandler)) field_to_test = 'bootloader_copy_files' data_to_set = {'package': [{"source1": "dest1"}, {"source2": "dest2"}]} self.MockHardwarepackHandler.metadata_dict = { field_to_test: data_to_set, } board_conf = BoardConfig() board_conf.set_metadata('ahwpack.tar.gz') self.assertEquals(data_to_set, board_conf.bootloader_copy_files) class TestGetMLOFile(TestCaseWithFixtures): def test_mlo_from_new_xloader(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = os.path.join( tempdir, 'usr', 'lib', 'x-loader', 'omap3530beagle') os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertEquals( mlo_path, _get_mlo_file(tempdir)) def test_mlo_from_old_xloader(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = os.path.join(tempdir, 'usr', 'lib', 'x-loader-omap4') os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertEquals( mlo_path, _get_mlo_file(tempdir)) def test_no_mlo_found(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, _get_mlo_file, tempdir) def test_more_than_one_mlo_found(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() for directory in ['x-loader-omap4', 'x-loader-omap3']: path = os.path.join(tempdir, 'usr', 'lib', directory) os.makedirs(path) mlo_path = os.path.join(path, 'MLO') open(mlo_path, 'w').close() self.assertRaises( AssertionError, _get_mlo_file, tempdir) def _create_uboot_dir(root, flavor): path = os.path.join(root, 'usr', 'lib', 'u-boot', flavor) os.makedirs(path) return path class TestGetSMDKSPL(TestCaseWithFixtures): def setUp(self): super(TestGetSMDKSPL, self).setUp() self.config = boards.SMDKV310Config() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_old_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'v310_mmc_spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) def test_prefers_old_path(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) old_spl_path = os.path.join(path, 'v310_mmc_spl.bin') new_spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(old_spl_path, 'w').close() open(new_spl_path, 'w').close() self.assertEquals(old_spl_path, self.config._get_samsung_spl(tempdir)) class TestGetSMDKUboot(TestCaseWithFixtures): def setUp(self): super(TestGetSMDKUboot, self).setUp() self.config = boards.SMDKV310Config() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_uses_uboot_flavour(self): chroot_dir = "chroot" uboot_file = os.path.join(chroot_dir, 'usr', 'lib', 'u-boot', self.config.bootloader_flavor, 'u-boot.bin') self.assertEquals( uboot_file, self.config._get_samsung_bootloader(chroot_dir)) class TestGetOrigenSPL(TestCaseWithFixtures): def setUp(self): super(TestGetOrigenSPL, self).setUp() self.config = boards.OrigenConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetOrigenUboot(TestGetSMDKUboot): def setUp(self): super(TestGetOrigenUboot, self).setUp() self.config = boards.OrigenConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 class TestGetOrigenQuadSPL(TestCaseWithFixtures): def setUp(self): super(TestGetOrigenQuadSPL, self).setUp() self.config = boards.OrigenQuadConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'origen_quad-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetOrigenQuadUboot(TestGetSMDKUboot): def setUp(self): super(TestGetOrigenQuadUboot, self).setUp() self.config = boards.OrigenQuadConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 class TestGetArndaleSPL(TestCaseWithFixtures): def setUp(self): super(TestGetArndaleSPL, self).setUp() self.config = boards.ArndaleConfig() self.config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_no_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( AssertionError, self.config._get_samsung_spl, tempdir) def test_new_file_present(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() path = _create_uboot_dir(tempdir, self.config.bootloader_flavor) spl_path = os.path.join(path, 'smdk5250-spl.bin') open(spl_path, 'w').close() self.assertEquals(spl_path, self.config._get_samsung_spl(tempdir)) class TestGetArndaleUboot(TestGetSMDKUboot): config = boards.ArndaleConfig class TestArndaleBootFiles(TestCaseWithFixtures): def test_arndale_make_boot_files_v2(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) board_conf = boards.ArndaleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.load_addr = '0x40008000' board_conf.boot_script = None board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.temp_bootdir_path = os.path.join(self.tempdir, 'boot') self.temp_bl0_path = os.path.join(self.tempdir, 'lib', 'firmware', 'arndale') k_img_file = os.path.join(self.tempdir, 'vmlinuz-1-arndale') i_img_file = os.path.join(self.tempdir, 'initrd.img-1-arndale') bl0_file = os.path.join(self.temp_bl0_path, 'arndale-bl1.bin') os.makedirs(self.temp_bl0_path) open(bl0_file, 'w').close() boot_env = {'ethact': 'smc911x-0', 'initrd_high': '0xffffffff', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'bootcmd': 'fatload mmc 0:2 None uImage; bootm None', 'bootargs': 'root=UUID=test_boot_env_uuid rootwait ro'} board_conf._make_boot_files_v2( boot_env=boot_env, chroot_dir=self.tempdir, boot_dir=self.temp_bootdir_path, boot_device_or_file='boot_device_or_file', k_img_data=k_img_file, i_img_data=i_img_file, d_img_data=None) expected_commands = [ ('sudo -E dd if=%s of=boot_device_or_file bs=512 conv=notrunc ' 'seek=1' % bl0_file), ('sudo -E mkimage -A arm -O linux -T kernel -C none -a %s -e %s ' '-n Linux -d %s %s/uImage' % (board_conf.load_addr, board_conf.load_addr, k_img_file, self.temp_bootdir_path)), ('sudo -E mkimage -A arm -O linux -T ramdisk -C none -a 0 -e 0 ' '-n initramfs -d %s %s/uInitrd' % (i_img_file, self.temp_bootdir_path))] self.assertEqual(expected_commands, popen_fixture.mock.commands_executed) shutil.rmtree(self.tempdir) class TestCreateToc(TestCaseWithFixtures): ''' Tests boards.SnowballEmmcConfig.create_toc()''' def setUp(self): ''' Create a temporary directory to work in''' super(TestCreateToc, self).setUp() self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() #Create the test's input data structures zero = '\x00\x00\x00\x00' line1 = zero + zero + zero + zero + zero + 'b' + zero + zero + \ '\x00\x00\x00' maxint = '\xFF\xFF\xFF\x7F' minint = '\xFF\xFF\xFF\xFF' line2 = maxint + maxint + zero + minint + minint + \ 'hello' + zero + '\x00\x00\x00' line3 = '\x01\x00\x00\x00' '\x64\x00\x00\x00' + zero + \ '\x05\x00\x00\x00' '\x05\x00\x00\x00' \ 'hello' + zero + '\x00\x00\x00' self.expected = line1 + line2 + line3 self.board_conf = boards.SnowballEmmcConfig() def create_files_structure(self, src_data): ''' Creates the data structure that the tested function needs as input''' files = [] for line in src_data: files.append({'section_name': line[5], 'filename': 'N/A', 'align': line[3], 'offset': line[0], 'size': line[1], 'load_adress': 'N/A'}) return files def test_create_toc_normal_case(self): ''' Creates a toc file, and then reads the created file and compares it to precomputed data''' correct_data = [(0, 0, 0, 0, 0, 'b'), (0x7FFFFFFF, 0x7FFFFFFF, 0x7FFFFFFF, -1, -1, 'hello'), (1, 100, 1000, 5, 10, 'hello')] files = self.create_files_structure(correct_data) filename = os.path.join(self.tempdir, 'toc') with open(filename, 'w') as f: self.board_conf.create_toc(f, files) with open(filename, 'r') as f: actual = f.read() self.assertEquals(96, len(actual)) for i in range(len(actual)): self.assertEquals( self.expected[i], actual[i], 'Mismatch at ix' ' %d, ref=%c, actual=%c' % (i, self.expected[i], actual[i])) def test_create_toc_error_too_large_section_name(self): '''Verify that trying to write past the end of the section name field raises an exception''' illegal_name_data = [(0, 0, 0, 0, 0, 'Too_longName')] files = self.create_files_structure(illegal_name_data) with open(os.path.join(self.tempdir, 'toc'), 'w') as f: self.assertRaises(AssertionError, self.board_conf.create_toc, f, files) def test_create_toc_error_negative_unsigned(self): '''Verify that trying to write a negative number to an unsigned field raises an exception''' illegal_unsigned_data = [(-3, 0, 0, 0, 0, 'xxx')] files = self.create_files_structure(illegal_unsigned_data) with open(os.path.join(self.tempdir, 'toc'), 'w') as f: self.assertRaises(struct.error, self.board_conf.create_toc, f, files) class TestSnowballBootFiles(TestCaseWithFixtures): ''' Tests boards.SnowballEmmcConfig.install_snowball_boot_loader()''' ''' Tests boards.SnowballEmmcConfig._make_boot_files()''' ''' Tests boards.SnowballEmmcConfig.get_file_info()''' def setUp(self): ''' Create temporary directory to work in''' super(TestSnowballBootFiles, self).setUp() self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.temp_bootdir_path = os.path.join(self.tempdir, 'boot') self.temp_configdir_path = os.path.join(self.tempdir, 'startfiles') if not os.path.exists(self.temp_bootdir_path): os.makedirs(self.temp_bootdir_path) if not os.path.exists(self.temp_configdir_path): os.makedirs(self.temp_configdir_path) self.snowball_config = get_board_config('snowball_emmc') self.snowball_config.hwpack_format = HardwarepackHandler.FORMAT_1 def setupFiles(self): return self.create_test_files(self.temp_bootdir_path) def setupAndroidFiles(self): return self.create_test_files(self.temp_configdir_path) def create_test_files(self, path): ''' Adds some files in the temp dir that the tested function can use as input: * A config file, which the tested function reads to discover which binary files should be written to the loader partition. * Test versions of the binary files themselves, containing dummy data. Returns the expected value that the tested function should return, given these input files. ''' src_data = [('ISSW', 'boot_image_issw.bin', -1, 0, '5'), ('X-LOADER', 'boot_image_x-loader.bin', -1, 0, '6'), ('MEM_INIT', 'mem_init.bin', 0, 0x160000, '7'), ('PWR_MGT', 'power_management.bin', 0, 0x170000, '8'), ('NORMAL', 'u-boot.bin', 0, 0xBA0000, '9'), ('UBOOT_ENV', 'u-boot-env.bin', 0, 0x00C1F000, '10')] # Create a config file cfg_file = os.path.join( path, self.snowball_config.snowball_startup_files_config) with open(cfg_file, 'w') as f: for line in src_data: # Write comments, so we test that the parser can read them f.write('#Yet another comment\n') # Write whitespace, so we test that the parser can handle them f.write(' \n\t\n \t \t \n') f.write('%s %s %i %#x %s\n' % line) expected = [] # Define dummy binary files, containing nothing but their own # section names. for line in src_data: with open(os.path.join(path, line[1]), 'w') as f: f.write(line[0]) #define the expected values read from the config file expected = [] ofs = [self.snowball_config.TOC_SIZE, self.snowball_config.TOC_SIZE + len('ISSW'), 0x160000, 0x170000, 0xBA0000, 0xC1F000] size = [len('ISSW'), len('X-LOADER'), len('MEM_INIT'), len('PWR_MGT'), len('NORMAL'), len('UBOOT_ENV')] i = 0 for line in src_data: filename = os.path.join(path, line[1]) expected.append({'section_name': line[0], 'filename': filename, 'align': int(line[2]), 'offset': ofs[i], 'size': long(size[i]), 'load_adress': line[4]}) i += 1 return expected def test_get_file_info_relative_path(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) uboot_file = 'u-boot.bin' with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ('NORMAL', uboot_file, 0, 0xBA0000, '9')) with open(os.path.join(self.temp_bootdir_path, uboot_file), 'w') as f: file_info = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(file_info[0]['filename'], os.path.join(self.temp_bootdir_path, uboot_file)) def test_get_file_info_abs_path(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) uboot_dir = tempfile.mkdtemp(dir=self.tempdir) uboot_file = os.path.join(uboot_dir, 'u-boot.bin') uboot_relative_file = uboot_file.replace(self.tempdir, '') with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ( 'NORMAL', uboot_relative_file, 0, 0xBA0000, '9')) with open(uboot_file, 'w') as f: file_info = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(file_info[0]['filename'], uboot_file) def test_get_file_info_raises(self): # Create a config file cfg_file = os.path.join( self.temp_bootdir_path, self.snowball_config.snowball_startup_files_config) with open(cfg_file, 'w') as f: f.write('%s %s %i %#x %s\n' % ('NORMAL', 'u-boot.bin', 0, 0xBA0000, '9')) self.assertRaises( AssertionError, self.snowball_config.get_file_info, self.tempdir, self.temp_bootdir_path) def test_file_name_size(self): ''' Test using a to large toc file ''' _, toc_filename = tempfile.mkstemp() atexit.register(os.unlink, toc_filename) filedata = 'X' bytes = self.snowball_config.TOC_SIZE + 1 tmpfile = open(toc_filename, 'wb') for n in xrange(bytes): tmpfile.write(filedata) tmpfile.close() files = self.setupFiles() self.assertRaises( AssertionError, self.snowball_config.install_snowball_boot_loader, toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S) def test_install_snowball_boot_loader_toc_dont_delete(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() self.snowball_config.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, self.snowball_config.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_snowball_boot_loader_toc_delete(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() self.snowball_config.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", self.snowball_config.SNOWBALL_LOADER_START_S, True) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, self.snowball_config.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s rm %s/boot_image_issw.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s rm %s/boot_image_x-loader.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s rm %s/power_management.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path), '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_snowball_boot_loader_toc_android(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) toc_filename = self.createTempFileAsFixture() files = self.setupFiles() board_conf = AndroidSnowballEmmcConfig() board_conf.install_snowball_boot_loader( toc_filename, files, "boot_device_or_file", board_conf.SNOWBALL_LOADER_START_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' ' seek=%s' % (sudo_args, toc_filename, board_conf.SNOWBALL_LOADER_START_S), '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_snowball_make_boot_files(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture( MockSomethingFixture(tempfile, 'mkstemp', lambda: (-1, '/tmp/temp_snowball_make_boot_files'))) self.setupFiles() k_img_file = os.path.join(self.tempdir, 'vmlinuz-1-ux500') i_img_file = os.path.join(self.tempdir, 'initrd.img-1-ux500') boot_env = self.snowball_config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=test_boot_env_uuid", i_img_data=None, d_img_data=None) self.snowball_config._make_boot_files(boot_env, self.tempdir, self.temp_bootdir_path, 'boot_device_or_file', k_img_file, i_img_file, None) expected = [ '%s mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e' ' 0x00008000 -n Linux -d %s %s/boot/uImage' % (sudo_args, k_img_file, self.tempdir), '%s cp /tmp/temp_snowball_make_boot_files %s/boot/boot.txt' % (sudo_args, self.tempdir), '%s mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n boot' ' script -d %s/boot/boot.txt %s/boot/flash.scr' % (sudo_args, self.tempdir, self.tempdir), '%s dd if=/tmp/temp_snowball_make_boot_files' ' of=boot_device_or_file bs=512 conv=notrunc seek=256' % (sudo_args), '%s dd if=%s/boot/boot_image_issw.bin of=boot_device_or_file' ' bs=512 conv=notrunc seek=257' % (sudo_args, self.tempdir), '%s rm %s/boot_image_issw.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/boot_image_x-loader.bin of=boot_device_or_file' ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.tempdir), '%s rm %s/boot_image_x-loader.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/mem_init.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=3072' % (sudo_args, self.tempdir), '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/power_management.bin of=boot_device_or_file' ' bs=512 conv=notrunc seek=3200' % (sudo_args, self.tempdir), '%s rm %s/power_management.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/u-boot.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=24064' % (sudo_args, self.tempdir), '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), '%s dd if=%s/boot/u-boot-env.bin of=boot_device_or_file bs=512' ' conv=notrunc seek=25080' % (sudo_args, self.tempdir), '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path), '%s rm /tmp/temp_snowball_make_boot_files' % (sudo_args), '%s rm %s/startfiles.cfg' % (sudo_args, self.temp_bootdir_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_missing_files(self): '''When the files cannot be read, an IOError should be raised''' self.assertRaises(IOError, self.snowball_config.get_file_info, self.tempdir, self.temp_bootdir_path) def test_normal_case(self): expected = self.setupFiles() actual = self.snowball_config.get_file_info( self.tempdir, self.temp_bootdir_path) self.assertEquals(expected, actual) class TestBootSteps(TestCaseWithFixtures): def setUp(self): super(TestBootSteps, self).setUp() self.funcs_calls = [] self.mock_all_boards_funcs() def mock_all_boards_funcs(self): """Mock functions of boards module with a call tracer.""" def mock_func_creator(name): return lambda *args, **kwargs: self.funcs_calls.append(name) for name in dir(boards): attr = getattr(boards, name) if isinstance(attr, types.FunctionType): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, name, mock_func_creator(name))) def mock_set_appropriate_serial_tty(self, config): def set_appropriate_serial_tty_mock(chroot_dir): config.serial_tty = config._serial_tty config.set_appropriate_serial_tty = MagicMock( side_effect=set_appropriate_serial_tty_mock) def make_boot_files(self, config): def _get_kflavor_files_mock(path): if config.dtb_name is None: return (path, path, None) return (path, path, path) config._get_kflavor_files = MagicMock( side_effect=_get_kflavor_files_mock) config.make_boot_files('', False, False, [], '', '', '', '') def test_vexpress_steps(self): board_conf = boards.VexpressConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd'] self.assertEqual(expected, self.funcs_calls) def test_vexpress_a9_steps(self): board_conf = boards.VexpressA9Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd'] self.assertEqual(expected, self.funcs_calls) def test_mx5_steps(self): board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.bootloader_flavor = 'bootloader_flavor' board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_mx5_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_smdkv310_steps(self): board_conf = boards.SMDKV310Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_origen_steps(self): board_conf = boards.OrigenConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_steps(self): board_conf = boards.OrigenQuadConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_arndale_steps(self): board_conf = boards.ArndaleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.install_samsung_boot_loader = MagicMock() board_conf.install_samsung_boot_loader.return_value = \ self.funcs_calls.append('install_samsung_boot_loader') self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: '1.0') self.make_boot_files(board_conf) expected = [ 'install_samsung_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_ux500_steps(self): board_conf = boards.Ux500Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_steps(self): board_conf = boards.SnowballSdConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.make_boot_files(board_conf) expected = ['make_uImage', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) def test_panda_steps(self): board_conf = boards.PandaConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_beagle_steps(self): board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_igep_steps(self): board_conf = boards.IgepConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_overo_steps(self): board_conf = boards.OveroConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_highbank_steps(self): board_conf = boards.HighBankConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 expected = [] self.assertEqual(expected, self.funcs_calls) def test_beaglbone_steps(self): board_conf = boards.BeagleBoneConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) self.make_boot_files(board_conf) expected = [ 'install_omap_boot_loader', 'make_uImage', 'make_uInitrd', 'make_dtb', 'make_boot_script', 'make_boot_ini'] self.assertEqual(expected, self.funcs_calls) def test_aa9_steps(self): board_conf = boards.Aa9Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 expected = [] self.assertEqual(expected, self.funcs_calls) class TestPopulateRawPartition(TestCaseWithFixtures): def setUp(self): super(TestPopulateRawPartition, self).setUp() self.funcs_calls = [] self.mock_all_boards_funcs() def mock_all_boards_funcs(self): """Mock functions of boards module with a call tracer.""" def mock_func_creator(name): return lambda *args, **kwargs: self.funcs_calls.append(name) for name in dir(boards): attr = getattr(boards, name) if isinstance(attr, types.FunctionType): self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards, name, mock_func_creator(name))) def populate_raw_partition(self, config): config.populate_raw_partition('', '') def test_snowball_config_raises(self): self.assertRaises(NotImplementedError, boards.SnowballSdConfig().snowball_config, '') def test_beagle_raw(self): self.populate_raw_partition(android_boards.AndroidBeagleConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_panda_raw(self): self.populate_raw_partition(android_boards.AndroidPandaConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_raw(self): self.populate_raw_partition(boards.SnowballSdConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_emmc_raw(self): def mock_func_creator(name): return classmethod( lambda *args, **kwargs: self.funcs_calls.append(name)) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'get_file_info', mock_func_creator('get_file_info'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'create_toc', mock_func_creator('create_toc'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'install_snowball_boot_loader', mock_func_creator('install_snowball_boot_loader'))) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.boards.SnowballEmmcConfig, 'delete_file', mock_func_creator('delete_file'))) self.populate_raw_partition(boards.SnowballEmmcConfig()) expected = ['get_file_info', 'create_toc', 'install_snowball_boot_loader', 'delete_file', 'delete_file'] # Test that we run the Snowball populate_raw_partition() and # delete both the toc and startfiles. self.assertEqual(expected, self.funcs_calls) def test_smdkv310_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.SMDKV310Config()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_mx53loco_raw(self): self.populate_raw_partition(boards.Mx53LoCoConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_origen_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.OrigenConfig()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raw(self): # Need to mock this since files do not exist here, and # an Exception is raised. self.useFixture( MockSomethingFixture(os.path, 'exists', lambda exists: True)) self.populate_raw_partition(boards.OrigenQuadConfig()) expected = ['_dd', '_dd', '_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raises(self): board_conf = boards.OrigenQuadConfig() self.assertRaises( boards.BoardException, board_conf.populate_raw_partition, '', '') def test_arndale_raw(self): self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(boards.ArndaleConfig()) expected = ['_dd', '_dd', '_dd'] self.assertEqual(expected, self.funcs_calls) def test_vexpress_a9_raw(self): self.populate_raw_partition(boards.VexpressA9Config()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_highbank_raw(self): self.populate_raw_partition(boards.HighBankConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_beaglebone_raw(self): self.populate_raw_partition(boards.BeagleBoneConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_aa9_raw(self): self.populate_raw_partition(boards.Aa9Config()) expected = [] self.assertEqual(expected, self.funcs_calls) class TestPopulateRawPartitionAndroid(TestCaseWithFixtures): def setUp(self): super(TestPopulateRawPartitionAndroid, self).setUp() self.funcs_calls = [] def populate_raw_partition(self, config): config.populate_raw_partition('', '') def test_beagle_raw(self): self.populate_raw_partition(android_boards.AndroidBeagleConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_panda_raw(self): self.populate_raw_partition(android_boards.AndroidPandaConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_sd_raw(self): self.populate_raw_partition(android_boards.AndroidSnowballSdConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_snowball_emmc_raw(self): def mock_func_creator(name): return classmethod( lambda *args, **kwargs: self.funcs_calls.append(name)) self.useFixture(MockSomethingFixture(os.path, 'exists', lambda file: True)) fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = ['sudo -E cp boot/u-boot.bin ./startupfiles'] self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'get_file_info', mock_func_creator('get_file_info'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'create_toc', mock_func_creator('create_toc'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'install_snowball_boot_loader', mock_func_creator('install_snowball_boot_loader'))) self.useFixture(MockSomethingFixture( android_boards.AndroidSnowballEmmcConfig, 'delete_file', mock_func_creator('delete_file'))) self.populate_raw_partition(android_boards.AndroidSnowballEmmcConfig()) expected_calls = ['get_file_info', 'create_toc', 'install_snowball_boot_loader', 'delete_file'] # Test that we copy the u-boot files to the local startupfiles dir. self.assertEqual(expected_commands, fixture.mock.commands_executed) # Test that we run the Snowball populate_raw_partition() and only # delete the toc. self.assertEqual(expected_calls, self.funcs_calls) def test_smdkv310_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ 'sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 seek=33', ('sudo -E dd if=boot/u-boot-mmc-spl.bin of= ' 'bs=512 conv=notrunc seek=1'), 'sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc seek=65'] self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(android_boards.AndroidSMDKV310Config()) expected_calls = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected_calls, self.funcs_calls) def test_mx53loco_raw(self): self.populate_raw_partition(android_boards.AndroidMx53LoCoConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) def test_origen_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ 'sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 seek=33', ('sudo -E dd if=boot/u-boot-mmc-spl.bin of= bs=512 ' 'conv=notrunc seek=1'), 'sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc seek=65'] self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) self.populate_raw_partition(android_boards.AndroidOrigenConfig()) expected = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected, self.funcs_calls) def test_origen_quad_raw(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) expected_commands = [ ('sudo -E dd if=/dev/zero of= bs=512 conv=notrunc count=32 ' 'seek=1601'), ('sudo -E dd if=boot/origen_quad.bl1.bin of= bs=512 ' 'conv=notrunc seek=1'), ('sudo -E dd if=boot/origen_quad-spl.bin.signed of= bs=512 ' 'conv=notrunc seek=31'), ('sudo -E dd if=boot/u-boot.bin of= bs=512 conv=notrunc ' 'seek=63'), ('sudo -E dd if=boot/exynos4x12.tzsw.signed.img of= bs=512 ' 'conv=notrunc seek=761') ] self.useFixture( MockSomethingFixture(os.path, 'exists', lambda exists: True)) self.populate_raw_partition(android_boards.AndroidOrigenQuadConfig()) expected = [] # Test that we dd the files self.assertEqual(expected_commands, fixture.mock.commands_executed) self.assertEqual(expected, self.funcs_calls) def test_vexpress_raw(self): self.populate_raw_partition(android_boards.AndroidVexpressConfig()) expected = [] self.assertEqual(expected, self.funcs_calls) class TestAlignPartition(TestCase): def test_align_up_none(self): self.assertEqual(1024, align_up(1024, 1)) def test_align_up_no_rounding(self): self.assertEqual(512, align_up(512, 512)) def test_align_up_rounding(self): self.assertEqual(512, align_up(1, 512)) def test_align_partition_4_mib_4_mib(self): expected = (4 * 1024 * 1024, 8 * 1024 * 1024 - 1, 4 * 1024 * 1024) self.assertEqual(expected, align_partition(1, 1, 4 * 1024 * 1024, 4 * 1024 * 1024)) def test_align_partition_none_4_mib(self): expected = (1, 4 * 1024 * 1024 - 1, 4 * 1024 * 1024 - 1) self.assertEqual(expected, align_partition(1, 1, 1, 4 * 1024 * 1024)) class TestFixForBug697824(TestCaseWithFixtures): def mock_set_appropriate_serial_tty(self, config): def set_appropriate_serial_tty_mock(arg): self.set_appropriate_serial_tty_called = True # Need to mock all the calls done from make_boot_files in order # to be able to correctly call it. config._get_kflavor_files = MagicMock(return_value=('', '', '')) config._get_boot_env = MagicMock(return_value=None) config._make_boot_files = MagicMock() config._make_boot_files_v2 = MagicMock() config.set_appropriate_serial_tty = MagicMock( side_effect=set_appropriate_serial_tty_mock) def test_omap_make_boot_files(self): self.set_appropriate_serial_tty_called = False board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.mock_set_appropriate_serial_tty(board_conf) # We don't need to worry about what's passed to make_boot_files() # because we mock the method which does the real work above and here # we're only interested in ensuring that OmapConfig.make_boot_files() # calls set_appropriate_serial_tty(). board_conf.make_boot_files( None, None, None, None, None, None, None, None) self.assertTrue( self.set_appropriate_serial_tty_called, "make_boot_files didn't call set_appropriate_serial_tty") def test_omap_make_boot_files_v2(self): self.set_appropriate_serial_tty_called = False board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_2 self.mock_set_appropriate_serial_tty(board_conf) # We don't need to worry about what's passed to make_boot_files() # because we mock the method which does the real work above and here # we're only interested in ensuring that OmapConfig.make_boot_files() # does not call set_appropriate_serial_tty(). board_conf.make_boot_files( None, None, None, None, None, None, None, None) self.assertFalse( self.set_appropriate_serial_tty_called, "make_boot_files called set_appropriate_serial_tty") def test_set_appropriate_serial_tty_old_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-2.6.35-23-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyS2', board_conf.serial_tty) def test_set_appropriate_serial_tty_new_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-2.6.36-13-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyO2', board_conf.serial_tty) def test_set_appropriate_serial_tty_three_dot_oh_kernel(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir boot_dir = os.path.join(tempdir, 'boot') os.makedirs(boot_dir) open(os.path.join(boot_dir, 'vmlinuz-3.0-13-foo'), 'w').close() board_conf = boards.BeagleConfig() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 board_conf.set_appropriate_serial_tty(tempdir) self.assertEquals('ttyO2', board_conf.serial_tty) class TestGetSfdiskCmd(TestCase): def set_up_config(self, config): config.hwpack_format = HardwarepackHandler.FORMAT_1 def test_default(self): board_conf = BoardConfig() self.set_up_config(board_conf) self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_default_aligned(self): board_conf = BoardConfig() self.set_up_config(board_conf) self.assertEqual( '8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd(should_align_boot_part=True)) def test_mx5(self): board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 self.assertEqual( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_sd(self): board_conf = boards.SnowballSdConfig() self.set_up_config(board_conf) self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_emmc(self): board_conf = boards.SnowballEmmcConfig() self.set_up_config(board_conf) self.assertEqual( '256,7936,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_smdkv310(self): board_conf = get_board_config('smdkv310') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen(self): board_conf = get_board_config('origen') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen_quad(self): board_conf = get_board_config('origen_quad') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_arndale(self): board_conf = get_board_config('arndale') self.set_up_config(board_conf) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_highbank(self): board_conf = get_board_config('highbank') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x83,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_beaglebone(self): board_conf = get_board_config('beaglebone') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_aa9(self): board_conf = get_board_config('aa9') self.set_up_config(board_conf) self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_panda_android(self): self.assertEqual( '63,270272,0x0C,*\n270336,1835008,L\n2105344,524288,L\n' '2629632,-,E\n2629632,1179648,L\n3809280,,,-', android_boards.AndroidPandaConfig().get_sfdisk_cmd()) def test_origen_android(self): self.assertEqual( '1,8191,0xDA\n8253,270274,0x0C,*\n278528,1835008,L\n' '2113536,-,E\n2113536,524288,L\n2637824,1179648,L\n3817472,,,-', android_boards.AndroidOrigenConfig().get_sfdisk_cmd()) def test_origen_quad_android(self): self.assertEqual( '1,8191,0xDA\n8253,270274,0x0C,*\n278528,1835008,L\n' '2113536,-,E\n2113536,524288,L\n2637824,1179648,L\n3817472,,,-', android_boards.AndroidOrigenQuadConfig().get_sfdisk_cmd()) def test_snowball_emmc_android(self): self.assertEqual( '256,7936,0xDA\n8192,262144,0x0C,*\n270336,1835008,L\n' '2105344,-,E\n2105344,524288,L\n2629632,1179648,L\n3809280,,,-', android_boards.AndroidSnowballEmmcConfig().get_sfdisk_cmd()) def test_vexpress_android(self): self.assertEqual( '63,270272,0x0E,*\n270336,1835008,L\n2105344,524288,L\n' '2629632,-,E\n2629632,1179648,L\n3809280,,,-', android_boards.AndroidVexpressConfig().get_sfdisk_cmd()) def test_mx5_android(self): self.assertEqual( '1,8191,0xDA\n8192,262144,0x0C,*\n270336,1835008,L\n' '2105344,-,E\n2105344,524288,L\n2629632,1179648,L\n3809280,,,-', android_boards.AndroidMx53LoCoConfig().get_sfdisk_cmd()) def test_mx6_android(self): self.assertEqual( '1,8191,0xDA\n8192,262144,0x0C,*\n270336,1835008,L\n' '2105344,-,E\n2105344,524288,L\n2629632,1179648,L\n3809280,,,-', android_boards.AndroidMx6QSabreliteConfig().get_sfdisk_cmd()) class TestGetSfdiskCmdV2(TestCase): def test_mx5(self): board_conf = boards.Mx5Config() board_conf.partition_layout = 'reserved_bootfs_rootfs' self.assertEqual( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_sd(self): board_conf = boards.SnowballSdConfig() board_conf.partition_layout = 'bootfs_rootfs' self.assertEqual( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_snowball_emmc(self): board_conf = boards.SnowballEmmcConfig() board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.loader_start_s = (128 * 1024) / SECTOR_SIZE self.assertEqual( '256,7936,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_smdkv310(self): board_conf = get_board_config('smdkv310') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen(self): board_conf = get_board_config('origen') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_origen_quad(self): board_conf = get_board_config('origen_quad') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_arndale(self): board_conf = get_board_config('arndale') board_conf.partition_layout = 'reserved_bootfs_rootfs' board_conf.LOADER_MIN_SIZE_S = ( board_conf.samsung_bl1_start + board_conf.samsung_bl1_len + board_conf.samsung_bl2_len + board_conf.samsung_env_len) self.assertEquals( '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_conf.get_sfdisk_cmd()) def test_highbank(self): board_conf = get_board_config('highbank') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x83,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_beaglebone(self): board_conf = get_board_config('beaglebone') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) def test_aa9(self): board_conf = get_board_config('aa9') board_conf.partition_layout = 'bootfs_rootfs' self.assertEquals( '63,106432,0x0C,*\n106496,,,-', board_conf.get_sfdisk_cmd()) class TestGetBootCmd(TestCase): def test_vexpress(self): board_conf = get_board_config('vexpress') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'console=ttyXXX root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_vexpress_a9(self): board_conf = get_board_config('vexpress-a9') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA0,38400n8 ' 'console=ttyXXX root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x60000000 uImage; ' 'fatload mmc 0:1 0x62000000 uInitrd; ' 'bootm 0x60000000 0x62000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_mx51(self): board_conf = boards.Mx51Config() boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="mx51.dtb") expected = { 'bootargs': 'console=tty0 console=ttymxc0,115200n8 ' 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x90000000 uImage; ' 'fatload mmc 0:2 0x92000000 uInitrd; ' 'fatload mmc 0:2 0x91ff0000 board.dtb; ' 'bootm 0x90000000 0x92000000 0x91ff0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_smdkv310(self): board_conf = get_board_config('smdkv310') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC1,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'ethact': 'smc911x-0', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_origen(self): board_conf = get_board_config('origen') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC2,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_origen_quad(self): board_conf = get_board_config('origen_quad') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=ttySAC2,115200n8 root=UUID=deadbeef ' 'rootwait ro', 'bootcmd': 'fatload mmc 0:2 0x40007000 uImage; ' 'fatload mmc 0:2 0x42000000 uInitrd; ' 'bootm 0x40007000 0x42000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_arndale(self): board_conf = get_board_config('arndale') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:2 None uImage; ' 'fatload mmc 0:2 None uInitrd; ' 'bootm None None', 'ethact': 'smc911x-0', 'ethaddr': '00:40:5c:26:0a:5b', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_ux500(self): board_conf = get_board_config('ux500') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'rootdelay=1 fixrtc nocompcache mem=96M@0 ' 'mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M', 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' 'fatload mmc 1:1 0x08000000 uInitrd; ' 'bootm 0x00100000 0x08000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_snowball_emmc(self): board_conf = get_board_config('snowball_emmc') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data=None) expected = { 'bootargs': 'console=tty0 console=ttyAMA2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'rootdelay=1 fixrtc nocompcache mem=96M@0 ' 'mem_modem=32M@96M mem=44M@128M pmem=22M@172M ' 'mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M ' 'hwmem=48M@302M mem=152M@360M', 'bootcmd': 'fatload mmc 0:2 0x00100000 uImage; ' 'fatload mmc 0:2 0x08000000 uInitrd; ' 'bootm 0x00100000 0x08000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_panda(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('panda') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="panda.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=48M omapfb.vram=0:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_beagle(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('beagle') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="beagle.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_igep(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = boards.IgepConfig() config.serial_tty = config._serial_tty boot_cmd = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="igep.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk fixrtc ' 'nocompcache vram=12M ' 'omapfb.mode=dvi:1280x720MR-16@60 mpurate=${mpurate}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_cmd) def test_overo(self): # XXX: To fix bug 697824 we have to change class attributes of our # OMAP board configs, and some tests do that so to make sure they # don't interfere with us we'll reset that before doing anything. config = get_board_config('overo') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="overo.dtb") expected = { 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' 'root=UUID=deadbeef rootwait ro earlyprintk ' 'mpurate=${mpurate} vram=12M ' 'omapdss.def_disp=${defaultdisplay} ' 'omapfb.mode=dvi:${dvimode}', 'bootcmd': 'fatload mmc 0:1 0x80000000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80000000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_highbank(self): board_conf = get_board_config('highbank') boot_commands = board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'ext2load scsi 0:1 0x00800000 uImage; ' 'ext2load scsi 0:1 0x01000000 uInitrd; ' 'ext2load scsi 0:1 0x00001000 board.dtb; ' 'bootm 0x00800000 0x01000000 0x00001000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_beaglebone(self): config = get_board_config('beaglebone') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'console=ttyO0,115200n8 ' 'root=UUID=deadbeef rootwait ro fixrtc', 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' 'fatload mmc 0:1 0x81600000 uInitrd; ' 'fatload mmc 0:1 0x815f0000 board.dtb; ' 'bootm 0x80200000 0x81600000 0x815f0000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) def test_aa9(self): config = get_board_config('aa9') config.serial_tty = config._serial_tty boot_commands = config._get_boot_env( is_live=False, is_lowmem=False, consoles=[], rootfs_id="UUID=deadbeef", i_img_data="initrd", d_img_data="board.dtb") expected = { 'bootargs': 'console=ttyS0,115200n8 ' 'root=UUID=deadbeef rootwait ro', 'bootcmd': 'fatload mmc 0:1 0x40000000 uImage; ' 'fatload mmc 0:1 0x41100000 uInitrd; ' 'fatload mmc 0:1 0x41000000 board.dtb; ' 'bootm 0x40000000 0x41100000 0x41000000', 'fdt_high': '0xffffffff', 'initrd_high': '0xffffffff'} self.assertEqual(expected, boot_commands) class TestExtraBootCmd(TestCaseWithFixtures): def setUp(self): super(TestExtraBootCmd, self).setUp() self.board_conf = BoardConfig() def test_extra_boot_args_options_is_picked_by_get_boot_env(self): boot_args = 'whatever' self.board_conf.extra_boot_args_options = boot_args boot_commands = self.board_conf._get_boot_env( is_live=False, is_lowmem=False, consoles=['ttyXXX'], rootfs_id="UUID=deadbeef", i_img_data=None, d_img_data=None) expected = ( 'console=ttyXXX root=UUID=deadbeef rootwait ro %s' % boot_args) self.assertEqual(expected, boot_commands['bootargs']) def test_passing_None_to_add_boot_args(self): boot_args = 'extra-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args(None) self.assertEqual(boot_args, self.board_conf.extra_boot_args_options) def test_passing_string_to_add_boot_args(self): boot_args = 'extra-args' extra_args = 'user-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args(extra_args) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) def test_passing_string_to_add_boot_args_with_no_default_extra_args(self): extra_args = 'user-args' self.board_conf.add_boot_args(extra_args) self.assertEqual(extra_args, self.board_conf.extra_boot_args_options) def test_add_boot_args_from_file(self): boot_args = 'extra-args' extra_args = 'user-args' boot_arg_path = self.createTempFileAsFixture() with open(boot_arg_path, 'w') as boot_arg_file: boot_arg_file.write(extra_args) self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(boot_arg_path) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) def test_passing_None_to_add_boot_args_from_file(self): boot_args = 'extra-args' self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(None) self.assertEqual(boot_args, self.board_conf.extra_boot_args_options) def test_add_boot_args_from_file_strips_whitespace_from_file(self): boot_args = 'extra-args' extra_args = 'user-args' boot_arg_path = self.createTempFileAsFixture() with open(boot_arg_path, 'w') as boot_arg_file: boot_arg_file.write('\n\n \t ' + extra_args + ' \n\n') self.board_conf.extra_boot_args_options = boot_args self.board_conf.add_boot_args_from_file(boot_arg_path) self.assertEqual( "%s %s" % (boot_args, extra_args), self.board_conf.extra_boot_args_options) class TestUnpackBinaryTarball(TestCaseWithFixtures): def setUp(self): super(TestUnpackBinaryTarball, self).setUp() self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) def test_unpack_binary_tarball(self): tmp_dir = self.useFixture(CreateTempDirFixture()).get_temp_dir() rc = unpack_binary_tarball( self.tarball_fixture.get_tarball(), tmp_dir, as_root=False) self.assertEqual(rc, 0) class TestGetUuid(TestCaseWithFixtures): def setUp(self): super(TestGetUuid, self).setUp() def test_get_uuid(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) get_uuid("/dev/rootfs") self.assertEquals( ["%s blkid -o udev -p -c /dev/null /dev/rootfs" % sudo_args], fixture.mock.commands_executed) def test_parse_blkid_output(self): output = ( "ID_FS_UUID=67d641db-ea7d-4acf-9f46-5f1f8275dce2\n" "ID_FS_UUID_ENC=67d641db-ea7d-4acf-9f46-5f1f8275dce2\n" "ID_FS_TYPE=ext4\n") uuid = _parse_blkid_output(output) self.assertEquals("67d641db-ea7d-4acf-9f46-5f1f8275dce2", uuid) class TestBoards(TestCaseWithFixtures): def _mock_get_file_matching(self): self.useFixture(MockSomethingFixture( boards, '_get_file_matching', lambda regex: regex)) def _mock_Popen(self): fixture = MockCmdRunnerPopenFixture() self.useFixture(fixture) return fixture def test_make_uImage(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_uImage('load_addr', 'parts_dir/vmlinuz-*-sub_arch', 'boot_disk') expected = [ '%s mkimage -A arm -O linux -T kernel -C none -a load_addr ' '-e load_addr -n Linux -d parts_dir/vmlinuz-*-sub_arch ' 'boot_disk/uImage' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_uInitrd(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_uInitrd('parts_dir/initrd.img-*-sub_arch', 'boot_disk') expected = [ '%s mkimage -A arm -O linux -T ramdisk -C none -a 0 -e 0 ' '-n initramfs -d parts_dir/initrd.img-*-sub_arch ' 'boot_disk/uInitrd' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_dtb(self): self._mock_get_file_matching() fixture = self._mock_Popen() make_dtb('parts_dir/dt-*-sub_arch/board_name.dtb', 'boot_disk') expected = [ '%s cp parts_dir/dt-*-sub_arch/board_name.dtb ' 'boot_disk/board.dtb' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_make_flashable_env_too_small_env(self): env = {'verylong': 'evenlonger'} self.assertRaises(AssertionError, make_flashable_env, env, 8) def test_make_flashable_env(self): env_file = self.createTempFileAsFixture() self.useFixture(MockSomethingFixture( tempfile, "mkstemp", lambda: (None, env_file))) env = {'a': 'b', 'x': 'y'} make_flashable_env(env, 12) with open(env_file, "r") as fd: self.assertEqual("\x80\x29\x2E\x89a=b\x00x=y\x00", fd.read()) def test_install_mx5_boot_loader(self): fixture = self._mock_Popen() imx_file = self.createTempFileAsFixture() install_mx5_boot_loader(imx_file, "boot_device_or_file", BoardConfig.LOADER_MIN_SIZE_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 ' 'conv=notrunc seek=2' % (sudo_args, imx_file)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_mx5_boot_loader_too_large(self): self.useFixture(MockSomethingFixture( os.path, "getsize", lambda s: (BoardConfig.LOADER_MIN_SIZE_S - 1) * SECTOR_SIZE + 1)) self.assertRaises( AssertionError, install_mx5_boot_loader, "imx_file", "boot_device_or_file", BoardConfig.LOADER_MIN_SIZE_S) def test_install_omap_boot_loader(self): fixture = self._mock_Popen() self.useFixture(MockSomethingFixture( boards, '_get_mlo_file', lambda chroot_dir: "%s/MLO" % chroot_dir)) board_conf = BoardConfig() board_conf.set_metadata([]) install_omap_boot_loader("chroot_dir", "boot_disk", board_conf) expected = [ '%s cp -v chroot_dir/MLO boot_disk' % sudo_args] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_smdk_u_boot(self): fixture = self._mock_Popen() board_conf = boards.SMDKV310Config() bootloader_flavor = board_conf.bootloader_flavor self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( "%s/%s/SPL" % ("chroot_dir", bootloader_flavor), "%s/%s/uboot" % ("chroot_dir", bootloader_flavor), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def _set_up_board_config(self, board_name): """Internal method to set-up correctly board config, with the appropriate mock objects. :param board_name: The name of the board to set-up. :return A 3-tuple: the config, the name of the bootloader, and the value of chroot_dir. """ board_conf = get_board_config(board_name) bootloader_flavor = board_conf.bootloader_flavor # Made-up value to be used as the chroot directory. chroot_dir_value = 'chroot_dir' board_conf._get_samsung_spl = MagicMock() board_conf._get_samsung_spl.return_value = ("%s/%s/SPL" % (chroot_dir_value, bootloader_flavor)) board_conf._get_samsung_bootloader = MagicMock() board_conf._get_samsung_bootloader.return_value = ("%s/%s/uboot" % (chroot_dir_value, bootloader_flavor)) board_conf.hardwarepack_handler = ( TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) board_conf.hardwarepack_handler.get_format = ( lambda: HardwarepackHandler.FORMAT_1) return board_conf, bootloader_flavor, chroot_dir_value def test_install_origen_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('origen') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_origen_quad_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('origen_quad') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_install_arndale_u_boot(self): fixture = self._mock_Popen() board_conf, bootloader_flavor, chroot_dir_value = \ self._set_up_board_config('arndale') self.useFixture(MockSomethingFixture(os.path, 'getsize', lambda file: 1)) board_conf.install_samsung_boot_loader( board_conf._get_samsung_spl(chroot_dir_value), board_conf._get_samsung_bootloader(chroot_dir_value), "boot_disk") expected = [ '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl1_start), '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' 'seek=%d' % (sudo_args, bootloader_flavor, board_conf.samsung_bl2_start)] self.assertEqual(expected, fixture.mock.commands_executed) def test_get_plain_boot_script_contents(self): boot_env = {'bootargs': 'mybootargs', 'bootcmd': 'mybootcmd', 'initrd_high': '0xffffffff', 'fdt_high': '0xffffffff'} boot_script_data = get_plain_boot_script_contents(boot_env) self.assertEqual(textwrap.dedent("""\ setenv initrd_high "0xffffffff" setenv fdt_high "0xffffffff" setenv bootcmd "mybootcmd" setenv bootargs "mybootargs" boot"""), boot_script_data) def test_make_boot_script(self): self.useFixture(MockSomethingFixture( tempfile, 'mkstemp', lambda: (-1, '/tmp/random-abxzr'))) tempdir = self.useFixture(CreateTempDirFixture()).tempdir self._mock_get_file_matching() fixture = self._mock_Popen() boot_script_path = os.path.join(tempdir, 'boot.scr') plain_boot_script_path = os.path.join(tempdir, 'boot.txt') boot_env = {'bootargs': 'mybootargs', 'bootcmd': 'mybootcmd', 'initrd_high': '0xffffffff', 'fdt_high': '0xffffffff'} make_boot_script(boot_env, boot_script_path) expected = [ '%s cp /tmp/random-abxzr %s' % ( sudo_args, plain_boot_script_path), '%s mkimage -A arm -O linux -T script -C none -a 0 -e 0 ' '-n boot script -d %s %s' % ( sudo_args, plain_boot_script_path, boot_script_path)] self.assertEqual(expected, fixture.mock.commands_executed) def test_get_file_matching(self): prefix = ''.join( random.choice(string.ascii_lowercase) for x in range(5)) file1 = self.createTempFileAsFixture(prefix) directory = os.path.dirname(file1) self.assertEqual( file1, _get_file_matching('%s/%s*' % (directory, prefix))) def test_get_file_matching_too_many_files_found(self): prefix = ''.join( random.choice(string.ascii_lowercase) for x in range(5)) file1 = self.createTempFileAsFixture(prefix) file2 = self.createTempFileAsFixture(prefix) directory = os.path.dirname(file1) assert directory == os.path.dirname(file2), ( "file1 and file2 should be in the same directory") self.assertRaises( ValueError, _get_file_matching, '%s/%s*' % (directory, prefix)) def test_get_kflavor_files_more_specific(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavorx = 'flavorX' flavorxy = 'flavorXY' board_conf = BoardConfig() board_conf.kernel_flavors = [flavorx, flavorxy] for f in reversed(board_conf.kernel_flavors): kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % f) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % f) open(kfile, "w").close() open(ifile, "w").close() self.assertEqual( (kfile, ifile, None), board_conf._get_kflavor_files(tempdir)) def test_get_dt_kflavor_files_more_specific(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavorx = 'flavorX' flavorxy = 'flavorXY' board_conf = BoardConfig() board_conf.kernel_flavors = [flavorx, flavorxy] board_conf.dtb_name = 'board_name.dtb' for f in reversed(board_conf.kernel_flavors): kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % f) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % f) dt = os.path.join(tempdir, 'dt-1-%s' % f) os.mkdir(dt) dfile = os.path.join(dt, board_conf.dtb_name) open(kfile, "w").close() open(ifile, "w").close() open(dfile, "w").close() self.assertEqual( (kfile, ifile, dfile), board_conf._get_kflavor_files(tempdir)) def test_get_kflavor_files_later_in_flavors(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % flavor1) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % flavor1) open(kfile, "w").close() open(ifile, "w").close() self.assertEqual( (kfile, ifile, None), board_conf._get_kflavor_files(tempdir)) def test_get_dt_kflavor_files_later_in_flavors(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] board_conf.dtb_name = 'board_name.dtb' kfile = os.path.join(tempdir, 'vmlinuz-1-%s' % flavor1) ifile = os.path.join(tempdir, 'initrd.img-1-%s' % flavor1) dt = os.path.join(tempdir, 'dt-1-%s' % flavor1) os.mkdir(dt) dfile = os.path.join(dt, board_conf.dtb_name) open(kfile, "w").close() open(ifile, "w").close() open(dfile, "w").close() self.assertEqual( (kfile, ifile, dfile), board_conf._get_kflavor_files(tempdir)) def test_get_kflavor_files_raises_when_no_match(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir flavor1 = 'flavorXY' flavor2 = 'flavorAA' board_conf = BoardConfig() board_conf.kernel_flavors = [flavor1, flavor2] self.assertRaises(ValueError, board_conf._get_kflavor_files, tempdir) def test_get_file_matching_no_files_found(self): self.assertEqual( None, _get_file_matching('/foo/bar/baz/*non-existent')) def test_run_mkimage(self): # Create a fake boot script. filename = self.createTempFileAsFixture() f = open(filename, 'w') f.write("setenv bootcmd 'fatload mmc 0:1 0x80000000 uImage;\nboot") f.close() img = self.createTempFileAsFixture() # Use that fake boot script to create a boot loader using mkimage. # Send stdout to /dev/null as mkimage will print to stdout and we # don't want that. retval = _run_mkimage( 'script', '0', '0', 'boot script', filename, img, stdout=open('/dev/null', 'w'), as_root=False) self.assertEqual(0, retval) class TestCreatePartitions(TestCaseWithFixtures): media = Media('/dev/xdz') def setUp(self): super(TestCreatePartitions, self).setUp() # Stub time.sleep() as create_partitions() use that. self.orig_sleep = time.sleep time.sleep = lambda s: None def tearDown(self): super(TestCreatePartitions, self).tearDown() time.sleep = self.orig_sleep def test_create_partitions_for_mx5(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = boards.Mx5Config() board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_smdkv310(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('smdkv310') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_origen(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('origen') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_origen_quad(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('origen_quad') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions( board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_arndale(self): # For this board we create a one cylinder partition at the beginning. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('arndale') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions( board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) # Notice that we create all partitions in a single sfdisk run because # every time we run sfdisk it actually repartitions the device, # erasing any partitions created previously. self.assertEqual( [('1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_for_beagle(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, self.media) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path), '%s sfdisk -l %s' % (sudo_args, self.media.path)], popen_fixture.mock.commands_executed) self.assertEqual( [('63,106432,0x0C,*\n106496,,,-', self.media.path)], sfdisk_fixture.mock.calls) def test_create_partitions_with_img_file(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) sfdisk_fixture = self.useFixture(MockRunSfdiskCommandsFixture()) tmpfile = self.createTempFileAsFixture() board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 create_partitions(board_conf, Media(tmpfile)) # Unlike the test for partitioning of a regular block device, in this # case parted was not called as there's no existing partition table # for us to overwrite on the image file. self.assertEqual( ['%s sfdisk -l %s' % (sudo_args, tmpfile), '%s sfdisk -l %s' % (sudo_args, tmpfile)], popen_fixture.mock.commands_executed) self.assertEqual( [('63,106432,0x0C,*\n106496,,,-', tmpfile)], sfdisk_fixture.mock.calls) def test_run_sfdisk_commands(self): tmpfile = self.createTempFileAsFixture() proc = cmd_runner.run( ['dd', 'of=%s' % tmpfile, 'bs=1', 'seek=10M', 'count=0'], stderr=open('/dev/null', 'w')) proc.communicate() stdout, stderr = run_sfdisk_commands( '2,16063,0xDA', tmpfile, as_root=False, stderr=subprocess.PIPE) self.assertIn('Successfully wrote the new partition table', stdout) def test_run_sfdisk_commands_raises_on_non_zero_returncode(self): tmpfile = self.createTempFileAsFixture() self.assertRaises( cmd_runner.SubcommandNonZeroReturnValue, run_sfdisk_commands, ',1,0xDA', tmpfile, as_root=False, stderr=subprocess.PIPE) def test_wait_partitions_to_settle(self): class Namespace: pass ns = Namespace() ns.count = 0 class MockCmdRunnerPopen(object): def __call__(self, cmd, *args, **kwargs): ns.count += 1 self.returncode = 0 if ns.count < 5: raise cmd_runner.SubcommandNonZeroReturnValue(args, 1) else: return self def communicate(self, input=None): self.wait() return '', '' def wait(self): return self.returncode self.useFixture(MockCmdRunnerPopenFixture()) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual(0, wait_partition_to_settle(media, 'mbr')) def test_wait_partitions_to_settle_raises_SubcommandNonZeroReturnValue( self): def mock_run(args, as_root=False, chroot=None, stdin=None, stdout=None, stderr=None, cwd=None): raise cmd_runner.SubcommandNonZeroReturnValue(args, 1) self.useFixture(MockSomethingFixture( cmd_runner, 'run', mock_run)) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertRaises(cmd_runner.SubcommandNonZeroReturnValue, wait_partition_to_settle, media, 'mbr') class TestPartitionSetup(TestCaseWithFixtures): def setUp(self): super(TestPartitionSetup, self).setUp() # Stub time.sleep() as create_partitions() use that. self.orig_sleep = time.sleep time.sleep = lambda s: None self.linux_image_size = 30 * 1024 ** 2 self.linux_offsets_and_sizes = [ (16384 * SECTOR_SIZE, 15746 * SECTOR_SIZE), (32768 * SECTOR_SIZE, (self.linux_image_size - 32768 * SECTOR_SIZE)) ] self.android_image_size = 256 * 1024 ** 2 # Extended partition info takes 32 sectors from the first ext partition ext_part_size = 32 self.android_offsets_and_sizes = [ (63 * SECTOR_SIZE, 32768 * SECTOR_SIZE), (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), (98367 * SECTOR_SIZE, 65536 * SECTOR_SIZE), ((294944 + ext_part_size) * SECTOR_SIZE, (131103 - ext_part_size) * SECTOR_SIZE), ((426016 + ext_part_size) * SECTOR_SIZE, self.android_image_size - (426016 + ext_part_size) * SECTOR_SIZE) ] self.android_snowball_offsets_and_sizes = [ (8192 * SECTOR_SIZE, 24639 * SECTOR_SIZE), (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), ((98367 + ext_part_size) * SECTOR_SIZE, (65536 - ext_part_size) * SECTOR_SIZE), (294975 * SECTOR_SIZE, 131072 * SECTOR_SIZE), ((426047 + ext_part_size) * SECTOR_SIZE, self.android_image_size - (426047 + ext_part_size) * SECTOR_SIZE) ] def tearDown(self): super(TestPartitionSetup, self).tearDown() time.sleep = self.orig_sleep def _create_tmpfile(self): # boot part at +8 MiB, root part at +16 MiB return self._create_qemu_img_with_partitions( '16384,15746,0x0C,*\n32768,,,-', '%s' % self.linux_image_size) def _create_android_tmpfile(self): # boot, system, cache, (extended), userdata and sdcard partitions return self._create_qemu_img_with_partitions( '63,32768,0x0C,*\n32831,65536,L\n98367,65536,L\n294975,-,E\n' '294975,131072,L\n426047,,,-', '%s' % self.android_image_size) def _create_snowball_android_tmpfile(self): # raw, boot, system, cache, (extended), userdata and sdcard partitions return self._create_qemu_img_with_partitions( '256,7936,0xDA\n8192,24639,0x0C,*\n32831,65536,L\n' '98367,-,E\n98367,65536,L\n294975,131072,L\n' '426047,,,-', '%s' % self.android_image_size) def test_check_min_size_small(self): """Check that we get back the minimum size as expeceted.""" self.assertEqual(MIN_IMAGE_SIZE, _check_min_size(12345)) def test_check_min_size_big(self): """Check that we get back the same value we pass.""" self.assertEqual(MIN_IMAGE_SIZE * 3, _check_min_size(3145728)) def test_convert_size_wrong_suffix(self): self.assertRaises(ValueError, get_partition_size_in_bytes, "123456H") def test_convert_size_no_suffix(self): self.assertEqual(2 ** 20, get_partition_size_in_bytes('123456')) def test_convert_size_one_mbyte(self): self.assertEqual(2 ** 20, get_partition_size_in_bytes('1M')) def test_convert_size_in_kbytes_to_bytes(self): self.assertEqual(2 * 2 ** 20, get_partition_size_in_bytes('2048K')) def test_convert_size_in_mbytes_to_bytes(self): self.assertEqual(100 * 2 ** 20, get_partition_size_in_bytes('100M')) def test_convert_size_in_gbytes_to_bytes(self): self.assertEqual(12 * 2 ** 30, get_partition_size_in_bytes('12G')) def test_convert_size_float_no_suffix(self): self.assertEqual(3 * 2 ** 20, get_partition_size_in_bytes('2348576.91')) def test_convert_size_float_in_kbytes_to_bytes(self): self.assertEqual(3 * 2 ** 20, get_partition_size_in_bytes('2345.8K')) def test_convert_size_float_in_mbytes_to_bytes_double(self): self.assertEqual(2 * 2 ** 20, get_partition_size_in_bytes('1.0000001M')) def test_convert_size_float_in_mbytes_to_bytes(self): self.assertEqual(877 * 2 ** 20, get_partition_size_in_bytes('876.123M')) def test_convert_size_float_in_gbytes_to_bytes(self): self.assertEqual(1946 * 2 ** 20, get_partition_size_in_bytes('1.9G')) def test_calculate_partition_size_and_offset(self): tmpfile = self._create_tmpfile() vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(tmpfile)) self.assertEqual( self.linux_offsets_and_sizes, [(vfat_offset, vfat_size), (linux_offset, linux_size)]) def test_calculate_android_partition_size_and_offset(self): tmpfile = self._create_android_tmpfile() device_info = calculate_android_partition_size_and_offset(tmpfile) # We use map(None, ...) since it would catch if the lists are not of # equal length and zip() would not in all cases. for device_pair, expected_pair in map(None, device_info, self.android_offsets_and_sizes): self.assertEqual(device_pair, expected_pair) def test_calculate_snowball_android_partition_size_and_offset(self): tmpfile = self._create_snowball_android_tmpfile() device_info = calculate_android_partition_size_and_offset(tmpfile) # We use map(None, ...) since it would catch if the lists are not of # equal length and zip() would not in all cases. snowball_info = map(None, device_info, self.android_snowball_offsets_and_sizes) for device_pair, expected_pair in snowball_info: self.assertEqual(device_pair, expected_pair) def test_partition_numbering(self): # another Linux partition at +24 MiB after the boot/root parts tmpfile = self._create_qemu_img_with_partitions( '16384,15746,0x0C,*\n32768,15427,,-\n49152,,,-', '%s' % self.linux_image_size) vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(tmpfile)) # check that the linux partition offset starts at +16 MiB so that it's # the partition immediately following the vfat one self.assertEqual(linux_offset, 32768 * 512) def test_get_boot_and_root_partitions_for_media_beagle(self): self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual( ("%s%d" % (tmpfile, 1), "%s%d" % (tmpfile, 2)), get_boot_and_root_partitions_for_media( media, get_board_config('beagle'))) def test_get_boot_and_root_partitions_for_media_mx5(self): self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) tmpfile = self.createTempFileAsFixture() media = Media(tmpfile) media.is_block_device = True self.assertEqual( ("%s%d" % (tmpfile, 2), "%s%d" % (tmpfile, 3)), get_boot_and_root_partitions_for_media(media, boards.Mx5Config())) def _create_qemu_img_with_partitions(self, sfdisk_commands, tempfile_size): tmpfile = self.createTempFileAsFixture() proc = cmd_runner.run( ['dd', 'of=%s' % tmpfile, 'bs=1', 'seek=%s' % tempfile_size, 'count=0'], stderr=open('/dev/null', 'w')) proc.communicate() stdout, stderr = run_sfdisk_commands( sfdisk_commands, tmpfile, as_root=False, # Throw away stderr as sfdisk complains a lot when operating on a # qemu image. stderr=subprocess.PIPE) self.assertIn('Successfully wrote the new partition table', stdout) return tmpfile def test_ensure_partition_is_not_mounted_for_mounted_partition(self): self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: True)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) ensure_partition_is_not_mounted('/dev/whatever') self.assertEqual( ['%s umount /dev/whatever' % sudo_args], popen_fixture.mock.commands_executed) def test_ensure_partition_is_not_mounted_for_umounted_partition(self): self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: False)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) ensure_partition_is_not_mounted('/dev/whatever') self.assertEqual(None, popen_fixture.mock.calls) def test_get_boot_and_root_loopback_devices(self): tmpfile = self._create_tmpfile() atexit_fixture = self.useFixture(MockSomethingFixture( atexit, 'register', AtExitRegister())) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # We can't test the return value of get_boot_and_root_loopback_devices # because it'd require running losetup as root, so we just make sure # it calls losetup correctly. get_boot_and_root_loopback_devices(tmpfile) self.assertEqual( ['%s losetup -f --show %s --offset %s --sizelimit %s' % (sudo_args, tmpfile, offset, size) for (offset, size) in self.linux_offsets_and_sizes], popen_fixture.mock.commands_executed) # get_boot_and_root_loopback_devices will also setup two exit handlers # to de-register the loopback devices set up above. self.assertEqual(2, len(atexit_fixture.mock.funcs)) popen_fixture.mock.calls = [] atexit_fixture.mock.run_funcs() # We did not really run losetup above (as it requires root) so here we # don't have a device to pass to 'losetup -d', but when a device is # setup it is passed to the atexit handler. self.assertEquals( ['%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args], popen_fixture.mock.commands_executed) def test_get_android_loopback_devices(self): tmpfile = self._create_android_tmpfile() atexit_fixture = self.useFixture(MockSomethingFixture( atexit, 'register', AtExitRegister())) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # We can't test the return value of get_boot_and_root_loopback_devices # because it'd require running losetup as root, so we just make sure # it calls losetup correctly. get_android_loopback_devices(tmpfile) self.assertEqual( ['%s losetup -f --show %s --offset %s --sizelimit %s' % (sudo_args, tmpfile, offset, size) for (offset, size) in self.android_offsets_and_sizes], popen_fixture.mock.commands_executed) # get_boot_and_root_loopback_devices will also setup two exit handlers # to de-register the loopback devices set up above. self.assertEqual(5, len(atexit_fixture.mock.funcs)) popen_fixture.mock.calls = [] atexit_fixture.mock.run_funcs() # We did not really run losetup above (as it requires root) so here we # don't have a device to pass to 'losetup -d', but when a device is # setup it is passed to the atexit handler. self.assertEquals( ['%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args], popen_fixture.mock.commands_executed) def test_setup_partitions_for_image_file(self): # In practice we could pass an empty image file to setup_partitions, # but here we mock Popen() and thanks to that the image is not setup # (via dd) inside setup_partitions. That's why we pass an # already setup image file. tmpfile = self._create_tmpfile() popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) def ensure_partition_not_mounted(part): raise AssertionError( "ensure_partition_is_not_mounted must not be called when " "generating image files. It makes no sense to do that and " "it depends on UDisks, thus making it hard to run on a " "chroot") self.useFixture(MockSomethingFixture( partitions, 'ensure_partition_is_not_mounted', ensure_partition_not_mounted)) self.useFixture(MockSomethingFixture( partitions, 'get_boot_and_root_loopback_devices', lambda image: ('/dev/loop99', '/dev/loop98'))) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 bootfs_dev, rootfs_dev = setup_partitions( board_conf, Media(tmpfile), '2G', 'boot', 'root', 'ext3', True, True, True) self.assertEqual( # This is the call that would create a 2 GiB image file. ['dd of=%s bs=1 seek=2147483648 count=0' % tmpfile, '%s sfdisk -l %s' % (sudo_args, tmpfile), # This call would partition the image file. '%s sfdisk --force -uS %s' % (sudo_args, tmpfile), # Make sure changes are written to disk. '%s sfdisk -l %s' % (sudo_args, tmpfile), '%s mkfs.vfat -F 32 %s -n boot' % (sudo_args, bootfs_dev), '%s mkfs.ext3 -F %s -L root' % (sudo_args, rootfs_dev)], popen_fixture.mock.commands_executed) def test_setup_partitions_for_block_device(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) # Pretend the partitions are mounted. self.useFixture(MockSomethingFixture( partitions, 'is_partition_mounted', lambda part: True)) tmpfile = self._create_tmpfile() self.useFixture(MockSomethingFixture( partitions, '_get_device_file_for_partition_number', lambda dev, partition: '%s%d' % (tmpfile, partition))) media = Media(tmpfile) # Pretend our tmpfile is a block device. media.is_block_device = True popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) board_conf = get_board_config('beagle') board_conf.hwpack_format = HardwarepackHandler.FORMAT_1 bootfs_dev, rootfs_dev = setup_partitions( board_conf, media, '2G', 'boot', 'root', 'ext3', True, True, True) self.assertEqual( ['%s parted -s %s mklabel msdos' % (sudo_args, tmpfile), '%s sfdisk -l %s' % (sudo_args, tmpfile), '%s sfdisk --force -uS %s' % (sudo_args, tmpfile), '%s sfdisk -l %s' % (sudo_args, tmpfile), # Since the partitions are mounted, setup_partitions will umount # them before running mkfs. '%s umount %s' % (sudo_args, bootfs_dev), '%s umount %s' % (sudo_args, rootfs_dev), '%s mkfs.vfat -F 32 %s -n boot' % (sudo_args, bootfs_dev), '%s mkfs.ext3 -F %s -L root' % (sudo_args, rootfs_dev)], popen_fixture.mock.commands_executed) def test_get_device_file_for_partition_number_raises_DBusException(self): def mock_get_udisks_device_path(d): raise dbus.exceptions.DBusException self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_path', mock_get_udisks_device_path)) tmpfile = self.createTempFileAsFixture() partition = get_board_config('beagle').mmc_part_offset self.useFixture(MockSomethingFixture( glob, 'glob', lambda pathname: ['%s%d' % (tmpfile, partition)])) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) media = Media(tmpfile) media.is_block_device = True self.assertRaises(dbus.exceptions.DBusException, _get_device_file_for_partition_number, media.path, partition) def test_get_device_file_for_partition_number(self): class Namespace: pass ns = Namespace() ns.count = 0 def mock_get_udisks_device_path(dev): ns.count += 1 if ns.count < 5: raise dbus.exceptions.DBusException else: return '/abc/123' def mock_get_udisks_device_file(dev, part): if ns.count < 5: raise dbus.exceptions.DBusException else: return '/abc/123' self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_path', mock_get_udisks_device_path)) self.useFixture(MockSomethingFixture( partitions, '_get_udisks_device_file', mock_get_udisks_device_file)) tmpfile = self.createTempFileAsFixture() partition = get_board_config('beagle').mmc_part_offset self.useFixture(MockSomethingFixture( glob, 'glob', lambda pathname: ['%s%d' % (tmpfile, partition)])) self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) media = Media(tmpfile) media.is_block_device = True self.assertIsNotNone(_get_device_file_for_partition_number( media.path, partition)) class TestException(Exception): """Just a test exception.""" class TestMountedPartitionContextManager(TestCaseWithFixtures): def test_basic(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def test_func(): with partition_mounted('foo', 'bar', '-t', 'proc'): pass test_func() expected = ['%s mount foo bar -t proc' % sudo_args, '%s umount bar' % sudo_args] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_exception_raised_inside_with_block(self): popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def test_func(): with partition_mounted('foo', 'bar'): raise TestException('something') try: test_func() except TestException: pass expected = ['%s mount foo bar' % sudo_args, '%s umount bar' % sudo_args] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_umount_failure(self): # We ignore a SubcommandNonZeroReturnValue from umount because # otherwise it could shadow an exception raised inside the 'with' # block. popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) def failing_umount(path): raise cmd_runner.SubcommandNonZeroReturnValue('umount', 1) self.useFixture(MockSomethingFixture( partitions, 'umount', failing_umount)) def test_func(): with partition_mounted('foo', 'bar'): pass test_func() expected = ['sudo -E mount foo bar'] self.assertEqual(expected, popen_fixture.mock.commands_executed) class TestPopulateBoot(TestCaseWithFixtures): expected_args = ( 'chroot_dir/boot', False, False, [], 'chroot_dir', 'rootfs_id', 'boot_disk', 'boot_device_or_file') expected_args_live = ( 'chroot_dir/casper', True, False, [], 'chroot_dir', 'rootfs_id', 'boot_disk', 'boot_device_or_file') expected_calls = [ 'mkdir -p boot_disk', '%s mount boot_partition boot_disk' % sudo_args, '%s umount boot_disk' % sudo_args] def save_args(self, *args): self.saved_args = args def prepare_config(self, config): self.config = config self.config.boot_script = 'boot_script' self.config.hardwarepack_handler = \ TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz') self.config.hardwarepack_handler.get_format = \ lambda: HardwarepackHandler.FORMAT_1 self.popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( self.config, 'make_boot_files', self.save_args)) self.config.hardwarepack_handler.get_file_from_package = \ self.get_file_from_package self.config.bootloader_copy_files = None def get_file_from_package(self, source_path, source_package): if source_package in self.config.bootloader_copy_files: for file_info in self.config.bootloader_copy_files[source_package]: if source_path in file_info: return source_path return None def prepare_config_v3(self, config): self.config = config self.config.boot_script = 'boot_script' self.config.hardwarepack_handler = \ TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz') self.config.hardwarepack_handler.get_format = lambda: '3.0' self.config.hardwarepack_handler.get_file = \ lambda file_alias: ['file1', 'file2'] self.config.hardwarepack_handler.get_file_from_package = \ self.get_file_from_package self.config.bootloader_copy_files = { "package1": [{"file1": "/boot/"}, {"file2": "/boot/grub/renamed"}]} self.popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( self.config, 'make_boot_files', self.save_args)) def call_populate_boot(self, config, is_live=False): config.populate_boot( 'chroot_dir', 'rootfs_id', 'boot_partition', 'boot_disk', 'boot_device_or_file', is_live, False, []) def test_populate_boot_live(self): self.prepare_config(BoardConfig()) self.call_populate_boot(self.config, is_live=True) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args_live, self.saved_args) def test_populate_boot_regular(self): self.prepare_config(BoardConfig()) self.call_populate_boot(self.config) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_flavor(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.call_populate_boot(self.config) self.assertEquals( self.expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_file_in_boot_part(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.config.bootloader_file_in_boot_part = True self.config.bootloader = "u_boot" self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert( 2, '%s cp -v chroot_dir/usr/lib/u-boot/bootloader_flavor/u-boot.bin ' 'boot_disk' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_bootloader_file_in_boot_part_false(self): self.prepare_config(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" self.config.bootloader_file_in_boot_part = False self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_bootloader_copy_files(self): self.prepare_config_v3(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" # Test that copy_files works per spec (puts stuff in boot partition) # even if bootloader not in_boot_part. self.config.bootloader_file_in_boot_part = False self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert(2, '%s mkdir -p boot_disk/boot' % sudo_args) expected_calls.insert(3, '%s cp -v file1 ' 'boot_disk/boot/' % sudo_args) expected_calls.insert(4, '%s mkdir -p boot_disk/boot/grub' % sudo_args) expected_calls.insert(5, '%s cp -v file2 ' 'boot_disk/boot/grub/renamed' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_bootloader_copy_files_bootloader_set(self): self.prepare_config_v3(BoardConfig()) self.config.bootloader_flavor = "bootloader_flavor" # Test that copy_files works per spec (puts stuff in boot partition) # even if bootloader not in_boot_part. self.config.bootloader_file_in_boot_part = False self.config.bootloader = "u_boot" self.call_populate_boot(self.config) expected_calls = self.expected_calls[:] expected_calls.insert(2, '%s mkdir -p boot_disk/boot' % sudo_args) expected_calls.insert(3, '%s cp -v file1 ' 'boot_disk/boot/' % sudo_args) expected_calls.insert(4, '%s mkdir -p boot_disk/boot/grub' % sudo_args) expected_calls.insert(5, '%s cp -v file2 ' 'boot_disk/boot/grub/renamed' % sudo_args) self.assertEquals( expected_calls, self.popen_fixture.mock.commands_executed) self.assertEquals(self.expected_args, self.saved_args) def test_populate_boot_no_bootloader_flavor(self): self.prepare_config(BoardConfig()) self.config.bootloader_file_in_boot_part = True self.assertRaises( AssertionError, self.call_populate_boot, self.config) class TestPopulateRootFS(TestCaseWithFixtures): lines_added_to_fstab = None create_flash_kernel_config_called = False def test_populate_rootfs(self): def fake_append_to_fstab(disk, additions): self.lines_added_to_fstab = additions def fake_create_flash_kernel_config(disk, mmc_device_id, partition_offset): self.create_flash_kernel_config_called = True # Mock stdout, cmd_runner.Popen(), append_to_fstab and # create_flash_kernel_config. self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) self.useFixture(MockSomethingFixture( rootfs, 'append_to_fstab', fake_append_to_fstab)) self.useFixture(MockSomethingFixture( rootfs, 'create_flash_kernel_config', fake_create_flash_kernel_config)) popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) # Store a dummy rootdisk and contents_dir in a tempdir. tempdir = self.useFixture(CreateTempDirFixture()).tempdir root_disk = os.path.join(tempdir, 'rootdisk') contents_dir = os.path.join(tempdir, 'contents') contents_bin = os.path.join(contents_dir, 'bin') contents_etc = os.path.join(contents_dir, 'etc') os.makedirs(contents_bin) os.makedirs(contents_etc) # Must mock rootfs._list_files() because populate_rootfs() uses its # return value but since we mock cmd_runner.run() _list_files() would # return an invalid value. def mock_list_files(directory): return [contents_bin, contents_etc] self.useFixture(MockSomethingFixture( rootfs, '_list_files', mock_list_files)) populate_rootfs( contents_dir, root_disk, partition='/dev/rootfs', rootfs_type='ext3', rootfs_id='UUID=uuid', should_create_swap=True, swap_size=100, mmc_device_id=0, partition_offset=0, os_release_id='ubuntu', board_config=None) self.assertEqual( ['UUID=uuid / ext3 errors=remount-ro 0 1', '/SWAP.swap none swap sw 0 0'], self.lines_added_to_fstab) self.assertEqual(True, self.create_flash_kernel_config_called) swap_file = os.path.join(root_disk, 'SWAP.swap') expected = [ '%s mount /dev/rootfs %s' % (sudo_args, root_disk), '%s mv %s %s %s' % ( sudo_args, contents_bin, contents_etc, root_disk), '%s dd if=/dev/zero of=%s bs=1M count=100' % ( sudo_args, swap_file), '%s mkswap %s' % (sudo_args, swap_file), '%s umount %s' % (sudo_args, root_disk)] self.assertEqual(expected, popen_fixture.mock.commands_executed) def test_create_flash_kernel_config(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) tempdir = self.useFixture(CreateTempDirFixture()).tempdir create_flash_kernel_config(tempdir, mmc_device_id=0, boot_partition_number=1) calls = fixture.mock.calls self.assertEqual(1, len(calls), calls) call = calls[0] # The call writes to a tmpfile and then moves it to the, # /etc/flash-kernel.conf, so the tmpfile is the next to last in the # list of arguments stored. tmpfile = call[-2] self.assertEqual( '%s mv -f %s %s/etc/flash-kernel.conf' % ( sudo_args, tmpfile, tempdir), fixture.mock.commands_executed[0]) self.assertEqual('UBOOT_PART=/dev/mmcblk0p1', open(tmpfile).read().rstrip()) def test_list_files(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir # We don't want to mock cmd_runner.run() because we're testing the # command that it runs, but we need to monkey-patch SUDO_ARGS because # we don't want to use 'sudo' in tests. orig_sudo_args = cmd_runner.SUDO_ARGS def restore_sudo_args(): cmd_runner.SUDO_ARGS = orig_sudo_args self.addCleanup(restore_sudo_args) cmd_runner.SUDO_ARGS = [] file1 = self.createTempFileAsFixture(dir=tempdir) self.assertEqual([file1], rootfs._list_files(tempdir)) def test_move_contents(self): tempdir = self.useFixture(CreateTempDirFixture()).tempdir popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) file1 = self.createTempFileAsFixture(dir=tempdir) def mock_list_files(directory): return [file1] self.useFixture(MockSomethingFixture( rootfs, '_list_files', mock_list_files)) move_contents(tempdir, '/tmp/') self.assertEqual(['%s mv %s /tmp/' % (sudo_args, file1)], popen_fixture.mock.commands_executed) def test_has_space_left_for_swap(self): statvfs = os.statvfs('/') space_left = statvfs.f_bavail * statvfs.f_bsize swap_size_in_megs = space_left / (1024 ** 2) self.assertTrue( has_space_left_for_swap('/', swap_size_in_megs)) def test_has_no_space_left_for_swap(self): statvfs = os.statvfs('/') space_left = statvfs.f_bavail * statvfs.f_bsize swap_size_in_megs = (space_left / (1024 ** 2)) + 1 self.assertFalse( has_space_left_for_swap('/', swap_size_in_megs)) def mock_write_data_to_protected_file(self, path, data): # Duplicate of write_data_to_protected_file() but does not sudo. _, tmpfile = tempfile.mkstemp() with open(tmpfile, 'w') as fd: fd.write(data) cmd_runner.run(['mv', '-f', tmpfile, path], as_root=False).wait() def test_update_interfaces_no_interfaces_no_update(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') update_network_interfaces(tempdir, BoardConfig()) self.assertFalse(os.path.exists(if_path)) def test_update_interfaces_creates_entry(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0'] expected = 'auto eth0\n' \ 'iface eth0 inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertEqual(expected, open(if_path).read()) def test_update_interfaces_creates_entries(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0', 'eth1'] board_conf.wireless_interfaces = ['wlan0'] expected = 'auto %(if)s\n' \ 'iface %(if)s inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertIn(expected % {'if': 'eth1'}, open(if_path).read()) self.assertIn(expected % {'if': 'eth0'}, open(if_path).read()) self.assertIn(expected % {'if': 'wlan0'}, open(if_path).read()) def test_update_interfaces_leaves_original(self): self.useFixture(MockSomethingFixture( rootfs, 'write_data_to_protected_file', self.mock_write_data_to_protected_file)) tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() os.makedirs(os.path.join(tempdir, 'etc', 'network')) if_path = os.path.join(tempdir, 'etc', 'network', 'interfaces') with open(if_path, 'w') as interfaces: interfaces.write('Original contents of file.\n') board_conf = BoardConfig() board_conf.wired_interfaces = ['eth0'] expected = 'Original contents of file.\n' \ 'auto eth0\n' \ 'iface eth0 inet dhcp\n' update_network_interfaces(tempdir, board_conf) self.assertEqual(expected, open(if_path).read()) def test_write_data_to_protected_file(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) data = 'foo' path = '/etc/nonexistant' write_data_to_protected_file(path, data) calls = fixture.mock.calls self.assertEqual(1, len(calls), calls) call = calls[0] # The call moves tmpfile to the given path, so tmpfile is the next to # last in the list of arguments stored. tmpfile = call[-2] self.assertEqual(['%s mv -f %s %s' % (sudo_args, tmpfile, path)], fixture.mock.commands_executed) self.assertEqual(data, open(tmpfile).read()) def test_rootfs_mount_options_for_btrfs(self): self.assertEqual("defaults", rootfs_mount_options('btrfs')) def test_rootfs_mount_options_for_ext4(self): self.assertEqual("errors=remount-ro", rootfs_mount_options('ext4')) def test_rootfs_mount_options_for_unknown(self): self.assertRaises(ValueError, rootfs_mount_options, 'unknown') def test_append_to_fstab(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() # we don't really need root (sudo) as we're not writing to a real # root owned /etc self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 0)) etc = os.path.join(tempdir, 'etc') os.mkdir(etc) fstab = os.path.join(etc, 'fstab') open(fstab, "w").close() append_to_fstab(tempdir, ['foo', 'bar']) f = open(fstab) contents = f.read() f.close() self.assertEquals("\nfoo\nbar\n", contents) class TestCheckDevice(TestCaseWithFixtures): def _mock_does_device_exist_true(self): self.useFixture(MockSomethingFixture( check_device, '_does_device_exist', lambda device: True)) def _mock_does_device_exist_false(self): self.useFixture(MockSomethingFixture( check_device, '_does_device_exist', lambda device: False)) def _mock_print_devices(self): self.useFixture(MockSomethingFixture( check_device, '_print_devices', lambda: None)) def _mock_select_device(self): self.useFixture(MockSomethingFixture( check_device, '_select_device', lambda device: True)) def _mock_deselect_device(self): self.useFixture(MockSomethingFixture( check_device, '_select_device', lambda device: False)) def _mock_sys_stdout(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open(os.devnull, 'w'))) def setUp(self): super(TestCheckDevice, self).setUp() self._mock_sys_stdout() self._mock_print_devices() def test_ensure_device_partitions_not_mounted(self): partitions_umounted = [] def ensure_partition_is_not_mounted_mock(part): partitions_umounted.append(part) self.useFixture(MockSomethingFixture( partitions, 'ensure_partition_is_not_mounted', ensure_partition_is_not_mounted_mock)) self.useFixture(MockSomethingFixture( glob, 'glob', lambda pattern: ['/dev/sdz1', '/dev/sdz2'])) check_device._ensure_device_partitions_not_mounted('/dev/sdz') self.assertEquals(['/dev/sdz1', '/dev/sdz2'], partitions_umounted) def test_check_device_and_select(self): self._mock_does_device_exist_true() self._mock_select_device() self.assertTrue( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) def test_check_device_and_deselect(self): self._mock_does_device_exist_true() self._mock_deselect_device() self.assertFalse( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) def test_check_device_not_found(self): self._mock_does_device_exist_false() self.assertFalse( check_device.confirm_device_selection_and_ensure_it_is_ready( None)) class AtExitRegister(object): funcs = None def __call__(self, func, *args, **kwargs): if self.funcs is None: self.funcs = [] self.funcs.append((func, args, kwargs)) def run_funcs(self): for func, args, kwargs in self.funcs: func(*args, **kwargs) class TestInstallHWPack(TestCaseWithFixtures): def create_minimal_v3_hwpack(self, location, name, version, architecture): metadata = "\n".join([ "name: " + name, "version: " + version, "architecture: " + architecture, "format: 3.0" ]) print metadata tar_file = tarfile.open(location, mode='w:gz') tarinfo = tarfile.TarInfo("metadata") tarinfo.size = len(metadata) tar_file.addfile(tarinfo, StringIO(metadata)) tar_file.close() def mock_prepare_chroot(self, chroot_dir, tmp_dir): def fake_prepare_chroot(chroot_dir, tmp_dir): cmd_runner.run(['prepare_chroot %s %s' % (chroot_dir, tmp_dir)], as_root=True).wait() self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'prepare_chroot', fake_prepare_chroot)) def test_temporarily_overwrite_file_on_dir(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) temporarily_overwrite_file_on_dir('/path/to/file', '/dir', '/tmp/dir') self.assertEquals( ['%s cp -a /path/to/file /dir' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f /dir/file' % sudo_args], fixture.mock.commands_executed) def test_copy_file(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) copy_file('/path/to/file', '/dir') self.assertEquals( ['%s cp /path/to/file /dir' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f /dir/file' % sudo_args], fixture.mock.commands_executed) def test_mount_chroot_proc(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) mount_chroot_proc('chroot') self.assertEquals( ['%s mount proc chroot/proc -t proc' % sudo_args], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s umount -v chroot/proc' % sudo_args], fixture.mock.commands_executed) def test_install_hwpack(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' hwpack_dir = tempfile.mkdtemp() hwpack_file_name = 'hwpack.tgz' hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_name = "foo" hwpack_version = "4" hwpack_architecture = "armel" extract_kpkgs = False self.create_minimal_v3_hwpack(hwpack_tgz_location, hwpack_name, hwpack_version, hwpack_architecture) force_yes = False install_hwpack(chroot_dir, hwpack_tgz_location, extract_kpkgs, force_yes, 'linaro-hwpack-install') self.assertEquals( ['%s cp %s %s' % (sudo_args, hwpack_tgz_location, chroot_dir), '%s %s %s linaro-hwpack-install --hwpack-version %s ' '--hwpack-arch %s --hwpack-name %s /%s' % (sudo_args, chroot_args, chroot_dir, hwpack_version, hwpack_architecture, hwpack_name, hwpack_file_name)], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f %s/hwpack.tgz' % (sudo_args, chroot_dir)], fixture.mock.commands_executed) def test_install_hwpack_extract(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' hwpack_dir = tempfile.mkdtemp() hwpack_file_name = 'hwpack.tgz' hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_name = "foo" hwpack_version = "4" hwpack_architecture = "armel" extract_kpkgs = True self.create_minimal_v3_hwpack(hwpack_tgz_location, hwpack_name, hwpack_version, hwpack_architecture) force_yes = False install_hwpack(chroot_dir, hwpack_tgz_location, extract_kpkgs, force_yes, 'linaro-hwpack-install') self.assertEquals( ['%s cp %s %s' % (sudo_args, hwpack_tgz_location, chroot_dir), '%s linaro-hwpack-install --hwpack-version %s ' '--hwpack-arch %s --hwpack-name %s --extract-kernel-only %s/%s' % (sudo_args, hwpack_version, hwpack_architecture, hwpack_name, chroot_dir, hwpack_file_name)], fixture.mock.commands_executed) fixture.mock.calls = [] run_local_atexit_funcs() self.assertEquals( ['%s rm -f %s/hwpack.tgz' % (sudo_args, chroot_dir)], fixture.mock.commands_executed) def test_install_hwpacks(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' tmp_dir = 'tmp_dir' self.mock_prepare_chroot(chroot_dir, tmp_dir) force_yes = True prefer_dir = preferred_tools_dir() hwpack_dir = tempfile.mkdtemp() hwpack_file_names = ['hwpack1.tgz', 'hwpack2.tgz'] hwpack_tgz_locations = [] hwpack_names = [] extract_kpkgs = False for hwpack_file_name in hwpack_file_names: hwpack_tgz_location = os.path.join(hwpack_dir, hwpack_file_name) hwpack_tgz_locations.append(hwpack_tgz_location) hwpack_names.append(hwpack_file_name) hwpack_version = "4" hwpack_architecture = "armel" self.create_minimal_v3_hwpack( hwpack_tgz_location, hwpack_file_name, hwpack_version, hwpack_architecture) install_hwpacks( chroot_dir, tmp_dir, prefer_dir, force_yes, [], extract_kpkgs, hwpack_tgz_locations[0], hwpack_tgz_locations[1]) linaro_hwpack_install = find_command( 'linaro-hwpack-install', prefer_dir=prefer_dir) expected = [ 'prepare_chroot %(chroot_dir)s %(tmp_dir)s', 'cp %(linaro_hwpack_install)s %(chroot_dir)s/usr/bin', 'mount proc %(chroot_dir)s/proc -t proc', 'chroot %(chroot_dir)s true', 'cp %(hwpack1)s %(chroot_dir)s', ('%(chroot_args)s %(chroot_dir)s linaro-hwpack-install ' '--hwpack-version %(hp_version)s ' '--hwpack-arch %(hp_arch)s --hwpack-name %(hp_name1)s' ' --force-yes /hwpack1.tgz'), 'cp %(hwpack2)s %(chroot_dir)s', ('%(chroot_args)s %(chroot_dir)s linaro-hwpack-install ' '--hwpack-version %(hp_version)s ' '--hwpack-arch %(hp_arch)s --hwpack-name %(hp_name2)s' ' --force-yes /hwpack2.tgz'), 'rm -f %(chroot_dir)s/hwpack2.tgz', 'rm -f %(chroot_dir)s/hwpack1.tgz', 'umount -v %(chroot_dir)s/proc', 'rm -f %(chroot_dir)s/usr/bin/linaro-hwpack-install'] keywords = dict( chroot_dir=chroot_dir, tmp_dir=tmp_dir, chroot_args=chroot_args, linaro_hwpack_install=linaro_hwpack_install, hwpack1=hwpack_tgz_locations[0], hwpack2=hwpack_tgz_locations[1], hp_version=hwpack_version, hp_name1=hwpack_names[0], hp_name2=hwpack_names[1], hp_arch=hwpack_architecture) expected = [ "%s %s" % (sudo_args, line % keywords) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_install_packages(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) chroot_dir = 'chroot_dir' tmp_dir = 'tmp_dir' self.mock_prepare_chroot(chroot_dir, tmp_dir) install_packages(chroot_dir, tmp_dir, 'pkg1', 'pkg2') expected = [ 'prepare_chroot %(chroot_dir)s %(tmp_dir)s', 'mount proc %(chroot_dir)s/proc -t proc', '%(chroot_args)s %(chroot_dir)s apt-get --yes install pkg1 pkg2', '%(chroot_args)s %(chroot_dir)s apt-get clean', 'umount -v %(chroot_dir)s/proc'] keywords = dict( chroot_dir=chroot_dir, tmp_dir=tmp_dir, chroot_args=chroot_args) expected = [ "%s %s" % (sudo_args, line % keywords) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_prepare_chroot(self): self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) fixture = self.useFixture(MockCmdRunnerPopenFixture()) prepare_chroot('chroot', '/tmp/dir') run_local_atexit_funcs() expected = [ 'cp -a /etc/resolv.conf chroot/etc', 'cp -a /etc/hosts chroot/etc', 'cp /usr/bin/qemu-armeb-static chroot/usr/bin', 'cp /usr/bin/qemu-arm-static chroot/usr/bin', 'cp /usr/bin/qemu-aarch64-static chroot/usr/bin', 'rm -f chroot/usr/bin/qemu-aarch64-static', 'rm -f chroot/usr/bin/qemu-arm-static', 'rm -f chroot/usr/bin/qemu-armeb-static', 'rm -f chroot/etc/hosts', 'rm -f chroot/etc/resolv.conf'] expected = [ "%s %s" % (sudo_args, line) for line in expected] self.assertEquals(expected, fixture.mock.commands_executed) def test_run_local_atexit_funcs(self): self.useFixture(MockSomethingFixture( sys, 'stderr', open('/dev/null', 'w'))) self.call_order = [] class TestException(Exception): pass def raising_func(): self.call_order.append('raising_func') raise TestException() def behaving_func(): self.call_order.append('behaving_func') self.behaving_func_called = True # run_local_atexit_funcs() runs the atexit handlers in LIFO order, but # even though the first function called (raising_func) will raise # an exception, the second one will still be called after it. linaro_image_tools.media_create.chroot_utils.local_atexit = [ behaving_func, raising_func] # run_local_atexit_funcs() also propagates the last exception raised # by one of the functions. chroot_utils = linaro_image_tools.media_create.chroot_utils self.assertRaises(TestException, chroot_utils.run_local_atexit_funcs) self.assertEquals( ['raising_func', 'behaving_func'], self.call_order) def test_hwpack_atexit(self): self.run_local_atexit_functions_called = False def mock_run_local_atexit_functions(): self.run_local_atexit_functions_called = True def mock_install_hwpack(p1, p2, p3, p4): raise Exception('hwpack mock exception') self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'install_hwpack', mock_install_hwpack)) self.useFixture(MockSomethingFixture( linaro_image_tools.media_create.chroot_utils, 'run_local_atexit_funcs', mock_run_local_atexit_functions)) force_yes = True exception_caught = False extract_kpkgs = False try: install_hwpacks( 'chroot', '/tmp/dir', preferred_tools_dir(), force_yes, [], extract_kpkgs, 'hwp.tgz', 'hwp2.tgz') except: exception_caught = True self.assertTrue(self.run_local_atexit_functions_called) self.assertTrue(exception_caught) def setUp(self): super(TestInstallHWPack, self).setUp() # Ensure the list of cleanup functions gets cleared to make sure tests # don't interfere with one another. def clear_atexits(): linaro_image_tools.media_create.chroot_utils.local_atexit = [] self.addCleanup(clear_atexits) linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/partitions.py0000644000175000017500000005576612724020110026752 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from contextlib import contextmanager from math import ceil import atexit import dbus import glob import logging import re import subprocess import time from parted import ( Device, Disk, PARTITION_NORMAL, PARTITION_EXTENDED, ) from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) HEADS = 128 SECTORS = 32 SECTOR_SIZE = 512 # bytes CYLINDER_SIZE = HEADS * SECTORS * SECTOR_SIZE DBUS_PROPERTIES = 'org.freedesktop.DBus.Properties' UDISKS = "org.freedesktop.UDisks" # Max number of attempts to sleep (total sleep time in seconds = # 1+2+...+MAX_TTS) MAX_TTS = 10 # Image size should be a multiple of 1MiB, expressed in bytes. This is also # the minimum image size possible. ROUND_IMAGE_TO = 2 ** 20 MIN_IMAGE_SIZE = ROUND_IMAGE_TO def setup_android_partitions(board_config, media, image_size, bootfs_label, should_create_partitions, should_align_boot_part=False): if not media.is_block_device: image_size_in_bytes = get_partition_size_in_bytes(image_size) proc = cmd_runner.run( ['dd', 'of=%s' % media.path, 'bs=1', 'seek=%s' % image_size_in_bytes, 'count=0'], stderr=open('/dev/null', 'w')) proc.wait() if should_create_partitions: create_partitions( board_config, media, should_align_boot_part=should_align_boot_part) if media.is_block_device: bootfs, system, cache, data, sdcard = \ get_android_partitions_for_media(media, board_config) ensure_partition_is_not_mounted(bootfs) ensure_partition_is_not_mounted(system) ensure_partition_is_not_mounted(cache) ensure_partition_is_not_mounted(data) ensure_partition_is_not_mounted(sdcard) else: partitions = get_android_loopback_devices(media.path) bootfs = partitions[0] system = partitions[1] cache = partitions[2] data = partitions[3] sdcard = partitions[4] print "\nFormating boot partition\n" proc = cmd_runner.run( ['mkfs.vfat', '-F', str(board_config.fat_size), bootfs, '-n', bootfs_label], as_root=True) proc.wait() ext4_partitions = {"system": system, "cache": cache, "userdata": data} for label, dev in ext4_partitions.iteritems(): mkfs = 'mkfs.%s' % "ext4" proc = cmd_runner.run( [mkfs, '-F', dev, '-L', label], as_root=True) proc.wait() proc = cmd_runner.run( ['mkfs.vfat', '-F32', sdcard, '-n', "sdcard"], as_root=True) proc.wait() return bootfs, system, cache, data, sdcard # I wonder if it'd make sense to convert this into a small shim which calls # the appropriate function for the given type of device? I think it's still # small enough that there's not much benefit in doing that, but if it grows we # might want to do it. def setup_partitions(board_config, media, image_size, bootfs_label, rootfs_label, rootfs_type, should_create_partitions, should_format_bootfs, should_format_rootfs, should_align_boot_part=False, part_table="mbr"): """Make sure the given device is partitioned to boot the given board. :param board_config: A BoardConfig class. :param media: The Media we should partition. :param image_size: The size of the image file, in case we're setting up a QEMU image. :param bootfs_label: Label for the boot partition. :param rootfs_label: Label for the root partition. :param rootfs_type: Filesystem for the root partition. :param should_create_partitions: Whether or not we should erase existing partitions and create new ones. :param should_format_bootfs: Whether to reuse the filesystem on the boot partition. :param should_format_rootfs: Whether to reuse the filesystem on the root partition. :param should_align_boot_part: Whether to align the boot partition too. :param part_table: Type of partition table, either 'mbr' or 'gpt'. """ if not media.is_block_device: image_size_in_bytes = get_partition_size_in_bytes(image_size) proc = cmd_runner.run( ['dd', 'of=%s' % media.path, 'bs=1', 'seek=%s' % image_size_in_bytes, 'count=0'], stderr=open('/dev/null', 'w')) proc.wait() if should_create_partitions: create_partitions( board_config, media, should_align_boot_part=should_align_boot_part, part_table=part_table) if media.is_block_device: bootfs, rootfs = get_boot_and_root_partitions_for_media( media, board_config) # It looks like KDE somehow automounts the partitions after you # repartition a disk so we need to unmount them here to create the # filesystem. ensure_partition_is_not_mounted(bootfs) ensure_partition_is_not_mounted(rootfs) else: bootfs, rootfs = get_boot_and_root_loopback_devices(media.path) if should_format_bootfs: print "\nFormating boot partition\n" mkfs = 'mkfs.%s' % board_config.bootfs_type if board_config.bootfs_type == 'vfat': proc = cmd_runner.run( [mkfs, '-F', str(board_config.fat_size), bootfs, '-n', bootfs_label], as_root=True) else: proc = cmd_runner.run( [mkfs, bootfs, '-L', bootfs_label], as_root=True) proc.wait() if should_format_rootfs: print "\nFormating root partition\n" mkfs = 'mkfs.%s' % rootfs_type proc = cmd_runner.run( [mkfs, '-F', rootfs, '-L', rootfs_label], as_root=True) proc.wait() return bootfs, rootfs def umount(path): # The old code used to ignore failures here, but I don't think that's # desirable so I'm using cmd_runner.run()'s standard behaviour, which will # fail on a non-zero return value. cmd_runner.run(['umount', path], as_root=True).wait() @contextmanager def partition_mounted(device, path, *args): """A context manager that mounts the given device and umounts when done. We use a try/finally to make sure the device is umounted even if there's an uncaught exception in the with block. :param *args: Extra arguments to the mount command. """ subprocess_args = ['mount', device, path] subprocess_args.extend(args) cmd_runner.run(subprocess_args, as_root=True).wait() try: yield finally: try: umount(path) except cmd_runner.SubcommandNonZeroReturnValue, e: logger.warn("Failed to umount %s, but ignoring it because of a " "previous error" % path) logger.warn(e) def get_uuid(partition): """Find UUID of the given partition.""" proc = cmd_runner.run( ['blkid', '-o', 'udev', '-p', '-c', '/dev/null', partition], as_root=True, stdout=subprocess.PIPE) blkid_output, _ = proc.communicate() return _parse_blkid_output(blkid_output) def _parse_blkid_output(output): for line in output.splitlines(): uuid_match = re.match("ID_FS_UUID=(.*)", line) if uuid_match: return uuid_match.group(1) return None def ensure_partition_is_not_mounted(partition): """Ensure the given partition is not mounted, umounting if necessary.""" if is_partition_mounted(partition): cmd_runner.run(['umount', partition], as_root=True).wait() def is_partition_mounted(partition): """Is the given partition mounted?""" device_path = _get_udisks_device_path(partition) device = dbus.SystemBus().get_object(UDISKS, device_path) return device.Get( device_path, 'DeviceIsMounted', dbus_interface=DBUS_PROPERTIES) def get_boot_and_root_loopback_devices(image_file): """Return the boot and root loopback devices for the given image file. Register the loopback devices as well. """ vfat_size, vfat_offset, linux_size, linux_offset = ( calculate_partition_size_and_offset(image_file)) boot_device = register_loopback(image_file, vfat_offset, vfat_size) root_device = register_loopback(image_file, linux_offset, linux_size) return boot_device, root_device def get_android_loopback_devices(image_file): """Return the loopback devices for the given image file. Assumes a particular order of devices in the file. Register the loopback devices as well. """ devices = [] device_info = calculate_android_partition_size_and_offset(image_file) for device_offset, device_size in device_info: devices.append(register_loopback(image_file, device_offset, device_size)) return devices def register_loopback(image_file, offset, size): """Register a loopback device with an atexit handler to de-register it.""" def undo(device): cmd_runner.run(['losetup', '-d', device], as_root=True).wait() proc = cmd_runner.run( ['losetup', '-f', '--show', image_file, '--offset', str(offset), '--sizelimit', str(size)], stdout=subprocess.PIPE, as_root=True) device, _ = proc.communicate() device = device.strip() atexit.register(undo, device) return device def calculate_partition_size_and_offset(image_file): """Return the size and offset of the boot and root partitions. Both the size and offset are in sectors. :param image_file: A string containing the path to the image_file. :return: A 4-tuple containing the offset and size of the boot partition followed by the offset and size of the root partition. """ # Here we can use parted.Device to read the partitions because we're # reading from a regular file rather than a block device. If it was a # block device we'd need root rights. disk = Disk(Device(image_file)) vfat_partition = None linux_partition = None for partition in disk.partitions: assert partition.type == PARTITION_NORMAL, ( "Parted should only return normal partitions but got type %i" % partition.type) if 'boot' in partition.getFlagsAsString(): geometry = partition.geometry vfat_offset = geometry.start * SECTOR_SIZE vfat_size = geometry.length * SECTOR_SIZE vfat_partition = partition elif vfat_partition is not None: # next partition after boot partition is the root partition # NB: don't use vfat_partition.nextPartition() as that might return # a partition of type PARTITION_FREESPACE; it's much easier to # iterate disk.partitions which only returns # parted.PARTITION_NORMAL partitions geometry = partition.geometry linux_offset = geometry.start * SECTOR_SIZE linux_size = geometry.length * SECTOR_SIZE linux_partition = partition break assert vfat_partition is not None, ( "Couldn't find boot partition on %s" % image_file) assert linux_partition is not None, ( "Couldn't find root partition on %s" % image_file) return vfat_size, vfat_offset, linux_size, linux_offset def calculate_android_partition_size_and_offset(image_file): """Return the size and offset of the android partitions. Both the size and offset are in bytes. :param image_file: A string containing the path to the image_file. :return: A list of (offset, size) pairs. """ # Here we can use parted.Device to read the partitions because we're # reading from a regular file rather than a block device. If it was a # block device we'd need root rights. vfat_partition = None disk = Disk(Device(image_file)) partition_info = [] for partition in disk.partitions: # Will ignore any partitions before boot and of type EXTENDED if 'boot' in partition.getFlagsAsString(): vfat_partition = partition geometry = partition.geometry partition_info.append((geometry.start * SECTOR_SIZE, geometry.length * SECTOR_SIZE)) elif (vfat_partition is not None and partition.type != PARTITION_EXTENDED): geometry = partition.geometry partition_info.append((geometry.start * SECTOR_SIZE, geometry.length * SECTOR_SIZE)) # NB: don't use vfat_partition.nextPartition() as that might return # a partition of type PARTITION_FREESPACE; it's much easier to # iterate disk.partitions which only returns # parted.PARTITION_NORMAL partitions assert vfat_partition is not None, ( "Couldn't find boot partition on %s" % image_file) assert len(partition_info) == 5 return partition_info def get_android_partitions_for_media(media, board_config): """Return the device files for all the Android partitions of media. For boot we use partition number 1 plus the board's defined partition offset and for root we use partition number 2 plus the board's offset. This function must only be used for block devices. """ assert media.is_block_device, ( "This function must only be used for block devices") boot_partition = _get_device_file_for_partition_number( media.path, 1 + board_config.mmc_part_offset) system_partition = _get_device_file_for_partition_number( media.path, 2 + board_config.mmc_part_offset) if board_config.mmc_part_offset != 1: cache_partition = _get_device_file_for_partition_number( media.path, 3 + board_config.mmc_part_offset) else: # In the current setup, partition 4 is always the # extended partition container, so we need to skip 4 cache_partition = _get_device_file_for_partition_number( media.path, 5) data_partition = _get_device_file_for_partition_number( media.path, 5 + board_config.mmc_part_offset) sdcard_partition = _get_device_file_for_partition_number( media.path, 6 + board_config.mmc_part_offset) assert boot_partition is not None, ( "Could not find boot partition for %s" % media.path) assert system_partition is not None, ( "Could not find system partition for %s" % media.path) assert cache_partition is not None, ( "Could not find cache partition for %s" % media.path) assert data_partition is not None, ( "Could not find data partition for %s" % media.path) assert sdcard_partition is not None, ( "Could not find sdcard partition for %s" % media.path) return boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition def get_boot_and_root_partitions_for_media(media, board_config): """Return the device files for the boot and root partitions of media. For boot we use partition number 1 plus the board's defined partition offset and for root we use partition number 2 plus the board's offset. This function must only be used for block devices. """ assert media.is_block_device, ( "This function must only be used for block devices") boot_partition = _get_device_file_for_partition_number( media.path, 1 + board_config.mmc_part_offset) root_partition = _get_device_file_for_partition_number( media.path, 2 + board_config.mmc_part_offset) assert boot_partition is not None and root_partition is not None, ( "Could not find boot/root partition for %s" % media.path) return boot_partition, root_partition def _get_device_file_for_partition_number(device, partition): """Return the device file for the partition number on the given device. e.g. /dev/sda1 for the first partition on device /dev/sda or /dev/mmcblk0p3 for the third partition on /dev/mmcblk0. """ # This could be simpler but UDisks doesn't make it easy for us: # https://bugs.freedesktop.org/show_bug.cgi?id=33113. time_to_sleep = 1 dev_files = glob.glob("%s?*" % device) i = 0 while i < len(dev_files): dev_file = dev_files[i] try: device_path = _get_udisks_device_path(dev_file) partition_str = _get_udisks_device_file(device_path, partition) if partition_str: return partition_str i += 1 except dbus.exceptions.DBusException, e: if time_to_sleep > MAX_TTS: print "We've waited long enough..." raise print "*" * 60 print "UDisks doesn't know about %s: %s" % (dev_file, e) bus = dbus.SystemBus() manager = dbus.Interface( bus.get_object(UDISKS, "/org/freedesktop/UDisks"), UDISKS) print "This is what UDisks know about: %s" % ( manager.EnumerateDevices()) print "Sleeping for %d seconds" % time_to_sleep time.sleep(time_to_sleep) time_to_sleep += 1 print "*" * 60 return None def _get_udisks_device_path(device): """Return the UDisks path for the given device.""" bus = dbus.SystemBus() udisks = dbus.Interface( bus.get_object(UDISKS, "/org/freedesktop/UDisks"), UDISKS) return udisks.get_dbus_method('FindDeviceByDeviceFile')(device) def _get_udisks_device_file(path, part): """Return the UNIX special device file for the given partition.""" udisks_dev = dbus.SystemBus().get_object(UDISKS, path) part_number = udisks_dev.Get( path, 'PartitionNumber', dbus_interface=DBUS_PROPERTIES) if part_number == part: return str(udisks_dev.Get( path, 'DeviceFile', dbus_interface=DBUS_PROPERTIES)) def get_partition_size_in_bytes(size): """Convert a size string in Kbytes, Mbytes or Gbytes to bytes. The conversion rounds-up the size to the nearest MiB, considering a minimum size of MIN_IMAGE_SIZE bytes. The conversion always assures to have a big enough size for the partition. """ unit = size[-1].upper() real_size = float(size[:-1]) # no unit? (ends with a digit) if unit in '0123456789': real_size = float(size) elif unit == 'K': real_size = real_size * 1024 elif unit == 'M': real_size = real_size * 1024 * 1024 elif unit == 'G': real_size = real_size * 1024 * 1024 * 1024 else: raise ValueError("Unknown size format: %s. Use K[bytes], M[bytes] " "or G[bytes]" % size) # Guarantee that is a multiple of ROUND_IMAGE_TO real_size = _check_min_size(int(ceil(real_size / ROUND_IMAGE_TO) * ROUND_IMAGE_TO)) return real_size def _check_min_size(size): """Check that the image size is at least MIN_IMAGE_SIZE bytes. :param size: The size of the image to check, as a number. """ if (size < MIN_IMAGE_SIZE): size = MIN_IMAGE_SIZE return size def run_sfdisk_commands(commands, device, as_root=True, stderr=None): """Run the given commands under sfdisk. Every time sfdisk is invoked it will repartition the device so to create multiple partitions you should craft a list of newline-separated commands to be executed in a single sfdisk run. :param commands: A string of sfdisk commands; each on a separate line. :return: A 2-tuple containing the subprocess' stdout and stderr. """ # --force is unfortunate, but a consequence of having partitions not # starting on cylinder boundaries: sfdisk will abort with "Warning: # partition 2 does not start at a cylinder boundary" args = ['sfdisk', '--force', '-uS'] args.append(device) proc = cmd_runner.run( args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=stderr, as_root=as_root) return proc.communicate("%s\n" % commands) def run_sgdisk_commands(commands, device, as_root=True, stderr=None): args = ['sgdisk', device] args.extend(commands.split()) proc = cmd_runner.run(args, stderr=stderr, as_root=as_root) proc.wait() def create_partitions(board_config, media, should_align_boot_part=False, part_table="mbr"): """Partition the given media according to the board requirements. :param board_config: A BoardConfig class. :param media: A setup_partitions.Media object to partition. :param should_align_boot_part: Whether to align the boot partition too. :param part_table Type of partition table, either 'mbr' or 'gpt'. """ label = 'msdos' if part_table == 'gpt': label = part_table if media.is_block_device: # Overwrite any existing partition tables with a fresh one. proc = cmd_runner.run( ['parted', '-s', media.path, 'mklabel', label], as_root=True) proc.wait() wait_partition_to_settle(media, part_table) if part_table == 'gpt': sgdisk_cmd = board_config.get_sgdisk_cmd( should_align_boot_part=should_align_boot_part) run_sgdisk_commands(sgdisk_cmd, media.path) else: # default partition table to mbr sfdisk_cmd = board_config.get_sfdisk_cmd( should_align_boot_part=should_align_boot_part) run_sfdisk_commands(sfdisk_cmd, media.path) # sleep to wait for the partition to settle. wait_partition_to_settle(media, part_table) def wait_partition_to_settle(media, part_table): """Sleep in a loop to wait partition to settle :param media: A setup_partitions.Media object to partition. """ tts = 1 while (tts > 0) and (tts <= MAX_TTS): try: logger.info("Sleeping for %s second(s) to wait " "for the partition to settle" % tts) time.sleep(tts) args = ['sfdisk', '-l', media.path] if part_table == 'gpt': args = ['sgdisk', '-L', media.path] proc = cmd_runner.run(args, as_root=True, stdout=open('/dev/null', 'w')) proc.wait() return 0 except cmd_runner.SubcommandNonZeroReturnValue: logger.info("Partition table is not available " "for device %s" % media.path) tts += 1 logger.error("Couldn't read partition table " "for a reasonable time for device %s" % media.path) raise class Media(object): """A representation of the media where Linaro will be installed.""" def __init__(self, path): self.path = path self.is_block_device = path.startswith('/dev/') linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/rootfs.py0000644000175000017500000001566612724020110026065 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import subprocess import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.partitions import partition_mounted def populate_partition(content_dir, root_disk, partition): os.makedirs(root_disk) with partition_mounted(partition, root_disk): move_contents(content_dir, root_disk) def rootfs_mount_options(rootfs_type): """Return mount options for the specific rootfs type.""" if rootfs_type == "btrfs": return "defaults" if rootfs_type in ('ext2', 'ext3', 'ext4'): return "errors=remount-ro" raise ValueError('Unsupported rootfs type') def populate_rootfs(content_dir, root_disk, partition, rootfs_type, rootfs_id, should_create_swap, swap_size, mmc_device_id, partition_offset, os_release_id, board_config=None): """Populate the rootfs and make the necessary tweaks to make it usable. This consists of: 1. Create a directory on the path specified by root_disk 2. Mount the given partition onto the created directory. 3. Setup an atexit handler to unmount the partition mounted above. 4. Move the contents of content_dir to that directory. 5. If should_create_swap, then create it with the given size. 6. Add fstab entries for the / filesystem and swap (if created). 7. Create a /etc/flash-kernel.conf containing the target's boot device. """ print "\nPopulating rootfs partition" print "Be patient, this may take a few minutes\n" # Create a directory to mount the rootfs partition. os.makedirs(root_disk) with partition_mounted(partition, root_disk): move_contents(content_dir, root_disk) mount_options = rootfs_mount_options(rootfs_type) fstab_additions = ["%s / %s %s 0 1" % ( rootfs_id, rootfs_type, mount_options)] if should_create_swap: print "\nCreating SWAP File\n" if has_space_left_for_swap(root_disk, swap_size): proc = cmd_runner.run([ 'dd', 'if=/dev/zero', 'of=%s/SWAP.swap' % root_disk, 'bs=1M', 'count=%s' % swap_size], as_root=True) proc.wait() proc = cmd_runner.run( ['mkswap', '%s/SWAP.swap' % root_disk], as_root=True) proc.wait() fstab_additions.append("/SWAP.swap none swap sw 0 0") else: print ("Swap file is bigger than space left on partition; " "continuing without swap.") append_to_fstab(root_disk, fstab_additions) if os_release_id == 'debian' or os_release_id == 'ubuntu' or \ os.path.exists('%s/etc/debian_version' % root_disk): print "\nCreating /etc/flash-kernel.conf\n" create_flash_kernel_config( root_disk, mmc_device_id, 1 + partition_offset) if board_config is not None: print "\nUpdating /etc/network/interfaces\n" update_network_interfaces(root_disk, board_config) def update_network_interfaces(root_disk, board_config): interfaces = [] if board_config.wired_interfaces is not None: interfaces.extend(board_config.wired_interfaces) if board_config.wireless_interfaces is not None: interfaces.extend(board_config.wireless_interfaces) if_path = os.path.join(root_disk, 'etc', 'network', 'interfaces') if os.path.exists(if_path): with open(if_path) as if_file: config = if_file.read() else: config = '' for interface in interfaces: if interface not in config: config += "auto %(if)s\niface %(if)s inet dhcp\n" % ( {'if': interface}) if config != '': write_data_to_protected_file(if_path, config) def create_flash_kernel_config(root_disk, mmc_device_id, boot_partition_number): """Create a flash-kernel.conf file under root_disk/etc. Uses the given partition number to figure out the boot partition. """ target_boot_dev = '/dev/mmcblk%dp%s' % ( mmc_device_id, boot_partition_number) flash_kernel = os.path.join(root_disk, 'etc', 'flash-kernel.conf') write_data_to_protected_file( flash_kernel, "UBOOT_PART=%s\n" % target_boot_dev) def _list_files(directory): """List the files and dirs under the given directory. Runs as root because we want to list everything, including stuff that may not be world-readable. """ p = cmd_runner.run( ['find', directory, '-maxdepth', '1', '-mindepth', '1', '!', '-name', 'lost+found'], stdout=subprocess.PIPE, as_root=True) stdout, _ = p.communicate() return stdout.split() def move_contents(from_, root_disk): """Move everything under from_ to the given root disk. Uses sudo for moving. """ assert os.path.isdir(from_), "%s is not a directory" % from_ files = _list_files(from_) mv_cmd = ['mv'] mv_cmd.extend(sorted(files)) mv_cmd.append(root_disk) cmd_runner.run(mv_cmd, as_root=True).wait() def has_space_left_for_swap(root_disk, swap_size_in_mega_bytes): """Is there enough space for a swap file in the given root disk?""" statvfs = os.statvfs(root_disk) free_space = statvfs.f_bavail * statvfs.f_bsize swap_size_in_bytes = int(swap_size_in_mega_bytes) * 1024 ** 2 if free_space >= swap_size_in_bytes: return True return False def append_to_fstab(root_disk, fstab_additions): fstab = os.path.join(root_disk, 'etc', 'fstab') data = open(fstab).read() + '\n' + '\n'.join(fstab_additions) + '\n' write_data_to_protected_file(fstab, data) def write_data_to_protected_file(path, data): """Write data to the file on the given path. This is meant to be used when the given file is only writable by root, and we overcome that by writing the data to a tempfile and then moving the tempfile on top of the given one using sudo. """ _, tmpfile = tempfile.mkstemp() with open(tmpfile, 'w') as fd: fd.write(data) cmd_runner.run(['mv', '-f', tmpfile, path], as_root=True).wait() linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/unpack_binary_tarball.py0000644000175000017500000000570612724020110031071 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import re import subprocess from linaro_image_tools import cmd_runner def unpack_android_binary_tarball(tarball, unpack_dir, as_root=True): if is_tar_support_selinux(): tar_cmd = ['tar', '--selinux', '--numeric-owner', '-C', unpack_dir, '-jxf', tarball] else: tar_cmd = ['tar', '--numeric-owner', '-C', unpack_dir, '-jxf', tarball] proc = cmd_runner.run(tar_cmd, as_root=as_root, stderr=subprocess.PIPE) stderr = proc.communicate()[1] selinux_warn_outputted = False selinux_warn1 = "tar: Ignoring unknown extended header keyword" selinux_warn2 = "tar: setfileconat: Cannot set SELinux context" for line in stderr.splitlines(): # following 2 messages will not occur at the same time index = line.find(selinux_warn1) index2 = line.find(selinux_warn2) if index == -1 and index2 == -1: print line continue elif not selinux_warn_outputted: # either index != -1 or index2 != -1 print line print ("WARNING: selinux will not work correctly since the\n" " --selinux option of tar command in this OS\n" " is not fully supported\n") selinux_warn_outputted = True else: # same line of selinux_warn1 or selinux_warn2 continue return proc.returncode def unpack_binary_tarball(tarball, unpack_dir, as_root=True): extract_opt = '-xf' if tarball.endswith('.xz'): extract_opt = '-Jxf' proc = cmd_runner.run( ['tar', '--numeric-owner', '-C', unpack_dir, extract_opt, tarball], as_root=as_root) proc.wait() return proc.returncode def is_tar_support_selinux(): try: tar_help, _ = cmd_runner.Popen( ['tar', '--help'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, ).communicate() except cmd_runner.SubcommandNonZeroReturnValue: return False for line in tar_help.splitlines(): selinux_support = re.search('--selinux', line) if selinux_support: return True return False linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/__init__.py0000644000175000017500000002531712724020110026302 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import argparse import subprocess import os from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.boards import board_configs from linaro_image_tools.media_create.android_boards import ( android_board_configs) from linaro_image_tools.__version__ import __version__ from linaro_image_tools.hwpack.hwpack_fields import ( DEFAULT_BOOTLOADER ) KNOWN_BOARDS = sorted(board_configs.keys()) ANDROID_KNOWN_BOARDS = sorted(android_board_configs.keys()) class Live256MegsAction(argparse.Action): """A custom argparse.Action for the --live-256m option. It is a store_true action for the given dest plus a store_true action for 'is_live'. """ def __init__(self, option_strings, dest, default=None, required=False, help=None, metavar=None): super(Live256MegsAction, self).__init__( option_strings=option_strings, dest=dest, nargs=0, default=False, required=required, help=help) def __call__(self, parser, namespace, values, option_string=None): setattr(namespace, self.dest, True) setattr(namespace, 'is_live', True) def get_version(): qemu_path = '/usr/bin/qemu-arm-static' if os.path.exists(qemu_path): # qemu-arm-static has -version option proc = cmd_runner.run([qemu_path, "-version"], stdout=subprocess.PIPE, stderr=subprocess.PIPE) (qemu_version, stderrdata) = proc.communicate() if (proc.returncode or stderrdata): qemu_version = "qemu-arm version unknown (%s)" % stderrdata else: qemu_version = "Cannot find %s." % qemu_path return "%s\n* %s" % (__version__, qemu_version) def add_common_options(parser): parser.add_argument( '--extra-boot-args', dest='extra_boot_args', required=False, help='Extra boot args.') parser.add_argument( '--extra-boot-args-file', dest='extra_boot_args_file', required=False, help=('File containing extra boot arguments.')) parser.add_argument("--debug", action="store_true") def get_args_parser(): """Get the ArgumentParser for the arguments given on the command line.""" parser = argparse.ArgumentParser( version='%(prog)s ' + get_version(), formatter_class=argparse.RawTextHelpFormatter) group = parser.add_mutually_exclusive_group() group.add_argument( '--mmc', dest='device', default="sd.img", help='The storage device to use.') group.add_argument( '--image-file', '--image_file', dest='device', default="sd.img", help='File where we should write an image file (defaults to sd.img ' 'if neither --image-file or --mmc are specified.)') parser.add_argument( '--output-directory', dest='directory', help='Directory where image and accessories should be written to.') parser.add_argument( '--read-hwpack', dest='readhwpack', action='store_true', help=('Read the hardware pack and print information about the ' 'supported boards and bootloaders.')) parser.add_argument( '--dev', dest='dev', choices=KNOWN_BOARDS, help='Generate an SD card or image for the given board.') parser.add_argument( '--part-table', default='mbr', choices=['mbr', 'gpt'], help='Type of partition table to use for the MMC image') parser.add_argument( '--rootfs', default='ext4', choices=['ext2', 'ext3', 'ext4', 'btrfs'], help='Type of filesystem to use for the rootfs') parser.add_argument( '--rfs-label', '--rfs_label', default='rootfs', help='Label to use for the root filesystem.') parser.add_argument( '--boot-label', '--boot_label', default='boot', help='Label to use for the boot filesystem.') parser.add_argument( '--swap-file', '--swap_file', type=int, help='Create a swap file of the given size (in MBs).') group = parser.add_mutually_exclusive_group() group.add_argument( '--live', dest='is_live', action='store_true', help=('Create boot command for casper/live images; if this is not ' 'provided the UUID for the rootfs is used as the root= option')) group.add_argument( '--live-256m', dest='is_lowmem', action=Live256MegsAction, help=('Create boot command for casper/live images; adds ' 'only-ubiquity option to allow use of live installer on ' 'boards with 256M memory - like beagle.')) parser.add_argument( '--console', action='append', dest='consoles', default=[], help=('Add a console to kernel boot parameter; this parameter can be ' 'defined multiple times.')) parser.add_argument( '--hwpack', action='append', dest='hwpacks', required=True, help=('A hardware pack that should be installed in the rootfs; this ' 'parameter can be defined multiple times.')) parser.add_argument( '--hwpack-sig', action='append', dest='hwpacksigs', required=False, default=[], help=('Signature file for verifying a hwpack; this ' 'parameter can be defined multiple times.')) parser.add_argument( '--hwpack-force-yes', action='store_true', help='Pass --force-yes to linaro-hwpack-install') parser.add_argument( '--image-size', '--image_size', default='3G', help=('The image size, specified in mega/giga bytes (e.g. 3000M or ' '3G); use with --image_file only')) parser.add_argument( '--binary', default='binary-tar.tar.gz', required=False, help=('The tarball containing the rootfs used to create the bootable ' 'system.')) parser.add_argument( '--binary-sig', dest='binarysig', required=False, help=('Signature file used for verifying the binary tarball.')) parser.add_argument( '--no-rootfs', dest='should_format_rootfs', action='store_false', help='Do not deploy the root filesystem.') parser.add_argument( '--no-bootfs', dest='should_format_bootfs', action='store_false', help='Do not deploy the boot filesystem.') parser.add_argument( '--no-part', dest='should_create_partitions', action='store_false', help='Reuse existing partitions on the given media.') parser.add_argument( '--align-boot-part', dest='should_align_boot_part', action='store_true', help='Align boot partition too (might break older x-loaders).') parser.add_argument( '--nocheck-mmc', dest='nocheck_mmc', action='store_true', help=('Assume yes to the question "Are you 100%% sure, ' 'on selecting [mmc]"')) parser.add_argument( '--bootloader', help="Select a bootloader from a hardware pack that contains more " "than one. If not specified, it will default to '%s'." % DEFAULT_BOOTLOADER) parser.add_argument( '--dtb-file', help="Select a DTB file from a hardware pack that contains more " "than one. If not specified, it will default to the first " "entry in 'dtb_files' list.") add_common_options(parser) return parser def get_android_args_parser(): """Get the ArgumentParser for the arguments given on the command line.""" parser = argparse.ArgumentParser( version='%(prog)s ' + get_version(), formatter_class=argparse.RawTextHelpFormatter) group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--mmc', dest='device', help='The storage device to use.') group.add_argument( '--image-file', '--image_file', dest='device', help='File where we should write the image file.') parser.add_argument( '--hwpack', required=False, help=('An Android hardware pack file with the board configuration.')) parser.add_argument( '--image-size', '--image_size', default='2G', help=('The image size, specified in mega/giga bytes (e.g. 3000M or ' '3G); use with --image_file only')) parser.add_argument( '--dev', required=True, dest='dev', choices=ANDROID_KNOWN_BOARDS, help='Generate an SD card or image for the given board.') parser.add_argument( '--boot-label', '--boot_label', default='boot', help='Label to use for the boot filesystem.') parser.add_argument( '--console', action='append', dest='consoles', default=[], help=('Add a console to kernel boot parameter; this parameter can be ' 'defined multiple times.')) #group for system partition content specification group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--system', dest="system", help=('The tarball containing the Android system paritition.' 'Like system.tar.bz2')) group.add_argument( '--systemimage', dest="systemimage", help=('The ext4 filesystem data file containing the Android ' 'system paritition. Like system.img')) #group for userdata partition content specification group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--userdata', dest="userdata", help=('The tarball containing the Android data paritition.' 'Like userdata.tar.bz2')) group.add_argument( '--userdataimage', dest="userdataimage", help=('The ext4 filesystem data containing the Android ' 'data paritition. Like userdata.img')) parser.add_argument( '--boot', default='boot.tar.bz2', required=True, help=('The tarball containing the Android root partition')) parser.add_argument( '--no-part', dest='should_create_partitions', action='store_false', help='Reuse existing partitions on the given media.') parser.add_argument( '--align-boot-part', dest='should_align_boot_part', action='store_true', help='Align boot partition too (might break older x-loaders).') add_common_options(parser) return parser linaro-image-tools-2016.05.orig/linaro_image_tools/media_create/android_boards.py0000644000175000017500000006425212724020110027516 0ustar voipiovoipio# Copyright (C) 2011 Linaro # # Author: Jeremy Chang # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . """Configuration for boards supported by linaro-android-media-create. To add support for a new board, you need to create a subclass of AndroidBoardConfig, create an Android hwpack as explained here: https://wiki.linaro.org/AndroidHardwarePacksV3 and add the board to 'android_board_configs' at the end of this file. """ import os import yaml import logging from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.hwpack_fields import FORMAT_FIELD from linaro_image_tools.media_create.partitions import SECTOR_SIZE from linaro_image_tools.media_create.boards import ( ArndaleConfig, ArndaleOctaConfig, BeagleConfig, BoardConfig, BoardConfigException, Mx53LoCoConfig, OrigenConfig, OrigenQuadConfig, PART_ALIGN_S, PandaConfig, SMDKV310Config, SnowballEmmcConfig, SnowballSdConfig, VexpressConfig, align_partition, align_up, install_mx5_boot_loader, make_boot_script, _dd, BoardException, ) from linaro_image_tools.utils import DEFAULT_LOGGER_NAME logger = logging.getLogger(DEFAULT_LOGGER_NAME) BOOT_MIN_SIZE_S = align_up(128 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE SYSTEM_MIN_SIZE_S = align_up(896 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE CACHE_MIN_SIZE_S = align_up(256 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE USERDATA_MIN_SIZE_S = align_up(576 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE SDCARD_MIN_SIZE_S = align_up(128 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE LOADER_MIN_SIZE_S = align_up(1 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE class AndroidBoardConfig(BoardConfig): def __init__(self): super(AndroidBoardConfig, self).__init__() self.dtb_name = None self._extra_serial_options = [] self._android_specific_args = [] self._extra_boot_args_options = [] self._live_serial_options = [] def _get_android_specific_args(self): android_args = self._android_specific_args if isinstance(android_args, list): android_args = ' '.join(self._android_specific_args) return android_args def _set_android_specific_args(self, value): self._android_specific_args = value android_specific_args = property(_get_android_specific_args, _set_android_specific_args) def _get_extra_boot_args_options(self): extra_boot_args = self._extra_boot_args_options if isinstance(extra_boot_args, list): extra_boot_args = ' '.join(self._extra_boot_args_options) return extra_boot_args def _set_extra_boot_args_options(self, value): self._extra_boot_args_options = value extra_boot_args_options = property(_get_extra_boot_args_options, _set_extra_boot_args_options) def _get_extra_serial_options(self): extra_serial = self._extra_serial_options if isinstance(extra_serial, list): extra_serial = ' '.join(self._extra_serial_options) return extra_serial def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def _get_live_serial_options(self): serial_options = self._live_serial_options if serial_options: if isinstance(serial_options, list): serial_options = ' '.join(self._live_serial_options) if self._check_placeholder_presence(serial_options, '%s'): serial_options = serial_options % self.serial_tty return serial_options def _set_live_serial_options(self, value): self._live_serial_options = value live_serial_options = property(_get_live_serial_options, _set_live_serial_options) def from_file(self, hwpack): """Loads the Android board configuration from an Android hardware pack configuration file and sets the config attributes with their values. :param hwpack: The Android hwpack configuration file. :return The configuration read from the file as a dictionary. """ try: with open(hwpack, 'r') as hw: config = yaml.safe_load(hw) self._set_attributes(config) return config except yaml.YAMLError, ex: logger.debug("Error loading YAML file %s" % hwpack, ex) raise BoardConfigException("Error reading Android hwpack %s" % hwpack) except IOError, ex: logger.debug("Error reading hwpack file %s" % hwpack, ex) raise BoardConfigException("Android hwpack %s could not be found" % hwpack) def _set_attributes(self, config): """Initialize the class attributes with the values read from the Android hardware pack configuration file. :param config: The config read from the Android hwpack. """ for name, value in config.iteritems(): if name == FORMAT_FIELD: setattr(self, 'hwpack_format', value) elif hasattr(self, name): setattr(self, name, value) else: logger.warning("Attribute '%s' does not belong to this " "instance of '%s'." % (name, self.__class__)) def _get_bootargs(self, consoles): """Get the bootargs for this board. In general subclasses should not have to override this. """ boot_args_options = 'rootwait ro' if self.extra_boot_args_options: boot_args_options += ' %s' % self.extra_boot_args_options boot_args_options += ' %s' % self.android_specific_args serial_options = self.extra_serial_options for console in consoles: serial_options += ' console=%s' % console replacements = dict( serial_options=serial_options, boot_args_options=boot_args_options) return ( "%(serial_options)s " "%(boot_args_options)s" % replacements).strip() def _get_boot_env(self, consoles): """Get the boot environment for this board. In general subclasses should not have to override this. """ boot_env = {} boot_env["bootargs"] = self._get_bootargs(consoles) initrd = False if self.initrd_addr: initrd = True # On Android, the DTB file is always built as part of the kernel it # comes from - and lives in the same directory in the boot tarball, so # here we don't need to pass the whole path to it. boot_env["bootcmd"] = self._get_bootcmd(initrd, self.dtb_name) boot_env["initrd_high"] = self.initrd_high boot_env["fdt_high"] = self.fdt_high return boot_env def populate_boot_script(self, boot_partition, boot_disk, consoles): cmd_runner.run(['mkdir', '-p', boot_disk]).wait() # TODO: Use partition_mounted() here to make sure the partition is # always umounted after we're done. cmd_runner.run(['mount', boot_partition, boot_disk], as_root=True).wait() boot_env = self._get_boot_env(consoles) cmdline_filepath = os.path.join(boot_disk, "cmdline") with open(cmdline_filepath, 'r') as cmdline_file: android_kernel_cmdline = cmdline_file.read().strip() boot_env['bootargs'] = boot_env['bootargs'] + ' ' + \ android_kernel_cmdline boot_dir = boot_disk boot_script_path = os.path.join(boot_dir, self.boot_script) make_boot_script(boot_env, boot_script_path) try: cmd_runner.run(['umount', boot_disk], as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass def get_sfdisk_cmd(self, should_align_boot_part=False, start_addr=0, extra_part=False): if self.fat_size == 32: partition_type = '0x0C' else: partition_type = '0x0E' # align on sector 63 for compatibility with broken versions of x-loader # unless align_boot_part is set boot_align = 63 if should_align_boot_part: boot_align = PART_ALIGN_S # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( start_addr + 1, BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room boot_len = boot_len - boot_len % 2 boot_end = boot_start + boot_len - 1 system_start, _system_end, _system_len = align_partition( boot_end + 1, SYSTEM_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) cache_start, _cache_end, _cache_len = align_partition( _system_end + 1, CACHE_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) userdata_start, _userdata_end, _userdata_len = align_partition( _cache_end + 1, USERDATA_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) sdcard_start, _sdcard_end, _sdcard_len = align_partition( _userdata_end + 1, SDCARD_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) # Snowball board needs a raw partition added to the beginning of image. # If extra_part is True an extra primary partition will be added. # Due to a maximum of 4 primary partitions cache data will be placed in # a extended partition if extra_part is True: assert start_addr > 0, ("Not possible to add extra partition" "when boot partition starts at '0'") return '%s,%s,%s,*\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,%s,L\n%s,,,-' % ( boot_start, boot_len, partition_type, system_start, _system_len, cache_start, cache_start, _cache_len, userdata_start, _userdata_len, sdcard_start) return '%s,%s,%s,*\n%s,%s,L\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,,,-' % ( boot_start, boot_len, partition_type, system_start, _system_len, cache_start, _cache_len, userdata_start, userdata_start, _userdata_len, sdcard_start) def populate_raw_partition(self, media, boot_dir): super(AndroidBoardConfig, self).populate_raw_partition(media, boot_dir) def install_boot_loader(self, boot_partition, boot_device_or_file): pass class AndroidOmapConfig(AndroidBoardConfig): """Placeholder class for OMAP configuration inheritance.""" class AndroidBeagleConfig(AndroidOmapConfig, BeagleConfig): """Placeholder class for Beagle configuration inheritance.""" def __init__(self): super(AndroidBeagleConfig, self).__init__() self._android_specific_args = 'init=/init androidboot.console=ttyO2' self._extra_serial_options = 'console=tty0 console=ttyO2,115200n8' class AndroidPandaConfig(AndroidBoardConfig, PandaConfig): """Placeholder class for Panda configuration inheritance.""" def __init__(self): super(AndroidPandaConfig, self).__init__() self.dtb_name = 'board.dtb' self._extra_serial_options = 'console=ttyO2,115200n8' self._extra_boot_args_options = ( 'earlyprintk fixrtc nocompcache vram=48M ' 'omapfb.vram=0:24M,1:24M mem=456M@0x80000000 mem=512M@0xA0000000') self._android_specific_args = 'init=/init androidboot.console=ttyO2' self.dtb_addr = '0x815f0000' self.bootloader_flavor = 'omap4_panda' class AndroidSnowballSdConfig(AndroidBoardConfig, SnowballSdConfig): """Placeholder class for Snowball Sd configuration inheritance.""" def __init__(self): super(AndroidSnowballSdConfig, self).__init__() self.dtb_name = 'board.dtb' self._android_specific_args = 'init=/init androidboot.console=ttyAMA2' self._extra_boot_args_options = ( 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M vmalloc=500M') self._extra_serial_options = 'console=ttyAMA2,115200n8' self.boot_script = 'boot.scr' self.fdt_high = '0x05000000' self.initrd_addr = '0x05000000' self.initrd_high = '0x06000000' self.dtb_addr = '0x8000000' class AndroidSnowballEmmcConfig(AndroidBoardConfig, SnowballEmmcConfig): """Class for Snowball Emmc configuration inheritance.""" def __init__(self): super(AndroidSnowballEmmcConfig, self).__init__() self.dtb_name = 'board.dtb' self._extra_boot_args_options = ( 'earlyprintk mem=128M@0 mali.mali_mem=64M@128M hwmem=168M@192M ' 'mem=22M@360M mem_issw=1M@383M mem=640M@384M vmalloc=500M') self._extra_serial_options = 'console=ttyAMA2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttyAMA2' self.boot_script = 'boot.scr' self.fdt_high = '0x05000000' self.initrd_addr = '0x05000000' self.initrd_high = '0x06000000' self.mmc_option = '0:2' self.dtb_addr = '0x8000000' def get_sfdisk_cmd(self, should_align_boot_part=False): loader_start, loader_end, loader_len = align_partition( SnowballEmmcConfig.SNOWBALL_LOADER_START_S, LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) command = super(AndroidSnowballEmmcConfig, self).get_sfdisk_cmd( should_align_boot_part=True, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) def populate_raw_partition(self, media, boot_dir): # To avoid adding a Snowball specific command line option, we assume # that the user already has unpacked the startfiles to ./startupfiles config_files_dir = self.snowball_config(boot_dir) assert os.path.exists(config_files_dir), ( "You need to unpack the Snowball startupfiles to the directory " "'startupfiles' in your current working directory. See " "igloocommunity.org for more information.") # We copy the u-boot files from the unpacked boot.tar.bz2 # and put it with the startfiles. boot_files = ['u-boot.bin'] for boot_file in boot_files: cmd_runner.run(['cp', os.path.join(boot_dir, 'boot', boot_file), config_files_dir], as_root=True).wait() super(AndroidSnowballEmmcConfig, self).populate_raw_partition( media, boot_dir) def snowball_config(self, chroot_dir): # The user is expected to have unpacked the startupfiles to this subdir # of their working dir. return os.path.join('.', 'startupfiles') @property def delete_startupfiles(self): # The startupfiles will have been unpacked to the user's working dir # and should not be deleted after they have been installed. return False class AndroidMx53LoCoConfig(AndroidBoardConfig, Mx53LoCoConfig): """Class for Mx53LoCo configuration inheritance.""" def __init__(self): super(AndroidMx53LoCoConfig, self).__init__() self._extra_boot_args_options = ( 'earlyprintk rootdelay=1 fixrtc nocompcache di1_primary tve') self._extra_serial_options = 'console=%s,115200n8' self._android_specific_args = 'init=/init androidboot.console=%s' def _get_extra_serial_options(self): serial_options = self._extra_serial_options if serial_options: if isinstance(serial_options, list): serial_options = ' '.join(serial_options) if self._check_placeholder_presence(serial_options, '%s'): serial_options = serial_options % self.serial_tty return serial_options def _set_extra_serial_options(self, value): self._extra_serial_options = value extra_serial_options = property(_get_extra_serial_options, _set_extra_serial_options) def _get_android_specific_args(self): android_args = self._android_specific_args if android_args: if isinstance(android_args, list): android_args = ' '.join(android_args) if self._check_placeholder_presence(android_args, '%s'): android_args = android_args % self.serial_tty return android_args def _set_android_specific_args(self, value): self._android_specific_args = value android_specific_args = property(_get_android_specific_args, _set_android_specific_args) def get_sfdisk_cmd(self, should_align_boot_part=False): loader_start, loader_end, loader_len = align_partition( 1, self.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) command = super(AndroidMx53LoCoConfig, self).get_sfdisk_cmd( should_align_boot_part=True, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) def install_boot_loader(self, boot_partition, boot_device_or_file): install_mx5_boot_loader( os.path.join(boot_device_or_file, "u-boot.imx"), boot_partition, self.LOADER_MIN_SIZE_S) class AndroidMx6QSabreliteConfig(AndroidMx53LoCoConfig): """Placeholder class for Mx6Q Sabrelite configuration inheritance.""" def __init__(self): super(AndroidMx6QSabreliteConfig, self).__init__() self.dtb_name = 'board.dtb' self.bootloader_flavor = 'mx6qsabrelite' self.kernel_addr = '0x10000000' self.initrd_addr = '0x12000000' self.load_addr = '0x10008000' self.dtb_addr = '0x11ff0000' class AndroidSamsungConfig(AndroidBoardConfig): def get_sfdisk_cmd(self, should_align_boot_part=False): loaders_min_len = (self.samsung_bl1_start + self.samsung_bl1_len + self.samsung_bl2_len + self.samsung_env_len) loader_start, loader_end, loader_len = align_partition( 1, loaders_min_len, 1, PART_ALIGN_S) command = super(AndroidSamsungConfig, self).get_sfdisk_cmd( should_align_boot_part=False, start_addr=loader_end, extra_part=True) return '%s,%s,0xDA\n%s' % ( loader_start, loader_len, command) class AndroidSMDKV310Config(AndroidSamsungConfig, SMDKV310Config): """Placeholder class for SMDKV310 configuration inheritance.""" def __init__(self): super(AndroidSMDKV310Config, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC1,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC1' class AndroidOrigenConfig(AndroidSamsungConfig, OrigenConfig): """Placeholder class for Origen configuration inheritance.""" def __init__(self): super(AndroidOrigenConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC2' class AndroidOrigenQuadConfig(AndroidSamsungConfig, OrigenQuadConfig): def __init__(self): super(AndroidOrigenQuadConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttySAC2,115200n8' self._android_specific_args = 'init=/init androidboot.console=ttySAC2' class AndroidVexpressConfig(AndroidBoardConfig, VexpressConfig): """Placeholder class for Vexpress configuration inheritance.""" def __init__(self): super(AndroidVexpressConfig, self).__init__() self._extra_serial_options = 'console=tty0 console=ttyAMA0,38400n8' self._android_specific_args = 'init=/init androidboot.console=ttyAMA0' class AndroidArndaleConfig(AndroidSamsungConfig, ArndaleConfig): """Placeholder class for Arndale configuration inheritance.""" def __init__(self): super(AndroidArndaleConfig, self).__init__() self.mmc_option = '0:1' self.kernel_addr = '0x40007000' self.initrd_addr = '0x41000000' self.dtb_addr = '0x41f00000' self.dtb_name = 'exynos5250-arndale.dtb' self._android_specific_args = ( 'init=/init androidboot.console=ttySAC2 console=ttySAC2 initrd=%s' % self.initrd_addr) self._extra_serial_options = 'ttySAC2,115200n8' self._extra_boot_args_options = 'rootdelay=3' def _get_bootcmd(self, i_img_data, d_img_data): """Get the bootcmd for this board. In general subclasses should not have to override this. """ replacements = dict( fatload_command=self.fatload_command, uimage_path=self.uimage_path, mmc_option=self.mmc_option, kernel_addr=self.kernel_addr, initrd_addr=self.initrd_addr, dtb_addr=self.dtb_addr) boot_script = ( ("%(fatload_command)s mmc %(mmc_option)s %(kernel_addr)s " + "%(uimage_path)suImage; ")) % replacements if i_img_data is not None: boot_script += ( ("%(fatload_command)s mmc %(mmc_option)s %(initrd_addr)s " + "%(uimage_path)suInitrd; ")) % replacements if d_img_data is not None: assert self.dtb_addr is not None, ( "Need a dtb_addr when passing d_img_data") boot_script += ("%(fatload_command)s mmc %(mmc_option)s " "%(dtb_addr)s " % replacements) boot_script += "%s; " % d_img_data boot_script += (("bootm %(kernel_addr)s")) % replacements if i_img_data is not None: boot_script += ((" %(initrd_addr)s")) % replacements if d_img_data is not None: boot_script += ((" %(dtb_addr)s")) % replacements return boot_script def populate_raw_partition(self, boot_device_or_file, chroot_dir): boot_bin_0 = {'name': 'arndale-bl1.bin', 'seek': 1} boot_bin_1 = {'name': 'u-boot-mmc-spl.bin', 'seek': 17} boot_bin_2 = {'name': 'u-boot.bin', 'seek': 49} boot_bins = [boot_bin_0, boot_bin_1, boot_bin_2] boot_partition = 'boot' # Zero the env so that the boot_script will get loaded _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) for boot_bin in boot_bins: name = boot_bin['name'] file_path = os.path.join(chroot_dir, boot_partition, name) if not os.path.exists(file_path): raise BoardException( "File '%s' does not exists. Cannot proceed." % name) _dd(file_path, boot_device_or_file, seek=boot_bin['seek']) class AndroidArndaleOctaConfig(AndroidArndaleConfig, ArndaleOctaConfig): """Placeholder class for Arndale-Octa configuration inheritance.""" def __init__(self): super(AndroidArndaleOctaConfig, self).__init__() self.samsung_env_start = 1231 self.mmc_option = '0:2' self.kernel_addr = '0x20007000' self.initrd_addr = '0x21000000' self.dtb_addr = '0x21f00000' self.dtb_name = 'exynos5420-arndale-octa.dtb' self._android_specific_args = ( 'init=/init androidboot.console=ttySAC3 console=ttySAC3 initrd=%s' % self.initrd_addr) self._extra_serial_options = 'ttySAC3,115200n8' self._extra_boot_args_options = 'rootdelay=3' def populate_raw_partition(self, boot_device_or_file, chroot_dir): boot_bin_0 = {'name': 'arndale-octa.bl1.bin', 'seek': 1} boot_bin_1 = {'name': 'smdk5420-spl.signed.bin', 'seek': 31} boot_bin_2 = {'name': 'u-boot.bin', 'seek': 63} boot_bin_3 = {'name': 'arndale-octa.tzsw.bin', 'seek': 719} boot_bins = [boot_bin_0, boot_bin_1, boot_bin_2, boot_bin_3] boot_partition = 'boot' # Zero the env so that the boot_script will get loaded _dd("/dev/zero", boot_device_or_file, count=self.samsung_env_len, seek=self.samsung_env_start) for boot_bin in boot_bins: name = boot_bin['name'] file_path = os.path.join(chroot_dir, boot_partition, name) if not os.path.exists(file_path): raise BoardException( "File '%s' does not exists. Cannot proceed." % name) _dd(file_path, boot_device_or_file, seek=boot_bin['seek']) # This dictionary is composed as follows: # : # The is the command line argument passed to l-a-m-c, the # is the corresponding config class in this file (not the instance). # If a new device does not have special needs, it is possible to use the # general AndroidBoardConfig class. android_board_configs = { 'arndale': AndroidArndaleConfig, 'arndale_octa': AndroidArndaleOctaConfig, 'beagle': AndroidBeagleConfig, 'iMX53': AndroidMx53LoCoConfig, 'mx53loco': AndroidMx53LoCoConfig, 'mx6qsabrelite': AndroidMx6QSabreliteConfig, 'origen': AndroidOrigenConfig, 'origen_quad': AndroidOrigenQuadConfig, 'panda': AndroidPandaConfig, 'smdkv310': AndroidSMDKV310Config, 'snowball_emmc': AndroidSnowballEmmcConfig, 'snowball_sd': AndroidSnowballSdConfig, 'vexpress': AndroidVexpressConfig, 'vexpress-a9': AndroidVexpressConfig, } def get_board_config(board): """Get the board configuration for the specified board. :param board: The name of the board to get the configuration of. :type board: str """ clazz = android_board_configs.get(board, None) if clazz: return clazz() else: raise BoardConfigException("Board name '%s' has no configuration " "available." % board) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/0000755000175000017500000000000012724020110023034 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/hardwarepack.py0000644000175000017500000006045212724020110026051 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import time import os import urlparse from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile from linaro_image_tools.hwpack.packages import ( FetchedPackage, get_packages_file, PackageMaker, ) from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, ) from linaro_image_tools.hwpack.hwpack_convert import ( dump, ) from hwpack_fields import ( BOARDS_FIELD, BOOTLOADERS_FIELD, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, DTB_ADDR_FIELD, DTB_FILE_FIELD, DTB_FILES_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, FORMAT_FIELD, INITRD_ADDR_FIELD, INITRD_FILE_FIELD, KERNEL_ADDR_FIELD, KERNEL_FILE_FIELD, LOAD_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, MAINTAINER_FIELD, METADATA_ARCH_FIELD, METADATA_VERSION_FIELD, MMC_ID_FIELD, NAME_FIELD, ORIGIN_FIELD, PARTITION_LAYOUT_FIELD, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SUPPORT_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, ) class Metadata(object): """Metadata for a hardware pack. This metadata is what is required and optional for the metadata file that ends up in the hardware pack. In addition str() can be used to create the contents of the metdata file for the hardware pack that this metadata file refers to. :ivar name: the name of the hardware pack. :type name: str :ivar version: the version of the hardware pack. It must not contain white spaces. :type version: str :ivar origin: the origin of the hardware pack, or None if the origin is not known. :type origin: str or None :ivar maintainer: the maintainer of the hardware pack, or None if not known. :type maintainer: str or None :ivar support: the support status of this hardware pack, or None if not know. :type support: str or None """ def __init__(self, name, version, architecture, origin=None, maintainer=None, support=None, format=HardwarePackFormatV1()): """Create the Metadata for a hardware pack. See the instance variables for a description of the arguments. """ self.format = format self.name = name if ' ' in version: raise AssertionError( 'Hardware pack version must not contain white ' 'spaces: "%s"' % version) self.version = version self.origin = origin self.maintainer = maintainer self.support = support self.architecture = architecture @classmethod def add_v2_config(self, serial_tty=None, kernel_addr=None, initrd_addr=None, load_addr=None, dtb_file=None, wired_interfaces=[], wireless_interfaces=[], partition_layout=None, mmc_id=None, boot_min_size=None, root_min_size=None, loader_min_size=None, vmlinuz=None, initrd=None, dtb_addr=None, extra_boot_options=None, env_dd=None, boot_script=None, bootloader_file_in_boot_part=None, uboot_dd=None, spl_in_boot_part=None, spl_dd=None, extra_serial_options=None, loader_start=None, snowball_startup_files_config=None, samsung_bl1_start=None, samsung_bl1_len=None, samsung_env_start=None, samsung_env_len=None, samsung_bl2_start=None, samsung_bl2_len=None): """Add fields that are specific to the new format. These fields are not present in earlier config files. """ self.u_boot = None self.spl = None self.serial_tty = serial_tty self.kernel_addr = kernel_addr self.initrd_addr = initrd_addr self.load_addr = load_addr self.wired_interfaces = wired_interfaces self.wireless_interfaces = wireless_interfaces self.partition_layout = partition_layout self.mmc_id = mmc_id self.boot_min_size = boot_min_size self.root_min_size = root_min_size self.loader_min_size = loader_min_size self.loader_start = loader_start self.vmlinuz = vmlinuz self.initrd = initrd self.dtb_file = dtb_file self.dtb_addr = dtb_addr self.extra_boot_options = extra_boot_options self.boot_script = boot_script self.bootloader_file_in_boot_part = bootloader_file_in_boot_part self.uboot_dd = uboot_dd self.spl_in_boot_part = spl_in_boot_part self.spl_dd = spl_dd self.env_dd = env_dd self.extra_serial_options = extra_serial_options self.snowball_startup_files_config = snowball_startup_files_config self.samsung_bl1_start = samsung_bl1_start self.samsung_bl1_len = samsung_bl1_len self.samsung_env_start = samsung_env_start self.samsung_env_len = samsung_env_len self.samsung_bl2_start = samsung_bl2_start self.samsung_bl2_len = samsung_bl2_len @classmethod def add_v3_config(self, boards=None, bootloaders=None, dtb_files=None): """Add fields that are specific to the v3 config format. These fields are not present in the earlier config files. :param boards: The boards section of the hwpack. :param bootloaders: The bootloaders section of the hwpack. :param dtb_files: The dtb_files section of the hwpack.""" self.boards = boards self.bootloaders = bootloaders self.dtb_files = dtb_files @classmethod def from_config(cls, config, version, architecture): """Create a Metadata from a Config object. As a Config will contain most of the information needed for a Metadata, we can provide this convenient way to construct one. Information that is not in the config has to be provided by the caller. :param config: the config to take values from. :type config: Config :param version: the version to record in the metadata. :type version: str :param architecture: the architecture that the hardware pack is targetting. :type architecture: str """ metadata = cls( config.name, version, architecture, origin=config.origin, maintainer=config.maintainer, support=config.support, format=config.format) if config.format.has_v2_fields: # Helper variable to adhere to the line length limit. snowball_startup_config = config.snowball_startup_files_config metadata.add_v2_config( boot_min_size=config.boot_min_size, boot_script=config.boot_script, dtb_addr=config.dtb_addr, dtb_file=config.dtb_file, env_dd=config.env_dd, extra_boot_options=config.extra_boot_options, extra_serial_options=config.extra_serial_options, initrd_addr=config.initrd_addr, initrd=config.initrd, kernel_addr=config.kernel_addr, load_addr=config.load_addr, loader_min_size=config.loader_min_size, loader_start=config.loader_start, mmc_id=config.mmc_id, partition_layout=config.partition_layout, root_min_size=config.root_min_size, samsung_bl1_len=config.samsung_bl1_len, samsung_bl1_start=config.samsung_bl1_start, samsung_bl2_len=config.samsung_bl2_len, samsung_bl2_start=config.samsung_bl2_start, samsung_env_len=config.samsung_env_len, samsung_env_start=config.samsung_env_start, serial_tty=config.serial_tty, snowball_startup_files_config=snowball_startup_config, spl_dd=config.spl_dd, spl_in_boot_part=config.spl_in_boot_part, uboot_dd=config.bootloader_dd, bootloader_file_in_boot_part=config. bootloader_file_in_boot_part, vmlinuz=config.vmlinuz, wired_interfaces=config.wired_interfaces, wireless_interfaces=config.wireless_interfaces, ) if config.format.format_as_string == '3.0': metadata.add_v3_config(boards=config.boards, bootloaders=config.bootloaders, dtb_files=config.dtb_files) return metadata def __str__(self): if self.format.format_as_string == '3.0': return self.create_metadata_new() else: return self.create_metadata_old() def create_metadata_new(self): """Get the contents of the metadata file. The metadata file is almost an identical copy of the hwpack configuration file. Only a couple of fields are different, and some are missing. :return A string. """ metadata = "" metadata += dump({FORMAT_FIELD: self.format.format_as_string}) metadata += dump({NAME_FIELD: self.name}) metadata += dump({METADATA_VERSION_FIELD: self.version}) # This is a single 'architecture' hwpack, each arch will get its own, # it is not retrieved from the Config. metadata += dump({METADATA_ARCH_FIELD: self.architecture}) if self.origin is not None: metadata += dump({ORIGIN_FIELD: self.origin}) if self.maintainer is not None: metadata += dump({MAINTAINER_FIELD: self.maintainer}) if self.support is not None: metadata += dump({SUPPORT_FIELD: self.support}) if self.boards is not None: metadata += dump({BOARDS_FIELD: self.boards}) if self.bootloaders is not None: metadata += dump({BOOTLOADERS_FIELD: self.bootloaders}) if self.serial_tty is not None: metadata += dump({SERIAL_TTY_FIELD: self.serial_tty}) if self.kernel_addr is not None: metadata += dump({KERNEL_ADDR_FIELD: self.kernel_addr}) if self.initrd_addr is not None: metadata += dump({INITRD_ADDR_FIELD: self.initrd_addr}) if self.load_addr is not None: metadata += dump({LOAD_ADDR_FIELD: self.load_addr}) if self.dtb_addr is not None: metadata += dump({DTB_ADDR_FIELD: self.dtb_addr}) if self.wired_interfaces != []: eth_interfaces = " ".join(self.wired_interfaces) metadata += dump({WIRED_INTERFACES_FIELD: eth_interfaces}) if self.wireless_interfaces != []: wifi_interfaces = " ".join(self.wireless_interfaces) metadata += dump({WIRELESS_INTERFACES_FIELD: wifi_interfaces}) if self.partition_layout is not None: metadata += dump({PARTITION_LAYOUT_FIELD: self.partition_layout}) if self.mmc_id is not None: metadata += dump({MMC_ID_FIELD: self.mmc_id}) if self.boot_min_size is not None: metadata += dump({BOOT_MIN_SIZE_FIELD: self.boot_min_size}) if self.root_min_size is not None: metadata += dump({ROOT_MIN_SIZE_FIELD: self.root_min_size}) if self.loader_min_size is not None: metadata += dump({LOADER_MIN_SIZE_FIELD: self.loader_min_size}) if self.loader_start is not None: metadata += dump({LOADER_START_FIELD: self.loader_start}) if self.vmlinuz is not None: metadata += dump({KERNEL_FILE_FIELD: self.vmlinuz}) if self.initrd is not None: metadata += dump({INITRD_FILE_FIELD: self.initrd}) if self.dtb_file is not None: # XXX In V3 this one should be a list, called dtb_files. metadata += dump({DTB_FILE_FIELD: self.dtb_file}) if self.dtb_files is not None: metadata += dump({DTB_FILES_FIELD: self.dtb_files}) if self.boot_script is not None: metadata += dump({BOOT_SCRIPT_FIELD: self.boot_script}) if self.extra_serial_options is not None: # XXX Check why and where once we get a list once a string. if isinstance(self.extra_serial_options, list): extra_serial_options = " ".join(self.extra_serial_options) else: extra_serial_options = self.extra_serial_options metadata += dump({ EXTRA_SERIAL_OPTIONS_FIELD: extra_serial_options}) if self.snowball_startup_files_config is not None: metadata += dump({SNOWBALL_STARTUP_FILES_CONFIG_FIELD: self.snowball_startup_files_config}) if self.samsung_bl1_start is not None: metadata += dump({SAMSUNG_BL1_START_FIELD: self.samsung_bl1_start}) if self.samsung_bl1_len is not None: metadata += dump({SAMSUNG_BL1_LEN_FIELD: self.samsung_bl1_len}) if self.samsung_env_start is not None: metadata += dump({SAMSUNG_ENV_START_FIELD: self.samsung_env_start}) if self.samsung_env_len is not None: metadata += dump({SAMSUNG_ENV_LEN_FIELD: self.samsung_env_len}) if self.samsung_bl2_start is not None: metadata += dump({SAMSUNG_BL2_START_FIELD: self.samsung_bl2_start}) if self.samsung_bl2_len is not None: metadata += dump({SAMSUNG_BL2_LEN_FIELD: self.samsung_bl2_len}) return metadata def create_metadata_old(self): """Get the contents of the metadata file. Creates a metadata file for v1 and v2 of the hwpack config file. """ metadata = "NAME=%s\n" % self.name metadata += "VERSION=%s\n" % self.version metadata += "ARCHITECTURE=%s\n" % self.architecture if self.origin is not None: metadata += "ORIGIN=%s\n" % self.origin if self.maintainer is not None: metadata += "MAINTAINER=%s\n" % self.maintainer if self.support is not None: metadata += "SUPPORT=%s\n" % self.support if not self.format.has_v2_fields: return metadata if self.u_boot is not None: metadata += "U_BOOT=%s\n" % self.u_boot if self.spl is not None: metadata += "SPL=%s\n" % self.spl if self.serial_tty is not None: metadata += "SERIAL_TTY=%s\n" % self.serial_tty if self.kernel_addr is not None: metadata += "KERNEL_ADDR=%s\n" % self.kernel_addr if self.initrd_addr is not None: metadata += "INITRD_ADDR=%s\n" % self.initrd_addr if self.load_addr is not None: metadata += "LOAD_ADDR=%s\n" % self.load_addr if self.dtb_addr is not None: metadata += "DTB_ADDR=%s\n" % self.dtb_addr if self.wired_interfaces != []: metadata += "WIRED_INTERFACES=%s\n" % " ".join( self.wired_interfaces) if self.wireless_interfaces != []: metadata += "WIRELESS_INTERFACES=%s\n" % " ".join( self.wireless_interfaces) if self.partition_layout is not None: metadata += "PARTITION_LAYOUT=%s\n" % self.partition_layout if self.mmc_id is not None: metadata += "MMC_ID=%s\n" % self.mmc_id if self.boot_min_size is not None: metadata += "BOOT_MIN_SIZE=%s\n" % self.boot_min_size if self.root_min_size is not None: metadata += "ROOT_MIN_SIZE=%s\n" % self.root_min_size if self.loader_min_size is not None: metadata += "LOADER_MIN_SIZE=%s\n" % self.loader_min_size if self.loader_start is not None: metadata += "LOADER_START=%s\n" % self.loader_start if self.vmlinuz is not None: metadata += "KERNEL_FILE=%s\n" % self.vmlinuz if self.initrd is not None: metadata += "INITRD_FILE=%s\n" % self.initrd if self.dtb_file is not None: metadata += "DTB_FILE=%s\n" % self.dtb_file if self.extra_boot_options is not None: metadata += "EXTRA_BOOT_OPTIONS=%s\n" % self.extra_boot_options if self.boot_script is not None: metadata += "BOOT_SCRIPT=%s\n" % self.boot_script if self.bootloader_file_in_boot_part is not None: metadata += ("U_BOOT_IN_BOOT_PART=%s\n" % self.bootloader_file_in_boot_part) if self.spl_in_boot_part is not None: metadata += "SPL_IN_BOOT_PART=%s\n" % self.spl_in_boot_part if self.uboot_dd is not None: metadata += "U_BOOT_DD=%s\n" % self.uboot_dd if self.spl_dd is not None: metadata += "SPL_DD=%s\n" % self.spl_dd if self.env_dd is not None: metadata += "ENV_DD=%s\n" % self.env_dd if self.extra_serial_options is not None: metadata += "EXTRA_SERIAL_OPTIONS=%s\n" % self.extra_serial_options if self.snowball_startup_files_config is not None: metadata += "SNOWBALL_STARTUP_FILES_CONFIG=%s\n" % ( self.snowball_startup_files_config) if self.samsung_bl1_start is not None: metadata += "SAMSUNG_BL1_START=%s\n" % self.samsung_bl1_start if self.samsung_bl1_len is not None: metadata += "SAMSUNG_BL1_LEN=%s\n" % self.samsung_bl1_len if self.samsung_env_start is not None: metadata += "SAMSUNG_ENV_START=%s\n" % self.samsung_env_start if self.samsung_env_len is not None: metadata += "SAMSUNG_ENV_LEN=%s\n" % self.samsung_env_len if self.samsung_bl2_start is not None: metadata += "SAMSUNG_BL2_START=%s\n" % self.samsung_bl2_start if self.samsung_bl2_len is not None: metadata += "SAMSUNG_BL2_LEN=%s\n" % self.samsung_bl2_len return metadata class HardwarePack(object): """The representation of a hardware pack. :ivar metadata: the metadata of this hardware pack. :type metadata: Metadata :ivar FORMAT: the format of hardware pack that should be created. :type FORMAT: str """ FORMAT_FILENAME = "FORMAT" METADATA_FILENAME = "metadata" MANIFEST_FILENAME = "manifest" PACKAGES_DIRNAME = "pkgs" PACKAGES_FILENAME = "%s/Packages" % PACKAGES_DIRNAME SOURCES_LIST_DIRNAME = "sources.list.d" SOURCES_LIST_GPG_DIRNAME = "sources.list.d.gpg" U_BOOT_DIR = "u-boot" SPL_DIR = "spl" BOOT_DIR = "boot" def __init__(self, metadata): """Create a HardwarePack. :param metadata: the metadata to use. :type metadata: Metadata """ self.metadata = metadata self.sources = {} self.packages = [] self.format = metadata.format self.files = [] def filename(self, extension=".tar.gz"): """The filename that this hardware pack should have. Returns the filename that the hardware pack should have, according to the convention used. :return: the filename that should be used. :rtype: str """ if self.metadata.support is None: support_suffix = "" else: support_suffix = "_%s" % self.metadata.support return "hwpack_%s_%s_%s%s%s" % ( self.metadata.name, self.metadata.version, self.metadata.architecture, support_suffix, extension) def add_apt_sources(self, sources): """Add APT sources to the hardware pack. Given a dict of names and the source lines this will add them to the hardware pack. The names should be an identifier for the source, and the source lines should be what is put in sources.list for that source, minus the "deb" part. If you pass an identifier that has already been passed to this method, then the previous value will be replaced with the new value. :param sources: the sources to use as a dict mapping identifiers to sources entries. :type sources: a dict mapping str to str """ if sources: self.sources.update(sources) def add_packages(self, packages): """Add packages to the hardware pack. Given a list of packages this will add them to the hardware pack. :param packages: the packages to add :type packages: FetchedPackage """ self.packages += packages def add_dependency_package(self, packages_spec): """Add a packge that depends on packages_spec to the hardware pack. :param packages_spec: A list of apt package specifications, e.g. ``['foo', 'bar (>= 1.2)']``. """ with PackageMaker() as maker: dep_package_name = 'hwpack-' + self.metadata.name relationships = {} if packages_spec: relationships = {'Depends': ', '.join(packages_spec)} deb_file_path = maker.make_package( dep_package_name, self.metadata.version, relationships, self.metadata.architecture) self.packages.append(FetchedPackage.from_deb(deb_file_path)) def add_file(self, dir, file): target_file = os.path.join(dir, os.path.basename(file)) self.files.append((file, target_file)) return target_file def manifest_text(self): manifest_content = "" for package in self.packages: manifest_content += "%s=%s\n" % ( package.name, package.version) return manifest_content def to_file(self, fileobj): """Write the hwpack to a file object. The full hardware pack will be written to the file object in gzip compressed tarball form as the spec requires. :param fileobj: the file object to write to. :type fileobj: a file-like object :return: None """ kwargs = {} kwargs["default_uid"] = 1000 kwargs["default_gid"] = 1000 kwargs["default_uname"] = "user" kwargs["default_gname"] = "group" kwargs["default_mtime"] = time.time() with writeable_tarfile(fileobj, mode="w:gz", **kwargs) as tf: tf.create_file_from_string( self.FORMAT_FILENAME, "%s\n" % self.format) tf.create_file_from_string( self.METADATA_FILENAME, str(self.metadata)) for fs_file_name, arc_file_name in self.files: tf.add(fs_file_name, arcname=arc_file_name) tf.create_dir(self.PACKAGES_DIRNAME) for package in self.packages: if package.content is not None: tf.create_file_from_string( self.PACKAGES_DIRNAME + "/" + package.filename, package.content.read()) tf.create_file_from_string( self.MANIFEST_FILENAME, self.manifest_text()) tf.create_file_from_string( self.PACKAGES_FILENAME, get_packages_file( [p for p in self.packages if p.content is not None])) tf.create_dir(self.SOURCES_LIST_DIRNAME) for source_name, source_info in self.sources.items(): url_parsed = urlparse.urlsplit(source_info) # Don't output sources with passwords in them if not url_parsed.password: tf.create_file_from_string( (self.SOURCES_LIST_DIRNAME + "/" + source_name + ".list"), "deb " + source_info + "\n") # TODO: include sources keys etc. tf.create_dir(self.SOURCES_LIST_GPG_DIRNAME) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/hwpack_reader.py0000644000175000017500000001531112724020110026206 0ustar voipiovoipio# Copyright (C) 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from linaro_image_tools.hwpack.handler import HardwarepackHandler from linaro_image_tools.hwpack.hwpack_fields import ( FORMAT_FIELD, NAME_FIELD, BOARDS_FIELD, BOOTLOADERS_FIELD, ) from os import linesep as LINE_SEP # Fields necessary for the string representation of the hardware pack supported # boards and bootlaoders. HALF_SEPARATOR = '+--------------------------------------' ENDING = '+' SEPARATOR = HALF_SEPARATOR * 2 + ENDING FORMAT = '{:<80}' CENTER_ALIGN = '{:^80}' ELEMENT_FORMAT = '{:<39}| {:<39}' class HwpackReaderError(Exception): """General error raised by HwpackReader.""" def __init__(self, value): self.value = value def __str__(self): return repr(self.value) class Hwpack(object): """A simple representation of an hardware pack and its value.""" def __init__(self): self._hwpack = None self._boards = None self._bootloaders = None self._name = None @property def hwpack(self): """The hardware pack it refers to.""" return self._hwpack def sethwpack(self, value): """Sets the hwpack field.""" self._hwpack = value @property def boards(self): """The boards field of this hardware pack.""" return self._boards def setboards(self, value): """Sets the boards field.""" self._boards = value @property def name(self): """The name field of this hardware pack.""" return self._name def setname(self, value): """Sets the name field.""" self._name = value @property def bootloaders(self): """The bootlaoders field of this hardware pack.""" return self._bootloaders def setbootloaders(self, value): """Sets the bootlaoders field.""" self._bootloaders = value def __eq__(self, other): """Equality method.""" equal = False if isinstance(other, Hwpack): equal = (self.name == other.name and self.boards == other.boards and self.hwpack == other.hwpack and self.bootloaders == other.bootloaders) return equal def __hash__(self): return hash(frozenset(self.bootloaders), frozenset(self.boards), self.name, self.hwpack) def __str__(self): """String representation of this hwapack supported elements.""" string = FORMAT.format("Read hardware pack: %s" % self.hwpack) string += LINE_SEP string += FORMAT.format(SEPARATOR) string += LINE_SEP string += ELEMENT_FORMAT.format("Supported boards", "Supported bootloaders") string += LINE_SEP string += FORMAT.format(SEPARATOR) string += LINE_SEP if self.boards: for key, value in self.boards.iteritems(): if value.get(BOOTLOADERS_FIELD, None) is not None: bootloaders = value.get(BOOTLOADERS_FIELD) supported_bootloaders = bootloaders.keys() else: supported_bootloaders = self.bootloaders.keys() string += ELEMENT_FORMAT.format( key, ",".join(supported_bootloaders)) string += LINE_SEP else: # If we pass a converted file with just a single board, we do not # have the boards section, and we default to the name of the hwpack if self.bootloaders: supported_bootloaders = self.bootloaders.keys() string += ELEMENT_FORMAT.format( self.name, ",".join(supported_bootloaders)) string += LINE_SEP else: string += CENTER_ALIGN.format("No supported boards and " "bootloaders") string += LINE_SEP string += FORMAT.format(SEPARATOR) return string + LINE_SEP class HwpackReader(object): """Reads the information contained in a hwpack """ def __init__(self, hwpacks): """Create a new instance. :param hwpacks: The list of hardware packs to read from.""" self.hwpacks = hwpacks # Where we store all the info from the hwpack. self._supported_elements = [] @property def supported_elements(self): """Gets the supported elements of by all the hardwapare packs.""" return self._supported_elements def _read_hwpacks_metadata(self): """Reads the hardware pack metadata file, and prints information about the supported boards and bootloaders.""" for tarball in self.hwpacks: with HardwarepackHandler([tarball]) as handler: hwpack_format = handler.get_field(FORMAT_FIELD)[0] if hwpack_format.format_as_string == "3.0": local_hwpack = Hwpack() local_hwpack.sethwpack(tarball) local_hwpack.setname(handler.get_field(NAME_FIELD)[0]) local_hwpack.setboards(handler.get_field(BOARDS_FIELD)[0]) local_hwpack.setbootloaders( handler.get_field(BOOTLOADERS_FIELD)[0]) self.supported_elements.append(local_hwpack) else: raise HwpackReaderError("Hardwarepack '%s' cannot be " "read, unsupported format." % (tarball)) def get_supported_boards(self): """Prints the necessary information. :return A string representation of the information.""" self._read_hwpacks_metadata() return str(self) def __str__(self): """The string representation of this reader. It is a printable representation of the necessary information.""" hwpack_reader = "" for element in self.supported_elements: hwpack_reader += str(element) return hwpack_reader linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/better_tarfile.py0000644000175000017500000001115212724020110026401 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager from StringIO import StringIO from tarfile import DIRTYPE, TarFile as StandardTarFile, TarInfo """Improvements to the standard library's tarfile module. In particular this module provides a tarfile.TarFile subclass that aids in adding paths to the tarfile that aren't present on the filesystem, with the ability to specify file content as strings, and provide default values for the mtime, uid, etc. of the created paths. """ @contextmanager def writeable_tarfile(backing_file, mode="w", **kwargs): """A context manager to get a writeable better tarfile. :param backing_file: a file object to write the tarfile contents to. :param mode: the mode to open the tarfile with. Default is "w". :param kwargs: other keyword arguments to pass to the TarFile constructor. """ tf = TarFile.open(mode=mode, fileobj=backing_file, **kwargs) try: yield tf finally: tf.close() class TarFile(StandardTarFile): """An improvement to tarfile that can add paths not on the filesystem. With the standard tarfile implementation adding paths that are not present on the filesystem is convoluted. This subclass adds methods to create paths in the tarfile that are not present on the filesystem. In addition, it can take constructor parameters to set the defaults of various attributes of the paths that it adds. """ def __init__(self, *args, **kwargs): """Create a TarFile. :param default_mtime: the default mtime to create paths with, an int or None to use the stdlib default. :param default_uid: the default user id to set as the owner of created paths, an int or None to use the stdlib default. :param default_gid: the default group id to set as the owner of created paths, an int or None to use the stdlib default. :param default_uname: the default user name to set as the owner of created paths, a string, or None to use the stdlib default. :param default_gname: the default group name ot set as the owner of created paths, a string, or None to use the stdlib default. """ self.default_mtime = kwargs.pop("default_mtime", None) self.default_uid = kwargs.pop("default_uid", None) self.default_gid = kwargs.pop("default_gid", None) self.default_uname = kwargs.pop("default_uname", None) self.default_gname = kwargs.pop("default_gname", None) super(TarFile, self).__init__(*args, **kwargs) def _set_defaults(self, tarinfo): if self.default_mtime is not None: tarinfo.mtime = self.default_mtime if self.default_uid is not None: tarinfo.uid = self.default_uid if self.default_gid is not None: tarinfo.gid = self.default_gid if self.default_uname is not None: tarinfo.uname = self.default_uname if self.default_gname is not None: tarinfo.gname = self.default_gname def create_file_from_string(self, filename, content): """Create a file with the contents passed as a string. :param filename: the path to put the file at inside the tarfile. :param content: the content to put in the created file. """ tarinfo = TarInfo(name=filename) tarinfo.size = len(content) self._set_defaults(tarinfo) fileobj = StringIO(content) self.addfile(tarinfo, fileobj=fileobj) def create_dir(self, path): """Create a directory within the tarfile. :param path: the path to put the directory at. """ tarinfo = TarInfo(name=path) tarinfo.type = DIRTYPE tarinfo.mode = 0755 self._set_defaults(tarinfo) self.addfile(tarinfo) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/packages.py0000644000175000017500000007621612724020110025200 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import hashlib import logging import os import re import shutil from string import Template import subprocess import tempfile import urlparse from apt.cache import Cache from apt.cache import FetchFailedException from apt.package import FetchError import apt_pkg from debian.debfile import DebFile from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) def get_packages_file(packages, extra_text=None, rel_to=None): """Get the Packages file contents indexing `packages`. :param packages: the packages to index. :type packages: an iterable of FetchedPackages. :param extra_text: extra text to insert in to each stanza. Should not end with a newline. :type extra_text: str or None :param rel_to: If present, generate the Filename: parts of the Packages file as paths relative to this location. If not present, Filename: will just include the file name (not the path). :return: the Packages file contents indexing `packages`. :rtype: str """ content = "" for package in packages: parts = [] parts.append('Package: %s' % package.name) if extra_text is not None: parts.append(extra_text) parts.append('Version: %s' % package.version) if rel_to is not None: filename = os.path.relpath(package.filepath, rel_to) else: filename = package.filename parts.append('Filename: %s' % filename) parts.append('Size: %d' % package.size) parts.append('Architecture: %s' % package.architecture) if package.depends: parts.append('Depends: %s' % package.depends) if package.pre_depends: parts.append('Pre-Depends: %s' % package.pre_depends) if package.multi_arch: parts.append('Multi-Arch: %s' % package.multi_arch) if package.conflicts: parts.append('Conflicts: %s' % package.conflicts) if package.recommends: parts.append('Recommends: %s' % package.recommends) if package.provides: parts.append('Provides: %s' % package.provides) if package.replaces: parts.append('Replaces: %s' % package.replaces) if package.breaks: parts.append('Breaks: %s' % package.breaks) parts.append('MD5sum: %s' % package.md5) parts.append('SHA256: %s' % package.sha256) content += "\n".join(parts) content += "\n\n" return content def stringify_relationship(pkg, relationship): """Given a Package, return a string of the specified relationship. apt.package.Version stores the relationship information of the package as objects. This function will convert those objects in to the string form that we are used to from debian/control or Packages files. :param pkg: the package to take the relationship information from. :type pkg: apt.package.Version :param relationship: the relationship to stringify, as understood by apt.package.Package.get_dependencies, e.g. "Depends", "PreDepends". :type relationship: str or None if the package has no relationships of that type. """ relationship_str = None pkg_dependencies = pkg.get_dependencies(relationship) if pkg_dependencies: relationship_list = [] for or_dep in pkg_dependencies: or_list = [] for or_alternative in or_dep.or_dependencies: suffix = "" if or_alternative.relation: relation = or_alternative.relation if relation in ('<', '>'): # The choice made here by python-apt is to report the # relationship in a Python spelling; as far as apt # knows, < is a deprecated spelling of <=; << is the # spelling of "strictly less than". Similarly for >. relation *= 2 suffix = " (%s %s)" % (relation, or_alternative.version) or_list.append("%s%s" % (or_alternative.name, suffix)) relationship_list.append(" | ".join(or_list)) relationship_str = ", ".join(relationship_list) return relationship_str class DummyProgress(object): """An AcquireProgress that silences all output. This can be used to ensure that apt produces no output when fetching files. """ def start(self): pass def ims_hit(self, item): pass def fail(self, item): pass def fetch(self, item): pass def pulse(self, owner): return True def media_change(self): return False def stop(self): pass class TemporaryDirectoryManager(object): def __init__(self): self._temporary_directories = None def __enter__(self): if self._temporary_directories is not None: raise AssertionError("__enter__ must not be called twice") self._temporary_directories = [] return self def __exit__(self, exc_type=None, exc_value=None, traceback=None): if self._temporary_directories is None: return for tmpdir in self._temporary_directories: shutil.rmtree(tmpdir) self._temporary_directories = None return False def make_temporary_directory(self): """Create a temporary directory and return its path. The created directory will be deleted on __exit__. """ if self._temporary_directories is None: raise AssertionError("__enter__ must be called") tmpdir = tempfile.mkdtemp() self._temporary_directories.append(tmpdir) return tmpdir class LocalArchiveMaker(TemporaryDirectoryManager): def sources_entry_for_debs(self, local_debs, label=None): tmpdir = self.make_temporary_directory() with open(os.path.join(tmpdir, 'Packages'), 'w') as packages_file: packages_file.write(get_packages_file(local_debs, rel_to=tmpdir)) if label: cmd_runner.run( ['apt-ftparchive', '-oAPT::FTPArchive::Release::Label=%s' % label, 'release', tmpdir], stdout=open(os.path.join(tmpdir, 'Release'), 'w')).wait() return 'file://%s ./' % (tmpdir, ) class PackageMaker(TemporaryDirectoryManager): """An object that can create binary debs on the fly. PackageMakers implement the context manager protocol to manage the temporary directories the debs are created in. """ # This template (and the code that uses it) is made more awkward by the # fact that blank lines are invalid in control files -- so in particular # when there are no relationships, there must be no blank line between the # Maintainer and the Description. control_file_template = Template('''\ Package: ${name} Version: ${version} Architecture: ${architecture} Maintainer: Nobody ${relationships}\ Description: Dummy package to install a hwpack This package was created automatically by linaro-media-create ''') def make_package(self, name, version, relationships, architecture='all', files=[]): tmp_dir = self.make_temporary_directory() filename = '%s_%s_%s' % (name, version, architecture) packaging_dir = os.path.join(tmp_dir, filename) os.mkdir(packaging_dir) os.mkdir(os.path.join(packaging_dir, 'DEBIAN')) relationship_strs = [] for relationship_name, relationship_value in relationships.items(): relationship_strs.append( '%s: %s\n' % (relationship_name, relationship_value)) subst_vars = dict( architecture=architecture, name=name, relationships=''.join(relationship_strs), version=version, ) control_file_text = self.control_file_template.safe_substitute( subst_vars) # If any files have been specified, create them for file_path in files: os.makedirs(os.path.join(packaging_dir, os.path.dirname(file_path))) with open(os.path.join(packaging_dir, file_path), 'w') as new_file: new_file.write(name + " " + file_path) with open(os.path.join( packaging_dir, 'DEBIAN', 'control'), 'w') as control_file: control_file.write(control_file_text) env = os.environ env['LC_ALL'] = 'C' env['NO_PKG_MANGLE'] = '1' proc = cmd_runner.Popen( ['dpkg-deb', '-b', '-Zgzip', packaging_dir], env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = proc.communicate() if proc.returncode: raise ValueError("dpkg-deb failed!\n%s" % stderrdata) if stderrdata: raise ValueError("dpkg-deb had warnings:\n%s" % stderrdata) deb_file_path_match = re.match( "dpkg-deb: building package `.*' in `(.*)'", stdoutdata) if not deb_file_path_match: raise ValueError( "failed to find filename in dpkg-deb output:\n%s" % stdoutdata) return deb_file_path_match.group(1) class FetchedPackage(object): """The result of fetching packages. :ivar name: the name of the fetched package. :type name: str :ivar version: the version of the fetched package. :type version: str :ivar filename: the filename that the package has. :type filename: str :ivar content: a file that the content of the package can be read from, or None if the content is not known. :type content: a file-like object or None :ivar size: the size of the package :type size: int :ivar md5: the hex representation of the md5sum of the contents of the package. :type md5: str :ivar sha256: the hex representation of the sha256sum of the contents of the package. :type sha256: str :ivar architecture: the architecture that the package is for, may be 'all'. :type architecture: str :ivar depends: the Depends string that the package has, i.e. the dependencies as specified in debian/control. May be None if the package has none. :type depends: str or None :ivar pre_depends: the Pre-Depends string that the package has, i.e. the pre-dependencies as specified in debian/control. May be None if the package has none. :type pre_depends: str or None :ivar multi_arch: the Multi-Arch string that the package has. May be None if the package has none. :type multi_arch: str or None :ivar conflicts: the Conflicts string that the package has, i.e. the conflicts as specified in debian/control. May be None if the package has none. :type conflicts: str or None :ivar recommends: the Recommends string that the package has, i.e. the recommends as specified in debian/control. May be None if the package has none. :type recommends: str or None :ivar provides: the Provides string that the package has, i.e. the provides as specified in debian/control. May be None if the package has none. :type provides: str or None :ivar replaces: the Replaces string that the package has, i.e. the replaces as specified in debian/control. May be None if the package has none. :type replaces: str or None :ivar breaks: the Breaks string that the package has, i.e. the breaks as specified in debian/control. May be None if the package has none. :type breaks: str or None """ def __init__(self, name, version, filename, size, md5, sha256, architecture, depends=None, pre_depends=None, multi_arch=None, conflicts=None, recommends=None, provides=None, replaces=None, breaks=None): """Create a FetchedPackage. See the instance variables for the arguments. """ self.name = name self.version = version self.filename = filename self.size = size self.md5 = md5 self.sha256 = sha256 self.architecture = architecture self.depends = depends self.pre_depends = pre_depends self.multi_arch = multi_arch self.conflicts = conflicts self.recommends = recommends self.provides = provides self.replaces = replaces self.breaks = breaks self.content = None self._file_path = None @property def filepath(self): if self._file_path is not None: return self._file_path else: return self.filename @classmethod def from_apt(cls, pkg, filename, content=None): """Create a FetchedPackage from a python-apt Version (package). This is an alternative constructor for FetchedPackages that takes most of the information from an apt.package.Version object (i.e. a single version of a package), with some additional information supplied by tha caller. :param pkg: the python-apt package to take the information from. :type pkg: apt.package.Version instance :param filename: the filename that the package has. :type filename: str :param content: the content of the package. :type content: file-like object """ depends = stringify_relationship(pkg, "Depends") pre_depends = stringify_relationship(pkg, "PreDepends") multi_arch = pkg.record.get("Multi-Arch") or None conflicts = stringify_relationship(pkg, "Conflicts") recommends = stringify_relationship(pkg, "Recommends") replaces = stringify_relationship(pkg, "Replaces") breaks = stringify_relationship(pkg, "Breaks") provides = ", ".join([a[0] for a in pkg._cand.provides_list]) or None pkg = cls( pkg.package.name, pkg.version, filename, pkg.size, pkg.md5, pkg.sha256, pkg.architecture, depends=depends, pre_depends=pre_depends, multi_arch=multi_arch, conflicts=conflicts, recommends=recommends, provides=provides, replaces=replaces, breaks=breaks) if content is not None: pkg.content = content return pkg @classmethod def from_deb(cls, deb_file_path): """Create a FetchedPackage from a binary package on disk.""" debcontrol = DebFile(deb_file_path).control.debcontrol() name = debcontrol['Package'] version = debcontrol['Version'] filename = os.path.basename(deb_file_path) size = os.path.getsize(deb_file_path) md5sum = hashlib.md5(open(deb_file_path).read()).hexdigest() sha256sum = hashlib.sha256(open(deb_file_path).read()).hexdigest() architecture = debcontrol['Architecture'] depends = debcontrol.get('Depends') pre_depends = debcontrol.get('Pre-Depends') multi_arch = debcontrol.get('Multi-Arch') conflicts = debcontrol.get('Conflicts') recommends = debcontrol.get('Recommends') provides = debcontrol.get('Provides') replaces = debcontrol.get('Replaces') breaks = debcontrol.get('Breaks') pkg = cls( name, version, filename, size, md5sum, sha256sum, architecture, depends, pre_depends, multi_arch, conflicts, recommends, provides, replaces, breaks) pkg.content = open(deb_file_path) pkg._file_path = deb_file_path return pkg # A list of attributes that are compared to determine equality. Note that # we don't include the contents here -- we assume that comparing the md5 # checksum is enough (more philosophically, FetchedPackages are equal if # they represent the same underlying package, even if they represent it in # slightly different ways) _equality_attributes = ( 'name', 'version', 'filename', 'size', 'md5', 'sha256', 'architecture', 'depends', 'pre_depends', 'multi_arch', 'conflicts', 'recommends', 'provides', 'replaces', 'breaks') @property def _equality_data(self): return tuple( getattr(self, attr) for attr in self._equality_attributes) def __eq__(self, other): return self._equality_data == other._equality_data def __ne__(self, other): return not self.__eq__(other) def __hash__(self): return hash(self._equality_data) def __repr__(self): has_content = self.content and "yes" or "no" return ( '<%s name=%s version=%s size=%s md5=%s architecture=%s ' 'depends="%s" pre_depends="%s" multi_arch="%s" conflicts="%s" ' 'recommends="%s" provides="%s" replaces="%s" breaks="%s" ' 'has_content=%s>' % (self.__class__.__name__, self.name, self.version, self.size, self.md5, self.architecture, self.depends, self.pre_depends, self.multi_arch, self.conflicts, self.recommends, self.provides, self.replaces, self.breaks, has_content)) class IsolatedAptCache(object): """A apt.cache.Cache wrapper that isolates it from the system it runs on. :ivar cache: the isolated cache. :type cache: apt.cache.Cache """ def __init__(self, sources, architecture=None, prefer_label=None, backports=False): """Create an IsolatedAptCache. :param sources: a list of sources such that they can be prefixed with "deb " and fed to apt. :type sources: an iterable of str :param architecture: the architecture to fetch packages for. :type architecture: str """ self.sources = sources self.architecture = architecture self.tempdir = None self.prefer_label = prefer_label self.backports = backports def prepare(self): """Prepare the IsolatedAptCache for use. Should be called before use, and after any modification to the list of sources. """ self.cleanup() logger.debug("Writing apt configs") self.tempdir = tempfile.mkdtemp(prefix="hwpack-apt-cache-") dirs = ["var/lib/dpkg", "etc/apt/sources.list.d", "var/cache/apt/archives/partial", "var/lib/apt/lists/partial", ] for d in dirs: os.makedirs(os.path.join(self.tempdir, d)) self.set_installed_packages([], reopen=False) sources_list = os.path.join( self.tempdir, "etc", "apt", "sources.list") with open(sources_list, 'w') as f: for source in self.sources: # To make a file URL look like an HTTP one (for urlparse) # We do this to use urlparse, which is probably more robust # than any regexp we come up with. mangled_source = source if re.search("file:/[^/]", source): mangled_source = re.sub("file:/", "file://", source) url_parsed = urlparse.urlsplit(mangled_source) # If the source uses authentication, don't put in sources.list if url_parsed.password: url_parts_without_user_pass = [url_parsed.scheme, url_parsed.hostname, url_parsed.path, url_parsed.query, url_parsed.fragment] auth_name = os.path.join( self.tempdir, "etc", "apt", "auth.conf") with open(auth_name, 'w') as auth: auth.write( "machine " + url_parsed.hostname + "\n" + "login " + url_parsed.username + "\n" + "password " + url_parsed.password + "\n") source = urlparse.urlunsplit(url_parts_without_user_pass) # Get rid of extra / in file URLs source = re.sub("file://", "file:/", source) f.write("deb %s\n" % source) if self.architecture is not None: apt_conf = os.path.join(self.tempdir, "etc", "apt", "apt.conf") with open(apt_conf, 'w') as f: f.write( 'Apt {\nArchitecture "%s";\n' 'Install-Recommends "true";\n}\n' % self.architecture) apt_preferences = os.path.join( self.tempdir, "etc", "apt", "preferences") if self.backports: with open(apt_preferences, 'w') as f: f.write( 'Package: *\n' 'Pin: release a=*-backports\n' 'Pin-Priority: 500\n\n') if self.prefer_label is not None: with open(apt_preferences, 'a') as f: f.write( 'Package: *\n' 'Pin: release l=%s\n' 'Pin-Priority: 1001\n' % self.prefer_label) # XXX: This is a temporary workaround for bug 885895. apt_pkg.config.set("Dir::bin::dpkg", "/bin/false") self.cache = Cache(rootdir=self.tempdir, memonly=True) logger.debug("Updating apt cache") try: self.cache.update() except FetchFailedException, e: obfuscated_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) raise FetchFailedException(obfuscated_e) self.cache.open() return self def set_installed_packages(self, packages, reopen=True): """Set a list of packages as those installed on the system. This does no installing, just changes dpkg's database to have the tools think the packages are installed. :param packages: a list of packages to "install" on the system, replacing any others. :type packages: an iterable of FetchedPackages. :param reopen: whether to reopen the apt cache after doing the operation. Default is to do so. Note that if it is not done, then the changes will not be visible in the cache until it is reopened. """ with open( os.path.join(self.tempdir, "var/lib/dpkg/status"), "w") as f: f.write( get_packages_file( packages, extra_text="Status: install ok installed")) if reopen: self.cache.open() __enter__ = prepare def cleanup(self): """Cleanup any remaining artefacts. Should be called on all IsolatedAptCache when they are finished with. """ if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) def __exit__(self, exc_type, exc_value, traceback): self.cleanup() return False class DependencyNotSatisfied(Exception): pass class PackageFetcher(object): """A class to fetch packages from a defined list of sources.""" def __init__(self, sources, architecture=None, prefer_label=None, backports=False): """Create a PackageFetcher. Once created a PackageFetcher should have its `prepare` method called before use. :param sources: a list of sources such that they can be prefixed with "deb " and fed to apt. :type sources: an iterable of str :param architecture: the architecture to fetch packages for. :type architecture: str """ self.cache = IsolatedAptCache( sources, architecture=architecture, prefer_label=prefer_label, backports=backports) def prepare(self): """Prepare the PackageFetcher for use. Should be called before use. """ self.cache.prepare() return self __enter__ = prepare def cleanup(self): """Cleanup any remaining artefacts. Should be called on all PackageFetchers when they are finished with. """ self.cache.cleanup() def __exit__(self, exc_type, exc_value, traceback): self.cleanup() return False def ignore_packages(self, packages): """Ignore packages such that they will not be fetched. If a package is ignored then neither it or any of its recursive dependencies will be fetched by fetch_packages. :param packages: the list of package names to ignore. :type packages: an iterable of str """ logger.debug("Ignoring %s" % packages) for package in packages: self.cache.cache[package].mark_install(auto_fix=False) if self.cache.cache.broken_count: raise DependencyNotSatisfied( "Unable to satisfy dependencies of %s" % ", ".join([p.name for p in self.cache.cache if p.is_inst_broken])) installed = [] for package in self.cache.cache.get_changes(): candidate = package.candidate base = os.path.basename(candidate.filename) installed.append(FetchedPackage.from_apt(candidate, base)) for package in self.cache.cache: if not package.is_installed: continue candidate = package.installed base = os.path.basename(candidate.filename) installed.append(FetchedPackage.from_apt(candidate, base)) logger.debug("Ignored %s" % package.name) self.cache.set_installed_packages(installed) broken = [p.name for p in self.cache.cache if p.is_inst_broken or p.is_now_broken] if broken: # If this happens then there is a bug, as we should have # caught this problem earlier raise AssertionError( "Weirdly unable to satisfy dependencies of %s" % ", ".join(broken)) def _filter_ignored(self, package_dict): seen_packages = set() for package in self.cache.cache.get_changes(): if package.name in package_dict: seen_packages.add(package.name) all_packages = set(package_dict.keys()) for unseen_package in all_packages.difference(seen_packages): logger.debug("%s is ignored, skipping" % unseen_package) del package_dict[unseen_package] def fetch_packages(self, packages, download_content=True): """Fetch the files for the given list of package names. The files, and all their dependencies are download, and the metadata and content returned as FetchedPackage objects. If download_content is False then only the metadata is returned (i.e. the FetchedPackages will have None for their content attribute), and only information about the specified packages will be returned, no dependencies. No packages that have been ignored, or are recursive dependencies of ignored packages will be returned. :param packages: a list of package names to install :type packages: an iterable of str :param download_content: whether to download the content of the packages. Default is to do so. :type download_content: bool :return: a list of the packages that were fetched, with relevant metdata and the contents of the files available. :rtype: an iterable of FetchedPackages. :raises KeyError: if any of the package names in the list couldn't be found. """ fetched = {} for package in packages: candidate = self.cache.cache[package].candidate base = os.path.basename(candidate.filename) result_package = FetchedPackage.from_apt(candidate, base) fetched[package] = result_package def check_no_broken_packages(): if self.cache.cache.broken_count: raise DependencyNotSatisfied( "Unable to satisfy dependencies of %s" % ", ".join([p.name for p in self.cache.cache if p.is_inst_broken])) for package in packages: try: self.cache.cache[package].mark_install(auto_fix=True) except SystemError: # Either we raise a DependencyNotSatisfied error # if some packages are broken, or we raise the original # error if there was another cause check_no_broken_packages() raise # Check that nothing was broken, even if mark_install didn't # raise SystemError, just to make sure. check_no_broken_packages() self._filter_ignored(fetched) if not download_content: self.cache.cache.clear() return fetched.values() acq = apt_pkg.Acquire(DummyProgress()) acqfiles = [] # re to remove the repo private key deb_url_auth_re = re.compile( r"(?P.*://)(?P.*):.*@(?P.*$)") for package in self.cache.cache.get_changes(): if (package.marked_delete or package.marked_keep): continue logger.debug("Fetching %s ..." % package) candidate = package.candidate base = os.path.basename(candidate.filename) if package.name not in fetched: result_package = FetchedPackage.from_apt(candidate, base) fetched[package.name] = result_package result_package = fetched[package.name] destfile = os.path.join(self.cache.tempdir, base) acqfile = apt_pkg.AcquireFile( acq, candidate.uri, candidate.md5, candidate.size, base, destfile=destfile) acqfiles.append((acqfile, result_package, destfile)) # check if we have a private key in the pkg url deb_url_auth = deb_url_auth_re.match(acqfile.desc_uri) if deb_url_auth: logger.debug(" ... from %s%s:***@%s" % deb_url_auth.groups()) else: logger.debug(" ... from %s" % acqfile.desc_uri) self.cache.cache.clear() acq.run() for acqfile, result_package, destfile in acqfiles: if acqfile.status != acqfile.STAT_DONE: raise FetchError( "The item %r could not be fetched: %s" % (acqfile.destfile, acqfile.error_text)) result_package.content = open(destfile) result_package._file_path = destfile return fetched.values() linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/hardwarepack_format.py0000644000175000017500000000430412724020110027413 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging logger = logging.getLogger(__name__) class HardwarePackFormat(object): def __init__(self): self.format_as_string = None self.is_deprecated = False self.is_supported = False self.has_v2_fields = False def __str__(self): if self.format_as_string is None: raise NotImplementedError() if self.is_deprecated: logger.warning("The format '%s' is deprecated, please update " "your hardware pack configuration." % self.format_as_string) return self.format_as_string class HardwarePackFormatV1(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV1, self).__init__() self.format_as_string = "1.0" self.is_supported = True self.is_deprecated = True class HardwarePackFormatV2(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV2, self).__init__() self.format_as_string = "2.0" self.is_supported = True self.is_deprecated = False self.has_v2_fields = True class HardwarePackFormatV3(HardwarePackFormat): def __init__(self): super(HardwarePackFormatV3, self).__init__() self.format_as_string = "3.0" self.is_supported = True self.is_deprecated = False self.has_v2_fields = True linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/builder.py0000644000175000017500000004172312724020110025043 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging import errno import subprocess import os import shutil from glob import iglob from debian.debfile import DebFile from debian.arfile import ArError from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.config import Config from linaro_image_tools.hwpack.hardwarepack import HardwarePack, Metadata from linaro_image_tools.hwpack.packages import ( FetchedPackage, LocalArchiveMaker, PackageFetcher, ) from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.hwpack.hwpack_fields import ( PACKAGE_FIELD, SPL_PACKAGE_FIELD, ) # The fields that hold packages to be installed. PACKAGE_FIELDS = [PACKAGE_FIELD, SPL_PACKAGE_FIELD] logger = logging.getLogger(__name__) LOCAL_ARCHIVE_LABEL = 'hwpack-local' class ConfigFileMissing(Exception): def __init__(self, filename): self.filename = filename super(ConfigFileMissing, self).__init__( "No such config file: '%s'" % self.filename) class HardwarePackBuilder(object): def __init__(self, config_path, version, local_debs, backports=False, out_name=None): try: with open(config_path) as fp: self.config = Config(fp, allow_unset_bootloader=True) except IOError, e: if e.errno == errno.ENOENT: raise ConfigFileMissing(config_path) raise self.config.validate() self.format = self.config.format self.version = version self.local_debs = local_debs self.package_unpacker = None self.hwpack = None self.packages = None self.packages_added_to_hwpack = [] self.out_name = out_name self.backports = backports def find_fetched_package(self, packages, wanted_package_name): wanted_package = None for package in packages: if package.name == wanted_package_name: wanted_package = package break else: raise AssertionError("Package '%s' was not fetched." % wanted_package_name) return wanted_package def add_file_to_hwpack(self, package, wanted_file, target_path): if (package.name, wanted_file) in self.packages_added_to_hwpack: # Don't bother adding the same package more than once. return tempfile_name = self.package_unpacker.get_file( package.filepath, wanted_file) self.packages_added_to_hwpack.append((package.name, target_path)) return self.hwpack.add_file(target_path, tempfile_name) def find_bootloader_packages(self, bootloaders_config): """Loop through the bootloaders dictionary searching for packages that should be installed, based on known keywords. :param bootloaders_config: The bootloaders dictionary to loop through. :return A list of packages, without duplicates.""" boot_packages = [] for key, value in bootloaders_config.iteritems(): if isinstance(value, dict): boot_packages.extend(self.find_bootloader_packages(value)) else: if key in PACKAGE_FIELDS: boot_packages.append(value) # Eliminate duplicates. return list(set(boot_packages)) def do_extract_file(self, package, source_path, dest_path): """Extract specified file from package to dest_path.""" package_ref = self.find_fetched_package(self.packages, package) return self.add_file_to_hwpack(package_ref, source_path, dest_path) def do_extract_files(self): """Go through a bootloader config, search for files to extract.""" base_dest_path = "" if self.config.board: base_dest_path = self.config.board base_dest_path = os.path.join(base_dest_path, self.config.bootloader) # Extract bootloader file if self.config.bootloader_package and self.config.bootloader_file: dest_path = os.path.join( base_dest_path, os.path.dirname(self.config.bootloader_file)) self.do_extract_file(self.config.bootloader_package, self.config.bootloader_file, dest_path) # Extract SPL file if self.config.spl_package and self.config.spl_file: dest_path = os.path.join(base_dest_path, os.path.dirname(self.config.spl_file)) self.do_extract_file(self.config.spl_package, self.config.spl_file, dest_path) def foreach_boards_and_bootloaders(self, function): """Call function for each board + bootloader combination in metadata""" if self.config.bootloaders is not None: for bootloader in self.config.bootloaders: self.config.board = None self.config.bootloader = bootloader function() if self.config.boards is not None: for board in self.config.boards: if self.config.bootloaders is not None: for bootloader in self.config.bootloaders: self.config.board = board self.config.bootloader = bootloader function() def extract_files(self): """Find bootloaders in config that may contain files to extract.""" if float(self.config.format.format_as_string) < 3.0: # extract files was introduced in version 3 configurations and is # a null operation for earlier configuration files return self.foreach_boards_and_bootloaders(self.do_extract_files) def do_find_copy_files_packages(self): """Find packages referenced by copy_files (single board, bootloader)""" copy_files = self.config.bootloader_copy_files if copy_files: self.copy_files_packages.extend(copy_files.keys()) def find_copy_files_packages(self): """Find all packages referenced by copy_files sections in metadata.""" self.copy_files_packages = [] self.foreach_boards_and_bootloaders( self.do_find_copy_files_packages) packages = self.copy_files_packages del self.copy_files_packages return packages def build(self): for architecture in self.config.architectures: logger.info("Building for %s" % architecture) metadata = Metadata.from_config( self.config, self.version, architecture) self.hwpack = HardwarePack(metadata) sources = self.config.sources with LocalArchiveMaker() as local_archive_maker: self.hwpack.add_apt_sources(sources) if sources: sources = sources.values() else: sources = [] self.packages = self.config.packages[:] # Loop through multiple bootloaders. # In V3 of hwpack configuration, all the bootloaders info and # packages are in the bootloaders section. if self.format.format_as_string == '3.0': if self.config.bootloaders is not None: self.packages.extend(self.find_bootloader_packages( self.config.bootloaders)) if self.config.boards is not None: self.packages.extend(self.find_bootloader_packages( self.config.boards)) self.packages.extend(self.find_copy_files_packages()) else: if self.config.bootloader_package is not None: self.packages.append(self.config.bootloader_package) if self.config.spl_package is not None: self.packages.append(self.config.spl_package) local_packages = [ FetchedPackage.from_deb(deb) for deb in self.local_debs] sources.append( local_archive_maker.sources_entry_for_debs( local_packages, LOCAL_ARCHIVE_LABEL)) self.packages.extend([lp.name for lp in local_packages]) logger.info("Fetching packages") fetcher = PackageFetcher( sources, architecture=architecture, backports=self.backports, prefer_label=LOCAL_ARCHIVE_LABEL) with fetcher: with PackageUnpacker() as self.package_unpacker: fetcher.ignore_packages(self.config.assume_installed) self.packages = fetcher.fetch_packages( self.packages, download_content=self.config.include_debs) if self.format.format_as_string == '3.0': self.extract_files() else: self._old_format_extract_files() self._add_packages_to_hwpack(local_packages) out_name = self.out_name if not out_name: out_name = self.hwpack.filename() manifest_name = os.path.splitext(out_name)[0] if manifest_name.endswith('.tar'): manifest_name = os.path.splitext(manifest_name)[0] manifest_name += '.manifest.txt' self._write_hwpack_and_manifest(out_name, manifest_name) cache_dir = fetcher.cache.tempdir self._extract_build_info(cache_dir, out_name, manifest_name) def _write_hwpack_and_manifest(self, out_name, manifest_name): """Write the real hwpack file and its manifest file. :param out_name: The name of the file to write. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Writing hwpack file") with open(out_name, 'w') as f: self.hwpack.to_file(f) logger.info("Wrote %s" % out_name) logger.debug("Writing manifest file content") with open(manifest_name, 'w') as f: f.write(self.hwpack.manifest_text()) def _old_format_extract_files(self): """Extract files for hwpack versions < 3.0.""" bootloader_package = None if self.config.bootloader_file is not None: assert(self.config.bootloader_package is not None) bootloader_package = self.find_fetched_package( self.packages, self.config.bootloader_package) self.hwpack.metadata.u_boot = self.add_file_to_hwpack( bootloader_package, self.config.bootloader_file, self.hwpack.U_BOOT_DIR) spl_package = None if self.config.spl_file is not None: assert self.config.spl_package is not None spl_package = self.find_fetched_package(self.packages, self.config.spl_package) self.hwpack.metadata.spl = self.add_file_to_hwpack( spl_package, self.config.spl_file, self.hwpack.SPL_DIR) # bootloader_package and spl_package can be identical if (bootloader_package is not None and bootloader_package in self.packages): self.packages.remove(bootloader_package) if (spl_package is not None and spl_package in self.packages): self.packages.remove(spl_package) def _add_packages_to_hwpack(self, local_packages): """Adds the packages to the hwpack. :param local_packages: The packages to add. :type local_packages: list """ logger.debug("Adding packages to hwpack") self.hwpack.add_packages(self.packages) for local_package in local_packages: if local_package not in self.packages: logger.warning("Local package '%s' not included", local_package.name) self.hwpack.add_dependency_package(self.config.packages) def _extract_build_info(self, cache_dir, out_name, manifest_name): """Extracts build-info from the packages. :param cache_dir: The cache directory where build-info should be located. :type cache_dir: str :param out_name: The name of the hwpack file. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Extracting build-info") build_info_dir = os.path.join(cache_dir, 'build-info') build_info_available = 0 for deb_pkg in self.packages: deb_pkg_file_path = deb_pkg.filepath # FIXME: test deb_pkg_dir to work around # https://bugs.launchpad.net/bugs/1067786 deb_pkg_dir = os.path.dirname(deb_pkg_file_path) if deb_pkg_dir != cache_dir: continue if os.path.islink(deb_pkg_file_path): # Skip symlink-ed debian package file # e.g. fetched package with dummy information continue try: # Extract Build-Info attribute from debian control deb_control = DebFile(deb_pkg_file_path).control.debcontrol() build_info = deb_control.get('Build-Info') except ArError: # Skip invalid debian package file # e.g. fetched package with dummy information continue if build_info is not None: build_info_available += 1 # Extract debian packages with build information env = os.environ env['LC_ALL'] = 'C' env['NO_PKG_MANGLE'] = '1' proc = cmd_runner.Popen(['dpkg-deb', '-x', deb_pkg_file_path, build_info_dir], env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdoutdata, stderrdata) = proc.communicate() if proc.returncode: raise ValueError('dpkg-deb extract failed!\n%s' % stderrdata) if stderrdata: raise ValueError('dpkg-deb extract had warnings:\n%s' % stderrdata) self._concatenate_build_info(build_info_available, build_info_dir, out_name, manifest_name) def _concatenate_build_info(self, build_info_available, build_info_dir, out_name, manifest_name): """Concatenates the build-info text if more than one is available. :param build_info_available: The number of available build-info. :type build_info_available: int :param build_info_dir: Where build-info files should be. :type build_info_dir: str :param out_name: The name of the hwpack file. :type out_name: str :param manifest_name: The name of the manifest file. :type manifest_name: str """ logger.debug("Concatenating build-info files") dst_file = open('BUILD-INFO.txt', 'wb') if build_info_available > 0: build_info_path = (r'%s/usr/share/doc/*/BUILD-INFO.txt' % build_info_dir) for src_file in iglob(build_info_path): with open(src_file, 'rb') as f: dst_file.write('\nFiles-Pattern: %s\n' % out_name) shutil.copyfileobj(f, dst_file) dst_file.write('\nFiles-Pattern: %s\nLicense-Type: open\n' % manifest_name) else: dst_file.write('Format-Version: 0.1\n' 'Files-Pattern: %s, %s\n' 'License-Type: open\n' % (out_name, manifest_name)) dst_file.close() linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/handler.py0000644000175000017500000002675712724020110025044 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . from StringIO import StringIO import ConfigParser import logging import os import re import shutil import tarfile import tempfile from linaro_image_tools.hwpack.config import Config from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.utils import DEFAULT_LOGGER_NAME logger = logging.getLogger(DEFAULT_LOGGER_NAME) class HardwarepackHandler(object): FORMAT_1 = '1.0' FORMAT_2 = '2.0' FORMAT_3 = '3.0' FORMAT_MIXED = '1.0and2.0' metadata_filename = 'metadata' format_filename = 'FORMAT' main_section = 'main' hwpack_tarfiles = [] tempdir = None def __init__(self, hwpacks, bootloader=None, board=None): self.hwpacks = hwpacks self.hwpack_tarfiles = [] self.bootloader = bootloader self.board = board self.tempdirs = {} # Used to store the config created from the metadata. self.config = None class FakeSecHead(object): """ Add a fake section header to the metadata file. This is done so we can use ConfigParser to parse the file. """ def __init__(self, fp): self.fp = fp self.sechead = '[%s]\n' % HardwarepackHandler.main_section def readline(self): if self.sechead: try: return self.sechead finally: self.sechead = None else: return self.fp.readline() def __enter__(self): self.tempdir = tempfile.mkdtemp() for hwpack in self.hwpacks: hwpack_tarfile = tarfile.open(hwpack, mode='r:gz') self.hwpack_tarfiles.append(hwpack_tarfile) return self def __exit__(self, type, value, traceback): for hwpack_tarfile in self.hwpack_tarfiles: if hwpack_tarfile is not None: hwpack_tarfile.close() self.hwpack_tarfiles = [] if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) for name in self.tempdirs: tempdir = self.tempdirs[name] if tempdir is not None and os.path.exists(tempdir): shutil.rmtree(tempdir) def _get_config_from_metadata(self, metadata): """ Retrieves a Config object associated with the metadata. :param metadata: The metadata to parse. :return: A Config instance. """ if not self.config: lines = metadata.readlines() if re.search("=", lines[0]) and not re.search(":", lines[0]): # Probably V2 hardware pack without [hwpack] on the first line lines = ["[hwpack]\n"] + lines self.config = Config(StringIO("".join(lines))) self.config.board = self.board self.config.bootloader = self.bootloader return self.config def get_field(self, field, return_keys=False): data = None hwpack_with_data = None keys = None for hwpack_tarfile in self.hwpack_tarfiles: metadata = hwpack_tarfile.extractfile(self.metadata_filename) parser = self._get_config_from_metadata(metadata) try: new_data = parser.get_option(field) if new_data is not None: assert data is None, "The metadata field '%s' is set to " \ "'%s' and new value '%s' is found" % (field, data, new_data) data = new_data hwpack_with_data = hwpack_tarfile if return_keys: keys = parser.get_last_used_keys() except ConfigParser.NoOptionError: continue if return_keys: return data, hwpack_with_data, keys return data, hwpack_with_data def get_format(self): format = None supported_formats = [self.FORMAT_1, self.FORMAT_2, self.FORMAT_3] for hwpack_tarfile in self.hwpack_tarfiles: format_file = hwpack_tarfile.extractfile(self.format_filename) format_string = format_file.read().strip() if not format_string in supported_formats: raise AssertionError( "Format version '%s' is not supported." % format_string) if format is None: format = format_string elif format != format_string: return self.FORMAT_MIXED return format def get_file(self, file_alias): """Get file(s) from a hwpack. :param file_alias: Property name (not field name) which contains file reference(s) :return: path to a file or list of paths to files """ file_names, hwpack_tarfile, keys = self.get_field(file_alias, return_keys=True) if not file_names: return file_names single = False if not isinstance(file_names, list): single = True file_names = [file_names] out_files = [] # Depending on if board and/or bootloader were used to look up the # file we are getting, we need to prepend those names to the path # to get the correct extracted file from the hardware pack. config_names = [("board", "boards"), ("bootloader", "bootloaders")] base_path = "" if keys: # If keys is non-empty, we have a V3 config option that was # modified by the bootloader and/or boot option... for name, key in config_names: if self.get_field(name): value = self.get_field(name)[0] if keys[0] == key: base_path = os.path.join(base_path, value) keys = keys[1:] for f in file_names: # Check that the base path is needed. If the file doesn't exist, # try without it (this provides fallback to V2 style directory # layouts with a V3 config). path_inc_board_and_bootloader = os.path.join(base_path, f) if path_inc_board_and_bootloader in hwpack_tarfile.getnames(): f = path_inc_board_and_bootloader hwpack_tarfile.extract(f, self.tempdir) f = os.path.join(self.tempdir, f) out_files.append(f) if single: return out_files[0] return out_files def list_packages(self): """Return list of (package names, TarFile object containing them)""" packages = [] for tf in self.hwpack_tarfiles: for name in tf.getnames(): if name.startswith("pkgs/") and name.endswith(".deb"): packages.append((tf, name)) return packages def find_package_for(self, name, version=None, revision=None, architecture=None): """Find a package that matches the name, version, rev and arch given. Packages are named according to the debian specification: http://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html _-_.deb DebianRevisionNumber seems to be optional. Use this spec to return a package matching the requirements given. """ for tar_file, package in self.list_packages(): file_name = os.path.basename(package) dpkg_chunks = re.search("^(.+)_(.+)_(.+)\.deb$", file_name) assert dpkg_chunks, "Could not split package file name into"\ "__.deb" pkg_name = dpkg_chunks.group(1) pkg_version = dpkg_chunks.group(2) pkg_architecture = dpkg_chunks.group(3) ver_chunks = re.search("^(.+)-(.+)$", pkg_version) if ver_chunks: pkg_version = ver_chunks.group(1) pkg_revision = ver_chunks.group(2) else: pkg_revision = None if name != pkg_name: continue if version is not None and str(version) != pkg_version: continue if revision is not None and str(revision) != pkg_revision: continue if (architecture is not None and str(architecture) != pkg_architecture): continue # Got a matching package - return its path inside the tarball return tar_file, package # Failed to find a matching package - return None return None def get_file_from_package(self, file_path, package_name, package_version=None, package_revision=None, package_architecture=None): """Extract named file from package specified by name, ver, rev, arch. File is extracted from the package matching the given specification to a temporary directory. The absolute path to the extracted file is returned. """ package_info = self.find_package_for(package_name, package_version, package_revision, package_architecture) if package_info is None: return None tar_file, package = package_info # Avoid unpacking hardware pack more than once by assigning each one # its own tempdir to unpack into. # TODO: update logic that uses self.tempdir so we can get rid of this # by sharing nicely. if not package in self.tempdirs: self.tempdirs[package] = tempfile.mkdtemp() tempdir = self.tempdirs[package] # We extract everything in the hardware pack so we don't have to worry # about chasing links (extract a link, find where it points to, extract # that...). This is slower, but more reliable. tar_file.extractall(tempdir) package_path = os.path.join(tempdir, package) with PackageUnpacker() as self.package_unpacker: extracted_file = self.package_unpacker.get_file(package_path, file_path) after_tmp = re.sub(self.package_unpacker.tempdir, "", extracted_file).lstrip("/\\") extract_dir = os.path.join(tempdir, "extracted", os.path.dirname(after_tmp)) os.makedirs(extract_dir) shutil.move(extracted_file, extract_dir) extracted_file = os.path.join(extract_dir, os.path.basename(extracted_file)) return extracted_file linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/0000755000175000017500000000000012724020110024176 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_better_tarfile.py0000644000175000017500000001722312724020110030607 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager from StringIO import StringIO import tarfile from testtools import TestCase from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile @contextmanager def standard_tarfile(backing_file, mode="r", seek=True): """A context manager to open a stdlib tarfile. :param backing_file: the file object to take the tarfile contents from. :param mode: the mode to open the tarfile with. :param seek: whether to seek the backing file to 0 before opening. """ if seek: backing_file.seek(0) tf = tarfile.TarFile.open(mode=mode, fileobj=backing_file) try: yield tf finally: tf.close() class TarFileTests(TestCase): def test_creates_empty_tarfile(self): backing_file = StringIO() with writeable_tarfile(backing_file): pass with standard_tarfile(backing_file, seek=False) as tf: self.assertEqual([], tf.getnames()) def create_simple_tarball(self, contents, **kwargs): backing_file = StringIO() with writeable_tarfile(backing_file, **kwargs) as tf: for path, content in contents: if path[-1] == '/': tf.create_dir(path) else: tf.create_file_from_string(path, content) return backing_file def test_create_file_from_string_adds_path(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(["foo"], tf.getnames()) def test_create_file_from_string_uses_content(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual("bar", tf.extractfile("foo").read()) def test_create_file_from_string_sets_size(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(3, tf.getmember("foo").size) def test_create_file_from_string_sets_mode(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0644, tf.getmember("foo").mode) def test_create_file_from_string_sets_type(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual(tarfile.REGTYPE, tf.getmember("foo").type) def test_create_file_from_string_sets_linkname(self): backing_file = self.create_simple_tarball([("foo", "bar")]) with standard_tarfile(backing_file) as tf: self.assertEqual('', tf.getmember("foo").linkname) def test_create_file_uses_default_mtime(self): now = 126793 backing_file = self.create_simple_tarball( [("foo", "bar")], default_mtime=now) with standard_tarfile(backing_file) as tf: self.assertEqual(now, tf.getmember("foo").mtime) def test_create_file_uses_default_uid(self): uid = 1259 backing_file = self.create_simple_tarball( [("foo", "bar")], default_uid=uid) with standard_tarfile(backing_file) as tf: self.assertEqual(uid, tf.getmember("foo").uid) def test_create_file_uses_default_gid(self): gid = 2259 backing_file = self.create_simple_tarball( [("foo", "bar")], default_gid=gid) with standard_tarfile(backing_file) as tf: self.assertEqual(gid, tf.getmember("foo").gid) def test_create_file_uses_default_uname(self): uname = "someperson" backing_file = self.create_simple_tarball( [("foo", "bar")], default_uname=uname) with standard_tarfile(backing_file) as tf: self.assertEqual(uname, tf.getmember("foo").uname) def test_create_file_uses_default_gname(self): gname = "somegroup" backing_file = self.create_simple_tarball( [("foo", "bar")], default_gname=gname) with standard_tarfile(backing_file) as tf: self.assertEqual(gname, tf.getmember("foo").gname) def test_create_dir_adds_path(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(["foo"], tf.getnames()) def test_create_dir_sets_name(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual("foo", tf.getmember("foo").name) def test_create_dir_sets_type(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(tarfile.DIRTYPE, tf.getmember("foo").type) def test_create_dir_sets_size(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0, tf.getmember("foo").size) def test_create_dir_sets_mode(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual(0755, tf.getmember("foo").mode) def test_create_dir_sets_linkname(self): backing_file = self.create_simple_tarball([("foo/", "")]) with standard_tarfile(backing_file) as tf: self.assertEqual('', tf.getmember("foo").linkname) def test_create_dir_uses_default_mtime(self): now = 126793 backing_file = self.create_simple_tarball( [("foo/", "")], default_mtime=now) with standard_tarfile(backing_file) as tf: self.assertEqual(now, tf.getmember("foo").mtime) def test_create_dir_uses_default_uid(self): uid = 1259 backing_file = self.create_simple_tarball( [("foo/", "")], default_uid=uid) with standard_tarfile(backing_file) as tf: self.assertEqual(uid, tf.getmember("foo").uid) def test_create_dir_uses_default_gid(self): gid = 2259 backing_file = self.create_simple_tarball( [("foo/", "")], default_gid=gid) with standard_tarfile(backing_file) as tf: self.assertEqual(gid, tf.getmember("foo").gid) def test_create_dir_uses_default_uname(self): uname = "someperson" backing_file = self.create_simple_tarball( [("foo/", "")], default_uname=uname) with standard_tarfile(backing_file) as tf: self.assertEqual(uname, tf.getmember("foo").uname) def test_create_dir_uses_default_gname(self): gname = "somegroup" backing_file = self.create_simple_tarball( [("foo/", "")], default_gname=gname) with standard_tarfile(backing_file) as tf: self.assertEqual(gname, tf.getmember("foo").gname) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_hwpack_reader.py0000644000175000017500000001050012724020110030402 0ustar voipiovoipio# Copyright (C) 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import tarfile from StringIO import StringIO from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, ) from linaro_image_tools.media_create.tests.fixtures import ( CreateTarballFixture, ) from linaro_image_tools.hwpack.hwpack_reader import ( Hwpack, HwpackReader, HwpackReaderError, ) class HwpackReaderTests(TestCaseWithFixtures): """Test class for the hwpack reader.""" def setUp(self): super(HwpackReaderTests, self).setUp() self.metadata = ("format: 3.0\nversion: '1'\nname: test-hwpack\n" "architecture: armel\norigin: Linaro") self.hwpack = Hwpack() self.hwpack.setname('test-hwpack') self.tar_dir_fixture = CreateTempDirFixture() self.useFixture(self.tar_dir_fixture) self.tarball_fixture = CreateTarballFixture( self.tar_dir_fixture.get_temp_dir()) self.useFixture(self.tarball_fixture) def tearDown(self): super(HwpackReaderTests, self).tearDown() self.hwpack = None self.metadata = "" def add_to_tarball(self, files, tarball=None): if tarball is None: tarball = self.tarball_fixture.get_tarball() tar_file = tarfile.open(tarball, mode='w:gz') for filename, data in files: tarinfo = tarfile.TarInfo(filename) tarinfo.size = len(data) tar_file.addfile(tarinfo, StringIO(data)) tar_file.close() return tarball def test_hwpack_class(self): hwpack = Hwpack() hwpack.setname('test-hwpack') hwpack.sethwpack('a_hwpack') self.hwpack.sethwpack('a_hwpack') self.assertEqual(self.hwpack, hwpack) def test_hwpack_class_not_equal(self): hwpack = Hwpack() hwpack.setname('test-hwpack') hwpack.sethwpack('a_hwpack') self.hwpack.sethwpack('b_hwpack') self.assertNotEqual(self.hwpack, hwpack) def test_hwpack_metadata_read(self): tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.assertEqual(self.hwpack, reader.supported_elements[0]) def test_raise_exception(self): new_metadata = ("format=2.0\nversion=4") tarball = self.add_to_tarball([('metadata', new_metadata)]) reader = HwpackReader([tarball]) self.assertRaises(HwpackReaderError, reader._read_hwpacks_metadata) def test_hwpack_metadata_read_with_boards(self): self.metadata += "\nboards:\n panda:\n support: supported\n" tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.hwpack.setboards({'panda': {'support': 'supported'}}) self.assertEqual(self.hwpack, reader.supported_elements[0]) def test_hwpack_metadata_read_with_bootloaders(self): self.metadata += ("\nboards:\n panda:\n support: supported\n " "bootloaders:\n u_boot:\n file: a_file\n") tarball = self.add_to_tarball([('metadata', self.metadata)]) reader = HwpackReader([tarball]) reader._read_hwpacks_metadata() self.hwpack.sethwpack(tarball) self.hwpack.setboards({'panda': {'support': 'supported', 'bootloaders': {'u_boot': {'file': 'a_file'}}}}) self.assertEqual(self.hwpack, reader.supported_elements[0]) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_hwpack_converter.py0000644000175000017500000002006312724020110031154 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tempfile from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, CreateTempFileFixture, ) from linaro_image_tools.hwpack.hwpack_convert import ( HwpackConverter, HwpackConverterException, check_and_validate_args, ) class Args(): """Defines the args for the command line options.""" def __init__(self, input_file, output_file=None): self.CONFIG_FILE = input_file self.out = output_file class HwpackConverterTests(TestCaseWithFixtures): """Test class for the hwpack converter.""" def setUp(self): super(HwpackConverterTests, self).setUp() def test_wrong_input_file(self): """Pass a non-existing file.""" input_file = '/tmp/foobaz' self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=input_file)) def test_wrong_input_dir(self): """Pass a directory instead of file.""" temp_file = tempfile.NamedTemporaryFile() temp_dir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=temp_file.name, output_file=temp_dir)) def test_same_input_output_file(self): """Pass the same existing file path to the two arguments.""" temp_file = self.useFixture(CreateTempFileFixture()).get_file_name() self.assertRaises( HwpackConverterException, check_and_validate_args, Args(input_file=temp_file, output_file=temp_file)) def test_basic_parse(self): ini_format = '[hwpack]\nformat=2.0\nsupport=supported' output_format = "format: '3.0'\nsupport: supported\n" input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(output_format, str(converter)) def test_architectures_section_creation(self): """Tests that we create the correct architectures list in the converted file. """ ini_format = '[hwpack]\nformat=2.0\narchitectures=armhf armel' output_format = "format: '3.0'\narchitectures:\n- armhf\n- armel\n" input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(output_format, str(converter)) def test_bootloaders(self): """Tests the correct creation of the bootloaders part.""" ini_format = ("[hwpack]\nformat=2.0\nu_boot_package=a_package\n" "u_boot_file=a_file\nu_boot_in_boot_part=Yes\n" "u_boot_dd=33") out_format = ("format: '3.0'\nbootloaders:\n u_boot:\n dd: '33'" "\n file: a_file\n in_boot_part: true\n" " package: a_package\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_extra_boot_options(self): """Tests the correct creation of the extra_boot_options part.""" ini_format = ("[hwpack]\nformat=2.0\nu_boot_package=a_package\n" "extra_boot_options=opt1 opt2") out_format = ("format: '3.0'\nbootloaders:\n u_boot:\n " " extra_boot_options:\n - opt1\n " "- opt2\n package: a_package\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_extra_serial_options(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\nextra_serial_options=opt1 opt2") out_format = ("format: '3.0'\nextra_serial_options:\n- opt1\n- opt2\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_assume_installed(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\nassume-installed=install1 " "install2") out_format = ("format: '3.0'\nassume_installed:\n- install1\n- " "install2\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_include_debs(self): """Tests the correct creation of the extra_serial_options part.""" ini_format = ("[hwpack]\nformat=2.0\ninclude-debs=yes") out_format = ("format: '3.0'\ninclude_debs: true\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_dtb_file(self): """Test the dtb_file conversion.""" ini_format = ("[hwpack]\nformat=2.0\ndtb_file=boot/a-*-path/file.dtb") out_format = ("format: '3.0'\ndtb_files:\n- board.dtb: " "boot/a-*-path/file.dtb\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) def test_mmc_id(self): """Test correct handling of mmc_id field. The mmc_id field has to be quoted coming out from the converter otherwise when reading the yaml file the value is read as a number, not a string.""" ini_format = ("[hwpack]\nformat=2.0\nmmc_id=1:1") out_format = ("format: '3.0'\nmmc_id: '1:1'\n") input_file = self.useFixture( CreateTempFileFixture(ini_format)).get_file_name() output_file = self.useFixture(CreateTempFileFixture()).get_file_name() converter = HwpackConverter(input_file, output_file) converter._parse() self.assertEqual(out_format, str(converter)) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_config.py0000644000175000017500000007624612724020110027073 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from StringIO import StringIO from testtools import TestCase from linaro_image_tools.hwpack.config import Config, HwpackConfigError from linaro_image_tools.hwpack.hwpack_fields import ( DEFINED_PARTITION_LAYOUTS, ) class ConfigTests(TestCase): valid_start = ( "[hwpack]\nname = ahwpack\npackages = foo\narchitectures = armel\n") valid_start_v2 = valid_start + "format = 2.0\n" valid_complete_v2 = (valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = usr/lib/u-boot/smdkv310/" "u-boot.bin\nserial_tty=ttySAC1\n" "partition_layout = bootfs_rootfs\n" "spl_package = x-loader-omap4-panda\n" "spl_file = usr/lib/x-loader/omap4430panda/MLO\n" "kernel_file = boot/vmlinuz-*-linaro-omap\n" "initrd_file = boot/initrd.img-*-linaro-omap\n" "dtb_file = boot/dt-*-linaro-omap/omap4-panda.dtb\n" "boot_script = boot.scr\n" + ("extra_serial_options = console=tty0 " "console=ttyO2,115200n8\n") + ("extra_boot_options = earlyprintk fixrtc " "nocompcache vram=48M omapfb.vram=0:24M " "mem=456M@0x80000000 mem=512M@0xA0000000\n") + "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = Yes\n") valid_end = "[ubuntu]\nsources-entry = foo bar\n" def test_create(self): config = Config(StringIO()) self.assertTrue(config is not None) def get_config(self, contents): return Config(StringIO(contents)) def assertConfigError(self, contents, f, *args, **kwargs): e = self.assertRaises(HwpackConfigError, f, *args, **kwargs) self.assertEqual(contents, str(e)) def assertValidationError(self, contents, config, function="validate"): self.assertConfigError(contents, config.get_option(function)) def test_validate_no_hwpack_section(self): config = self.get_config("") self.assertValidationError("No [hwpack] section", config) def test_validate_no_name(self): config = self.get_config("[hwpack]\n") self.assertValidationError("Empty value for name", config) def test_validate_empty_name(self): config = self.get_config("[hwpack]\nname = \n") self.assertValidationError("Empty value for name", config) def test_validate_invalid_name(self): config = self.get_config("[hwpack]\nname = ~~\n") self.assertValidationError("Invalid name: ~~", config) def test_validate_invalid_include_debs(self): config = self.get_config( "[hwpack]\nname = ahwpack\n" "include-debs = if you don't mind\n") self.assertValidationError( "Invalid value for include-debs: Not a boolean: if you don't mind", config) def test_validate_invalid_supported(self): config = self.get_config( "[hwpack]\nname = ahwpack\nsupport = if you pay us\n") self.assertValidationError( "Invalid value for support: if you pay us", config) def test_validate_no_packages(self): config = self.get_config( "[hwpack]\nname = ahwpack\n\n") self.assertValidationError( "No packages in the [hwpack] section", config) def test_validate_empty_packages(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = \n") self.assertValidationError( "No packages in the [hwpack] section", config) def test_validate_invalid_package_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo ~~ bar\n") self.assertValidationError( "Invalid value in packages in the [hwpack] section: ~~", config) def test_validate_no_architectures(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n") self.assertValidationError( "No architectures in the [hwpack] section", config) def test_validate_empty_architectures(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = \n") self.assertValidationError( "No architectures in the [hwpack] section", config) def test_validate_invalid_package_name_in_assume_installed(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = armel\nassume-installed = bar ~~\n") self.assertValidationError( "Invalid value in assume-installed in the [hwpack] section: ~~", config) def test_validate_no_other_sections(self): config = self.get_config(self.valid_start + "\n") self.assertValidationError( "No sections other than [hwpack]", config) def test_validate_other_section_no_sources_entry(self): config = self.get_config(self.valid_start + "\n[ubuntu]\n") self.assertValidationError( "No sources-entry in the [ubuntu] section", config) def test_validate_other_section_empty_sources_entry(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = \n") self.assertValidationError( "The sources-entry in the [ubuntu] section is missing the URI", config) def test_validate_other_section_only_uri_in_sources_entry(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = foo\n") self.assertValidationError( "The sources-entry in the [ubuntu] section is missing the " "distribution", config) def test_validate_other_section_sources_entry_starting_with_deb(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = deb http://example.org/ " "foo main\n") self.assertValidationError( "The sources-entry in the [ubuntu] section shouldn't start " "with 'deb'", config) def test_validate_other_section_sources_entry_starting_with_deb_src(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = deb-src http://example.org/ " "foo main\n") self.assertValidationError( "The sources-entry in the [ubuntu] section shouldn't start " "with 'deb'", config) def test_validate_valid_config(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = foo bar\n") self.assertEqual(None, config.validate()) def test_validate_valid_config_with_dash_in_package_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\n" "packages = u-boot\n" "architectures = armel\n\n" "[ubuntu]\nsources-entry = foo bar\n") self.assertEqual(None, config.validate()) def test_validate_supported_format(self): contents = self.valid_start + "format = 0.9\n" config = Config(StringIO(contents)) self.assertRaises(HwpackConfigError, config.validate) def test_validate_invalid_u_boot_package_name(self): config = self.get_config( self.valid_start_v2 + "u_boot_package = ~~\n") self.assertValidationError( "Invalid value in u_boot_package in the [hwpack] section: ~~", config) def test_validate_invalid_u_boot_file(self): config = self.get_config(self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_kernel_file(self): config = self.get_config(self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = ~~\n") self.assertValidationError("Invalid path: ~~", config, "_validate_vmlinuz") def test_validate_empty_kernel_file(self): config = self.get_config(self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = \n") self.assertValidationError("No kernel_file in the [hwpack] section", config, "_validate_vmlinuz") def test_validate_invalid_initrd_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = ~~\n") self.assertValidationError("Invalid path: ~~", config, "_validate_initrd") def test_validate_empty_initrd_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = \n") self.assertValidationError("No initrd_file in the [hwpack] section", config, "_validate_initrd") def test_validate_invalid_boot_script(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "mmc_id = 0:1\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "u_boot_in_boot_part = No\n" "boot_script = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_dtb_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "u_boot_in_boot_part = No\n" "mmc_id = 0:1\n" "dtb_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_invalid_spl_package_name(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = usr/bin/version/MLO\n" "partition_layout = bootfs_rootfs\n" "mmc_id = 0:1\n" "spl_package = ~~\n") self.assertValidationError( "Invalid value in spl_package in the [hwpack] section: ~~", config) def test_validate_invalid_spl_file(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = usr/bin/version/MLO\n" "partition_layout = bootfs_rootfs\n" "spl_package = x-loader--linaro-s5pv310\n" "spl_file = ~~\n") self.assertValidationError("Invalid path: ~~", config) def test_validate_partition_layout(self): partition_layout = 'apafs_bananfs' config = self.get_config(self.valid_start_v2 + "u_boot_package = " "u-boot-linaro-s5pv310\nu_boot_file = " "u-boot.bin\npartition_layout = %s\n" % partition_layout) self.assertValidationError( "Undefined partition layout %s in the [%s] section. " "Valid partition layouts are %s." % (partition_layout, 'hwpack', ", ".join(DEFINED_PARTITION_LAYOUTS)), config, "_validate_partition_layout") def test_validate_wired_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_wireless_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_u_boot_in_boot_part(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = Nope\n") self.assertValidationError( "Invalid value for u_boot_in_boot_part: Nope", config) def test_validate_u_boot_in_boot_part_bool(self): config = self.get_config( self.valid_start_v2 + "u-boot-package = u-boot-linaro-s5pv310\n" "u-boot-file = u-boot.bin\n" "partition_layout = bootfs_rootfs\n" "kernel_file = boot/vmlinuz-3.0.0-1002-linaro-omap\n" "initrd_file = boot/initrd.img-3.0.0-1002-linaro-omap\n" "boot_script = boot.scr\n" "mmc_id = 0:1\n" "u_boot_in_boot_part = True\n") self.assertValidationError( "Invalid value for u_boot_in_boot_part: True", config) def test_validate_serial_tty(self): config = self.get_config( self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = u-boot.bin\nserial_tty=tty\n") self.assertValidationError("Invalid serial tty: tty", config, "_validate_serial_tty") config = self.get_config( self.valid_start_v2 + "u_boot_package = u-boot-linaro-s5pv310\n" "u_boot_file = u-boot.bin\n" "serial_tty=ttxSAC1\n") self.assertValidationError("Invalid serial tty: ttxSAC1", config, "_validate_serial_tty") def test_validate_mmc_id(self): config = self.get_config(self.valid_complete_v2 + "mmc_id = x\n") self.assertValidationError("Invalid mmc_id x", config) def test_validate_boot_min_size(self): config = self.get_config(self.valid_complete_v2 + "boot_min_size = x\n") self.assertValidationError("Invalid boot min size x", config) def test_validate_root_min_size(self): config = self.get_config(self.valid_complete_v2 + "root_min_size = x\n") self.assertValidationError("Invalid root min size x", config) def test_validate_loader_min_size(self): config = self.get_config(self.valid_complete_v2 + "loader_min_size = x\n") self.assertValidationError("Invalid loader min size x", config) def test_validate_kernel_addr(self): config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8000000\n") self.assertValidationError("Invalid kernel address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8000000x\n") self.assertValidationError( "Invalid kernel address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 80000000\n") self.assertValidationError("Invalid kernel address: 80000000", config) def test_validate_initrd_addr(self): config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8000000\n") self.assertValidationError("Invalid initrd address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8000000x\n") self.assertValidationError( "Invalid initrd address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 80000000\n") self.assertValidationError("Invalid initrd address: 80000000", config) def test_validate_load_addr(self): config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8000000\n") self.assertValidationError("Invalid load address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8000000x\n") self.assertValidationError("Invalid load address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "load_addr = 80000000\n") self.assertValidationError("Invalid load address: 80000000", config) def test_validate_dtb_addr(self): config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8000000\n") self.assertValidationError("Invalid dtb address: 0x8000000", config) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8000000x\n") self.assertValidationError("Invalid dtb address: 0x8000000x", config) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 80000000\n") self.assertValidationError("Invalid dtb address: 80000000", config) def test_wired_interfaces(self): config = self.get_config(self.valid_complete_v2 + "wired_interfaces = eth0\n" + self.valid_end) config.validate() self.assertEqual(["eth0"], config.wired_interfaces) config = self.get_config(self.valid_complete_v2 + "wired_interfaces = eth0 eth1 usb2\n" + self.valid_end) config.validate() self.assertEqual(["eth0", "eth1", "usb2"], config.wired_interfaces) def test_wireless_interfaces(self): config = self.get_config(self.valid_complete_v2 + "wireless_interfaces = wlan0\n" + self.valid_end) config.validate() self.assertEqual(["wlan0"], config.wireless_interfaces) config = self.get_config(self.valid_complete_v2 + "wireless_interfaces = wlan0 wl1 usb2\n" + self.valid_end) config.validate() self.assertEqual(["wlan0", "wl1", "usb2"], config.wireless_interfaces) def test_partition_layout(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("bootfs_rootfs", config.partition_layout) def test_u_boot_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("usr/lib/u-boot/smdkv310/u-boot.bin", config.bootloader_file) def test_u_boot_package(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("u-boot-linaro-s5pv310", config.bootloader_package) def test_spl_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("usr/lib/x-loader/omap4430panda/MLO", config.spl_file) def test_kernel_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/vmlinuz-*-linaro-omap", config.vmlinuz) def test_initrd_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/initrd.img-*-linaro-omap", config.initrd) def test_dtb_file(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot/dt-*-linaro-omap/omap4-panda.dtb", config.dtb_file) def test_extra_boot_options(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual( "earlyprintk fixrtc nocompcache vram=48M " "omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000", config.extra_boot_options) def test_extra_serial_options(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("console=tty0 console=ttyO2,115200n8", config.extra_serial_options) def test_boot_script(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("boot.scr", config.boot_script) def test_u_boot_in_boot_part(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("Yes", config.bootloader_file_in_boot_part) def test_spl_package(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("x-loader-omap4-panda", config.spl_package) def test_serial_tty(self): config = self.get_config(self.valid_complete_v2 + self.valid_end) config.validate() self.assertEqual("ttySAC1", config.serial_tty) def test_mmc_id(self): config = self.get_config(self.valid_complete_v2 + "mmc_id = 0:1\n" + self.valid_end) config.validate() self.assertEqual("0:1", config.mmc_id) def test_boot_min_size(self): config = self.get_config(self.valid_complete_v2 + "boot_min_size = 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.boot_min_size) def test_root_min_size(self): config = self.get_config(self.valid_complete_v2 + "root_min_size = 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.root_min_size) def test_loader_min_size(self): config = self.get_config(self.valid_complete_v2 + "loader_min_size = 2\n" + self.valid_end) config.validate() self.assertEqual("2", config.loader_min_size) def test_kernel_addr(self): config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.kernel_addr) config = self.get_config(self.valid_complete_v2 + "kernel_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.kernel_addr) def test_initrd_addr(self): config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.initrd_addr) config = self.get_config(self.valid_complete_v2 + "initrd_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.initrd_addr) def test_load_addr(self): config = self.get_config(self.valid_complete_v2 + "load_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.load_addr) config = self.get_config(self.valid_complete_v2 + "load_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.load_addr) def test_dtb_addr(self): config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.dtb_addr) config = self.get_config(self.valid_complete_v2 + "dtb_addr = 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8aBcdEFf", config.dtb_addr) def test_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" "architectures = armel\n") self.assertEqual("ahwpack", config.name) def test_include_debs(self): config = self.get_config(self.valid_start + "include-debs = false\n") self.assertEqual(False, config.include_debs) def test_include_debs_defaults_true(self): config = self.get_config(self.valid_start) self.assertEqual(True, config.include_debs) def test_include_debs_defaults_true_on_empty(self): config = self.get_config(self.valid_start + "include-debs = \n") self.assertEqual(True, config.include_debs) def test_origin(self): config = self.get_config(self.valid_start + "origin = linaro\n") self.assertEqual("linaro", config.origin) def test_origin_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.origin) def test_origin_None_on_empty(self): config = self.get_config(self.valid_start + "origin = \n") self.assertEqual(None, config.origin) def test_maintainer(self): maintainer = "Linaro Developers " config = self.get_config( self.valid_start + "maintainer = %s\n" % maintainer) self.assertEqual(maintainer, config.maintainer) def test_maintainer_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.maintainer) def test_maintainer_None_on_empty(self): config = self.get_config(self.valid_start + "maintainer = \n") self.assertEqual(None, config.maintainer) def test_support_supported(self): config = self.get_config(self.valid_start + "support = supported\n") self.assertEqual("supported", config.support) def test_support_unsupported(self): config = self.get_config(self.valid_start + "support = unsupported\n") self.assertEqual("unsupported", config.support) def test_support_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.support) def test_support_None_on_empty(self): config = self.get_config(self.valid_start + "support = \n") self.assertEqual(None, config.support) def test_packages(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo bar\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\n bar\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo bar foo\n" "architectures=armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_sources_single(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = http://example.org foo\n") self.assertEqual({"ubuntu": "http://example.org foo"}, config.sources) def test_sources_multiple(self): config = self.get_config( self.valid_start + "\n[ubuntu]\nsources-entry = http://example.org foo\n" + "\n[linaro]\nsources-entry = http://example.org bar\n") self.assertEqual( {"ubuntu": "http://example.org foo", "linaro": "http://example.org bar"}, config.sources) def test_architectures(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=foo bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=foo\n bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\n" "architectures=foo bar foo\n") self.assertEqual(["foo", "bar"], config.architectures) def test_assume_installed(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_with_newline(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo\n bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_filters_duplicates(self): config = self.get_config( "[hwpack]\nname=ahwpack\npackages=foo\narchitectures=armel\n" "assume-installed=foo bar foo\n") self.assertEqual(["foo", "bar"], config.assume_installed) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_tarfile_matchers.py0000644000175000017500000002245012724020110031126 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tarfile from testtools import TestCase from linaro_image_tools.hwpack.tarfile_matchers import ( TarfileHasFile, TarfileMissingPathMismatch, TarfileWrongValueMismatch, ) from linaro_image_tools.hwpack.testing import test_tarfile class TarfileMissingPathMismatchTests(TestCase): def test_describe(self): mismatch = TarfileMissingPathMismatch("foo", "bar") self.assertEqual('"foo" has no path "bar"', mismatch.describe()) def test_eq(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("foo", "bar") self.assertEqual(mismatch1, mismatch2) self.assertFalse(mismatch1 != mismatch2) def test_no_eq_different_tarball(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("baz", "bar") self.assertNotEqual(mismatch1, mismatch2) def test_no_eq_different_path(self): mismatch1 = TarfileMissingPathMismatch("foo", "bar") mismatch2 = TarfileMissingPathMismatch("foo", "baz") self.assertNotEqual(mismatch1, mismatch2) class TarfileWrongTypeMismatchTests(TestCase): def test_describe(self): mismatch = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) self.assertEqual( 'The path "bar" in "foo" has type 2, expected 1', mismatch.describe()) def test_eq(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) self.assertEqual(mismatch1, mismatch2) self.assertFalse(mismatch1 != mismatch2) def test_not_eq_different_attribute(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("size", "foo", "bar", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_tarball(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "baz", "bar", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_path(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "baz", 1, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_expected(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 3, 2) self.assertNotEqual(mismatch1, mismatch2) def test_not_eq_different_actual(self): mismatch1 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 2) mismatch2 = TarfileWrongValueMismatch("type", "foo", "bar", 1, 3) self.assertNotEqual(mismatch1, mismatch2) class TarfileHasFileTests(TestCase): def test_str(self): matcher = TarfileHasFile("foo") self.assertEqual('tarfile has file "foo"', str(matcher)) def test_matches(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo") self.assertIs(None, matcher.match(tf)) def test_matches_directory(self): with test_tarfile(contents=[("foo/", "")]) as tf: matcher = TarfileHasFile("foo", type=tarfile.DIRTYPE) self.assertIs(None, matcher.match(tf)) def test_matches_directory_content(self): with test_tarfile(contents=[("foo/", ""), ("foo/bar", "")]) as tf: matcher = TarfileHasFile( "foo", type=tarfile.DIRTYPE, content=["bar"]) self.assertIs(None, matcher.match(tf)) def test_mismatches_missing_path(self): with test_tarfile() as tf: matcher = TarfileHasFile("foo") mismatch = matcher.match(tf) self.assertIsInstance(mismatch, TarfileMissingPathMismatch) self.assertEqual(TarfileMissingPathMismatch(tf, "foo"), mismatch) def assertValueMismatch(self, mismatch, tarball, path, attribute, expected, actual): self.assertIsInstance(mismatch, TarfileWrongValueMismatch) expected_mismatch = TarfileWrongValueMismatch( attribute, tarball, path, expected, actual) self.assertEqual(expected_mismatch, mismatch) def test_mismatches_wrong_type(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", type=tarfile.DIRTYPE) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "type", tarfile.DIRTYPE, tarfile.REGTYPE) def test_mismatches_wrong_size(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", size=1235) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "size", 1235, 0) def test_mismatches_wrong_mtime(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=54321) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "mtime", 54321, 12345) def test_mismatches_wrong_mode(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", mode=0000) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "mode", 0000, 0644) def test_mismatches_wrong_linkname(self): with test_tarfile(contents=[("foo", "")]) as tf: matcher = TarfileHasFile("foo", linkname="somelink") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "linkname", "somelink", "") def test_mismatches_wrong_uid(self): with test_tarfile(contents=[("foo", "")], default_uid=100) as tf: matcher = TarfileHasFile("foo", uid=99) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "uid", 99, 100) def test_mismatches_wrong_gid(self): with test_tarfile(contents=[("foo", "")], default_gid=100) as tf: matcher = TarfileHasFile("foo", gid=99) mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "gid", 99, 100) def test_mismatches_wrong_uname(self): with test_tarfile( contents=[("foo", "")], default_uname="someuser") as tf: matcher = TarfileHasFile("foo", uname="otheruser") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "uname", "otheruser", "someuser") def test_mismatches_wrong_gname(self): with test_tarfile( contents=[("foo", "")], default_gname="somegroup") as tf: matcher = TarfileHasFile("foo", gname="othergroup") mismatch = matcher.match(tf) self.assertValueMismatch( mismatch, tf, "foo", "gname", "othergroup", "somegroup") def test_mismatches_wrong_content(self): with test_tarfile(contents=[("foo", "somecontent")]) as tf: matcher = TarfileHasFile("foo", content="othercontent") mismatch = matcher.match(tf) self.assertEquals( "'othercontent' != 'somecontent': The content of " "path \"foo\" did not match", mismatch.describe()) def test_mismatches_wrong_directory_content(self): with test_tarfile(contents=[("foo/", ""), ("foo/bar", "")]) as tf: matcher = TarfileHasFile( "foo", type=tarfile.DIRTYPE, content=["baz"]) mismatch = matcher.match(tf) self.assertEquals( "['baz'] != ['bar']: The content of " "path \"foo\" did not match", mismatch.describe()) def test_matches_mtime_with_skew(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=12346, mtime_skew=1) mismatch = matcher.match(tf) self.assertIs(None, mismatch) def test_matches_mtime_without_skew(self): with test_tarfile(contents=[("foo", "")], default_mtime=12345) as tf: matcher = TarfileHasFile("foo", mtime=12345) mismatch = matcher.match(tf) self.assertIs(None, mismatch) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_hardwarepack.py0000644000175000017500000006470312724020110030255 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from StringIO import StringIO import re import tarfile from testtools import TestCase from testtools.matchers import Equals, MismatchError from linaro_image_tools.hwpack.hardwarepack import HardwarePack, Metadata from linaro_image_tools.hwpack.packages import get_packages_file from linaro_image_tools.hwpack.testing import ( DummyFetchedPackage, HardwarePackHasFile, MatchesAsPackagesFile, MatchesAsPackageContent, MatchesPackageRelationshipList, MatchesStructure, Not, ) from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, HardwarePackFormatV2, HardwarePackFormatV3, ) class MetadataTests(TestCase): def setUp(self): super(MetadataTests, self).setUp() self.metadata = Metadata("ahwpack", "3", "armel") def test_name(self): self.assertEqual("ahwpack", self.metadata.name) def test_version(self): self.assertEqual("3", self.metadata.version) def test_version_with_whitespace(self): self.assertRaises( AssertionError, Metadata, "ahwpack", "3 (with extras)", "armel") def test_architecture(self): self.assertEqual("armel", self.metadata.architecture) def test_default_origin_is_None(self): self.assertEqual(None, self.metadata.origin) def test_origin(self): metadata = Metadata("ahwpack", "4", "armel", origin="linaro") self.assertEqual("linaro", metadata.origin) def test_default_maintainer_is_None(self): metadata = Metadata("ahwpack", "4", "armel") self.assertEqual(None, metadata.maintainer) def test_maintainer(self): metadata = Metadata( "ahwpack", "4", "armel", maintainer="Some maintainer") self.assertEqual("Some maintainer", metadata.maintainer) def test_default_support_is_None(self): metadata = Metadata("ahwpack", "4", "armel") self.assertEqual(None, metadata.support) def test_support(self): metadata = Metadata("ahwpack", "4", "armel", support="supported") self.assertEqual("supported", metadata.support) def test_str(self): metadata = Metadata("ahwpack", "4", "armel") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_origin(self): metadata = Metadata("ahwpack", "4", "armel", origin="linaro") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "ORIGIN=linaro\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_maintainer(self): metadata = Metadata( "ahwpack", "4", "armel", maintainer="Some Maintainer") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "MAINTAINER=Some Maintainer\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_support(self): metadata = Metadata("ahwpack", "4", "armel", support="unsupported") expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "SUPPORT=unsupported\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_serial_tty(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(serial_tty='ttyO2') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "SERIAL_TTY=ttyO2\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_kernel_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(kernel_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "KERNEL_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_initrd_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(initrd_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "INITRD_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_load_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(load_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "LOAD_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_dtb_addr(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(dtb_addr='0x80000000') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "DTB_ADDR=0x80000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_wired_interfaces(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(wired_interfaces=['eth0', 'usb0']) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "WIRED_INTERFACES=eth0 usb0\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_wireless_interfaces(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(wireless_interfaces=['wlan0', 'wl0']) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "WIRELESS_INTERFACES=wlan0 wl0\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_partition_layout(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(partition_layout='bootfs_rootfs') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "PARTITION_LAYOUT=bootfs_rootfs\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_mmc_id(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(mmc_id='1') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "MMC_ID=1\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_boot_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(boot_min_size='50') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "BOOT_MIN_SIZE=50\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_root_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(root_min_size='100') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "ROOT_MIN_SIZE=100\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_loader_min_size(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(loader_min_size='1') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "LOADER_MIN_SIZE=1\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_kernel_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(vmlinuz='boot/vmlinuz-3.0.0-1002-linaro-omap') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "KERNEL_FILE=boot/vmlinuz-3.0.0-1002-linaro-omap\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_initrd_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(initrd='boot/initrd.img-3.0.0-1002-linaro-omap') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "INITRD_FILE=boot/initrd.img-3.0.0-1002-linaro-omap\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_dtb_file(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( dtb_file='boot/dt-3.0.0-1002-linaro-omap/omap4-panda.dtb') expected_out = ("NAME=ahwpack\nVERSION=4\n" "ARCHITECTURE=armel\nDTB_FILE=" "boot/dt-3.0.0-1002-linaro-omap/omap4-panda.dtb\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_boot_script(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config(boot_script='boot.scr') expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "BOOT_SCRIPT=boot.scr\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_extra_boot_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( extra_boot_options=( 'earlyprintk fixrtc nocompcache vram=48M omapfb.vram=0:24M ' 'mem=456M@0x80000000 mem=512M@0xA0000000')) expected_out = ("NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" "EXTRA_BOOT_OPTIONS=earlyprintk fixrtc nocompcache " "vram=48M omapfb.vram=0:24M " "mem=456M@0x80000000 mem=512M@0xA0000000\n") self.assertEqual(expected_out, str(metadata)) def test_str_with_extra_serial_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV2()) metadata.add_v2_config( extra_serial_options='console=tty0 console=ttyO2,115200n8') expected_out = ("NAME=ahwpack\nVERSION=4\n" "ARCHITECTURE=armel\nEXTRA_SERIAL_OPTIONS=" "console=tty0 console=ttyO2,115200n8\n") self.assertEqual(expected_out, str(metadata)) def test_from_config(self): class Config: name = "foo" origin = "linaro" maintainer = "someone" support = "supported" format = HardwarePackFormatV1() config = Config() metadata = Metadata.from_config(config, "2.0", "i386") self.assertEqual(config.name, metadata.name) self.assertEqual(config.origin, metadata.origin) self.assertEqual(config.maintainer, metadata.maintainer) self.assertEqual(config.support, metadata.support) self.assertEqual("2.0", metadata.version) self.assertEqual("i386", metadata.architecture) class NewMetadataTests(TestCase): def setUp(self): super(NewMetadataTests, self).setUp() def test_format(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) # Need to call also this one! metadata.add_v2_config() metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\n") self.assertEqual(expected_out, str(metadata)) def test_section_bootloaders(self): bootloaders = {'u_boot': {'file': 'a_file'}} metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) # Need to call also this one! metadata.add_v2_config() metadata.add_v3_config(bootloaders=bootloaders) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nbootloaders:\n u_boot:\n" " file: a_file\n") self.assertEqual(expected_out, str(metadata)) def test_section_wireless(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) wireless_list = ['wlan0', 'wl0'] # Need to call also this one! metadata.add_v2_config(wireless_interfaces=wireless_list) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nwireless_interfaces: wlan0 " "wl0\n") self.assertEqual(expected_out, str(metadata)) def test_section_wired(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) wired_list = ['eth0', 'usb0'] # Need to call also this one! metadata.add_v2_config(wired_interfaces=wired_list) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nwired_interfaces: eth0 usb0\n") self.assertEqual(expected_out, str(metadata)) def test_section_extra_serial_options(self): metadata = Metadata("ahwpack", "4", "armel", format=HardwarePackFormatV3()) options = ['option1', 'option2,option3'] # Need to call also this one! metadata.add_v2_config(extra_serial_options=options) metadata.add_v3_config(bootloaders=None) expected_out = ("format: '3.0'\nname: ahwpack\nversion: '4'\n" "architecture: armel\nextra_serial_options: option1 " "option2,option3\n") self.assertEqual(expected_out, str(metadata)) class HardwarePackTests(TestCase): def setUp(self): super(HardwarePackTests, self).setUp() self.metadata = Metadata("ahwpack", "4", "armel") def test_format_is_correct(self): format = '1.0' hwpack = HardwarePack(self.metadata) self.assertEqual(format, hwpack.format.__str__()) def test_format_has_no_spaces(self): hwpack = HardwarePack(self.metadata) self.assertIs(None, re.search('\s', hwpack.format.__str__()), "hwpack.format contains spaces.") def test_filename(self): hwpack = HardwarePack(self.metadata) self.assertEqual("hwpack_ahwpack_4_armel.tar.gz", hwpack.filename()) def test_filename_with_support(self): metadata = Metadata("ahwpack", "4", "armel", support="supported") hwpack = HardwarePack(metadata) self.assertEqual( "hwpack_ahwpack_4_armel_supported.tar.gz", hwpack.filename()) def test_filename_with_extension(self): hwpack = HardwarePack(self.metadata) self.assertEqual( "hwpack_ahwpack_4_armel.txt", hwpack.filename('.txt')) def get_tarfile(self, hwpack): fileobj = StringIO() hwpack.to_file(fileobj) fileobj.seek(0) tf = tarfile.open(mode="r:gz", fileobj=fileobj) self.addCleanup(tf.close) return tf def test_creates_FORMAT_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("FORMAT", content=hwpack.format.__str__() + "\n")) def test_creates_metadata_file(self): metadata = Metadata( "ahwpack", "4", "armel", origin="linaro", maintainer="Some Maintainer", support="unsupported") hwpack = HardwarePack(metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("metadata", content=str(metadata))) def test_creates_manifest_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("manifest")) def test_manifest_file_empty_with_no_packages(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("manifest", content="")) def test_manifest_contains_package_info(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.2") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("manifest", content="foo=1.1\nbar=1.2\n")) def test_creates_pkgs_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs", type=tarfile.DIRTYPE)) def test_adds_packages(self): package = DummyFetchedPackage("foo", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package.filename, content=package.content.read())) def test_adds_multiple_packages_at_once(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package1.filename, content=package1.content.read())) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package2.filename, content=package2.content.read())) def test_adds_multiple_in_multiple_steps(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1]) hwpack.add_packages([package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package1.filename, content=package1.content.read())) self.assertThat( tf, HardwarePackHasFile("pkgs/%s" % package2.filename, content=package2.content.read())) def test_add_packages_without_content_leaves_out_debs(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1]) tf = self.get_tarfile(hwpack) self.assertThat( tf, Not(HardwarePackHasFile("pkgs/%s" % package1.filename))) def test_add_dependency_package_adds_package(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package([]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/%s_%s_%s.deb" % ( 'hwpack-' + self.metadata.name, self.metadata.version, self.metadata.architecture), content_matcher=MatchesAsPackageContent( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=Equals(None), version=Equals(self.metadata.version))))) def test_add_dependency_package_adds_package_with_dependency(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package(["foo", "bar (= 1.0)"]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/%s_%s_%s.deb" % ( 'hwpack-' + self.metadata.name, self.metadata.version, self.metadata.architecture), content_matcher=MatchesAsPackageContent( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=MatchesPackageRelationshipList( [Equals("foo"), Equals("bar (= 1.0)")]), version=Equals(self.metadata.version))))) def test_add_dependency_package_adds_package_to_Packages(self): hwpack = HardwarePack(self.metadata) hwpack.add_dependency_package(["foo", "bar (= 1.0)"]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content_matcher=MatchesAsPackagesFile( MatchesStructure( name=Equals('hwpack-' + self.metadata.name), architecture=Equals(self.metadata.architecture), depends=MatchesPackageRelationshipList( [Equals("foo"), Equals("bar (= 1.0)")]), version=Equals(self.metadata.version))))) def test_creates_Packages_file(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs/Packages")) def test_Packages_file_empty_with_no_packages(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat(tf, HardwarePackHasFile("pkgs/Packages", content="")) def test_Packages_file_correct_contents_with_packages(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content=get_packages_file([package1, package2]))) def test_Packages_file_empty_with_no_deb_content(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) package2 = DummyFetchedPackage("bar", "1.1", no_content=True) hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("pkgs/Packages", content="")) def test_Packages_file_correct_content_with_some_deb_content(self): package1 = DummyFetchedPackage("foo", "1.1", no_content=True) package2 = DummyFetchedPackage("bar", "1.1") hwpack = HardwarePack(self.metadata) hwpack.add_packages([package1, package2]) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile( "pkgs/Packages", content=get_packages_file([package2]))) def test_creates_sources_list_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d", type=tarfile.DIRTYPE)) def test_adds_sources_list_file(self): hwpack = HardwarePack(self.metadata) source = 'http://example.org/ ubuntu' hwpack.add_apt_sources({'ubuntu': source}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + source + "\n")) def test_adds_multiple_sources_list_files(self): hwpack = HardwarePack(self.metadata) source1 = 'http://example.org/ ubuntu main universe' source2 = 'http://example.org/ linaro' hwpack.add_apt_sources({'ubuntu': source1, 'linaro': source2}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + source1 + "\n")) self.assertThat( tf, HardwarePackHasFile("sources.list.d/linaro.list", content="deb " + source2 + "\n")) def test_overwrites_sources_list_file(self): hwpack = HardwarePack(self.metadata) old_source = 'http://example.org/ ubuntu' hwpack.add_apt_sources({'ubuntu': old_source}) new_source = 'http://example.org/ ubuntu main universe' hwpack.add_apt_sources({'ubuntu': new_source}) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d/ubuntu.list", content="deb " + new_source + "\n")) def test_creates_sources_list_gpg_dir(self): hwpack = HardwarePack(self.metadata) tf = self.get_tarfile(hwpack) self.assertThat( tf, HardwarePackHasFile("sources.list.d.gpg", type=tarfile.DIRTYPE)) def test_password_removed_from_urls(self): hwpack = HardwarePack(self.metadata) url = "https://username:password@hostname/url precise main" hwpack.add_apt_sources({"protected": url}) tf = self.get_tarfile(hwpack) try: self.assertThat( tf, HardwarePackHasFile("sources.list.d/protected.list", content="deb " + url + "\n")) except MismatchError: pass # Expect to not find the password protected URL else: self.assertTrue(False, "Found password protected URL in hwpack") linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_packages.py0000644000175000017500000015110612724020110027371 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import re import shutil from StringIO import StringIO import subprocess import tempfile import textwrap import apt_pkg from debian.debfile import DebFile from debian import deb822 from testtools import TestCase from testtools.matchers import Equals from linaro_image_tools.hwpack.packages import ( DependencyNotSatisfied, DummyProgress, FetchedPackage, get_packages_file, IsolatedAptCache, LocalArchiveMaker, PackageFetcher, PackageMaker, stringify_relationship, TemporaryDirectoryManager, ) from linaro_image_tools.hwpack.testing import ( AptSourceFixture, ContextManagerFixture, DummyFetchedPackage, MatchesPackage, ) from linaro_image_tools.testing import TestCaseWithFixtures class GetPackagesFileTests(TestCase): def test_single_stanza(self): package = DummyFetchedPackage("foo", "1.1", architecture="armel") self.assertEqual(textwrap.dedent("""\ Package: foo Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: armel MD5sum: %(md5)s SHA256: %(sha256)s \n""" % {'filename': package.filename, 'size': package.size, 'md5': package.md5, 'sha256': package.sha256, }), get_packages_file([package])) def test_two_stanzas(self): package1 = DummyFetchedPackage("foo", "1.1") package2 = DummyFetchedPackage("bar", "1.2") self.assertEqual( get_packages_file([package1]) + get_packages_file([package2]), get_packages_file([package1, package2])) def get_stanza(self, package, relationships=""): stanza = textwrap.dedent("""\ Package: foo Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: all """ % { 'filename': package.filename, 'size': package.size, }) stanza += relationships stanza += "MD5sum: %s\n" % package.md5 stanza += "SHA256: %s\n\n" % package.sha256 return stanza def test_with_depends(self): package = DummyFetchedPackage("foo", "1.1", depends="bar | baz") self.assertEqual( self.get_stanza(package, "Depends: bar | baz\n"), get_packages_file([package])) def test_with_pre_depends(self): package = DummyFetchedPackage("foo", "1.1", pre_depends="bar | baz") self.assertEqual( self.get_stanza(package, "Pre-Depends: bar | baz\n"), get_packages_file([package])) def test_with_conflicts(self): package = DummyFetchedPackage("foo", "1.1", conflicts="bar | baz") self.assertEqual( self.get_stanza(package, "Conflicts: bar | baz\n"), get_packages_file([package])) def test_with_recommends(self): package = DummyFetchedPackage("foo", "1.1", recommends="bar | baz") self.assertEqual( self.get_stanza(package, "Recommends: bar | baz\n"), get_packages_file([package])) def test_with_provides(self): package = DummyFetchedPackage("foo", "1.1", provides="bar") self.assertEqual( self.get_stanza(package, "Provides: bar\n"), get_packages_file([package])) def test_with_replaces(self): package = DummyFetchedPackage("foo", "1.1", replaces="bar (<< 2.0)") self.assertEqual( self.get_stanza(package, "Replaces: bar (<< 2.0)\n"), get_packages_file([package])) def test_with_breaks(self): package = DummyFetchedPackage("foo", "1.1", breaks="bar (<< 2.0)") self.assertEqual( self.get_stanza(package, "Breaks: bar (<< 2.0)\n"), get_packages_file([package])) def test_with_extra_text(self): package = DummyFetchedPackage("foo", "1.1") self.assertEqual(textwrap.dedent("""\ Package: foo Status: bar Version: 1.1 Filename: %(filename)s Size: %(size)d Architecture: all MD5sum: %(md5)s SHA256: %(sha256)s \n""" % {'filename': package.filename, 'size': package.size, 'md5': package.md5, 'sha256': package.sha256, }), get_packages_file([package], extra_text="Status: bar")) class StringifyRelationshipTests(TestCaseWithFixtures): def test_no_relationship(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( None, stringify_relationship(candidate, "Depends")) def test_single_package(self): target_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar", stringify_relationship(candidate, "Depends")) def test_multiple_package(self): target_package = DummyFetchedPackage("foo", "1.0", depends="bar, baz") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar, baz", stringify_relationship(candidate, "Depends")) def test_alternative_packages(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="bar | baz") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "bar | baz", stringify_relationship(candidate, "Depends")) def test_package_with_le(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (<= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (<= 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_lt(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (<< 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (<< 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_eq(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (= 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_gt(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (>> 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (>> 2.0)", stringify_relationship(candidate, "Depends")) def test_package_with_ge(self): target_package = DummyFetchedPackage( "foo", "1.0", depends="baz (>= 2.0)") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate self.assertEqual( "baz (>= 2.0)", stringify_relationship(candidate, "Depends")) class TemporaryDirectoryManagerTests(TestCaseWithFixtures): def test_enter_twice_fails(self): maker = TemporaryDirectoryManager() maker.__enter__() self.assertRaises(AssertionError, maker.__enter__) def test_exit_without_enter_silent(self): maker = TemporaryDirectoryManager() maker.__exit__() def test_make_temporary_directory_without_enter_fails(self): maker = TemporaryDirectoryManager() self.assertRaises(AssertionError, maker.make_temporary_directory) def test_make_temporary_directory_makes_directory(self): maker = TemporaryDirectoryManager() self.useFixture(ContextManagerFixture(maker)) tmpdir = maker.make_temporary_directory() self.assertTrue(os.path.isdir(tmpdir)) def test_exit_removes_temporary_directory(self): maker = TemporaryDirectoryManager() self.useFixture(ContextManagerFixture(maker)) tmpdir = maker.make_temporary_directory() maker.__exit__() self.assertFalse(os.path.isdir(tmpdir)) class LocalArchiveMakerTests(TestCaseWithFixtures): def test_sources_entry_for_debs(self): package = DummyFetchedPackage("foo", "1.0") local_archive_maker = LocalArchiveMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) entry = local_archive_maker.sources_entry_for_debs([package]) with IsolatedAptCache([entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, package.filename) self.assertThat(created_package, MatchesPackage(package)) def test_packages_from_sources_entry_for_debs_are_fetchable(self): local_archive_maker = LocalArchiveMaker() package_maker = PackageMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) self.useFixture(ContextManagerFixture(package_maker)) deb_file_path = package_maker.make_package("foo", "1.0", {}) target_package = FetchedPackage.from_deb(deb_file_path) entry = local_archive_maker.sources_entry_for_debs( [target_package]) tmpdir = tempfile.mkdtemp() self.addCleanup(shutil.rmtree, tmpdir) with IsolatedAptCache([entry]) as cache: candidate = cache.cache['foo'].candidate acq = apt_pkg.Acquire(DummyProgress()) base = os.path.basename(candidate.filename) acqfile = apt_pkg.AcquireFile( acq, candidate.uri, candidate.md5, candidate.size, base, destfile=os.path.join(tmpdir, 'deb')) acq.run() self.assertThat(acqfile.status, Equals(acqfile.STAT_DONE)) def test_sources_entry_for_debs_creates_release_with_label(self): package = DummyFetchedPackage("foo", "1.0") local_archive_maker = LocalArchiveMaker() self.useFixture(ContextManagerFixture(local_archive_maker)) label_text = 'random-label' entry = local_archive_maker.sources_entry_for_debs( [package], label=label_text) loc = re.match('file://([^ ]*).*', entry).group(1) release = deb822.Release(open(os.path.join(loc, 'Release'))) self.assertThat(release['Label'], Equals(label_text)) class PackageMakerTests(TestCaseWithFixtures): def test_make_package_creates_file(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) self.assertTrue(os.path.exists(deb_path)) def test_make_package_creates_deb_with_given_package_name(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) deb_pkg = DebFile(deb_path) self.assertEqual('foo', deb_pkg.control.debcontrol()['Package']) def test_make_package_creates_deb_with_correct_file_name(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) proc = subprocess.Popen( ['dpkg-name', deb_path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) output = proc.communicate()[0] # dpkg-name prints 'dpkg-name: warning: skipping $path' when you give # it a package that is already correctly named. self.assertTrue( 'skipping' in output, "'skipping' was not found in dpkg-name output:\n%s" % output) def test_make_package_creates_deb_with_correct_file_name_arch(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}, 'armel') proc = subprocess.Popen( ['dpkg-name', deb_path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) output = proc.communicate()[0] self.assertTrue( 'skipping' in output, "'skipping' was not found in dpkg-name output:\n%s" % output) def test_make_package_creates_deb_with_given_version(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0ubuntu1', {}) deb_pkg = DebFile(deb_path) self.assertEqual( '1.0ubuntu1', deb_pkg.control.debcontrol()['Version']) def create_package_and_assert_control_fields_preserved(self, fields): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package( 'foo', '1.0', fields) deb_pkg = DebFile(deb_path) for key, value in fields.items(): self.assertEqual( value, deb_pkg.control.debcontrol()[key]) def test_make_package_creates_deb_with_given_depends(self): self.create_package_and_assert_control_fields_preserved( {'Depends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_predepends(self): self.create_package_and_assert_control_fields_preserved( {'Pre-Depends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_conflicts(self): self.create_package_and_assert_control_fields_preserved( {'Conflicts': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_recommends(self): self.create_package_and_assert_control_fields_preserved( {'Recommends': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_provides(self): self.create_package_and_assert_control_fields_preserved( {'Provides': 'bar, baz (= 1.0)'}) def test_make_package_creates_deb_with_given_replaces(self): self.create_package_and_assert_control_fields_preserved( {'Replaces': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_given_breaks(self): self.create_package_and_assert_control_fields_preserved( {'Breaks': 'bar, baz (>= 1.0)'}) def test_make_package_creates_deb_with_all_given_fields(self): self.create_package_and_assert_control_fields_preserved({ 'Depends': 'bar, baz (>= 1.0)', 'Pre-Depends': 'bar, baz (>= 1.0)', 'Conflicts': 'bar, baz (>= 1.0)', 'Recommends': 'bar, baz (>= 1.0)', 'Provides': 'bar, baz (= 1.0)', 'Replaces': 'bar, baz (>= 1.0)', 'Breaks': 'bar, baz (>= 1.0)', }) def test_unknown_field_name_fails(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) self.assertRaises( ValueError, maker.make_package, 'foo', '1.0', {'InvalidField': 'value'}) def test_arch_all_by_default(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}) deb_pkg = DebFile(deb_path) self.assertEqual("all", deb_pkg.control.debcontrol()['Architecture']) def test_custom_architecture(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_path = maker.make_package('foo', '1.0', {}, 'armel') deb_pkg = DebFile(deb_path) self.assertEqual('armel', deb_pkg.control.debcontrol()['Architecture']) class FetchedPackageTests(TestCaseWithFixtures): def test_attributes(self): package = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertEqual("foo", package.name) self.assertEqual("1.1", package.version) self.assertEqual("foo_1.1.deb", package.filename) self.assertEqual(None, package.content) self.assertEqual(4, package.size) self.assertEqual("aaaa", package.md5) self.assertEqual("bbbb", package.sha256) self.assertEqual("armel", package.architecture) def test_equal(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertEqual(package1, package2) self.assertFalse(package1 != package2) def test_not_equal_different_name(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "bar", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_version(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.2", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_filename(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "afoo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_size(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 5, "aaaa", "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_md5(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "bbbb", "bbbb", "armel") self.assertNotEqual(package1, package2) def test_not_equal_different_architecture(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "i386") self.assertNotEqual(package1, package2) def test_not_equal_different_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_depends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends=None) self.assertNotEqual(package1, package2) def test_equal_same_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", depends="bar") self.assertEqual(package1, package2) def test_not_equal_different_pre_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_pre_depends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends=None) self.assertNotEqual(package1, package2) def test_equal_same_pre_depends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", pre_depends="bar") self.assertEqual(package1, package2) def test_not_equal_different_conflicts(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_conflicts_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts=None) self.assertNotEqual(package1, package2) def test_equal_same_conflicts(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", conflicts="bar") self.assertEqual(package1, package2) def test_not_equal_different_recommends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_recommends_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends=None) self.assertNotEqual(package1, package2) def test_equal_same_recommends(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", recommends="bar") self.assertEqual(package1, package2) def test_not_equal_different_provides(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_provides_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides=None) self.assertNotEqual(package1, package2) def test_equal_same_provides(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", provides="bar") self.assertEqual(package1, package2) def test_not_equal_different_replaces(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_replaces_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces=None) self.assertNotEqual(package1, package2) def test_equal_same_replaces(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", replaces="bar") self.assertEqual(package1, package2) def test_not_equal_different_breaks(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks="baz") self.assertNotEqual(package1, package2) def test_not_equal_different_breaks_one_None(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks=None) self.assertNotEqual(package1, package2) def test_equal_same_breaks(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks="bar") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel", breaks="bar") self.assertEqual(package1, package2) def test_equal_different_contents(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package1.content = StringIO("xxxx") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2.content = StringIO("yyyy") self.assertEqual(package1, package2) def test_equal_one_with_contents_one_not(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package1.content = StringIO("xxxx") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertEqual(package1, package2) def test_equal_packages_hash_the_same(self): package1 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") package2 = FetchedPackage( "foo", "1.1", "foo_1.1.deb", 4, "aaaa", "bbbb", "armel") self.assertEqual(hash(package1), hash(package2)) def test_from_apt(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def create_package_and_assert_from_apt_translates_relationship( self, relationship): kwargs = {} kwargs[relationship] = "bar | baz (>= 1.0), zap" target_package = DummyFetchedPackage("foo", "1.0", **kwargs) source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def test_from_apt_with_depends(self): self.create_package_and_assert_from_apt_translates_relationship( 'depends') def test_from_apt_with_pre_depends(self): self.create_package_and_assert_from_apt_translates_relationship( 'pre_depends') def test_from_apt_with_conflicts(self): self.create_package_and_assert_from_apt_translates_relationship( 'conflicts') def test_from_apt_with_recommends(self): self.create_package_and_assert_from_apt_translates_relationship( 'recommends') def test_from_apt_with_replaces(self): self.create_package_and_assert_from_apt_translates_relationship( 'replaces') def test_from_apt_with_breaks(self): self.create_package_and_assert_from_apt_translates_relationship( 'breaks') def test_from_apt_with_provides(self): target_package = DummyFetchedPackage("foo", "1.0", provides="bar") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename, content=target_package.content) self.assertEqual(target_package, created_package) def test_from_apt_without_content(self): target_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([target_package])) with IsolatedAptCache([source.sources_entry]) as cache: candidate = cache.cache['foo'].candidate created_package = FetchedPackage.from_apt( candidate, target_package.filename) self.assertEqual(None, created_package.content) def test_from_deb(self): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_file_path = maker.make_package('foo', '1.0', {}) target_package = DummyFetchedPackage( "foo", "1.0", content=open(deb_file_path).read()) created_package = FetchedPackage.from_deb(deb_file_path) self.assertEqual(target_package, created_package) def create_package_and_assert_from_deb_translates_relationships( self, relationships): maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) deb_file_path = maker.make_package('foo', '1.0', relationships) dummy_relationships = {} for relationship, value in relationships.items(): dummy_relationships[relationship.lower().replace('-', '_')] = value target_package = DummyFetchedPackage( "foo", "1.0", content=open(deb_file_path).read(), **dummy_relationships) created_package = FetchedPackage.from_deb(deb_file_path) self.assertThat(created_package, MatchesPackage(target_package)) def test_from_deb_with_depends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Depends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_pre_depends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Pre-Depends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_conflicts(self): self.create_package_and_assert_from_deb_translates_relationships( {'Conflicts': 'bar, baz (>= 1.0)'}) def test_from_deb_with_recommends(self): self.create_package_and_assert_from_deb_translates_relationships( {'Recommends': 'bar, baz (>= 1.0)'}) def test_from_deb_with_replaces(self): self.create_package_and_assert_from_deb_translates_relationships( {'Replaces': 'bar, baz (>= 1.0)'}) def test_from_deb_with_breaks(self): self.create_package_and_assert_from_deb_translates_relationships( {'breaks': 'bar, baz (>= 1.0)'}) class AptCacheTests(TestCaseWithFixtures): def test_cleanup_removes_tempdir(self): cache = IsolatedAptCache([]) cache.prepare() tempdir = cache.tempdir cache.cleanup() self.assertFalse(os.path.exists(tempdir)) def test_cleanup_ignores_missing_tempdir(self): cache = IsolatedAptCache([]) cache.prepare() cache.cleanup() # Check that there is no problem removing it again cache.cleanup() def test_cleanup_before_prepare(self): cache = IsolatedAptCache([]) # Check that there is no problem cleaning up before we start cache.cleanup() def test_prepare_creates_tempdir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue(os.path.isdir(cache.tempdir)) def test_prepare_creates_var_lib_dpkg_status_file(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( '', open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_prepare_creates_var_cache_apt_archives_partial_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "var", "cache", "apt", "archives", "partial"))) def test_prepare_creates_var_lib_apt_lists_partial_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "var", "lib", "apt", "lists", "partial"))) def test_prepare_creates_etc_apt_sources_list_file(self): source1 = self.useFixture(AptSourceFixture([])) source2 = self.useFixture(AptSourceFixture([])) cache = IsolatedAptCache( [source1.sources_entry, source2.sources_entry]) self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( "deb %s\ndeb %s\n" % ( source1.sources_entry, source2.sources_entry), open(os.path.join( cache.tempdir, "etc", "apt", "sources.list")).read()) def test_prepare_creates_etc_apt_sources_list_dot_d_dir(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() self.assertTrue( os.path.isdir(os.path.join( cache.tempdir, "etc", "apt", "sources.list.d"))) def test_prepare_with_arch_creates_etc_apt_apt_conf(self): cache = IsolatedAptCache([], architecture="arch") self.addCleanup(cache.cleanup) cache.prepare() self.assertEqual( 'Apt {\nArchitecture "arch";\nInstall-Recommends "true";\n}\n', open(os.path.join( cache.tempdir, "etc", "apt", "apt.conf")).read()) def test_prepare_with_prefer_label_creates_etc_apt_preferences(self): label_text = 'random-label' cache = IsolatedAptCache([], prefer_label=label_text) self.addCleanup(cache.cleanup) cache.prepare() self.assertIn( label_text, open(os.path.join( cache.tempdir, "etc", "apt", "preferences")).read()) def test_context_manager(self): # A smoketest that IsolatedAptCache can be used as a context # manager with IsolatedAptCache([]) as cache: tempdir = cache.tempdir self.assertTrue(os.path.isdir(tempdir)) self.assertFalse(os.path.exists(tempdir)) def test_set_installed_packages(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() packages = [DummyFetchedPackage("foo", "1.0")] cache.set_installed_packages(packages) self.assertEqual( get_packages_file( packages, extra_text="Status: install ok installed"), open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_set_installed_packages_empty_list(self): cache = IsolatedAptCache([]) self.addCleanup(cache.cleanup) cache.prepare() cache.set_installed_packages([]) self.assertEqual( "", open(os.path.join( cache.tempdir, "var", "lib", "dpkg", "status")).read()) def test_package_list_has_no_passwords(self): source1 = self.useFixture(AptSourceFixture([])) path = re.sub("file:/", "file://user:pass@", source1.sources_entry) cache = IsolatedAptCache([path]) self.addCleanup(cache.cleanup) cache.prepare() cache.set_installed_packages([]) sources_list_location = os.path.join(cache.tempdir, "etc", "apt", "sources.list") sources_list = open(sources_list_location).read() self.assertEqual("deb %s\n" % source1.sources_entry, sources_list) class PackageFetcherTests(TestCaseWithFixtures): def test_context_manager(self): # A smoketest that PackageFetcher can be used as a context # manager with PackageFetcher([]) as fetcher: tempdir = fetcher.cache.tempdir self.assertTrue(os.path.isdir(tempdir)) self.assertFalse(os.path.exists(tempdir)) def get_fetcher(self, sources, architecture=None, prefer_label=None): fetcher = PackageFetcher( [s.sources_entry for s in sources], architecture=architecture, prefer_label=prefer_label) self.addCleanup(fetcher.cleanup) fetcher.prepare() return fetcher def test_fetch_packages_not_found_because_no_sources(self): fetcher = self.get_fetcher([]) self.assertRaises(KeyError, fetcher.fetch_packages, ["nothere"]) def test_fetch_packages_not_found_because_not_in_sources(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertRaises(KeyError, fetcher.fetch_packages, ["nothere"]) def test_fetch_packages_not_found_one_of_two_missing(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertRaises( KeyError, fetcher.fetch_packages, ["foo", "nothere"]) def test_fetch_packages_fetches_no_packages(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual(0, len(fetcher.fetch_packages([]))) def test_fetch_packages_fetches_single_package(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual(1, len(fetcher.fetch_packages(["foo"]))) def test_fetch_packages_fetches_correct_package(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) self.assertEqual( available_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_preferred_label(self): lower_package = DummyFetchedPackage("foo", "1.0") higher_package = DummyFetchedPackage("foo", "2.0") label_text = 'random-label' source1 = self.useFixture(AptSourceFixture([higher_package])) source2 = self.useFixture( AptSourceFixture([lower_package], label=label_text)) fetcher = self.get_fetcher( [source1, source2], prefer_label=label_text) self.assertEqual( lower_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_multiple_packages(self): available_packages = [ DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("foo", "1.0"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) self.assertEqual(2, len(fetcher.fetch_packages(["foo", "bar"]))) def test_fetch_packages_fetches_multiple_packages_correctly(self): available_packages = [ DummyFetchedPackage("foo", "1.0"), DummyFetchedPackage("bar", "1.0"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) fetched = fetcher.fetch_packages(["foo", "bar"]) self.assertEqual(available_packages[0], fetched[0]) self.assertEqual(available_packages[1], fetched[1]) def test_fetch_packages_fetches_newest(self): available_packages = [ DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("bar", "1.1"), ] source = self.useFixture(AptSourceFixture(available_packages)) fetcher = self.get_fetcher([source]) fetched = fetcher.fetch_packages(["bar"]) self.assertEqual(available_packages[1], fetched[0]) def test_fetch_packages_fetches_newest_from_multiple_sources(self): old_source_packages = [DummyFetchedPackage("bar", "1.0")] new_source_packages = [DummyFetchedPackage("bar", "1.1")] old_source = self.useFixture(AptSourceFixture(old_source_packages)) new_source = self.useFixture(AptSourceFixture(new_source_packages)) fetcher = self.get_fetcher([old_source, new_source]) fetched = fetcher.fetch_packages(["bar"]) self.assertEqual(new_source_packages[0], fetched[0]) def test_fetch_packages_records_correct_architecture(self): available_package = DummyFetchedPackage( "foo", "1.0", architecture="nonexistant") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source], architecture="nonexistant") self.assertEqual( "nonexistant", fetcher.fetch_packages(["foo"])[0].architecture) def test_fetch_packages_fetches_from_correct_architecture(self): wanted_package = DummyFetchedPackage( "foo", "1.0", architecture="arch1") unwanted_package = DummyFetchedPackage( "foo", "1.1", architecture="arch2") source = self.useFixture( AptSourceFixture([wanted_package, unwanted_package])) fetcher = self.get_fetcher([source], architecture="arch1") self.assertEqual( wanted_package, fetcher.fetch_packages(["foo"])[0]) def test_fetch_packages_fetches_with_relationships(self): depends = "foo" pre_depends = "bar (>= 1.0)" conflicts = "baz | zap" recommends = "zing, zang" dependency_packages = [ DummyFetchedPackage("foo", "1.0"), DummyFetchedPackage("bar", "1.0"), DummyFetchedPackage("baz", "1.0"), DummyFetchedPackage("zap", "1.0"), DummyFetchedPackage("zing", "1.0"), DummyFetchedPackage("zang", "1.0"), ] wanted_package = DummyFetchedPackage( "top", "1.0", depends=depends, pre_depends=pre_depends, conflicts=conflicts, recommends=recommends) source = self.useFixture( AptSourceFixture([wanted_package] + dependency_packages)) fetcher = self.get_fetcher([source]) self.assertIn( wanted_package, fetcher.fetch_packages(["top"])) def test_fetch_packages_download_content_False_doesnt_set_content(self): available_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([available_package])) fetcher = self.get_fetcher([source]) fetched_package = fetcher.fetch_packages( ["foo"], download_content=False)[0] self.assertIs(None, fetched_package.content) def test_fetches_dependencies(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", depends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( [wanted_package1, wanted_package2], fetcher.fetch_packages(["foo"])) def test_fetches_recommends(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", recommends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( [wanted_package1, wanted_package2], fetcher.fetch_packages(["foo"])) def test_broken_dependencies(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) e = self.assertRaises( DependencyNotSatisfied, fetcher.fetch_packages, ["foo"]) self.assertEqual( "Unable to satisfy dependencies of foo", str(e)) def test_fetch_packages_leaves_no_marked_changes(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.fetch_packages(["foo"]) self.assertEqual([], list(fetcher.cache.cache.get_changes())) def test_fetch_packages_without_content_leaves_no_marked_changes(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.fetch_packages(["foo"], download_content=False) self.assertEqual([], list(fetcher.cache.cache.get_changes())) def test_fetch_packages_can_remove_package(self): """Check that removed packages aren't included in the hwpack When installing the hwpack would cause an "assume-installed" package to be removed the hwpack shouldn't contain that package. """ wanted_package = DummyFetchedPackage( "foo", "1.0", conflicts="provided", provides="provided", depends="zoo", replaces="provided") top_package = DummyFetchedPackage( "top", "1.0", recommends="bar, baz") dep_package = DummyFetchedPackage( "bar", "1.0", depends="baz") conflict_package = DummyFetchedPackage( "baz", "1.0", recommends="bar", provides="provided", conflicts="provided", replaces="provided") extra_package = DummyFetchedPackage("zoo", "1.0") source = self.useFixture(AptSourceFixture( [wanted_package, dep_package, conflict_package, extra_package, top_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["top"]) fetched = fetcher.fetch_packages(["foo"]) self.assertEqual([wanted_package, extra_package], fetched) def test_ignore_with_provides(self): ignored_package = DummyFetchedPackage( "ubuntu-minimal", "1.0", depends="apt-utils") middle_package = DummyFetchedPackage( "apt-utils", "1.0", depends="libapt-pkg") provides_package = DummyFetchedPackage( "apt", "1.0", provides="libapt-pkg", replaces="someotherpackage") source = self.useFixture( AptSourceFixture( [ignored_package, middle_package, provides_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["ubuntu-minimal"]) def test_download_content_False_fetches_no_dependencies(self): wanted_package1 = DummyFetchedPackage("foo", "1.0", depends="bar") wanted_package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package1, wanted_package2])) fetcher = self.get_fetcher([source]) self.assertEqual( 1, len(fetcher.fetch_packages(["foo"], download_content=False))) def test_ignore_packages(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") ignored_package = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([wanted_package, ignored_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["bar"]) self.assertEqual( [wanted_package], fetcher.fetch_packages(["foo"])) def test_ignore_dependency_of_ignored(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="baz") ignored_package = DummyFetchedPackage("bar", "1.0", depends="baz") dependency = DummyFetchedPackage("baz", "1.0") source = self.useFixture( AptSourceFixture([wanted_package, ignored_package, dependency])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["bar"]) self.assertEqual( [wanted_package], fetcher.fetch_packages(["foo"])) def test_ignore_unknown_package(self): source = self.useFixture(AptSourceFixture([])) fetcher = self.get_fetcher([source]) self.assertRaises(KeyError, fetcher.ignore_packages, ["unknown"]) def test_ignore_cant_satisfy_depenencies(self): wanted_package = DummyFetchedPackage("foo", "1.0", depends="bar") source = self.useFixture(AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) e = self.assertRaises( DependencyNotSatisfied, fetcher.ignore_packages, ["foo"]) self.assertEqual( "Unable to satisfy dependencies of foo", str(e)) def test_ignored_arent_fetched_even_if_explicitly_requested(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture( AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], fetcher.fetch_packages(["foo"])) def test_no_metadata_for_ignored_even_if_explicitly_requested(self): wanted_package = DummyFetchedPackage("foo", "1.0") source = self.useFixture( AptSourceFixture([wanted_package])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], fetcher.fetch_packages(["foo"], download_content=False)) def test_ignore_is_cumulative(self): package1 = DummyFetchedPackage("foo", "1.0") package2 = DummyFetchedPackage("bar", "1.0") source = self.useFixture( AptSourceFixture([package1, package2])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) fetcher.ignore_packages(["bar"]) self.assertEqual( [], fetcher.fetch_packages(["foo", "bar"], download_content=False)) def test_ignore_leaves_no_marked_changes(self): package1 = DummyFetchedPackage("foo", "1.0") source = self.useFixture(AptSourceFixture([package1])) fetcher = self.get_fetcher([source]) fetcher.ignore_packages(["foo"]) self.assertEqual( [], list(fetcher.cache.cache.get_changes())) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_testing.py0000644000175000017500000002205112724020110027264 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import doctest import re from StringIO import StringIO import sys from testtools import TestCase from testtools.matchers import ( Annotate, Equals, Mismatch, NotEquals, ) from linaro_image_tools.hwpack.testing import ( DummyFetchedPackage, EachOf, MatchesAsPackagesFile, MatchesPackage, MatchesPackageRelationshipList, MatchesStructure, MatchesSetwise, parse_packages_file_content, ) from linaro_image_tools.hwpack.packages import ( get_packages_file, ) def run_doctest(obj, name): p = doctest.DocTestParser() t = p.get_doctest( obj.__doc__, sys.modules[obj.__module__].__dict__, name, '', 0) r = doctest.DocTestRunner() output = StringIO() r.run(t, out=output.write) return r.failures, output.getvalue() class TestEachOf(TestCase): def test_docstring(self): failure_count, output = run_doctest(EachOf, "EachOf") if failure_count: self.fail("Doctest failed with %s" % output) class TestMatchesPackageRelationshipList(TestCase): def test_docstring(self): failure_count, output = run_doctest( MatchesPackageRelationshipList, "MatchesPackageRelationshipList") if failure_count: self.fail("Doctest failed with %s" % output) class TestMatchesStructure(TestCase): class SimpleClass: def __init__(self, x): self.x = x def test_matches(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=Equals(1))) def test_mismatch(self): self.assertRaises( AssertionError, self.assertThat, self.SimpleClass(1), MatchesStructure(x=NotEquals(1))) def test_fromExample(self): self.assertThat( self.SimpleClass(1), MatchesStructure.fromExample(self.SimpleClass(1), 'x')) def test_update(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=NotEquals(1)).update(x=Equals(1))) def test_update_none(self): self.assertThat( self.SimpleClass(1), MatchesStructure(x=Equals(1), y=NotEquals(42)).update( y=None)) class TestMatchesPackage(TestCase): def test_simple(self): observed = DummyFetchedPackage("foo", "1.1", architecture="armel") expected = DummyFetchedPackage("foo", "1.1", architecture="armel") self.assertThat( observed, MatchesPackage(expected)) def test_mismatch(self): observed = DummyFetchedPackage("foo", "1.1", depends="bar") expected = DummyFetchedPackage("foo", "1.1", depends="baz") self.assertRaises(AssertionError, self.assertThat, observed, MatchesPackage(expected)) def test_skip_one_attribute(self): observed = DummyFetchedPackage("foo", "1.1", depends="bar") expected = DummyFetchedPackage("foo", "1.1", depends="baz") self.assertThat( observed, MatchesPackage(expected).update(depends=None)) class MatchesRegex(object): def __init__(self, pattern, flags=0): self.pattern = pattern self.flags = flags def match(self, value): if not re.match(self.pattern, value, self.flags): return Mismatch("%r did not match %r" % (self.pattern, value)) class TestMatchesSetwise(TestCase): def assertMismatchWithDescriptionMatching(self, value, matcher, description_matcher): mismatch = matcher.match(value) if mismatch is None: self.fail("%s matched %s" % (matcher, value)) actual_description = mismatch.describe() self.assertThat( actual_description, Annotate( "%s matching %s" % (matcher, value), description_matcher)) def test_matches(self): self.assertIs( None, MatchesSetwise(Equals(1), Equals(2)).match([2, 1])) def test_mismatches(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex('.*There was 1 mismatch$', re.S)) def test_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(1), Equals(2), Equals(3)), Equals('There was 1 matcher left over: Equals(1)')) def test_too_many_values(self): self.assertMismatchWithDescriptionMatching( [1, 2, 3], MatchesSetwise(Equals(1), Equals(2)), Equals('There was 1 value left over: [3]')) def test_two_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [3], MatchesSetwise(Equals(1), Equals(2), Equals(3)), MatchesRegex( 'There were 2 matchers left over: Equals\([12]\), ' 'Equals\([12]\)')) def test_two_too_many_values(self): self.assertMismatchWithDescriptionMatching( [1, 2, 3, 4], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( 'There were 2 values left over: \[[34], [34]\]')) def test_mismatch_and_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [2, 3], MatchesSetwise(Equals(0), Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 1 extra matcher: Equals\([01]\)', re.S)) def test_mismatch_and_too_many_values(self): self.assertMismatchWithDescriptionMatching( [2, 3, 4], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 1 extra value: \[[34]\]', re.S)) def test_mismatch_and_two_too_many_matchers(self): self.assertMismatchWithDescriptionMatching( [3, 4], MatchesSetwise( Equals(0), Equals(1), Equals(2), Equals(3)), MatchesRegex( '.*There was 1 mismatch and 2 extra matchers: ' 'Equals\([012]\), Equals\([012]\)', re.S)) def test_mismatch_and_two_too_many_values(self): self.assertMismatchWithDescriptionMatching( [2, 3, 4, 5], MatchesSetwise(Equals(1), Equals(2)), MatchesRegex( '.*There was 1 mismatch and 2 extra values: \[[145], [145]\]', re.S)) class TestParsePackagesFileContent(TestCase): def test_one(self): observed = DummyFetchedPackage("foo", "1.1") packages_content = get_packages_file([observed]) parsed = parse_packages_file_content(packages_content) self.assertThat(len(parsed), Equals(1)) self.assertThat(parsed[0], MatchesPackage(observed)) def test_several(self): observed1 = DummyFetchedPackage("foo", "1.1") observed2 = DummyFetchedPackage("bar", "1.2") observed3 = DummyFetchedPackage("baz", "1.5") packages_content = get_packages_file( [observed1, observed2, observed3]) parsed = parse_packages_file_content(packages_content) self.assertThat(parsed, MatchesSetwise( MatchesPackage(observed3), MatchesPackage(observed2), MatchesPackage(observed1))) class TestMatchesAsPackagesFile(TestCase): def test_one(self): observed = DummyFetchedPackage("foo", "1.1") packages_content = get_packages_file([observed]) self.assertThat( packages_content, MatchesAsPackagesFile( MatchesPackage(observed))) def test_ignore_one_md5(self): # This is what I actually care about: being able to specify that a # packages file matches a set of packages, ignoring just a few # details on just one package. observed1 = DummyFetchedPackage("foo", "1.1") observed2 = DummyFetchedPackage("bar", "1.2") observed3 = DummyFetchedPackage("baz", "1.5") packages_content = get_packages_file( [observed1, observed2, observed3]) oldmd5 = observed3.md5 observed3._content = ''.join(reversed(observed3._content_str())) self.assertNotEqual(oldmd5, observed3.md5) self.assertThat(packages_content, MatchesAsPackagesFile( MatchesPackage(observed1), MatchesPackage(observed2), MatchesPackage(observed3).update(md5=None))) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_config_v3.py0000644000175000017500000010623412724020110027472 0ustar voipiovoipio# Copyright (C) 2010 - 2012 Linaro # # Author: James Tunnicliffe # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import re from StringIO import StringIO from testtools import TestCase from linaro_image_tools.hwpack.config import Config, HwpackConfigError from linaro_image_tools.hwpack.hwpack_fields import ( DEFINED_PARTITION_LAYOUTS, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, ) class ConfigTests(TestCase): valid_start = ( "name: ahwpack\npackages: foo\narchitectures: armel\n") valid_start_v3 = valid_start + "format: 3.0\n" valid_complete_v3 = (valid_start_v3 + "serial_tty: ttySAC1\n" "partition_layout:\n" " - bootfs_rootfs\n" "boot_script: boot.scr\n" "extra_serial_options:\n" " - console=tty0\n" " - console=ttyO2,115200n8\n" "mmc_id: 0:1\n" "kernel_file: boot/vmlinuz-*-linaro-omap\n" "initrd_file: boot/initrd.img-*-linaro-omap\n" "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb\n" "bootloaders:\n" " u_boot:\n" " package: u-boot-linaro-s5pv310\n" " file: usr/lib/u-boot/smdkv310/u-boot.bin\n" " spl_package: x-loader-omap4-panda\n" " spl_file: usr/lib/x-loader/omap4430panda/MLO\n" " in_boot_part: True\n" " extra_boot_options:\n" " - earlyprintk\n" " - fixrtc\n" " - nocompcache\n" " - vram=48M\n" " - omapfb.vram=0:24M\n" " - mem=456M@0x80000000\n" " - mem=512M@0xA0000000\n") valid_end = "sources:\n sources-entry: foo bar\n" def test_create(self): config = Config(StringIO()) self.assertTrue(config is not None) def get_config(self, contents): if not re.search("\s*format\s*:", contents): contents = "format: 3.0\n" + contents return Config(StringIO(contents), bootloader="u_boot") def assertConfigError(self, contents, f, *args, **kwargs): e = self.assertRaises(HwpackConfigError, f, *args, **kwargs) self.assertEqual(contents, str(e)) def assertValidationError(self, contents, validate_function): self.assertConfigError(contents, validate_function) def test_validate_empty_name(self): config = self.get_config("name: ") self.assertValidationError("Empty value for name", config._validate_name) def test_validate_invalid_name(self): config = self.get_config("name: ~~\n") self.assertValidationError("Invalid name: ~~", config._validate_name) def test_validate_invalid_include_debs(self): config = self.get_config( "name: ahwpack\n" "include_debs: if you don't mind\n") self.assertValidationError( "Invalid value for include-debs: if you don't mind", config._validate_include_debs) def test_validate_invalid_supported(self): config = self.get_config( "name: ahwpack\nsupport: if you pay us\n") self.assertValidationError( "Invalid value for support: if you pay us", config._validate_support) def test_validate_no_packages(self): config = self.get_config( "name: ahwpack\n\n") self.assertValidationError( "No packages found in the metadata", config._validate_packages) def test_validate_empty_packages(self): config = self.get_config( "name: ahwpack\npackages: \n") self.assertValidationError( "No packages found in the metadata", config._validate_packages) def test_validate_invalid_package_name(self): config = self.get_config( "name: ahwpack\npackages: foo ~~ bar\n") self.assertValidationError( "Invalid value in packages in the metadata: ~~", config._validate_packages) def test_validate_no_architectures(self): config = self.get_config( "name: ahwpack\npackages: foo\n") self.assertValidationError( "No architectures found in the metadata", config._validate_architectures) def test_validate_empty_architectures(self): config = self.get_config( "name: ahwpack\npackages: foo\n" "architectures: \n") self.assertValidationError( "No architectures found in the metadata", config._validate_architectures) def test_validate_invalid_package_name_in_assume_installed(self): config = self.get_config( "name: ahwpack\npackages: foo\n" "architectures: armel\nassume_installed:\n - bar\n - ~~\n") self.assertValidationError( "Invalid value in assume-installed in the metadata: ~~", config._validate_assume_installed) def test_validate_other_section_empty_sources_entry(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: \n") self.assertValidationError( "The sources-entry, ubuntu is missing the URI", config._validate_sources) def test_validate_other_section_only_uri_in_sources_entry(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: foo\n") self.assertValidationError( "The sources-entry, ubuntu is missing the distribution", config._validate_sources) def test_validate_other_section_sources_entry_starting_with_deb(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: deb http://example.org/ foo main\n") self.assertValidationError( "The sources-entry, ubuntu shouldn't start with 'deb'", config._validate_sources) def test_validate_other_section_sources_entry_starting_with_deb_src(self): config = self.get_config( self.valid_start + "sources:\n ubuntu: deb-src http://example.org/ foo main\n") self.assertValidationError( "The sources-entry, ubuntu shouldn't start with 'deb'", config._validate_sources) def test_validate_valid_config(self): config = self.get_config(self.valid_complete_v3) self.assertEqual(None, config.validate()) def test_validate_supported_format(self): config = self.get_config(self.valid_start + "format: 0.9\n") self.assertValidationError( "Format version '0.9' is not supported.", config._validate_format) def test_validate_invalid_u_boot_package_name(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " package: ~~\n") self.assertValidationError( "Invalid value in u_boot_package in the metadata: ~~", config._validate_bootloader_package) def test_validate_invalid_bootloader_file(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_bootloader_file) def test_validate_invalid_kernel_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_vmlinuz) def test_validate_empty_kernel_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: \n") self.assertValidationError("No kernel_file found in the metadata", config._validate_vmlinuz) def test_validate_invalid_initrd_file(self): config = self.get_config(self.valid_start_v3 + "initrd_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_initrd) def test_validate_empty_initrd_file(self): config = self.get_config(self.valid_start_v3 + "kernel_file: \n") self.assertValidationError("No initrd_file found in the metadata", config._validate_initrd) def test_validate_invalid_boot_script(self): config = self.get_config(self.valid_start_v3 + "boot_script: ~~") self.assertValidationError("Invalid path: ~~", config._validate_boot_script) def test_validate_invalid_dtb_file(self): config = self.get_config(self.valid_start_v3 + "dtb_file: ~~\n") self.assertValidationError("Invalid path: ~~", config._validate_dtb_file) def test_validate_invalid_spl_package_name(self): config = self.get_config(self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " spl_package: ~~\n") config.board = "panda" self.assertValidationError( "Invalid value in spl_package in the metadata: ~~", config._validate_spl_package) def test_validate_invalid_spl_file(self): config = self.get_config(self.valid_start_v3 + "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " spl_file: ~~\n") config.board = "panda" self.assertValidationError("Invalid path: ~~", config._validate_spl_file) def test_validate_partition_layout(self): partition_layout = 'apafs_bananfs' config = self.get_config(self.valid_start_v3 + "partition_layout: " + partition_layout) self.assertValidationError( "Undefined partition layout %s. " "Valid partition layouts are %s." % (partition_layout, ", ".join(DEFINED_PARTITION_LAYOUTS)), config._validate_partition_layout) def test_validate_wired_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_wireless_interfaces(self): self.assertTrue("XXX What is an invalid interface name?") def test_validate_bootloader_in_boot_part_bool(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: Nope\n") self.assertValidationError( "Invalid value for bootloader_in_boot_part: Nope", config._validate_bootloader_file_in_boot_part) def test_find_board_specific_variable(self): config = self.get_config( self.valid_start_v3 + "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config.board = "panda" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_board_specific_overwrites_global(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: No\n" "boards:\n" " panda:\n" " bootloaders:\n" " u_boot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config.board = "panda" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_multiple_bootloaders(self): config = self.get_config( self.valid_start_v3 + "bootloaders:\n" " u_boot:\n" " in_boot_part: No\n" " anotherboot:\n" " in_boot_part: Yes\n") config.bootloader = "u_boot" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "no") config.bootloader = "anotherboot" config._validate_bootloader_file_in_boot_part() self.assertEqual(config.bootloader_file_in_boot_part, "yes") def test_validate_serial_tty(self): config = self.get_config(self.valid_start_v3 + "serial_tty: tty\n") self.assertValidationError("Invalid serial tty: tty", config._validate_serial_tty) config = self.get_config(self.valid_start_v3 + "serial_tty: ttxSAC1\n") self.assertValidationError("Invalid serial tty: ttxSAC1", config._validate_serial_tty) def test_validate_mmc_id_wrong(self): # The mmc_id value, if coming from a yaml file, has to be quoted. # Make sure the test does not accept a valid-unquoted value. config = self.get_config(self.valid_complete_v3 + "mmc_id: 1:1\n") self.assertRaises(HwpackConfigError, config._validate_mmc_id) def test_validate_mmc_id(self): config = self.get_config(self.valid_complete_v3 + "mmc_id: x\n") self.assertValidationError("Invalid mmc_id x", config._validate_mmc_id) def test_validate_boot_min_size(self): config = self.get_config(self.valid_complete_v3 + "boot_min_size: x\n") self.assertValidationError("Invalid boot min size x", config._validate_boot_min_size) def test_validate_root_min_size(self): config = self.get_config(self.valid_complete_v3 + "root_min_size: x\n") self.assertValidationError("Invalid root min size x", config._validate_root_min_size) def test_validate_loader_min_size(self): config = self.get_config(self.valid_complete_v3 + "loader_min_size: x\n") self.assertValidationError("Invalid loader min size x", config._validate_loader_min_size) def test_validate_kernel_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8000000\n") config._validate_kernel_addr() config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8000000x\n") self.assertValidationError( "Invalid kernel address: 0x8000000x", config._validate_kernel_addr) config = self.get_config(self.valid_complete_v3 + "kernel_addr: 80000000\n") config._validate_kernel_addr() def test_validate_initrd_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8000000\n") config._validate_initrd_addr() config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8000000x\n") self.assertValidationError( "Invalid initrd address: 0x8000000x", config._validate_initrd_addr) config = self.get_config(self.valid_complete_v3 + "initrd_addr: 80000000\n") config._validate_initrd_addr() def test_validate_load_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8000000\n") config._validate_load_addr() config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8000000x\n") self.assertValidationError("Invalid load address: 0x8000000x", config._validate_load_addr) config = self.get_config(self.valid_complete_v3 + "load_addr: 80000000\n") config._validate_load_addr() def test_validate_dtb_addr(self): # V3 change: All numerical inputs are good addresses (since YAML # converts them to ingegers and we convert them back to the correct # format). We don't need 8 digit hex values for addresses. config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8000000\n") config._validate_dtb_addr() config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8000000x\n") self.assertValidationError("Invalid dtb address: 0x8000000x", config._validate_dtb_addr) config = self.get_config(self.valid_complete_v3 + "dtb_addr: 80000000\n") config._validate_dtb_addr() def test_wired_interfaces(self): config = self.get_config(self.valid_complete_v3 + "wired_interfaces:\n - eth0\n" + self.valid_end) config.validate() self.assertEqual(["eth0"], config.wired_interfaces) config = self.get_config(self.valid_complete_v3 + "wired_interfaces:\n" " - eth0\n" " - eth1\n" " - usb2\n" + self.valid_end) config.validate() self.assertEqual(["eth0", "eth1", "usb2"], config.wired_interfaces) def test_wireless_interfaces(self): config = self.get_config(self.valid_complete_v3 + "wireless_interfaces:\n" " - wlan0\n" + self.valid_end) config.validate() self.assertEqual(["wlan0"], config.wireless_interfaces) config = self.get_config(self.valid_complete_v3 + "wireless_interfaces:\n" " - wlan0\n" " - wl1\n" " - usb2\n" + self.valid_end) config.validate() self.assertEqual(["wlan0", "wl1", "usb2"], config.wireless_interfaces) def test_partition_layout(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("bootfs_rootfs", config.partition_layout) def test_bootloader_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("usr/lib/u-boot/smdkv310/u-boot.bin", config.bootloader_file) def test_u_boot_package(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("u-boot-linaro-s5pv310", config.bootloader_package) def test_spl_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("usr/lib/x-loader/omap4430panda/MLO", config.spl_file) def test_kernel_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/vmlinuz-*-linaro-omap", config.vmlinuz) def test_initrd_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/initrd.img-*-linaro-omap", config.initrd) def test_dtb_file(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot/dt-*-linaro-omap/omap4-panda.dtb", config.dtb_file) def test_extra_boot_options(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual( "earlyprintk fixrtc nocompcache vram=48M " "omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000", config.extra_boot_options) def test_extra_serial_options(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual('console=tty0 console=ttyO2,115200n8', config.extra_serial_options) def test_boot_script(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("boot.scr", config.boot_script) def test_u_boot_in_boot_part(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("yes", config.bootloader_file_in_boot_part) def test_spl_package(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("x-loader-omap4-panda", config.spl_package) def test_serial_tty(self): config = self.get_config(self.valid_complete_v3 + self.valid_end) config.validate() self.assertEqual("ttySAC1", config.serial_tty) def test_mmc_id(self): config = self.get_config(self.valid_complete_v3 + "mmc_id: 0:1\n" + self.valid_end) config.validate() self.assertEqual("0:1", config.mmc_id) def test_boot_min_size(self): config = self.get_config(self.valid_complete_v3 + "boot_min_size: 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.boot_min_size) def test_root_min_size(self): config = self.get_config(self.valid_complete_v3 + "root_min_size: 50\n" + self.valid_end) config.validate() self.assertEqual("50", config.root_min_size) def test_loader_min_size(self): config = self.get_config(self.valid_complete_v3 + "loader_min_size: 2\n" + self.valid_end) config.validate() self.assertEqual("2", config.loader_min_size) def test_kernel_addr(self): config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.kernel_addr) config = self.get_config(self.valid_complete_v3 + "kernel_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.kernel_addr) def test_initrd_addr(self): config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.initrd_addr) config = self.get_config(self.valid_complete_v3 + "initrd_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.initrd_addr) def test_load_addr(self): config = self.get_config(self.valid_complete_v3 + "load_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.load_addr) config = self.get_config(self.valid_complete_v3 + "load_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.load_addr) def test_dtb_addr(self): config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x80000000\n" + self.valid_end) config.validate() self.assertEqual("0x80000000", config.dtb_addr) config = self.get_config(self.valid_complete_v3 + "dtb_addr: 0x8aBcdEFf\n" + self.valid_end) config.validate() self.assertEqual("0x8abcdeff", config.dtb_addr) def test_name(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures: armel\n") self.assertEqual("ahwpack", config.name) def test_include_debs(self): config = self.get_config(self.valid_start + "include_debs: false\n") self.assertEqual(False, config.include_debs) def test_include_debs_defaults_true(self): config = self.get_config(self.valid_start) self.assertEqual(True, config.include_debs) def test_include_debs_defaults_true_on_empty(self): config = self.get_config(self.valid_start + "include_debs: \n") self.assertEqual(True, config.include_debs) def test_origin(self): config = self.get_config(self.valid_start + "origin: linaro\n") self.assertEqual("linaro", config.origin) def test_origin_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.origin) def test_origin_None_on_empty(self): config = self.get_config(self.valid_start + "origin: \n") self.assertEqual(None, config.origin) def test_maintainer(self): maintainer = "Linaro Developers " config = self.get_config( self.valid_start + "maintainer: %s\n" % maintainer) self.assertEqual(maintainer, config.maintainer) def test_maintainer_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.maintainer) def test_maintainer_None_on_empty(self): config = self.get_config(self.valid_start + "maintainer: \n") self.assertEqual(None, config.maintainer) def test_support_supported(self): config = self.get_config(self.valid_start + "support: supported\n") self.assertEqual("supported", config.support) def test_support_unsupported(self): config = self.get_config(self.valid_start + "support: unsupported\n") self.assertEqual("unsupported", config.support) def test_support_default_None(self): config = self.get_config(self.valid_start) self.assertEqual(None, config.support) def test_support_None_on_empty(self): config = self.get_config(self.valid_start + "support: \n") self.assertEqual(None, config.support) def test_packages(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" " - bar\n" "architectures: armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_packages_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" " - bar\n" " - foo\n" "architectures: armel\n") self.assertEqual(["foo", "bar"], config.packages) def test_sources_single(self): config = self.get_config( self.valid_start + "sources:\n" " ubuntu: http://example.org foo\n") self.assertEqual({"ubuntu": "http://example.org foo"}, config.sources) def test_sources_multiple(self): config = self.get_config( self.valid_start + "sources:\n" " ubuntu: http://example.org foo\n" " linaro: http://example.org bar\n") self.assertEqual( {"ubuntu": "http://example.org foo", "linaro": "http://example.org bar"}, config.sources) def test_architectures(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures:\n" " - foo\n" " - bar\n") self.assertEqual(["foo", "bar"], config.architectures) def test_architectures_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages: foo\n" "architectures:\n" " - foo\n" " - bar\n" " - foo\n") self.assertEqual(["foo", "bar"], config.architectures) def test_assume_installed(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" "architectures:\n" " - armel\n" "assume_installed:\n" " - foo\n" " - bar\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_assume_installed_filters_duplicates(self): config = self.get_config( "name: ahwpack\n" "packages:\n" " - foo\n" "architectures:\n" " - armel\n" "assume_installed:\n" " - foo\n" " - bar\n" " - foo\n") self.assertEqual(["foo", "bar"], config.assume_installed) def test_invalid_key_in_root(self): config = self.get_config("foo: bar") self.assertValidationError("Unknown key in metadata: 'foo'", config._validate_keys) def test_invalid_key_value_root(self): config = self.get_config("bootloaders: bar") self.assertValidationError("Invalid structure in metadata. Expected " "key: value pairs, found: 'bootloaders: " "bar'", config._validate_keys) def test_invalid_key_value_bootloaders(self): config = self.get_config("\n".join([ "bootloaders:", " u_boot:", " foo: bar" ])) self.assertValidationError("Unknown key in metadata: 'bootloaders: " "u_boot: foo'", config._validate_keys) def test_invalid_key_in_board(self): config = self.get_config("\n".join([ "boards:", " pandaboard:", " foo: bar" ])) self.assertValidationError("Unknown key in metadata: " "'boards: pandaboard: foo'", config._validate_keys) def test_invalid_key_in_board_2(self): config = self.get_config("\n".join([ "boards:", " pandaboard:", " name: bar", " snowball:", " foo: bar", ])) self.assertValidationError("Unknown key in metadata: " "'boards: snowball: foo'", config._validate_keys) def test_valid_samsung_bl1_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL1_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl1_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL1_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl2_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL2_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_bl2_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_BL2_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_env_len_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_ENV_LEN_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_valid_samsung_env_start_field(self): config = self.get_config(self.valid_start_v3 + SAMSUNG_ENV_START_FIELD + ': 1\n') self.assertEqual(None, config._validate_keys()) def test_samsung_field_wrong(self): config = self.get_config(self.valid_start_v3 + 'samsung_wrong_field: 1\n') self.assertRaises(HwpackConfigError, config._validate_keys) # Tests for dtb_files support def test_dtb_files(self): dtb_files = ('dtb_files:\n' + ' - adest.dtb : boot/dt-*-linaro-omap/omap4-panda.dtb\n' + ' - bdest.dtb : ' + 'boot/dt-*-linaro-omap2/omap4-panda2.dtb\n') expected = [{'adest.dtb': 'boot/dt-*-linaro-omap/omap4-panda.dtb'}, {'bdest.dtb': 'boot/dt-*-linaro-omap2/omap4-panda2.dtb'}] config = self.get_config(self.valid_complete_v3 + dtb_files) config.validate() self.assertEqual(expected, config.dtb_files) def test_dtb_files_one_wrong(self): dtb_files = ('dtb_files:\n' + ' - adest.dtb : boot/dt-*-linaro-omap/omap4-panda.dtb\n' + ' - bdest.dtb : ~~~\n') config = self.get_config(self.valid_start_v3 + dtb_files) self.assertRaises(HwpackConfigError, config._validate_dtb_files) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_builder.py0000644000175000017500000005305212724020110027242 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import tarfile from testtools import TestCase from testtools.matchers import Equals from linaro_image_tools.hwpack.builder import ( ConfigFileMissing, HardwarePackBuilder, logger as builder_logger, ) from linaro_image_tools.hwpack.package_unpacker import PackageUnpacker from linaro_image_tools.hwpack.config import HwpackConfigError from linaro_image_tools.hwpack.hardwarepack import Metadata from linaro_image_tools.hwpack.packages import ( FetchedPackage, PackageMaker, ) from linaro_image_tools.hwpack.tarfile_matchers import TarfileHasFile from linaro_image_tools.hwpack.testing import ( AppendingHandler, AptSourceFixture, ChdirToTempdirFixture, ConfigFileFixture, ContextManagerFixture, DummyFetchedPackage, EachOf, IsHardwarePack, MatchesStructure, Not, ) from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( MockSomethingFixture, MockCmdRunnerPopenFixture, ) class ConfigFileMissingTests(TestCase): def test_str(self): exc = ConfigFileMissing("path") self.assertEqual("No such config file: 'path'", str(exc)) class PackageUnpackerTests(TestCaseWithFixtures): def test_creates_tempdir(self): with PackageUnpacker() as package_unpacker: self.assertTrue(os.path.exists(package_unpacker.tempdir)) def test_tempfiles_are_removed(self): tempdir = None with PackageUnpacker() as package_unpacker: tempdir = package_unpacker.tempdir self.assertFalse(os.path.exists(tempdir)) def test_unpack_package(self): fixture = MockCmdRunnerPopenFixture(assert_child_finished=False) self.useFixture(fixture) package_file_name = "package-to-unpack" with PackageUnpacker() as package_unpacker: package_unpacker.unpack_package(package_file_name) package_dir = package_unpacker.get_path(package_file_name) self.assertEquals( ["tar -C %s -xf -" % package_dir, "dpkg --fsys-tarfile %s" % package_file_name], fixture.mock.commands_executed) def test_get_file_returns_tempfile(self): package = 'package' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.useFixture(MockSomethingFixture( os.path, 'exists', lambda file: True)) tempfile = package_unpacker.get_file(package, file) self.assertEquals(tempfile, os.path.join(package_unpacker.get_path(package), file)) def test_get_file_raises(self): package = 'package' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.assertRaises(AssertionError, package_unpacker.get_file, package, file) def test_get_file_no_clash(self): # Test that PackageUnpacker, asked to get the same file path # from 2 different packages, return reference to *different* # temporary files package1 = 'package1' package2 = 'package2' file = 'dummyfile' with PackageUnpacker() as package_unpacker: self.useFixture(MockSomethingFixture( package_unpacker, 'unpack_package', lambda package: None)) self.useFixture(MockSomethingFixture( os.path, 'exists', lambda file: True)) tempfile1 = package_unpacker.get_file(package1, file) tempfile2 = package_unpacker.get_file(package2, file) self.assertNotEquals(tempfile1, tempfile2) class HardwarePackBuilderTests(TestCaseWithFixtures): config_v3 = "\n".join(["format: 3.0", "name: ahwpack", "architectures: armel", "serial_tty: ttySAC1", "partition_layout:", " - bootfs_rootfs", "boot_script: boot.scr", "mmc_id: 0:1", "kernel_file: boot/vmlinuz-*-linaro-omap", "initrd_file: boot/initrd.img-*-linaro-omap", "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb", "packages:", " - %s", " - %s", ""]) bootloader_config = "\n".join([" package: %s", " in_boot_part: %s", ""]) def setUp(self): super(HardwarePackBuilderTests, self).setUp() self.useFixture(ChdirToTempdirFixture()) self.extra_config = { 'format': '2.0', 'u-boot-package': 'wanted-package', 'u-boot-file': 'wanted-file', 'partition_layout': 'bootfs_rootfs', 'x_loader_package': 'x-loader-omap4-panda', 'x_loader_file': 'usr/lib/x-loader/omap4430panda/MLO', 'kernel_file': 'boot/vmlinuz-3.0.0-1002-linaro-omap', 'initrd_file': 'boot/initrd.img-3.0.0-1002-linaro-omap', 'boot_script': 'boot.scr', 'mmc_id': '0:1', 'u_boot_in_boot_part': 'no'} def test_raises_on_missing_configuration(self): e = self.assertRaises( ConfigFileMissing, HardwarePackBuilder, "nonexistant", "1.0", []) self.assertEqual("nonexistant", e.filename) def test_validates_configuration(self): config = self.useFixture(ConfigFileFixture('')) self.assertRaises( HwpackConfigError, HardwarePackBuilder, config.filename, "1.0", []) def makeMetaDataAndConfigFixture( self, packages, sources, hwpack_name="ahwpack", hwpack_version="1.0", architecture="armel", extra_config={}): config_text = ( '[hwpack]\n' 'name=%s\n' 'packages=%s\n' 'architectures=%s\n' % (hwpack_name, ' '.join(packages), architecture)) for key, value in extra_config.iteritems(): config_text += '%s=%s\n' % (key, value) config_text += '\n' for source_id, source in sources.iteritems(): config_text += '\n' config_text += '[%s]\n' % source_id config_text += 'sources-entry=%s\n' % source config = self.useFixture(ConfigFileFixture(config_text)) return Metadata(hwpack_name, hwpack_version, architecture), config def test_find_fetched_package_finds(self): package_name = "dummy-package" wanted_package_name = "wanted-package" available_package = DummyFetchedPackage(package_name, "1.1") wanted_package = DummyFetchedPackage(wanted_package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package, wanted_package]) _, config = self.makeMetaDataAndConfigFixture( [package_name, wanted_package_name], sources_dict, extra_config=self.extra_config) builder = HardwarePackBuilder(config.filename, "1.0", []) found_package = builder.find_fetched_package( [available_package, wanted_package], wanted_package_name) self.assertEquals(wanted_package, found_package) def test_find_fetched_package_raises(self): package_name = "dummy-package" wanted_package_name = "wanted-package" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) _, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config=self.extra_config) builder = HardwarePackBuilder(config.filename, "1.0", []) packages = [available_package] self.assertRaises(AssertionError, builder.find_fetched_package, packages, wanted_package_name) def test_creates_external_manifest(self): available_package = DummyFetchedPackage("foo", "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( ["foo"], sources_dict) builder = HardwarePackBuilder(config.filename, "1.0", []) builder.build() self.assertTrue( os.path.isfile("hwpack_ahwpack_1.0_armel.manifest.txt")) def sourcesDictForPackages(self, packages): source = self.useFixture(AptSourceFixture(packages)) return {'ubuntu': source.sources_entry} def test_builds_one_pack_per_arch(self): available_package = DummyFetchedPackage("foo", "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( ["foo"], sources_dict, architecture="i386 armel") builder = HardwarePackBuilder(config.filename, "1.0", []) builder.build() self.assertTrue(os.path.isfile("hwpack_ahwpack_1.0_i386.tar.gz")) self.assertTrue(os.path.isfile("hwpack_ahwpack_1.0_armel.tar.gz")) def test_builds_correct_contents(self): package_name = "foo" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) def test_builds_correct_contents_multiple_packages(self): package_name1 = "foo" package_name2 = "goo" available_package1 = DummyFetchedPackage(package_name1, "1.1") available_package2 = DummyFetchedPackage(package_name2, "1.2") sources_dict = self.sourcesDictForPackages( [available_package1, available_package2]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name1, package_name2], sources_dict) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() hwpack_filename = "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture) self.assertThat( hwpack_filename, IsHardwarePack( metadata, [available_package1, available_package2], sources_dict, package_spec='%s, %s' % (package_name1, package_name2))) self.assertThat( hwpack_filename, IsHardwarePack( metadata, [available_package2, available_package1], sources_dict, package_spec='%s, %s' % (package_name1, package_name2))) def test_obeys_include_debs(self): package_name = "foo" available_package = DummyFetchedPackage(package_name, "1.1") sources_dict = self.sourcesDictForPackages([available_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'include-debs': 'no'}) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, packages_without_content=[available_package], package_spec=package_name)) def test_obeys_assume_installed(self): package_name = "foo" assume_installed = "bar" available_package = DummyFetchedPackage( package_name, "1.1", depends=assume_installed) dependency_package = DummyFetchedPackage(assume_installed, "1.1") sources_dict = self.sourcesDictForPackages( [available_package, dependency_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'assume-installed': assume_installed}) builder = HardwarePackBuilder(config.filename, metadata.version, []) builder.build() filename = "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture) self.assertThat( filename, IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) tf = tarfile.open(filename, mode="r:gz") try: self.assertThat( tf, Not(TarfileHasFile("pkgs/%s" % dependency_package.filename))) finally: tf.close() def test_includes_local_debs(self): package_name = "foo" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) local_path = maker.make_package(package_name, "1.2", {}) available_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [available_package], sources_dict, package_spec=package_name)) def test_prefers_local_debs(self): package_name = "foo" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) # The point here is that remote_package has a later version than # local_package, but local_package is still preferred. remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(package_name, "1.0", {}) local_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [local_package], sources_dict, package_spec=package_name)) def test_includes_local_debs_even_if_not_in_config(self): package_name = "foo" local_name = "bar" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(local_name, "1.0", {}) local_package = FetchedPackage.from_deb(local_path) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [remote_package, local_package], sources_dict, package_spec=package_name)) def test_warn_if_not_including_local_deb(self): package_name = "foo" local_name = "bar" maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) remote_package = DummyFetchedPackage(package_name, "1.1") local_path = maker.make_package(local_name, "1.0", {}) sources_dict = self.sourcesDictForPackages([remote_package]) metadata, config = self.makeMetaDataAndConfigFixture( [package_name], sources_dict, extra_config={'assume-installed': local_name}) builder = HardwarePackBuilder( config.filename, metadata.version, [local_path]) handler = AppendingHandler() builder_logger.addHandler(handler) self.addCleanup(builder_logger.removeHandler, handler) builder.build() self.assertThat( "hwpack_%s_%s_%s.tar.gz" % ( metadata.name, metadata.version, metadata.architecture), IsHardwarePack( metadata, [remote_package], sources_dict, package_spec=package_name)) self.assertThat( handler.messages, EachOf([MatchesStructure(levelname=Equals('WARNING'))])) self.assertThat( handler.messages[0].getMessage(), Equals("Local package 'bar' not included")) def test_global_and_board_bootloader(self): package_names = ['package0', 'package1', 'package2', 'package3'] files = { package_names[0]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "usr/share/doc/u-boot-linaro-omap4-panda/copyright"], package_names[1]: ["usr/lib/u-boot/omap4_panda/u-boot.img", "some/path/file"], package_names[2]: [], package_names[3]: [], } config_v3 = self.config_v3 + "\n".join([ "bootloaders:", " u_boot:", self.bootloader_config, " file: " + files[package_names[0]][0], " copy_files:", " " + package_names[2] + ":", " - some_file", "boards:", " board1:", " bootloaders:", " u_boot:", " package: %s", " file: " + files[package_names[1]][0], " copy_files:", " " + package_names[3] + ":", " - some_file", " in_boot_part: true", "sources:", " ubuntu: %s"]) # Generate some test packages available_packages = [] maker = PackageMaker() self.useFixture(ContextManagerFixture(maker)) for package_name in package_names: # The files parameter to make_package is a list of files to create. # These files are text files containing package_name and their # path. Since package_name is different for each package, this # gives each file a unique content. deb_file_path = maker.make_package(package_name, '1.0', {}, files=files[package_name]) dummy_package = DummyFetchedPackage( package_name, "1.0", content=open(deb_file_path).read()) available_packages.append(dummy_package) source = self.useFixture(AptSourceFixture(available_packages)) # Generate a V3 config config_v3 = config_v3 % (package_names[0], package_names[1], package_names[0], "True", package_names[1], source.sources_entry) config_file_fixture = self.useFixture(ConfigFileFixture(config_v3)) # Build a hardware pack builder = HardwarePackBuilder( config_file_fixture.filename, "1.0", [os.path.join(source.rootdir, package.filepath) for package in available_packages]) builder.build() stored_package_names = [p.name for p in builder.packages] for package_name in package_names: self.assertIn(package_name, stored_package_names) # Read the contents of the hardware pack, making sure it is as expected tf = tarfile.open("hwpack_ahwpack_1.0_armel.tar.gz", mode="r:gz") # We check the content of each file when content != None. For our test # files this is " " so they can be # uniquely identified. expected_files = [ ("u_boot/" + files[package_names[0]][0], package_names[0] + " " + files[package_names[0]][0]), ("board1/u_boot/" + files[package_names[1]][0], package_names[1] + " " + files[package_names[1]][0])] for expected_file, contents in expected_files: self.assertThat( tf, TarfileHasFile(expected_file, content=contents)) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/test_script.py0000644000175000017500000002052612724020110027120 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import subprocess from testtools.matchers import DocTestMatches from linaro_image_tools.hwpack.hardwarepack import Metadata from linaro_image_tools.hwpack.testing import ( AptSourceFixture, ChdirToTempdirFixture, ConfigFileFixture, DummyFetchedPackage, IsHardwarePack, ) from linaro_image_tools.hwpack.config import Config from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.utils import find_command import os from StringIO import StringIO class ScriptTests(TestCaseWithFixtures): """Tests that execute the linaro-hwpack-create script.""" def setUp(self): super(ScriptTests, self).setUp() # Work out root of checkout. # We do this here because when running in PyCharm # the assumption in find_command that os.path.isabs(__file__) is # only true when not running from a Bazaar checkout is broken. # Thankfully find_command allows us to work around this by specifying # prefer_dir. dir = os.path.dirname(__file__) while True: path = os.path.join(dir, "linaro-hwpack-create") if dir == "/" or dir == "": # Didn't find linaro-media-create. Continue as if we haven't # tried to work out prefer_dir. dir = None break if os.path.exists(path) and os.access(path, os.X_OK): break dir = os.path.split(dir)[0] self.script_path = find_command("linaro-hwpack-create", prefer_dir=dir) self.useFixture(ChdirToTempdirFixture()) def run_script(self, args, expected_returncode=0): cmdline = [self.script_path] + args proc = subprocess.Popen( cmdline, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEqual( expected_returncode, proc.returncode, "%s exited with code %d. stdout: %s\nstderr: %s\n" % (str(cmdline), proc.returncode, stdout, stderr)) return stdout, stderr def test_errors_on_missing_configfile(self): stdout, stderr = self.run_script( ["nonexistant", "1.0"], expected_returncode=1) self.assertEqual("No such config file: 'nonexistant'\n", stderr) self.assertEqual("", stdout) def test_errors_on_missing_configfile_argument(self): stdout, stderr = self.run_script([], expected_returncode=2) self.assertEqual( "linaro-hwpack-create: error: too few arguments", stderr.splitlines()[-1]) self.assertEqual("", stdout) def test_errors_on_missing_version_argument(self): stdout, stderr = self.run_script(["somefile"], expected_returncode=2) self.assertEqual( "linaro-hwpack-create: error: too few arguments", stderr.splitlines()[-1]) self.assertEqual("", stdout) def test_builds_a_hwpack(self): package_name = 'foo' available_package = DummyFetchedPackage( package_name, "1.1", architecture="armel") source = self.useFixture(AptSourceFixture([available_package])) config = self.useFixture(ConfigFileFixture( '[hwpack]\nname=ahwpack\npackages=%s\narchitectures=armel\n' '\n[ubuntu]\nsources-entry=%s\n' % ( package_name, source.sources_entry))) stdout, stderr = self.run_script([config.filename, "1.0"]) metadata = Metadata("ahwpack", "1.0", "armel") self.assertThat( "hwpack_ahwpack_1.0_armel.tar.gz", IsHardwarePack( metadata, [available_package], {"ubuntu": source.sources_entry}, package_spec=package_name)) def test_builds_a_v3_hwpack_from_config_with_2_bootloaders(self): config_v3 = ("format: 3.0\n" "name: ahwpack\n" "architectures: armel\n" "serial_tty: ttySAC1\n" "partition_layout:\n" " - bootfs_rootfs\n" "boot_script: boot.scr\n" "extra_serial_options:\n" " - console=tty0\n" " - console=ttyO2,115200n8\n" "mmc_id: 0:1\n" "kernel_file: boot/vmlinuz-*-linaro-omap\n" "initrd_file: boot/initrd.img-*-linaro-omap\n" "dtb_file: boot/dt-*-linaro-omap/omap4-panda.dtb\n" "packages:\n" " - %s\n" " - %s\n") bootloader_config = (" package: %s\n" " in_boot_part: %s\n" " extra_boot_options:\n" " - earlyprintk\n" " - fixrtc\n" " - nocompcache\n" " - vram=48M\n" " - omapfb.vram=0:24M\n" " - mem=456M@0x80000000\n" " - mem=512M@0xA0000000\n") config_v3 += ("bootloaders:\n" " u_boot:\n" + bootloader_config + " u_boot_2:\n" + bootloader_config) config_v3 += ("sources:\n" " ubuntu: %s\n") package_names = ['foo', 'bar'] available_packages = [] for package_name in package_names: available_packages.append( DummyFetchedPackage(package_name, "1.1", architecture="armel")) source = self.useFixture(AptSourceFixture(available_packages)) config_v3 = config_v3 % (package_names[0], package_names[1], package_names[0], "True", package_names[1], "False", source.sources_entry) config_file_fixture = self.useFixture(ConfigFileFixture(config_v3)) self.run_script([config_file_fixture.filename, "1.0"]) # We now need a real config object to test against the configuration # in the hardware pack we have created. config = Config(StringIO(config_v3)) config.bootloader = "u_boot" metadata = Metadata.from_config(config, "1.0", "armel") self.assertThat( "hwpack_ahwpack_1.0_armel.tar.gz", IsHardwarePack( metadata, available_packages, {"ubuntu": source.sources_entry}, package_spec=",".join(package_names), format="3.0")) def test_log_output(self): package_name = 'foo' architecture = 'armel' available_package = DummyFetchedPackage( package_name, "1.1", architecture=architecture) source = self.useFixture(AptSourceFixture([available_package])) config = self.useFixture(ConfigFileFixture( '[hwpack]\nname=ahwpack\npackages=%s\narchitectures=armel\n' '\n[ubuntu]\nsources-entry=%s\n' % ( package_name, source.sources_entry))) stdout, stderr = self.run_script([config.filename, "1.0"]) # XXX Adding in the format deprecation message below is just a hack # until the test can be fixed up to test a new format hwpack. self.assertThat( stderr, DocTestMatches( "Building for %(arch)s\nFetching packages\n" "The format '1.0' is deprecated, please update your hardware " "pack configuration.\n" "Wrote hwpack_ahwpack_1.0_%(arch)s.tar.gz" "\n" % dict(arch=architecture))) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tests/__init__.py0000644000175000017500000000322512724020110026311 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import unittest def test_suite(): module_names = [ 'linaro_image_tools.hwpack.tests.test_better_tarfile', 'linaro_image_tools.hwpack.tests.test_builder', 'linaro_image_tools.hwpack.tests.test_config', 'linaro_image_tools.hwpack.tests.test_config_v3', 'linaro_image_tools.hwpack.tests.test_hardwarepack', 'linaro_image_tools.hwpack.tests.test_hwpack_converter', 'linaro_image_tools.hwpack.tests.test_hwpack_reader', 'linaro_image_tools.hwpack.tests.test_packages', 'linaro_image_tools.hwpack.tests.test_script', 'linaro_image_tools.hwpack.tests.test_tarfile_matchers', 'linaro_image_tools.hwpack.tests.test_testing', ] loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) return suite linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/testing.py0000644000175000017500000005525112724020110025073 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. from contextlib import contextmanager import hashlib import logging import os import shutil import subprocess import tempfile from StringIO import StringIO import tarfile import time from debian.deb822 import Packages from testtools.matchers import Annotate, Equals, Matcher, Mismatch from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile from linaro_image_tools.hwpack.tarfile_matchers import TarfileHasFile from linaro_image_tools.hwpack.packages import ( get_packages_file, FetchedPackage, ) @contextmanager def test_tarfile(contents=[], **kwargs): """Create a tarfile with the given contents, then re-open it for reading. This context manager creates a tarfile with the given contents, then reopens it for reading and yields it for use in a with block. When the block ends the tarfile will be closed. The contents can be specified as a list of tuples of (path, contents), where if the path ends with '/' it is considered to be a directory and the contents ignored. :param contents: the contents to put in the tarball, defaults to the empty list. :type contents: a list of tuples of (str, str) :param kwargs: keyword arguments for the better_tarfile.TarFile constructor. """ backing_file = StringIO() with writeable_tarfile(backing_file, **kwargs) as tf: for path, content in contents: if path[-1] == "/": tf.create_dir(path) else: tf.create_file_from_string(path, content) if contents: backing_file.seek(0) tf = tarfile.TarFile.open(mode="r", fileobj=backing_file) try: yield tf finally: tf.close() class DummyFetchedPackage(FetchedPackage): """A FetchedPackage with dummy information. See FetchedPackage for the instance variables. """ def __init__(self, name, version, architecture="all", depends=None, pre_depends=None, multi_arch=None, conflicts=None, recommends=None, provides=None, replaces=None, breaks=None, no_content=False, content=None): self.name = name self.version = version self.architecture = architecture self.depends = depends self.pre_depends = pre_depends self.multi_arch = multi_arch self.conflicts = conflicts self.recommends = recommends self.provides = provides self.replaces = replaces self.breaks = breaks self._no_content = no_content self._content = content self._file_path = None @property def filename(self): return "%s_%s_all.deb" % (self.name, self.version) def _content_str(self): return "Content of %s" % self.filename @property def content(self): if self._no_content: return None elif self._content is not None: return StringIO(self._content) return StringIO(self._content_str()) @property def size(self): return len(self.content.read()) @property def md5(self): md5sum = hashlib.md5() md5sum.update(self.content.read()) return md5sum.hexdigest() @property def sha256(self): sha256sum = hashlib.sha256() sha256sum.update(self.content.read()) return sha256sum.hexdigest() class AptSourceFixture(object): """A fixture that provides an apt source, with packages and indices. An apt source provides a set of package files, and a Packages file that allows apt to determine the contents of the source. :ivar sources_entry: the URI and suite to give to apt to view the source (i.e. a sources.list line without the "deb" prefix :type sources_entry: str """ def __init__(self, packages, label=None): """Create an AptSourceFixture. :param packages: a list of packages to add to the source and index. :type packages: an iterable of FetchedPackages """ self.packages = packages self.label = label def setUp(self): self.rootdir = tempfile.mkdtemp(prefix="hwpack-apt-source-") for package in self.packages: with open(os.path.join(self.rootdir, package.filename), 'wb') as f: f.write(package.content.read()) with open(os.path.join(self.rootdir, "Packages"), 'wb') as f: f.write(get_packages_file(self.packages)) if self.label is not None: subprocess.check_call( ['apt-ftparchive', '-oAPT::FTPArchive::Release::Label=%s' % self.label, 'release', self.rootdir], stdout=open(os.path.join(self.rootdir, 'Release'), 'w')) def tearDown(self): if os.path.exists(self.rootdir): shutil.rmtree(self.rootdir) @property def sources_entry(self): return "file:" + os.path.abspath(self.rootdir) + " ./" class ConfigFileFixture(object): def __init__(self, contents): self.contents = contents self.filename = None def setUp(self): fh, self.filename = tempfile.mkstemp(prefix="hwpack-test-config-") with os.fdopen(fh, 'w') as f: f.write(self.contents) def tearDown(self): if self.filename is not None and os.path.exists(self.filename): os.unlink(self.filename) class ContextManagerFixture(object): """Adapt a context manager to be a usable with `useFixture`.""" def __init__(self, context_manager): self.manager = context_manager def setUp(self): self.manager.__enter__() def tearDown(self): # It might be nice to pass exc_type, exc_value, traceback in here in # the failure case, if that's possible. self.manager.__exit__(None, None, None) class ChdirToTempdirFixture(object): def __init__(self): self._orig_dir = None self.tempdir = None def setUp(self): self.tearDown() self._orig_dir = os.getcwd() self.tempdir = tempfile.mkdtemp(prefix="hwpack-tests-") os.chdir(self.tempdir) def tearDown(self): if self._orig_dir is not None: os.chdir(self._orig_dir) self._orig_dir = None if self.tempdir is not None and os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) self.tempdir = None class MismatchesAll(Mismatch): """A mismatch with many child mismatches.""" def __init__(self, mismatches): self.mismatches = mismatches def describe(self): descriptions = ["Differences: ["] for mismatch in self.mismatches: descriptions.append(mismatch.describe()) descriptions.append("]\n") return '\n'.join(descriptions) class MatchesAll(object): def __init__(self, *matchers): self.matchers = matchers def __str__(self): return 'MatchesAll(%s)' % ', '.join(map(str, self.matchers)) def match(self, matchee): results = [] for matcher in self.matchers: mismatch = matcher.match(matchee) if mismatch is not None: results.append(mismatch) if results: return MismatchesAll(results) else: return None class Not: """Inverts a matcher.""" def __init__(self, matcher): self.matcher = matcher def __str__(self): return 'Not(%s)' % (self.matcher,) def match(self, other): mismatch = self.matcher.match(other) if mismatch is None: return MatchedUnexpectedly(self.matcher, other) else: return None class MatchedUnexpectedly: """A thing matched when it wasn't supposed to.""" def __init__(self, matcher, other): self.matcher = matcher self.other = other def describe(self): return "%r matches %s" % (self.other, self.matcher) class HardwarePackHasFile(TarfileHasFile): """A subclass of TarfileHasFile specific to hardware packs. We default to a set of attributes expected for files in a hardware pack. """ def __init__(self, path, **kwargs): """Create a HardwarePackHasFile matcher. The kwargs are the keyword arguments taken by TarfileHasFile. If they are not given then defaults will be checked: - The type should be a regular file - If the content is given then the size will be checked to ensure it indicates the length of the content correctly. - the mode is appropriate for the type. If the type is regular file this is 0644, otherwise if it is a directory then it is 0755. - the linkname should be the empty string. - the uid and gid should be 1000 - the uname and gname should be "user" and "group" respectively. :param path: the path that should be present. :type path: str """ kwargs.setdefault("type", tarfile.REGTYPE) if "content" in kwargs and kwargs['type'] != tarfile.DIRTYPE: kwargs.setdefault("size", len(kwargs["content"])) if kwargs["type"] == tarfile.DIRTYPE: kwargs.setdefault("mode", 0755) else: kwargs.setdefault("mode", 0644) kwargs.setdefault("linkname", "") kwargs.setdefault("uid", 1000) kwargs.setdefault("gid", 1000) kwargs.setdefault("uname", "user") kwargs.setdefault("gname", "group") kwargs.setdefault("mtime", time.time()) # Enough that it won't fail if the test is slow to execute, but # not enough that we can have a wildly wrong timestamp. kwargs.setdefault("mtime_skew", 100) super(HardwarePackHasFile, self).__init__(path, **kwargs) class IsHardwarePack(Matcher): def __init__(self, metadata, packages, sources, packages_without_content=None, package_spec=None, format="1.0"): self.metadata = metadata self.packages = packages self.sources = sources self.packages_without_content = packages_without_content or [] self.package_spec = package_spec self.format = format + "\n" def match(self, path): with tarfile.open(name=path, mode="r:gz") as tf: matchers = [] matchers.append(HardwarePackHasFile("FORMAT", content=self.format)) matchers.append(HardwarePackHasFile( "metadata", content=str(self.metadata))) manifest_lines = [] for package in self.packages: manifest_lines.append( "%s=%s" % (package.name, package.version)) manifest_lines.append( "%s=%s" % ( 'hwpack-' + self.metadata.name, self.metadata.version)) matchers.append( HardwarePackHasFile( "manifest", content_matcher=AfterPreproccessing( str.splitlines, MatchesSetwise(*map(Equals, manifest_lines))))) matchers.append(HardwarePackHasFile("pkgs", type=tarfile.DIRTYPE)) packages_with_content = [p for p in self.packages if p not in self.packages_without_content] for package in packages_with_content: matchers.append(HardwarePackHasFile( "pkgs/%s" % package.filename, content=package.content.read())) package_matchers = [ MatchesPackage(p) for p in packages_with_content] dep_package_matcher = MatchesStructure( name=Equals('hwpack-' + self.metadata.name), version=Equals(self.metadata.version), architecture=Equals(self.metadata.architecture), filename=Equals('hwpack-%s_%s_%s.deb' % ( self.metadata.name, self.metadata.version, self.metadata.architecture))) if self.package_spec: dep_package_matcher = dep_package_matcher.update( depends=MatchesPackageRelationshipList( [Equals(p.strip()) for p in self.package_spec.split(',')])) package_matchers.append(dep_package_matcher) matchers.append(HardwarePackHasFile( "pkgs/Packages", content_matcher=MatchesAsPackagesFile( *package_matchers))) matchers.append(HardwarePackHasFile( "sources.list.d", type=tarfile.DIRTYPE)) for source_id, sources_entry in self.sources.items(): matchers.append(HardwarePackHasFile( "sources.list.d/%s.list" % source_id, content="deb " + sources_entry + "\n")) matchers.append(HardwarePackHasFile( "sources.list.d.gpg", type=tarfile.DIRTYPE)) return MatchesAll(*matchers).match(tf) def __str__(self): return "Is a valid hardware pack." class EachOf(object): """Matches if each matcher matches the corresponding value. More easily explained by example than in words: >>> EachOf([Equals(1)]).match([1]) >>> EachOf([Equals(1), Equals(2)]).match([1, 2]) >>> EachOf([Equals(1), Equals(2)]).match([2, 1]) #doctest: +ELLIPSIS <...Mismatch...> """ def __init__(self, matchers): self.matchers = matchers def match(self, values): mismatches = [] length_mismatch = Annotate( "Length mismatch", Equals(len(self.matchers))).match(len(values)) if length_mismatch: mismatches.append(length_mismatch) for matcher, value in zip(self.matchers, values): mismatch = matcher.match(value) if mismatch: mismatches.append(mismatch) if mismatches: return MismatchesAll(mismatches) class MatchesStructure(object): """Matcher that matches an object structurally. 'Structurally' here means that attributes of the object being matched are compared against given matchers. `fromExample` allows the creation of a matcher from a prototype object and then modified versions can be created with `update`. """ def __init__(self, **kwargs): self.kws = kwargs @classmethod def fromExample(cls, example, *attributes): kwargs = {} for attr in attributes: kwargs[attr] = Equals(getattr(example, attr)) return cls(**kwargs) def update(self, **kws): new_kws = self.kws.copy() for attr, matcher in kws.iteritems(): if matcher is None: new_kws.pop(attr, None) else: new_kws[attr] = matcher return type(self)(**new_kws) def match(self, value): matchers = [] values = [] for attr, matcher in self.kws.iteritems(): matchers.append(Annotate(attr, matcher)) values.append(getattr(value, attr)) return EachOf(matchers).match(values) def MatchesPackage(example): """Create a `MatchesStructure` object from a `FetchedPackage`.""" return MatchesStructure.fromExample( example, *example._equality_attributes) class MatchesSetwise(object): """Matches if all the matchers match elements of the value being matched. The difference compared to `EachOf` is that the order of the matchings does not matter. """ def __init__(self, *matchers): self.matchers = matchers def match(self, observed): remaining_matchers = set(self.matchers) not_matched = [] for value in observed: for matcher in remaining_matchers: if matcher.match(value) is None: remaining_matchers.remove(matcher) break else: not_matched.append(value) if not_matched or remaining_matchers: remaining_matchers = list(remaining_matchers) # There are various cases that all should be reported somewhat # differently. # There are two trivial cases: # 1) There are just some matchers left over. # 2) There are just some values left over. # Then there are three more interesting cases: # 3) There are the same number of matchers and values left over. # 4) There are more matchers left over than values. # 5) There are more values left over than matchers. if len(not_matched) == 0: if len(remaining_matchers) > 1: msg = "There were %s matchers left over: " % ( len(remaining_matchers),) else: msg = "There was 1 matcher left over: " msg += ', '.join(map(str, remaining_matchers)) return Mismatch(msg) elif len(remaining_matchers) == 0: if len(not_matched) > 1: return Mismatch( "There were %s values left over: %s" % ( len(not_matched), not_matched)) else: return Mismatch( "There was 1 value left over: %s" % ( not_matched, )) else: common_length = min(len(remaining_matchers), len(not_matched)) if common_length == 0: raise AssertionError("common_length can't be 0 here") if common_length > 1: msg = "There were %s mismatches" % (common_length,) else: msg = "There was 1 mismatch" if len(remaining_matchers) > len(not_matched): extra_matchers = remaining_matchers[common_length:] msg += " and %s extra matcher" % (len(extra_matchers), ) if len(extra_matchers) > 1: msg += "s" msg += ': ' + ', '.join(map(str, extra_matchers)) elif len(not_matched) > len(remaining_matchers): extra_values = not_matched[common_length:] msg += " and %s extra value" % (len(extra_values), ) if len(extra_values) > 1: msg += "s" msg += ': ' + str(extra_values) return Annotate( msg, EachOf(remaining_matchers[:common_length]) ).match(not_matched[:common_length]) def parse_packages_file_content(file_content): packages = [] for para in Packages.iter_paragraphs(StringIO(file_content)): args = {} for key, value in para.iteritems(): key = key.lower() if key == 'md5sum': key = 'md5' elif key == 'package': key = 'name' elif key == 'size': value = int(value) if key in FetchedPackage._equality_attributes: args[key] = value packages.append(FetchedPackage(**args)) return packages class AfterPreproccessing(object): """Matches if the value matches after passing through a function.""" def __init__(self, preprocessor, matcher): self.preprocessor = preprocessor self.matcher = matcher def __str__(self): return "AfterPreproccessing(%s, %s)" % ( self.preprocessor, self.matcher) def match(self, value): value = self.preprocessor(value) return self.matcher.match(value) def MatchesAsPackagesFile(*package_matchers): """Matches the contents of a Packages file against the given matchers. The contents of the Packages file is turned into a list of FetchedPackages using `parse_packages_file_content` above. """ return AfterPreproccessing( parse_packages_file_content, MatchesSetwise(*package_matchers)) def MatchesAsPackageContent(package_matcher): """Match a package on disk against `package_matcher`.""" def load_from_disk(content): fd, path = tempfile.mkstemp() try: os.write(fd, content) os.close(fd) return FetchedPackage.from_deb(path) finally: os.remove(path) return AfterPreproccessing(load_from_disk, package_matcher) class DoesNotStartWith(Mismatch): def __init__(self, matchee, expected): """Create a DoesNotStartWith Mismatch. :param matchee: the string that did not match. :param expected: the string that `matchee` was expected to start with. """ self.matchee = matchee self.expected = expected def describe(self): return "'%s' does not start with '%s'." % ( self.matchee, self.expected) class StartsWith(Matcher): """Checks whether one string starts with another.""" def __init__(self, expected): """Create a StartsWith Matcher. :param expected: the string that matchees should start with. """ self.expected = expected def __str__(self): return "Starts with '%s'." % self.expected def match(self, matchee): if not matchee.startswith(self.expected): return DoesNotStartWith(matchee, self.expected) return None def MatchesPackageRelationshipList(relationship_matchers): """Matches a set of matchers against a package relationship specification. >>> MatchesPackageRelationshipList( ... [Equals('foo'), StartsWith('bar (')]).match('bar (= 1.0), foo') >>> """ def process(relationships): if relationships is None: return [] return [rel.strip() for rel in relationships.split(',')] return AfterPreproccessing( process, MatchesSetwise(*relationship_matchers)) class AppendingHandler(logging.Handler): """A logging handler that simply appends messages to a list.""" def __init__(self): logging.Handler.__init__(self) self.messages = [] def emit(self, message): self.messages.append(message) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/hwpack_fields.py0000644000175000017500000001244112724020110026213 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. # This file contains all the valid fields for an hwpack v3. # Reference wiki page: https://wiki.linaro.org/HardwarePacksV3 # # Try to keep it alphabetically sorted per section. # ARCHITECTURES_FIELD = 'architectures' ARCHITECTURE_FIELD = 'architecture' ASSUME_INSTALLED_FIELD = 'assume_installed' BOARDS_FIELD = 'boards' BOOTLOADERS_FIELD = 'bootloaders' BOOT_MIN_SIZE_FIELD = 'boot_min_size' BOOT_SCRIPT_FIELD = 'boot_script' COPY_FILES_FIELD = 'copy_files' DTB_ADDR_FIELD = 'dtb_addr' DTB_FILE_FIELD = 'dtb_file' DTB_FILES_FIELD = 'dtb_files' EXTRA_SERIAL_OPTIONS_FIELD = 'extra_serial_options' FORMAT_FIELD = 'format' INCLUDE_DEBS_FIELD = 'include_debs' INITRD_ADDR_FIELD = 'initrd_addr' INITRD_FILE_FIELD = 'initrd_file' KERNEL_ADDR_FIELD = 'kernel_addr' KERNEL_FILE_FIELD = 'kernel_file' LOAD_ADDR_FIELD = 'load_addr' LOADER_MIN_SIZE_FIELD = 'loader_min_size' LOADER_START_FIELD = 'loader_start' MAINTAINER_FIELD = 'maintainer' MMC_ID_FIELD = 'mmc_id' NAME_FIELD = 'name' ORIGIN_FIELD = 'origin' PACKAGES_FIELD = 'packages' PARTITION_LAYOUT_FIELD = 'partition_layout' ROOT_MIN_SIZE_FIELD = 'root_min_size' SERIAL_TTY_FIELD = 'serial_tty' SOURCES_FIELD = 'sources' SUPPORT_FIELD = 'support' WIRED_INTERFACES_FIELD = 'wired_interfaces' WIRELESS_INTERFACES_FIELD = 'wireless_interfaces' VERSION_FIELD = 'version' # Bootloaders specific fields COPY_FILES_FIELD = 'copy_files' DD_FIELD = 'dd' ENV_DD_FIELD = 'env_dd' EXTRA_BOOT_OPTIONS_FIELD = 'extra_boot_options' FILE_FIELD = 'file' IN_BOOT_PART_FIELD = 'in_boot_part' PACKAGE_FIELD = 'package' SPL_DD_FIELD = 'spl_dd' SPL_FILE_FIELD = 'spl_file' SPL_IN_BOOT_PART_FIELD = 'spl_in_boot_part' SPL_PACKAGE_FIELD = 'spl_package' # Samsung fields SAMSUNG_BL1_LEN_FIELD = 'samsung_bl1_len' SAMSUNG_BL1_START_FIELD = 'samsung_bl1_start' SAMSUNG_BL2_LEN_FIELD = 'samsung_bl2_len' SAMSUNG_BL2_START_FIELD = 'samsung_bl2_start' SAMSUNG_ENV_LEN_FIELD = 'samsung_env_len' SAMSUNG_ENV_START_FIELD = 'samsung_env_start' # Snowball fields SNOWBALL_STARTUP_FILES_CONFIG_FIELD = 'snowball_startup_files_config' # Fields that might be necessary for the metadata file METADATA_ARCH_FIELD = 'architecture' METADATA_VERSION_FIELD = 'version' # The allowed partition layouts. BOOTFS16 = 'bootfs16_rootfs' BOOTFS = 'bootfs_rootfs' RESERVED_BOOTFS = 'reserved_bootfs_rootfs' DEFINED_PARTITION_LAYOUTS = [ BOOTFS16, BOOTFS, RESERVED_BOOTFS, ] # Supported bootloaders U_BOOT = 'u_boot' UEFI = 'uefi' DEFAULT_BOOTLOADER = U_BOOT # Define where fields are valid, so we can test them. # If a key has a value None, this indicates there is either a value or # list of values that can be associated with it. # If a key contains a dictionary, this means that the key can # contain a dictionary. # The string "root" indicates that the key can contain the root # structure. This is used for the boards section, where each # board can contain the full or partial layout, overwriting the global # settings. hwpack_v3_layout = { FORMAT_FIELD: None, NAME_FIELD: None, ARCHITECTURES_FIELD: None, ORIGIN_FIELD: None, MAINTAINER_FIELD: None, SUPPORT_FIELD: None, ASSUME_INSTALLED_FIELD: None, INCLUDE_DEBS_FIELD: None, DTB_FILE_FIELD: None, DTB_FILES_FIELD: None, DTB_ADDR_FIELD: None, SERIAL_TTY_FIELD: None, EXTRA_SERIAL_OPTIONS_FIELD: None, MMC_ID_FIELD: None, PACKAGES_FIELD: None, PARTITION_LAYOUT_FIELD: None, KERNEL_FILE_FIELD: None, KERNEL_ADDR_FIELD: None, INITRD_FILE_FIELD: None, INITRD_ADDR_FIELD: None, LOAD_ADDR_FIELD: None, BOOT_SCRIPT_FIELD: None, LOADER_START_FIELD: None, WIRED_INTERFACES_FIELD: None, WIRELESS_INTERFACES_FIELD: None, BOOT_MIN_SIZE_FIELD: None, ROOT_MIN_SIZE_FIELD: None, LOADER_MIN_SIZE_FIELD: None, SAMSUNG_BL1_LEN_FIELD: None, SAMSUNG_BL1_START_FIELD: None, SAMSUNG_ENV_LEN_FIELD: None, SAMSUNG_ENV_START_FIELD: None, SAMSUNG_BL2_LEN_FIELD: None, SAMSUNG_BL2_START_FIELD: None, SNOWBALL_STARTUP_FILES_CONFIG_FIELD: None, SOURCES_FIELD: None, BOOTLOADERS_FIELD: { "*": { PACKAGE_FIELD: None, FILE_FIELD: None, IN_BOOT_PART_FIELD: None, COPY_FILES_FIELD: None, DD_FIELD: None, EXTRA_BOOT_OPTIONS_FIELD: None, SPL_PACKAGE_FIELD: None, SPL_FILE_FIELD: None, SPL_IN_BOOT_PART_FIELD: None, SPL_DD_FIELD: None, ENV_DD_FIELD: None, } }, BOARDS_FIELD: "root", } linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/hwpack_convert.py0000644000175000017500000003106612724020110026431 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # Author: Milo Casagrande # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import ConfigParser import logging import os import os.path import re import yaml from hwpack_fields import ( ARCHITECTURES_FIELD, ASSUME_INSTALLED_FIELD, BOOTLOADERS_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, SOURCES_FIELD, FORMAT_FIELD, PACKAGES_FIELD, PACKAGE_FIELD, FILE_FIELD, IN_BOOT_PART_FIELD, DD_FIELD, DTB_FILE_FIELD, DTB_FILES_FIELD, ENV_DD_FIELD, SPL_IN_BOOT_PART_FIELD, SPL_DD_FIELD, SPL_PACKAGE_FIELD, SPL_FILE_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, INCLUDE_DEBS_FIELD, ) # This is the main section of an INI-style hwpack config file. MAIN_SECTION = 'hwpack' # The suffix for the new file NEW_FILE_SUFFIX = '.yaml' # How many spaces should be used for indentation. INDENT_STEP = 1 # Regular expression to convert from Yes/No values into Boolean. YES_REGEX = '[Yy]es' NO_REGEX = '[Nn]o' # The default format number. DEFAULT_FORMAT = '3.0' # Old INI style u_boot keys name. UBOOT_PACKAGE_KEY = "u_boot_package" UBOOT_FILE_KEY = "u_boot_file" UBOOT_IN_BOOT_PART_KEY = 'u_boot_in_boot_part' UBOOT_DD_KEY = 'u_boot_dd' # All the u_boot defined keys in a list. UBOOT_KEYS = [UBOOT_PACKAGE_KEY, UBOOT_FILE_KEY, UBOOT_IN_BOOT_PART_KEY, UBOOT_DD_KEY] # Old field, the only one with a dash: since the format is new, convert it. ASSUME_INSTALLED_OLD = 'assume-installed' INCLUDE_DEBS_OLD = 'include-debs' # The default bootloader for the bootloaders section. DEFAULT_BOOTLOADER = 'u_boot' # All the SPL keys SPL_KEYS = [SPL_IN_BOOT_PART_FIELD, SPL_DD_FIELD, SPL_PACKAGE_FIELD, SPL_FILE_FIELD, ENV_DD_FIELD] # The default name used for renaming dtb file DEFAULT_DTB_NAME = 'board.dtb' logger = logging.getLogger(__name__) class HwpackConverterException(Exception): """General exception class for the converter.""" class HwpackConverter(object): """Simple and basic class that converts an INI-style format file into the new YAML format. The old format number is maintained. :param input_file: the input file to parse, has to be an INI-style file. :param output_file: where to write the new file, if not given, the name of the input file will be used adding the 'yaml' suffix. """ def __init__(self, input_file=None, output_file=None): """Initializie the class.""" self.input_file = input_file self.output_file = output_file # Where we store the list of sources. self.sources = {} # Where we store all the information of the hwpack config file # In this case we have one board per hwpack config file. self.hwpack = {} # List of supported architectures. self.architectures = [] # Where we hold bootloaders info self.bootloaders = {} # List to store extra boot options. self.extra_boot_options = [] # The list of packages. self.packages = [] # List of the extra_serial_options. self.extra_serial_options = [] # Lists for network interfaces. self.wired_interfaces = [] self.wireless_interfaces = [] # SPL entries self.spl = {} # The list of packages that should be installed. self.assume_installed = [] # The dtb_files section self.dtb_files = [] def _parse(self): """Parses the config file and stores its values.""" if self.input_file is not None: parser = ConfigParser.RawConfigParser() with open(self.input_file, 'r') as fp: parser.readfp(fp) # Iterate through all the file sections. for section in parser.sections(): if section == MAIN_SECTION: for key, value in parser.items(section): if value is not None: if re.match("[Yy]es", value): value = True elif re.match("[Nn]o", value): value = False if key == ARCHITECTURES_FIELD: self.parse_list_string( self.architectures, value) continue elif key == EXTRA_BOOT_OPTIONS_FIELD: self.parse_list_string( self.extra_boot_options, value) continue elif key == EXTRA_SERIAL_OPTIONS_FIELD: self.parse_list_string( self.extra_serial_options, value) continue elif key == WIRED_INTERFACES_FIELD: self.parse_list_string(self.wired_interfaces, value) continue elif key == WIRELESS_INTERFACES_FIELD: self.parse_list_string( self.wireless_interfaces, value) continue elif key in SPL_KEYS: self.spl[key] = value continue elif key == FORMAT_FIELD: value = DEFAULT_FORMAT elif key == PACKAGES_FIELD: self.parse_list_string(self.packages, value) continue elif key in UBOOT_KEYS: self._set_bootloaders(key, value) continue # Create list. elif key == ASSUME_INSTALLED_OLD: self.parse_list_string( self.assume_installed, value) continue elif key == DTB_FILE_FIELD: self.dtb_files.append({DEFAULT_DTB_NAME: value}) continue elif key == INCLUDE_DEBS_OLD: key = INCLUDE_DEBS_FIELD self.hwpack[key] = value else: # Here we have only sources sections. for _, value in parser.items(section): if value is not None: self.sources[section] = value def _set_bootloaders(self, key, value): """Sets the bootloaders dictionary of a new YAML file. Converts from the old INI keys name into the new ones. :param key: The key of the bootloader. :param value: The key value.""" if key == UBOOT_PACKAGE_KEY: self.bootloaders[PACKAGE_FIELD] = value elif key == UBOOT_FILE_KEY: self.bootloaders[FILE_FIELD] = value elif key == UBOOT_IN_BOOT_PART_KEY: self.bootloaders[IN_BOOT_PART_FIELD] = value elif key == UBOOT_DD_KEY: self.bootloaders[DD_FIELD] = value def parse_list_string(self, store, string, split=" "): """Parses a string of listed values, and stores the single splitted value in the provided list. :param store: The list where to store the values. :param string: The string that should be splitted. :param split: The separator to use, defaults to empty space. """ if not isinstance(store, list): raise HwpackConverterException("Can use this method only with " "list.") store.extend(string.split(" ")) def _to_file(self): """Writes the converted hwpack to file.""" with open(self.output_file, 'w') as fp: fp.write(str(self)) def convert(self): """Converts the input file into the output file with the new format. """ self._parse() self._to_file() def __str__(self): """Readable representation of the converted hwpack. :return A YAML-string representation of the hwpack configuration. """ converted = '' if self.hwpack: converted += dump(self.hwpack) if self.architectures: archs = {ARCHITECTURES_FIELD: self.architectures} converted += dump(archs) if self.assume_installed: installed = {ASSUME_INSTALLED_FIELD: self.assume_installed} converted += dump(installed) if self.extra_serial_options: serial_options = {EXTRA_SERIAL_OPTIONS_FIELD: self.extra_serial_options} converted += dump(serial_options) if self.packages: packages = {PACKAGES_FIELD: self.packages} converted += dump(packages) if self.wired_interfaces: wired = {WIRED_INTERFACES_FIELD: self.wired_interfaces} converted += dump(wired) if self.wireless_interfaces: converted += dump(self.wireless_interfaces) if self.sources: sources = {SOURCES_FIELD: self.sources} converted += dump(sources) if self.dtb_files: dtb = {DTB_FILES_FIELD: self.dtb_files} converted += dump(dtb) if self.bootloaders or self.extra_boot_options or self.spl: # The bootloaders section in the new YAML file is a dictionary # containing a dictionary which can contains also other # dictionaries. In this case we only have list and normal values. nested_value = {} if self.bootloaders: for key, value in self.bootloaders.iteritems(): nested_value[key] = value if self.extra_boot_options: nested_value[EXTRA_BOOT_OPTIONS_FIELD] = \ self.extra_boot_options if self.spl: for key, value in self.spl.iteritems(): nested_value[key] = value default_bootloader = {DEFAULT_BOOTLOADER: nested_value} bootloaders = {BOOTLOADERS_FIELD: default_bootloader} converted += dump(bootloaders) return converted def dump(python_object): """Serialize a Python object in a YAML string format. :param python_object: The object to serialize. """ return yaml.dump(python_object, default_flow_style=False) def check_and_validate_args(args): """Assures that the args passed are valid. :param args: the args as defined in linaro-hwpack-convert. """ input_file = args.CONFIG_FILE output_file = args.out if not os.path.exists(input_file) or not os.path.isfile(input_file): raise HwpackConverterException("The configuration file '%s' is not a " "regular file." % input_file) if output_file is not None: if os.path.exists(output_file) or os.path.isdir(output_file): raise HwpackConverterException("The output file name provided " "'%s' already exists, or is a " "directory." % output_file) elif not os.path.isabs(output_file): # If we output file is just a name, write it in the current dir. output_file = os.path.join(os.getcwd(), output_file) else: output_file = input_file + NEW_FILE_SUFFIX return (input_file, output_file) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/config.py0000644000175000017500000014151412724020110024661 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import ConfigParser from operator import attrgetter import os import re import string import yaml from linaro_image_tools.hwpack.hardwarepack_format import ( HardwarePackFormatV1, HardwarePackFormatV2, HardwarePackFormatV3, ) from hwpack_fields import ( ARCHITECTURES_FIELD, ARCHITECTURE_FIELD, ASSUME_INSTALLED_FIELD, BOARDS_FIELD, BOOTLOADERS_FIELD, BOOT_MIN_SIZE_FIELD, BOOT_SCRIPT_FIELD, COPY_FILES_FIELD, DD_FIELD, DTB_ADDR_FIELD, DTB_FILE_FIELD, DEFAULT_BOOTLOADER, DTB_FILES_FIELD, ENV_DD_FIELD, EXTRA_BOOT_OPTIONS_FIELD, EXTRA_SERIAL_OPTIONS_FIELD, FILE_FIELD, FORMAT_FIELD, INCLUDE_DEBS_FIELD, IN_BOOT_PART_FIELD, INITRD_ADDR_FIELD, INITRD_FILE_FIELD, KERNEL_ADDR_FIELD, KERNEL_FILE_FIELD, LOAD_ADDR_FIELD, LOADER_MIN_SIZE_FIELD, LOADER_START_FIELD, MAINTAINER_FIELD, MMC_ID_FIELD, NAME_FIELD, ORIGIN_FIELD, PACKAGE_FIELD, PACKAGES_FIELD, PARTITION_LAYOUT_FIELD, ROOT_MIN_SIZE_FIELD, SAMSUNG_BL1_LEN_FIELD, SAMSUNG_BL1_START_FIELD, SAMSUNG_BL2_LEN_FIELD, SAMSUNG_BL2_START_FIELD, SAMSUNG_ENV_LEN_FIELD, SAMSUNG_ENV_START_FIELD, SERIAL_TTY_FIELD, SNOWBALL_STARTUP_FILES_CONFIG_FIELD, SOURCES_FIELD, SPL_DD_FIELD, SPL_FILE_FIELD, SPL_IN_BOOT_PART_FIELD, SPL_PACKAGE_FIELD, SUPPORT_FIELD, WIRED_INTERFACES_FIELD, WIRELESS_INTERFACES_FIELD, DEFINED_PARTITION_LAYOUTS, VERSION_FIELD, hwpack_v3_layout, ) import logging class HwpackConfigError(Exception): pass class Config(object): """Encapsulation of a hwpack-create configuration.""" translate_v2_to_v3 = {} translate_v2_metadata = {} MAIN_SECTION = "hwpack" NAME_REGEX = r"[a-z0-9][a-z0-9+\-.]+$" SOURCES_ENTRY_KEY = "sources-entry" PACKAGE_REGEX = NAME_REGEX PATH_REGEX = r"\w[\w+\-./_]+$" GLOB_REGEX = r"[\w+\-./_\*]+$" INCLUDE_DEBS_KEY = "include-debs" translate_v2_to_v3[INCLUDE_DEBS_KEY] = INCLUDE_DEBS_FIELD translate_v2_metadata[ARCHITECTURES_FIELD] = "ARCHITECTURE" ASSUME_INSTALLED_KEY = "assume-installed" translate_v2_to_v3[ASSUME_INSTALLED_KEY] = ASSUME_INSTALLED_FIELD BOOTLOADER_PACKAGE_KEY = "u_boot_package" translate_v2_to_v3[BOOTLOADER_PACKAGE_KEY] = PACKAGE_FIELD BOOTLOADER_FILE_KEY = "u_boot_file" translate_v2_to_v3[BOOTLOADER_FILE_KEY] = FILE_FIELD translate_v2_metadata[BOOTLOADER_FILE_KEY] = "U_BOOT" SPL_FILE_KEY = "spl_file" translate_v2_metadata[SPL_FILE_KEY] = "SPL" BOOTLOADER_IN_BOOT_PART_KEY = 'u_boot_in_boot_part' translate_v2_to_v3[BOOTLOADER_IN_BOOT_PART_KEY] = IN_BOOT_PART_FIELD BOOTLOADER_DD_KEY = 'u_boot_dd' translate_v2_to_v3[BOOTLOADER_DD_KEY] = DD_FIELD last_used_keys = [] board = None def __init__(self, fp, bootloader=None, board=None, allow_unset_bootloader=False): """Create a Config. :param fp: a file-like object containing the configuration. :param allow_unset_bootloader: Bool. If you have more than 1 bootloader in the config object and don't set which one to use, accessing bootloader related parameters will throw an exception. By setting this None will be returned instead. """ # This Config class is used in two places: # 1. Generating hardware packs # 2. Combining a hardware pack with an OS image to create a bootable # disk image. # # In both cases we are providing a file format independant interface # to configuration data to the rest of Linaro Image Tools. # # In case 1 we want all information to be put in the hardware pack and # there is no possibility of picking a booloader (all bootloaders are # put in the hardware pack and one is picked later). In this case we # don't want to trip up other code by throwing an exception when a # bootloader dependant parameter is queried so we return None. In # reality this information isn't used, but sometimes gets queried by # tests. This flag allows us to keep things simple. # # In case 2 we may have multiple bootloaders specified, but only one # can be used by the OS image so we need to pick one. If a bootloader # isn't specified we want to throw an error when a choice would make a # difference to what is returned when querying the object. # # self.allow_unset_bootloader allows for both modes of operation. self.logger = logging.getLogger('linaro_image_tools') self.allow_unset_bootloader = allow_unset_bootloader self.board = board self._bootloader = bootloader obfuscated_e = None obfuscated_yaml_e = "" try: self.parser = ConfigParser.RawConfigParser() self.parser.readfp(fp) except ConfigParser.Error, e: obfuscated_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) if obfuscated_e: # obfuscated_e being set indicates that something went wrong. # It could be that the input is in fact YAML. Try the YAML # parser. try: fp.seek(0) self.parser = yaml.safe_load(fp) except yaml.YAMLError, e: obfuscated_yaml_e = re.sub(r"([^ ]https://).+?(@)", r"\1***\2", str(e)) else: # If YAML parsed OK, we don't have an error. obfuscated_e = None if obfuscated_e: # If INI parsing from ConfigParser or YAML parsing failed, # print both error messages. msg = ("Failed to parse hardware pack configuration. Tried to " "parse as both INI and YAML. INI parsing error:\n" + obfuscated_e + "\n" + "YAML parser error:\n" + obfuscated_yaml_e) raise ConfigParser.Error(msg) def _get_bootloader(self): """Returns the bootloader associated with this config. If bootloader is None / empty and there is only one bootloader available, use that.""" bootloader = self._bootloader if not bootloader: # Auto-detect bootloader. If there is a single bootloader specified # then use it, else, error. bootloaders = self.bootloaders if isinstance(bootloaders, dict): # We have a list of bootloaders in the expected format bootloaders = bootloaders.keys() bootloader = bootloaders[0] if len(bootloaders) > 1: # We have more than one bootloader, use 'u_boot'. if DEFAULT_BOOTLOADER in bootloaders: bootloader = DEFAULT_BOOTLOADER self.logger.warning('WARNING: no bootloader specified ' 'on the command line. Defaulting ' 'to \'%s\'.' % DEFAULT_BOOTLOADER) self.logger.warning('WARNING: specify another ' 'bootloader if this is not the ' 'correct one to use.') else: self.logger.warning('Default bootloader \'%s\' not ' 'found. Will try to use \'%s\'. ' 'instead.' % (DEFAULT_BOOTLOADER, bootloader)) # bootloader is None: since we are here, set it so we do not # have to go through all the config retrieval again. self._bootloader = bootloader return bootloader def _set_bootloader(self, value): """Set bootloader used to look up configuration in bootloader section. """ self._bootloader = value bootloader = property(_get_bootloader, _set_bootloader) def get_bootloader_list(self): if isinstance(self.bootloaders, dict): # We have a list of bootloaders in the expected format return self.bootloaders.keys() return [] def validate_bootloader_fields(self): self._validate_bootloader_package() self._validate_bootloader_file() self._validate_spl_package() self._validate_spl_file() self._validate_bootloader_file_in_boot_part() self._validate_bootloader_dd() self._validate_spl_in_boot_part() self._validate_spl_dd() self._validate_env_dd() def validate(self): """Check that this configuration follows the schema. :raises HwpackConfigError: if it does not. """ if isinstance(self.parser, ConfigParser.RawConfigParser): if not self.parser.has_section(self.MAIN_SECTION): raise HwpackConfigError("No [%s] section" % self.MAIN_SECTION) self._validate_keys() self._validate_format() self._validate_name() self._validate_include_debs() self._validate_support() self._validate_packages() self._validate_architectures() self._validate_assume_installed() if self.format.has_v2_fields: # Check config for all bootloaders if one isn't specified. if not self.bootloader and self._is_v3: for bootloader in self.get_bootloader_list(): self.bootloader = bootloader self.validate_bootloader_fields() else: self.validate_bootloader_fields() self._validate_serial_tty() self._validate_kernel_addr() self._validate_initrd_addr() self._validate_load_addr() self._validate_dtb_addr() self._validate_wired_interfaces() self._validate_wireless_interfaces() self._validate_partition_layout() self._validate_boot_min_size() self._validate_root_min_size() self._validate_loader_min_size() self._validate_loader_start() self._validate_vmlinuz() self._validate_initrd() self._validate_dtb_file() self._validate_dtb_files() self._validate_mmc_id() self._validate_extra_boot_options() self._validate_boot_script() self._validate_extra_serial_options() self._validate_snowball_startup_files_config() self._validate_samsung_bl1_start() self._validate_samsung_bl1_len() self._validate_samsung_env_start() self._validate_samsung_env_len() self._validate_samsung_bl2_start() self._validate_samsung_bl2_len() self._validate_sources() @property def format(self): """The format of the hardware pack. A subclass of HardwarePackFormat. """ if isinstance(self.parser, ConfigParser.RawConfigParser): try: format_string = self.parser.get(self.MAIN_SECTION, FORMAT_FIELD) except (ConfigParser.NoOptionError, ConfigParser.NoSectionError): # Default to 1.0 to aviod breaking existing hwpack files. # When this code no longer supports 1.0, it effectively makes # explicitly specifying format in hwpack files mandatory. format_string = "1.0" else: format_string = self.parser.get(FORMAT_FIELD) if format_string == '1.0': return HardwarePackFormatV1() elif format_string == '2.0': return HardwarePackFormatV2() elif format_string == 3.0 or format_string == '3.0': return HardwarePackFormatV3() else: raise HwpackConfigError("Format version '%s' is not supported." % format_string) @property def name(self): """The name of the hardware pack. A str.""" return self._get_option(NAME_FIELD) @property def version(self): return self._get_option(VERSION_FIELD) @property def include_debs(self): """Whether the hardware pack should contain .debs. A bool.""" try: if self._get_option(self.INCLUDE_DEBS_KEY) is None: return True try: return self._get_option_bool(self.INCLUDE_DEBS_KEY) except ValueError as e: raise HwpackConfigError("Invalid value for include-debs: %s" % e) except ConfigParser.NoOptionError: return True @property def bootloaders(self): """Bootloaders available in the hardware pack""" return self._get_option(BOOTLOADERS_FIELD) @property def boards(self): """Multiple boards available in the hardware pack.""" return self._get_option(BOARDS_FIELD) @property def bootloader_file_in_boot_part(self): """Whether uboot binary should be put in the boot partition. A str.""" return self._get_bootloader_option(self.BOOTLOADER_IN_BOOT_PART_KEY) @property def bootloader_dd(self): """If the uboot binary should be dd:d to the boot partition this field specifies the offset. An int.""" return self._get_bootloader_option(self.BOOTLOADER_DD_KEY) @property def spl_in_boot_part(self): """Whether spl binary should be put in the boot partition. A str.""" return self._get_bootloader_option(SPL_IN_BOOT_PART_FIELD) @property def bootloader_copy_files(self): """Extra files to copy to boot partition. This can be stored in several formats. We always present in a common one: {source_package: [{source_file_path: dest_file_path}]. dest_file_path (in the above example) is always absolute. """ #copy_files: # source_package: # - source_file_path : dest_file_path # - source_file_without_explicit_destination #copy_files: # - file1 # - file2: dest_path # # Note that the list of files is always that - a list. copy_files = self._get_bootloader_option(COPY_FILES_FIELD) if copy_files is None: return None if not isinstance(copy_files, dict): copy_files = {self.bootloader_package: copy_files} for package in copy_files: new_list = [] for value in copy_files[package]: if not isinstance(value, dict): dest_path = "/boot" source_path = value else: if len(value.keys()) > 1: raise HwpackConfigError("copy_files entry found with" "more than one destination") source_path = value.keys()[0] dest_path = value[source_path] if not dest_path.startswith("/boot"): # Target path should be relative, or start with /boot - we # don't support to copying to anywhere other than /boot. if dest_path[0] == "/": raise HwpackConfigError( "copy_files destinations must" "be relative to /boot or start with /boot.") dest_path = os.path.join("/boot", dest_path) new_list.append({source_path: dest_path}) copy_files[package] = new_list return copy_files @property def spl_dd(self): """If the spl binary should be dd:d to the boot partition this field specifies the offset. An int.""" return self._get_bootloader_option(SPL_DD_FIELD) @property def env_dd(self): """If the env should be dd:d to the boot partition. 'Yes' or 'No'.""" return self._get_bootloader_option(ENV_DD_FIELD) def _get_option_bool(self, key): """Gets a boolean value from the key.""" if self.format.format_as_string == '3.0': value = self._get_option(key, convert_to="disable") if isinstance(value, bool): return value else: raise ValueError(value) else: try: return self.parser.getboolean(self.MAIN_SECTION, key) except ConfigParser.NoOptionError: return None def _get_bootloader_option(self, key, join_list_with=False, convert_to=None): """Get an option inside the current bootloader section.""" if self._is_v3: if not self.bootloader: if self.allow_unset_bootloader: return None raise ValueError("bootloader not set.") if not isinstance(key, list): keys = [key] keys = [BOOTLOADERS_FIELD, self.bootloader] + keys else: keys = key return self._get_option(keys, join_list_with, convert_to) def _bool_to_string(self, value): """Convert value, treated as boolean, to string "yes" or "no".""" if value: return "yes" else: return "no" def _hex_addrress(self, value): """Convert value to 8 character hex string""" converted_value = value if not isinstance(value, str): converted_value = "0x%08x" % value return converted_value def _v2_key_to_v3(self, key): """Convert V2 key to a V3 key""" if key in self.translate_v2_to_v3: key = self.translate_v2_to_v3[key] return key def _get_v3_option(self, keys): """Find value in config dictionary based on supplied list (keys).""" result = self.parser for key in keys: key = self._v2_key_to_v3(key) if result is not None: result = result.get(key, None) self.last_used_keys = keys return result def get_last_used_keys(self): """Used so you can work out which boards + boot loader was used. Configuration data is stored in a dictionary. This returns a list of keys used to traverse into the dictionary the last time an item was looked up. This can be used to see where a bit of information came from - we store data that may be indexed differently depending on which board and bootloader are set. """ return self.last_used_keys def get_option(self, name): """Return the value of an attribute by name. Used when you can't use a property. """ return attrgetter(name)(self) def _get_option(self, key, join_list_with=False, convert_to=None): """Return value for the given key. Precedence to board specific values. :param key: the key to return the value for. :type key: str. :param join_list_with: Used to convert lists to strings. :type join_list_with: str :param convert_to: Used to convert stored value to another type. :type convert_to: type or function. :return: the value for that key, or None if the key is not present or the value is empty. :rtype: str or None. """ if self.format.format_as_string == "3.0": if not isinstance(key, list): keys = [key] else: keys = key result = None # Just mark result as not set yet... # If board is set, search board specific keys first if self.board: result = self._get_v3_option([BOARDS_FIELD, self.board] + keys) # If a board specific value isn't found, look for a global one if result is None: result = self._get_v3_option(keys) # If no value is found, bail early (return None) if result is None: return None # 0 except: raise HwpackConfigError( "Invalid root min size %s" % (root_min_size)) def _validate_boot_min_size(self): boot_min_size = self.boot_min_size if boot_min_size is None: return try: assert int(boot_min_size) > 0 except: raise HwpackConfigError( "Invalid boot min size %s" % (boot_min_size)) def _validate_loader_min_size(self): loader_min_size = self.loader_min_size if loader_min_size is None: return try: assert int(loader_min_size) > 0 except: raise HwpackConfigError( "Invalid loader min size %s" % (loader_min_size)) def _validate_loader_start(self): loader_start = self.loader_start if loader_start is None: return try: assert int(loader_start) > 0 except: raise HwpackConfigError( "Invalid loader start %s" % (loader_start)) def _validate_include_debs(self): try: self.include_debs except ValueError: raise HwpackConfigError( "Invalid value for include-debs: %s" % self.include_debs) @property def _is_v3(self): """Checks if format is 3.0.""" return self.format.format_as_string == '3.0' def _validate_bool(self, value): """Checks if a value is boolean or not, represented by "yes" or "no". """ if not isinstance(value, str): return False return string.lower(value) in ['yes', 'no'] def _validate_bootloader_file_in_boot_part(self): if not self._validate_bool(self.bootloader_file_in_boot_part): if self._is_v3: name = "bootloader" else: name = "u_boot" raise HwpackConfigError( "Invalid value for %s_in_boot_part: %s" % (name, self.bootloader_file_in_boot_part)) def _validate_spl_in_boot_part(self): spl_in_boot_part = self.spl_in_boot_part if spl_in_boot_part is None: return if string.lower(spl_in_boot_part) not in ['yes', 'no']: raise HwpackConfigError( "Invalid value for spl_in_boot_part: %s" % self.spl_in_boot_part) def _validate_env_dd(self): env_dd = self.env_dd if env_dd is None: return if string.lower(env_dd) not in ['yes', 'no']: raise HwpackConfigError( "Invalid value for env_dd: %s" % self.env_dd) def _validate_bootloader_dd(self): bootloader_dd = self.bootloader_dd if bootloader_dd is None: return try: assert int(bootloader_dd) > 0 except: if self._is_v3: name = "bootloader" else: name = "u_boot" raise HwpackConfigError( "Invalid %s_dd %s" % (name, bootloader_dd)) def _validate_spl_dd(self): spl_dd = self.spl_dd if spl_dd is None: return try: assert int(spl_dd) > 0 except: raise HwpackConfigError( "Invalid spl_dd %s" % (spl_dd)) def _validate_support(self): support = self.support if support not in (None, "supported", "unsupported"): raise HwpackConfigError( "Invalid value for support: %s" % support) def _invalid_package_message(self, package_name, section_name, value): if self._is_v3: message = ("Invalid value in %s in the metadata: %s" % (package_name, value)) else: message = ("Invalid value in %s in the [%s] section: %s" % (package_name, section_name, value)) return message def _validate_packages(self): packages = self.packages if not packages: raise HwpackConfigError(self._not_found_message(PACKAGES_FIELD)) for package in packages: self._assert_matches_pattern( self.PACKAGE_REGEX, package, self._invalid_package_message( PACKAGES_FIELD, self.MAIN_SECTION, package)) def _validate_bootloader_package(self): bootloader_package = self.bootloader_package if bootloader_package is not None: self._assert_matches_pattern( self.PACKAGE_REGEX, bootloader_package, self._invalid_package_message( self.BOOTLOADER_PACKAGE_KEY, self.MAIN_SECTION, bootloader_package)) def _validate_spl_package(self): spl_package = self.spl_package if spl_package is not None: self._assert_matches_pattern( self.PACKAGE_REGEX, spl_package, self._invalid_package_message(SPL_PACKAGE_FIELD, self.MAIN_SECTION, spl_package)) def _validate_samsung_bl1_start(self): samsung_bl1_start = self.samsung_bl1_start if samsung_bl1_start is None: return try: assert int(samsung_bl1_start) > 0 except: raise HwpackConfigError( "Invalid samsung_bl1_start %s" % (samsung_bl1_start)) def _validate_samsung_bl1_len(self): samsung_bl1_len = self.samsung_bl1_len if samsung_bl1_len is None: return try: assert int(samsung_bl1_len) > 0 except: raise HwpackConfigError( "Invalid samsung_bl1_len %s" % (samsung_bl1_len)) def _validate_samsung_env_start(self): samsung_env_start = self.samsung_env_start if samsung_env_start is None: return try: assert int(samsung_env_start) > 0 except: raise HwpackConfigError( "Invalid samsung_env_start %s" % (samsung_env_start)) def _validate_samsung_env_len(self): samsung_env_len = self.samsung_env_len if samsung_env_len is None: return try: assert int(samsung_env_len) > 0 except: raise HwpackConfigError( "Invalid samsung_env_len %s" % (samsung_env_len)) def _validate_samsung_bl2_start(self): samsung_bl2_start = self.samsung_bl2_start if samsung_bl2_start is None: return try: assert int(samsung_bl2_start) > 0 except: raise HwpackConfigError( "Invalid samsung_bl2_start %s" % (samsung_bl2_start)) def _validate_samsung_bl2_len(self): samsung_bl2_len = self.samsung_bl2_len if samsung_bl2_len is None: return try: assert int(samsung_bl2_len) > 0 except: raise HwpackConfigError( "Invalid samsung_bl2_len %s" % (samsung_bl2_len)) def _validate_architectures(self): architectures = self.architectures if not architectures: raise HwpackConfigError( self._not_found_message(ARCHITECTURES_FIELD)) def _validate_assume_installed(self): assume_installed = self.assume_installed for package in assume_installed: self._assert_matches_pattern( self.PACKAGE_REGEX, package, self._invalid_package_message(self.ASSUME_INSTALLED_KEY, self.MAIN_SECTION, package)) def _message_start(self, key, section_name): if self._is_v3: message = "The %s, %s " % (key, section_name) else: message = "The %s in the [%s] section " % (key, section_name) return message def _validate_source(self, section_name): if self._is_v3: sources_entry = self._get_option([SOURCES_FIELD] + [section_name]) else: try: sources_entry = self.parser.get( section_name, self.SOURCES_ENTRY_KEY) except ConfigParser.NoOptionError: raise HwpackConfigError( "No %s in the [%s] section" % (self.SOURCES_ENTRY_KEY, section_name)) if not sources_entry: raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "is missing the URI") if len(sources_entry.split(" ", 1)) < 2: raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "is missing the distribution") if sources_entry.startswith("deb"): raise HwpackConfigError( self._message_start(self.SOURCES_ENTRY_KEY, section_name) + "shouldn't start with 'deb'") def _validate_sources(self): if self._is_v3: source_dict = self.parser.get(SOURCES_FIELD) if not source_dict: return if isinstance(source_dict, dict): sources = source_dict.keys() else: raise HwpackConfigError( "The %s in the [%s] section is missing the URI" % (self.SOURCES_ENTRY_KEY, source_dict)) else: sources = self.parser.sections() found = False for source_name in sources: if source_name == self.MAIN_SECTION: continue self._validate_source(source_name) found = True if not found: raise HwpackConfigError( "No sections other than [%s]" % self.MAIN_SECTION) def _validate_keys(self): """Check the dictionary created by the YAML parser for unknown keys""" if not self._is_v3: # We don't check V1 or V2 configurations in this way return self._validate_keys_layout = hwpack_v3_layout self._do_validate_keys_prefix = [] self._do_validate_keys(self._validate_keys_layout, self.parser) def _do_validate_keys_push_prefix(self, prefix): self._do_validate_keys_prefix.append(prefix) prefix = ": ".join(self._do_validate_keys_prefix)[2:] if prefix: prefix += ": " return prefix def _do_validate_keys(self, expected, config, prefix=""): prefix = self._do_validate_keys_push_prefix(prefix) if not isinstance(config, dict): raise HwpackConfigError("Invalid structure in metadata. Expected " "key: value pairs, found: '%s'" % (prefix + str(config))) for key in config.keys(): # If expected == {"*": {...}} then we can accept any key if("*" in expected and expected.keys() == ["*"] and isinstance(expected["*"], dict)): # Have found a sub-dictionary to check. Recurse. self._do_validate_keys(expected["*"], config[key], key) continue # Check to see if the key is valid if key not in expected: raise HwpackConfigError("Unknown key in metadata: '%s'" % (prefix + str(key))) # Have a valid key. If it should point to a dictionary, recurse if expected[key]: if isinstance(expected[key], dict): # Have found a sub-dictionary to check. Recurse. self._do_validate_keys(expected[key], config[key], key) continue if expected[key] == "root": config = config[key] prefix = self._do_validate_keys_push_prefix(key) for key in config.keys(): self._do_validate_keys(self._validate_keys_layout, config[key], key) self._do_validate_keys_prefix.pop() self._do_validate_keys_prefix.pop() linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/tarfile_matchers.py0000644000175000017500000001625212724020110026730 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import tarfile from testtools.matchers import Annotate, Equals, Matcher, Mismatch class TarfileMissingPathMismatch(Mismatch): """A Mismatch indicating that a required path was missing from a tarfile. """ def __init__(self, tarball, path): """Create a TarfileMissingPathMismatch Mismatch. :param tarball: the tarfile that was checked. :param path: the path that was expected to be present. """ self.tarball = tarball self.path = path def describe(self): return '"%s" has no path "%s"' % (self.tarball, self.path) def __eq__(self, other): return self.tarball == other.tarball and self.path == other.path def __ne__(self, other): return not self.__eq__(other) class TarfileWrongValueMismatch(Mismatch): """A Mismatch indicating that an entry in a tarfile was not as expected. """ def __init__(self, attribute, tarball, path, expected, actual): """Create a TarfileWrongValueMismatch Mismatch. :param attribute: the attribute that was not as expected. :type attribute: str :param tarball: the tarfile that was checked. :param path: the path that was checked. :param expected: the expected value of the attribute. :param actual: the value that was found. """ self.attribute = attribute self.tarball = tarball self.path = path self.expected = expected self.actual = actual def describe(self): return 'The path "%s" in "%s" has %s %s, expected %s' % ( self.path, self.tarball, self.attribute, self.actual, self.expected) def __eq__(self, other): return (self.attribute == other.attribute and self.tarball == other.tarball and self.path == other.path and self.expected == other.expected and self.actual == other.actual) def __ne__(self, other): return not self.__eq__(other) class TarfileHasFile(Matcher): """Check that a tarfile has an entry with certain values.""" def __init__(self, path, type=None, size=None, mtime=None, mtime_skew=None, mode=None, linkname=None, uid=None, gid=None, uname=None, gname=None, content=None, content_matcher=None): """Create a TarfileHasFile Matcher. :param path: the path that must be present. :type path: str :param type: the type of TarInfo that must be at `path`, or None to not check. :param size: the size that the entry at `path` must have, or None to not check. :param mtime: the mtime that the entry at `path` must have, or None to not check. :param mtime_skew: the number of seconds that the file mtime can be different to the required. :param mode: the mode that the entry at `path` must have, or None to not check. :param linkname: the linkname that the entry at `path` must have, or None to not check. :param uid: the user id that the entry at `path` must have, or None to not check. :param gid: the group id that the entry at `path` must have, or None to not check. :param uname: the username that the entry at `path` must have, or None to not check. :param gname: the group name that the entry at `path` must have, or None to not check. :param content: the content that `path` must have when extracted, or None to not check. :param content_matcher: a matcher to match the content that `path` has when extracted, or None to not check. You can't specify both content_matcher and content. """ self.path = path self.type = type self.size = size self.mtime = mtime self.mtime_skew = mtime_skew self.mode = mode self.linkname = linkname self.uid = uid self.gid = gid self.uname = uname self.gname = gname if content is not None: if content_matcher is not None: raise ValueError( "doesn't make sense to specify content and " "content_matcher") content_matcher = Equals(content) if content_matcher is not None: self.content_matcher = Annotate( 'The content of path "%s" did not match' % path, content_matcher) else: self.content_matcher = None def match(self, tarball): """Match a tarfile.TarFile against the expected values.""" if self.path not in tarball.getnames(): return TarfileMissingPathMismatch(tarball, self.path) info = tarball.getmember(self.path) for attr in ("type", "size", "mode", "linkname", "uid", "gid", "uname", "gname"): expected = getattr(self, attr, None) if expected is not None: actual = getattr(info, attr) if expected != actual: return TarfileWrongValueMismatch( attr, tarball, self.path, expected, actual) if self.mtime is not None: mtime_skew = self.mtime_skew or 0 if abs(self.mtime - info.mtime) > mtime_skew: return TarfileWrongValueMismatch( "mtime", tarball, self.path, self.mtime, info.mtime) if self.content_matcher is not None: if info.type == tarfile.DIRTYPE: contents = [] path_frags = self.path.split('/') for name in tarball.getnames(): name_frags = name.split('/') if (len(name_frags) == len(path_frags) + 1 and name_frags[:-1] == path_frags): contents.append(name_frags[-1]) content_mismatch = self.content_matcher.match(contents) if content_mismatch: return content_mismatch else: actual = tarball.extractfile(self.path).read() content_mismatch = self.content_matcher.match(actual) if content_mismatch: return content_mismatch return None def __str__(self): return 'tarfile has file "%s"' % (self.path, ) linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/package_unpacker.py0000644000175000017500000000476112724020110026701 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2013 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging import os import tempfile from subprocess import PIPE from shutil import rmtree from linaro_image_tools import cmd_runner logger = logging.getLogger(__name__) class PackageUnpacker(object): def __enter__(self): self.tempdir = tempfile.mkdtemp() return self def __exit__(self, type, value, traceback): if self.tempdir is not None and os.path.exists(self.tempdir): rmtree(self.tempdir) def get_path(self, package_file_name, file_name=''): """Get package or file path in unpacker tmp dir.""" package_dir = os.path.basename(package_file_name) return os.path.join(self.tempdir, package_dir, file_name) def unpack_package(self, package_file_name): # We could extract only a single file, but since dpkg will pipe # the entire package through tar anyway we might as well extract all. unpack_dir = self.get_path(package_file_name) if not os.path.isdir(unpack_dir): os.mkdir(unpack_dir) p = cmd_runner.run(["tar", "-C", unpack_dir, "-xf", "-"], stdin=PIPE) cmd_runner.run(["dpkg", "--fsys-tarfile", package_file_name], stdout=p.stdin).communicate() p.communicate() def get_file(self, package, file): # File path passed here must not be absolute, or file from # real filesystem will be referenced. assert file and file[0] != '/' self.unpack_package(package) logger.debug("Unpacked package %s." % package) temp_file = self.get_path(package, file) assert os.path.exists(temp_file), "The file '%s' was " \ "not found in the package '%s'." % (file, package) return temp_file linaro-image-tools-2016.05.orig/linaro_image_tools/hwpack/__init__.py0000644000175000017500000000207312724020110025147 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import logging from linaro_image_tools.utils import DEFAULT_LOGGER_NAME class NullHandler(logging.Handler): def emit(self, record): pass h = NullHandler() logging.getLogger(DEFAULT_LOGGER_NAME).addHandler(h) linaro-image-tools-2016.05.orig/linaro_image_tools/tests/0000755000175000017500000000000012724020110022721 5ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro_image_tools/tests/test_pep8.py0000644000175000017500000000273412724020110025214 0ustar voipiovoipio# Copyright (C) 2012 Linaro Ltd. # # Author: Loic Minier # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import subprocess from testtools import TestCase class TestPep8(TestCase): def test_pep8(self): # Errors we have to ignore for now: # * E202 whitespace before ')' or ']' # E202 is actually only reported with the natty version of pep8 and # can be re-enabled once we drop support for natty. ignore = ['E202'] # Ignore return code. proc = subprocess.Popen( ['pep8', '--repeat', '--ignore=%s' % ','.join(ignore), '.'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEquals('', stdout) self.assertEquals('', stderr) linaro-image-tools-2016.05.orig/linaro_image_tools/tests/fixtures.py0000644000175000017500000001133712724020110025151 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import shutil import tempfile from StringIO import StringIO from linaro_image_tools import cmd_runner class CreateTempDirFixture(object): def __init__(self): self.tempdir = None def setUp(self): self.tempdir = tempfile.mkdtemp() def tearDown(self): if os.path.exists(self.tempdir): shutil.rmtree(self.tempdir) def get_temp_dir(self): return self.tempdir class CreateTempFileFixture(object): """Class to create a temporary file to be used in a test.""" def __init__(self, string=None): """Initialize the fixture. :param string: the string to write in the file. """ self.temp_file = None self.string = string def setUp(self): self.temp_file = tempfile.NamedTemporaryFile() if self.string is not None: self.temp_file.write(self.string) # Go back to the initial position, we just need to write something # and be able to read from the beginning of the file. self.temp_file.seek(0) def tearDown(self): # We don't need to do anything, file is automatically deleted. pass def get_file_name(self): return self.temp_file.name class MockSomethingFixture(object): """A fixture which mocks something on the given object. Replaces attr_name on obj with the given mock, undoing that upon tearDown(). """ def __init__(self, obj, attr_name, mock): self.obj = obj self.attr_name = attr_name self.mock = mock self.orig_attr = getattr(obj, attr_name) def setUp(self): setattr(self.obj, self.attr_name, self.mock) def tearDown(self): setattr(self.obj, self.attr_name, self.orig_attr) class MockCmdRunnerPopen(object): """A mock for cmd_runner.Popen() which stores the args given to it.""" calls = None # A variable that is set to False in __call__() and only set back to True # when wait() is called, to indicate that the subprocess has finished. Is # used in tests to make sure all callsites wait for their child. child_finished = True def __init__(self, output_string='', assert_child_finished=True): self.assert_child_finished = assert_child_finished self.output_string = output_string def __call__(self, cmd, *args, **kwargs): if self.assert_child_finished and not self.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") self.child_finished = False if self.calls is None: self.calls = [] if isinstance(cmd, basestring): all_args = [cmd] else: all_args = cmd all_args.extend(args) self.calls.append(all_args) self.returncode = 0 return self def communicate(self, input=None): self.wait() return self.output_string, '' def wait(self): self.child_finished = True return self.returncode @property def commands_executed(self): return [' '.join(args) for args in self.calls] @property def stdin(self): return StringIO() class MockCmdRunnerPopenFixture(MockSomethingFixture): """A test fixture which mocks cmd_runner.do_run with the given mock. If no mock is given, a MockCmdRunnerPopen instance is used. """ def __init__(self, output_string='', assert_child_finished=True): super(MockCmdRunnerPopenFixture, self).__init__( cmd_runner, 'Popen', MockCmdRunnerPopen(output_string, assert_child_finished)) def tearDown(self): super(MockCmdRunnerPopenFixture, self).tearDown() if self.mock.assert_child_finished and not self.mock.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") linaro-image-tools-2016.05.orig/linaro_image_tools/tests/test_utils.py0000644000175000017500000003764512724020110025511 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import stat import subprocess import sys import logging import tempfile import tarfile from StringIO import StringIO from linaro_image_tools import cmd_runner, utils from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( CreateTempDirFixture, MockCmdRunnerPopenFixture, MockSomethingFixture, ) from linaro_image_tools.utils import ( IncompatibleOptions, InvalidHwpackFile, UnableToFindPackageProvidingCommand, additional_android_option_checks, additional_option_checks, android_hwpack_in_boot_tarball, check_file_integrity_and_log_errors, ensure_command, find_command, install_package_providing, path_in_tarfile_exists, preferred_tools_dir, prep_media_path, try_import, verify_file_integrity, ) sudo_args = " ".join(cmd_runner.SUDO_ARGS) class TestPathInTarfile(TestCaseWithFixtures): def setUp(self): super(TestPathInTarfile, self).setUp() tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() self.tarfile_name = os.path.join(tempdir, 'test_tarfile.tar.gz') self.tempfile_added = self.createTempFileAsFixture() self.tempfile_unused = self.createTempFileAsFixture() with tarfile.open(self.tarfile_name, 'w:gz') as tar: tar.add(self.tempfile_added) def test_file_exists(self): self.assertTrue(path_in_tarfile_exists(self.tempfile_added[1:], self.tarfile_name)) def test_file_does_not_exist(self): self.assertFalse(path_in_tarfile_exists(self.tempfile_unused[1:], self.tarfile_name)) class TestVerifyFileIntegrity(TestCaseWithFixtures): filenames_in_shafile = ['verified-file1', 'verified-file2'] class MockCmdRunnerPopen(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n', '' def wait(self): return self.returncode class MockCmdRunnerPopen_sha1sum_fail(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': ERROR\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': ERROR\n', '' def wait(self): return self.returncode class MockCmdRunnerPopen_wait_fails(object): def __call__(self, cmd, *args, **kwargs): self.returncode = 0 return self def communicate(self, input=None): self.wait() return ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n', '' def wait(self): stdout = ': OK\n'.join( TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n' raise cmd_runner.SubcommandNonZeroReturnValue([], 1, stdout, None) class FakeTempFile(): name = "/tmp/1" def close(self): pass def read(self): return "" def test_verify_files(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(tempfile, 'NamedTemporaryFile', self.FakeTempFile)) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" verify_file_integrity([signature_filename]) self.assertEqual( ['gpg --status-file=%s --verify %s' % (self.FakeTempFile.name, signature_filename), 'sha1sum -c %s' % hash_filename], fixture.mock.commands_executed) def test_verify_files_returns_files(self): self.useFixture(MockSomethingFixture(cmd_runner, 'Popen', self.MockCmdRunnerPopen())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" verified_files, _, _ = verify_file_integrity([signature_filename]) self.assertEqual(self.filenames_in_shafile, verified_files) def test_check_file_integrity_and_print_errors(self): self.useFixture(MockSomethingFixture(cmd_runner, 'Popen', self.MockCmdRunnerPopen())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual(self.filenames_in_shafile, verified_files) # The sha1sums are faked as passing and all commands return 0, so # it should look like GPG passed self.assertTrue(result) def test_check_file_integrity_and_print_errors_fail_sha1sum(self): logging.getLogger().setLevel(100) # Disable logging messages to screen self.useFixture(MockSomethingFixture( cmd_runner, 'Popen', self.MockCmdRunnerPopen_sha1sum_fail())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual([], verified_files) # The sha1sums are faked as failing and all commands return 0, so # it should look like GPG passed self.assertFalse(result) logging.getLogger().setLevel(logging.WARNING) def test_check_file_integrity_and_print_errors_fail_gpg(self): logging.getLogger().setLevel(100) # Disable logging messages to screen self.useFixture(MockSomethingFixture( cmd_runner, 'Popen', self.MockCmdRunnerPopen_wait_fails())) hash_filename = "dummy-file.txt" signature_filename = hash_filename + ".asc" result, verified_files = check_file_integrity_and_log_errors( [signature_filename], self.filenames_in_shafile[0], [self.filenames_in_shafile[1]]) self.assertEqual([], verified_files) # The sha1sums are faked as passing and all commands return 1, so # it should look like GPG failed self.assertFalse(result) logging.getLogger().setLevel(logging.WARNING) class TestEnsureCommand(TestCaseWithFixtures): install_pkg_providing_called = False def setUp(self): super(TestEnsureCommand, self).setUp() self.useFixture(MockSomethingFixture( sys, 'stdout', open('/dev/null', 'w'))) def test_command_already_present(self): self.mock_install_package_providing() ensure_command('apt-get') self.assertFalse(self.install_pkg_providing_called) def test_command_not_present(self): self.mock_install_package_providing() ensure_command('apt-get-two-o') self.assertTrue(self.install_pkg_providing_called) def mock_install_package_providing(self): def mock_func(command): self.install_pkg_providing_called = True self.useFixture(MockSomethingFixture( utils, 'install_package_providing', mock_func)) class TestFindCommand(TestCaseWithFixtures): def test_preferred_dir(self): tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() lmc = 'linaro-media-create' path = os.path.join(tempdir, lmc) open(path, 'w').close() os.chmod(path, stat.S_IXUSR) self.assertEquals(path, find_command(lmc, tempdir)) def test_existing_command(self): lmc = 'linaro-media-create' prefer_dir = preferred_tools_dir() if prefer_dir is None: expected, _ = cmd_runner.run( ['which', lmc, ], stdout=subprocess.PIPE).communicate() expected = expected.strip() else: expected = os.path.join(prefer_dir, lmc) self.assertEquals(expected, find_command(lmc)) def test_nonexisting_command(self): self.assertEquals(find_command('linaro-moo'), None) class TestInstallPackageProviding(TestCaseWithFixtures): # This is the package we need to fake the installation of, it is a # slightly changed version of 'apt-get -s install' output. # It is used as an argument to MockCmdRunnerPopenFixture in order to # pass a string that should be expected as output from the command that # is being executed. output_string = 'Inst dosfstools (3.0.12-1ubuntu1 Ubuntu:12.04)' def test_package_installation_accepted(self): self.useFixture(MockSomethingFixture(sys, 'stdout', open('/dev/null', 'w'))) # We need this since we are getting user input via raw_input # and we need a 'Y' to proceed with the operations. self.useFixture(MockSomethingFixture(sys, 'stdin', StringIO('Y'))) fixture = self.useFixture( MockCmdRunnerPopenFixture(self.output_string)) try: install_package_providing('mkfs.vfat') except UnableToFindPackageProvidingCommand as inst: self.assertEqual("CommandNotFound python module does not exist.", inst.args[0]) else: self.assertEqual( ['apt-get -s install dosfstools', '%s apt-get --yes install dosfstools' % sudo_args], fixture.mock.commands_executed) def test_package_installation_refused(self): self.useFixture(MockSomethingFixture(sys, 'stdout', open('/dev/null', 'w'))) # We need this since we are getting user input via raw_input # and we need a 'n' to mimic a refused package installation. self.useFixture(MockSomethingFixture(sys, 'stdin', StringIO('n'))) self.useFixture(MockCmdRunnerPopenFixture(self.output_string)) CommandNotFound = try_import('CommandNotFound.CommandNotFound') if CommandNotFound is None: self.assertRaises(UnableToFindPackageProvidingCommand, install_package_providing, 'mkfs.vfat') else: self.assertRaises(SystemExit, install_package_providing, 'mkfs.vfat') def test_not_found_package(self): self.assertRaises(UnableToFindPackageProvidingCommand, install_package_providing, 'mkfs.lean') class Args(): def __init__(self, directory, device, board): self.directory = directory self.device = device self.board = board class TestPrepMediaPath(TestCaseWithFixtures): def test_prep_media_path(self): self.useFixture(MockSomethingFixture(os.path, 'abspath', lambda x: x)) self.useFixture(MockSomethingFixture(os, "makedirs", lambda x: x)) self.assertEqual("testdevice", prep_media_path(Args(directory=None, device="testdevice", board="testboard"))) self.assertEqual("/foo/bar/testdevice", prep_media_path(Args(directory="/foo/bar", device="testdevice", board="testboard"))) class TestAdditionalOptionChecks(TestCaseWithFixtures): def test_additional_option_checks(self): self.useFixture(MockSomethingFixture(os.path, 'abspath', lambda x: x)) self.useFixture(MockSomethingFixture(os, "makedirs", lambda x: x)) self.assertRaises(IncompatibleOptions, additional_option_checks, Args(directory="/foo/bar", device="/testdevice", board="testboard")) sys.argv.append("--mmc") self.assertRaises(IncompatibleOptions, additional_option_checks, Args(directory="/foo/bar", device="testdevice", board="testboard")) sys.argv.remove("--mmc") class TestAndroidOptionChecks(TestCaseWithFixtures): def test_hwpack_is_file(self): class HwPacksArgs: def __init__(self, hwpack): self.hwpack = hwpack try: tmpdir = tempfile.mkdtemp() self.assertRaises(InvalidHwpackFile, additional_android_option_checks, HwPacksArgs(tmpdir)) finally: os.rmdir(tmpdir) def test_android_hwpack_in_boot(self): """Test presence of config file in boot directory.""" try: tmpdir = tempfile.mkdtemp() boot_dir = os.path.join(tmpdir, "boot") os.mkdir(boot_dir) config_file = os.path.join(boot_dir, "config") expected = (True, config_file) with open(config_file, "w"): self.assertEqual(expected, android_hwpack_in_boot_tarball(tmpdir)) finally: os.unlink(config_file) os.removedirs(boot_dir) def test_android_hwpack_not_in_boot(self): """Test missing config file.""" try: tmpdir = tempfile.mkdtemp() boot_dir = os.path.join(tmpdir, "boot") os.mkdir(boot_dir) config_file = os.path.join(boot_dir, "config") expected = (False, config_file) self.assertEqual(expected, android_hwpack_in_boot_tarball(tmpdir)) finally: os.removedirs(boot_dir) class TestHwpackIsFile(TestCaseWithFixtures): """Testing '--hwpack' option only allows regular files.""" def test_hwpack_is_file(self): class HwPackArgs: def __init__(self, hwpack): self.hwpacks = [hwpack] self.directory = None try: tmpdir = tempfile.mkdtemp() self.assertRaises(InvalidHwpackFile, additional_option_checks, HwPackArgs(hwpack=tmpdir)) finally: os.rmdir(tmpdir) def test_hwpacks_are_files(self): """ Tests that multiple hwpacks are regular files. Tests against a file and a directory, to avoid circumstances in which 'additional_option_checks' is tweaked. """ class HwPacksArgs: def __init__(self, hwpacks): self.hwpacks = hwpacks self.directory = None try: tmpdir = tempfile.mkdtemp() _, tmpfile = tempfile.mkstemp() self.assertRaises(InvalidHwpackFile, additional_option_checks, HwPacksArgs([tmpfile, tmpdir])) finally: os.rmdir(tmpdir) os.remove(tmpfile) linaro-image-tools-2016.05.orig/linaro_image_tools/tests/test_pyflakes.py0000644000175000017500000000225312724020110026152 0ustar voipiovoipio# Copyright (C) 2011 Linaro # # Author: Loic Minier # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import subprocess from testtools import TestCase class TestPyflakes(TestCase): def test_pyflakes(self): # ignore return code proc = subprocess.Popen(['pyflakes', '.'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() self.assertEquals('', stdout) self.assertEquals('', stderr) linaro-image-tools-2016.05.orig/linaro_image_tools/tests/integration.txt0000644000175000017500000000247712724020110026017 0ustar voipiovoipioA few integration tests we can run to test things on a higher level. These will probably require root or access to specific block devices so they are not meant to be automated. # This should print nothing to stdout but will unpack the given binary # tarball under the given directory. >>> from linaro_image_tools.media_create.unpack_binary_tarball import unpack_binary_tarball >>> unpack_binary_tarball(, ) # Partition (for real!) /dev/sdb for a beagle board and return the devices # for the boot and root partitions. >>> from linaro_image_tools.media_create.partitions import Media, setup_partitions >>> from linaro_image_tools.media_create.boards import board_configs >>> setup_partitions( ... board_configs['beagle'], Media('/dev/sdb'), None, 'boot', ... 'root', 'ext3', True, True, True) Checking that no-one is using this disk right now ... ('/dev/sdb1', '/dev/sdb2') # Partition /tmp/beagle.img for a beagle board and return the loopback # devices for the boot and root partitions. >>> setup_partitions( ... board_configs['beagle'], Media('/tmp/beagle.img'), '2G', 'boot', ... 'root', 'ext3', True, True, True) Warning: /tmp/beagle.img is not a block device ... ('/dev/loop0', '/dev/loop1') linaro-image-tools-2016.05.orig/linaro_image_tools/tests/__init__.py0000644000175000017500000000335712724020110025042 0ustar voipiovoipio# Copyright (C) 2010, 2011, 2012 Linaro # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os import unittest from linaro_image_tools.hwpack.tests import test_suite as hwpack_suite from linaro_image_tools.media_create.tests import ( test_suite as media_create_suite, ) from linaro_image_tools.utils import has_command def test_suite(): module_names = [ 'linaro_image_tools.tests.test_cmd_runner', 'linaro_image_tools.tests.test_utils', ] # if pyflakes is installed and we're running from a bzr checkout... if has_command('pyflakes') and not os.path.isabs(__file__): # ...also run the pyflakes test module_names.append('linaro_image_tools.tests.test_pyflakes') # if pep8 is installed and we're running from a bzr checkout... if has_command('pep8') and not os.path.isabs(__file__): # ...also run the pep8 test module_names.append('linaro_image_tools.tests.test_pep8') loader = unittest.TestLoader() suite = loader.loadTestsFromNames(module_names) suite.addTests(hwpack_suite()) suite.addTests(media_create_suite()) return suite linaro-image-tools-2016.05.orig/linaro_image_tools/tests/test_cmd_runner.py0000644000175000017500000001027512724020110026473 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import os from linaro_image_tools import cmd_runner from linaro_image_tools.testing import TestCaseWithFixtures from linaro_image_tools.tests.fixtures import ( MockCmdRunnerPopenFixture, MockSomethingFixture, ) sudo_args = " ".join(cmd_runner.SUDO_ARGS) chroot_args = " ".join(cmd_runner.CHROOT_ARGS) class TestSanitizePath(TestCaseWithFixtures): def setUp(self): super(TestSanitizePath, self).setUp() self.env = {} def test_path_unset(self): cmd_runner.sanitize_path(self.env) self.assertEqual(cmd_runner.DEFAULT_PATH, self.env['PATH']) def test_path_missing_dirs(self): path = '/bin:/sbin:/foo:/usr/local/sbin' self.env['PATH'] = path cmd_runner.sanitize_path(self.env) expected = '%s:/usr/local/bin:/usr/sbin:/usr/bin' % path self.assertEqual(expected, self.env['PATH']) def test_idempotent(self): self.env['PATH'] = cmd_runner.DEFAULT_PATH cmd_runner.sanitize_path(self.env) self.assertEqual(cmd_runner.DEFAULT_PATH, self.env['PATH']) class TestCmdRunner(TestCaseWithFixtures): def test_run(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) proc = cmd_runner.run(['foo', 'bar', 'baz']) # Call wait or else MockCmdRunnerPopenFixture() raises an # AssertionError(). proc.wait() self.assertEqual(0, proc.returncode) self.assertEqual(['foo bar baz'], fixture.mock.commands_executed) def test_run_as_root_with_sudo(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 1000)) cmd_runner.run(['foo', 'bar'], as_root=True).wait() self.assertEqual( ['%s foo bar' % sudo_args], fixture.mock.commands_executed) def test_run_as_root_as_root(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 0)) cmd_runner.run(['foo', 'bar'], as_root=True).wait() self.assertEqual(['foo bar'], fixture.mock.commands_executed) def test_tuple_with_sudo(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture(os, 'getuid', lambda: 1000)) cmd_runner.run(('foo', 'bar',), as_root=True).wait() self.assertEqual( ['%s foo bar' % sudo_args], fixture.mock.commands_executed) def test_chrooted(self): fixture = self.useFixture(MockCmdRunnerPopenFixture()) cmd_runner.run(['foo', 'bar'], chroot='chroot_dir').wait() self.assertEqual( ['%s %s chroot_dir foo bar' % (sudo_args, chroot_args)], fixture.mock.commands_executed) def test_run_succeeds_on_zero_return_code(self): proc = cmd_runner.run(['true']) # Need to wait() here as we're using the real Popen. proc.wait() self.assertEqual(0, proc.returncode) def test_run_raises_exception_on_non_zero_return_code(self): def run_and_wait(): proc = cmd_runner.run(['false']) proc.wait() self.assertRaises( cmd_runner.SubcommandNonZeroReturnValue, run_and_wait) def test_run_must_be_given_list_as_args(self): self.assertRaises(AssertionError, cmd_runner.run, 'true') def test_Popen(self): proc = cmd_runner.Popen('true') returncode = proc.wait() self.assertEqual(0, returncode) linaro-image-tools-2016.05.orig/linaro_image_tools/cmd_runner.py0000644000175000017500000001026512724020110024271 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import os import subprocess DEFAULT_PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' CHROOT_ARGS = ['chroot'] SUDO_ARGS = ['sudo', '-E'] def sanitize_path(env): """Makes sure PATH is set and has important directories""" dirs = env.get('PATH', DEFAULT_PATH).split(os.pathsep) for d in DEFAULT_PATH.split(os.pathsep): if d not in dirs: dirs.append(d) env['PATH'] = os.pathsep.join(dirs) def run(args, as_root=False, chroot=None, stdin=None, stdout=None, stderr=None, cwd=None): """Run the given command as a sub process. Return a Popen instance. Callsites must wait() or communicate() with the returned Popen instance. :param command: A list or tuple containing the command to run and the arguments that should be passed to it. :param as_root: Should the given command be run as root (with sudo)? :param chroot: A directory to chroot into (implies as_root). :param stdin: Same as in subprocess.Popen(). :param stdout: Same as in subprocess.Popen(). :param stderr: Same as in subprocess.Popen(). """ assert isinstance(args, (list, tuple)), ( "The command to run must be a list or tuple, found: %s" % type(args)) if isinstance(args, tuple): args = list(args) if chroot is not None: args = CHROOT_ARGS + [chroot] + args as_root = True if as_root and os.getuid() != 0: args = SUDO_ARGS + args return Popen(args, stdin=stdin, stdout=stdout, stderr=stderr, cwd=cwd) class Popen(subprocess.Popen): """A version of Popen which raises an error on non-zero returncode. Once the subprocess completes we check its returncode and raise SubcommandNonZeroReturnValue if it's non-zero. """ def __init__(self, args, env=None, **kwargs): self._my_args = args self.except_on_cmd_fail = True if env is None: env = os.environ.copy() env['LC_ALL'] = 'C' # ensure a proper PATH before calling Popen sanitize_path(os.environ) # and for subcommands sanitize_path(env) super(Popen, self).__init__(args, env=env, **kwargs) def communicate(self, input=None): self.except_on_cmd_fail = False stdout, stderr = super(Popen, self).communicate(input) self.except_on_cmd_fail = True if self.returncode != 0: raise SubcommandNonZeroReturnValue(self._my_args, self.returncode, stdout, stderr) return stdout, stderr def wait(self): returncode = super(Popen, self).wait() if returncode != 0 and self.except_on_cmd_fail: raise SubcommandNonZeroReturnValue(self._my_args, returncode) return returncode class SubcommandNonZeroReturnValue(Exception): def __init__(self, command, return_value, stdout=None, stderr=None): self.command = command self.retval = return_value self.stdout = stdout self.stderr = stderr def __str__(self): message = 'Sub process "%s" returned a non-zero value: %d' % ( self.command, self.retval) if self.stdout: message += '\nstdout was\n{0}'.format(self.stdout) if self.stderr: message += '\nstderr was\n{0}'.format(self.stderr) return message linaro-image-tools-2016.05.orig/linaro_image_tools/testing.py0000644000175000017500000000411112724020110023603 0ustar voipiovoipio# Copyright (C) 2010, 2011 Linaro # # Author: James Westby # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. import os import tempfile from testtools import TestCase class TestCaseWithFixtures(TestCase): """A TestCase with the ability to easily add 'fixtures'. A fixture is an object which can be created and cleaned up, and this test case knows how to manage them to ensure that they will always be cleaned up at the end of the test. """ def useFixture(self, fixture): """Make use of a fixture, ensuring that it will be cleaned up. Given a fixture, this method will run the `setUp` method of the fixture, and ensure that its `tearDown` method will be called at the end of the test, regardless of success or failure. :param fixture: the fixture to use. :type fixture: an object with setUp and tearDown methods. :return: the fixture that was passed in. """ self.addCleanup(fixture.tearDown) fixture.setUp() return fixture def createTempFileAsFixture(self, prefix='tmp', dir=None): """Create a temp file and make sure it is removed on tearDown. :return: The filename of the file created. """ _, filename = tempfile.mkstemp(prefix=prefix, dir=dir) self.addCleanup(os.unlink, filename) return filename linaro-image-tools-2016.05.orig/linaro_image_tools/__init__.py0000644000175000017500000000000012724020110023656 0ustar voipiovoipiolinaro-image-tools-2016.05.orig/linaro-hwpack-replace0000755000175000017500000002204012724020110022005 0ustar voipiovoipio#!/usr/bin/env python # Copyright (C) 2010, 2011 Linaro # # Author: Deepti B. Kalakeri # # This file is part of Linaro Image Tools. It adds the feature # to include/replace a debian package into the given hwpack. # We might need to change the manifest and Packages file in the # future to match the hardware pack v2 changes when available. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . # import os import sys import shutil import glob import tarfile import tempfile import argparse import datetime import fileinput from debian.deb822 import Packages from linaro_image_tools.hwpack.packages import get_packages_file from linaro_image_tools.hwpack.packages import FetchedPackage from linaro_image_tools.utils import get_logger parser = argparse.ArgumentParser() parser.add_argument("-t", "--hwpack-name", dest="hwpack_name", help="Specific hwpack-name to use (default: None)") parser.add_argument("-p", "--deb-pack", dest="deb_pack", help="Specific debian package to replace (default: None).") parser.add_argument("-r", "--prefix-pkg-remove", dest="prefix_pkg_remove", help="Specify the prefix of the old debian package to "\ "replace (default: None).") parser.add_argument("-n", "--append-build-number", dest="build_number", help="Specify the build number if any to be used in new "\ "hwpack name (default: None).") parser.add_argument("-i", "--in-place", action="store_true", dest="inplace", help="Modify the hwpack rather than creating a new one") parser.add_argument("-d", "--debug-output", action="store_true", dest="debug", help="Verbose messages are displayed when specified") logger = None class DummyStanza(object): def __init__(self, info): self.info = info def dump(self, fd): fd.write(get_packages_file([self.info])) def get_hwpack_name(old_hwpack, build_number): # The build_number would be the job build number. # Valid value for the build_number would be available for ex # when l-h-r is used in the jenkins. timestamp = [datetime.datetime.utcnow().strftime("%Y%m%d-%H%M")] hwpack_name_parts = (old_hwpack.split('_', 3)) new_hwpack_name = [('_'.join(hwpack_name_parts[:2] + timestamp))] if build_number is not None: job_build_number = [''.join('b' + build_number)] new_hwpack_name = [('_'.join(new_hwpack_name + job_build_number))] return('_'.join(new_hwpack_name + hwpack_name_parts[3:])) def should_remove(package_name, prefix_pkg_remove): # hwpack-* Package is a metadata package that contain reference to the # linux-linaro-omap that was previously present in the hwpack. # We need to make sure we dont write the hwpack-* related # package information into Package, otherwise it would try to download the old # kernel package that was present in the hwpack than installing the new one. if (package_name.startswith(prefix_pkg_remove) or package_name.startswith("hwpack-")): return True return False def verify_existing_debians(debpack_dirname, prefix_pkg_remove): """ Find if the debian file with the same name exists, if it exists then remove it and replace with the new deb file If similar debian file exists then remove it """ deb_file_to_remove = None try: for deb_filename in os.listdir(debpack_dirname): root, ext = os.path.splitext(deb_filename) if should_remove(root, prefix_pkg_remove) and ext == '.deb': deb_file_to_remove = os.path.join(debpack_dirname, deb_filename) os.remove(deb_file_to_remove) except Exception, details: logger.error("Error Details: %s", details) def modify_manifest_info(tempdir, new_debpack_info, prefix_pkg_remove): """ Modify the manifest file to include the new debian information """ debpack_manifest_fname = os.path.join(tempdir, "manifest") if new_debpack_info is not None: new_debpack_line = '%s=%s\n' % (new_debpack_info.name, new_debpack_info.version) for line in fileinput.FileInput(debpack_manifest_fname, inplace=1): if not should_remove(line, prefix_pkg_remove): sys.stdout.write(line) if new_debpack_info is not None: logger.debug("Adding the new debian package info to manifest") fout = open(debpack_manifest_fname, "a") fout.write(new_debpack_line) fout.close() else: logger.debug("Removed the debian package info from manifest") def modify_Packages_info(debpack_dirname, new_debpack_info, prefix_pkg_remove): """ Modify the Packages file to include the new debian information """ debpack_Packages_fname = os.path.join(debpack_dirname, "Packages") try: output = [] f = open(debpack_Packages_fname, "r+") for stanza in Packages.iter_paragraphs(f): if not should_remove(stanza["Package"], prefix_pkg_remove): output.append(stanza) if new_debpack_info is not None: output.append(DummyStanza(new_debpack_info)) f.seek(0,0) f.truncate() for stanza in output: stanza.dump(f) f.write("\n") finally: f.close() def main(): # Validate that all the required information is passed on the command line args = parser.parse_args() if (args.hwpack_name == None or args.prefix_pkg_remove == None): parser.print_help() parser.error("You must specify both hwpack name "\ "and the debian package information\n") return 1 global logger logger = get_logger(debug=args.debug) old_hwpack = args.hwpack_name new_deb_file_to_copy = args.deb_pack prefix_pkg_remove = args.prefix_pkg_remove build_number = args.build_number status = 0 tempdir = "" try: # Get the new hardware pack name hwpack_name = get_hwpack_name(old_hwpack, build_number) if hwpack_name == None: logger.error("Did not get a valid hwpack name, exiting") return status # untar the hardware pack and extract all the files in it tar = tarfile.open(old_hwpack, "r:gz") tempdir = tempfile.mkdtemp() tar.extractall(tempdir) tar.close() # Search if a similar package with the same name exists, if yes then # replace it. IF the old and new debian have the same name then we # are still replacing the old one with the new one. debpack_dirname = os.path.join(tempdir, 'pkgs/') if not os.path.exists(debpack_dirname): logger.error("Failed to extract the hwpack: %s ", old_hwpack) return status new_debpack_info = None if new_deb_file_to_copy is not None: new_debpack_info = FetchedPackage.from_deb(new_deb_file_to_copy) verify_existing_debians(debpack_dirname, prefix_pkg_remove) # Copy the new debian file to the pkgs dir, if new_deb_file_to_copy is not None: shutil.copy2(new_deb_file_to_copy, debpack_dirname) modify_manifest_info(tempdir, new_debpack_info, prefix_pkg_remove) modify_Packages_info(debpack_dirname, new_debpack_info, prefix_pkg_remove) # Compress the hardware pack with the new debian file included in it tar = tarfile.open(hwpack_name , "w:gz") origdir = os.getcwd() os.chdir(tempdir) for file_name in glob.glob('*'): tar.add(file_name, recursive=True) tar.close() # Retain old hwpack name instead of using a new name os.chdir(origdir) if args.inplace: os.rename(hwpack_name, old_hwpack) hwpack_name = old_hwpack # Export the updated manifest file manifest_name = hwpack_name.replace('.tar.gz', '.manifest.txt') shutil.copy2(os.path.join(tempdir, 'manifest'), manifest_name) except Exception, details: logger.error("Error Details: %s", details) status = 1 finally: if os.path.exists(tempdir): shutil.rmtree(tempdir) if status == 0: logger.info("The debian package '%s' has been been included in '%s'", new_deb_file_to_copy, hwpack_name) print hwpack_name else: logger.error("Injecting the debian package '%s' failed", new_deb_file_to_copy) return status if __name__ == '__main__': sys.exit(main()) linaro-image-tools-2016.05.orig/initrd-do0000755000175000017500000000763212724020110017540 0ustar voipiovoipio#!/bin/sh # initrd-do - run a command within an initrd # Copyright (C) 2008 Loïc Minier # Copyright (C) 2011 Linaro Limited # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), # to deal in the Software without restriction, including without limitation # the rights to use, copy, modify, merge, publish, distribute, sublicense, # and/or sell copies of the Software, and to permit persons to whom the # Software is furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL # SOFTWARE IN THE PUBLIC INTEREST, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER # DEALINGS IN THE SOFTWARE. # # Except as contained in this notice, the name of the author shall not be used # in advertising or otherwise to promote the sale, use or other dealings in # this Software without prior written authorization from the author. # # depends: sudo set -e self="$(basename "$0")" work_dir="" new_initrd="" log() { echo "$*" >&2 } log_i() { log "I:" "$@" } log_w() { log "W:" "$@" } die() { log "E:" "$@" exit 1 } usage() { log "Usage: $self --initrd [-c [ ...]]" } getshell() { if [ -n "$SHELL" ]; then echo "$SHELL" return fi PASSWD="$(getent passwd $(id -u) | sed -n 's/.*://p')" if [ -n "$PASSWD" ]; then echo "$PASSWD" return fi echo "/bin/sh" } escape() { echo "$*" | sed "s/'/'\"'\"'/g; s/.*/'&'/" } # unused; use eval unescape () { eval "echo" "$*" } cleanup() { if [ -n "$new_initrd" ]; then rm -f "$new_initrd" fi if [ -n "$work_dir" ]; then rm -rf "$work_dir" fi } trap "cleanup" 0 1 2 3 9 11 13 15 initrd="" command="$(escape "$(getshell)")" while [ $# -gt 0 ]; do case $1 in --help) usage exit 0 ;; -i|--initrd) shift initrd="$1" if ! shift; then die "Need an initrd file after --initrd" fi ;; -c) shift if [ $# -le 0 ]; then die "Need a command after -c" fi command="" while :; do command="$command $(escape "$1")" shift if [ $# -eq 0 ]; then break fi done break ;; *) # clever handling of args: set initrd first if unset, otherwise set # command if [ -z "$initrd" ]; then initrd="$1" shift else command="" while :; do command="$command $(escape "$1")" shift if [ $# -eq 0 ]; then break fi done fi ;; esac done if [ -z "$initrd" ]; then usage exit 1 fi work_dir="$(mktemp -dt "$self.XXXXXXXXXX")" # subshell to not touch cwd as we will nuke it ( cd "$work_dir" log_i "Unpacking initrd $initrd" gunzip -c <"$initrd" | cpio -i --quiet log_i "Running command:" $command eval $command ) err=$? if [ 0 != $err ]; then log_w "Command exited with $err; aborting" exit 1 fi log_i "Generating new initrd" new_initrd="$(mktemp -t "$self.XXXXXXXXXX")" # subshell to not touch cwd as we will nuke it ( cd "$work_dir" find . | cpio -o -H newc -R 0:0 --quiet | gzip -c >"$new_initrd" ) log_i "Moving new initrd to $initrd" mv -f "$new_initrd" "$initrd" linaro-image-tools-2016.05.orig/README0000644000175000017500000000340612724020110016574 0ustar voipiovoipioThis project contains a collection of tools to help creating Linaro bootable images to be used in development boards. = Running linaro-media-create = To run linaro-media-create you'll need to make sure the following packages are installed: - parted - dosfstools - u-boot-tools or uboot-mkimage - python-argparse - python-dbus - python-debian >= 0.1.16ubuntu1 - python-parted - qemu-user-static >= 0.13.0 (only if you're running on x86) - btrfs-tools - command-not-found - python-yaml = Running tests = Before running any tests you need to install the following packages: - testrepository - python-testtools >= 0.9.12 (available at https://launchpad.net/~linaro-maintainers/+archive/tools) - python-debian >= 0.1.16ubuntu1 - python-argparse - dpkg-dev - python-parted - python-dbus (and dbus, udisks) - python-apt - qemu-kvm - sfdisk (from util-linux) - apt-utils - pep8 - python-mock - python-commandnotfound (as of Ubuntu 12.10) Also consider installing pyflakes, which is optional but will enable more tests. Some of the packages above require fixes, which are as of Ubuntu 12.04 are not yet in upstream, so using Linaro PPA is required: $ sudo add-apt-repository ppa:linaro-maintainers/tools Then, you can install all packages listed above in one go with the following command: $ sudo apt-get install testrepository python-testtools python-debian \ python-argparse dpkg-dev python-parted dbus udisks python-dbus \ python-apt qemu-kvm util-linux apt-utils pep8 pyflakes \ python-commandnotfound To initialized testsuite, run the following command: $ testr init After that you can run the whole test suite with: $ testr run If you want to learn more about testrepository: $ testr quickstart linaro-image-tools-2016.05.orig/linaro-media-create0000755000175000017500000002424312724020110021446 0ustar voipiovoipio#!/usr/bin/env python # Copyright (C) 2010, 2011 Linaro # # Author: Guilherme Salgado # # This file is part of Linaro Image Tools. # # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # Linaro Image Tools is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with Linaro Image Tools. If not, see . import atexit import os import sys import tempfile from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.boards import get_board_config from linaro_image_tools.media_create.check_device import ( confirm_device_selection_and_ensure_it_is_ready) from linaro_image_tools.media_create.chroot_utils import ( install_hwpacks, install_packages, ) from linaro_image_tools.hwpack.hwpack_reader import ( HwpackReader, HwpackReaderError, ) from linaro_image_tools.media_create.partitions import ( Media, setup_partitions, get_uuid, ) from linaro_image_tools.media_create.rootfs import populate_rootfs from linaro_image_tools.media_create.unpack_binary_tarball import ( unpack_binary_tarball, ) from linaro_image_tools.media_create import get_args_parser from linaro_image_tools.utils import ( additional_option_checks, check_file_integrity_and_log_errors, check_required_args, ensure_command, IncompatibleOptions, is_arm_host, MissingRequiredOption, path_in_tarfile_exists, prep_media_path, get_logger, UnableToFindPackageProvidingCommand, disable_automount, enable_automount, ) # Just define the global variables TMP_DIR = None ROOTFS_DIR = None BOOT_DISK = None ROOT_DISK = None # Registered as the first atexit handler as we want this to be the last # handler to execute. def cleanup_tempdir(): """Remove TEMP_DIR with all its contents. Before doing so, make sure BOOT_DISK and ROOT_DISK are not mounted. """ devnull = open('/dev/null', 'w') # ignore non-zero return codes for disk in BOOT_DISK, ROOT_DISK: if disk is not None: try: cmd_runner.run(['umount', disk], stdout=devnull, stderr=devnull, as_root=True).wait() except cmd_runner.SubcommandNonZeroReturnValue: pass # Remove TMP_DIR as root because some files written there are # owned by root. if TMP_DIR is not None: cmd_runner.run(['rm', '-rf', TMP_DIR], as_root=True).wait() def ensure_required_commands(args): """Ensure we have the commands that we know are going to be used.""" required_commands = [ 'mkfs.vfat', 'sfdisk', 'mkimage', 'parted', 'gpg', 'sha1sum', 'sgdisk'] if not is_arm_host(): required_commands.append('qemu-arm-static') if args.rootfs in ['btrfs', 'ext2', 'ext3', 'ext4']: required_commands.append('mkfs.%s' % args.rootfs) else: raise AssertionError('Unsupported rootfs type %s' % args.rootfs) for command in required_commands: try: ensure_command(command) except UnableToFindPackageProvidingCommand: logger.error("Could not look up command %s. Please ensure that command %s is installed." % (command, command)) raise if __name__ == '__main__': parser = get_args_parser() args = parser.parse_args() logger = get_logger(debug=args.debug) try: additional_option_checks(args) except IncompatibleOptions as e: parser.print_help() logger.error(e.value) sys.exit(1) if args.readhwpack: try: reader = HwpackReader(args.hwpacks) logger.info(reader.get_supported_boards()) sys.exit(0) except HwpackReaderError as e: logger.error(e.value) sys.exit(1) try: check_required_args(args) except MissingRequiredOption as e: parser.print_help() logger.error(e.value) sys.exit(1) # Do this by default, disable automount options and re-enable them at exit. disable_automount() atexit.register(enable_automount) board_config = get_board_config(args.dev) board_config.set_metadata(args.hwpacks, args.bootloader, args.dev, args.dtb_file) board_config.add_boot_args(args.extra_boot_args) board_config.add_boot_args_from_file(args.extra_boot_args_file) media = Media(prep_media_path(args)) if media.is_block_device: if not board_config.supports_writing_to_mmc: logger.error("The board '%s' does not support the --mmc option. " "Please use --image_file to create an image file for " "this board." % args.dev) sys.exit(1) if not confirm_device_selection_and_ensure_it_is_ready( args.device, args.nocheck_mmc): sys.exit(1) elif not args.should_format_rootfs or not args.should_format_bootfs: logger.error("Do not use --no-boot or --no-part in conjunction with " "--image_file.") sys.exit(1) # If --help was specified this won't execute. # Create temp dir and initialize rest of path vars. TMP_DIR = tempfile.mkdtemp() BOOT_DISK = os.path.join(TMP_DIR, 'boot-disc') ROOT_DISK = os.path.join(TMP_DIR, 'root-disc') BIN_DIR = os.path.join(TMP_DIR, 'rootfs') os.mkdir(BIN_DIR) logger.info('Searching correct rootfs path') # Identify the correct path for the rootfs filesystem_dir = '' if path_in_tarfile_exists('binary/etc', args.binary): filesystem_dir = 'binary' elif path_in_tarfile_exists('binary/boot/filesystem.dir', args.binary): # The binary image is in the new live format. filesystem_dir = 'binary/boot/filesystem.dir' ROOTFS_DIR = os.path.join(BIN_DIR, filesystem_dir) try: ensure_required_commands(args) except UnableToFindPackageProvidingCommand: sys.exit(1) sig_file_list = args.hwpacksigs[:] if args.binarysig is not None: sig_file_list.append(args.binarysig) # Check that the signatures that we have been provided (if any) match # the hwpack and OS binaries we have been provided. If they don't, quit. files_ok, verified_files = check_file_integrity_and_log_errors( sig_file_list, args.binary, args.hwpacks) if not files_ok: sys.exit(1) atexit.register(cleanup_tempdir) unpack_binary_tarball(args.binary, BIN_DIR) # if compatible system, extract all packages os_release_id = 'linux' os_release_file = '%s/etc/os-release' % ROOTFS_DIR if os.path.exists(os_release_file): for line in open(os_release_file): if line.startswith('ID='): os_release_id = line[(len('ID=')):] os_release_id = os_release_id.strip('\"\n') break if os_release_id == 'debian' or os_release_id == 'ubuntu' or \ os.path.exists('%s/etc/debian_version' % ROOTFS_DIR): extract_kpkgs = False elif os_release_id == 'fedora': extract_kpkgs = False else: extract_kpkgs = True hwpacks = args.hwpacks lmc_dir = os.path.dirname(__file__) if lmc_dir == '': lmc_dir = None install_hwpacks(ROOTFS_DIR, TMP_DIR, lmc_dir, args.hwpack_force_yes, verified_files, extract_kpkgs, *hwpacks) if args.rootfs == 'btrfs': if not extract_kpkgs: logger.info("Desired rootfs type is 'btrfs', trying to " "auto-install the 'btrfs-tools' package") install_packages(ROOTFS_DIR, TMP_DIR, "btrfs-tools") else: logger.info("Desired rootfs type is 'btrfs', please make sure the " "rootfs also includes 'btrfs-tools'") boot_partition, root_partition = setup_partitions( board_config, media, args.image_size, args.boot_label, args.rfs_label, args.rootfs, args.should_create_partitions, args.should_format_bootfs, args.should_format_rootfs, args.should_align_boot_part, args.part_table) uuid = get_uuid(root_partition) # In case we're only extracting the kernel packages, avoid # using uuid because we don't have a working initrd if extract_kpkgs: # XXX: workaround https://bugs.launchpad.net/bugs/1208815 # When we use OE, we don't have initrd/UUID and fallback to pass # root=/dev/mmcblk0p3 to the kernel. It's based on mmc_option value # provided by the hardware configuration (mmc_id: '0:2'). # At U-Boot stage, the value is correct and we load from mmc 0:2. # At the kernel stage, the value becomes incorrect because Arndale # has eMMC and rootfs can be found on /dev/mmcblk1p3. # Since the boot commands are calculated based on the same mmc_id # parameter, Arndale can't boot in this use case. if board_config.board == 'arndale': board_config.mmc_device_id = board_config.mmc_device_id + 1 # XXX: this needs to be smarter as we can't always assume mmcblk devices rootfs_id = '/dev/mmcblk%dp%s' % ( board_config.mmc_device_id, 2 + board_config.mmc_part_offset) else: rootfs_id = "UUID=%s" % uuid if args.should_format_bootfs: board_config.populate_boot( ROOTFS_DIR, rootfs_id, boot_partition, BOOT_DISK, media.path, args.is_live, args.is_lowmem, args.consoles) if args.should_format_rootfs: create_swap = False if args.swap_file is not None: create_swap = True populate_rootfs(ROOTFS_DIR, ROOT_DISK, root_partition, args.rootfs, rootfs_id, create_swap, str(args.swap_file), board_config.mmc_device_id, board_config.mmc_part_offset, os_release_id, board_config) logger.info("Done creating Linaro image on %s" % media.path) linaro-image-tools-2016.05.orig/setup.py0000755000175000017500000000122312724020110017424 0ustar voipiovoipio#!/usr/bin/env python # https://launchpad.net/python-distutils-extra import DistUtilsExtra.auto from linaro_image_tools.__version__ import __version__ DistUtilsExtra.auto.setup( name="linaro-image-tools", version=__version__, description="Tools to create and write Linaro images", url="https://launchpad.net/linaro-image-tools", license="GPL v3 or later", author='Linaro Infrastructure team', author_email="linaro-dev@lists.linaro.org", scripts=[ "initrd-do", "linaro-hwpack-create", "linaro-hwpack-install", "linaro-media-create", "linaro-android-media-create", "linaro-hwpack-replace"], )