pax_global_header00006660000000000000000000000064130156170660014517gustar00rootroot0000000000000052 comment=1d398b56d2e11a43e909904554f625920dc2a7b4 nageru-1.4.2/000077500000000000000000000000001301561706600130045ustar00rootroot00000000000000nageru-1.4.2/.gitignore000066400000000000000000000002761301561706600150010ustar00rootroot00000000000000*.d *.o *.pb.h *.pb.cc *.moc.cpp ui_aboutdialog.h ui_audio_expanded_view.h ui_audio_miniview.h ui_display.h ui_input_mapping.h ui_mainwindow.h ui_midi_mapping.h nageru benchmark_audio_mixer nageru-1.4.2/.gitmodules000066400000000000000000000001041301561706600151540ustar00rootroot00000000000000[submodule "bmusb"] path = bmusb url = http://git.sesse.net/bmusb nageru-1.4.2/COPYING000066400000000000000000001045131301561706600140430ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . nageru-1.4.2/Makefile000066400000000000000000000070411301561706600144460ustar00rootroot00000000000000CXX=g++ PROTOC=protoc INSTALL=install EMBEDDED_BMUSB=no PKG_MODULES := Qt5Core Qt5Gui Qt5Widgets Qt5OpenGLExtensions Qt5OpenGL libusb-1.0 movit lua52 libmicrohttpd epoxy x264 protobuf libpci CXXFLAGS ?= -O2 -g -Wall # Will be overridden by environment. CXXFLAGS += -std=gnu++11 -fPIC $(shell pkg-config --cflags $(PKG_MODULES)) -pthread -DMOVIT_SHADER_DIR=\"$(shell pkg-config --variable=shaderdir movit)\" -Idecklink/ ifeq ($(EMBEDDED_BMUSB),yes) CPPFLAGS += -Ibmusb/ else PKG_MODULES += bmusb endif LDLIBS=$(shell pkg-config --libs $(PKG_MODULES)) -pthread -lva -lva-drm -lva-x11 -lX11 -lavformat -lavcodec -lavutil -lswscale -lavresample -lzita-resampler -lasound -ldl # Qt objects OBJS_WITH_MOC = glwidget.o mainwindow.o vumeter.o lrameter.o compression_reduction_meter.o correlation_meter.o aboutdialog.o input_mapping_dialog.o midi_mapping_dialog.o nonlinear_fader.o OBJS += $(OBJS_WITH_MOC) OBJS += $(OBJS_WITH_MOC:.o=.moc.o) ellipsis_label.moc.o clickable_label.moc.o OBJS += vu_common.o piecewise_interpolator.o main.o OBJS += midi_mapper.o midi_mapping.pb.o # Mixer objects AUDIO_MIXER_OBJS = audio_mixer.o alsa_input.o alsa_pool.o ebu_r128_proc.o stereocompressor.o resampling_queue.o flags.o correlation_measurer.o filter.o input_mapping.o state.pb.o OBJS += mixer.o pbo_frame_allocator.o context.o ref_counted_frame.o theme.o httpd.o flags.o image_input.o alsa_output.o disk_space_estimator.o $(AUDIO_MIXER_OBJS) # Streaming and encoding objects OBJS += quicksync_encoder.o x264_encoder.o x264_speed_control.o video_encoder.o metacube2.o mux.o audio_encoder.o ffmpeg_raii.o # DeckLink OBJS += decklink_capture.o decklink/DeckLinkAPIDispatch.o # bmusb ifeq ($(EMBEDDED_BMUSB),yes) OBJS += bmusb/bmusb.o bmusb/fake_capture.o endif # Benchmark program. BM_OBJS = benchmark_audio_mixer.o $(AUDIO_MIXER_OBJS) flags.o %.o: %.cpp $(CXX) -MMD -MP $(CPPFLAGS) $(CXXFLAGS) -o $@ -c $< %.o: %.cc $(CXX) -MMD -MP $(CPPFLAGS) $(CXXFLAGS) -o $@ -c $< %.pb.cc %.pb.h : %.proto $(PROTOC) --cpp_out=. $< %.h: %.ui uic $< -o $@ %.moc.cpp: %.h moc $< -o $@ all: nageru benchmark_audio_mixer nageru: $(OBJS) $(CXX) -o $@ $^ $(LDFLAGS) $(LDLIBS) benchmark_audio_mixer: $(BM_OBJS) $(CXX) -o $@ $^ $(LDFLAGS) $(LDLIBS) # Extra dependencies that need to be generated. aboutdialog.o: ui_aboutdialog.h alsa_pool.o: state.pb.h audio_mixer.o: state.pb.h input_mapping.o: state.pb.h input_mapping_dialog.o: ui_input_mapping.h mainwindow.o: ui_mainwindow.h ui_display.h ui_audio_miniview.h ui_audio_expanded_view.h ui_midi_mapping.h mainwindow.o: midi_mapping.pb.h midi_mapper.o: midi_mapping.pb.h midi_mapping_dialog.o: ui_midi_mapping.h midi_mapping.pb.h DEPS=$(OBJS:.o=.d) $(BM_OBJS:.o=.d) -include $(DEPS) clean: $(RM) $(OBJS) $(BM_OBJS) $(DEPS) nageru benchmark_audio_mixer ui_aboutdialog.h ui_mainwindow.h ui_display.h ui_about.h ui_audio_miniview.h ui_audio_expanded_view.h ui_input_mapping.h ui_midi_mapping.h chain-*.frag *.dot *.pb.cc *.pb.h $(OBJS_WITH_MOC:.o=.moc.cpp) ellipsis_label.moc.cpp clickable_label.moc.cpp PREFIX=/usr/local install: $(INSTALL) -m 755 -o root -g root -d $(DESTDIR)$(PREFIX)/bin $(DESTDIR)$(PREFIX)/share/nageru $(INSTALL) -m 755 -o root -g root nageru $(DESTDIR)$(PREFIX)/bin/nageru $(INSTALL) -m 644 -o root -g root theme.lua $(DESTDIR)$(PREFIX)/share/nageru/theme.lua $(INSTALL) -m 644 -o root -g root simple.lua $(DESTDIR)$(PREFIX)/share/nageru/simple.lua $(INSTALL) -m 644 -o root -g root bg.jpeg $(DESTDIR)$(PREFIX)/share/nageru/bg.jpeg $(INSTALL) -m 644 -o root -g root akai_midimix.midimapping $(DESTDIR)$(PREFIX)/share/nageru/akai_midimix.midimapping nageru-1.4.2/NEWS000066400000000000000000000151741301561706600135130ustar00rootroot00000000000000Nageru 1.4.2, November 24th, 2016 - Fix a thread race that would sometimes cause x264 streaming to go awry. Nageru 1.4.1, November 6th, 2016 - Various bugfixes. Nageru 1.4.0, October 26th, 2016 - Support for multichannel (or more accurately, multi-bus) audio, choosable from the UI or using the --multichannel command-line flag. In multichannel mode, you can take in inputs from multiple different sources (or different channels on the same source, for multichannel sound cards), apply effects to them separately and then mix them together. This includes both audio from the video cards as well as ALSA inputs, including hotplug. Ola Gundelsby contributed invaluable feedback on this feature throughout the entire development cycle. - Support for having MIDI controllers control various aspects of the audio UI, with relatively flexible mapping. Note that different MIDI controllers can vary significantly in what protocol they speak, so Nageru will not necessarily work with all. (The primary testing controller has been the Akai MIDImix, and a pre-made mapping for that is included. The Korg nanoKONTROL2 has also been tested and works, but it requires some Korg-specific SysEx commands to make the buttons and lights work.) - Add a disk space indicator to the main window. - Various bugfixes. In particular, an issue where the audio would pitch up sharply after a series of many dropped frames has been fixed. Nageru 1.3.4, August 2nd, 2016 - Various bugfixes. Nageru 1.3.3, July 27th, 2016 - Various changes to make distribution packaging easier; in particular, theme data can be picked up from /usr/local/share/nageru. - Fix various FFmpeg deprecation warnings, now that we need FFmpeg 3.1 for other reasons anyway. Nageru 1.3.2, July 23rd, 2016 - Allow limited hotplugging (unplugging and replugging) of USB cards. You can use the new command-line option --num-fake-cards (-C) to add fake cards that show only a single color and that will be replaced by real cards as you plug them in; you can also unplug cards and have them be replaced by fake cards. Fake cards can also be used for testing Nageru without actually having any video cards available. - Add Metacube timestamping of every keyframe, for easier detection of streams not keeping up. Works with the new timestamp feature of Cubemap 1.3.1. Will be ignored (save for some logging) in older Cubemap versions. - The included default theme has been reworked and cleaned up to be more understandable and extensible. - Add more command-line options for initial audio setup. Nageru 1.3.1, July 1st, 2016 - Various display bugfixes. Nageru 1.3.0, June 26th, 2016 - It is now possible, given enough CPU power (e.g., a quad-core Haswell or faster desktop CPU), to output a stream that is suitable for streaming directly to end users without further transcoding. In particular, this includes support for encoding the network stream with x264 (the stream saved to disk is still done using Quick Sync), for Metacube framing (for streaming to the Cubemap reflector), and for choosing the network stream mux. For more information, see the README. - Add a flag (--disable-alsa-output) to disable ALSA monitoring output. - Do texture uploads from the main thread instead of from separate threads; may or may not improve stability with NVIDIA's proprietary drivers. - When beginning a new video segment, the shutdown of the old encoder is now done in a background thread, in order to not disturb the external stream. The audio still goes into a somewhat random stream, though. - You can now override the default stream-to-card mapping with --map-signal= on the command line. - Nageru now tries to lock itself into RAM if it has the permissions to do so, for better realtime behavior. (Writing the stream to disk tends to fill the buffer cache, eventually paging less-used parts of Nageru out.) - Various fixes for deadlocks, memory leaks, and many other errors. Nageru 1.2.1, April 15th, 2016 - Images are now updated from disk about every second, so that it is possible to update e.g. overlays during streaming, although somewhat slowly. - Fix support for PNG images. - You can now send SIGHUP to start a new cut instead of using the menu. - Added a --help option. - Various tweaks to OpenGL fence handling. Nageru 1.2.0, April 6th, 2016 - Support for Blackmagic's PCI and Thunderbolt cards, using the official (closed-source) Blackmagic drivers. (You do not need the SDK installed, though.) You can use PCI and USB cards pretty much interchangeably. - Much more stable handling of frame queues on non-master cards. In particular, you can have a master card on 50 Hz and another card on 60 Hz without getting lots of warning messages and a 10+ frame latency on the second card. - Many new options in the right click menu on cards: Adjustable video inputs, adjustable audio inputs, adjustable resolutions, ability to select card for master clock. - Add support for starting with almost all audio processing turned off (--flat-audio). - The UI now marks inputs with red or green to mark them as participating in the live or preview signal, respectively. Red takes priority. (Actually, it merely asks the theme for a color for each input; the theme contains the logic.) - Add support for uncompressed video instead of H.264 on the HTTP server, while still storing H.264 to files (--http-uncompressed-video). Note that depending on your client, this might not actually be more CPU efficient even on localhost, so be sure to check. - Add a simpler, less featureful theme (simple.lua) that should be easier to understand for beginners. Themes are now also choosable with -t on the command line. - Too many bugfixes and small tweaks to list. In particular, many memory leaks in the streaming part have been identified and fixed. Nageru 1.1.0, February 24th, 2016 - Support doing the H.264 encoding on a different graphics device from the one doing the mixing. In particular, this makes it possible to use Nageru on an NVIDIA GPU while still encoding H.264 video using Intel Quick Sync (NVENC is not supported yet) -- it is less efficient since the data needs to be read back via the CPU, but the NVIDIA cards and drivers are so much faster that it doesn't really matter. Tested on a GTX 950 with the proprietary drivers. - In the included example theme, fix fading to/from deinterlaced sources. - Various smaller compilation, distribution and documentation fixes. Nageru 1.0.0, January 30th, 2016 - Initial release. nageru-1.4.2/README000066400000000000000000000236341301561706600136740ustar00rootroot00000000000000Nageru is a live video mixer, based around the standard M/E workflow. Features: - High performance on modest hardware (720p60 with two input streams on my Thinkpad X240[1]); almost all pixel processing is done on the GPU. - High output quality; Lanczos3 scaling, subpixel precision everywhere, white balance adjustment, mix of 16- and 32-bit floating point for intermediate calculations, dithered output. - Proper sound support: Syncing of multiple unrelated sources through high-quality resampling, multichannel mixing with separate effects per-bus, cue out for headphones, dynamic range compression, three-band graphical EQ (pluss a fixed low-cut), level meters conforming to EBU R128, automation via MIDI controllers. - Theme engine encapsulating the design demands of each individual event; Lua code is responsible for setting up the pixel processing pipelines, running transitions etc., so that the visual look is consistent between operators. [1] For reference, that is: Core i7 4600U (dualcore 2.10GHz, clocks down to 800 MHz after 30 seconds due to thermal constraints), Intel HD Graphics 4400 (ie., without the extra L4 cache from Iris Pro), single-channel DDR3 RAM (so 12.8 GB/sec theoretical memory bandwidth, shared between CPU and GPU). Nageru is in beta stage. It currently needs: - An Intel processor with Intel Quick Sync, or otherwise some hardware H.264 encoder exposed through VA-API. Note that you can use VA-API over DRM instead of X11, to use a non-Intel GPU for rendering but still use Quick Sync (by giving e.g. “--va-display /dev/dri/renderD128”). - Two or more Blackmagic USB3 or PCI cards, either HDMI or SDI. The PCI cards need Blackmagic's own drivers installed. The USB3 cards are driven through the “bmusb” driver, using libusb-1.0. If you want zerocopy USB, you need libusb 1.0.21 or newer, as well as a recent kernel (4.6.0 or newer). Zerocopy USB helps not only for performance, but also for stability. - Movit, my GPU-based video filter library (https://movit.sesse.net). You will need at least version 1.3.1, but at least 1.4.0 is recommended. - Qt 5.5 or newer for the GUI. - libmicrohttpd for the embedded web server. - x264 for encoding high-quality video suitable for streaming to end users. - ffmpeg for muxing, and for encoding audio. You will need at least version 3.1. - Working OpenGL; Movit works with almost any modern OpenGL implementation. Nageru has been tested with Intel on Mesa (you want 11.2 or newer, due to critical stability bugfixes), and with NVIDIA's proprietary drivers. AMD's proprietary drivers (fglrx) are known not to work due to driver bugs; I am in contact with AMD to try to get this resolved. - libzita-resampler, for resampling sound sources so that they are in sync between sources, and also for oversampling for the peak meter. - Lua, for driving the theme engine. - libpci, for printing friendly PCI device names in an error message. If on Debian stretch or something similar, you can install everything you need with: apt install qtbase5-dev libqt5opengl5-dev qt5-default pkg-config libmicrohttpd-dev \ libusb-1.0-0-dev liblua5.2-dev libzita-resampler-dev libva-dev \ libavcodec-dev libavformat-dev libswscale-dev libavresample-dev \ libmovit-dev libegl1-mesa-dev libasound2-dev libx264-dev libbmusb-dev \ protobuf-compiler libprotobuf-dev libpci-dev Exceptions as of November 2016: - libusb 1.0.21 is not yet in stretch; you need to fetch it from sid. Same with a bmusb version built against it. The patches/ directory contains a patch that helps zita-resampler performance. It is meant for upstream, but was not in at the time Nageru was released. It is taken to be by Steinar H. Gunderson (ie., my ex-work email), and under the same license as zita-resampler itself. To start it, just hook up your equipment, type “make” and then “./nageru”. It is strongly recommended to have the rights to run at real-time priority; it will make the USB3 threads do so, which will make them a lot more stable. (A reasonable hack for testing is probably just to run it as root using sudo, although you might not want to do that in production.) Note also that if you are running a desktop compositor, it will steal significant amounts of GPU performance. The same goes for PulseAudio. Nageru will open a HTTP server at port 9095, where you can extract a live H264+PCM signal in nut mux (e.g. http://127.0.0.1:9095/stream.nut). It is probably too high bitrate (~25 Mbit/sec depending on content) to send to users, but you can easily send it around in your internal network and then transcode it in e.g. VLC. A copy of the stream (separately muxed) will also be saved live to local disk. If you have a fast CPU (typically a quadcore desktop; most laptops will spend most of their CPU on running Nageru itself), you can use x264 for the outgoing stream instead of Quick Sync; it is much better quality for the same bitrate, and also has proper bitrate controls. Simply add --http-x264-video on the command line. (You may also need to add something like "--x264-preset veryfast", since the default "medium" preset might be too CPU-intensive, but YMMV.) The stream saved to disk will still be the Quick Sync-encoded stream, as it is typically higher bitrate and thus also higher quality. Note that if you add ".metacube" at the end of the URL (e.g. "http://127.0.0.1:9095/stream.ts.metacube"), you will get a stream suitable for streaming through the Cubemap video reflector (cubemap.sesse.net). A typical example would be: ./nageru --http-x264-video --x264-preset veryfast --x264-tune film \ --http-mux mp4 --http-audio-codec libfdk_aac --http-audio-bitrate 128 If you are comfortable with using all your remaining CPU power on the machine for x264, try --x264-speedcontrol, which will try to adjust the preset dynamically for maximum quality, at the expense of somewhat higher delay. See --help for more information on options in general. The name “Nageru” is a play on the Japanese verb 投げる (nageru), which means to throw or cast. (I also later learned that it could mean to face defeat or give up, but that's not the intended meaning.) Nageru's home page is at https://nageru.sesse.net/, where you can also find contact information and link to the latest version. Legalese: TL;DR: Everything is GPLv3-or-newer compatible, and see Intel's copyright license at quicksync_encoder.h. Nageru is Copyright (C) 2015 Steinar H. Gunderson . Portions Copyright (C) 2003 Rune Holm. Portions Copyright (C) 2010-2015 Fons Adriaensen . Portions Copyright (C) 2012-2015 Fons Adriaensen . Portions Copyright (C) 2008-2015 Fons Adriaensen . Portions Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Portions of quicksync_encoder.h and quicksync_encoder.cpp: Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sub license, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. All files in decklink/: Copyright (c) 2009 Blackmagic Design Copyright (c) 2015 Blackmagic Design Permission is hereby granted, free of charge, to any person or organization obtaining a copy of the software and accompanying documentation covered by this license (the "Software") to use, reproduce, display, distribute, execute, and transmit the Software, and to prepare derivative works of the Software, and to permit third-parties to whom the Software is furnished to do so, all subject to the following: The copyright notices in the Software and this entire statement, including the above license grant, this restriction and the following disclaimer, must be included in all copies of the Software, in whole or in part, and all derivative works of the Software, unless such copies or derivative works are solely in the form of machine-executable object code generated by a source language processor. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. nageru-1.4.2/aboutdialog.cpp000066400000000000000000000004131301561706600160000ustar00rootroot00000000000000#include "aboutdialog.h" #include #include "ui_aboutdialog.h" using namespace std; AboutDialog::AboutDialog() : ui(new Ui::AboutDialog) { ui->setupUi(this); connect(ui->button_box, &QDialogButtonBox::accepted, [this]{ this->close(); }); } nageru-1.4.2/aboutdialog.h000066400000000000000000000004561301561706600154540ustar00rootroot00000000000000#ifndef _ABOUTDIALOG_H #define _ABOUTDIALOG_H 1 #include #include class QObject; namespace Ui { class AboutDialog; } // namespace Ui class AboutDialog : public QDialog { Q_OBJECT public: AboutDialog(); private: Ui::AboutDialog *ui; }; #endif // !defined(_ABOUTDIALOG_H) nageru-1.4.2/akai_midimix.midimapping000066400000000000000000000135101301561706600176510ustar00rootroot00000000000000# Example mapping for the Akai MIDImix. This one is written by hand, # and serves as a simple example of the basic features. The MIDImix # doesn't have a ton of controls, so not everything is mapped up, # and some "wrong" mappings need to be done; in particular, we've set up # two controller banks and switch between them with the BANK LEFT and # BANK RIGHT buttons (which are normally meant to switch between channels # 1–8 and 9–16, as I understand it). # # The mappings for the 270° pots on each bus are: # # Bank 1: Treble, mid, bass # Bank 2: Gain, compressor threshold, (globals) # # The “(globals)” here are only for use on the two rightmost buses: # The third pot on bus 7 controls the lo-cut cutoff, and the pot on # bus 8 controls the limiter threshold. # # The mute button controls muting (obviously) for that bus, and the solo # button (accessible by holding the global solo button and pressing the # mute button for the bus) is abused for toggling auto gain staging. # # The REC ARM button for each bus is abused to be a “has peaked” meter; # pressing it will reset the measurement. # # Finally, the faders work pretty much as you'd expect; each bus' fader # is connected to the volume for that bus, and the master fader is # connected to the global makeup gain. num_controller_banks: 2 treble_bank: 0 mid_bank: 0 bass_bank: 0 gain_bank: 1 compressor_threshold_bank: 1 locut_bank: 1 limiter_threshold_bank: 1 # Bus 1. We also store the master controller here. bus_mapping { treble { controller_number: 16 } mid { controller_number: 17 } bass { controller_number: 18 } gain { controller_number: 16 } compressor_threshold { controller_number: 17 } fader { controller_number: 19 } toggle_mute { note_number: 1 } toggle_auto_gain_staging { note_number: 2 } clear_peak { note_number: 3 } # Master. makeup_gain { controller_number: 62 } select_bank_1 { note_number: 25 # Bank left. } select_bank_2 { note_number: 26 # Bank right. } # Lights. is_muted { note_number: 1 } auto_gain_staging_is_on { note_number: 2 } has_peaked { note_number: 3 } # Global lights. bank_1_is_selected { note_number: 25 } bank_2_is_selected { note_number: 26 } } # Bus 2. bus_mapping { treble { controller_number: 20 } mid { controller_number: 21 } bass { controller_number: 22 } gain { controller_number: 20 } compressor_threshold { controller_number: 21 } fader { controller_number: 23 } toggle_mute { note_number: 4 } toggle_auto_gain_staging { note_number: 5 } clear_peak { note_number: 6 } # Lights. is_muted { note_number: 4 } auto_gain_staging_is_on { note_number: 5 } has_peaked { note_number: 6 } } # Bus 3. bus_mapping { treble { controller_number: 24 } mid { controller_number: 25 } bass { controller_number: 26 } gain { controller_number: 24 } compressor_threshold { controller_number: 25 } fader { controller_number: 27 } toggle_mute { note_number: 7 } toggle_auto_gain_staging { note_number: 8 } clear_peak { note_number: 9 } # Lights. is_muted { note_number: 7 } auto_gain_staging_is_on { note_number: 8 } has_peaked { note_number: 9 } } # Bus 4. bus_mapping { treble { controller_number: 28 } mid { controller_number: 29 } bass { controller_number: 30 } gain { controller_number: 28 } compressor_threshold { controller_number: 29 } fader { controller_number: 31 } toggle_mute { note_number: 10 } toggle_auto_gain_staging { note_number: 11 } clear_peak { note_number: 12 } # Lights. is_muted { note_number: 10 } auto_gain_staging_is_on { note_number: 11 } has_peaked { note_number: 12 } } # Bus 5. Note the discontinuity in the controller numbers, # but not in the note numbers. bus_mapping { treble { controller_number: 46 } mid { controller_number: 47 } bass { controller_number: 48 } gain { controller_number: 46 } compressor_threshold { controller_number: 47 } fader { controller_number: 49 } toggle_mute { note_number: 13 } toggle_auto_gain_staging { note_number: 14 } clear_peak { note_number: 15 } # Lights. is_muted { note_number: 13 } auto_gain_staging_is_on { note_number: 14 } has_peaked { note_number: 15 } } # Bus 6. bus_mapping { treble { controller_number: 50 } mid { controller_number: 51 } bass { controller_number: 52 } gain { controller_number: 50 } compressor_threshold { controller_number: 51 } fader { controller_number: 53 } toggle_mute { note_number: 16 } toggle_auto_gain_staging { note_number: 17 } clear_peak { note_number: 18 } # Lights. is_muted { note_number: 16 } auto_gain_staging_is_on { note_number: 17 } has_peaked { note_number: 18 } } # Bus 7. bus_mapping { treble { controller_number: 54 } mid { controller_number: 55 } bass { controller_number: 56 } gain { controller_number: 54 } compressor_threshold { controller_number: 55 } fader { controller_number: 57 } toggle_mute { note_number: 19 } toggle_auto_gain_staging { note_number: 20 } clear_peak { note_number: 21 } # Lights. is_muted { note_number: 19 } auto_gain_staging_is_on { note_number: 20 } has_peaked { note_number: 21 } # Global controllers. locut { controller_number: 56 } } # Bus 8. bus_mapping { treble { controller_number: 58 } mid { controller_number: 59 } bass { controller_number: 60 } gain { controller_number: 58 } compressor_threshold { controller_number: 59 } fader { controller_number: 61 } toggle_mute { note_number: 22 } toggle_auto_gain_staging { note_number: 23 } clear_peak { note_number: 24 } # Lights. is_muted { note_number: 22 } auto_gain_staging_is_on { note_number: 23 } has_peaked { note_number: 24 } # Global controllers. limiter_threshold { controller_number: 60 } } nageru-1.4.2/alsa_input.cpp000066400000000000000000000231271301561706600156540ustar00rootroot00000000000000#include "alsa_input.h" #include #include #include #include #include #include #include "alsa_pool.h" #include "bmusb/bmusb.h" #include "timebase.h" using namespace std; using namespace std::placeholders; #define RETURN_ON_ERROR(msg, expr) do { \ int err = (expr); \ if (err < 0) { \ fprintf(stderr, "[%s] " msg ": %s\n", device.c_str(), snd_strerror(err)); \ if (err == -ENODEV) return CaptureEndReason::DEVICE_GONE; \ return CaptureEndReason::OTHER_ERROR; \ } \ } while (false) #define RETURN_FALSE_ON_ERROR(msg, expr) do { \ int err = (expr); \ if (err < 0) { \ fprintf(stderr, "[%s] " msg ": %s\n", device.c_str(), snd_strerror(err)); \ return false; \ } \ } while (false) #define WARN_ON_ERROR(msg, expr) do { \ int err = (expr); \ if (err < 0) { \ fprintf(stderr, "[%s] " msg ": %s\n", device.c_str(), snd_strerror(err)); \ } \ } while (false) ALSAInput::ALSAInput(const char *device, unsigned sample_rate, unsigned num_channels, audio_callback_t audio_callback, ALSAPool *parent_pool, unsigned internal_dev_index) : device(device), sample_rate(sample_rate), num_channels(num_channels), audio_callback(audio_callback), parent_pool(parent_pool), internal_dev_index(internal_dev_index) { } bool ALSAInput::open_device() { RETURN_FALSE_ON_ERROR("snd_pcm_open()", snd_pcm_open(&pcm_handle, device.c_str(), SND_PCM_STREAM_CAPTURE, 0)); // Set format. snd_pcm_hw_params_t *hw_params; snd_pcm_hw_params_alloca(&hw_params); if (!set_base_params(device.c_str(), pcm_handle, hw_params, &sample_rate)) { return false; } RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_set_channels()", snd_pcm_hw_params_set_channels(pcm_handle, hw_params, num_channels)); // Fragment size of 64 samples (about 1 ms at 48 kHz; a frame at 60 // fps/48 kHz is 800 samples.) We ask for 64 such periods in our buffer // (~85 ms buffer); more than that, and our jitter is probably so high // that the resampling queue can't keep up anyway. // The entire thing with periods and such is a bit mysterious to me; // seemingly I can get 96 frames at a time with no problems even if // the period size is 64 frames. And if I set num_periods to e.g. 1, // I can't have a big buffer. num_periods = 16; int dir = 0; RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_set_periods_near()", snd_pcm_hw_params_set_periods_near(pcm_handle, hw_params, &num_periods, &dir)); period_size = 64; dir = 0; RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_set_period_size_near()", snd_pcm_hw_params_set_period_size_near(pcm_handle, hw_params, &period_size, &dir)); buffer_frames = 64 * 64; RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_set_buffer_size_near()", snd_pcm_hw_params_set_buffer_size_near(pcm_handle, hw_params, &buffer_frames)); RETURN_FALSE_ON_ERROR("snd_pcm_hw_params()", snd_pcm_hw_params(pcm_handle, hw_params)); //snd_pcm_hw_params_free(hw_params); // Figure out which format the card actually chose. RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_current()", snd_pcm_hw_params_current(pcm_handle, hw_params)); snd_pcm_format_t chosen_format; RETURN_FALSE_ON_ERROR("snd_pcm_hw_params_get_format()", snd_pcm_hw_params_get_format(hw_params, &chosen_format)); audio_format.num_channels = num_channels; audio_format.bits_per_sample = 0; switch (chosen_format) { case SND_PCM_FORMAT_S16_LE: audio_format.bits_per_sample = 16; break; case SND_PCM_FORMAT_S24_LE: audio_format.bits_per_sample = 24; break; case SND_PCM_FORMAT_S32_LE: audio_format.bits_per_sample = 32; break; default: assert(false); } //printf("num_periods=%u period_size=%u buffer_frames=%u sample_rate=%u bits_per_sample=%d\n", // num_periods, unsigned(period_size), unsigned(buffer_frames), sample_rate, audio_format.bits_per_sample); buffer.reset(new uint8_t[buffer_frames * num_channels * audio_format.bits_per_sample / 8]); snd_pcm_sw_params_t *sw_params; snd_pcm_sw_params_alloca(&sw_params); RETURN_FALSE_ON_ERROR("snd_pcm_sw_params_current()", snd_pcm_sw_params_current(pcm_handle, sw_params)); RETURN_FALSE_ON_ERROR("snd_pcm_sw_params_set_start_threshold", snd_pcm_sw_params_set_start_threshold(pcm_handle, sw_params, num_periods * period_size / 2)); RETURN_FALSE_ON_ERROR("snd_pcm_sw_params()", snd_pcm_sw_params(pcm_handle, sw_params)); RETURN_FALSE_ON_ERROR("snd_pcm_nonblock()", snd_pcm_nonblock(pcm_handle, 1)); RETURN_FALSE_ON_ERROR("snd_pcm_prepare()", snd_pcm_prepare(pcm_handle)); return true; } bool ALSAInput::set_base_params(const char *device_name, snd_pcm_t *pcm_handle, snd_pcm_hw_params_t *hw_params, unsigned *sample_rate) { int err; err = snd_pcm_hw_params_any(pcm_handle, hw_params); if (err < 0) { fprintf(stderr, "[%s] snd_pcm_hw_params_any(): %s\n", device_name, snd_strerror(err)); return false; } err = snd_pcm_hw_params_set_access(pcm_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); if (err < 0) { fprintf(stderr, "[%s] snd_pcm_hw_params_set_access(): %s\n", device_name, snd_strerror(err)); return false; } snd_pcm_format_mask_t *format_mask; snd_pcm_format_mask_alloca(&format_mask); snd_pcm_format_mask_set(format_mask, SND_PCM_FORMAT_S16_LE); snd_pcm_format_mask_set(format_mask, SND_PCM_FORMAT_S24_LE); snd_pcm_format_mask_set(format_mask, SND_PCM_FORMAT_S32_LE); err = snd_pcm_hw_params_set_format_mask(pcm_handle, hw_params, format_mask); if (err < 0) { fprintf(stderr, "[%s] snd_pcm_hw_params_set_format_mask(): %s\n", device_name, snd_strerror(err)); return false; } err = snd_pcm_hw_params_set_rate_near(pcm_handle, hw_params, sample_rate, 0); if (err < 0) { fprintf(stderr, "[%s] snd_pcm_hw_params_set_rate_near(): %s\n", device_name, snd_strerror(err)); return false; } return true; } ALSAInput::~ALSAInput() { if (pcm_handle) { WARN_ON_ERROR("snd_pcm_close()", snd_pcm_close(pcm_handle)); } } void ALSAInput::start_capture_thread() { should_quit = false; capture_thread = thread(&ALSAInput::capture_thread_func, this); } void ALSAInput::stop_capture_thread() { should_quit = true; capture_thread.join(); } void ALSAInput::capture_thread_func() { parent_pool->set_card_state(internal_dev_index, ALSAPool::Device::State::STARTING); // If the device hasn't been opened already, we need to do so // before we can capture. while (!should_quit && pcm_handle == nullptr) { if (!open_device()) { fprintf(stderr, "[%s] Waiting one second and trying again...\n", device.c_str()); sleep(1); } } if (should_quit) { // Don't call free_card(); that would be a deadlock. WARN_ON_ERROR("snd_pcm_close()", snd_pcm_close(pcm_handle)); pcm_handle = nullptr; return; } // Do the actual capture. (Termination condition within loop.) for ( ;; ) { switch (do_capture()) { case CaptureEndReason::REQUESTED_QUIT: // Don't call free_card(); that would be a deadlock. WARN_ON_ERROR("snd_pcm_close()", snd_pcm_close(pcm_handle)); pcm_handle = nullptr; return; case CaptureEndReason::DEVICE_GONE: parent_pool->free_card(internal_dev_index); WARN_ON_ERROR("snd_pcm_close()", snd_pcm_close(pcm_handle)); pcm_handle = nullptr; return; case CaptureEndReason::OTHER_ERROR: parent_pool->set_card_state(internal_dev_index, ALSAPool::Device::State::STARTING); fprintf(stderr, "[%s] Sleeping one second and restarting capture...\n", device.c_str()); sleep(1); break; } } } ALSAInput::CaptureEndReason ALSAInput::do_capture() { parent_pool->set_card_state(internal_dev_index, ALSAPool::Device::State::STARTING); RETURN_ON_ERROR("snd_pcm_start()", snd_pcm_start(pcm_handle)); parent_pool->set_card_state(internal_dev_index, ALSAPool::Device::State::RUNNING); uint64_t num_frames_output = 0; while (!should_quit) { int ret = snd_pcm_wait(pcm_handle, /*timeout=*/100); if (ret == 0) continue; // Timeout. if (ret == -EPIPE) { fprintf(stderr, "[%s] ALSA overrun\n", device.c_str()); snd_pcm_prepare(pcm_handle); snd_pcm_start(pcm_handle); continue; } RETURN_ON_ERROR("snd_pcm_wait()", ret); snd_pcm_sframes_t frames = snd_pcm_readi(pcm_handle, buffer.get(), buffer_frames); if (frames == -EPIPE) { fprintf(stderr, "[%s] ALSA overrun\n", device.c_str()); snd_pcm_prepare(pcm_handle); snd_pcm_start(pcm_handle); continue; } if (frames == 0) { fprintf(stderr, "snd_pcm_readi() returned 0\n"); break; } RETURN_ON_ERROR("snd_pcm_readi()", frames); const int64_t prev_pts = frames_to_pts(num_frames_output); const int64_t pts = frames_to_pts(num_frames_output + frames); bool success; do { if (should_quit) return CaptureEndReason::REQUESTED_QUIT; success = audio_callback(buffer.get(), frames, audio_format, pts - prev_pts); } while (!success); num_frames_output += frames; } return CaptureEndReason::REQUESTED_QUIT; } int64_t ALSAInput::frames_to_pts(uint64_t n) const { return (n * TIMEBASE) / sample_rate; } nageru-1.4.2/alsa_input.h000066400000000000000000000046701301561706600153230ustar00rootroot00000000000000#ifndef _ALSA_INPUT_H #define _ALSA_INPUT_H 1 // ALSA sound input, running in a separate thread and sending audio back // in callbacks. // // Note: “frame” here generally refers to the ALSA definition of frame, // which is a set of samples, exactly one for each channel. The only exception // is in frame_length, where it means the TIMEBASE length of the buffer // as a whole, since that's what AudioMixer::add_audio() wants. #include #include #include #include #include #include #include #include #include "bmusb/bmusb.h" class ALSAPool; class ALSAInput { public: typedef std::function audio_callback_t; ALSAInput(const char *device, unsigned sample_rate, unsigned num_channels, audio_callback_t audio_callback, ALSAPool *parent_pool, unsigned internal_dev_index); ~ALSAInput(); // If not called before start_capture_thread(), the capture thread // will call it until it succeeds. bool open_device(); // Not valid before the device has been successfully opened. // NOTE: Might very well be different from the sample rate given to the // constructor, since the card might not support the one you wanted. unsigned get_sample_rate() const { return sample_rate; } void start_capture_thread(); void stop_capture_thread(); // Set access, sample rate and format parameters on the given ALSA PCM handle. // Returns the computed parameter set and the chosen sample rate. Note that // sample_rate is an in/out parameter; you send in the desired rate, // and ALSA picks one as close to that as possible. static bool set_base_params(const char *device_name, snd_pcm_t *pcm_handle, snd_pcm_hw_params_t *hw_params, unsigned *sample_rate); private: void capture_thread_func(); int64_t frames_to_pts(uint64_t n) const; enum class CaptureEndReason { REQUESTED_QUIT, DEVICE_GONE, OTHER_ERROR }; CaptureEndReason do_capture(); std::string device; unsigned sample_rate, num_channels, num_periods; snd_pcm_uframes_t period_size; snd_pcm_uframes_t buffer_frames; bmusb::AudioFormat audio_format; audio_callback_t audio_callback; snd_pcm_t *pcm_handle = nullptr; std::thread capture_thread; std::atomic should_quit{false}; std::unique_ptr buffer; ALSAPool *parent_pool; unsigned internal_dev_index; }; #endif // !defined(_ALSA_INPUT_H) nageru-1.4.2/alsa_output.cpp000066400000000000000000000067641301561706600160650ustar00rootroot00000000000000#include "alsa_output.h" #include #include #include #include #include using namespace std; namespace { void die_on_error(const char *func_name, int err) { if (err < 0) { fprintf(stderr, "%s: %s\n", func_name, snd_strerror(err)); exit(1); } } } // namespace ALSAOutput::ALSAOutput(int sample_rate, int num_channels) : sample_rate(sample_rate), num_channels(num_channels) { die_on_error("snd_pcm_open()", snd_pcm_open(&pcm_handle, "default", SND_PCM_STREAM_PLAYBACK, 0)); // Set format. snd_pcm_hw_params_t *hw_params; snd_pcm_hw_params_alloca(&hw_params); die_on_error("snd_pcm_hw_params_any()", snd_pcm_hw_params_any(pcm_handle, hw_params)); die_on_error("snd_pcm_hw_params_set_access()", snd_pcm_hw_params_set_access(pcm_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED)); die_on_error("snd_pcm_hw_params_set_format()", snd_pcm_hw_params_set_format(pcm_handle, hw_params, SND_PCM_FORMAT_FLOAT_LE)); die_on_error("snd_pcm_hw_params_set_rate()", snd_pcm_hw_params_set_rate(pcm_handle, hw_params, sample_rate, 0)); die_on_error("snd_pcm_hw_params_set_channels", snd_pcm_hw_params_set_channels(pcm_handle, hw_params, num_channels)); // Fragment size of 512 samples. (A frame at 60 fps/48 kHz is 800 samples.) // We ask for 16 such periods (~170 ms buffer). unsigned int num_periods = 16; int dir = 0; die_on_error("snd_pcm_hw_params_set_periods_near()", snd_pcm_hw_params_set_periods_near(pcm_handle, hw_params, &num_periods, &dir)); period_size = 512; dir = 0; die_on_error("snd_pcm_hw_params_set_period_size_near()", snd_pcm_hw_params_set_period_size_near(pcm_handle, hw_params, &period_size, &dir)); die_on_error("snd_pcm_hw_params()", snd_pcm_hw_params(pcm_handle, hw_params)); //snd_pcm_hw_params_free(hw_params); snd_pcm_sw_params_t *sw_params; snd_pcm_sw_params_alloca(&sw_params); die_on_error("snd_pcm_sw_params_current()", snd_pcm_sw_params_current(pcm_handle, sw_params)); die_on_error("snd_pcm_sw_params_set_start_threshold", snd_pcm_sw_params_set_start_threshold(pcm_handle, sw_params, num_periods * period_size / 2)); die_on_error("snd_pcm_sw_params()", snd_pcm_sw_params(pcm_handle, sw_params)); die_on_error("snd_pcm_nonblock", snd_pcm_nonblock(pcm_handle, 1)); die_on_error("snd_pcm_prepare()", snd_pcm_prepare(pcm_handle)); } void ALSAOutput::write(const vector &samples) { buffer.insert(buffer.end(), samples.begin(), samples.end()); try_again: int periods_to_write = buffer.size() / (period_size * num_channels); if (periods_to_write == 0) { return; } int ret = snd_pcm_writei(pcm_handle, buffer.data(), periods_to_write * period_size); if (ret == -EPIPE) { fprintf(stderr, "warning: snd_pcm_writei() reported underrun\n"); snd_pcm_recover(pcm_handle, ret, 1); goto try_again; } else if (ret == -EAGAIN) { ret = 0; } else if (ret < 0) { fprintf(stderr, "error: snd_pcm_writei() returned '%s'\n", snd_strerror(ret)); exit(1); } else if (ret > 0) { buffer.erase(buffer.begin(), buffer.begin() + ret * num_channels); } if (buffer.size() >= period_size * num_channels) { // Still more to write. if (ret == 0) { if (buffer.size() >= period_size * num_channels * 8) { // OK, almost 100 ms. Giving up. fprintf(stderr, "warning: ALSA overrun, dropping some audio (%d ms)\n", int(buffer.size() * 1000 / (num_channels * sample_rate))); buffer.clear(); } } else if (ret > 0) { // Not a completely failure (effectively a short write), // possibly due to a signal. goto try_again; } } } nageru-1.4.2/alsa_output.h000066400000000000000000000012711301561706600155160ustar00rootroot00000000000000#ifndef _ALSA_OUTPUT_H #define _ALSA_OUTPUT_H 1 // Extremely minimalistic ALSA output. Will not resample to fit // sound card clock, will not care much about over- or underflows // (so it will not block), will not care about A/V sync. // // This means that if you run it for long enough, clocks will // probably drift out of sync enough to make a little pop. #include #include class ALSAOutput { public: ALSAOutput(int sample_rate, int num_channels); void write(const std::vector &samples); private: snd_pcm_t *pcm_handle; std::vector buffer; snd_pcm_uframes_t period_size; int sample_rate, num_channels; }; #endif // !defined(_ALSA_OUTPUT_H) nageru-1.4.2/alsa_pool.cpp000066400000000000000000000357501301561706600154730ustar00rootroot00000000000000#include "alsa_pool.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "alsa_input.h" #include "audio_mixer.h" #include "defs.h" #include "input_mapping.h" #include "state.pb.h" using namespace std; using namespace std::placeholders; ALSAPool::ALSAPool() { should_quit_fd = eventfd(/*initval=*/0, /*flags=*/0); assert(should_quit_fd != -1); } ALSAPool::~ALSAPool() { for (Device &device : devices) { if (device.input != nullptr) { device.input->stop_capture_thread(); } } should_quit = true; const uint64_t one = 1; if (write(should_quit_fd, &one, sizeof(one)) != sizeof(one)) { perror("write(should_quit_fd)"); exit(1); } inotify_thread.join(); while (retry_threads_running > 0) { this_thread::sleep_for(std::chrono::milliseconds(100)); } } std::vector ALSAPool::get_devices() { lock_guard lock(mu); for (Device &device : devices) { device.held = true; } return devices; } void ALSAPool::hold_device(unsigned index) { lock_guard lock(mu); assert(index < devices.size()); devices[index].held = true; } void ALSAPool::release_device(unsigned index) { lock_guard lock(mu); if (index < devices.size()) { devices[index].held = false; } } void ALSAPool::enumerate_devices() { // Enumerate all cards. for (int card_index = -1; snd_card_next(&card_index) == 0 && card_index >= 0; ) { char address[256]; snprintf(address, sizeof(address), "hw:%d", card_index); snd_ctl_t *ctl; int err = snd_ctl_open(&ctl, address, 0); if (err < 0) { printf("%s: %s\n", address, snd_strerror(err)); continue; } unique_ptr ctl_closer(ctl, snd_ctl_close); // Enumerate all devices on this card. for (int dev_index = -1; snd_ctl_pcm_next_device(ctl, &dev_index) == 0 && dev_index >= 0; ) { probe_device_with_retry(card_index, dev_index); } } } void ALSAPool::probe_device_with_retry(unsigned card_index, unsigned dev_index) { char address[256]; snprintf(address, sizeof(address), "hw:%d,%d", card_index, dev_index); lock_guard lock(add_device_mutex); if (add_device_tries_left.count(address)) { // Some thread is already busy retrying this, // so just reset its count. add_device_tries_left[address] = num_retries; return; } // Try (while still holding the lock) to add the device synchronously. ProbeResult result = probe_device_once(card_index, dev_index); if (result == ProbeResult::SUCCESS) { return; } else if (result == ProbeResult::FAILURE) { return; } assert(result == ProbeResult::DEFER); // Add failed for whatever reason (probably just that the device // isn't up yet. Set up a count so that nobody else starts a thread, // then start it ourselves. fprintf(stderr, "Trying %s again in one second...\n", address); add_device_tries_left[address] = num_retries; ++retry_threads_running; thread(&ALSAPool::probe_device_retry_thread_func, this, card_index, dev_index).detach(); } void ALSAPool::probe_device_retry_thread_func(unsigned card_index, unsigned dev_index) { char address[256]; snprintf(address, sizeof(address), "hw:%d,%d", card_index, dev_index); for ( ;; ) { // Termination condition within the loop. sleep(1); // See if there are any retries left. lock_guard lock(add_device_mutex); if (should_quit || !add_device_tries_left.count(address) || add_device_tries_left[address] == 0) { add_device_tries_left.erase(address); fprintf(stderr, "Giving up probe of %s.\n", address); break; } // Seemingly there were. Give it a try (we still hold the mutex). ProbeResult result = probe_device_once(card_index, dev_index); if (result == ProbeResult::SUCCESS) { add_device_tries_left.erase(address); fprintf(stderr, "Probe of %s succeeded.\n", address); break; } else if (result == ProbeResult::FAILURE || --add_device_tries_left[address] == 0) { add_device_tries_left.erase(address); fprintf(stderr, "Giving up probe of %s.\n", address); break; } // Failed again. assert(result == ProbeResult::DEFER); fprintf(stderr, "Trying %s again in one second (%d tries left)...\n", address, add_device_tries_left[address]); } --retry_threads_running; } ALSAPool::ProbeResult ALSAPool::probe_device_once(unsigned card_index, unsigned dev_index) { char address[256]; snprintf(address, sizeof(address), "hw:%d", card_index); snd_ctl_t *ctl; int err = snd_ctl_open(&ctl, address, 0); if (err < 0) { printf("%s: %s\n", address, snd_strerror(err)); return ALSAPool::ProbeResult::DEFER; } unique_ptr ctl_closer(ctl, snd_ctl_close); snd_pcm_info_t *pcm_info; snd_pcm_info_alloca(&pcm_info); snd_pcm_info_set_device(pcm_info, dev_index); snd_pcm_info_set_subdevice(pcm_info, 0); snd_pcm_info_set_stream(pcm_info, SND_PCM_STREAM_CAPTURE); if (snd_ctl_pcm_info(ctl, pcm_info) < 0) { // Not available for capture. printf("%s: Not available for capture.\n", address); return ALSAPool::ProbeResult::DEFER; } snprintf(address, sizeof(address), "hw:%d,%d", card_index, dev_index); unsigned num_channels = 0; // Find all channel maps for this device, and pick out the one // with the most channels. snd_pcm_chmap_query_t **cmaps = snd_pcm_query_chmaps_from_hw(card_index, dev_index, 0, SND_PCM_STREAM_CAPTURE); if (cmaps != nullptr) { for (snd_pcm_chmap_query_t **ptr = cmaps; *ptr; ++ptr) { num_channels = max(num_channels, (*ptr)->map.channels); } snd_pcm_free_chmaps(cmaps); } if (num_channels == 0) { // Device had no channel maps. We need to open it to query. // TODO: Do this asynchronously. snd_pcm_t *pcm_handle; int err = snd_pcm_open(&pcm_handle, address, SND_PCM_STREAM_CAPTURE, 0); if (err < 0) { printf("%s: %s\n", address, snd_strerror(err)); return ALSAPool::ProbeResult::DEFER; } snd_pcm_hw_params_t *hw_params; snd_pcm_hw_params_alloca(&hw_params); unsigned sample_rate; if (!ALSAInput::set_base_params(address, pcm_handle, hw_params, &sample_rate)) { snd_pcm_close(pcm_handle); return ALSAPool::ProbeResult::DEFER; } err = snd_pcm_hw_params_get_channels_max(hw_params, &num_channels); if (err < 0) { fprintf(stderr, "[%s] snd_pcm_hw_params_get_channels_max(): %s\n", address, snd_strerror(err)); snd_pcm_close(pcm_handle); return ALSAPool::ProbeResult::DEFER; } snd_pcm_close(pcm_handle); } if (num_channels == 0) { printf("%s: No channel maps with channels\n", address); return ALSAPool::ProbeResult::FAILURE; } snd_ctl_card_info_t *card_info; snd_ctl_card_info_alloca(&card_info); snd_ctl_card_info(ctl, card_info); string name = snd_ctl_card_info_get_name(card_info); string info = snd_pcm_info_get_name(pcm_info); unsigned internal_dev_index; string display_name; { lock_guard lock(mu); internal_dev_index = find_free_device_index(name, info, num_channels, address); devices[internal_dev_index].address = address; devices[internal_dev_index].name = name; devices[internal_dev_index].info = info; devices[internal_dev_index].num_channels = num_channels; // Note: Purposefully does not overwrite held. display_name = devices[internal_dev_index].display_name(); } fprintf(stderr, "%s: Probed successfully.\n", address); reset_device(internal_dev_index); // Restarts it if it is held (ie., we just replaced a dead card). DeviceSpec spec{InputSourceType::ALSA_INPUT, internal_dev_index}; global_audio_mixer->set_display_name(spec, display_name); global_audio_mixer->trigger_state_changed_callback(); return ALSAPool::ProbeResult::SUCCESS; } void ALSAPool::unplug_device(unsigned card_index, unsigned dev_index) { char address[256]; snprintf(address, sizeof(address), "hw:%d,%d", card_index, dev_index); for (unsigned i = 0; i < devices.size(); ++i) { if (devices[i].state != Device::State::EMPTY && devices[i].state != Device::State::DEAD && devices[i].address == address) { free_card(i); } } } void ALSAPool::init() { inotify_thread = thread(&ALSAPool::inotify_thread_func, this); enumerate_devices(); } void ALSAPool::inotify_thread_func() { int inotify_fd = inotify_init(); if (inotify_fd == -1) { perror("inotify_init()"); fprintf(stderr, "No hotplug of ALSA devices available.\n"); return; } int watch_fd = inotify_add_watch(inotify_fd, "/dev/snd", IN_MOVE | IN_CREATE | IN_DELETE); if (watch_fd == -1) { perror("inotify_add_watch()"); fprintf(stderr, "No hotplug of ALSA devices available.\n"); close(inotify_fd); return; } int size = sizeof(inotify_event) + NAME_MAX + 1; unique_ptr buf(new char[size]); while (!should_quit) { pollfd fds[2]; fds[0].fd = inotify_fd; fds[0].events = POLLIN; fds[0].revents = 0; fds[1].fd = should_quit_fd; fds[1].events = POLLIN; fds[1].revents = 0; int ret = poll(fds, 2, -1); if (ret == -1) { if (errno == EINTR) { continue; } else { perror("poll(inotify_fd)"); return; } } if (ret == 0) { continue; } if (fds[1].revents) break; // should_quit_fd asserted. ret = read(inotify_fd, buf.get(), size); if (ret == -1) { if (errno == EINTR) { continue; } else { perror("read(inotify_fd)"); close(watch_fd); close(inotify_fd); return; } } if (ret < int(sizeof(inotify_event))) { fprintf(stderr, "inotify read unexpectedly returned %d, giving up hotplug of ALSA devices.\n", int(ret)); close(watch_fd); close(inotify_fd); return; } for (int i = 0; i < ret; ) { const inotify_event *event = reinterpret_cast(&buf[i]); i += sizeof(inotify_event) + event->len; if (event->mask & IN_Q_OVERFLOW) { fprintf(stderr, "WARNING: inotify overflowed, may lose ALSA hotplug events.\n"); continue; } unsigned card, device; char type; if (sscanf(event->name, "pcmC%uD%u%c", &card, &device, &type) == 3 && type == 'c') { if (event->mask & (IN_MOVED_FROM | IN_DELETE)) { printf("Deleted capture device: Card %u, device %u\n", card, device); unplug_device(card, device); } if (event->mask & (IN_MOVED_TO | IN_CREATE)) { printf("Adding capture device: Card %u, device %u\n", card, device); probe_device_with_retry(card, device); } } } } close(watch_fd); close(inotify_fd); close(should_quit_fd); } void ALSAPool::reset_device(unsigned index) { lock_guard lock(mu); Device *device = &devices[index]; if (inputs[index] != nullptr) { inputs[index]->stop_capture_thread(); } if (!device->held) { inputs[index].reset(); } else { // TODO: Put on a background thread instead of locking? auto callback = bind(&AudioMixer::add_audio, global_audio_mixer, DeviceSpec{InputSourceType::ALSA_INPUT, index}, _1, _2, _3, _4); inputs[index].reset(new ALSAInput(device->address.c_str(), OUTPUT_FREQUENCY, device->num_channels, callback, this, index)); inputs[index]->start_capture_thread(); } device->input = inputs[index].get(); } unsigned ALSAPool::get_capture_frequency(unsigned index) { lock_guard lock(mu); assert(devices[index].held); if (devices[index].input) return devices[index].input->get_sample_rate(); else return OUTPUT_FREQUENCY; } ALSAPool::Device::State ALSAPool::get_card_state(unsigned index) { lock_guard lock(mu); assert(devices[index].held); return devices[index].state; } void ALSAPool::set_card_state(unsigned index, ALSAPool::Device::State state) { { lock_guard lock(mu); devices[index].state = state; } DeviceSpec spec{InputSourceType::ALSA_INPUT, index}; bool silence = (state != ALSAPool::Device::State::RUNNING); while (!global_audio_mixer->silence_card(spec, silence)) ; global_audio_mixer->trigger_state_changed_callback(); } unsigned ALSAPool::find_free_device_index(const string &name, const string &info, unsigned num_channels, const string &address) { // First try to find an exact match on a dead card. for (unsigned i = 0; i < devices.size(); ++i) { if (devices[i].state == Device::State::DEAD && devices[i].address == address && devices[i].name == name && devices[i].info == info && devices[i].num_channels == num_channels) { devices[i].state = Device::State::READY; return i; } } // Then try to find a match on everything but the address // (probably that devices were plugged back in a different order). // If we have two cards that are equal, this might get them mixed up, // but we don't have anything better. for (unsigned i = 0; i < devices.size(); ++i) { if (devices[i].state == Device::State::DEAD && devices[i].name == name && devices[i].info == info && devices[i].num_channels == num_channels) { devices[i].state = Device::State::READY; return i; } } // OK, so we didn't find a match; see if there are any empty slots. for (unsigned i = 0; i < devices.size(); ++i) { if (devices[i].state == Device::State::EMPTY) { devices[i].state = Device::State::READY; devices[i].held = false; return i; } } // Failing that, we just insert the new device at the end. Device new_dev; new_dev.state = Device::State::READY; new_dev.held = false; devices.push_back(new_dev); inputs.emplace_back(nullptr); return devices.size() - 1; } unsigned ALSAPool::create_dead_card(const string &name, const string &info, unsigned num_channels) { lock_guard lock(mu); // See if there are any empty slots. If not, insert one at the end. vector::iterator free_device = find_if(devices.begin(), devices.end(), [](const Device &device) { return device.state == Device::State::EMPTY; }); if (free_device == devices.end()) { devices.push_back(Device()); inputs.emplace_back(nullptr); free_device = devices.end() - 1; } free_device->state = Device::State::DEAD; free_device->name = name; free_device->info = info; free_device->num_channels = num_channels; free_device->held = true; return distance(devices.begin(), free_device); } void ALSAPool::serialize_device(unsigned index, DeviceSpecProto *serialized) { lock_guard lock(mu); assert(index < devices.size()); assert(devices[index].held); serialized->set_type(DeviceSpecProto::ALSA_INPUT); serialized->set_index(index); serialized->set_display_name(devices[index].display_name()); serialized->set_alsa_name(devices[index].name); serialized->set_alsa_info(devices[index].info); serialized->set_num_channels(devices[index].num_channels); serialized->set_address(devices[index].address); } void ALSAPool::free_card(unsigned index) { DeviceSpec spec{InputSourceType::ALSA_INPUT, index}; while (!global_audio_mixer->silence_card(spec, true)) ; { lock_guard lock(mu); if (devices[index].held) { devices[index].state = Device::State::DEAD; } else { devices[index].state = Device::State::EMPTY; inputs[index].reset(); } while (!devices.empty() && devices.back().state == Device::State::EMPTY) { devices.pop_back(); inputs.pop_back(); } } global_audio_mixer->trigger_state_changed_callback(); } nageru-1.4.2/alsa_pool.h000066400000000000000000000126761301561706600151420ustar00rootroot00000000000000#ifndef _ALSA_POOL_H #define _ALSA_POOL_H 1 #include #include #include #include #include #include #include class ALSAInput; class DeviceSpecProto; // The class dealing with the collective of all ALSA cards in the system. // In particular, it deals with enumeration of cards, and hotplug of new ones. class ALSAPool { public: ALSAPool(); ~ALSAPool(); struct Device { enum class State { // There is no card here. (There probably used to be one, // but it got removed.) We don't insert a card before // we've actually probed it, ie., we know whether it // can be captured from at all, and what its name is. EMPTY, // This card is ready for capture, as far as we know. // (It could still be used by someone else; we don't know // until we try to open it.) READY, // We are trying to start capture from this card, but we are not // streaming yet. Note that this could in theory go on forever, // if the card is in use by some other process; in the UI, // we will show this state as “(busy)”. STARTING, // The card is capturing and sending data. If there's a fatal error, // it could go back to STARTING, or it could go to DEAD // (depending on the error). RUNNING, // The card is gone (e.g., unplugged). However, since there's // still a bus using it, we can't just remove the entry. // If the card comes back (ie., a new card is plugged in, // and we believe it has the same configuration), it could be // installed in place of this card, and then presumably be put // back into STARTING or RUNNING. DEAD } state = State::EMPTY; std::string address; // E.g. “hw:0,0”. std::string name, info; unsigned num_channels; ALSAInput *input = nullptr; // nullptr iff EMPTY or DEAD. // Whether the AudioMixer is interested in this card or not. // “Interested” could mean either of two things: Either it is part of // a bus mapping, or it is in the process of enumerating devices // (to show to the user). A card that is _not_ held can disappear // at any given time as a result of an error or hotplug event; // a card that is held will go to the DEAD state instead. bool held = false; std::string display_name() const { return name + " (" + info + ")"; } }; void init(); // Get the list of all current devices. Note that this will implicitly mark // all of the returned devices as held, since the input mapping UI needs // some kind of stability when the user is to choose. Thus, when you are done // with the list and have set a new mapping, you must go through all the devices // you don't want and release them using release_device(). std::vector get_devices(); void hold_device(unsigned index); void release_device(unsigned index); // Note: index is allowed to go out of bounds. // If device is held, start or restart capture. If device is not held, // stop capture if it isn't already. void reset_device(unsigned index); // Note: The card must be held. Returns OUTPUT_FREQUENCY if the card is in EMPTY or DEAD. unsigned get_capture_frequency(unsigned index); // Note: The card must be held. Device::State get_card_state(unsigned index); // Only for ALSAInput. void set_card_state(unsigned index, Device::State state); // Just a short form for taking and then moving the card to // EMPTY or DEAD state. Only for ALSAInput and for internal use. void free_card(unsigned index); // Create a new card, mark it immediately as DEAD and hold it. // Returns the new index. unsigned create_dead_card(const std::string &name, const std::string &info, unsigned num_channels); // Make a protobuf representation of the given card, so that it can be // matched against at a later stage. For AudioMixer only. // The given card must be held. void serialize_device(unsigned index, DeviceSpecProto *serialized); private: mutable std::mutex mu; std::vector devices; // Under mu. std::vector> inputs; // Under mu, corresponds 1:1 to devices. // Keyed on device address (e.g. “hw:0,0”). If there's an entry here, // it means we already have a thread doing retries, so we shouldn't // start a new one. std::unordered_map add_device_tries_left; // Under add_device_mutex. std::mutex add_device_mutex; static constexpr int num_retries = 10; void inotify_thread_func(); void enumerate_devices(); // Try to add an input at the given card/device. If it succeeds, return // synchronously. If not, fire off a background thread to try up to // times. void probe_device_with_retry(unsigned card_index, unsigned dev_index); void probe_device_retry_thread_func(unsigned card_index, unsigned dev_index); enum class ProbeResult { SUCCESS, DEFER, FAILURE }; ProbeResult probe_device_once(unsigned card_index, unsigned dev_index); void unplug_device(unsigned card_index, unsigned dev_index); // Must be called with held. Will allocate a new entry if needed. // The returned entry will be set to READY state. unsigned find_free_device_index(const std::string &name, const std::string &info, unsigned num_channels, const std::string &address); std::atomic should_quit{false}; int should_quit_fd; std::thread inotify_thread; std::atomic retry_threads_running{0}; friend class ALSAInput; }; #endif // !defined(_ALSA_POOL_H) nageru-1.4.2/audio_encoder.cpp000066400000000000000000000130251301561706600163110ustar00rootroot00000000000000#include "audio_encoder.h" extern "C" { #include #include #include #include #include #include #include #include #include #include } #include #include #include #include #include #include #include #include "defs.h" #include "mux.h" #include "timebase.h" using namespace std; AudioEncoder::AudioEncoder(const string &codec_name, int bit_rate, const AVOutputFormat *oformat) { AVCodec *codec = avcodec_find_encoder_by_name(codec_name.c_str()); if (codec == nullptr) { fprintf(stderr, "ERROR: Could not find codec '%s'\n", codec_name.c_str()); exit(1); } ctx = avcodec_alloc_context3(codec); ctx->bit_rate = bit_rate; ctx->sample_rate = OUTPUT_FREQUENCY; ctx->sample_fmt = codec->sample_fmts[0]; ctx->channels = 2; ctx->channel_layout = AV_CH_LAYOUT_STEREO; ctx->time_base = AVRational{1, TIMEBASE}; if (oformat->flags & AVFMT_GLOBALHEADER) { ctx->flags |= CODEC_FLAG_GLOBAL_HEADER; } if (avcodec_open2(ctx, codec, NULL) < 0) { fprintf(stderr, "Could not open codec '%s'\n", codec_name.c_str()); exit(1); } resampler = avresample_alloc_context(); if (resampler == nullptr) { fprintf(stderr, "Allocating resampler failed.\n"); exit(1); } av_opt_set_int(resampler, "in_channel_layout", AV_CH_LAYOUT_STEREO, 0); av_opt_set_int(resampler, "out_channel_layout", AV_CH_LAYOUT_STEREO, 0); av_opt_set_int(resampler, "in_sample_rate", OUTPUT_FREQUENCY, 0); av_opt_set_int(resampler, "out_sample_rate", OUTPUT_FREQUENCY, 0); av_opt_set_int(resampler, "in_sample_fmt", AV_SAMPLE_FMT_FLT, 0); av_opt_set_int(resampler, "out_sample_fmt", ctx->sample_fmt, 0); if (avresample_open(resampler) < 0) { fprintf(stderr, "Could not open resample context.\n"); exit(1); } audio_frame = av_frame_alloc(); } AudioEncoder::~AudioEncoder() { av_frame_free(&audio_frame); avresample_free(&resampler); avcodec_free_context(&ctx); } void AudioEncoder::encode_audio(const vector &audio, int64_t audio_pts) { if (ctx->frame_size == 0) { // No queueing needed. assert(audio_queue.empty()); assert(audio.size() % 2 == 0); encode_audio_one_frame(&audio[0], audio.size() / 2, audio_pts); return; } int64_t sample_offset = audio_queue.size(); audio_queue.insert(audio_queue.end(), audio.begin(), audio.end()); size_t sample_num; for (sample_num = 0; sample_num + ctx->frame_size * 2 <= audio_queue.size(); sample_num += ctx->frame_size * 2) { int64_t adjusted_audio_pts = audio_pts + (int64_t(sample_num) - sample_offset) * TIMEBASE / (OUTPUT_FREQUENCY * 2); encode_audio_one_frame(&audio_queue[sample_num], ctx->frame_size, adjusted_audio_pts); } audio_queue.erase(audio_queue.begin(), audio_queue.begin() + sample_num); last_pts = audio_pts + audio.size() * TIMEBASE / (OUTPUT_FREQUENCY * 2); } void AudioEncoder::encode_audio_one_frame(const float *audio, size_t num_samples, int64_t audio_pts) { audio_frame->pts = audio_pts; audio_frame->nb_samples = num_samples; audio_frame->channel_layout = AV_CH_LAYOUT_STEREO; audio_frame->format = ctx->sample_fmt; audio_frame->sample_rate = OUTPUT_FREQUENCY; if (av_samples_alloc(audio_frame->data, nullptr, 2, num_samples, ctx->sample_fmt, 0) < 0) { fprintf(stderr, "Could not allocate %ld samples.\n", num_samples); exit(1); } if (avresample_convert(resampler, audio_frame->data, 0, num_samples, (uint8_t **)&audio, 0, num_samples) < 0) { fprintf(stderr, "Audio conversion failed.\n"); exit(1); } int err = avcodec_send_frame(ctx, audio_frame); if (err < 0) { fprintf(stderr, "avcodec_send_frame() failed with error %d\n", err); exit(1); } for ( ;; ) { // Termination condition within loop. AVPacket pkt; av_init_packet(&pkt); pkt.data = nullptr; pkt.size = 0; int err = avcodec_receive_packet(ctx, &pkt); if (err == 0) { pkt.stream_index = 1; pkt.flags = 0; for (Mux *mux : muxes) { mux->add_packet(pkt, pkt.pts, pkt.dts); } av_packet_unref(&pkt); } else if (err == AVERROR(EAGAIN)) { break; } else { fprintf(stderr, "avcodec_receive_frame() failed with error %d\n", err); exit(1); } } av_freep(&audio_frame->data[0]); av_frame_unref(audio_frame); } void AudioEncoder::encode_last_audio() { if (!audio_queue.empty()) { // Last frame can be whatever size we want. assert(audio_queue.size() % 2 == 0); encode_audio_one_frame(&audio_queue[0], audio_queue.size() / 2, last_pts); audio_queue.clear(); } if (ctx->codec->capabilities & AV_CODEC_CAP_DELAY) { // Collect any delayed frames. for ( ;; ) { AVPacket pkt; av_init_packet(&pkt); pkt.data = nullptr; pkt.size = 0; int err = avcodec_receive_packet(ctx, &pkt); if (err == 0) { pkt.stream_index = 1; pkt.flags = 0; for (Mux *mux : muxes) { mux->add_packet(pkt, pkt.pts, pkt.dts); } av_packet_unref(&pkt); } else if (err == AVERROR_EOF) { break; } else { fprintf(stderr, "avcodec_receive_frame() failed with error %d\n", err); exit(1); } } } } AVCodecParametersWithDeleter AudioEncoder::get_codec_parameters() { AVCodecParameters *codecpar = avcodec_parameters_alloc(); avcodec_parameters_from_context(codecpar, ctx); return AVCodecParametersWithDeleter(codecpar, avcodec_parameters_free_unique); } nageru-1.4.2/audio_encoder.h000066400000000000000000000021431301561706600157550ustar00rootroot00000000000000// A class to encode audio (using ffmpeg) and send it to a Mux. #ifndef _AUDIO_ENCODER_H #define _AUDIO_ENCODER_H 1 #include #include #include #include extern "C" { #include #include #include #include } #include "ffmpeg_raii.h" class Mux; class AudioEncoder { public: AudioEncoder(const std::string &codec_name, int bit_rate, const AVOutputFormat *oformat); ~AudioEncoder(); void add_mux(Mux *mux) { // Does not take ownership. muxes.push_back(mux); } void encode_audio(const std::vector &audio, int64_t audio_pts); void encode_last_audio(); AVCodecParametersWithDeleter get_codec_parameters(); private: void encode_audio_one_frame(const float *audio, size_t num_samples, int64_t audio_pts); std::vector audio_queue; int64_t last_pts = 0; // The first pts after all audio we've encoded. AVCodecContext *ctx; AVAudioResampleContext *resampler; AVFrame *audio_frame = nullptr; std::vector muxes; }; #endif // !defined(_AUDIO_ENCODER_H) nageru-1.4.2/audio_mixer.cpp000066400000000000000000001065011301561706600160200ustar00rootroot00000000000000#include "audio_mixer.h" #include #include #include #include #ifdef __SSE2__ #include #endif #include #include #include #include #include #include #include #include #include #include #include "db.h" #include "flags.h" #include "state.pb.h" #include "timebase.h" using namespace bmusb; using namespace std; using namespace std::placeholders; namespace { // TODO: If these prove to be a bottleneck, they can be SSSE3-optimized // (usually including multiple channels at a time). void convert_fixed16_to_fp32(float *dst, size_t out_channel, size_t out_num_channels, const uint8_t *src, size_t in_channel, size_t in_num_channels, size_t num_samples) { assert(in_channel < in_num_channels); assert(out_channel < out_num_channels); src += in_channel * 2; dst += out_channel; for (size_t i = 0; i < num_samples; ++i) { int16_t s = le16toh(*(int16_t *)src); *dst = s * (1.0f / 32768.0f); src += 2 * in_num_channels; dst += out_num_channels; } } void convert_fixed24_to_fp32(float *dst, size_t out_channel, size_t out_num_channels, const uint8_t *src, size_t in_channel, size_t in_num_channels, size_t num_samples) { assert(in_channel < in_num_channels); assert(out_channel < out_num_channels); src += in_channel * 3; dst += out_channel; for (size_t i = 0; i < num_samples; ++i) { uint32_t s1 = src[0]; uint32_t s2 = src[1]; uint32_t s3 = src[2]; uint32_t s = s1 | (s1 << 8) | (s2 << 16) | (s3 << 24); *dst = int(s) * (1.0f / 2147483648.0f); src += 3 * in_num_channels; dst += out_num_channels; } } void convert_fixed32_to_fp32(float *dst, size_t out_channel, size_t out_num_channels, const uint8_t *src, size_t in_channel, size_t in_num_channels, size_t num_samples) { assert(in_channel < in_num_channels); assert(out_channel < out_num_channels); src += in_channel * 4; dst += out_channel; for (size_t i = 0; i < num_samples; ++i) { int32_t s = le32toh(*(int32_t *)src); *dst = s * (1.0f / 2147483648.0f); src += 4 * in_num_channels; dst += out_num_channels; } } float find_peak_plain(const float *samples, size_t num_samples) __attribute__((unused)); float find_peak_plain(const float *samples, size_t num_samples) { float m = fabs(samples[0]); for (size_t i = 1; i < num_samples; ++i) { m = max(m, fabs(samples[i])); } return m; } #ifdef __SSE__ static inline float horizontal_max(__m128 m) { __m128 tmp = _mm_shuffle_ps(m, m, _MM_SHUFFLE(1, 0, 3, 2)); m = _mm_max_ps(m, tmp); tmp = _mm_shuffle_ps(m, m, _MM_SHUFFLE(2, 3, 0, 1)); m = _mm_max_ps(m, tmp); return _mm_cvtss_f32(m); } float find_peak(const float *samples, size_t num_samples) { const __m128 abs_mask = _mm_castsi128_ps(_mm_set1_epi32(0x7fffffffu)); __m128 m = _mm_setzero_ps(); for (size_t i = 0; i < (num_samples & ~3); i += 4) { __m128 x = _mm_loadu_ps(samples + i); x = _mm_and_ps(x, abs_mask); m = _mm_max_ps(m, x); } float result = horizontal_max(m); for (size_t i = (num_samples & ~3); i < num_samples; ++i) { result = max(result, fabs(samples[i])); } #if 0 // Self-test. We should be bit-exact the same. float reference_result = find_peak_plain(samples, num_samples); if (result != reference_result) { fprintf(stderr, "Error: Peak is %f [%f %f %f %f]; should be %f.\n", result, _mm_cvtss_f32(_mm_shuffle_ps(m, m, _MM_SHUFFLE(0, 0, 0, 0))), _mm_cvtss_f32(_mm_shuffle_ps(m, m, _MM_SHUFFLE(1, 1, 1, 1))), _mm_cvtss_f32(_mm_shuffle_ps(m, m, _MM_SHUFFLE(2, 2, 2, 2))), _mm_cvtss_f32(_mm_shuffle_ps(m, m, _MM_SHUFFLE(3, 3, 3, 3))), reference_result); abort(); } #endif return result; } #else float find_peak(const float *samples, size_t num_samples) { return find_peak_plain(samples, num_samples); } #endif void deinterleave_samples(const vector &in, vector *out_l, vector *out_r) { size_t num_samples = in.size() / 2; out_l->resize(num_samples); out_r->resize(num_samples); const float *inptr = in.data(); float *lptr = &(*out_l)[0]; float *rptr = &(*out_r)[0]; for (size_t i = 0; i < num_samples; ++i) { *lptr++ = *inptr++; *rptr++ = *inptr++; } } } // namespace AudioMixer::AudioMixer(unsigned num_cards) : num_cards(num_cards), limiter(OUTPUT_FREQUENCY), correlation(OUTPUT_FREQUENCY) { for (unsigned bus_index = 0; bus_index < MAX_BUSES; ++bus_index) { locut[bus_index].init(FILTER_HPF, 2); eq[bus_index][EQ_BAND_BASS].init(FILTER_LOW_SHELF, 1); // Note: EQ_BAND_MID isn't used (see comments in apply_eq()). eq[bus_index][EQ_BAND_TREBLE].init(FILTER_HIGH_SHELF, 1); compressor[bus_index].reset(new StereoCompressor(OUTPUT_FREQUENCY)); level_compressor[bus_index].reset(new StereoCompressor(OUTPUT_FREQUENCY)); set_bus_settings(bus_index, get_default_bus_settings()); } set_limiter_enabled(global_flags.limiter_enabled); set_final_makeup_gain_auto(global_flags.final_makeup_gain_auto); if (!global_flags.input_mapping_filename.empty()) { current_mapping_mode = MappingMode::MULTICHANNEL; InputMapping new_input_mapping; if (!load_input_mapping_from_file(get_devices(), global_flags.input_mapping_filename, &new_input_mapping)) { fprintf(stderr, "Failed to load input mapping from '%s', exiting.\n", global_flags.input_mapping_filename.c_str()); exit(1); } set_input_mapping(new_input_mapping); } else { set_simple_input(/*card_index=*/0); if (global_flags.multichannel_mapping_mode) { current_mapping_mode = MappingMode::MULTICHANNEL; } } r128.init(2, OUTPUT_FREQUENCY); r128.integr_start(); // hlen=16 is pretty low quality, but we use quite a bit of CPU otherwise, // and there's a limit to how important the peak meter is. peak_resampler.setup(OUTPUT_FREQUENCY, OUTPUT_FREQUENCY * 4, /*num_channels=*/2, /*hlen=*/16, /*frel=*/1.0); global_audio_mixer = this; alsa_pool.init(); } void AudioMixer::reset_resampler(DeviceSpec device_spec) { lock_guard lock(audio_mutex); reset_resampler_mutex_held(device_spec); } void AudioMixer::reset_resampler_mutex_held(DeviceSpec device_spec) { AudioDevice *device = find_audio_device(device_spec); if (device->interesting_channels.empty()) { device->resampling_queue.reset(); } else { // TODO: ResamplingQueue should probably take the full device spec. // (It's only used for console output, though.) device->resampling_queue.reset(new ResamplingQueue(device_spec.index, device->capture_frequency, OUTPUT_FREQUENCY, device->interesting_channels.size())); } device->next_local_pts = 0; } bool AudioMixer::add_audio(DeviceSpec device_spec, const uint8_t *data, unsigned num_samples, AudioFormat audio_format, int64_t frame_length) { AudioDevice *device = find_audio_device(device_spec); unique_lock lock(audio_mutex, defer_lock); if (!lock.try_lock_for(chrono::milliseconds(10))) { return false; } if (device->resampling_queue == nullptr) { // No buses use this device; throw it away. return true; } unsigned num_channels = device->interesting_channels.size(); assert(num_channels > 0); // Convert the audio to fp32. unique_ptr audio(new float[num_samples * num_channels]); unsigned channel_index = 0; for (auto channel_it = device->interesting_channels.cbegin(); channel_it != device->interesting_channels.end(); ++channel_it, ++channel_index) { switch (audio_format.bits_per_sample) { case 0: assert(num_samples == 0); break; case 16: convert_fixed16_to_fp32(audio.get(), channel_index, num_channels, data, *channel_it, audio_format.num_channels, num_samples); break; case 24: convert_fixed24_to_fp32(audio.get(), channel_index, num_channels, data, *channel_it, audio_format.num_channels, num_samples); break; case 32: convert_fixed32_to_fp32(audio.get(), channel_index, num_channels, data, *channel_it, audio_format.num_channels, num_samples); break; default: fprintf(stderr, "Cannot handle audio with %u bits per sample\n", audio_format.bits_per_sample); assert(false); } } // Now add it. int64_t local_pts = device->next_local_pts; device->resampling_queue->add_input_samples(local_pts / double(TIMEBASE), audio.get(), num_samples); device->next_local_pts = local_pts + frame_length; return true; } bool AudioMixer::add_silence(DeviceSpec device_spec, unsigned samples_per_frame, unsigned num_frames, int64_t frame_length) { AudioDevice *device = find_audio_device(device_spec); unique_lock lock(audio_mutex, defer_lock); if (!lock.try_lock_for(chrono::milliseconds(10))) { return false; } if (device->resampling_queue == nullptr) { // No buses use this device; throw it away. return true; } unsigned num_channels = device->interesting_channels.size(); assert(num_channels > 0); vector silence(samples_per_frame * num_channels, 0.0f); for (unsigned i = 0; i < num_frames; ++i) { device->resampling_queue->add_input_samples(device->next_local_pts / double(TIMEBASE), silence.data(), samples_per_frame); // Note that if the format changed in the meantime, we have // no way of detecting that; we just have to assume the frame length // is always the same. device->next_local_pts += frame_length; } return true; } bool AudioMixer::silence_card(DeviceSpec device_spec, bool silence) { AudioDevice *device = find_audio_device(device_spec); unique_lock lock(audio_mutex, defer_lock); if (!lock.try_lock_for(chrono::milliseconds(10))) { return false; } if (device->silenced && !silence) { reset_resampler_mutex_held(device_spec); } device->silenced = silence; return true; } AudioMixer::BusSettings AudioMixer::get_default_bus_settings() { BusSettings settings; settings.fader_volume_db = 0.0f; settings.muted = false; settings.locut_enabled = global_flags.locut_enabled; for (unsigned band_index = 0; band_index < NUM_EQ_BANDS; ++band_index) { settings.eq_level_db[band_index] = 0.0f; } settings.gain_staging_db = global_flags.initial_gain_staging_db; settings.level_compressor_enabled = global_flags.gain_staging_auto; settings.compressor_threshold_dbfs = ref_level_dbfs - 12.0f; // -12 dB. settings.compressor_enabled = global_flags.compressor_enabled; return settings; } AudioMixer::BusSettings AudioMixer::get_bus_settings(unsigned bus_index) const { lock_guard lock(audio_mutex); BusSettings settings; settings.fader_volume_db = fader_volume_db[bus_index]; settings.muted = mute[bus_index]; settings.locut_enabled = locut_enabled[bus_index]; for (unsigned band_index = 0; band_index < NUM_EQ_BANDS; ++band_index) { settings.eq_level_db[band_index] = eq_level_db[bus_index][band_index]; } settings.gain_staging_db = gain_staging_db[bus_index]; settings.level_compressor_enabled = level_compressor_enabled[bus_index]; settings.compressor_threshold_dbfs = compressor_threshold_dbfs[bus_index]; settings.compressor_enabled = compressor_enabled[bus_index]; return settings; } void AudioMixer::set_bus_settings(unsigned bus_index, const AudioMixer::BusSettings &settings) { lock_guard lock(audio_mutex); fader_volume_db[bus_index] = settings.fader_volume_db; mute[bus_index] = settings.muted; locut_enabled[bus_index] = settings.locut_enabled; for (unsigned band_index = 0; band_index < NUM_EQ_BANDS; ++band_index) { eq_level_db[bus_index][band_index] = settings.eq_level_db[band_index]; } gain_staging_db[bus_index] = settings.gain_staging_db; last_gain_staging_db[bus_index] = gain_staging_db[bus_index]; level_compressor_enabled[bus_index] = settings.level_compressor_enabled; compressor_threshold_dbfs[bus_index] = settings.compressor_threshold_dbfs; compressor_enabled[bus_index] = settings.compressor_enabled; } AudioMixer::AudioDevice *AudioMixer::find_audio_device(DeviceSpec device) { switch (device.type) { case InputSourceType::CAPTURE_CARD: return &video_cards[device.index]; case InputSourceType::ALSA_INPUT: return &alsa_inputs[device.index]; case InputSourceType::SILENCE: default: assert(false); } return nullptr; } // Get a pointer to the given channel from the given device. // The channel must be picked out earlier and resampled. void AudioMixer::find_sample_src_from_device(const map> &samples_card, DeviceSpec device_spec, int source_channel, const float **srcptr, unsigned *stride) { static float zero = 0.0f; if (source_channel == -1 || device_spec.type == InputSourceType::SILENCE) { *srcptr = &zero; *stride = 0; return; } AudioDevice *device = find_audio_device(device_spec); assert(device->interesting_channels.count(source_channel) != 0); unsigned channel_index = 0; for (int channel : device->interesting_channels) { if (channel == source_channel) break; ++channel_index; } assert(channel_index < device->interesting_channels.size()); const auto it = samples_card.find(device_spec); assert(it != samples_card.end()); *srcptr = &(it->second)[channel_index]; *stride = device->interesting_channels.size(); } // TODO: Can be SSSE3-optimized if need be. void AudioMixer::fill_audio_bus(const map> &samples_card, const InputMapping::Bus &bus, unsigned num_samples, float *output) { if (bus.device.type == InputSourceType::SILENCE) { memset(output, 0, num_samples * 2 * sizeof(*output)); } else { assert(bus.device.type == InputSourceType::CAPTURE_CARD || bus.device.type == InputSourceType::ALSA_INPUT); const float *lsrc, *rsrc; unsigned lstride, rstride; float *dptr = output; find_sample_src_from_device(samples_card, bus.device, bus.source_channel[0], &lsrc, &lstride); find_sample_src_from_device(samples_card, bus.device, bus.source_channel[1], &rsrc, &rstride); for (unsigned i = 0; i < num_samples; ++i) { *dptr++ = *lsrc; *dptr++ = *rsrc; lsrc += lstride; rsrc += rstride; } } } vector AudioMixer::get_active_devices() const { vector ret; for (unsigned card_index = 0; card_index < MAX_VIDEO_CARDS; ++card_index) { const DeviceSpec device_spec{InputSourceType::CAPTURE_CARD, card_index}; if (!find_audio_device(device_spec)->interesting_channels.empty()) { ret.push_back(device_spec); } } for (unsigned card_index = 0; card_index < MAX_ALSA_CARDS; ++card_index) { const DeviceSpec device_spec{InputSourceType::ALSA_INPUT, card_index}; if (!find_audio_device(device_spec)->interesting_channels.empty()) { ret.push_back(device_spec); } } return ret; } namespace { void apply_gain(float db, float last_db, vector *samples) { if (fabs(db - last_db) < 1e-3) { // Constant over this frame. const float gain = from_db(db); for (size_t i = 0; i < samples->size(); ++i) { (*samples)[i] *= gain; } } else { // We need to do a fade. unsigned num_samples = samples->size() / 2; float gain = from_db(last_db); const float gain_inc = pow(from_db(db - last_db), 1.0 / num_samples); for (size_t i = 0; i < num_samples; ++i) { (*samples)[i * 2 + 0] *= gain; (*samples)[i * 2 + 1] *= gain; gain *= gain_inc; } } } } // namespace vector AudioMixer::get_output(double pts, unsigned num_samples, ResamplingQueue::RateAdjustmentPolicy rate_adjustment_policy) { map> samples_card; vector samples_bus; lock_guard lock(audio_mutex); // Pick out all the interesting channels from all the cards. for (const DeviceSpec &device_spec : get_active_devices()) { AudioDevice *device = find_audio_device(device_spec); samples_card[device_spec].resize(num_samples * device->interesting_channels.size()); if (device->silenced) { memset(&samples_card[device_spec][0], 0, samples_card[device_spec].size() * sizeof(float)); } else { device->resampling_queue->get_output_samples( pts, &samples_card[device_spec][0], num_samples, rate_adjustment_policy); } } vector samples_out, left, right; samples_out.resize(num_samples * 2); samples_bus.resize(num_samples * 2); for (unsigned bus_index = 0; bus_index < input_mapping.buses.size(); ++bus_index) { fill_audio_bus(samples_card, input_mapping.buses[bus_index], num_samples, &samples_bus[0]); apply_eq(bus_index, &samples_bus); { lock_guard lock(compressor_mutex); // Apply a level compressor to get the general level right. // Basically, if it's over about -40 dBFS, we squeeze it down to that level // (or more precisely, near it, since we don't use infinite ratio), // then apply a makeup gain to get it to -14 dBFS. -14 dBFS is, of course, // entirely arbitrary, but from practical tests with speech, it seems to // put ut around -23 LUFS, so it's a reasonable starting point for later use. if (level_compressor_enabled[bus_index]) { float threshold = 0.01f; // -40 dBFS. float ratio = 20.0f; float attack_time = 0.5f; float release_time = 20.0f; float makeup_gain = from_db(ref_level_dbfs - (-40.0f)); // +26 dB. level_compressor[bus_index]->process(samples_bus.data(), samples_bus.size() / 2, threshold, ratio, attack_time, release_time, makeup_gain); gain_staging_db[bus_index] = to_db(level_compressor[bus_index]->get_attenuation() * makeup_gain); } else { // Just apply the gain we already had. float db = gain_staging_db[bus_index]; float last_db = last_gain_staging_db[bus_index]; apply_gain(db, last_db, &samples_bus); } last_gain_staging_db[bus_index] = gain_staging_db[bus_index]; #if 0 printf("level=%f (%+5.2f dBFS) attenuation=%f (%+5.2f dB) end_result=%+5.2f dB\n", level_compressor.get_level(), to_db(level_compressor.get_level()), level_compressor.get_attenuation(), to_db(level_compressor.get_attenuation()), to_db(level_compressor.get_level() * level_compressor.get_attenuation() * makeup_gain)); #endif // The real compressor. if (compressor_enabled[bus_index]) { float threshold = from_db(compressor_threshold_dbfs[bus_index]); float ratio = 20.0f; float attack_time = 0.005f; float release_time = 0.040f; float makeup_gain = 2.0f; // +6 dB. compressor[bus_index]->process(samples_bus.data(), samples_bus.size() / 2, threshold, ratio, attack_time, release_time, makeup_gain); // compressor_att = compressor.get_attenuation(); } } add_bus_to_master(bus_index, samples_bus, &samples_out); deinterleave_samples(samples_bus, &left, &right); measure_bus_levels(bus_index, left, right); } { lock_guard lock(compressor_mutex); // Finally a limiter at -4 dB (so, -10 dBFS) to take out the worst peaks only. // Note that since ratio is not infinite, we could go slightly higher than this. if (limiter_enabled) { float threshold = from_db(limiter_threshold_dbfs); float ratio = 30.0f; float attack_time = 0.0f; // Instant. float release_time = 0.020f; float makeup_gain = 1.0f; // 0 dB. limiter.process(samples_out.data(), samples_out.size() / 2, threshold, ratio, attack_time, release_time, makeup_gain); // limiter_att = limiter.get_attenuation(); } // printf("limiter=%+5.1f compressor=%+5.1f\n", to_db(limiter_att), to_db(compressor_att)); } // At this point, we are most likely close to +0 LU (at least if the // faders sum to 0 dB and the compressors are on), but all of our // measurements have been on raw sample values, not R128 values. // So we have a final makeup gain to get us to +0 LU; the gain // adjustments required should be relatively small, and also, the // offset shouldn't change much (only if the type of audio changes // significantly). Thus, we shoot for updating this value basically // “whenever we process buffers”, since the R128 calculation isn't exactly // something we get out per-sample. // // Note that there's a feedback loop here, so we choose a very slow filter // (half-time of 30 seconds). double target_loudness_factor, alpha; double loudness_lu = r128.loudness_M() - ref_level_lufs; target_loudness_factor = final_makeup_gain * from_db(-loudness_lu); // If we're outside +/- 5 LU (after correction), we don't count it as // a normal signal (probably silence) and don't change the // correction factor; just apply what we already have. if (fabs(loudness_lu) >= 5.0 || !final_makeup_gain_auto) { alpha = 0.0; } else { // Formula adapted from // https://en.wikipedia.org/wiki/Low-pass_filter#Simple_infinite_impulse_response_filter. const double half_time_s = 30.0; const double fc_mul_2pi_delta_t = 1.0 / (half_time_s * OUTPUT_FREQUENCY); alpha = fc_mul_2pi_delta_t / (fc_mul_2pi_delta_t + 1.0); } { lock_guard lock(compressor_mutex); double m = final_makeup_gain; for (size_t i = 0; i < samples_out.size(); i += 2) { samples_out[i + 0] *= m; samples_out[i + 1] *= m; m += (target_loudness_factor - m) * alpha; } final_makeup_gain = m; } update_meters(samples_out); return samples_out; } namespace { void apply_filter_fade(StereoFilter *filter, float *data, unsigned num_samples, float cutoff_hz, float db, float last_db) { // A granularity of 32 samples is an okay tradeoff between speed and // smoothness; recalculating the filters is pretty expensive, so it's // good that we don't do this all the time. static constexpr unsigned filter_granularity_samples = 32; const float cutoff_linear = cutoff_hz * 2.0 * M_PI / OUTPUT_FREQUENCY; if (fabs(db - last_db) < 1e-3) { // Constant over this frame. if (fabs(db) > 0.01f) { filter->render(data, num_samples, cutoff_linear, 0.5f, db / 40.0f); } } else { // We need to do a fade. (Rounding up avoids division by zero.) unsigned num_blocks = (num_samples + filter_granularity_samples - 1) / filter_granularity_samples; const float inc_db_norm = (db - last_db) / 40.0f / num_blocks; float db_norm = db / 40.0f; for (size_t i = 0; i < num_samples; i += filter_granularity_samples) { size_t samples_this_block = std::min(num_samples - i, filter_granularity_samples); filter->render(data + i * 2, samples_this_block, cutoff_linear, 0.5f, db_norm); db_norm += inc_db_norm; } } } } // namespace void AudioMixer::apply_eq(unsigned bus_index, vector *samples_bus) { constexpr float bass_freq_hz = 200.0f; constexpr float treble_freq_hz = 4700.0f; // Cut away everything under 120 Hz (or whatever the cutoff is); // we don't need it for voice, and it will reduce headroom // and confuse the compressor. (In particular, any hums at 50 or 60 Hz // should be dampened.) if (locut_enabled[bus_index]) { locut[bus_index].render(samples_bus->data(), samples_bus->size() / 2, locut_cutoff_hz * 2.0 * M_PI / OUTPUT_FREQUENCY, 0.5f); } // Apply the rest of the EQ. Since we only have a simple three-band EQ, // we can implement it with two shelf filters. We use a simple gain to // set the mid-level filter, and then offset the low and high bands // from that if we need to. (We could perhaps have folded the gain into // the next part, but it's so cheap that the trouble isn't worth it.) // // If any part of the EQ has changed appreciably since last frame, // we fade smoothly during the course of this frame. const float bass_db = eq_level_db[bus_index][EQ_BAND_BASS]; const float mid_db = eq_level_db[bus_index][EQ_BAND_MID]; const float treble_db = eq_level_db[bus_index][EQ_BAND_TREBLE]; const float last_bass_db = last_eq_level_db[bus_index][EQ_BAND_BASS]; const float last_mid_db = last_eq_level_db[bus_index][EQ_BAND_MID]; const float last_treble_db = last_eq_level_db[bus_index][EQ_BAND_TREBLE]; assert(samples_bus->size() % 2 == 0); const unsigned num_samples = samples_bus->size() / 2; apply_gain(mid_db, last_mid_db, samples_bus); apply_filter_fade(&eq[bus_index][EQ_BAND_BASS], samples_bus->data(), num_samples, bass_freq_hz, bass_db - mid_db, last_bass_db - last_mid_db); apply_filter_fade(&eq[bus_index][EQ_BAND_TREBLE], samples_bus->data(), num_samples, treble_freq_hz, treble_db - mid_db, last_treble_db - last_mid_db); last_eq_level_db[bus_index][EQ_BAND_BASS] = bass_db; last_eq_level_db[bus_index][EQ_BAND_MID] = mid_db; last_eq_level_db[bus_index][EQ_BAND_TREBLE] = treble_db; } void AudioMixer::add_bus_to_master(unsigned bus_index, const vector &samples_bus, vector *samples_out) { assert(samples_bus.size() == samples_out->size()); assert(samples_bus.size() % 2 == 0); unsigned num_samples = samples_bus.size() / 2; const float new_volume_db = mute[bus_index] ? -90.0f : fader_volume_db[bus_index].load(); if (fabs(new_volume_db - last_fader_volume_db[bus_index]) > 1e-3) { // The volume has changed; do a fade over the course of this frame. // (We might have some numerical issues here, but it seems to sound OK.) // For the purpose of fading here, the silence floor is set to -90 dB // (the fader only goes to -84). float old_volume = from_db(max(last_fader_volume_db[bus_index], -90.0f)); float volume = from_db(max(new_volume_db, -90.0f)); float volume_inc = pow(volume / old_volume, 1.0 / num_samples); volume = old_volume; if (bus_index == 0) { for (unsigned i = 0; i < num_samples; ++i) { (*samples_out)[i * 2 + 0] = samples_bus[i * 2 + 0] * volume; (*samples_out)[i * 2 + 1] = samples_bus[i * 2 + 1] * volume; volume *= volume_inc; } } else { for (unsigned i = 0; i < num_samples; ++i) { (*samples_out)[i * 2 + 0] += samples_bus[i * 2 + 0] * volume; (*samples_out)[i * 2 + 1] += samples_bus[i * 2 + 1] * volume; volume *= volume_inc; } } } else if (new_volume_db > -90.0f) { float volume = from_db(new_volume_db); if (bus_index == 0) { for (unsigned i = 0; i < num_samples; ++i) { (*samples_out)[i * 2 + 0] = samples_bus[i * 2 + 0] * volume; (*samples_out)[i * 2 + 1] = samples_bus[i * 2 + 1] * volume; } } else { for (unsigned i = 0; i < num_samples; ++i) { (*samples_out)[i * 2 + 0] += samples_bus[i * 2 + 0] * volume; (*samples_out)[i * 2 + 1] += samples_bus[i * 2 + 1] * volume; } } } last_fader_volume_db[bus_index] = new_volume_db; } void AudioMixer::measure_bus_levels(unsigned bus_index, const vector &left, const vector &right) { assert(left.size() == right.size()); const float volume = mute[bus_index] ? 0.0f : from_db(fader_volume_db[bus_index]); const float peak_levels[2] = { find_peak(left.data(), left.size()) * volume, find_peak(right.data(), right.size()) * volume }; for (unsigned channel = 0; channel < 2; ++channel) { // Compute the current value, including hold and falloff. // The constants are borrowed from zita-mu1 by Fons Adriaensen. static constexpr float hold_sec = 0.5f; static constexpr float falloff_db_sec = 15.0f; // dB/sec falloff after hold. float current_peak; PeakHistory &history = peak_history[bus_index][channel]; history.historic_peak = max(history.historic_peak, peak_levels[channel]); if (history.age_seconds < hold_sec) { current_peak = history.last_peak; } else { current_peak = history.last_peak * from_db(-falloff_db_sec * (history.age_seconds - hold_sec)); } // See if we have a new peak to replace the old (possibly falling) one. if (peak_levels[channel] > current_peak) { history.last_peak = peak_levels[channel]; history.age_seconds = 0.0f; // Not 100% correct, but more than good enough given our frame sizes. current_peak = peak_levels[channel]; } else { history.age_seconds += float(left.size()) / OUTPUT_FREQUENCY; } history.current_level = peak_levels[channel]; history.current_peak = current_peak; } } void AudioMixer::update_meters(const vector &samples) { // Upsample 4x to find interpolated peak. peak_resampler.inp_data = const_cast(samples.data()); peak_resampler.inp_count = samples.size() / 2; vector interpolated_samples; interpolated_samples.resize(samples.size()); { lock_guard lock(audio_measure_mutex); while (peak_resampler.inp_count > 0) { // About four iterations. peak_resampler.out_data = &interpolated_samples[0]; peak_resampler.out_count = interpolated_samples.size() / 2; peak_resampler.process(); size_t out_stereo_samples = interpolated_samples.size() / 2 - peak_resampler.out_count; peak = max(peak, find_peak(interpolated_samples.data(), out_stereo_samples * 2)); peak_resampler.out_data = nullptr; } } // Find R128 levels and L/R correlation. vector left, right; deinterleave_samples(samples, &left, &right); float *ptrs[] = { left.data(), right.data() }; { lock_guard lock(audio_measure_mutex); r128.process(left.size(), ptrs); correlation.process_samples(samples); } send_audio_level_callback(); } void AudioMixer::reset_meters() { lock_guard lock(audio_measure_mutex); peak_resampler.reset(); peak = 0.0f; r128.reset(); r128.integr_start(); correlation.reset(); } void AudioMixer::send_audio_level_callback() { if (audio_level_callback == nullptr) { return; } lock_guard lock(audio_measure_mutex); double loudness_s = r128.loudness_S(); double loudness_i = r128.integrated(); double loudness_range_low = r128.range_min(); double loudness_range_high = r128.range_max(); vector bus_levels; bus_levels.resize(input_mapping.buses.size()); { lock_guard lock(compressor_mutex); for (unsigned bus_index = 0; bus_index < bus_levels.size(); ++bus_index) { bus_levels[bus_index].current_level_dbfs[0] = to_db(peak_history[bus_index][0].current_level); bus_levels[bus_index].current_level_dbfs[1] = to_db(peak_history[bus_index][1].current_level); bus_levels[bus_index].peak_level_dbfs[0] = to_db(peak_history[bus_index][0].current_peak); bus_levels[bus_index].peak_level_dbfs[1] = to_db(peak_history[bus_index][1].current_peak); bus_levels[bus_index].historic_peak_dbfs = to_db( max(peak_history[bus_index][0].historic_peak, peak_history[bus_index][1].historic_peak)); bus_levels[bus_index].gain_staging_db = gain_staging_db[bus_index]; if (compressor_enabled[bus_index]) { bus_levels[bus_index].compressor_attenuation_db = -to_db(compressor[bus_index]->get_attenuation()); } else { bus_levels[bus_index].compressor_attenuation_db = 0.0; } } } audio_level_callback(loudness_s, to_db(peak), bus_levels, loudness_i, loudness_range_low, loudness_range_high, to_db(final_makeup_gain), correlation.get_correlation()); } map AudioMixer::get_devices() { lock_guard lock(audio_mutex); map devices; for (unsigned card_index = 0; card_index < num_cards; ++card_index) { const DeviceSpec spec{ InputSourceType::CAPTURE_CARD, card_index }; const AudioDevice *device = &video_cards[card_index]; DeviceInfo info; info.display_name = device->display_name; info.num_channels = 8; devices.insert(make_pair(spec, info)); } vector available_alsa_devices = alsa_pool.get_devices(); for (unsigned card_index = 0; card_index < available_alsa_devices.size(); ++card_index) { const DeviceSpec spec{ InputSourceType::ALSA_INPUT, card_index }; const ALSAPool::Device &device = available_alsa_devices[card_index]; DeviceInfo info; info.display_name = device.display_name(); info.num_channels = device.num_channels; info.alsa_name = device.name; info.alsa_info = device.info; info.alsa_address = device.address; devices.insert(make_pair(spec, info)); } return devices; } void AudioMixer::set_display_name(DeviceSpec device_spec, const string &name) { AudioDevice *device = find_audio_device(device_spec); lock_guard lock(audio_mutex); device->display_name = name; } void AudioMixer::serialize_device(DeviceSpec device_spec, DeviceSpecProto *device_spec_proto) { lock_guard lock(audio_mutex); switch (device_spec.type) { case InputSourceType::SILENCE: device_spec_proto->set_type(DeviceSpecProto::SILENCE); break; case InputSourceType::CAPTURE_CARD: device_spec_proto->set_type(DeviceSpecProto::CAPTURE_CARD); device_spec_proto->set_index(device_spec.index); device_spec_proto->set_display_name(video_cards[device_spec.index].display_name); break; case InputSourceType::ALSA_INPUT: alsa_pool.serialize_device(device_spec.index, device_spec_proto); break; } } void AudioMixer::set_simple_input(unsigned card_index) { InputMapping new_input_mapping; InputMapping::Bus input; input.name = "Main"; input.device.type = InputSourceType::CAPTURE_CARD; input.device.index = card_index; input.source_channel[0] = 0; input.source_channel[1] = 1; new_input_mapping.buses.push_back(input); lock_guard lock(audio_mutex); current_mapping_mode = MappingMode::SIMPLE; set_input_mapping_lock_held(new_input_mapping); fader_volume_db[0] = 0.0f; } unsigned AudioMixer::get_simple_input() const { lock_guard lock(audio_mutex); if (input_mapping.buses.size() == 1 && input_mapping.buses[0].device.type == InputSourceType::CAPTURE_CARD && input_mapping.buses[0].source_channel[0] == 0 && input_mapping.buses[0].source_channel[1] == 1) { return input_mapping.buses[0].device.index; } else { return numeric_limits::max(); } } void AudioMixer::set_input_mapping(const InputMapping &new_input_mapping) { lock_guard lock(audio_mutex); set_input_mapping_lock_held(new_input_mapping); current_mapping_mode = MappingMode::MULTICHANNEL; } AudioMixer::MappingMode AudioMixer::get_mapping_mode() const { lock_guard lock(audio_mutex); return current_mapping_mode; } void AudioMixer::set_input_mapping_lock_held(const InputMapping &new_input_mapping) { map> interesting_channels; for (const InputMapping::Bus &bus : new_input_mapping.buses) { if (bus.device.type == InputSourceType::CAPTURE_CARD || bus.device.type == InputSourceType::ALSA_INPUT) { for (unsigned channel = 0; channel < 2; ++channel) { if (bus.source_channel[channel] != -1) { interesting_channels[bus.device].insert(bus.source_channel[channel]); } } } } // Reset resamplers for all cards that don't have the exact same state as before. for (unsigned card_index = 0; card_index < MAX_VIDEO_CARDS; ++card_index) { const DeviceSpec device_spec{InputSourceType::CAPTURE_CARD, card_index}; AudioDevice *device = find_audio_device(device_spec); if (device->interesting_channels != interesting_channels[device_spec]) { device->interesting_channels = interesting_channels[device_spec]; reset_resampler_mutex_held(device_spec); } } for (unsigned card_index = 0; card_index < MAX_ALSA_CARDS; ++card_index) { const DeviceSpec device_spec{InputSourceType::ALSA_INPUT, card_index}; AudioDevice *device = find_audio_device(device_spec); if (interesting_channels[device_spec].empty()) { alsa_pool.release_device(card_index); } else { alsa_pool.hold_device(card_index); } if (device->interesting_channels != interesting_channels[device_spec]) { device->interesting_channels = interesting_channels[device_spec]; alsa_pool.reset_device(device_spec.index); reset_resampler_mutex_held(device_spec); } } input_mapping = new_input_mapping; } InputMapping AudioMixer::get_input_mapping() const { lock_guard lock(audio_mutex); return input_mapping; } unsigned AudioMixer::num_buses() const { lock_guard lock(audio_mutex); return input_mapping.buses.size(); } void AudioMixer::reset_peak(unsigned bus_index) { lock_guard lock(audio_mutex); for (unsigned channel = 0; channel < 2; ++channel) { PeakHistory &history = peak_history[bus_index][channel]; history.current_level = 0.0f; history.historic_peak = 0.0f; history.current_peak = 0.0f; history.last_peak = 0.0f; history.age_seconds = 0.0f; } } AudioMixer *global_audio_mixer = nullptr; nageru-1.4.2/audio_mixer.h000066400000000000000000000320521301561706600154640ustar00rootroot00000000000000#ifndef _AUDIO_MIXER_H #define _AUDIO_MIXER_H 1 // The audio mixer, dealing with extracting the right signals from // each capture card, resampling signals so that they are in sync, // processing them with effects (if desired), and then mixing them // all together into one final audio signal. // // All operations on AudioMixer (except destruction) are thread-safe. #include #include #include #include #include #include #include #include #include #include #include #include "alsa_pool.h" #include "correlation_measurer.h" #include "db.h" #include "defs.h" #include "ebu_r128_proc.h" #include "filter.h" #include "input_mapping.h" #include "resampling_queue.h" #include "stereocompressor.h" class DeviceSpecProto; namespace bmusb { struct AudioFormat; } // namespace bmusb enum EQBand { EQ_BAND_BASS = 0, EQ_BAND_MID, EQ_BAND_TREBLE, NUM_EQ_BANDS }; class AudioMixer { public: AudioMixer(unsigned num_cards); void reset_resampler(DeviceSpec device_spec); void reset_meters(); // Add audio (or silence) to the given device's queue. Can return false if // the lock wasn't successfully taken; if so, you should simply try again. // (This is to avoid a deadlock where a card hangs on the mutex in add_audio() // while we are trying to shut it down from another thread that also holds // the mutex.) frame_length is in TIMEBASE units. bool add_audio(DeviceSpec device_spec, const uint8_t *data, unsigned num_samples, bmusb::AudioFormat audio_format, int64_t frame_length); bool add_silence(DeviceSpec device_spec, unsigned samples_per_frame, unsigned num_frames, int64_t frame_length); // If a given device is offline for whatever reason and cannot deliver audio // (by means of add_audio() or add_silence()), you can call put it in silence mode, // where it will be taken to only output silence. Note that when taking it _out_ // of silence mode, the resampler will be reset, so that old audio will not // affect it. Same true/false behavior as add_audio(). bool silence_card(DeviceSpec device_spec, bool silence); std::vector get_output(double pts, unsigned num_samples, ResamplingQueue::RateAdjustmentPolicy rate_adjustment_policy); float get_fader_volume(unsigned bus_index) const { return fader_volume_db[bus_index]; } void set_fader_volume(unsigned bus_index, float level_db) { fader_volume_db[bus_index] = level_db; } bool get_mute(unsigned bus_index) const { return mute[bus_index]; } void set_mute(unsigned bus_index, bool muted) { mute[bus_index] = muted; } // Note: This operation holds all ALSA devices (see ALSAPool::get_devices()). // You will need to call set_input_mapping() to get the hold state correctly, // or every card will be held forever. std::map get_devices(); // See comments on ALSAPool::get_card_state(). ALSAPool::Device::State get_alsa_card_state(unsigned index) { return alsa_pool.get_card_state(index); } // See comments on ALSAPool::create_dead_card(). DeviceSpec create_dead_card(const std::string &name, const std::string &info, unsigned num_channels) { unsigned dead_card_index = alsa_pool.create_dead_card(name, info, num_channels); return DeviceSpec{InputSourceType::ALSA_INPUT, dead_card_index}; } void set_display_name(DeviceSpec device_spec, const std::string &name); // Note: The card should be held (currently this isn't enforced, though). void serialize_device(DeviceSpec device_spec, DeviceSpecProto *device_spec_proto); enum class MappingMode { // A single bus, only from a video card (no ALSA devices), // only channel 1 and 2, locked to +0 dB. Note that this is // only an UI abstraction around exactly the same audio code // as MULTICHANNEL; it's just less flexible. SIMPLE, // Full, arbitrary mappings. MULTICHANNEL }; // Automatically sets mapping mode to MappingMode::SIMPLE. void set_simple_input(unsigned card_index); // If mapping mode is not representable as a MappingMode::SIMPLE type // mapping, returns numeric_limits::max(). unsigned get_simple_input() const; // Implicitly sets mapping mode to MappingMode::MULTICHANNEL. void set_input_mapping(const InputMapping &input_mapping); MappingMode get_mapping_mode() const; InputMapping get_input_mapping() const; unsigned num_buses() const; void set_locut_cutoff(float cutoff_hz) { locut_cutoff_hz = cutoff_hz; } float get_locut_cutoff() const { return locut_cutoff_hz; } void set_locut_enabled(unsigned bus, bool enabled) { locut_enabled[bus] = enabled; } bool get_locut_enabled(unsigned bus) { return locut_enabled[bus]; } void set_eq(unsigned bus_index, EQBand band, float db_gain) { assert(band >= 0 && band < NUM_EQ_BANDS); eq_level_db[bus_index][band] = db_gain; } float get_eq(unsigned bus_index, EQBand band) const { assert(band >= 0 && band < NUM_EQ_BANDS); return eq_level_db[bus_index][band]; } float get_limiter_threshold_dbfs() const { return limiter_threshold_dbfs; } float get_compressor_threshold_dbfs(unsigned bus_index) const { return compressor_threshold_dbfs[bus_index]; } void set_limiter_threshold_dbfs(float threshold_dbfs) { limiter_threshold_dbfs = threshold_dbfs; } void set_compressor_threshold_dbfs(unsigned bus_index, float threshold_dbfs) { compressor_threshold_dbfs[bus_index] = threshold_dbfs; } void set_limiter_enabled(bool enabled) { limiter_enabled = enabled; } bool get_limiter_enabled() const { return limiter_enabled; } void set_compressor_enabled(unsigned bus_index, bool enabled) { compressor_enabled[bus_index] = enabled; } bool get_compressor_enabled(unsigned bus_index) const { return compressor_enabled[bus_index]; } void set_gain_staging_db(unsigned bus_index, float gain_db) { std::unique_lock lock(compressor_mutex); level_compressor_enabled[bus_index] = false; gain_staging_db[bus_index] = gain_db; } float get_gain_staging_db(unsigned bus_index) const { std::unique_lock lock(compressor_mutex); return gain_staging_db[bus_index]; } void set_gain_staging_auto(unsigned bus_index, bool enabled) { std::unique_lock lock(compressor_mutex); level_compressor_enabled[bus_index] = enabled; } bool get_gain_staging_auto(unsigned bus_index) const { std::unique_lock lock(compressor_mutex); return level_compressor_enabled[bus_index]; } void set_final_makeup_gain_db(float gain_db) { std::unique_lock lock(compressor_mutex); final_makeup_gain_auto = false; final_makeup_gain = from_db(gain_db); } float get_final_makeup_gain_db() { std::unique_lock lock(compressor_mutex); return to_db(final_makeup_gain); } void set_final_makeup_gain_auto(bool enabled) { std::unique_lock lock(compressor_mutex); final_makeup_gain_auto = enabled; } bool get_final_makeup_gain_auto() const { std::unique_lock lock(compressor_mutex); return final_makeup_gain_auto; } void reset_peak(unsigned bus_index); struct BusLevel { float current_level_dbfs[2]; // Digital peak of last frame, left and right. float peak_level_dbfs[2]; // Digital peak with hold, left and right. float historic_peak_dbfs; float gain_staging_db; float compressor_attenuation_db; // A positive number; 0.0 for no attenuation. }; typedef std::function bus_levels, float global_level_lufs, float range_low_lufs, float range_high_lufs, float final_makeup_gain_db, float correlation)> audio_level_callback_t; void set_audio_level_callback(audio_level_callback_t callback) { audio_level_callback = callback; } typedef std::function state_changed_callback_t; void set_state_changed_callback(state_changed_callback_t callback) { state_changed_callback = callback; } state_changed_callback_t get_state_changed_callback() const { return state_changed_callback; } void trigger_state_changed_callback() { if (state_changed_callback != nullptr) { state_changed_callback(); } } // A combination of all settings for a bus. Useful if you want to get // or store them as a whole without bothering to call all of the get_* // or set_* functions for that bus. struct BusSettings { float fader_volume_db; bool muted; bool locut_enabled; float eq_level_db[NUM_EQ_BANDS]; float gain_staging_db; bool level_compressor_enabled; float compressor_threshold_dbfs; bool compressor_enabled; }; static BusSettings get_default_bus_settings(); BusSettings get_bus_settings(unsigned bus_index) const; void set_bus_settings(unsigned bus_index, const BusSettings &settings); private: struct AudioDevice { std::unique_ptr resampling_queue; int64_t next_local_pts = 0; std::string display_name; unsigned capture_frequency = OUTPUT_FREQUENCY; // Which channels we consider interesting (ie., are part of some input_mapping). std::set interesting_channels; bool silenced = false; }; const AudioDevice *find_audio_device(DeviceSpec device_spec) const { return const_cast(this)->find_audio_device(device_spec); } AudioDevice *find_audio_device(DeviceSpec device_spec); void find_sample_src_from_device(const std::map> &samples_card, DeviceSpec device_spec, int source_channel, const float **srcptr, unsigned *stride); void fill_audio_bus(const std::map> &samples_card, const InputMapping::Bus &bus, unsigned num_samples, float *output); void reset_resampler_mutex_held(DeviceSpec device_spec); void apply_eq(unsigned bus_index, std::vector *samples_bus); void update_meters(const std::vector &samples); void add_bus_to_master(unsigned bus_index, const std::vector &samples_bus, std::vector *samples_out); void measure_bus_levels(unsigned bus_index, const std::vector &left, const std::vector &right); void send_audio_level_callback(); std::vector get_active_devices() const; void set_input_mapping_lock_held(const InputMapping &input_mapping); unsigned num_cards; mutable std::timed_mutex audio_mutex; ALSAPool alsa_pool; AudioDevice video_cards[MAX_VIDEO_CARDS]; // Under audio_mutex. AudioDevice alsa_inputs[MAX_ALSA_CARDS]; // Under audio_mutex. std::atomic locut_cutoff_hz{120}; StereoFilter locut[MAX_BUSES]; // Default cutoff 120 Hz, 24 dB/oct. std::atomic locut_enabled[MAX_BUSES]; StereoFilter eq[MAX_BUSES][NUM_EQ_BANDS]; // The one for EQBand::MID isn't actually used (see comments in apply_eq()). // First compressor; takes us up to about -12 dBFS. mutable std::mutex compressor_mutex; std::unique_ptr level_compressor[MAX_BUSES]; // Under compressor_mutex. Used to set/override gain_staging_db if . float gain_staging_db[MAX_BUSES]; // Under compressor_mutex. float last_gain_staging_db[MAX_BUSES]; // Under compressor_mutex. bool level_compressor_enabled[MAX_BUSES]; // Under compressor_mutex. static constexpr float ref_level_dbfs = -14.0f; // Chosen so that we end up around 0 LU in practice. static constexpr float ref_level_lufs = -23.0f; // 0 LU, more or less by definition. StereoCompressor limiter; std::atomic limiter_threshold_dbfs{ref_level_dbfs + 4.0f}; // 4 dB. std::atomic limiter_enabled{true}; std::unique_ptr compressor[MAX_BUSES]; std::atomic compressor_threshold_dbfs[MAX_BUSES]; std::atomic compressor_enabled[MAX_BUSES]; // Note: The values here are not in dB. struct PeakHistory { float current_level = 0.0f; // Peak of the last frame. float historic_peak = 0.0f; // Highest peak since last reset; no falloff. float current_peak = 0.0f; // Current peak of the peak meter. float last_peak = 0.0f; float age_seconds = 0.0f; // Time since "last_peak" was set. }; PeakHistory peak_history[MAX_BUSES][2]; // Separate for each channel. Under audio_mutex. double final_makeup_gain = 1.0; // Under compressor_mutex. Read/write by the user. Note: Not in dB, we want the numeric precision so that we can change it slowly. bool final_makeup_gain_auto = true; // Under compressor_mutex. MappingMode current_mapping_mode; // Under audio_mutex. InputMapping input_mapping; // Under audio_mutex. std::atomic fader_volume_db[MAX_BUSES] {{ 0.0f }}; std::atomic mute[MAX_BUSES] {{ false }}; float last_fader_volume_db[MAX_BUSES] { 0.0f }; // Under audio_mutex. std::atomic eq_level_db[MAX_BUSES][NUM_EQ_BANDS] {{{ 0.0f }}}; float last_eq_level_db[MAX_BUSES][NUM_EQ_BANDS] {{ 0.0f }}; audio_level_callback_t audio_level_callback = nullptr; state_changed_callback_t state_changed_callback = nullptr; mutable std::mutex audio_measure_mutex; Ebu_r128_proc r128; // Under audio_measure_mutex. CorrelationMeasurer correlation; // Under audio_measure_mutex. Resampler peak_resampler; // Under audio_measure_mutex. std::atomic peak{0.0f}; }; extern AudioMixer *global_audio_mixer; #endif // !defined(_AUDIO_MIXER_H) nageru-1.4.2/benchmark_audio_mixer.cpp000066400000000000000000000114351301561706600200330ustar00rootroot00000000000000// Rather simplistic benchmark of AudioMixer. Sets up a simple mapping // with the default settings, feeds some white noise to the inputs and // runs a while. Useful for e.g. profiling. #include #include #include #include #include #include #include #include #include #include "audio_mixer.h" #include "db.h" #include "defs.h" #include "input_mapping.h" #include "resampling_queue.h" #include "timebase.h" #define NUM_BENCHMARK_CARDS 4 #define NUM_WARMUP_FRAMES 100 #define NUM_BENCHMARK_FRAMES 1000 #define NUM_TEST_FRAMES 10 #define NUM_CHANNELS 8 #define NUM_SAMPLES 1024 using namespace std; using namespace std::chrono; // 16-bit samples, white noise at full volume. uint8_t samples16[(NUM_SAMPLES * NUM_CHANNELS + 1024) * sizeof(uint16_t)]; // 24-bit samples, white noise at low volume (-48 dB). uint8_t samples24[(NUM_SAMPLES * NUM_CHANNELS + 1024) * 3]; static uint32_t seed = 1234; // We use our own instead of rand() to get deterministic behavior. // Quality doesn't really matter much. uint32_t lcgrand() { seed = seed * 1103515245u + 12345u; return seed; } void reset_lcgrand() { seed = 1234; } void callback(float level_lufs, float peak_db, std::vector bus_levels, float global_level_lufs, float range_low_lufs, float range_high_lufs, float final_makeup_gain_db, float correlation) { // Empty. } vector process_frame(unsigned frame_num, AudioMixer *mixer) { // Feed the inputs. for (unsigned card_index = 0; card_index < NUM_BENCHMARK_CARDS; ++card_index) { bmusb::AudioFormat audio_format; audio_format.bits_per_sample = card_index == 3 ? 24 : 16; audio_format.num_channels = NUM_CHANNELS; unsigned num_samples = NUM_SAMPLES + (lcgrand() % 9) - 5; bool ok = mixer->add_audio(DeviceSpec{InputSourceType::CAPTURE_CARD, card_index}, card_index == 3 ? samples24 : samples16, num_samples, audio_format, NUM_SAMPLES * TIMEBASE / OUTPUT_FREQUENCY); assert(ok); } double pts = double(frame_num) * NUM_SAMPLES / OUTPUT_FREQUENCY; return mixer->get_output(pts, NUM_SAMPLES, ResamplingQueue::ADJUST_RATE); } void init_mapping(AudioMixer *mixer) { InputMapping mapping; InputMapping::Bus bus1; bus1.device = DeviceSpec{InputSourceType::CAPTURE_CARD, 0}; bus1.source_channel[0] = 0; bus1.source_channel[1] = 1; mapping.buses.push_back(bus1); InputMapping::Bus bus2; bus2.device = DeviceSpec{InputSourceType::CAPTURE_CARD, 3}; bus2.source_channel[0] = 6; bus2.source_channel[1] = 4; mapping.buses.push_back(bus2); mixer->set_input_mapping(mapping); } void do_test(const char *filename) { AudioMixer mixer(NUM_BENCHMARK_CARDS); mixer.set_audio_level_callback(callback); init_mapping(&mixer); reset_lcgrand(); vector output; for (unsigned i = 0; i < NUM_TEST_FRAMES; ++i) { vector frame_output = process_frame(i, &mixer); output.insert(output.end(), frame_output.begin(), frame_output.end()); } FILE *fp = fopen(filename, "rb"); if (fp == nullptr) { fprintf(stderr, "%s not found, writing new reference.\n", filename); fp = fopen(filename, "wb"); fwrite(&output[0], output.size() * sizeof(float), 1, fp); fclose(fp); return; } vector ref; ref.resize(output.size()); fread(&ref[0], output.size() * sizeof(float), 1, fp); fclose(fp); float max_err = 0.0f, sum_sq_err = 0.0f; for (unsigned i = 0; i < output.size(); ++i) { float err = output[i] - ref[i]; max_err = max(max_err, fabs(err)); sum_sq_err += err * err; } printf("Largest error: %.6f (%+.1f dB)\n", max_err, to_db(max_err)); printf("RMS error: %+.1f dB\n", to_db(sqrt(sum_sq_err) / output.size())); } void do_benchmark() { AudioMixer mixer(NUM_BENCHMARK_CARDS); mixer.set_audio_level_callback(callback); init_mapping(&mixer); size_t out_samples = 0; reset_lcgrand(); steady_clock::time_point start, end; for (unsigned i = 0; i < NUM_WARMUP_FRAMES + NUM_BENCHMARK_FRAMES; ++i) { if (i == NUM_WARMUP_FRAMES) { start = steady_clock::now(); } vector output = process_frame(i, &mixer); if (i >= NUM_WARMUP_FRAMES) { out_samples += output.size(); } } end = steady_clock::now(); double elapsed = duration(end - start).count(); double simulated = double(out_samples) / (OUTPUT_FREQUENCY * 2); printf("%ld samples produced in %.1f ms (%.1f%% CPU, %.1fx realtime).\n", out_samples, elapsed * 1e3, 100.0 * elapsed / simulated, simulated / elapsed); } int main(int argc, char **argv) { for (unsigned i = 0; i < NUM_SAMPLES * NUM_CHANNELS + 1024; ++i) { samples16[i * 2] = lcgrand() & 0xff; samples16[i * 2 + 1] = lcgrand() & 0xff; samples24[i * 3] = lcgrand() & 0xff; samples24[i * 3 + 1] = lcgrand() & 0xff; samples24[i * 3 + 2] = 0; } if (argc == 2) { do_test(argv[1]); } do_benchmark(); } nageru-1.4.2/bg.jpeg000066400000000000000000003443401301561706600142530ustar00rootroot00000000000000JFIFC       C " H !1AQ"aq2B#R3b$Cr4SDcsT/!1A"Q2BaRq#3 ?|C*&K~N럙tm_]]_U\)4.]m€se`EIim (kzR-7YLU|H oIC6Ma;۴ꥮ+vE2z.\ l;xU9FI,WlScZlqQ[! qt6 j{ٰAb+#AT>p/Ķo'KZ:*Z918Tl2DwTI=9rjH4'$Cr.pTJrBjA7U6DHQ#WGB((Q'Ks#%CZtgEN/; ={dCq'@E/cs4Ut#헹v*HmQ&0MDɖfHY(p[(t fëj!e&v"apB?S}WdX.瀗ͯB0$'g\es=F3=emz,u2iI^P:& C IV2d-l#LHZe13{v^#5#S*%_kl#GloTKz-.^čFmuφU:dč*~\:gDG6]s2LTh$HAh_?ZhIl!}N0P=K!dT r$r!VR#$w$Ac򴨚g{-\HU.<@t]  }XJЕvOU_S.fW't~T!.~i[BNa \ T|J?J52_ELf ѷ2K!,/2Ka$uLX[09n)*OfEOU0>jF ] 5Nc 7طr|j k?%W54595p;49}[~J6iJ#ύ+rWZ1م#tƦSlm3o:!GS7:CEԾke{_4.tW&q= >L1wGf1 Ak^(mPԝYox"*@kt!5*>l:UHپQAEO[_/åjcaq-ӼC$|UDyI/gDhCx0y[Co*OA@yCnQU:'<2&櫔 "" 7UuT:?~s `?tMCrI$zQLbWIn< #)nV#[7] kzg|8/ AEËʢPXtޛQXo t?'m.amM0s-g}Oةoܺ&P.;)|-/3Ue?esuFOnz =w *JbiN57DGC!٧+=E' i: mD7H)lR;ltsn1a@KJ`9hi)j8R `-# #535#FZ:x 5 b!Kqpk٨d8V,klnxW ) tDG TlU!lIle+G*ַ(,tV|Dс KqDΆPǃt$)A3[`1eϔ-gI/ǁn E9ɲ[Wlx>SJm(I}gOd8XǵHX}-!4v ce؞L(,9B;e[?M0Ro%A"}K9&)Tmʩ# U=lH EOPzcn oLg Y=k!0,lok֘&kRLNn2j42:OAnAŸ {B)ݴ+8Wt_͐vm5=*dc¥w=6UHۭr!$dr9R#E lI3dʲ-.-uNE#'pZU.64*u9EBq͊› _mY41"ᔜܥD$'C`<9BgQ)hh@蟴 ǥ7i d%F.@:9+9L`U.hJvP"Sk8葃-]S%pʬLnGu\seS,{nSK_3B7]#qPLD~ߦ8NC6=J㥓aDO9^F9R9L~hmʛh5Q Qr V2DqȽyDB-ՠHG%Ll@=+3'=+ E)bR,dɦ[U!>f*<&}Rhɒ;$YklRIfmeԧ=^ꃗRh 'SZv|m7)eG 2TN!?ѳTQ]}V6]JrwL@=MGUDE٣ıF\肓]فAEF NR)3o$>VG vtT-E2bit/D29AtHSfsbD:+[bBݰfJQ"@{&3" BIȦŵZLH[/^!6\-bvԀm6+tS lq1#6C`0Ve{m1#6TE _U25#KI^/uc 9蘑d K(%XUdGŗ](oP|ET@8_(o% ) v!E#aeoC}Lk${jhI3w+VrJ?2d^a>pz5 B2 6knZ,lJ>p.L-ID3A?mQ Z \ݧrm : oVa`,Hْ>+I-=M/eIB]kc4XYDߣUy@{#UilQ?͒}DJ81JF{[+oG3{Q;eWFbB&ܧ&tR~E*f$:T^2SeDr$|=0[d98^[c{/hex')`y藵"7.5*k+ 4L%dC+$iG#jal<׷wJ$c{g]7;VҩY^_G{^ \pfeS^oFܭЊd7|7rFUd'NB z2Df:Oa- -up i[ꯖ"IPcz[)ɀe 7<)l":Vmvx*@6T@GmREQa6Sk/|]rc?n,|G)<3<'UX8:ksLO$ 4$z&/ˏWYOfu(y7r-l4q~SB]$.,;,l4.mLQazZ3-?3ZE;uF4[5eqGZ j@r%$_5W |h DګGk~85cͨTFYE;GYA$sϢ02~-?e6}a{B^& o.Io꫌?f֯pErfh>I|V副Y%_ݐ*''At49(މOBSDGE#ɬt&:n#LBtllt@rltDNF! tpC qfthS6Sۛ+YӁX"kyL@;lٲ`tSXt&}S1OEh,:ālt 3 Mh=U@l` p 'mMH͕rK+C{Yp1 6Ch+V41IVw-SR;eYpPS[=\b LHݠCQQ+@P7%y35#oSQ賓]qj765dr{IA#WE^ Y&Vn~}U2s=4[DNQ}ntXZ[M;Tۧ7LHo ߥ^IRlS3&܂QE27/'ьCD}mK64{&lcE[rHZyH#;G2fS'=QQґubo"B=Ot&1WE: 4LR2$'ܫ6hVn>I}4CU:m>6n=,,*o4 锐GOͯFg[Q4@Җ73/dclL Y%G6,=޸\|`m ےtTWYD~)zs4 0hz`us,}fl6ǰ\Ooe19oQ.#ϮjWck@TGӚa#AWuF+Wmjj>,aa9|s@^ [kQ.}E0`}j,. ZG5֖KQ+P9JGQPO)[IfYWM<ϖG3ʛJyKn,#H\ef`.o#vU*AX;eL16d͕ A\X:`EISll e|4S6 NDʚ쮤kyE@1W`&]Qi 't6"(<斊Ǎc\y4x6_KBaqtb&<^Tqb 6—O֊?Z7q6tc>kh`h0!#bh|-Q-+VNX \}ۃ:-Elf([ȝ`[ZtbXj)mܤΊFL7X[aV MMIl l| }~ r_ԬvʤI|]{yWh˃` 3yh;Ӫau0V"xxu.͚ m,6e+J&!룳;!*W&6 JEKMHVEA=me=rU}p:6]gtj uT$fEUW_ Mj+0>wm U'45ߩc~^m.S^E=c^NLT|` t3Kp lM3: ]nW)lCNcҵI$t2l TdtX? Qwg;mj*Z vj슾uɁK_4y[j1 R "gO3/F^O pI Z~mT/w FG~fRuS#$jd"{,n ~wZ{jm62/Eџ,OC*H6T]dq~KhnB5_HqQWQÖKQSVxkY)fǍUʍmtHu}23\lNX5UG1}_u1rRgP o&]fy!U3+g/ysrnWݚ{GD6SV$naceC/:Lu NqFU yJ|) Pyʱ`25#x@fyJN!hXItG{T=͠`mtt{MR+e#kB:38MG{STD6[u{(`~="&eͿW6 X`0)ʇꊇE$B1TV7[&)@(fbXUZ7AK$ccf1\3 cZ:uB[+۰FGH7JTzU$.TL̈4tz5T3FHchoZԣR3K/%H8۰TMZts7H{4"%. r0CԪs)'4]9Qk3G_L!3Tx!(f~)BO\X}j?+Ûfܮ#[i4iCj{v?BU]N&HG Uz%UNEҹ#chEX-7[[:CVj$$|' A_Ca}jbvq3&zx|DFPכ$E~w}~9WF#qp?)[9?;Rݶ_3;$ҾGwqWeNGPZJtW)JoRyſRA=W E7L6NjH6.l'npٓ=jaV͕6g];V>m=T<*buxi&tK!h4 j09azplt. p Ix"y6`2?[Oj…N+bt,Ki%n v-%)sydt :vGC,K.VD Z|\tzX})4K`38q`E(*Y9qS̤Z`cq 0*DbD]#lI*%VvHE={of\GAe7 0AU>h>0;B⻣WCll+`'&.1]@Hʍ:.Ypɇ8 RZǻHz`+bѼ<,vmdxEg$` ,RRi,8 ,S}B x "C^Njr"ι3M%XsKA [: Hf9Wy?ł4}B:;԰9f5hMǧ|VͺAd;^G ;j\Sp;¹ ρζWi!>L{Kv󄚢FjLNv ;{> 4MtdHKhÛ~m4| ~[ n 7 \{mх&S6mpɜ:[dph쌎<]Y̥./ g0Hk[ʪqE :j9g 5/Z~Uԁ>.dlcR#ZCͰ(ڊ#7G ֒GhP%u:A>tXऔ n5*s,q&wwOG\ TkE^_=.(i{ORxG8]rًSm;I yܤ>* HU죑܄5a~Y9u_%_: VI= il?N8 θL s3:4ac{a14]e$ɩlԁ͸i!#,P.R&nЎi >Dll 4wL.>LTc&Vvl)XGLL{Rݏ<9~.#(9;!6UTs(tQ~LALZ;۹%Ll .( lLc`Ҭ jFr.HWajG#il d;TI^H\$ ;F0I*Eo@4])p{.=ێq&;$@_nR TԎHiXn3uez)7{JbFlD.ы] t!5H|2x S Oatz{3')#=$GqN 廿 7 hM6gH%l(@MH^- URhENz) 1%5# N^U%>Z$ꋋN;K#Lyκ*=wp& hִubv +LyyV#Xִq`Qh'(>Wm{.i3AMZ3V B&$i44R[#%,uLl,th='Q7+?4lpp[0,7Fss=Yo#d4m.Zۏ#/$4B펙 f ;@cTrP6tw(vKu7MuƱ7Tt lUtp]Eꕎ:}vi~suU=VWyf;fz=Zm+u$"ol{[3Z;[M+Æͼj2!x9:W|.5-&mi-&?J[Ƌ }O-M! : Ep``z/+STLQ\o F)y|$TDu; F&_Q0= Zly*mx "`˻{wLԬ}ݤTد4SCKgË](nsw_6lUT7/y+lwu;/]`"āGz4 g^9]ZgE.ܮes4c'OUD!4%I EDNi ɍ,TuAyS')^Pf2|*$!Mcp0\~)!q'*򱊪iGRp,ADv-MIgaTk@{ABe b3F?xEQZo<`Pм#hM3sնHyj:ͽSK1HZaD[Yq>ѱ^FkRFާF:1p[=Ĉnn|Ul2c".Jm^s˅jg%-!%nF(:+{|K­ i(Ц:6RkL rn]9̤YgB_f\!r)sd$yb^U\{5gkD mr9:fbȻWY>wi5q+:@*NгZd2sw\3trѤK?oTշda|GD፨l7рw Y3ٳ^Fß茆['kKUe-d.C!૎[Яfį4UH&uL#Ӕܧ[" ^ !P̚֊8 '|Ѱࠜ6;(eD:>aW6AlXx1ݬR& {- 0vS]py+F_*.S4 INgwuqˍ@RO' rTG;) ٺjGISnΊTJ/ksx6* U.MH#H{$k DǧO!Se@^xh3EiJcx`Dԅ|QiS<hBFY5!O?I-Ϫ>47{Y0+@ ]ι=tO#e gx~Pu)u6 ~ROr}(7WCEUQN#9#s˖8>jC}:->"=u(;%Dq6 BwTH?)f!=f@m"1i +MtW͛SLqOn>$5QV@Rik聟Qq~)eXIi.wmI@1!@UjT4k3?dD=vW8;\Put83-nYP gc9>OBU?cz~ඍo_sק'W=qqw.&UNP7,KR8*$,\5Ikwwme[\ʹ)t>b_ҎxS;6u+^n=0KOcbe?fn2G嬿&N^ZG3UHM;r7&LqrCהLL+DS5il[F״>/b4; C-?ӹb{^BYwrʊ:{"swQRYZP8cv}k|)pP:UW{~nS0+k4u:xT7WvzcjO >$%Ɔ4(~ZojM}}Xuwfuj gntwʹLEnHպdx ZJ:Xr}6`KK2PMs&a#<<_Wǿ bcv\kOʹY >>4eClQQ:gDnG[o.ib-τ.fM=J0R\Qe&t \b&; `!hD jh23YFH<76/t|Zg1-T>i5j&rR9n ]z[NkLѐ-o3gTYZxS=pJ[Sm& ExEcpqad WΏpKj B&UL6j8Z^6 $G &3{ۅLl7o 1k/Sa5m'el#K@7!pBݸnjci[kGD+p]%P2 G>^.ʬ0,g;J7EFxbqu{)wMHDu+6w蘅HVttIT %B~ */?#bB: "1.}S`s`4q(Kض5Ls5Dm5!mlMn ưW|#j`!:dU.ۢflDCUNFȎz9K `:nv\>e5/jlr&#u\oUBk:lsOP\NMЏsUMX]p{MLhZ|@PIS|vgę쉟`Tpݴg ~l/,P^㵭sSP]ޥ}?DרG'J g첷SKhly_esT7~%%~/CNkmpijx DS:KһA&QΈ; D͊ tn7n!V!jՉ_ 'Y@r GKꪒ>DoU=ρW8 q' sE{j$V=#ƪ~dIjOWXmq`ϺCۀGKv? v/ .YO>=,/  hEٌpe o`Qxs=,P^&yq{|ERS{@V2ZsT/qJcO.v˱ii vG,OFoQ[Q߅=Ytc`RS[ECJ{]K l2- @Ùu n&[% V6ِvȁgD-RXm Yv)*/4CiQBAFZQ b`P 6CnVm+tן1hhB4ۭ@kEO*Alدn] Bˡ O[se@ AWvYf!+C<ݿU׃2t  -{ :-^;H+]E3KE?ir:5`9f'6H;S[ cm*{pEE2Xf}mCQu,,$g)6PgrTEr3w#b\a7T#q+X,kk#$7x(VDH\QЊaDCe ~#G+ϧl5+iw &S\|tOK#XZؚh;H;̰ͰYEQ[q3uO<ʱ~KB -X_I0dkNל,$N<lG<].CL)YA*0#l {@6U-]^ww3Z4cړEdYh9wT]*DO_@ͭ4`:UT_|mƑ܈пKNLfx#'&'c+L\)ӂ㝂-]+*cppt=GUFGԬGusLN>^oK'tueGR;  ;Ry7+隖'c#wM~S2*~1{`4۹]v1Nih Jj:}0mu8Mh͕fq1tf/~W>3[R)[RǖuA ۷_ HuUc0dbF NiDL?TLq|ˑ6,U`ftlΉ1l@Fɋ4~3- ƎJ)[STn5 G)%,8yJ8'U- EVN#_@qfu}94Jˏٔ;YC %1 -/Ǣ%T,px9“̉C>C{""7U7q쯉3@]!/ms@>208Z+9xfKs},q>Bl8GӼ5#7 &u\\ A$a1C36 YLw6P] #d_{ׅ,(rh^CSlo',|GF.hNcȐ:Y!\8wtlto?%å@Gæ[ U{`$&)EGLYE6 nAR` BGOW0,kut}tT?)L4dL:eeAF@SR5C֩4tknA_-'MHroA7Xh,fj%L$ tYk!:Pk,}]%q\7+Q6ϙ1 kȉZ̵wY.:~R9u78ٗ'T]U/$ךi6:ȃ}tO/4W.L%5i 3o4EO9YC _,w'cvzQϠUy>6-ǩXkj7sԪwɺ/b~cVW`>i$Kk1Ieulq%C'J d)}[ReՌ>j?#܇X雾f:`Ux(! A[mR|8j$lM\y_dWeW?Io,;] >paS[=K$:GwqC>L\]/yckzaRoAIRTH}lbJ-V,yG|[cTo@@6-gw 7\ߊZ6(;-{]1x~6]ܛ0/$ɝy͢1L?`=ZM n*x6O~V2>;;'Q6&0д(Ǘi!]]jFnA dBklK.}h⥧w`Ia[eRK3DqW"#^$7s^y%nPwe^?$peEP_=Q@0_BaNܥS,_S"ʘ"~8.2TрQmeq%>ʶ\:Q( leLIsvcR. gٽm8䫚Mб.d@K#ԁ[F ѷQ cFU͍6 ԃZRrЍ.Ecq@jk(d EGSD7EKA$Xc@U!mMi_QwdJ[;g O+]kiu^LPi -׶;&eMG&Y=ldDtraYM6LP,SŪ`u 'ɖ־V7Q15?ᄵL$0Ӈpe`g( ]69\ )]Dq|6ɭXʍ@Bϲ:^bBWDkI+77CP,f(K;/G/72ٚ$v9YY }Tt{$kz S9`n0Oޙ%=3U~# KkGwR3ZC%2X<ϑ''~Ɨa-0Iu:L70p[kQ*'3Ѝ7V2g0h#o"ckp|U#M1kI62 -#% ћy#H_ZYP`YKi c=@Q6=`ɬŕ,fz]f5733+vaimA@VQ58zp+x!/ct~cU~L~?W@M7zc&$DgI_UJ\8aYjVI>yк'X0RӸpu!s\0BTay8R\z3ύ{De0ǀR0R;wdd3:--{)uj(=8PTΐ}4.pЇ-L*FEPI7\*Ɩ;':kUGq h M3f~Jɣ?UŁm;pB=mϲz5"F=/emk oZ2GHMD6"FuhϺtޚ(éVt+~OVϟ@S8H_f+c@kN}[pomR|:|r &;FMsC脖4rjË/S@1y pq.rRh &c4Z4e7MH_w>kڕw%2T8(hU:&#w%U<{/<lIJXSFk xP5zp;[2VBu]ufrTM)qe1E;@3GUUGIQ|pDS8zE{Qs̒V(W9$*038ngHI\6 ԣKuQѓr7#rJSWchv0CY䥤' =2UR (c^)w8\I9Ckf뽊# \fWd:OGzd @S}k6/sMʦa.{6¾-<"GRI6 r=ɺgU;kqLA:YHzyScK NJ.ce) ]1 UZ-njLvoad|0OUxz@FFr8% -D7[' Vx ְRJDÜ1r׽v8KQSw CJX$wtLlwU{ W2 :]-%q7( dGuDFt[#k~se4QqcG65-s#n V2T2$L}l$Pg*5In+C]TŅ}=Frʝi! $5(k'$ `<γ1EG$-Jڗ{')v \q_/, PPuu*JH]hqފVq"LDU+޹\uU!*mPǜR?D``ހl 7d>v O t-6W2_!b}1ҸhbiR4(xZocHIbe?ާ bJq$[T)31WnoE^?rcƓFdi^Ugan\}sl$ $?F X&qA+_#DBF_;i o|?3dϷSק\5-=f®~fqu_lknetH}+ai7|FM7.7UGR54B(<ד|mS6`ʌwڱdd %͍ [r)kYw.m5ʽb^؊r"kD\ b{fhAV2;v}?Õ쬤D1 khY+]{U fݯ ,,ѯ!i~4O`m {׏0}U煏 RـpR||"O/~PH]c2ǜ :A aZ{eyp63TxODzgACmBЦ\mlܘ-ֳ G]so;$J"^ci7acbecA# @k%Rf\JlamWvEJ[CwFtu9Nig'=^ ,0RJuN*+Ð -P46sD[U[ʙ3HK.+O_| |?;heќ4/&R- @RP6Ng5zCGrCD@2:K 5:q٫ )(mܦ pPm <"UDbĐ3CۆOlF2AUH.Gp4gL T|$U02gB==X5lj(V@cM`ŢU8h<>e/CGKd oiaL{aPyLPr2hUdh=Ja颱MǨ4Z9jST$w/CIN?F9#+U-P_QMH]3!ĵ~Aͨ>`>K*6tdO143`~3뻁M@ɨ);ѩɓ F_@˩< Iij;JlV5%kI˝!@L 1J*5Z{'$C&!ӅK=u->fs^+&E8 Y*ܸWhp%}NQR+' ́}K(]o@|oPJJ.C`F F¥'Xz!1/_clO4:Bq{ 6m=EUۢ,e1"J[GO, E>(ţ`RY98TL'ިK*\[Xz/mM;I!ˊ fN4 _^LY/\֛tULeu)〰$l݂puiņz~0T]()CŒ1#a*8cƨ 2*gde$sWEJ9#(ةShFEO [FSG`|t4d)D8ܥ;2_Xgca쪒ۛ;lbZ 1ԁWE.K5VmJy}V6cRZ7}II3aqO1٧oRi[1>4 v#Kʢp$uĥ6CAh9꩝ۤ#p2<-Fq62W Y\ 6B;,U pnT`p=Pʐ $ZGe7*@ߪ(B_[il0e1M0AV6p0ᄟdS4~5DtgŘơwض:f=Qi}[GUKl!.*ߋc?QI56\-ZyǢ8z}Ukmq1l]XӾLGi-@h')>-uMS8hf*Q'wհ 1|جF+K_ |"U6fA[; l-Ϧ+1꩜3"A}[<򥎨BD9w*zGXWXiߴz 8|C+ʊkmaoTb.)LJ\$a#.4$ɒmK~ܡjh#N/|'_3pBb}RZ2Ӽ4\rj&Xȩ> 0AZ5PFw4 zeed~ Dђd,`o--_V)ZZ]u)E4F3g# [ N]ʱRXL)FxC[qetTŎQ!~=$*;6 DxDJbqusHʡ.ϲ&]4 :x,'DփOu<ťS[[ex3O>)N@ZE2鵬)>5خT08errNрҘeb2kTHŜܤa+ΨrxcHid#čx{A;!h\NK t^q~ʰEB!u(ijZ|3 PS '\}Wd>Ft+=Y%U5lqr]Jm;)%N<˜5ʐajC74_Q` ^CRCkyrՎbU9FJK_TrKq*}W G yR`{Q諞MVUzDdV}2CZJ1gaog@K_5Mv>1$Y%Ksݯ~%HS=h$R鏠Q+sԣ[FSE}'BO1 iMm ]-ָi$>@豷o5"k<mT&=הƟDey-\6E 6/M{&$dvM){vVZE5 4ӌgX{&lmV#)wTJ:X}:,5 {g~.>r@`JR:He˴~Q L`#{Vm+Mn.]rNrE, '7SN9,)/[HV7BK{ *S*zg,] n)SѾ׵Zj;X)G2)ol&ю!9`H.?=Ylc4##tq9vCq 8ZEGmuoƎ<0}\tgӪ!,R5ikeMZ]IPTIR=(d[(g;HMGUZMe֚J'%Rgq^"fKnGtGJ~߲*=:Gd6[E`6I3.}Ϻ~>w ^rŶ7uS@u#R x mX]Fl[A%x &Yi>Eo)|TЦ̈́ڨ>>o_bȄ=I|Sφ0* }*G Ԥ0N{7+1=Ԩ_p j*g+ ZP|nqO'Ԣѥ285D@> \熵3Umr@:,l>ކ4ZysS_kUP'湭#h)c,vA!yn jKk+camtاbGl=]24ͽ ]={."<DTgv\kZ8YDÄl<*i6E2HcF eEYWMR:evqBMlw -7엒$-sɔ)u\d]ιD8؋XX5-W꠱w[:[3OL,VE_,Ƶsl68w E}= ^D{#B%1i7Ѐ }dёFƀl{G9zd\ X\sGlb]עĄWdIS'x\,}כo) |0TAo*@`*#8(`ۣ(9@UF4UWU5>HDH * 7%ÛeT+hm&1o%ic^`b7sET M0ܘWMN_ h=OȸFDT y#'-ĺ- kmŮȯi3Q.dEGA!2v̉JB,XaQQ\R٫v~ ]B/D. 96\m{yܳjprv4V%y_f~ #ze)Ze[tgI9'~RRUq1pIJN,uLy|-O1> M(&5h>!wY*(oi62/5!9\&g46S,Iq(i?҈˃e5Iby[v()uOH6_I%vB܎( $5s MH1-W]!%>y$8ЮhaɰMHR]:򻁓SkpD&$ to=~ʇIsʬRZ@7uZZ #H=s7QzZQywk3J6g4] '(J+uF~;Ps$Ͳ)w#j:qV$Jvmp@E^W6V VQÅ&@0Q); BI_6m,?ŸFeLR. ipdtc+A:-1 mdќѢeRC<Ͱ촔F.-47;S-&zF!R|'"Bϔ0>6ܢDpE8\tCU#́ *I'/MFn*9FH蹁.p ҋr X\U[U:K( Ic:EP/[ׄx^^@|ײlɧv,DrnxRtRؔH^qBo1qH6!s&T7 ?d}.KTcܩ)aԺwOK&L7dwo!$d|PaϪ& 7;!TlT.zn{#bf*QԥDEd1d qPQ20l=-Jr]I7Ytv(wotlǐk`Zq~гS/}>'4<D`|Ty6tڍT/ !@tg(I;Mɺs(Z15*_T .;HZ˭%rBKm+ i[f5RC#XlG BJ@d T;GJɿ*?Z5vtwpUwE؏U z;}WD\G~$lk@P}[G%2W!eԛ|t:}p*+M~5E[2plɨF\:q~ܤ~.w6O[fmaKPWp&;5duο7*b6~/m f^-0rs%X*rdauE%C \I.vSY쎇Dz؅{NYi~Lh-|m)Qp_piQD*NEMVVU [d1 Ŭ7S5BV62m6rV{t4*K"y ]FxXZG~Q#Alt,msJ֩2zkYH*`h,HYj27kn ȸ˒DI\|kV.HD4Z㑌*_&ǝۂ[ujꮎN%QP/ !ayG ?4DIkT]cCث6i<Z4EDyyC62y%O"6(ZqKEl "6=YUrQrSxoETK<3ZY Vn!{x;?AYcf#VCOcnz/e;a췕:pp ͦ$,$dGjܵwkE˖)Dn.S`aQ@R4nX8L hkqx*|VNZ48 + N?6rcB}`Y\ƴ+GQ#cS `Noю T+8ߢkG}Q-1ʉ)+d!2 hm9LU-**NI!<8ִ"~0x# }f;6 ZH i{ypqtPh͎)yz-8KZ:7oڗbqw~G=nƃwVx. =V?QkZTh*ݿRpmmd[[Ve:)<"j["y(JM$ROqMC`Y)_ UDwԭ%hq˽PGzt4~&؊(X\6=],tĒCGrɪ4 ).NsTQܠm/60di6NY\J D3xXLqMfUXi4q5H_m̈́-e<&z#q[$}e5"N -w&O ,%HE6n$*Yv.S'}ٶ;N7L)x{h$jjQyzHW\F}2=|)%ԩqG}76Ȓ + `aOpP'ԣ &$ȗbvODŧr4 2<#bɵ&)Y۸P鶰dV'HǑkqMa)IT:F0p>eE5F3`m2e#.5VqP\Ħ$, |ʧռ[.uDwkq7' *̞!DEAGs%*&oU$)P2rd$FR^A*R&#t[<.SR8-{wzٓtZ\;.oFOO3GwN:bs2*FynQэ$tw FC@M 4zQ7\Y= ^;k]-p߁a2{fr)XؘL(hyϢ-1`S՞b zF8𩒱p*7{(>c$b|^!ikN&y4j1F0nkmY*uJg֜UuphUkmϟqC%T;܄Ĥvu(T|B{H8S)%\-{*%u Gt8: !1QEޡck7@PKh/~l C粸84uU#MDN&p FNZΔeF%lb\`]a>MHx2PJ%M( [)sI)\}cIy$'Tc#c +$kF@OWE,sT@s/gLsSj nR gq*f?!4'CLJ'6FBnT ᑒtM9`>ր6<̴:ELt ichjJy,E0F@MYG} Ky_ضQц5\KrM'U6݊t.m+G*{}&ReKC){iAdO6 M3xLa°Ł) !y՘>y.T\"h:S4'T +tئP5jc/}$ycxr28TjZttp9E5s ~ J'Q75$c^7Dϊnx苏lۨPdK$vpn#< }^Waw(,ux51,w_ꬔ8qAG<| 6XSiABOI*68% ">+;@Y0tFM_nrZVnOv]L;yW\[*ȽϞk LS RĬp-,oS^^\>xy8U%1w9&BHV]b؉Ԯh]윾, ܏e 𽊛Lނ舩l6v?dDT@8CpOZoo2<#㤲1jXKkemWL>gtMGxué%xA~̃ 8]X^H~A6"Qdzp#D_!-a>623Fѫ>R)[(nßD]%=-=u֋6z*c01(6iᛈ<9㔶j q|Vf=i! =~76]!u$6u] o.H_CScrd _0 wU^0lTn{ P6!~P_#"kUUph)yI C?MFw{5d[<%+9Nփ33d=eC'ȿ"td5W5̒uHJ6ۃw_YL`(tS-`܅g7FD`ܶY&X6đ;[[YX DSz0#|ײ! }}kK$ϝۭ|Ǿp^;Dz Nߒ p윤?#KKUu6Y۱O)X)$:`떷 }z[9a5R8 LB%t2/briZ{N䆥 JC&তUPM\QAdvXJ'f<ݎFzI3pp)b*0 tͰopJc.GP[P%d-~{o J*̠mij+i]7$\{GgOvOu``(ߵ'舏J{so@HאSO+cC=Ldb[d <"҉> 1:[O @&̈dt:P}ӸX\ ͂j3bȴX#"pXiT` 3 TcPuȽFH#_Uˁm-$J`]RdF}S8`eWb @֏3FCLAc-Qt/\w8L#eӄz8AF6ŒYM%gEA 6;A=ʽUGK|{WE)r*%ԣ,_G ;&}o@Sg}$kj@@G+-6\vwi&Su >VUZعˏ_&4ٛ\;MRl (iBAo!U녚KdW~!J}W +5 ߈m$ 95N~˽F!UnyC 1lA6^ZE] GekGd3dhܭBal w9\PUtcz |r2CXkbg'Jzk%tt{@ d$Z}D]AC+]9=R8Sj]&6>% 7 3.R/Z qMhɃ!k}ymH:f6tKX o 2x%2F0}  槕Dz{@B%vdՔcjRe-lV>W2ӄɴ]Z~ Ŷ.m1&HFz5ML@F|+|E΅f)<5w;^}XCT,\&6[~g}Ocn2t5"V{UVM6{ YL=Ǯ&KfqrȪa/PU'$\&Q$k‰bjt4q[@B—E g'ٵA+ n88S]NbgX[}~X 92BË^ ïe vGUe;Ŷ[*#OLL:6=j!8.FMϐ&} k<"%?eDl8ZZz cs]\=Q8Yl=wxڊGn}U>[T}J{i]v-kkr4nNdf.Rw ?NEnVp*iopi[)w hŭGFo.%yju{x/)muSp}!4Rs]Y։ѮUYrdJ-%k59ww#py֠x7S{H̢VXfWݰ{Cy_lmɤu吸!-͇ʈ(M))"`܎gbt Kl;stn~}סn~8dgN1(4$h" Qp~I rz*c*ǖq=Ǻ9"{i j8gz?">Ik͕[{Vvw9[JFۂneyk i?e("ds-j8 ~c1 "&c Z z".?ŧŠ;&Lzit쵖\L-m(DQ?0jbV}O?(cg"]¦Jb0=7nLo@=_4l ^%G)|P縜Q/(|QI7?Q.\)8U[ U(Fj5"Ny%@fK" I2TI\Q.$vOU(l]yFx $F+ º:g8ᩉ)#mm\4y@@v:W.*!%5M827Ӓbc{)HhYr&|vs8VJvw(-44Z&l r}1ܧ4}:.Nj]EBH5EoUBBz[xDhV|M.+r#LrCIO%|Q܃t^bev.RdMoKSvyGsUSn旗C>bARl]3IUЬ"DK=8]`LU G*|{UUw\2sCR-U%tuT:kQ-Ny IQ/cN:NiX&v;=:?2I\s8 G_Sdrg*(jCF,m5@a#.\leA$T5Ǝk*cה{?]0ZkB=_UR1\\Np(F]}VZ nv dmiLEs)mmuۡ킈9*_ W^.}TMrml߆|ܡyϲܗ"l[aʢJn,2y#%j *]r_*9@ٺ4X T^S&zNW &E-qqͬvy*%OM6VC&cTB w 뇶AS'd`wK]umq(\ 7v!WMyZxʗ:]bjl4AwM969X]&п99 YE> GzzfE{ ;!.۱ %uc=913$m`#[- drb00\p,S&}B!PEM{*R8; EF\FEʣz&b=|V<] l3ʦZKlSz(:a]ʬ5Zy=.Sj+$psMNdh nIDqxyR>_$-j(Z<i螣'쌵- L&b1rnTY wG2+\靀{"fYDŽz.? A^O)Ș/pUcTxv gukZMgf<;oﲠHQI/q% Ҍt둥XoQ@eմ?6dsM&-hAdlkߕeT sM>s! >S)ZXD6"66<\)Vm1O AIWol7&S- 3eoAUK)$`%oٌǕ!=R #)aY=Cv$ Pc;8[; au5wWSZ^ުJjjfk5nZH ڋJ%{I?W8@[d4׻fK[_&C$Tz*( K!m9̀fTֺwu l_ .t t-dIPqo2zЦLHz(ykv6]3ex0u V D@Z +e\)@{@M2L cuy@ꉧy>^ct0fEUTm7+aȓ{vUh]`HەoV0ÐVIФq$28+UJͅ1e}U8 CkIS5spS128gŢ1Ac]27{Z%S1$`p*Q4U0SrH 4AI<0Ӧ3^>6m ᐒz8HP@lgv):UqY]krc*Ԡj*ͺowuAj 6Y2y(:j/@ϩ؜#5Ỿ +3>!s H3yU |{tĕS\F3M>ddmůrnY.$e[U:Q#q |-3P\7YV:L4M 8 Bf+r3wۺFBFF$nBZ+HyV8Ec\<;F[,RY /գΩzb1 };M<=D2E#/lwI0-prX ev C1TUK3P8 = )QpIa9'~ճN^@!ݐw~DV9N Z"9/{& 6#\ٯNR 0nNGD}=Cdnt#$ GSɩ ւdnFw dM=,re\ mkY:gA7TmlAr_B0uMU3'*Zf~MfN@_Hs[pR.1so蘐#Z(rWLZB9͒6P놶uTNXM*k|ɐ] `A%!lnK㵣NH[4%qO)kE|L)+ˈ%MRdxϲFu'j٥9q UUpF=Seh,쌤$RU:&Ȫ N"U9  -=/ScO~:ƞ٣.<RQs]α YSSF=_ A \21²w84\ё̸kcDkgZa:0{LYo|-,"F J3GGOLH\YUrWRǡJ YKM/˹ܕòwT]I\\{/Jv۪q#]ܫq![;}=i>ž[3nLYGF]g9Kc3+{7Ld0E'Ee }ww#ač׀%6c)vH)6=0= *2sgkp&S-Sq訒=s#.:Cijk*~A.%f^ R>Ϥl5Xbdv[7+YUT_N@?A~ٍ$ʒbfU3\EJ^]$=6i-rr2ۆSctoDҿmِ+@Si$fu=ƱS !M(:Ԩu!nVĖ7˕~Xp @T:).u^*Tڵ" t62.\0zA.UFK .Htx~Oyϰ5"HzA E[O+t:QaqPˍ7]MA AVa4u5YsZ \}VIk)2 c)'O426kmcIyaAB[ѕ`to!ƤSHm6{_!rPZ=-OŚ8mn}X+%;d/;#~?a.ë'e54^gv1zu7y sg*nK>o.j|gWoy:14rS]UU2So#*&i.6 7@sJ]R\&XZ CpiyRccUUUJ*Eq{tǢeYx/MlUψ֋R矺 ePtR^-I<Ъd/uY } 땢".)m/eߘLu+v [nU2'tgE tD6]p6z68Kι?u1TْU)J뛃2wwB8Q?)v3hSpd! ѷ>,rK8iF3^IQUtweIk P=5|E0C+6ċsvWrBSR6zmatQ\`.C ;MtI;(14m/hpxLGQ2_>636ga\,loUisN盥2S'ZCSLy^Np1]\iPFZZZ_> S'i>}K&JZo M=K}#$7[G5*OV;+7_HAݴGYǐZbWi.Ŏi7+\f,eJmS=.h#GX} stTtw5\3k{|Z\ ̮{ɁuBhh6dw>ZL276轺̈y)EGOn:pm<}S;X+IW `3#u oz*d\LK@.>U^G rU] FQ&O6-aɻ`ާ֪ ,yRUٵԼY7h:7%fQW1w;rnVrZ=nN; y)G00ˍw6cn7alfŲ=el` %uL@f=9^*悫܅cGa\_;!4lAފִ.q,MDZ9r g HO_J|9U:bI*ry[D.rm~Wp9+tck npD.nG[\="E%U(chOSh<16cfv+Sf@'(Z M%᱁SM0 Jq&F-QX7\78 }J&fcS`L= `.lgwmSB8V;ZtMl4Rk-*z\Ao*q8od -D#4\앾MS#v3֦Hl;W)Y ?¥n;-rWGD~g\<칮tZQUW#y$q?ꈕ S$ "BԜ%mw-L%fe->6 3sJ[nDe0mm,VTX{u}UCZԫ֟5q)k=RP/q;{V 1nǥ:oc{%  ]͕S:;~2ۅ0+=.+ .^+6v〼z!_UQAh)ئʃz.@ flUQHgÅh!T ؊EW2Ba@mXI G67:'m18W2Z9精:RH;mپ/_!Et 2t– k @-µ Nj)!tj(Soi(Bol ##zx4Q)|*_ &PhT#H~}<XZ˱O.HD3O?dG)&/ƞ%( 9ƌu)ubz,ǗP6@M=\ڞPuJz|`[D<[]xoă/h5}LmWdKo'oS]rI>;Po~UNY`Ax& }TH]XCïzN0wrrKqTHv.d}_1uDS.pɒ6kcLK!#0--sy }dDySMP^R9|r5ɷVvrsd[<$%iQ8ٲk uDwDa*s|<5? TtqҸ#RvD1V f{xwd,ձ20s Cp2)5o)Kk{\ vz;HtxF£R!Qe47keXfr`%rUɟT/+- o$D~O^JGS8%/w c2 m2[%\6h%9vOaTd{I+PP\5v誊2⍊3dԌoG\q.=9E 5G#oO!=~ˍ.hU |A'lC>-A̤WQ$O IRm[cD;wq=Pmߎzsx#2 0>Cfeko`.&ۮК1bs܄(m eM4XS[ JMe)_m>CmdʞQQ Ɍ>OY f)6J.X# SVA(k>r쏅Ǿ6$ƥx]pk2 *GXz "IB# JĒKT.1r'3t.yH\EѤ^{+H98WҖ;6ELF/$'BHԶЊZ&۠D_$7tk);|;)UQ4,Ɲr|pU#x6Fiy9 |,53ڦvD!g%2G"3d(߅,fy)EtAq@vY-WPy~醥UrHA^wJJe"097RѭW :+zf6؝X"dH bК`M]sͧqC)VÕca$_j- +Y =EP+i S-t`YU68jߟ-ePzVSle0l= #ȾN g ͦRS;D[*{=-1:l]^mܢbffTS *)g:h&eޡROcܫbqUTN"k cA5Xslrj.;IurJdz^mLdnK'_\n՚6zdϰ)kdzZyrs>#OŮEX|A/Tt%li-nrգs qZu  U"[~_";ZY.IjT1rcS'Q4E`\˚%UMK&1c`7]_J'syңe_#ܕk^0PqpW52XJQJ$nޡQPcȇ(np"L)74X]t洃BNK/#'!LAe8spUDO 㭡$p4 4Tœ@8U-xխɴ | i* )hxvܮQ _NN]Nd?r8M3P2-Ӫh@RC(kncE2 e&U\تg| $Pʧm'I*A ͍Q':a1Ɍ('`T>,z$p0l:iK)D6KjCg(qeCLSn vݥ5 ,QZIXou|58LHͮULa.hI+ \_Y \36Pm O6)|VӇxHjtnF:z$ybZ ]U I[V#H| w<'(G8=L5CɶV& /d@(]֍E 3@8?FLqKB#~VUBNIWFmנBVk=z%${"I3i>TG@aߪZkw\unZ7tG v(Ɋܿi|ڊx*AYz)q%&:F#p s⶞ B{r;~-yQ?2莨c+^I5(ݧog)Nɥ'!7KM{T $ụqv:5QBLOU9xsӭlh"yfk|O[QpxK*,-'(9#{uHvcs],ǹ(gU=Pc/p 5+6TE[-S|ο<*3-~rU9T:G8Un{쬍9(!"Miq&scdd@SE슍}ʥky*_ &E>B X HQ3UyP57BrׅDE o)ˮUr[euYpB7a$LOkE=JKceGX FqBRuʥӼ l`lV2;ULtDsq%T\ (ZƱ64Jiv'X Za$V"Eo'QtrH^l[OU݄l@tIҾL~剼"}dҰ=9?dgFSҶ/@(ZuAسW L6`މ|;X!++͍0,(mOw* $:l_3,NUeg~{e\?$x6hG)}L!,V sAGun֛U+a_%*a`WGk`rBX#cY'J.]fr]Dt7#)CY>hX!fX L{+~Fl'C"4 >`.=,7ұ0v^-SϪum8?۩SV$7v۾6s0sfC *Mp$2t]['8jG23:P]skZ*nS$;-5zg,~q߲a Ft"qR;;{ea肟['$^yD+]sZئ'1bU̐焹-uU픑`2" T9:8pɮC*3\X(7I_\\Z)p.*~N [u1#/Yk>AU@2Ht9p-#_׺eLtX$@Ѥi "gHDEkrNӊJj|0OweBiM?$T1fpa퉁vVQm;G}bY&8҆G[K@_7X8>5t4Q键-qXZ+y?fz-UNx`{z~&SDwCJu:gyZG^uLP߳5GmC`E|(̲:H+ ;p蝰B'?q菑9FR<\wUUCf0dPg{2T0b;> -,g{VnhBKWeuiIS?z_Kg/8Gf/{+*Z֎|Ū*dWͬAĶq2 X h+4p񯆮T_tHр,yNe[dk esձ<\ρ[r|ɳ [&P{ B`r> ] .n]t鸎t80d3i5kSؚ9>t$TgϲkT&պ,FKJjE/iQTvnY:bk|Vܶ'?Ij5ڇ>:|vXV8CWFKaf$ǭ=d%P}fr|k^#Bt$;p ،)-RqG=bcL[NEk#K GI06*T/n=y[غfNY,\ҫ]dAǢ~&Twwઢ<͕TBYUU'sM3*|Õ X6O'? <>9~707n}2(!@u.>X5þA2MdM>&3hj.Z;Pb8/=_ \|BQ=,EV]zx)'o6^`AUPiQ<[Uў9(û\hoUGquӏ_%TK8NXђ B8{(F^$qx{'I=Q% mNtGSʼnJ L-!ױN+C7;U>2ܫc3syutִ]OV-^*r8*ǜ!2iL .zt]Iz*b#sKs8L N^<\ܩP杦U.n*\[BOOQ\P;(fg%o"vbw8GJ>#ẁ+df{C}lUEo)փ%9W)p~0FI6Nt6*cqm=<#cI1"rOElqX}(>3ʿETi/ OM$6Xy*(!!"CB`qhoMV.|SL:8%1:̺K wQ4[.ёԥr{8*'˩z X pᖛ]5LlpkچIN2 GvM>3[m!ùYmOuOs$VܐOUSRg'=EVf0SãFŵd/V<)rQ4>`nU\F|I6w)2vKPg5(XGedP=V7-'t4X('3u~qMsWvԦeN_%Wr.SQtXln<!QC;v^",;BTE.)HâxCpW~%HL[Ôt<Ў/)ЈCg};͚rzUP>9k#~ ;8N`FF7LMOAQ1M5ј,?h(h)v<441n[_茎S-FK" [bzW6!7{*!ډ yZ>/{G'cH6+[oTUo]Ԃ }JH,nr{Ǫu7\͇Yq Hs~p96HJ3po Rv/Z8URw dcu٥ nP/ԶkUa -KYB>6 vpTA%9 KX3)'+VmH8{.̲6>a?*y6:zi0G=IGI3OF4| ]EI zE-n#Oܸ {?TAV4Jw?h|7UDxM#We$ %7q:#(i([pMXNK|ȪE U1#)憆4Yk0{)$%%8IM=CzNj 8sDz\M* aS+flM!W=`,nXgKɡ@<%ִb|7r_S\TN3Pz::sT ^I+ t}r; *u@nnպDAhbK)qn'?}^D͛o[??L_ER$(%\Gs#\]84 n{'U7Tyc^?(Վ<ܻվp\Սɿ'_(Y;uD{K[n\P7Hf0rS67qLLU-ԭ769gZ/w:?:pSK>mY;/fIIj5کR6mj@Ƃ7R:5#FXϪy%˟] [ֳ|5p] \Pe0W#_8߽ӭ3Y7 {Ny"S"ȥƩhm-{'hR3 ^ܬF{8<=U1I+ M.cmf+ k/L,SZ[Q0dLèɧ/߅ncGWF"tUTr|HGqۂ9Q2S8Ea|MIPm{xo/./隵8a篺ZȈV ]#U' Àدi~F!x?(//,ufUTsҧr?76x?u]Ui-m{m>Я#a488y~~Q CQ ?3SpQ n9_ൺ`oiK$\"kCzƩ.W:^7\8,~"iki7t^ǯߺʫ+{ܟ5}6<6q}M"GU#l=SM3Bf5ցf*TvbN3~exqĵ5=(Ӏ,pExu$G~46L[.RoqmđC(`@}-cZySo^L.,e@N (l_;t w%c݁׺ rc znWnP۝u0 d)w;词U32\m@DZ, j:ذ2(@Վp6XV UL-5 "ip V}ٍ'$XK"rIsU/hPϪ PT=&NU_)t)H*³b넋_ ^Wp2ODR*{'헩̯ +I`p" "v8etk}ԩ,. JEG|=J "7i\L՗$S_Tl],۩{?&T~ž #7CuXm+x2=}MSa|"Doh C sTF39503sGeN*Ll Jt5 )bqغVOQ9eYjdľNs~SLy*'0l4&H5=P0gcٜKR`=S\ vPuĹĺG_+}Vcp£QoHKvDĈ,{N\rOR)vu^U1? -|tԫ.xCLzYQ2AFY96B(=Byo)#%6S8GVM:_[k=6~]6ḷeC+A_odj1̕rPj n7]K5RW'?R ]Lͩ4w}b6lѢR7$96,_ˏjHDxlu8՞%cn> 핔;=;C~yn#<\TTK! {')Ήs]TZOu!.&J T<ªb˗[Kd^ ]pgpSgFE$~TIEGK}&:-~؝7Ov |u,a׷pȴ2vId a.tE [荢Xd|lZ5ȷ"p/)zQ?.:0Z8[8`CDrfjhch%ĔXл;cYY}K0GcH`-sMQaCa^JhbS@|o !+1vмsF5٩3 =|6V-.ӹ'V4^Hۑlpjh{~#B\6Y'|V/GqW=!VghE@]꘴ܠ <6 )YI{w}U 1.}ڏxaq{,h"-ccлz@Lv<_ʼnA԰9elsTd MRԲ$3fDΪ-rOkfyK] #< q][߲6Y<[N$z-U\NHWTu!FF Hp ,핂7|ʥ詍ŮJ:H˺!^L8_쩒2lLp;,ޞ^)^SQx*$-#NP|FǵfEǿmt,]S q'<ɳ(wȹtRqڋ,nө*9 {ܑ9N7]뭘 K5,狡jCMN (7( YvhLTϩwɲJ!ms 4.G:wOfY+;Bn- -@gVmy%!%yB)SKNeowg#.&M1 |3T= X|LեpcvVo>7sSĀ鎉9/hBQ㕤Lxw8N-<'mpZ}IJBåKq\wK$,7ஔTz`Sָ=wP^?S0:sW4rj[<7CX_̈oٕG7i?EόSqBx~^2ƛd<>)o]Ƭy1?in[i񴸋O,9B \Ӗv!j|9Tid3n,_B-;,uJyo?g<(]- 6׷/ Ve=3y\U}_Ondf9p%-lUY1WzzTG:|M7h3S3"Jiabb',wxHEsO$R0U`ђr[ZAF*TpI\ٷq*eΰ[1?J1|T_ ub|=O Ȳq#}=݀n8FLDU>X0<Y?E 1GU_\t~-9-@eN/}E 6pX4%@ IwNj]#$4 ٨>Z%e{rY7;Ӏ Z "a膞\JI6T 2\Aōwx\fc m:+0u\G"gM+Gn,;icy͹T zh\eL7ѽJqQ4:d_{=B}tсKJ `9>}65k:4?Gtc%} xr @ֵMM#22J~4cm&;La`C7pi瑕;AzO5yX=WS.t%UknsTIe@ŇӖs:Y,vM]4Yg5G KZ13NOcP wE*Bza7cO MFg%5z0L7{uLK\jD\0J*&&@ ..7mr8 R!b<!),Sdrz)G"z\*zwwyqpgo~zח ll[dTI8 ˿ ^z s\F_ЊVOT͒ݍ=HmKGUZ]DE$Ѱ&mL36G7BsִL>P H:\!Α-c\iCm`=VCM<1 WB վ>hT&Y -!Uo-?h^~pd$[j,E6`Rm;NHN)9 i7{l/U__Zr8]l[T+-ʓnf[* ~Wz$7H ÔʆH߆t诡T  jtXێZz\:7ufC/~T.S[~q&ުtrCu pn>p{V5{#fipT3u%J v#(m N/o곴.aoT 7[,8 BfȽF;vqI`u0m7 @aA[%+nm*>4ѭZ76KM,6 %4Sͽ;~K)hVῦyjvFO sdpN?``Q^b;qLntipUnKc%\ooTPrT䥦~֝ilH[GQшkZB`v朎I sK͍>d<\=be聡Gm.i7&P7ͣ1# og8f,l+Cש5=Sէ#S{-=J}O|;3ئGfpm[^?&-i9Q#{D5p-CE+!izde[F.bwi/z=y'ײ:$zvO)vZ 㦷';D4aβe V1Oĩm̑yo3^ǫ-q Bg;.rOqjZ r_<+X Na Y3$}Ň+%",>}5d{&QQQlS1)O+ k*6Q3mrJWQ3[ XO`bcTT>mPw7j%l3q7s*+Ff9FK^=JwT{*{-KD(v9nJzBպhDA1\)s cr=hu|qqQSf~t۠BނH(\3m<&Yw*㑀,P(b|6PCXrE=.RqU"4 >W@#71,]5.iz%Pvo6g3Fi3u z-LPș4#剿UQ X#c v6|7{g8J+5j,ꖹ;ŁB}c詬"ݞ% #xhfd%yrq90S]Bc"sy;Ry^{rTI(ʪe.Ddpr`bTܛECyqNH$Hzy8vG%Azy fץ(gH]|9N$zR׏}ΤLe"ǏE60_Q'/pU9n}ꯎ ss H^2nD_n/ nz=|O卭e;thU2 byVmvl[eEiVR9]M[ vç,تeArTRMh#wi`lN\*@S!GSl[z#{~ҽsvl›e`l2d79Ϣ* 6#c5x@!H}>r6@=JőS8ZГYDw})`f$ڟJ+4W6 NjklMdMoD` 'Es!Y\vj<5@ɂ)ΕPDNh|ܩI#Z-d_1,:CozqhW*61!zQkʶM@Pͧo.V폎}R9]'݌3yuc͐na7xLH9 ů}J&FIrj=9!FJ"j;aփr=ҺJVz^[ g;'jDW;YLIvRkn$&/iu1ap'o麛~8p*F6b6ԣR96,zơB۷,Mt>i1 ʧ%fZB_Q+nITU7tCfk9GYuøGRjk}Ry*;p6tk}(9WM>W;!권F B[.V{3`Ca6RdwXTWRj/:ї4mPIPFaܟT=s&hQZSW$9U,8!.msnv;S@A-sm#Pnh GFlT$kN Y"U)]㹶_$V7o(=t[5a 2=cI>ɕ.#]DF:HM%]ŗcҦd[qQmEB|اo-M)hL.ig쨌Z:=U"-P?ue;"I[,q^}ިj%W]땡dTmT3Z1dN}W&E %ោ hCA%M;n$vZ/_1 ?QÅ ҹw'ڕuj:l'ICuq͓/No>gm*tzZTmvG갾&OMhkˈm> K>rmtUjTɁ^=c7| CԴҾ):_-p0Gul h_j|;j9vCH4X;4:h54%U1N 8r4qԅ~Q#n1-v>ʳ8A_[$]W%HD@\X$Za0}H%~\!e^Zee6Et YfH+ eU$$yq!iCƌd:Y>+ٸ-񡒈Lh* 7q'&G~  6 K&\ {!о*Y$(ǟ$}4`S;5p[ viݐnCo%r|O%BTվEsC#kEˏ`nW|E -Se]/}{.Sw]B xIUcs(l7-}Kk{II:LfVk#hfZ:I<_Q+ћm_-uKZ3$<c uQ\G"1>z\o?q쭞x) %졠ӳKfӑu[ ziO|c,RY(@6TM-^.cgWgꊶz;+=COuLe!]n{e.ئ콳VXYMQdH\-˶)Fʰ守Sg#v/o6Dܮ\bfONA-,*.XbR}OfB\Sm&6$i]t.<~SM&dOk0tS)Y 9dtt!5vNۮ!1.Uc$ *.!~.ldr6 䫙OqY X] |육}Qʙ~ ;ChY+KOi21tE!Q%i { )v QO_}yvOԄ *][MVY741rΌ:ZU0#F>5PslmW<$`8qdEl;OČ]>:u+diKߕmliw%8Z)-5AI?E)7TDdYCZ^/cW[GDVӂɣ37 *y{[ONDc۹&_M2F/"#`cPKLHNM"\n.,ԼCKLKZ+9W D#7%5#ϼˤk'iE#@IA7`;m; s%s7D@6B[ĵhA.&2i&Gψ>҄=-BIMkb܎d`? +=\]sԴv6 5E "p1CJ¥:v@.x#SjWI<8贺f0Kx5;(iQT etCtڻrB.շ7]%UGe\5 cb?,R2dw#O"_lCUV1^LCd? 3S$8d{M=T,_4uħ#= Rjk@Z;B@sJ_Hwā 2>_hu8 i|EAR*.>ڿwSq-6)hS\ռ+/e9WG/SgۃZ <;hZ)g:Ҽyi>Le`+&6oZg$XtS`Ŭjox0Xo-pֿP:o `$a }U1C֘k``hV5Ƴx5m(k~$'[n>K" TnYZn09T]rm{Y`iu\qiy[)3tL6-9s~a-elۢ=[FSmlhw(A"T=MH`9YG*CnY-GRUu6܀stكC$Z|PRlO33Ij? ϳnS-lU-Q͏AgLltwç/r /5ʟբɥw|۶RjYXFؠ?ONdc톧]{O^߫y?T[''h;Ji2>|_+DpբaN9"h!9vbm-mHm3C ŢI/"_-%6Fִef.#$pUK56 MW`e_ݬ7=H*f%FY$&5[ *rqOUP)KGz@e.yrݗq/Xww !Sih[;+IVf$ؗAFw{%J]|fsI.*MUjAx]"Lk{iokB-tOTm6O8KHj@Hra8S.R[LbA԰C]9Y{B 8^\m^_(lGE͞@pr9YM,Op9.nːz)uSe{WV+c6cK`SH54Уm~P&?ŒRxzGӻæ1YeEmҫ;]КE #Ԅ<6زa /)z`L[舎6F fELW\CMG1Sl.9?K```gOP$]^ȇeX B˩#Dz5f{,k8[an7TNvI7UMu]\ =w^ġ \abV{7طm򣿲ۭ>']n\2cPdq?`]L^Ds{>W>!'hS?|jCN &^i"OTb?7}" ͤ T#vOwQW/" ]_QL_b\BZ7FSO]MJ%NX9-\]ůK)cT%Sa-{iGyJJHdD7fCin$59Jp?Má쏒n9TT^eZgi˅qg\)-GO's~l's[#r;,npE-8F rTP: &L8&\:lGn=h hhYR@ ͊G5!SĢ$ +d&B6EغC }u t$llvЩ7 xּLAyc .MS[lܔY$GhUoG9<ܔh# \9 mM=>}3T1YFym+#?i)lNȽ*/ 6䅱CCv/RVgxꁞFr Rk&Fp9I-eދ9SPXO@V2Wjf/)eF심\B;QV]/wSWɵ+(I"I*A)s NUqŒ#JEGPG&Ÿ2ӄukq*ߕ)Ȫ#o~Uό ч5)M=F ~OU|Q珪- '얩 Zj'Iqe>Р!xc #0T-*l z,ޟ?ksVE#A]4U⩮4vJ72@@v[ԧݥXT3sJh׏5эͺUSFI7mګMty"<%54c$ 'FAui_G \[ QOP>_ nWLFNܦT^2n;Ux+!>tvCYLJ֑evxjm; EKYK@4%)I* qq ] @nQxXуqFTZX%!?ᕮj*M_F])?Ry. S-s#cӢv'nڙSj :VtMQ*KWNA#إRo_E5x4Uc,'(-+Pq_M4'50K,19ԟP;;I7ϼ#Qsږu흃svRZYnSGu(o+} Pmiʉ;TFӓk.LQh?'Ůk64 hii&ۇ_eomͷVPvJp7Յ#TO+]lv.'GWLzpC'$Jf? iꚧ&XcVShpNy򘼏#'I/پ=!_8SR~l.>c?{ߌu=ԴZםm0SIU,tҼܹ}$:.aolozV3tb61<tښ/hbOÞ<Ql#lQ@p{Sk0n9O~8W_g7'׼Ikڄ|P  \u9 a #6eLZ@\Z!$(isd2@ވɽh>&uEk1B7@Y.itǐ(І6[ӂ# 9ov;OMe}-0!{fQKsR'[^yP%کHEsucipҸi45!ltqΏI/,mI6  o6Y~ąҶ6٭\1ޖCLi85B 0]IjE-vȺ&W&92eZ^sr'Dloe3cQu›#*A4}Q:S HMVu"!TK1aaAYI*B(4`ldZ}1=8EKn)UPhZ8htTo-Cb6IvIy6 5BCc.D6Y 'm3!*!ʛ#2`%b2q›b<+KZ\UIY /ORf.cbZE//SyAKTpVк3mEttҐM!ǜ%gANU7x^b˷[$Cas⁎ O!p;!̠YCqQ:S(BF\U[Wp y%^6W XFr5Iś{qMu:e5#Fc-WgVOkwn16r"qUa^W8zgʨ:/8Q:6VGÅ]5q؞9#iRe9' "R)]S#)es߲tKeG["Z=m}D 1Us}=VBX3)pʇyn 15=6ÁLjZ!sʲJ6F{kh\J9i,'Vc%kc{ ΋9Y']Fo{)k# -z]>jb\"wInʦBr0xI*ʛ1~12,n8>ԩi\{ԼLݴs &v^ԛ(No+.G dwl?5}PN?h<]1"I"Ղ09t uh v>k_/'({ pVa`:' kx"-a*\ax >4če'6JnHM6@LBh\:"g1uY=JHdKGU AAU6Y6y6 -l 'z=D?Q#y-qd*[@(w^Y TsGC+!k 7m4aꊃL; $B-t =ػ T-6ӦvKkck?6cpd]=Ttz|lqO#&=!h6mLkG? XQ_?DLr*%&; E8o&SMVG%ORuTZnkz(FHB]otGo; vk( TTmCC=N GԚYy.lVPsa'[m zu 7N~.>kW@EQ5}g_!6 <(t*:]ɷG6 I/ 6U姧o HFUO貳{tU;IIn=߫l-%O_DT>R;+R:&fIfەjLM~ʰV:k S6LtԽ{^ leTz|1'iZj.=) [2qlAII5?'9Pk碹~ц_h%vQa^X}V(?UE~/6X\R+ \#PjpH.i|~S۞ PJMݔʛ$u)(?N. ];agi/ ոm,6=gi[TrUNtf w\L* |,.f-('d{uuƂI/,:lԄhУ_*J0vG,v=OIqOgnd?ZpL үwU_8UQx{L|`~P_QsPu92>ϒGеE+^[}5c:s?P!߶'m.< 54*x.`bx瞤QMT u`U *(4=ec[}ShCVg oeAᙛ#jc-<-~k ?<7\<7,ϋ|gR) ]Ҹ?%PJR0>_Ӵ1}{?4rV^>ltIqAO-:;턟w/Ks(37\X?h 4b{5)x-dQ! MliGͼgjye&$TzǺqO횇" @0}WkItT,ҽ@˸ߥ1QuULYMࠎ`tʩ,S!`hM!<`%a+[)9Q;J y%W2 %$۞I(z{3i \EJO@9>eͲiO@R(,Ӈ.Ξ[(.8;V(nťƟO>i%OC~QK4ͧo ~O-d*0 Xz""݂GLnQLEL7*DO Sk}Q usb.d zɪ(l^p T )ض Jc|ޥ>6@Hhb j!*ݟ29[VGQ^r~WT߸U%NTKsr"eCs_Oİx? -tpm_(]ve~Tl(1I *pJ5'hxx: m@ Z {m` yB-"b|:7 htdNi41 o# `\.OtRʍOĵQ8D_NynHW0S'=c{Dɸgx\doTr gFnф➡,puuDq3^koS%2|3MJ9Lo^?)bʫmk+[ GTK_c~b(eQ4; 寖[l=rIBo2YU8FGpw}$ɒ.^]=|?d70 VzflKZvJW(n\U  5msG)?:P_z\ YP.A`jZO=5hdmv iUdq͊$W2KqnMAxq!3xf;( .?5a,ni C/,ujS7Y@ʺ L/h䂶?*+ nzHX4pjo3 C2 :1q2F׎k)7^ ne )ÝYǴ B8<.j$mp R?)G!C,-Ob6#.>V}%&GdTt'̖ȦSS&}i ʚ+as轷&MS}@1}@56?*n2T,vvh jH1=yitacfcEYüolތx6mh`Qyw>5MOG0vC/θ=Cv1ϲ&?]5l9 tZ]3\ْOULrCC 6/;#"z:6o38{e:P*+{x._cӦ0~:%`"eLiz &*I[i'f߂8$!?]GB\fk&7/ ui/C-Yec]^*ُuLuv_ 8N%ѝ~Js2`=̜mob,Sqod(&f4me*ʱΆLlDK`SQd-6aʪ9-eI/!r.82n6M R'!MKw|ςË_$ ]"a23P|n4rr|m8.VN_T5! d %Ό}%cSZ!`$uDH%<; cnfpޗY`cwB=nHȾmʙOF]&=@!Px^/8kq󐴄U4b uNa-C7‘;tgui#T+}#F_wvZ#OG畁͛DH-0S74Dy.}:7;-9<:fS':uWi:l61Ju oi4JPK oƶI=v^TΞW!o֛rT>z5nSBd4t&%g2-aw?L`^qH+uSR'`y:U-d2% S<2nQH[ 54NScBUOyGjBIZ,ju+C!h.w"ȖT$e 18nۧ=,-~t?\:Z|1~]HR2g|;Lc)7M^Ъ}HD0(X9&hmf-7x%JuQ ^ZꪗvW!54 (qvS:}=|L`c>˞Mt1!u>]bY`е1 5싎D@R*%ۡW.)h6lCnk;)tAp0>?tC!/ɮdK 0dvW6"y mUŪW0=U\ g.RhԬKbS[aʄ0/J\nrUy?DrcY+|4z fs~PVeq%"S V ./\2z[Ef &[_DZA&^Dʇsvey<rs`rW es|I#]$'n*E䪈͚8LHKqԚkSkX tčd;x)s# z)ߢW2,PuV5l6 ڍ"Y<5'>LsfFOtf:,> I=ShMPE50Q4UqLj%&EMb;W:kpTLDHqtƀQOv:)da#`bߩTL9MiuT4OBۀ^n1<@O~ͭ=?'J8=@AmőSlNa#Hi*[@!D4yWC=ksB6e"PSK CN$&9ev^}J&l6MWo$))Bm +2,m(|hٓUt,̧4R5[tSɜ|>=\Xn -CZ ZWG{dwBQ%Dѹ>U2cq@:W{,ak& $_'|W)t(68 lTHxBMՏe9jC)2# DrodĊ"KLvM+lsaoUJ)^s/odmaTm Cr:8WKYp7G1NIM(x WQMcr3SFe [+K=bg} .mtƦ܂# õP}K,7xi|MhPE8PqnNOA8*EqLFHNt76..QDY-\-hi3llݺdLkN6Yfes s-V*X"OGnD*yh=RJ A㡷 iюF@UQ-K'ބ }Wv@ qiS \ޣAy.q<*nq߅zHtP'$˟T,my2ga.L[uK.'`'|u1c\W+=˭l5 A6RS3Ll\{2-r)d)rn[C!|U'O2|:O(Sxnܬu31j$Ӛ,/E@΍Q>W\#)翙/EN_dKjܵ[UuGD.vY]OLCfvNMO3EV IsvB;i)o \e>DI/'ꋎ}*-v;$s?Er_24]Pf%а=V1Y3qФ{TVo>sI;Y,@7u=Km4Mw_)sOϕ}"now辣?$f[ݹJ46с4}vLm _!ݡ,ՏFmޗLӋ.@!%"g={&2a͞6|wcfFHVAէMMECE =R{5.Ycys ~ 2\{e#+ˍ]y;B{ҟoh1IR웁O{fH=Vz}6wf|Wm:##,ϔ{;v7eVu|Z|71?Q3!*qQ5ùLӣa`7D?bt!7۪o!gNRىmJnJQnIG5z\ljiCmU%$S7`<;|:CM,E.18RLj$ v_0cFlGNk}EKH姢v;uɑ٥ә83D3%oDҎT~2@.HUzTMd)0ߢH諩_+nn-tAyLF,US'.&y-Ff pa7@!ij1^4rS`V:} jS=#=:&f+Ӈѡlyeu),trj5 ?4X)qFh>?SͧLci\xK }\P39z >bǴ3íjY\`b}AwR듓eA)H $gYq{{oLHb9A 9IZ &$6X5\K;vZ:*opVvH TI%|ȺdBmqw\"byPj)RO;e%5T ۡNc͈h)_t/ULl[`4/\"P7BoЁBy~?PFܿ$tGcͣ:Gll;&oyCѤwߪ2 MuHF3["gM^L߈ùdڃZ4uPx)ɯgUI{$m- +`fKClv%fX!]+-SQwxϗv`z4Hi=UG0d]QW@I9FU;v0CPhkPnz/G1Ч;نyLIO9ar锔y>5h1mt{wĎ6 -hPW5%Wc)+-ˮ}N׳ڝR/LVN 'Ԁ$MT4Q@/88G C5h O)+q9AKKJf:0,~锓4hMuS!qfOcުQWzA<"ˉljVl^+?9>!S64n-?3s(CK.YPDWJ>}nBX GkZvpKj|7R`whӝP^Qo)[{qO 5FRBv>4|MyC;BSL#ZS _BgX|?b?n eWUSD,qhB;Ql;A,i?iZ ko|Qoicg=育䷠̝~q$ EM&N>]w)DLl&!1cOF[_lݡFIFӷ; $MA-οva<7q]l~ێ%]ro"BKne.6@766 ǰ!czhGN3M)(4 WyMLw?rEfK^ ȈHs.07^ 8cxiu}0]'#ӲSEI$M]8dnE5ҿ^ }V=,ߎ|^!s718o6diiZ~*+*%3z6W@7C88ĒI$I^mm\?THg(:N /Q$ AW?sz ^O0ҒG{.\.@SQi9x9M) 62$MtB};k,. t~1R+ `q X757D2:}ҝYMl#zZ5B[ {@TI4@ /] <D dУh64gϐnwDd=P5D8VM48i̎{))/;}Ϝ˨XY)kott|֠8ւPU.Ǫ{/Ql#uJSPR)02PսT?@ uMpi rԅhǚm,L&C'7R\-`߅cIUsԫqW>6Lv]ATpVœb [fܮI$纲8" 6od`{lDLtÁu=Ѱ#tj-"b.>F!x)4/ܪ}[Shsg9Gw&M,2y4tUA*gL[&$lb~)<8t>o(Lْ&[1UеO50۹!ɜqD5WNTN=v>p.<4D3{@#,̺## \XvShr^ .nph *ndh=)gficrs$|xhj+s.)%oe02pMKJ0mCO'5:^NKce "ʘ4]jH0F@`lFzZ{C%:̯hM 4 %n4ԖOTjܸA})J1tpÛ~Rh;5LbqkZ-ZAOTC^*ˣ5t3wP5ujay43|[CKe֖e"*6ꮝZktC=Dd=P2McbU&) EͅlUMґ>8 Ck_ v= jQְ2WYupJcO3!85"rѥl--mt6E<`vʖ>n*(ik$W y5}Mb?ʖ@'p 浠M@K '#rGĥ0+v(;'bAvS$`Yc-=~e1 ЭB^x%;4\D4|F{Z"hi %mԣ+@jѭnߢa/bJETXNU]Q4˙D&xTO$41sL9S}uIÝ)[ CGGRkXhÁcK4dbo[QHw %:~ˢC KzIM;>&mީΒ\SԳu: dO9-=}ZZHbid!cpz_Lk;~seFZO5J(ivg$tMׯ(ޘEEX-A]S\6:%ZA ?q2ܲ05%:[<jIvʩ9Ilq%|Nvtp|-%&ցf1bu@4Fmhj"`/`cƗ8cS< &=Tpm*a.T:O+0?jeI}"9&,~ة.>GE͔2Y ښ%WG9i&[ 4߭pZXBwG ČS_ԔYεHZƋpTڋpBe-kUW?X9BE-u;ZPi@G'Nn~۞ri 5H)5H.quiԚO.'qfaz' @>#hdw@ЈBSP ȡ1q/ - $U,gG:NDMYwɒ=NF?U? hѭOrS|/;oԫRk5:+2M)'`; TUd_+K\ 9R+H}K*;sB5-GEC # vHFK+M" fp/ H Giq&`xMeA+ӆ6a(hj*:~ ͻ H*`/ꌎd:| pd Ue:Ee hhh^ ŀd"oƒW H$F_ , }xe<ZB:Zm2}VME?߀4zLP-&qR RϾؾLdMc0hEGOe Ե{| t4Bo,0ňZ T9\te['uW.|~I*I*̄T 2jDUYuDPs9)XdmPi(I'}PORB"H:_-5%k8 hl/C鲴yԶni@1m|/-}}UzVd<,Qā%9-TٛuzHGRIM!c=%u×TEuP,MuNz-m{"骭pmXt{C%p!['vZ"8{,;10]I`SUI"-#j5n񞽓7Ss-jhVN5h&8ϋ|uE+o|ϊgߢn>8H?Yg]S&yw6$\[L>T70EqfT"AWo/w+ Oc T$ u&BF̿>yZ.S~1Pr׺$m !wźgɍGcCH*Zfšy`RQ;o':R; {& ti\i-^H#_'.M\ IN{z-,2qqBr^Ż )\M mz>M-M6$9ԣ[RB>@:cϣ/KO6vٽmMSg?Ɏ6R&̩X{*;4z+ncgfAM{pMOF=r وkF{GUvY3<-S?EJP꿯jmk7Qj糏'Lx?䖴 MWDw9O!tRlё_Wb O ?)/ -Ǟ(;tCl~k]촱Qvt( Sq,,>l Y2.@, gn<:9ۻIt QzֵHuD!w=[i?@{+_,:_ xRT/ch˭{ߩF] ,mV䔹\X ֔I esm{ "pr%ǰU9_z~N W4]M$ͯf] eɕ.krSieAY4T`2SdR7ZaOY@55.xq-%٠{# D6+[Fv= HVmʴ7襲6) e;YO@.l=6裒rn~&t!ҍtrBE$O@?rCN h(6O#!lSCCNo nd\t8RUw Q,kNJV"G%A$¶2&rIK+ua$~>RwINgku^|tcg$%uZrUjG>kEv<%1*`lUXŕW:?6Zc6; \+M2M&.4 6,LEEhF['M8nTEk 0E3Ua6;T9D2"Xp5z+l]̄WO##.!9ЊX'Li4y_bd֛Nķ{W"uGN:(ӧ#6pj}3;q00<7JSG)68EcZ;h}!rkIb諈Tc #YH i7Liomhb[dS[eT`eQ%A{Z=1S0E3 zEaRns-Ԝ5zl /<_/i̬8\ v.ֿsR9k*Ő٫[u#}_UuFYPm10>_4.>Et߀&&j wY"斚^Y=JYW47l-G=a%n{(Yx0; e|8^q_y^ ! rK}fP9^x"KtCl*J, uAI9qU9ČLP^!pW$yGJn2y蚐i$VC|"{{[:B4ZkV/m O@3(4}ɒ#PJ OqƸk)`MbMHiνkg.)vXzKrS7L6*mRl.${ MH,]MʔqCԇ}r6Rxӯ "3mi`m>88ꙺ?5z#^ <]z&QTTw\Gyh0=-zl3Rc+5Mem2Frxox&ʇ@L2z'bw약9zEnC\JI{~PSձ!2;od EC8eJi$oXr=ZQ3(y Xw(6T 9 ITniZiz"9v=VX.6=SJds،hB썊Ӎ 6 }tJs d)*z }U0^T%Kt)cNO9g&Y*qVj|$s&.caM)Ee@p6(9 b~W> ͕F-V.}J$y8`MCɧ l}Pd7{_sH`{ 2d=9}SUH>($4rFA@sQ.Ap>/PKmd%sx$v)%M nTM sK-:PQTˤ3ilݼR.o(R| E?uTOGE% &Br%۷ї 2*P߁NљcJ Fld`g"Fq+pq፮]~YnyD2 DS)KBH1K[f %e3::M $ pulbDPy>RJ\.s.@Ҝ쐴Z +v sjtC+@_$+ƐĆpȦj_ꔆ,UD(j3SָFEwn9 DT>Aa!5߂eǀbzj*ZĞ :]dR|ybvrQJ*jZM+gEѕ H9 h\²V3a)n~_J,%舩y 4<gwЅJE![44J'T(Ywu첵Uڌ44ps'Q[MNp4j|K+<J6FORL d_Z#y_1?#RH!e*oO}u}ePE#N\G_F.x:_(~=LbPsSX;x kڄR%9q8h֎,g[^j$dI̒ hcmﵣRݶD:*epe)Z]+pyTÞ܌{ FVT<œJ |" +ͦL0˔TTނH_V{m L!<@pR4 6QPA`0V@R|Ȩ0NE" 1 V׭XfAJe$ ˼* W7H(-[htE M):]$|; vۻ#)= X qS4tEO襬jEEN{"n-rQ==#7M miBLhw9 Mxز#vI7}8Uɝ?I%pkzlfW& MnH#~RjNWc^rl䚯S7#u@[R9aY&jkddBTWk}O||kf4SmTsb짞E2qʰZ.U@+cX=D^,8V\eEQ޹e&?[d?зzrDMѡMM!1 uY7 TۭOFk~M)c=@ֲ1h=)o^\;S8` E~!ު)ٍ& t2a.&p:Cf}.c)';bʸU\`IM)hN)4xsܦ1ցlFo1%qS1fKB6U)RIdD|K9-!t l7zW}-(s%ۥyu7Sb |([GO YЕZ&6/iY3F1%8Ã:\3v2G׍@Q !rU:_MF$ֿ{sPTjM Q aJ:ئ)%ɣd2NxM;$n[p pl՜%ѮQLSKڮv`ZzzK.=[}U펥÷w 5~A~L3[!-C۷nu pfY*^Y#=SCQt76:I^mƛBy[Nyg]D|))p%jcpT*%65GӶySFFݬ h*'N)eEc[2.JRһtOn7ϳV6pUjsu^V}XkQy_UR_(mE;d':جhq;ئD5j W\vA0p7Fb.ش:3nߙT1|Gk&m~:3"Pͽ"pۢ> ^< hےUuC34&_(eelqp{sۢQ1g׺o[d5OE|W]+O|?+`=(i-1Q5(:y=ހŒ͆,p)%[U2Mܵ-Q/-]\@vu~UI H+U~R.unipL 9Uvs#ch`QX-p`ԋ~MHMccm_gC#ǁdDpzGÀN T"ݒꎊmCOZMgjrr;m6Q0yKGy51 !B![I'XpKN# ":]':pxsN἞/=iz)c뇖IpSݝ' si4:I$qsw9ǒI`W5dC*gv$y?;GqZE|Ko ӕ [ET= #AI FHYnl2U_YNn6:@@D2H:FaSld tL4edt4D#` :$Ն$4{Gȩj&8l>[sY6:l"AnEOd!"@y]ԩ\uX Ǣѓ \dD4&䩆3HIP`U=#*vVc"jA-kFm4_W/_ϵGaq"JsŽQqS J[[a 5EAF5&ԺH)މ g:).#c<0 wD7!'oepPcsN ѱ>PO>JR tC} t'`' -}GE 2e$l>^Ϧk!LY= %5! q.tLX n=o^]GOLqЅI ]V)[DM :Ih}z dJw#}ORKYR Tg0n8ucnqtԄ:$,7.DP\ZE27Ѝ-5HgO<)c]`^ QI XTޞ==E5Z7FRzD~;EK*eh^d.i.EXr/e is.BQ5F{z{e+ 6.-m:T?dx7E=Pr]Q\7(}NJW+q$:Zq'fU`>0} I):4aЃ6w(TBҋ`]YNi߾'tt+~mdഎ Qᕊ5Hk x]'ɉWF)CcF6Z+ hOnlm S;4XdH `[e\\b?OVgk8nWZ=w8Y[\۲נgWFO*^ݸNAT6=dzAJgIPi]ۻ\4{r/Hvw {Q .Bw.n<"JOf^Q ep1;DƜ0 @`UC<6D@?5aV@ MS(G H돁\Kk}JI>4. n!0$%T.}K-(g6L 0Y ~Tj5-? {W|[J:rtqp)N~`2z!y(E)< t 盎O 3nrj50cat*hRqsJ&TUO h˞sSMo{@2鬁m̆Xf/6Gr5- [K~E=,Frրke7EA0R[waT>8̕S6&&zXGM_}/f[-QYQznM?4 }*:I*ǯ __!Q yV{ UɚeMe=K V=I?xIUkHtT9+ԹY( <[&u:~RB3_j΃%U8>`6jλUs!ʛ"/5-!:"nwccccll KBK+̑{WJOE664|v5GNNMѰӸ؆ite"ripř<|Y>nɗBZ%ƦKgI``kE$qm^F "h hoVS1rHt(u} { auT+ k't|q681Vf?w@4Tp0 5C#6VF4#]̮dV!{nZ|ߢnԌu16dꃩ#1 URѴtYBZ⊤|=AQ˜k #C}$t㍨Coڻ,<A[Uv=J= Tc„nh Z퉻rTJ^yt/7IryP3mp;PΔ&&SR7 u$nEU8$-B2~1 $:SepPMb[;$ܨ|_8UJ}s|ƩE?sW~rS@\^e eO &쩢 ]NJvz&3e? uPu=EҌغJ}%Wg77DU 7J%? k""e63w$N ;mRT[N#BR<pQYG!:Q;|&u㠽g\7( w  dCc.GR"?# fe5 [VUb D\s]%j]ăS.|VFJG|H߷)|qMPsnfiMHD`8>jtxmg+ ƛMNC&qsF7Gt{ gOX۴={dkY'\lpz5-`R &?W %M9z#gc!@>[Ԅ^ߡW i$c B6*=+cF t %]pUL hh+R,7c;-Q~CM5cDw뢨m= Wͳ!*ng1s,j>2εFG[sOaVR/z/ @yR +[Ճ&>a u#qcٴn!_H<&$DeiI-ap!ѫ#ꖘCnl`~SF#{ASdhi'2d/aSeXy6 $j)I{e5` 0_۔%ХkG`m(|(1'7s=8he1 Ď)CF('obx3X C-nÈlXd|$`z3+ r:1QLpile"nq`9`h;iUIYJBT9DdF  അ.״-pG57 }@;:IPB.8&}hU8#c477%X"qOE5 63)eguCoch!m Sͷ]Z6/=CU{~QmS3h HA\l=M3 GrU!$Zέ/ÄpI*֏VjnJۓBci_ $G&Z8 7C,}YstTITOSR0VԢ#M!ϠGCsG R~R+e5SӵWCnB7ѺELӴ6MvZc<^*ORu9,ndv69?@1޺ {&O7"H.QhWrGHְwq8f|IG 2r=f4oy+,c]IH/%uzԏY'<d.d5:GA8RH1m]i-'̛( T/ɓ:td ~0[>Ҏ 'yzvW%v/=e2kl$ ҁ+!sY\)PlݕR.8/~ͶПpes[iő$2}S9]|?Tm.<$ShPB{h0ӷlM `Myi\Kh<|K[܃G~JbԶ*xe | 61IЋ}_EKmw{㫓D҉egkì#u;I*U_ Rn~2QHy+c}??\EtW袧d 2q#w_飷v8 <T^uLǐ \wL-zjr%rPM@GE w>ik zPsfG Eet^iB#ܡ[d~vI|=e-#2)xBLowSY/ ;=b**H<|G6gf~ JZl,|qqM j>>+]xIERPD+BVެiL_$s{9ǹ'f<5CxV3m=x!z;O! 9*gkA%Bod>Q$\6O.TymbFc{_*+&ȶI2e8E UC@-GqM w@I *wsp^:!7D((UU}zawrJ~aD镕XZi~+=CY&&?CUv+Qxr,۞{#[)/;)X,ӫpdvtELu+266Ja>RuI dt4K8RǕA$/ROVjq>cTHy_G8 'HboJj5*Ի9[t5HҫR̔Uj]7/SCC+]5`ࠧ$ ~%՗\3>{u6CtC!hˑ!G#EnVT'(6 :0*Ʋ mscR) LF?yaEenBl{ap7P| YjB!]$-EIZC>\LRQ\I*p2u GQ2~3{"@X^ֵ<&4|1{Z.JJFO`t|i.%,^C>sf[\"訡6DP4t˯~nQq>D!L!(Kr8qsui;[ `R2I|y&xth'ףYQYMN[L| #Iz3ٖ:mCyv\T=+͒ &{; fܐB4lJd(* '-.aC ltQv&er<#$ Mn.S%?X™i۰O+-R܁`] %5#6FͶmu0G*mJ;e r22.Ue(3pxvt(o'n= lZ&b­Ԏ>Zd-x] E+V6'?Wf=BӜ.6kIMev{tD& 1Nvؾ .)E6(!X]sʨ3ɩ,t†Op467:PEU:aۅMHi;xT=xE$mK*es꜐rᩃRi!7q&&!cNO zaT)6Ht\s ݊씻ʁIH>CQT@sjBmN2;|%Roc<$T-ٺ-2mORx{lm5i j.qwOK]:ze/t)eVgR7zj>stTw%ZKZK'dWl0$2v]=Pm*˷#=Ü YFJи6qmy !_3Qԕ3S8:)n|SDpM}P}n=۾noE 7Ni =-vmK㺢k\ʖO*I[^7 Z+x:e8|wR#7s숊i-pE(lkQ#1h0p+YGDz8d@q.[=x\{p2PW Lp#huѷw_;Su(_qEDRʽ&)YK3PbagUsYّ`t)4Lp#umTMAc\db̟3tsHQ0xr <ѷaJ*|9S%Z9~%I]|Tf;"cn\ Q| 3L|ܭ,Zf0m(fv!y\xv!Js@UقюHl/ݲĐe*u ꆻ;Q.C\[쿺!dG5 RFbUl!ga ԥgR d}ZHFE0)I[?-7QZekk8~c6tq(%:D\NpGTE+wHJM}^&;s Ap{kWURFnvɺA7bĤ4VJ%.l}:W y@ \F/ 7Ghgkyq{̦nlO`-ݐ{ yx$~ݍKHJz*K'(_l҉>k6Z2;&gtiN2w44ѵs?L}TnHQY)il:4pBwŪ;A8kk^,м=7HVǪ{4n;p,=xpю6ߍ|/6>>#N⺏UF`cSDeL-mWǜ|_g61b~ꙪCUs{* e.\ {|.lwu|t+o{+/ [b ucbSkZ=1-hQ.񒇰 /U[[ZUB-Uc f'g.C,QHKKCWZE O G]!yxOd*`B'FM28,Lb ( ܅#LpCchk,/ u* @pkPhh9'[]QHռM4K4pfku*1FTӸ&6,n\HI2|TPr3I$K*u_̕WjQbFM 6z }Cl0OWd!I \lnqJX𘖆TȮd]D{Y\غ-H"d}8숞l4Y\ 6U{a }_(hw{k#^z{>LQ$ֽCI@pkbg)OO9%[\AP\N~ʗ~K=j>.>ʲV_JzGq=t=-|4S-7Dpht_IT.z6-֏`KQ7GFZ3Ӹ $\oe,e`̼&rEmDG# TSӋjI\R ax SU9lchK%{K\^DsS*I %PUtܒ./pTK1(O"쨓o~W so.灋]A tUHTQ.Axn ,z4yztNHtnD5ިVM$=+Zakq?오BHnvZ9 p?sMPiyoeh+Z{l=] na-Se[Gתtv[oEͤosz؉(픆>F{ߥeK)yʱ1SP=t>dĝ>dƖIZ@J::ZxCIη #V-rˠ;G-V2kNI"b"G#|!Ap],cl5*\:jPsͺe \쬛E<<ձ Ň(GHuna] %dYPlQݻ鲃Req'𬍎&r(ҷB=Υq>QdNw}m 6$tpIO(tWΜTMvַת ]tOSKL S!Yܒnrk_ }R6]5>v;m7Xk߾ʗ "Mh%9 GqPfIRu+<> N:ÃF/Kٟ(>!q;>'WE[ d95:#,|mbpϖkX"'T\+[ <-t'4=Cd2j}[nh`֛ X˹d`i_fTDIQmdlw+iI#&hnt"2~B>+ϳVKFAq򛲂8 Z<3aUvoZZ-D<B";oRn7O.# =\l$C0'Lnw |6q?ʘSxb%H~=.g UٰBO\ 4z%kyIW]f7bײՙ_:^ʤL8m6GJ[!9?evRhy7]'4ݦN曓V,Dov,L::GȘi#cȿ~QHMd|ϪUicTo/etLt0$g]Ws3:9Se)J D=[v͕%O} =.<( jy#?i7C߂AL})6Alj4y Nt 1oM-ffVHN[t3oZt*kG.NV؈IkȜGi)%I\5cbF4bXOCq ?w1cLSi^'쭏KyҜe_3ZX4yJ+q6߄/d3#7;]ttq9=QuJ1cqʍ''h]L/]5y> Hi'/2Ԕ٩(l6O)]y?R2p=MҶ\imiR뢢=3æ8{#b@w9?E][u.stm?y }n^NhiG׿zH˝4twyoh:yst c'['ftꚙ$7{CKRb:lH$7t7srNJZ!$9$sV)z㒆|v^]l%5Amʛb%^qv"ޗ*c.|+ HN9ScM^k@\pD{9T_ب:[ ExYme=SB:Ӽ0}I'J{5-!@G{a>Ӽ2\ZꁼnGA Zc&QaRߑH]EG@ CH?-&8}mTTfȆDk:i3HRA$\ aYP;<~*s %ek:t`$6fƻc?e뵒nKW=jH~:A6I'ΔT%% l׹ܗMYrqhdƣPzobwAe>cCTMUꄒθoCkߗ6DO&i!nI%ZȽ,:+V: "6>3V|=] H}6/kU{cc1+c LGa;c^P~)K8\c%of IUoqpq|/a7B8h! -FyB1< ݛ =UnwU.8 ̒s13Nj>f[ Z#d25Uss_+KVDP5"BicnRi`@[glSh=m<6b66]SYѴ+wP;v'ۢB$S;%:,٩l.q%ǕK:3֣WqGftZܺS!=TCTmVgEoUTIP '^ur;aE 谗ʃeQ V.`;E7cT:Lc򡸻DeIqfx7nM%DzR0Qݧz.珙:Ql+ŷRSekm]tnbJiP7S(@'wӄ=DUlu]OzVHꃅ[Lʯ윤n/6#eo{G|z4gGNRzތ&"[e2;fJ]B,*zC=,.DG("7lCC 83u}q -4d{Ph?V 5'ð-D[̟@:utDԁRE)Rp*tXmO 9N"b^FSiƻeIbkOkt|u;HP{5W6n+|jf_V<dSoÍnh#a=rM (@j:c}fNnU ͩӲ6( z%PvGCCtITy$PΩw!ߔ;TWwZ c&㺜U퐂ǩ^ '44ёۇtW2V2p~jS5Hv,׉}6\ck#e:|Ҏ\|'e+@v*[{^?lDX7SdNqa 7,ѐ.*ӜlNPmL!m9 >FCIvB0Kn[z+-ls(\PkuBcr[ p}k)>ʗ%&Ƨ p5+3X ^AETWBKq#ԙ.C,?U1nDd!fTSvXGH;݈(ow\y(t-2PKGԅl53kZyo.)Ox|=C?FB7?TdMi=Pmjog}5ow?C dߋ3|? s3eGˉ?@GD?$,kdh<14lfy%TűP 6po*%yP44Z] [ x{U;PoqG^4H~>[ ^bIa7Tjҷt[G'aiá)wGy[䯞/~-ǩx;kʗGGO= 4H`hkX'0QlEQշh*{,ѪD|439ꕶ\dC]vzjVnd%|Q 1:\d$TqF⭠Kf㕓w>H5ro)-Vxuv/ UĒw$:qT87r[=qutbH42 r]=a͝ J<$t H2ZÕ7T8&]dw=S5|ΰd]8mȺ!zYs H[usc=F\2bzdp1ӓEGJ,TS db' f^Q':% =K[Brm'lt.CI8NI&'&̔LSѸBI)<S ߪjѸCQ{d7yVac=؞ i Z rH[-Soh Ji#=kidmk[=9ZXY4CmLM,%RFi-`2wB[XzYW_ ݺKtVlcj^,v5=:8n4I)r1x{= ^:]DtOUqobgn7/\\\.핫vwqQܫ.(9녠:-2z-́\|}.@od.'=%rξJbFh^e 9`=SR4q=,n -l6GCm&3d G.?E&ݗE߇GodI'6ʘ15#$PIV+rlmcb'`족c zlRI(b1 ,5B)$Z~t2Δ'{Z6z*!>!zL0F1UW NHOuϯtD:|uO~P.մP&z|UC&'}BȽ*%>]Dllygto(Lp2lg-d mT:o>;:JП;}2rp¹mM<3[Ot^*ҥhn+`;3Br Hc쪂O5\ik)ꍮGBj@zZuC9G㑔7#jCƊROn]hMizJ`- I j&EDm&8|\ְ6(@n>\BI4}SeXUiI{15^*_LAl윥, { md4d DT9ej2b:QZ7p`J`?/Z_ }@U 8;oE3Okyh\+eq?u/N2Q54y[]lt&F%bBy*O7LH͌r|4y"P $. DĴc{$ke>%fLBvTx [U//7X3Zhg|lٶ8ݯ(uYsXXg7GA˾tSy1VVX<}צ YCctΗAo'ja$Ez#`kPf&vF,4L[N.Kqx`n W}\3(Yk_b~g/rK^ݗY9W| !~F 9ܮϨ|xdf$蛣c]P֭-gZq:4F8U +ul$ԷEN?/!*PWQ,q|.' UMX !Uf[^"H-9dQy<}Ny&DKdΙ\ .Û-Ѻ-xP|!9snq8GQhu:P4'z29 cKƋA K۪`4|qЧxACN>%=k0d\T  J)7D&@<$: D#ֲ8Z_#֎* 5(-wsj^!bK'_fN )O'9uItU.$o15+5:w甖S9%UZJuobUчAhiS )]F^NRjˉ'$-I&d=J9ʼ#Jr$t&D{zY6蹱2rQÜza-iEp1tLTpRm`b 3DP=לuKuư* pp8 ;aԀ)twUPrOdفϜIuK(y%#r&M~e['`"cO@4|Ίl 7O*z$ŲնÐM̉ 0 ^lTYTdGkfᶲۣ>Hg=Cmf'76,y@'4}`1IoTRƹM){_A5-".Z LRMW-n.}KPeUXfevj"Yfq%*J|J8 BbaBGtkR{ʗLJ7VmN-HQ kuD9{e{B`.2(DMLR pZۜS͑襶&f7Rs{̞-]5#9ӅݗR 'V,*l6VD1ʹpt;؁s" |Fsk02qؾ浇::8a˼ׅkIyu>e!fz,2e<MM{u}S gET\Ð5!tExV3{f`nؘj@] R@SJji͵DzSnĒE0$bqb|kNEsV.ThߔWaV7}%1Vc?\. f~ܯ%b]=c>F=i׏W \? چL}R+tl{'41$vŊrbkf~w9e6YS)\7Mis45؃M '}UDf\&j$Mz1f3kN:4tMI sS+iČ{y}P%O7ku|Ts3-M;o /A>p*fuLodih/T̋%vASZ< K.AV\6sM1:bWU^څ7qPXq4o RRz9(TK)4td{'`koӴC㵿jH?E/7lHlEGY\_w{d"?CG7og2j.!DzCG45:C${ÑdT w9bNc=OPIZ;6uX)MdV&Nj]g'1#}Py+.D? a/{^vTFbqLYlm'"'>WC-}$8,/|9td?ZojlMQgP GA`ϏKe?kM =4wpDKS2<7Q38ܠ?Xٴ0=]o|ƺJSYQώ(o_xUһKq-_Ĩ}, h˞>=霯=hHx.raE׸O:gi'VtיfnJj}zRE%6pHe;][UH~ )iYN5׷a`dӼ8K8sX:_4Qӹ/VŚLs1T@ϴ`.%.%t}.ߢoo7&xf"5|RO8U:AU QI[u(!P[{3[-]QsfǥTMmkx1}n8qe9htF1&)!$$7 8=%hh# 7\TdPcuiENE쭎/@!!iDCc $9%UЂ.$`pzd$z4`.I[RQj~#k}8 ;Y:X{xvEYrQU_Εj<)u:.\TjW[=q$2ԗuUF- AW:G|]7NJkEӔEQ snU"EHbEQB⋊-d=&8Ok% "X"">-Ċ#d\46(CEޯKtEqSaphQt4Ӂ1"%ܔ@\S\*.L2Q\M~|$?ULuTQ>dd*%U[*\; LH|)%'$~V2-F@

h#p>ts^0i#$i~cCi-` lv`cؓM#4>Z=0,̦hEI%% WLiSYS4 R[$ xd5䐋}ljcW3.}V;UHkY[;{rx&y=xTr|VWMsSJ\MԌ$7(ttdʖ݁AIʚpN} -)Ґmz&E,?ʞ~w3y=Ud߲\*p)#R RanMa ߇|YXX$qXeʐ꧷](͐buwCqtԎ# D]ʹ:Q V6?DCbxϲl6؍WGqGCB-d8ʢq }C@ }T* ̡L:\ZU -mSp .WHX(_ԅtQ}4mhLH _ ܡcM! $ϐ@de6P씓9l7H[{F~W w&i}INvt8mmhtx|ҝϨdm,DF:J!w\3Gm j/z2]NH:MWn.7](Zj7lV䞫W$:UEq<ey\aH6P"/@IU,0}$ēܫ0iIH#`q7Uk)mԎv 31"6# ˿ ~RlIP##`TMY 7`M)J5P&p?Ēw,N./|}EXvI$4 &kaUb*.(L5>Ţ8tχdQ=Vh'GUDZ5 _SXWERu#Մ:eMe߾U[Gv6_sk~}^U%9T>\W5pUvoYuz耵jȼo%q1fB믴~~G{}vitM©/SZkMt2NGpLY}oC$vk5y^ZoNA'7 oSzo|RY%-\QX+.7UQbo?V//Ϯo*!~lc\;)~,?O 4ϭ2D>]i 5ik^ؚ9tE@j^'UYpe>ԏXEsFŽ UqYHHO%X [G,|xmn=q=Ȧ ǻ)a,b&k+_ ?3<[ ᬬphN?+-WN6\r%pϰ*la;F/oxK^gސ|SG{P;o&ww |i1+(4T1g|yPhlOlc+e |s-&Q·h*<|vުOg?]ju U#Tn1=KnQ/a/G*s+g9l.I?ÊTD}:mjqGGmܣcH$i>"uMC#fG㚱GѴv>R> ~ N{0ѵ\\Ǟl]Ojw6[N?QSZ%}TOPu!3kCCL1RsE]Ŗ.FJ、|N?* ɺ_YrT|)$C־WY%?|#Y^XX+#r"xdLq..ll/LƈJic,K.}ԗ.~ލvFֆDi#"5ɽ8  ˝b+&Ŕ`S8顦fHh Z-Kc!`ISY o_,BzI2R]S; 7S],>ccV-ÀarJ$)$rc8%%*Aފ1ӀX~0 !65ҜF܄ɣ$/{ho)>;wr?jϦa~jzX 2oT["ikvY sұ|__һ{& Y]=TYd.=+pxtz^?nageru-1.4.2/bmusb/000077500000000000000000000000001301561706600141145ustar00rootroot00000000000000nageru-1.4.2/clickable_label.h000066400000000000000000000006731301561706600162330ustar00rootroot00000000000000#ifndef _CLICKABLE_LABEL_H #define _CLICKABLE_LABEL_H 1 // Just like a normal QLabel, except that it can also emit a clicked signal. #include class QMouseEvent; class ClickableLabel : public QLabel { Q_OBJECT public: ClickableLabel(QWidget *parent) : QLabel(parent) {} signals: void clicked(); protected: void mousePressEvent(QMouseEvent *event) override { emit clicked(); } }; #endif // !defined(_CLICKABLE_LABEL_H) nageru-1.4.2/compression_reduction_meter.cpp000066400000000000000000000053401301561706600213230ustar00rootroot00000000000000#include "compression_reduction_meter.h" #include #include #include "piecewise_interpolator.h" #include "vu_common.h" class QPaintEvent; class QResizeEvent; using namespace std; namespace { vector control_points = { { 60.0f, 6.0f }, { 30.0f, 5.0f }, { 18.0f, 4.0f }, { 12.0f, 3.0f }, { 6.0f, 2.0f }, { 3.0f, 1.0f }, { 0.0f, 0.0f } }; PiecewiseInterpolator interpolator(control_points); } // namespace CompressionReductionMeter::CompressionReductionMeter(QWidget *parent) : QWidget(parent) { } void CompressionReductionMeter::resizeEvent(QResizeEvent *event) { recalculate_pixmaps(); } void CompressionReductionMeter::paintEvent(QPaintEvent *event) { QPainter painter(this); float level_db; { unique_lock lock(level_mutex); level_db = this->level_db; } int on_pos = lrint(db_to_pos(level_db)); QRect on_rect(0, 0, width(), on_pos); QRect off_rect(0, on_pos, width(), height()); painter.drawPixmap(on_rect, on_pixmap, on_rect); painter.drawPixmap(off_rect, off_pixmap, off_rect); } void CompressionReductionMeter::recalculate_pixmaps() { constexpr int y_offset = text_box_height / 2; constexpr int text_margin = 5; float margin = 0.5 * (width() - meter_width); on_pixmap = QPixmap(width(), height()); QPainter on_painter(&on_pixmap); on_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(on_painter, width(), meter_height(), margin, 2.0, true, min_level, max_level, /*flip=*/true, y_offset); draw_scale(&on_painter, 0.5 * width() + 0.5 * meter_width + text_margin); off_pixmap = QPixmap(width(), height()); QPainter off_painter(&off_pixmap); off_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(off_painter, width(), meter_height(), margin, 2.0, false, min_level, max_level, /*flip=*/true, y_offset); draw_scale(&off_painter, 0.5 * width() + 0.5 * meter_width + text_margin); } void CompressionReductionMeter::draw_scale(QPainter *painter, int x_pos) { QFont font; font.setPointSize(8); painter->setPen(Qt::black); painter->setFont(font); for (size_t i = 0; i < control_points.size(); ++i) { char buf[256]; snprintf(buf, 256, "%.0f", control_points[i].db_value); double y = db_to_pos(control_points[i].db_value); painter->drawText(QRect(x_pos, y - text_box_height / 2, text_box_width, text_box_height), Qt::AlignCenter | Qt::AlignVCenter, buf); } } double CompressionReductionMeter::db_to_pos(double level_db) const { float value = interpolator.db_to_fraction(level_db); return height() - lufs_to_pos(value, meter_height(), min_level, max_level) - text_box_height / 2; } int CompressionReductionMeter::meter_height() const { return height() - text_box_height; } nageru-1.4.2/compression_reduction_meter.h000066400000000000000000000025411301561706600207700ustar00rootroot00000000000000#ifndef COMPRESSION_REDUCTION_METER_H #define COMPRESSION_REDUCTION_METER_H // A meter that goes downwards instead of upwards, and has a non-linear scale. #include #include #include #include #include #include "piecewise_interpolator.h" class QObject; class QPaintEvent; class QResizeEvent; class CompressionReductionMeter : public QWidget { Q_OBJECT public: CompressionReductionMeter(QWidget *parent); void set_reduction_db(float level_db) { std::unique_lock lock(level_mutex); this->level_db = level_db; QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); } private: void resizeEvent(QResizeEvent *event) override; void paintEvent(QPaintEvent *event) override; void recalculate_pixmaps(); void draw_scale(QPainter *painter, int x_pos); double db_to_pos(double db) const; int meter_height() const; std::mutex level_mutex; float level_db = 0.0f; static constexpr float min_level = 0.0f; // Must match control_points (in the .cpp file). static constexpr float max_level = 6.0f; // Same. static constexpr int meter_width = 20; // Size of the text box. The meter will be shrunk to make room for the text box // (half the height) on both sides. static constexpr int text_box_width = 15; static constexpr int text_box_height = 10; QPixmap on_pixmap, off_pixmap; }; #endif nageru-1.4.2/context.cpp000066400000000000000000000071721301561706600152030ustar00rootroot00000000000000#include #include extern "C" { #include } #include #include #include #include #include #include QGLWidget *global_share_widget = nullptr; bool using_egl = false; using namespace std; namespace { string get_pci_device_name(const char *node_name) { char vendor_path[256]; snprintf(vendor_path, sizeof(vendor_path), "/sys/class/drm/%s/device/vendor", node_name); FILE *vendor_file = fopen(vendor_path, "r"); if (vendor_file == nullptr) { return "could not look up vendor ID"; } int vendor; if (fscanf(vendor_file, "%i", &vendor) != 1) { fclose(vendor_file); return "could not parse vendor ID"; } fclose(vendor_file); char device_path[256]; snprintf(device_path, sizeof(device_path), "/sys/class/drm/%s/device/device", node_name); FILE *device_file = fopen(device_path, "r"); if (device_file == nullptr) { return "could not look up device ID"; } int device; if (fscanf(device_file, "%i", &device) != 1) { fclose(device_file); return "could not parse device ID"; } fclose(device_file); pci_access *pci = pci_alloc(); if (pci == nullptr) { return "could not init libpci"; } pci_init(pci); char buf[256]; const char *name = pci_lookup_name(pci, buf, sizeof(buf), PCI_LOOKUP_VENDOR | PCI_LOOKUP_DEVICE, vendor, device); pci_cleanup(pci); if (name == nullptr) { snprintf(buf, sizeof(buf), "%04x:%04x", vendor, device); } return buf; } void print_available_drm_nodes() { glob_t g; int err = glob("/dev/dri/renderD*", 0, nullptr, &g); // TODO: Accept /dev/dri/card*, too? if (err != 0) { fprintf(stderr, "Couldn't list render nodes (%s).\n", strerror(errno)); return; } if (g.gl_pathc == 0) { fprintf(stderr, "\n"); fprintf(stderr, "No render nodes found in /dev/dri.\n"); } else { fprintf(stderr, "Available devices (these may or may not support VA-API encoding):\n\n"); for (size_t i = 0; i < g.gl_pathc; ++i) { const char *node_name = basename(g.gl_pathv[i]); fprintf(stderr, " %s (%s)\n", g.gl_pathv[i], get_pci_device_name(node_name).c_str()); } } globfree(&g); } } // namespace QSurface *create_surface(const QSurfaceFormat &format) { QOffscreenSurface *surface = new QOffscreenSurface; surface->setFormat(format); surface->create(); if (!surface->isValid()) { fprintf(stderr, "ERROR: surface not valid!\n"); if (using_egl) { fprintf(stderr, "\n\n"); fprintf(stderr, "OpenGL initialization failed. This is most likely because your driver does not\n"); fprintf(stderr, "support EGL (e.g. NVIDIA drivers). You can turn off EGL by specifying the\n"); fprintf(stderr, "VA-API path directly, assuming you have another GPU with VA-API support\n"); fprintf(stderr, "(typically an integrated Intel GPU -- note that it you might need to manually\n"); fprintf(stderr, "enable it in the BIOS, as it might be turned off when a discrete GPU is detected).\n"); fprintf(stderr, "\n"); fprintf(stderr, "Specify the VA-API device using “--va-display /dev/dri/”.\n"); print_available_drm_nodes(); } exit(1); } return surface; } QSurface *create_surface_with_same_format(const QSurface *surface) { return create_surface(surface->format()); } QOpenGLContext *create_context(const QSurface *surface) { QOpenGLContext *context = new QOpenGLContext; context->setShareContext(global_share_widget->context()->contextHandle()); context->setFormat(surface->format()); context->create(); return context; } bool make_current(QOpenGLContext *context, QSurface *surface) { return context->makeCurrent(surface); } void delete_context(QOpenGLContext *context) { delete context; } nageru-1.4.2/context.h000066400000000000000000000010351301561706600146400ustar00rootroot00000000000000 // Needs to be in its own file because Qt and libepoxy seemingly don't coexist well // within the same file. class QSurface; class QOpenGLContext; class QSurfaceFormat; class QGLWidget; extern bool using_egl; extern QGLWidget *global_share_widget; QSurface *create_surface(const QSurfaceFormat &format); QSurface *create_surface_with_same_format(const QSurface *surface); QOpenGLContext *create_context(const QSurface *surface); bool make_current(QOpenGLContext *context, QSurface *surface); void delete_context(QOpenGLContext *context); nageru-1.4.2/correlation_measurer.cpp000066400000000000000000000043111301561706600177330ustar00rootroot00000000000000// Adapted from Adriaensen's project Zita-mu1 (as of January 2016). // Original copyright follows: // // Copyright (C) 2008-2015 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 3 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program. If not, see . #include "correlation_measurer.h" #include #include #include using namespace std; CorrelationMeasurer::CorrelationMeasurer(unsigned sample_rate, float lowpass_cutoff_hz, float falloff_seconds) : w1(2.0 * M_PI * lowpass_cutoff_hz / sample_rate), w2(1.0 / (falloff_seconds * sample_rate)) { } void CorrelationMeasurer::reset() { zl = zr = zll = zlr = zrr = 0.0f; } void CorrelationMeasurer::process_samples(const std::vector &samples) { assert(samples.size() % 2 == 0); // The compiler isn't always happy about modifying members, // since it doesn't always know they can't alias on . // Help it out a bit. float l = zl, r = zr, ll = zll, lr = zlr, rr = zrr; const float w1c = w1, w2c = w2; for (size_t i = 0; i < samples.size(); i += 2) { // The 1e-15f epsilon is to avoid denormals. // TODO: Just set the SSE flush-to-zero flags instead. l += w1c * (samples[i + 0] - l) + 1e-15f; r += w1c * (samples[i + 1] - r) + 1e-15f; lr += w2c * (l * r - lr); ll += w2c * (l * l - ll); rr += w2c * (r * r - rr); } zl = l; zr = r; zll = ll; zlr = lr; zrr = rr; } float CorrelationMeasurer::get_correlation() const { // The 1e-12f epsilon is to avoid division by zero. // zll and zrr are both always non-negative, so we do not risk negative values. return zlr / sqrt(zll * zrr + 1e-12f); } nageru-1.4.2/correlation_measurer.h000066400000000000000000000042271301561706600174060ustar00rootroot00000000000000#ifndef _CORRELATION_MEASURER_H #define _CORRELATION_MEASURER_H 1 // Measurement of left/right stereo correlation. +1 is pure mono // (okay but not ideal), 0 is no correlation (usually bad, unless // it is due to silence), strongly negative values means inverted // phase (bad). Typical values for e.g. music would be somewhere // around +0.7, although you can expect it to vary a bit. // // This is, of course, based on the regular Pearson correlation, // where µ_L and µ_R is taken to be 0 (ie., no DC offset). It is // filtered through a simple IIR filter so that older values are // weighed less than newer, depending on . // // // Adapted from Adriaensen's project Zita-mu1 (as of January 2016). // Original copyright follows: // // Copyright (C) 2008-2015 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 3 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program. If not, see . #include class CorrelationMeasurer { public: CorrelationMeasurer(unsigned sample_rate, float lowpass_cutoff_hz = 1000.0f, float falloff_seconds = 0.150f); void process_samples(const std::vector &samples); // Taken to be stereo, interleaved. void reset(); float get_correlation() const; private: float w1, w2; // Filtered values of left and right channel, respectively. float zl = 0.0f, zr = 0.0f; // Filtered values of l², r² and lr (where l and r are the filtered // versions, given by zl and zr). Together, they make up what we need // to calculate the correlation. float zll = 0.0f, zlr = 0.0f, zrr = 0.0f; }; #endif // !defined(_CORRELATION_MEASURER_H) nageru-1.4.2/correlation_meter.cpp000066400000000000000000000032241301561706600172260ustar00rootroot00000000000000#include "correlation_meter.h" #include #include #include #include #include #include class QPaintEvent; class QResizeEvent; using namespace std; CorrelationMeter::CorrelationMeter(QWidget *parent) : QWidget(parent) { } void CorrelationMeter::resizeEvent(QResizeEvent *event) { on_pixmap = QPixmap(width(), height()); QPainter on_painter(&on_pixmap); QLinearGradient on(0, 0, width(), 0); on.setColorAt(0.0f, QColor(255, 0, 0)); on.setColorAt(0.5f, QColor(255, 255, 0)); on.setColorAt(0.8f, QColor(0, 255, 0)); on.setColorAt(0.95f, QColor(255, 255, 0)); on_painter.fillRect(0, 0, width(), height(), Qt::black); on_painter.fillRect(1, 1, width() - 2, height() - 2, on); off_pixmap = QPixmap(width(), height()); QPainter off_painter(&off_pixmap); QLinearGradient off(0, 0, width(), 0); off.setColorAt(0.0f, QColor(127, 0, 0)); off.setColorAt(0.5f, QColor(127, 127, 0)); off.setColorAt(0.8f, QColor(0, 127, 0)); off.setColorAt(0.95f, QColor(127, 127, 0)); off_painter.fillRect(0, 0, width(), height(), Qt::black); off_painter.fillRect(1, 1, width() - 2, height() - 2, off); } void CorrelationMeter::paintEvent(QPaintEvent *event) { QPainter painter(this); float correlation; { unique_lock lock(correlation_mutex); correlation = this->correlation; } // Just in case. correlation = std::min(std::max(correlation, -1.0f), 1.0f); int pos = 3 + lrintf(0.5f * (correlation + 1.0f) * (width() - 6)); QRect off_rect(0, 0, width(), height()); QRect on_rect(pos - 2, 0, 5, height()); painter.drawPixmap(off_rect, off_pixmap, off_rect); painter.drawPixmap(on_rect, on_pixmap, on_rect); } nageru-1.4.2/correlation_meter.h000066400000000000000000000012711301561706600166730ustar00rootroot00000000000000#ifndef CORRELATION_METER_H #define CORRELATION_METER_H #include #include #include #include class QObject; class QPaintEvent; class QResizeEvent; class CorrelationMeter : public QWidget { Q_OBJECT public: CorrelationMeter(QWidget *parent); void set_correlation(float correlation) { std::unique_lock lock(correlation_mutex); this->correlation = correlation; QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); } private: void resizeEvent(QResizeEvent *event) override; void paintEvent(QPaintEvent *event) override; std::mutex correlation_mutex; float correlation = 0.0f; QPixmap on_pixmap, off_pixmap; }; #endif nageru-1.4.2/db.h000066400000000000000000000004141301561706600135410ustar00rootroot00000000000000#ifndef _DB_H #define _DB_H 1 // Utility routines for working with decibels. #include static inline double from_db(double db) { return pow(10.0, db / 20.0); } static inline double to_db(double val) { return 20.0 * log10(val); } #endif // !defined(_DB_H) nageru-1.4.2/decklink/000077500000000000000000000000001301561706600145705ustar00rootroot00000000000000nageru-1.4.2/decklink/DeckLinkAPI.h000077500000000000000000001355631301561706600167770ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPI_H #define BMD_DECKLINKAPI_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif /* DeckLink API */ #include #include "LinuxCOM.h" #include "DeckLinkAPITypes.h" #include "DeckLinkAPIModes.h" #include "DeckLinkAPIDiscovery.h" #include "DeckLinkAPIConfiguration.h" #include "DeckLinkAPIDeckControl.h" #define BLACKMAGIC_DECKLINK_API_MAGIC 1 // Type Declarations // Interface ID Declarations BMD_CONST REFIID IID_IDeckLinkVideoOutputCallback = /* 20AA5225-1958-47CB-820B-80A8D521A6EE */ {0x20,0xAA,0x52,0x25,0x19,0x58,0x47,0xCB,0x82,0x0B,0x80,0xA8,0xD5,0x21,0xA6,0xEE}; BMD_CONST REFIID IID_IDeckLinkInputCallback = /* DD04E5EC-7415-42AB-AE4A-E80C4DFC044A */ {0xDD,0x04,0xE5,0xEC,0x74,0x15,0x42,0xAB,0xAE,0x4A,0xE8,0x0C,0x4D,0xFC,0x04,0x4A}; BMD_CONST REFIID IID_IDeckLinkEncoderInputCallback = /* ACF13E61-F4A0-4974-A6A7-59AFF6268B31 */ {0xAC,0xF1,0x3E,0x61,0xF4,0xA0,0x49,0x74,0xA6,0xA7,0x59,0xAF,0xF6,0x26,0x8B,0x31}; BMD_CONST REFIID IID_IDeckLinkMemoryAllocator = /* B36EB6E7-9D29-4AA8-92EF-843B87A289E8 */ {0xB3,0x6E,0xB6,0xE7,0x9D,0x29,0x4A,0xA8,0x92,0xEF,0x84,0x3B,0x87,0xA2,0x89,0xE8}; BMD_CONST REFIID IID_IDeckLinkAudioOutputCallback = /* 403C681B-7F46-4A12-B993-2BB127084EE6 */ {0x40,0x3C,0x68,0x1B,0x7F,0x46,0x4A,0x12,0xB9,0x93,0x2B,0xB1,0x27,0x08,0x4E,0xE6}; BMD_CONST REFIID IID_IDeckLinkIterator = /* 50FB36CD-3063-4B73-BDBB-958087F2D8BA */ {0x50,0xFB,0x36,0xCD,0x30,0x63,0x4B,0x73,0xBD,0xBB,0x95,0x80,0x87,0xF2,0xD8,0xBA}; BMD_CONST REFIID IID_IDeckLinkAPIInformation = /* 7BEA3C68-730D-4322-AF34-8A7152B532A4 */ {0x7B,0xEA,0x3C,0x68,0x73,0x0D,0x43,0x22,0xAF,0x34,0x8A,0x71,0x52,0xB5,0x32,0xA4}; BMD_CONST REFIID IID_IDeckLinkOutput = /* CC5C8A6E-3F2F-4B3A-87EA-FD78AF300564 */ {0xCC,0x5C,0x8A,0x6E,0x3F,0x2F,0x4B,0x3A,0x87,0xEA,0xFD,0x78,0xAF,0x30,0x05,0x64}; BMD_CONST REFIID IID_IDeckLinkInput = /* AF22762B-DFAC-4846-AA79-FA8883560995 */ {0xAF,0x22,0x76,0x2B,0xDF,0xAC,0x48,0x46,0xAA,0x79,0xFA,0x88,0x83,0x56,0x09,0x95}; BMD_CONST REFIID IID_IDeckLinkEncoderInput = /* 270587DA-6B7D-42E7-A1F0-6D853F581185 */ {0x27,0x05,0x87,0xDA,0x6B,0x7D,0x42,0xE7,0xA1,0xF0,0x6D,0x85,0x3F,0x58,0x11,0x85}; BMD_CONST REFIID IID_IDeckLinkVideoFrame = /* 3F716FE0-F023-4111-BE5D-EF4414C05B17 */ {0x3F,0x71,0x6F,0xE0,0xF0,0x23,0x41,0x11,0xBE,0x5D,0xEF,0x44,0x14,0xC0,0x5B,0x17}; BMD_CONST REFIID IID_IDeckLinkMutableVideoFrame = /* 69E2639F-40DA-4E19-B6F2-20ACE815C390 */ {0x69,0xE2,0x63,0x9F,0x40,0xDA,0x4E,0x19,0xB6,0xF2,0x20,0xAC,0xE8,0x15,0xC3,0x90}; BMD_CONST REFIID IID_IDeckLinkVideoFrame3DExtensions = /* DA0F7E4A-EDC7-48A8-9CDD-2DB51C729CD7 */ {0xDA,0x0F,0x7E,0x4A,0xED,0xC7,0x48,0xA8,0x9C,0xDD,0x2D,0xB5,0x1C,0x72,0x9C,0xD7}; BMD_CONST REFIID IID_IDeckLinkVideoInputFrame = /* 05CFE374-537C-4094-9A57-680525118F44 */ {0x05,0xCF,0xE3,0x74,0x53,0x7C,0x40,0x94,0x9A,0x57,0x68,0x05,0x25,0x11,0x8F,0x44}; BMD_CONST REFIID IID_IDeckLinkVideoFrameAncillary = /* 732E723C-D1A4-4E29-9E8E-4A88797A0004 */ {0x73,0x2E,0x72,0x3C,0xD1,0xA4,0x4E,0x29,0x9E,0x8E,0x4A,0x88,0x79,0x7A,0x00,0x04}; BMD_CONST REFIID IID_IDeckLinkEncoderPacket = /* B693F36C-316E-4AF1-B6C2-F389A4BCA620 */ {0xB6,0x93,0xF3,0x6C,0x31,0x6E,0x4A,0xF1,0xB6,0xC2,0xF3,0x89,0xA4,0xBC,0xA6,0x20}; BMD_CONST REFIID IID_IDeckLinkEncoderVideoPacket = /* 4E7FD944-E8C7-4EAC-B8C0-7B77F80F5AE0 */ {0x4E,0x7F,0xD9,0x44,0xE8,0xC7,0x4E,0xAC,0xB8,0xC0,0x7B,0x77,0xF8,0x0F,0x5A,0xE0}; BMD_CONST REFIID IID_IDeckLinkEncoderAudioPacket = /* 49E8EDC8-693B-4E14-8EF6-12C658F5A07A */ {0x49,0xE8,0xED,0xC8,0x69,0x3B,0x4E,0x14,0x8E,0xF6,0x12,0xC6,0x58,0xF5,0xA0,0x7A}; BMD_CONST REFIID IID_IDeckLinkH265NALPacket = /* 639C8E0B-68D5-4BDE-A6D4-95F3AEAFF2E7 */ {0x63,0x9C,0x8E,0x0B,0x68,0xD5,0x4B,0xDE,0xA6,0xD4,0x95,0xF3,0xAE,0xAF,0xF2,0xE7}; BMD_CONST REFIID IID_IDeckLinkAudioInputPacket = /* E43D5870-2894-11DE-8C30-0800200C9A66 */ {0xE4,0x3D,0x58,0x70,0x28,0x94,0x11,0xDE,0x8C,0x30,0x08,0x00,0x20,0x0C,0x9A,0x66}; BMD_CONST REFIID IID_IDeckLinkScreenPreviewCallback = /* B1D3F49A-85FE-4C5D-95C8-0B5D5DCCD438 */ {0xB1,0xD3,0xF4,0x9A,0x85,0xFE,0x4C,0x5D,0x95,0xC8,0x0B,0x5D,0x5D,0xCC,0xD4,0x38}; BMD_CONST REFIID IID_IDeckLinkGLScreenPreviewHelper = /* 504E2209-CAC7-4C1A-9FB4-C5BB6274D22F */ {0x50,0x4E,0x22,0x09,0xCA,0xC7,0x4C,0x1A,0x9F,0xB4,0xC5,0xBB,0x62,0x74,0xD2,0x2F}; BMD_CONST REFIID IID_IDeckLinkNotificationCallback = /* B002A1EC-070D-4288-8289-BD5D36E5FF0D */ {0xB0,0x02,0xA1,0xEC,0x07,0x0D,0x42,0x88,0x82,0x89,0xBD,0x5D,0x36,0xE5,0xFF,0x0D}; BMD_CONST REFIID IID_IDeckLinkNotification = /* 0A1FB207-E215-441B-9B19-6FA1575946C5 */ {0x0A,0x1F,0xB2,0x07,0xE2,0x15,0x44,0x1B,0x9B,0x19,0x6F,0xA1,0x57,0x59,0x46,0xC5}; BMD_CONST REFIID IID_IDeckLinkAttributes = /* ABC11843-D966-44CB-96E2-A1CB5D3135C4 */ {0xAB,0xC1,0x18,0x43,0xD9,0x66,0x44,0xCB,0x96,0xE2,0xA1,0xCB,0x5D,0x31,0x35,0xC4}; BMD_CONST REFIID IID_IDeckLinkKeyer = /* 89AFCAF5-65F8-421E-98F7-96FE5F5BFBA3 */ {0x89,0xAF,0xCA,0xF5,0x65,0xF8,0x42,0x1E,0x98,0xF7,0x96,0xFE,0x5F,0x5B,0xFB,0xA3}; BMD_CONST REFIID IID_IDeckLinkVideoConversion = /* 3BBCB8A2-DA2C-42D9-B5D8-88083644E99A */ {0x3B,0xBC,0xB8,0xA2,0xDA,0x2C,0x42,0xD9,0xB5,0xD8,0x88,0x08,0x36,0x44,0xE9,0x9A}; BMD_CONST REFIID IID_IDeckLinkDeviceNotificationCallback = /* 4997053B-0ADF-4CC8-AC70-7A50C4BE728F */ {0x49,0x97,0x05,0x3B,0x0A,0xDF,0x4C,0xC8,0xAC,0x70,0x7A,0x50,0xC4,0xBE,0x72,0x8F}; BMD_CONST REFIID IID_IDeckLinkDiscovery = /* CDBF631C-BC76-45FA-B44D-C55059BC6101 */ {0xCD,0xBF,0x63,0x1C,0xBC,0x76,0x45,0xFA,0xB4,0x4D,0xC5,0x50,0x59,0xBC,0x61,0x01}; /* Enum BMDVideoOutputFlags - Flags to control the output of ancillary data along with video. */ typedef uint32_t BMDVideoOutputFlags; enum _BMDVideoOutputFlags { bmdVideoOutputFlagDefault = 0, bmdVideoOutputVANC = 1 << 0, bmdVideoOutputVITC = 1 << 1, bmdVideoOutputRP188 = 1 << 2, bmdVideoOutputDualStream3D = 1 << 4 }; /* Enum BMDPacketType - Type of packet */ typedef uint32_t BMDPacketType; enum _BMDPacketType { bmdPacketTypeStreamInterruptedMarker = /* 'sint' */ 0x73696E74, // A packet of this type marks the time when a video stream was interrupted, for example by a disconnected cable bmdPacketTypeStreamData = /* 'sdat' */ 0x73646174 // Regular stream data }; /* Enum BMDFrameFlags - Frame flags */ typedef uint32_t BMDFrameFlags; enum _BMDFrameFlags { bmdFrameFlagDefault = 0, bmdFrameFlagFlipVertical = 1 << 0, /* Flags that are applicable only to instances of IDeckLinkVideoInputFrame */ bmdFrameHasNoInputSource = 1 << 31 }; /* Enum BMDVideoInputFlags - Flags applicable to video input */ typedef uint32_t BMDVideoInputFlags; enum _BMDVideoInputFlags { bmdVideoInputFlagDefault = 0, bmdVideoInputEnableFormatDetection = 1 << 0, bmdVideoInputDualStream3D = 1 << 1 }; /* Enum BMDVideoInputFormatChangedEvents - Bitmask passed to the VideoInputFormatChanged notification to identify the properties of the input signal that have changed */ typedef uint32_t BMDVideoInputFormatChangedEvents; enum _BMDVideoInputFormatChangedEvents { bmdVideoInputDisplayModeChanged = 1 << 0, bmdVideoInputFieldDominanceChanged = 1 << 1, bmdVideoInputColorspaceChanged = 1 << 2 }; /* Enum BMDDetectedVideoInputFormatFlags - Flags passed to the VideoInputFormatChanged notification to describe the detected video input signal */ typedef uint32_t BMDDetectedVideoInputFormatFlags; enum _BMDDetectedVideoInputFormatFlags { bmdDetectedVideoInputYCbCr422 = 1 << 0, bmdDetectedVideoInputRGB444 = 1 << 1, bmdDetectedVideoInputDualStream3D = 1 << 2 }; /* Enum BMDDeckLinkCapturePassthroughMode - Enumerates whether the video output is electrically connected to the video input or if the clean switching mode is enabled */ typedef uint32_t BMDDeckLinkCapturePassthroughMode; enum _BMDDeckLinkCapturePassthroughMode { bmdDeckLinkCapturePassthroughModeDirect = /* 'pdir' */ 0x70646972, bmdDeckLinkCapturePassthroughModeCleanSwitch = /* 'pcln' */ 0x70636C6E }; /* Enum BMDOutputFrameCompletionResult - Frame Completion Callback */ typedef uint32_t BMDOutputFrameCompletionResult; enum _BMDOutputFrameCompletionResult { bmdOutputFrameCompleted, bmdOutputFrameDisplayedLate, bmdOutputFrameDropped, bmdOutputFrameFlushed }; /* Enum BMDReferenceStatus - GenLock input status */ typedef uint32_t BMDReferenceStatus; enum _BMDReferenceStatus { bmdReferenceNotSupportedByHardware = 1 << 0, bmdReferenceLocked = 1 << 1 }; /* Enum BMDAudioFormat - Audio Format */ typedef uint32_t BMDAudioFormat; enum _BMDAudioFormat { bmdAudioFormatPCM = /* 'lpcm' */ 0x6C70636D // Linear signed PCM samples }; /* Enum BMDAudioSampleRate - Audio sample rates supported for output/input */ typedef uint32_t BMDAudioSampleRate; enum _BMDAudioSampleRate { bmdAudioSampleRate48kHz = 48000 }; /* Enum BMDAudioSampleType - Audio sample sizes supported for output/input */ typedef uint32_t BMDAudioSampleType; enum _BMDAudioSampleType { bmdAudioSampleType16bitInteger = 16, bmdAudioSampleType32bitInteger = 32 }; /* Enum BMDAudioOutputStreamType - Audio output stream type */ typedef uint32_t BMDAudioOutputStreamType; enum _BMDAudioOutputStreamType { bmdAudioOutputStreamContinuous, bmdAudioOutputStreamContinuousDontResample, bmdAudioOutputStreamTimestamped }; /* Enum BMDDisplayModeSupport - Output mode supported flags */ typedef uint32_t BMDDisplayModeSupport; enum _BMDDisplayModeSupport { bmdDisplayModeNotSupported = 0, bmdDisplayModeSupported, bmdDisplayModeSupportedWithConversion }; /* Enum BMDTimecodeFormat - Timecode formats for frame metadata */ typedef uint32_t BMDTimecodeFormat; enum _BMDTimecodeFormat { bmdTimecodeRP188VITC1 = /* 'rpv1' */ 0x72707631, // RP188 timecode where DBB1 equals VITC1 (line 9) bmdTimecodeRP188VITC2 = /* 'rp12' */ 0x72703132, // RP188 timecode where DBB1 equals VITC2 (line 9 for progressive or line 571 for interlaced/PsF) bmdTimecodeRP188LTC = /* 'rplt' */ 0x72706C74, // RP188 timecode where DBB1 equals LTC (line 10) bmdTimecodeRP188Any = /* 'rp18' */ 0x72703138, // For capture: return the first valid timecode in {VITC1, LTC ,VITC2} - For playback: set the timecode as VITC1 bmdTimecodeVITC = /* 'vitc' */ 0x76697463, bmdTimecodeVITCField2 = /* 'vit2' */ 0x76697432, bmdTimecodeSerial = /* 'seri' */ 0x73657269 }; /* Enum BMDAnalogVideoFlags - Analog video display flags */ typedef uint32_t BMDAnalogVideoFlags; enum _BMDAnalogVideoFlags { bmdAnalogVideoFlagCompositeSetup75 = 1 << 0, bmdAnalogVideoFlagComponentBetacamLevels = 1 << 1 }; /* Enum BMDAudioOutputAnalogAESSwitch - Audio output Analog/AESEBU switch */ typedef uint32_t BMDAudioOutputAnalogAESSwitch; enum _BMDAudioOutputAnalogAESSwitch { bmdAudioOutputSwitchAESEBU = /* 'aes ' */ 0x61657320, bmdAudioOutputSwitchAnalog = /* 'anlg' */ 0x616E6C67 }; /* Enum BMDVideoOutputConversionMode - Video/audio conversion mode */ typedef uint32_t BMDVideoOutputConversionMode; enum _BMDVideoOutputConversionMode { bmdNoVideoOutputConversion = /* 'none' */ 0x6E6F6E65, bmdVideoOutputLetterboxDownconversion = /* 'ltbx' */ 0x6C746278, bmdVideoOutputAnamorphicDownconversion = /* 'amph' */ 0x616D7068, bmdVideoOutputHD720toHD1080Conversion = /* '720c' */ 0x37323063, bmdVideoOutputHardwareLetterboxDownconversion = /* 'HWlb' */ 0x48576C62, bmdVideoOutputHardwareAnamorphicDownconversion = /* 'HWam' */ 0x4857616D, bmdVideoOutputHardwareCenterCutDownconversion = /* 'HWcc' */ 0x48576363, bmdVideoOutputHardware720p1080pCrossconversion = /* 'xcap' */ 0x78636170, bmdVideoOutputHardwareAnamorphic720pUpconversion = /* 'ua7p' */ 0x75613770, bmdVideoOutputHardwareAnamorphic1080iUpconversion = /* 'ua1i' */ 0x75613169, bmdVideoOutputHardwareAnamorphic149To720pUpconversion = /* 'u47p' */ 0x75343770, bmdVideoOutputHardwareAnamorphic149To1080iUpconversion = /* 'u41i' */ 0x75343169, bmdVideoOutputHardwarePillarbox720pUpconversion = /* 'up7p' */ 0x75703770, bmdVideoOutputHardwarePillarbox1080iUpconversion = /* 'up1i' */ 0x75703169 }; /* Enum BMDVideoInputConversionMode - Video input conversion mode */ typedef uint32_t BMDVideoInputConversionMode; enum _BMDVideoInputConversionMode { bmdNoVideoInputConversion = /* 'none' */ 0x6E6F6E65, bmdVideoInputLetterboxDownconversionFromHD1080 = /* '10lb' */ 0x31306C62, bmdVideoInputAnamorphicDownconversionFromHD1080 = /* '10am' */ 0x3130616D, bmdVideoInputLetterboxDownconversionFromHD720 = /* '72lb' */ 0x37326C62, bmdVideoInputAnamorphicDownconversionFromHD720 = /* '72am' */ 0x3732616D, bmdVideoInputLetterboxUpconversion = /* 'lbup' */ 0x6C627570, bmdVideoInputAnamorphicUpconversion = /* 'amup' */ 0x616D7570 }; /* Enum BMDVideo3DPackingFormat - Video 3D packing format */ typedef uint32_t BMDVideo3DPackingFormat; enum _BMDVideo3DPackingFormat { bmdVideo3DPackingSidebySideHalf = /* 'sbsh' */ 0x73627368, bmdVideo3DPackingLinebyLine = /* 'lbyl' */ 0x6C62796C, bmdVideo3DPackingTopAndBottom = /* 'tabo' */ 0x7461626F, bmdVideo3DPackingFramePacking = /* 'frpk' */ 0x6672706B, bmdVideo3DPackingLeftOnly = /* 'left' */ 0x6C656674, bmdVideo3DPackingRightOnly = /* 'righ' */ 0x72696768 }; /* Enum BMDIdleVideoOutputOperation - Video output operation when not playing video */ typedef uint32_t BMDIdleVideoOutputOperation; enum _BMDIdleVideoOutputOperation { bmdIdleVideoOutputBlack = /* 'blac' */ 0x626C6163, bmdIdleVideoOutputLastFrame = /* 'lafa' */ 0x6C616661, bmdIdleVideoOutputDesktop = /* 'desk' */ 0x6465736B }; /* Enum BMDVideoEncoderFrameCodingMode - Video frame coding mode */ typedef uint32_t BMDVideoEncoderFrameCodingMode; enum _BMDVideoEncoderFrameCodingMode { bmdVideoEncoderFrameCodingModeInter = /* 'inte' */ 0x696E7465, bmdVideoEncoderFrameCodingModeIntra = /* 'intr' */ 0x696E7472 }; /* Enum BMDDNxHRLevel - DNxHR Levels */ typedef uint32_t BMDDNxHRLevel; enum _BMDDNxHRLevel { bmdDNxHRLevelSQ = /* 'dnsq' */ 0x646E7371, bmdDNxHRLevelLB = /* 'dnlb' */ 0x646E6C62, bmdDNxHRLevelHQ = /* 'dnhq' */ 0x646E6871, bmdDNxHRLevelHQX = /* 'dhqx' */ 0x64687178, bmdDNxHRLevel444 = /* 'd444' */ 0x64343434 }; /* Enum BMDLinkConfiguration - Video link configuration */ typedef uint32_t BMDLinkConfiguration; enum _BMDLinkConfiguration { bmdLinkConfigurationSingleLink = /* 'lcsl' */ 0x6C63736C, bmdLinkConfigurationDualLink = /* 'lcdl' */ 0x6C63646C, bmdLinkConfigurationQuadLink = /* 'lcql' */ 0x6C63716C }; /* Enum BMDDeviceInterface - Device interface type */ typedef uint32_t BMDDeviceInterface; enum _BMDDeviceInterface { bmdDeviceInterfacePCI = /* 'pci ' */ 0x70636920, bmdDeviceInterfaceUSB = /* 'usb ' */ 0x75736220, bmdDeviceInterfaceThunderbolt = /* 'thun' */ 0x7468756E }; /* Enum BMDDeckLinkAttributeID - DeckLink Attribute ID */ typedef uint32_t BMDDeckLinkAttributeID; enum _BMDDeckLinkAttributeID { /* Flags */ BMDDeckLinkSupportsInternalKeying = /* 'keyi' */ 0x6B657969, BMDDeckLinkSupportsExternalKeying = /* 'keye' */ 0x6B657965, BMDDeckLinkSupportsHDKeying = /* 'keyh' */ 0x6B657968, BMDDeckLinkSupportsInputFormatDetection = /* 'infd' */ 0x696E6664, BMDDeckLinkHasReferenceInput = /* 'hrin' */ 0x6872696E, BMDDeckLinkHasSerialPort = /* 'hspt' */ 0x68737074, BMDDeckLinkHasAnalogVideoOutputGain = /* 'avog' */ 0x61766F67, BMDDeckLinkCanOnlyAdjustOverallVideoOutputGain = /* 'ovog' */ 0x6F766F67, BMDDeckLinkHasVideoInputAntiAliasingFilter = /* 'aafl' */ 0x6161666C, BMDDeckLinkHasBypass = /* 'byps' */ 0x62797073, BMDDeckLinkSupportsDesktopDisplay = /* 'extd' */ 0x65787464, BMDDeckLinkSupportsClockTimingAdjustment = /* 'ctad' */ 0x63746164, BMDDeckLinkSupportsFullDuplex = /* 'fdup' */ 0x66647570, BMDDeckLinkSupportsFullFrameReferenceInputTimingOffset = /* 'frin' */ 0x6672696E, BMDDeckLinkSupportsSMPTELevelAOutput = /* 'lvla' */ 0x6C766C61, BMDDeckLinkSupportsDualLinkSDI = /* 'sdls' */ 0x73646C73, BMDDeckLinkSupportsQuadLinkSDI = /* 'sqls' */ 0x73716C73, BMDDeckLinkSupportsIdleOutput = /* 'idou' */ 0x69646F75, BMDDeckLinkHasLTCTimecodeInput = /* 'hltc' */ 0x686C7463, /* Integers */ BMDDeckLinkMaximumAudioChannels = /* 'mach' */ 0x6D616368, BMDDeckLinkMaximumAnalogAudioChannels = /* 'aach' */ 0x61616368, BMDDeckLinkNumberOfSubDevices = /* 'nsbd' */ 0x6E736264, BMDDeckLinkSubDeviceIndex = /* 'subi' */ 0x73756269, BMDDeckLinkPersistentID = /* 'peid' */ 0x70656964, BMDDeckLinkTopologicalID = /* 'toid' */ 0x746F6964, BMDDeckLinkVideoOutputConnections = /* 'vocn' */ 0x766F636E, BMDDeckLinkVideoInputConnections = /* 'vicn' */ 0x7669636E, BMDDeckLinkAudioOutputConnections = /* 'aocn' */ 0x616F636E, BMDDeckLinkAudioInputConnections = /* 'aicn' */ 0x6169636E, BMDDeckLinkDeviceBusyState = /* 'dbst' */ 0x64627374, BMDDeckLinkVideoIOSupport = /* 'vios' */ 0x76696F73, // Returns a BMDVideoIOSupport bit field BMDDeckLinkDeckControlConnections = /* 'dccn' */ 0x6463636E, BMDDeckLinkDeviceInterface = /* 'dbus' */ 0x64627573, // Returns a BMDDeviceInterface BMDDeckLinkAudioInputRCAChannelCount = /* 'airc' */ 0x61697263, BMDDeckLinkAudioInputXLRChannelCount = /* 'aixc' */ 0x61697863, BMDDeckLinkAudioOutputRCAChannelCount = /* 'aorc' */ 0x616F7263, BMDDeckLinkAudioOutputXLRChannelCount = /* 'aoxc' */ 0x616F7863, /* Floats */ BMDDeckLinkVideoInputGainMinimum = /* 'vigm' */ 0x7669676D, BMDDeckLinkVideoInputGainMaximum = /* 'vigx' */ 0x76696778, BMDDeckLinkVideoOutputGainMinimum = /* 'vogm' */ 0x766F676D, BMDDeckLinkVideoOutputGainMaximum = /* 'vogx' */ 0x766F6778, BMDDeckLinkMicrophoneInputGainMinimum = /* 'migm' */ 0x6D69676D, BMDDeckLinkMicrophoneInputGainMaximum = /* 'migx' */ 0x6D696778, /* Strings */ BMDDeckLinkSerialPortDeviceName = /* 'slpn' */ 0x736C706E, BMDDeckLinkVendorName = /* 'vndr' */ 0x766E6472, BMDDeckLinkDisplayName = /* 'dspn' */ 0x6473706E, BMDDeckLinkModelName = /* 'mdln' */ 0x6D646C6E }; /* Enum BMDDeckLinkAPIInformationID - DeckLinkAPI information ID */ typedef uint32_t BMDDeckLinkAPIInformationID; enum _BMDDeckLinkAPIInformationID { BMDDeckLinkAPIVersion = /* 'vers' */ 0x76657273 }; /* Enum BMDDeviceBusyState - Current device busy state */ typedef uint32_t BMDDeviceBusyState; enum _BMDDeviceBusyState { bmdDeviceCaptureBusy = 1 << 0, bmdDevicePlaybackBusy = 1 << 1, bmdDeviceSerialPortBusy = 1 << 2 }; /* Enum BMDVideoIOSupport - Device video input/output support */ typedef uint32_t BMDVideoIOSupport; enum _BMDVideoIOSupport { bmdDeviceSupportsCapture = 1 << 0, bmdDeviceSupportsPlayback = 1 << 1 }; /* Enum BMD3DPreviewFormat - Linked Frame preview format */ typedef uint32_t BMD3DPreviewFormat; enum _BMD3DPreviewFormat { bmd3DPreviewFormatDefault = /* 'defa' */ 0x64656661, bmd3DPreviewFormatLeftOnly = /* 'left' */ 0x6C656674, bmd3DPreviewFormatRightOnly = /* 'righ' */ 0x72696768, bmd3DPreviewFormatSideBySide = /* 'side' */ 0x73696465, bmd3DPreviewFormatTopBottom = /* 'topb' */ 0x746F7062 }; /* Enum BMDNotifications - Events that can be subscribed through IDeckLinkNotification */ typedef uint32_t BMDNotifications; enum _BMDNotifications { bmdPreferencesChanged = /* 'pref' */ 0x70726566 }; #if defined(__cplusplus) // Forward Declarations class IDeckLinkVideoOutputCallback; class IDeckLinkInputCallback; class IDeckLinkEncoderInputCallback; class IDeckLinkMemoryAllocator; class IDeckLinkAudioOutputCallback; class IDeckLinkIterator; class IDeckLinkAPIInformation; class IDeckLinkOutput; class IDeckLinkInput; class IDeckLinkEncoderInput; class IDeckLinkVideoFrame; class IDeckLinkMutableVideoFrame; class IDeckLinkVideoFrame3DExtensions; class IDeckLinkVideoInputFrame; class IDeckLinkVideoFrameAncillary; class IDeckLinkEncoderPacket; class IDeckLinkEncoderVideoPacket; class IDeckLinkEncoderAudioPacket; class IDeckLinkH265NALPacket; class IDeckLinkAudioInputPacket; class IDeckLinkScreenPreviewCallback; class IDeckLinkGLScreenPreviewHelper; class IDeckLinkNotificationCallback; class IDeckLinkNotification; class IDeckLinkAttributes; class IDeckLinkKeyer; class IDeckLinkVideoConversion; class IDeckLinkDeviceNotificationCallback; class IDeckLinkDiscovery; /* Interface IDeckLinkVideoOutputCallback - Frame completion callback. */ class IDeckLinkVideoOutputCallback : public IUnknown { public: virtual HRESULT ScheduledFrameCompleted (/* in */ IDeckLinkVideoFrame *completedFrame, /* in */ BMDOutputFrameCompletionResult result) = 0; virtual HRESULT ScheduledPlaybackHasStopped (void) = 0; protected: virtual ~IDeckLinkVideoOutputCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkInputCallback - Frame arrival callback. */ class IDeckLinkInputCallback : public IUnknown { public: virtual HRESULT VideoInputFormatChanged (/* in */ BMDVideoInputFormatChangedEvents notificationEvents, /* in */ IDeckLinkDisplayMode *newDisplayMode, /* in */ BMDDetectedVideoInputFormatFlags detectedSignalFlags) = 0; virtual HRESULT VideoInputFrameArrived (/* in */ IDeckLinkVideoInputFrame* videoFrame, /* in */ IDeckLinkAudioInputPacket* audioPacket) = 0; protected: virtual ~IDeckLinkInputCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderInputCallback - Frame arrival callback. */ class IDeckLinkEncoderInputCallback : public IUnknown { public: virtual HRESULT VideoInputSignalChanged (/* in */ BMDVideoInputFormatChangedEvents notificationEvents, /* in */ IDeckLinkDisplayMode *newDisplayMode, /* in */ BMDDetectedVideoInputFormatFlags detectedSignalFlags) = 0; virtual HRESULT VideoPacketArrived (/* in */ IDeckLinkEncoderVideoPacket* videoPacket) = 0; virtual HRESULT AudioPacketArrived (/* in */ IDeckLinkEncoderAudioPacket* audioPacket) = 0; protected: virtual ~IDeckLinkEncoderInputCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkMemoryAllocator - Memory allocator for video frames. */ class IDeckLinkMemoryAllocator : public IUnknown { public: virtual HRESULT AllocateBuffer (/* in */ uint32_t bufferSize, /* out */ void **allocatedBuffer) = 0; virtual HRESULT ReleaseBuffer (/* in */ void *buffer) = 0; virtual HRESULT Commit (void) = 0; virtual HRESULT Decommit (void) = 0; }; /* Interface IDeckLinkAudioOutputCallback - Optional callback to allow audio samples to be pulled as required. */ class IDeckLinkAudioOutputCallback : public IUnknown { public: virtual HRESULT RenderAudioSamples (/* in */ bool preroll) = 0; }; /* Interface IDeckLinkIterator - enumerates installed DeckLink hardware */ class IDeckLinkIterator : public IUnknown { public: virtual HRESULT Next (/* out */ IDeckLink **deckLinkInstance) = 0; }; /* Interface IDeckLinkAPIInformation - DeckLinkAPI attribute interface */ class IDeckLinkAPIInformation : public IUnknown { public: virtual HRESULT GetFlag (/* in */ BMDDeckLinkAPIInformationID cfgID, /* out */ bool *value) = 0; virtual HRESULT GetInt (/* in */ BMDDeckLinkAPIInformationID cfgID, /* out */ int64_t *value) = 0; virtual HRESULT GetFloat (/* in */ BMDDeckLinkAPIInformationID cfgID, /* out */ double *value) = 0; virtual HRESULT GetString (/* in */ BMDDeckLinkAPIInformationID cfgID, /* out */ const char **value) = 0; protected: virtual ~IDeckLinkAPIInformation () {} // call Release method to drop reference count }; /* Interface IDeckLinkOutput - Created by QueryInterface from IDeckLink. */ class IDeckLinkOutput : public IUnknown { public: virtual HRESULT DoesSupportVideoMode (/* in */ BMDDisplayMode displayMode, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDVideoOutputFlags flags, /* out */ BMDDisplayModeSupport *result, /* out */ IDeckLinkDisplayMode **resultDisplayMode) = 0; virtual HRESULT GetDisplayModeIterator (/* out */ IDeckLinkDisplayModeIterator **iterator) = 0; virtual HRESULT SetScreenPreviewCallback (/* in */ IDeckLinkScreenPreviewCallback *previewCallback) = 0; /* Video Output */ virtual HRESULT EnableVideoOutput (/* in */ BMDDisplayMode displayMode, /* in */ BMDVideoOutputFlags flags) = 0; virtual HRESULT DisableVideoOutput (void) = 0; virtual HRESULT SetVideoOutputFrameMemoryAllocator (/* in */ IDeckLinkMemoryAllocator *theAllocator) = 0; virtual HRESULT CreateVideoFrame (/* in */ int32_t width, /* in */ int32_t height, /* in */ int32_t rowBytes, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDFrameFlags flags, /* out */ IDeckLinkMutableVideoFrame **outFrame) = 0; virtual HRESULT CreateAncillaryData (/* in */ BMDPixelFormat pixelFormat, /* out */ IDeckLinkVideoFrameAncillary **outBuffer) = 0; virtual HRESULT DisplayVideoFrameSync (/* in */ IDeckLinkVideoFrame *theFrame) = 0; virtual HRESULT ScheduleVideoFrame (/* in */ IDeckLinkVideoFrame *theFrame, /* in */ BMDTimeValue displayTime, /* in */ BMDTimeValue displayDuration, /* in */ BMDTimeScale timeScale) = 0; virtual HRESULT SetScheduledFrameCompletionCallback (/* in */ IDeckLinkVideoOutputCallback *theCallback) = 0; virtual HRESULT GetBufferedVideoFrameCount (/* out */ uint32_t *bufferedFrameCount) = 0; /* Audio Output */ virtual HRESULT EnableAudioOutput (/* in */ BMDAudioSampleRate sampleRate, /* in */ BMDAudioSampleType sampleType, /* in */ uint32_t channelCount, /* in */ BMDAudioOutputStreamType streamType) = 0; virtual HRESULT DisableAudioOutput (void) = 0; virtual HRESULT WriteAudioSamplesSync (/* in */ void *buffer, /* in */ uint32_t sampleFrameCount, /* out */ uint32_t *sampleFramesWritten) = 0; virtual HRESULT BeginAudioPreroll (void) = 0; virtual HRESULT EndAudioPreroll (void) = 0; virtual HRESULT ScheduleAudioSamples (/* in */ void *buffer, /* in */ uint32_t sampleFrameCount, /* in */ BMDTimeValue streamTime, /* in */ BMDTimeScale timeScale, /* out */ uint32_t *sampleFramesWritten) = 0; virtual HRESULT GetBufferedAudioSampleFrameCount (/* out */ uint32_t *bufferedSampleFrameCount) = 0; virtual HRESULT FlushBufferedAudioSamples (void) = 0; virtual HRESULT SetAudioCallback (/* in */ IDeckLinkAudioOutputCallback *theCallback) = 0; /* Output Control */ virtual HRESULT StartScheduledPlayback (/* in */ BMDTimeValue playbackStartTime, /* in */ BMDTimeScale timeScale, /* in */ double playbackSpeed) = 0; virtual HRESULT StopScheduledPlayback (/* in */ BMDTimeValue stopPlaybackAtTime, /* out */ BMDTimeValue *actualStopTime, /* in */ BMDTimeScale timeScale) = 0; virtual HRESULT IsScheduledPlaybackRunning (/* out */ bool *active) = 0; virtual HRESULT GetScheduledStreamTime (/* in */ BMDTimeScale desiredTimeScale, /* out */ BMDTimeValue *streamTime, /* out */ double *playbackSpeed) = 0; virtual HRESULT GetReferenceStatus (/* out */ BMDReferenceStatus *referenceStatus) = 0; /* Hardware Timing */ virtual HRESULT GetHardwareReferenceClock (/* in */ BMDTimeScale desiredTimeScale, /* out */ BMDTimeValue *hardwareTime, /* out */ BMDTimeValue *timeInFrame, /* out */ BMDTimeValue *ticksPerFrame) = 0; virtual HRESULT GetFrameCompletionReferenceTimestamp (/* in */ IDeckLinkVideoFrame *theFrame, /* in */ BMDTimeScale desiredTimeScale, /* out */ BMDTimeValue *frameCompletionTimestamp) = 0; protected: virtual ~IDeckLinkOutput () {} // call Release method to drop reference count }; /* Interface IDeckLinkInput - Created by QueryInterface from IDeckLink. */ class IDeckLinkInput : public IUnknown { public: virtual HRESULT DoesSupportVideoMode (/* in */ BMDDisplayMode displayMode, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDVideoInputFlags flags, /* out */ BMDDisplayModeSupport *result, /* out */ IDeckLinkDisplayMode **resultDisplayMode) = 0; virtual HRESULT GetDisplayModeIterator (/* out */ IDeckLinkDisplayModeIterator **iterator) = 0; virtual HRESULT SetScreenPreviewCallback (/* in */ IDeckLinkScreenPreviewCallback *previewCallback) = 0; /* Video Input */ virtual HRESULT EnableVideoInput (/* in */ BMDDisplayMode displayMode, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDVideoInputFlags flags) = 0; virtual HRESULT DisableVideoInput (void) = 0; virtual HRESULT GetAvailableVideoFrameCount (/* out */ uint32_t *availableFrameCount) = 0; virtual HRESULT SetVideoInputFrameMemoryAllocator (/* in */ IDeckLinkMemoryAllocator *theAllocator) = 0; /* Audio Input */ virtual HRESULT EnableAudioInput (/* in */ BMDAudioSampleRate sampleRate, /* in */ BMDAudioSampleType sampleType, /* in */ uint32_t channelCount) = 0; virtual HRESULT DisableAudioInput (void) = 0; virtual HRESULT GetAvailableAudioSampleFrameCount (/* out */ uint32_t *availableSampleFrameCount) = 0; /* Input Control */ virtual HRESULT StartStreams (void) = 0; virtual HRESULT StopStreams (void) = 0; virtual HRESULT PauseStreams (void) = 0; virtual HRESULT FlushStreams (void) = 0; virtual HRESULT SetCallback (/* in */ IDeckLinkInputCallback *theCallback) = 0; /* Hardware Timing */ virtual HRESULT GetHardwareReferenceClock (/* in */ BMDTimeScale desiredTimeScale, /* out */ BMDTimeValue *hardwareTime, /* out */ BMDTimeValue *timeInFrame, /* out */ BMDTimeValue *ticksPerFrame) = 0; protected: virtual ~IDeckLinkInput () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderInput - Created by QueryInterface from IDeckLink. */ class IDeckLinkEncoderInput : public IUnknown { public: virtual HRESULT DoesSupportVideoMode (/* in */ BMDDisplayMode displayMode, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDVideoInputFlags flags, /* out */ BMDDisplayModeSupport *result, /* out */ IDeckLinkDisplayMode **resultDisplayMode) = 0; virtual HRESULT GetDisplayModeIterator (/* out */ IDeckLinkDisplayModeIterator **iterator) = 0; /* Video Input */ virtual HRESULT EnableVideoInput (/* in */ BMDDisplayMode displayMode, /* in */ BMDPixelFormat pixelFormat, /* in */ BMDVideoInputFlags flags) = 0; virtual HRESULT DisableVideoInput (void) = 0; virtual HRESULT GetAvailablePacketsCount (/* out */ uint32_t *availablePacketsCount) = 0; virtual HRESULT SetMemoryAllocator (/* in */ IDeckLinkMemoryAllocator *theAllocator) = 0; /* Audio Input */ virtual HRESULT EnableAudioInput (/* in */ BMDAudioFormat audioFormat, /* in */ BMDAudioSampleRate sampleRate, /* in */ BMDAudioSampleType sampleType, /* in */ uint32_t channelCount) = 0; virtual HRESULT DisableAudioInput (void) = 0; virtual HRESULT GetAvailableAudioSampleFrameCount (/* out */ uint32_t *availableSampleFrameCount) = 0; /* Input Control */ virtual HRESULT StartStreams (void) = 0; virtual HRESULT StopStreams (void) = 0; virtual HRESULT PauseStreams (void) = 0; virtual HRESULT FlushStreams (void) = 0; virtual HRESULT SetCallback (/* in */ IDeckLinkEncoderInputCallback *theCallback) = 0; /* Hardware Timing */ virtual HRESULT GetHardwareReferenceClock (/* in */ BMDTimeScale desiredTimeScale, /* out */ BMDTimeValue *hardwareTime, /* out */ BMDTimeValue *timeInFrame, /* out */ BMDTimeValue *ticksPerFrame) = 0; protected: virtual ~IDeckLinkEncoderInput () {} // call Release method to drop reference count }; /* Interface IDeckLinkVideoFrame - Interface to encapsulate a video frame; can be caller-implemented. */ class IDeckLinkVideoFrame : public IUnknown { public: virtual long GetWidth (void) = 0; virtual long GetHeight (void) = 0; virtual long GetRowBytes (void) = 0; virtual BMDPixelFormat GetPixelFormat (void) = 0; virtual BMDFrameFlags GetFlags (void) = 0; virtual HRESULT GetBytes (/* out */ void **buffer) = 0; virtual HRESULT GetTimecode (/* in */ BMDTimecodeFormat format, /* out */ IDeckLinkTimecode **timecode) = 0; virtual HRESULT GetAncillaryData (/* out */ IDeckLinkVideoFrameAncillary **ancillary) = 0; protected: virtual ~IDeckLinkVideoFrame () {} // call Release method to drop reference count }; /* Interface IDeckLinkMutableVideoFrame - Created by IDeckLinkOutput::CreateVideoFrame. */ class IDeckLinkMutableVideoFrame : public IDeckLinkVideoFrame { public: virtual HRESULT SetFlags (/* in */ BMDFrameFlags newFlags) = 0; virtual HRESULT SetTimecode (/* in */ BMDTimecodeFormat format, /* in */ IDeckLinkTimecode *timecode) = 0; virtual HRESULT SetTimecodeFromComponents (/* in */ BMDTimecodeFormat format, /* in */ uint8_t hours, /* in */ uint8_t minutes, /* in */ uint8_t seconds, /* in */ uint8_t frames, /* in */ BMDTimecodeFlags flags) = 0; virtual HRESULT SetAncillaryData (/* in */ IDeckLinkVideoFrameAncillary *ancillary) = 0; virtual HRESULT SetTimecodeUserBits (/* in */ BMDTimecodeFormat format, /* in */ BMDTimecodeUserBits userBits) = 0; protected: virtual ~IDeckLinkMutableVideoFrame () {} // call Release method to drop reference count }; /* Interface IDeckLinkVideoFrame3DExtensions - Optional interface implemented on IDeckLinkVideoFrame to support 3D frames */ class IDeckLinkVideoFrame3DExtensions : public IUnknown { public: virtual BMDVideo3DPackingFormat Get3DPackingFormat (void) = 0; virtual HRESULT GetFrameForRightEye (/* out */ IDeckLinkVideoFrame* *rightEyeFrame) = 0; protected: virtual ~IDeckLinkVideoFrame3DExtensions () {} // call Release method to drop reference count }; /* Interface IDeckLinkVideoInputFrame - Provided by the IDeckLinkVideoInput frame arrival callback. */ class IDeckLinkVideoInputFrame : public IDeckLinkVideoFrame { public: virtual HRESULT GetStreamTime (/* out */ BMDTimeValue *frameTime, /* out */ BMDTimeValue *frameDuration, /* in */ BMDTimeScale timeScale) = 0; virtual HRESULT GetHardwareReferenceTimestamp (/* in */ BMDTimeScale timeScale, /* out */ BMDTimeValue *frameTime, /* out */ BMDTimeValue *frameDuration) = 0; protected: virtual ~IDeckLinkVideoInputFrame () {} // call Release method to drop reference count }; /* Interface IDeckLinkVideoFrameAncillary - Obtained through QueryInterface() on an IDeckLinkVideoFrame object. */ class IDeckLinkVideoFrameAncillary : public IUnknown { public: virtual HRESULT GetBufferForVerticalBlankingLine (/* in */ uint32_t lineNumber, /* out */ void **buffer) = 0; virtual BMDPixelFormat GetPixelFormat (void) = 0; virtual BMDDisplayMode GetDisplayMode (void) = 0; protected: virtual ~IDeckLinkVideoFrameAncillary () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderPacket - Interface to encapsulate an encoded packet. */ class IDeckLinkEncoderPacket : public IUnknown { public: virtual HRESULT GetBytes (/* out */ void **buffer) = 0; virtual long GetSize (void) = 0; virtual HRESULT GetStreamTime (/* out */ BMDTimeValue *frameTime, /* in */ BMDTimeScale timeScale) = 0; virtual BMDPacketType GetPacketType (void) = 0; protected: virtual ~IDeckLinkEncoderPacket () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderVideoPacket - Provided by the IDeckLinkEncoderInput video packet arrival callback. */ class IDeckLinkEncoderVideoPacket : public IDeckLinkEncoderPacket { public: virtual BMDPixelFormat GetPixelFormat (void) = 0; virtual HRESULT GetHardwareReferenceTimestamp (/* in */ BMDTimeScale timeScale, /* out */ BMDTimeValue *frameTime, /* out */ BMDTimeValue *frameDuration) = 0; virtual HRESULT GetTimecode (/* in */ BMDTimecodeFormat format, /* out */ IDeckLinkTimecode **timecode) = 0; protected: virtual ~IDeckLinkEncoderVideoPacket () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderAudioPacket - Provided by the IDeckLinkEncoderInput audio packet arrival callback. */ class IDeckLinkEncoderAudioPacket : public IDeckLinkEncoderPacket { public: virtual BMDAudioFormat GetAudioFormat (void) = 0; protected: virtual ~IDeckLinkEncoderAudioPacket () {} // call Release method to drop reference count }; /* Interface IDeckLinkH265NALPacket - Obtained through QueryInterface() on an IDeckLinkEncoderVideoPacket object */ class IDeckLinkH265NALPacket : public IDeckLinkEncoderVideoPacket { public: virtual HRESULT GetUnitType (/* out */ uint8_t *unitType) = 0; virtual HRESULT GetBytesNoPrefix (/* out */ void **buffer) = 0; virtual long GetSizeNoPrefix (void) = 0; protected: virtual ~IDeckLinkH265NALPacket () {} // call Release method to drop reference count }; /* Interface IDeckLinkAudioInputPacket - Provided by the IDeckLinkInput callback. */ class IDeckLinkAudioInputPacket : public IUnknown { public: virtual long GetSampleFrameCount (void) = 0; virtual HRESULT GetBytes (/* out */ void **buffer) = 0; virtual HRESULT GetPacketTime (/* out */ BMDTimeValue *packetTime, /* in */ BMDTimeScale timeScale) = 0; protected: virtual ~IDeckLinkAudioInputPacket () {} // call Release method to drop reference count }; /* Interface IDeckLinkScreenPreviewCallback - Screen preview callback */ class IDeckLinkScreenPreviewCallback : public IUnknown { public: virtual HRESULT DrawFrame (/* in */ IDeckLinkVideoFrame *theFrame) = 0; protected: virtual ~IDeckLinkScreenPreviewCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkGLScreenPreviewHelper - Created with CoCreateInstance(). */ class IDeckLinkGLScreenPreviewHelper : public IUnknown { public: /* Methods must be called with OpenGL context set */ virtual HRESULT InitializeGL (void) = 0; virtual HRESULT PaintGL (void) = 0; virtual HRESULT SetFrame (/* in */ IDeckLinkVideoFrame *theFrame) = 0; virtual HRESULT Set3DPreviewFormat (/* in */ BMD3DPreviewFormat previewFormat) = 0; protected: virtual ~IDeckLinkGLScreenPreviewHelper () {} // call Release method to drop reference count }; /* Interface IDeckLinkNotificationCallback - DeckLink Notification Callback Interface */ class IDeckLinkNotificationCallback : public IUnknown { public: virtual HRESULT Notify (/* in */ BMDNotifications topic, /* in */ uint64_t param1, /* in */ uint64_t param2) = 0; }; /* Interface IDeckLinkNotification - DeckLink Notification interface */ class IDeckLinkNotification : public IUnknown { public: virtual HRESULT Subscribe (/* in */ BMDNotifications topic, /* in */ IDeckLinkNotificationCallback *theCallback) = 0; virtual HRESULT Unsubscribe (/* in */ BMDNotifications topic, /* in */ IDeckLinkNotificationCallback *theCallback) = 0; }; /* Interface IDeckLinkAttributes - DeckLink Attribute interface */ class IDeckLinkAttributes : public IUnknown { public: virtual HRESULT GetFlag (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ bool *value) = 0; virtual HRESULT GetInt (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ int64_t *value) = 0; virtual HRESULT GetFloat (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ double *value) = 0; virtual HRESULT GetString (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ const char **value) = 0; protected: virtual ~IDeckLinkAttributes () {} // call Release method to drop reference count }; /* Interface IDeckLinkKeyer - DeckLink Keyer interface */ class IDeckLinkKeyer : public IUnknown { public: virtual HRESULT Enable (/* in */ bool isExternal) = 0; virtual HRESULT SetLevel (/* in */ uint8_t level) = 0; virtual HRESULT RampUp (/* in */ uint32_t numberOfFrames) = 0; virtual HRESULT RampDown (/* in */ uint32_t numberOfFrames) = 0; virtual HRESULT Disable (void) = 0; protected: virtual ~IDeckLinkKeyer () {} // call Release method to drop reference count }; /* Interface IDeckLinkVideoConversion - Created with CoCreateInstance(). */ class IDeckLinkVideoConversion : public IUnknown { public: virtual HRESULT ConvertFrame (/* in */ IDeckLinkVideoFrame* srcFrame, /* in */ IDeckLinkVideoFrame* dstFrame) = 0; protected: virtual ~IDeckLinkVideoConversion () {} // call Release method to drop reference count }; /* Interface IDeckLinkDeviceNotificationCallback - DeckLink device arrival/removal notification callbacks */ class IDeckLinkDeviceNotificationCallback : public IUnknown { public: virtual HRESULT DeckLinkDeviceArrived (/* in */ IDeckLink* deckLinkDevice) = 0; virtual HRESULT DeckLinkDeviceRemoved (/* in */ IDeckLink* deckLinkDevice) = 0; protected: virtual ~IDeckLinkDeviceNotificationCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkDiscovery - DeckLink device discovery */ class IDeckLinkDiscovery : public IUnknown { public: virtual HRESULT InstallDeviceNotifications (/* in */ IDeckLinkDeviceNotificationCallback* deviceNotificationCallback) = 0; virtual HRESULT UninstallDeviceNotifications (void) = 0; protected: virtual ~IDeckLinkDiscovery () {} // call Release method to drop reference count }; /* Functions */ extern "C" { IDeckLinkIterator* CreateDeckLinkIteratorInstance (void); IDeckLinkDiscovery* CreateDeckLinkDiscoveryInstance (void); IDeckLinkAPIInformation* CreateDeckLinkAPIInformationInstance (void); IDeckLinkGLScreenPreviewHelper* CreateOpenGLScreenPreviewHelper (void); IDeckLinkVideoConversion* CreateVideoConversionInstance (void); } #endif // defined(__cplusplus) #endif /* defined(BMD_DECKLINKAPI_H) */ nageru-1.4.2/decklink/DeckLinkAPIConfiguration.h000066400000000000000000000314551301561706600215170ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPICONFIGURATION_H #define BMD_DECKLINKAPICONFIGURATION_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif // Type Declarations // Interface ID Declarations BMD_CONST REFIID IID_IDeckLinkConfiguration = /* CB71734A-FE37-4E8D-8E13-802133A1C3F2 */ {0xCB,0x71,0x73,0x4A,0xFE,0x37,0x4E,0x8D,0x8E,0x13,0x80,0x21,0x33,0xA1,0xC3,0xF2}; BMD_CONST REFIID IID_IDeckLinkEncoderConfiguration = /* 138050E5-C60A-4552-BF3F-0F358049327E */ {0x13,0x80,0x50,0xE5,0xC6,0x0A,0x45,0x52,0xBF,0x3F,0x0F,0x35,0x80,0x49,0x32,0x7E}; /* Enum BMDDeckLinkConfigurationID - DeckLink Configuration ID */ typedef uint32_t BMDDeckLinkConfigurationID; enum _BMDDeckLinkConfigurationID { /* Serial port Flags */ bmdDeckLinkConfigSwapSerialRxTx = /* 'ssrt' */ 0x73737274, /* Video Input/Output Flags */ bmdDeckLinkConfigUse1080pNotPsF = /* 'fpro' */ 0x6670726F, /* Video Input/Output Integers */ bmdDeckLinkConfigHDMI3DPackingFormat = /* '3dpf' */ 0x33647066, bmdDeckLinkConfigBypass = /* 'byps' */ 0x62797073, bmdDeckLinkConfigClockTimingAdjustment = /* 'ctad' */ 0x63746164, /* Audio Input/Output Flags */ bmdDeckLinkConfigAnalogAudioConsumerLevels = /* 'aacl' */ 0x6161636C, /* Video output flags */ bmdDeckLinkConfigFieldFlickerRemoval = /* 'fdfr' */ 0x66646672, bmdDeckLinkConfigHD1080p24ToHD1080i5994Conversion = /* 'to59' */ 0x746F3539, bmdDeckLinkConfig444SDIVideoOutput = /* '444o' */ 0x3434346F, bmdDeckLinkConfigBlackVideoOutputDuringCapture = /* 'bvoc' */ 0x62766F63, bmdDeckLinkConfigLowLatencyVideoOutput = /* 'llvo' */ 0x6C6C766F, bmdDeckLinkConfigDownConversionOnAllAnalogOutput = /* 'caao' */ 0x6361616F, bmdDeckLinkConfigSMPTELevelAOutput = /* 'smta' */ 0x736D7461, /* Video Output Integers */ bmdDeckLinkConfigVideoOutputConnection = /* 'vocn' */ 0x766F636E, bmdDeckLinkConfigVideoOutputConversionMode = /* 'vocm' */ 0x766F636D, bmdDeckLinkConfigAnalogVideoOutputFlags = /* 'avof' */ 0x61766F66, bmdDeckLinkConfigReferenceInputTimingOffset = /* 'glot' */ 0x676C6F74, bmdDeckLinkConfigVideoOutputIdleOperation = /* 'voio' */ 0x766F696F, bmdDeckLinkConfigDefaultVideoOutputMode = /* 'dvom' */ 0x64766F6D, bmdDeckLinkConfigDefaultVideoOutputModeFlags = /* 'dvof' */ 0x64766F66, bmdDeckLinkConfigSDIOutputLinkConfiguration = /* 'solc' */ 0x736F6C63, /* Video Output Floats */ bmdDeckLinkConfigVideoOutputComponentLumaGain = /* 'oclg' */ 0x6F636C67, bmdDeckLinkConfigVideoOutputComponentChromaBlueGain = /* 'occb' */ 0x6F636362, bmdDeckLinkConfigVideoOutputComponentChromaRedGain = /* 'occr' */ 0x6F636372, bmdDeckLinkConfigVideoOutputCompositeLumaGain = /* 'oilg' */ 0x6F696C67, bmdDeckLinkConfigVideoOutputCompositeChromaGain = /* 'oicg' */ 0x6F696367, bmdDeckLinkConfigVideoOutputSVideoLumaGain = /* 'oslg' */ 0x6F736C67, bmdDeckLinkConfigVideoOutputSVideoChromaGain = /* 'oscg' */ 0x6F736367, /* Video Input Flags */ bmdDeckLinkConfigVideoInputScanning = /* 'visc' */ 0x76697363, // Applicable to H264 Pro Recorder only bmdDeckLinkConfigUseDedicatedLTCInput = /* 'dltc' */ 0x646C7463, // Use timecode from LTC input instead of SDI stream bmdDeckLinkConfigSDIInput3DPayloadOverride = /* '3dds' */ 0x33646473, /* Video Input Integers */ bmdDeckLinkConfigVideoInputConnection = /* 'vicn' */ 0x7669636E, bmdDeckLinkConfigAnalogVideoInputFlags = /* 'avif' */ 0x61766966, bmdDeckLinkConfigVideoInputConversionMode = /* 'vicm' */ 0x7669636D, bmdDeckLinkConfig32PulldownSequenceInitialTimecodeFrame = /* 'pdif' */ 0x70646966, bmdDeckLinkConfigVANCSourceLine1Mapping = /* 'vsl1' */ 0x76736C31, bmdDeckLinkConfigVANCSourceLine2Mapping = /* 'vsl2' */ 0x76736C32, bmdDeckLinkConfigVANCSourceLine3Mapping = /* 'vsl3' */ 0x76736C33, bmdDeckLinkConfigCapturePassThroughMode = /* 'cptm' */ 0x6370746D, /* Video Input Floats */ bmdDeckLinkConfigVideoInputComponentLumaGain = /* 'iclg' */ 0x69636C67, bmdDeckLinkConfigVideoInputComponentChromaBlueGain = /* 'iccb' */ 0x69636362, bmdDeckLinkConfigVideoInputComponentChromaRedGain = /* 'iccr' */ 0x69636372, bmdDeckLinkConfigVideoInputCompositeLumaGain = /* 'iilg' */ 0x69696C67, bmdDeckLinkConfigVideoInputCompositeChromaGain = /* 'iicg' */ 0x69696367, bmdDeckLinkConfigVideoInputSVideoLumaGain = /* 'islg' */ 0x69736C67, bmdDeckLinkConfigVideoInputSVideoChromaGain = /* 'iscg' */ 0x69736367, /* Audio Input Flags */ bmdDeckLinkConfigMicrophonePhantomPower = /* 'mphp' */ 0x6D706870, /* Audio Input Integers */ bmdDeckLinkConfigAudioInputConnection = /* 'aicn' */ 0x6169636E, /* Audio Input Floats */ bmdDeckLinkConfigAnalogAudioInputScaleChannel1 = /* 'ais1' */ 0x61697331, bmdDeckLinkConfigAnalogAudioInputScaleChannel2 = /* 'ais2' */ 0x61697332, bmdDeckLinkConfigAnalogAudioInputScaleChannel3 = /* 'ais3' */ 0x61697333, bmdDeckLinkConfigAnalogAudioInputScaleChannel4 = /* 'ais4' */ 0x61697334, bmdDeckLinkConfigDigitalAudioInputScale = /* 'dais' */ 0x64616973, bmdDeckLinkConfigMicrophoneInputGain = /* 'micg' */ 0x6D696367, /* Audio Output Integers */ bmdDeckLinkConfigAudioOutputAESAnalogSwitch = /* 'aoaa' */ 0x616F6161, /* Audio Output Floats */ bmdDeckLinkConfigAnalogAudioOutputScaleChannel1 = /* 'aos1' */ 0x616F7331, bmdDeckLinkConfigAnalogAudioOutputScaleChannel2 = /* 'aos2' */ 0x616F7332, bmdDeckLinkConfigAnalogAudioOutputScaleChannel3 = /* 'aos3' */ 0x616F7333, bmdDeckLinkConfigAnalogAudioOutputScaleChannel4 = /* 'aos4' */ 0x616F7334, bmdDeckLinkConfigDigitalAudioOutputScale = /* 'daos' */ 0x64616F73, bmdDeckLinkConfigHeadphoneVolume = /* 'hvol' */ 0x68766F6C, /* Device Information Strings */ bmdDeckLinkConfigDeviceInformationLabel = /* 'dila' */ 0x64696C61, bmdDeckLinkConfigDeviceInformationSerialNumber = /* 'disn' */ 0x6469736E, bmdDeckLinkConfigDeviceInformationCompany = /* 'dico' */ 0x6469636F, bmdDeckLinkConfigDeviceInformationPhone = /* 'diph' */ 0x64697068, bmdDeckLinkConfigDeviceInformationEmail = /* 'diem' */ 0x6469656D, bmdDeckLinkConfigDeviceInformationDate = /* 'dida' */ 0x64696461, /* Deck Control Integers */ bmdDeckLinkConfigDeckControlConnection = /* 'dcco' */ 0x6463636F }; /* Enum BMDDeckLinkEncoderConfigurationID - DeckLink Encoder Configuration ID */ typedef uint32_t BMDDeckLinkEncoderConfigurationID; enum _BMDDeckLinkEncoderConfigurationID { /* Video Encoder Integers */ bmdDeckLinkEncoderConfigPreferredBitDepth = /* 'epbr' */ 0x65706272, bmdDeckLinkEncoderConfigFrameCodingMode = /* 'efcm' */ 0x6566636D, /* HEVC/H.265 Encoder Integers */ bmdDeckLinkEncoderConfigH265TargetBitrate = /* 'htbr' */ 0x68746272, /* DNxHR/DNxHD Compression ID */ bmdDeckLinkEncoderConfigDNxHRCompressionID = /* 'dcid' */ 0x64636964, /* DNxHR/DNxHD Level */ bmdDeckLinkEncoderConfigDNxHRLevel = /* 'dlev' */ 0x646C6576, /* Encoded Sample Decriptions */ bmdDeckLinkEncoderConfigMPEG4SampleDescription = /* 'stsE' */ 0x73747345, // Full MPEG4 sample description (aka SampleEntry of an 'stsd' atom-box). Useful for MediaFoundation, QuickTime, MKV and more bmdDeckLinkEncoderConfigMPEG4CodecSpecificDesc = /* 'esds' */ 0x65736473 // Sample description extensions only (atom stream, each with size and fourCC header). Useful for AVFoundation, VideoToolbox, MKV and more }; // Forward Declarations class IDeckLinkConfiguration; class IDeckLinkEncoderConfiguration; /* Interface IDeckLinkConfiguration - DeckLink Configuration interface */ class IDeckLinkConfiguration : public IUnknown { public: virtual HRESULT SetFlag (/* in */ BMDDeckLinkConfigurationID cfgID, /* in */ bool value) = 0; virtual HRESULT GetFlag (/* in */ BMDDeckLinkConfigurationID cfgID, /* out */ bool *value) = 0; virtual HRESULT SetInt (/* in */ BMDDeckLinkConfigurationID cfgID, /* in */ int64_t value) = 0; virtual HRESULT GetInt (/* in */ BMDDeckLinkConfigurationID cfgID, /* out */ int64_t *value) = 0; virtual HRESULT SetFloat (/* in */ BMDDeckLinkConfigurationID cfgID, /* in */ double value) = 0; virtual HRESULT GetFloat (/* in */ BMDDeckLinkConfigurationID cfgID, /* out */ double *value) = 0; virtual HRESULT SetString (/* in */ BMDDeckLinkConfigurationID cfgID, /* in */ const char *value) = 0; virtual HRESULT GetString (/* in */ BMDDeckLinkConfigurationID cfgID, /* out */ const char **value) = 0; virtual HRESULT WriteConfigurationToPreferences (void) = 0; protected: virtual ~IDeckLinkConfiguration () {} // call Release method to drop reference count }; /* Interface IDeckLinkEncoderConfiguration - DeckLink Encoder Configuration interface. Obtained from IDeckLinkEncoderInput */ class IDeckLinkEncoderConfiguration : public IUnknown { public: virtual HRESULT SetFlag (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* in */ bool value) = 0; virtual HRESULT GetFlag (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* out */ bool *value) = 0; virtual HRESULT SetInt (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* in */ int64_t value) = 0; virtual HRESULT GetInt (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* out */ int64_t *value) = 0; virtual HRESULT SetFloat (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* in */ double value) = 0; virtual HRESULT GetFloat (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* out */ double *value) = 0; virtual HRESULT SetString (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* in */ const char *value) = 0; virtual HRESULT GetString (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* out */ const char **value) = 0; virtual HRESULT GetBytes (/* in */ BMDDeckLinkEncoderConfigurationID cfgID, /* out */ void *buffer /* optional */, /* in, out */ uint32_t *bufferSize) = 0; protected: virtual ~IDeckLinkEncoderConfiguration () {} // call Release method to drop reference count }; /* Functions */ extern "C" { } #endif /* defined(BMD_DECKLINKAPICONFIGURATION_H) */ nageru-1.4.2/decklink/DeckLinkAPIDeckControl.h000066400000000000000000000306741301561706600211210ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPIDECKCONTROL_H #define BMD_DECKLINKAPIDECKCONTROL_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif // Type Declarations // Interface ID Declarations BMD_CONST REFIID IID_IDeckLinkDeckControlStatusCallback = /* 53436FFB-B434-4906-BADC-AE3060FFE8EF */ {0x53,0x43,0x6F,0xFB,0xB4,0x34,0x49,0x06,0xBA,0xDC,0xAE,0x30,0x60,0xFF,0xE8,0xEF}; BMD_CONST REFIID IID_IDeckLinkDeckControl = /* 8E1C3ACE-19C7-4E00-8B92-D80431D958BE */ {0x8E,0x1C,0x3A,0xCE,0x19,0xC7,0x4E,0x00,0x8B,0x92,0xD8,0x04,0x31,0xD9,0x58,0xBE}; /* Enum BMDDeckControlMode - DeckControl mode */ typedef uint32_t BMDDeckControlMode; enum _BMDDeckControlMode { bmdDeckControlNotOpened = /* 'ntop' */ 0x6E746F70, bmdDeckControlVTRControlMode = /* 'vtrc' */ 0x76747263, bmdDeckControlExportMode = /* 'expm' */ 0x6578706D, bmdDeckControlCaptureMode = /* 'capm' */ 0x6361706D }; /* Enum BMDDeckControlEvent - DeckControl event */ typedef uint32_t BMDDeckControlEvent; enum _BMDDeckControlEvent { bmdDeckControlAbortedEvent = /* 'abte' */ 0x61627465, // This event is triggered when a capture or edit-to-tape operation is aborted. /* Export-To-Tape events */ bmdDeckControlPrepareForExportEvent = /* 'pfee' */ 0x70666565, // This event is triggered a few frames before reaching the in-point. IDeckLinkInput::StartScheduledPlayback() should be called at this point. bmdDeckControlExportCompleteEvent = /* 'exce' */ 0x65786365, // This event is triggered a few frames after reaching the out-point. At this point, it is safe to stop playback. /* Capture events */ bmdDeckControlPrepareForCaptureEvent = /* 'pfce' */ 0x70666365, // This event is triggered a few frames before reaching the in-point. The serial timecode attached to IDeckLinkVideoInputFrames is now valid. bmdDeckControlCaptureCompleteEvent = /* 'ccev' */ 0x63636576 // This event is triggered a few frames after reaching the out-point. }; /* Enum BMDDeckControlVTRControlState - VTR Control state */ typedef uint32_t BMDDeckControlVTRControlState; enum _BMDDeckControlVTRControlState { bmdDeckControlNotInVTRControlMode = /* 'nvcm' */ 0x6E76636D, bmdDeckControlVTRControlPlaying = /* 'vtrp' */ 0x76747270, bmdDeckControlVTRControlRecording = /* 'vtrr' */ 0x76747272, bmdDeckControlVTRControlStill = /* 'vtra' */ 0x76747261, bmdDeckControlVTRControlShuttleForward = /* 'vtsf' */ 0x76747366, bmdDeckControlVTRControlShuttleReverse = /* 'vtsr' */ 0x76747372, bmdDeckControlVTRControlJogForward = /* 'vtjf' */ 0x76746A66, bmdDeckControlVTRControlJogReverse = /* 'vtjr' */ 0x76746A72, bmdDeckControlVTRControlStopped = /* 'vtro' */ 0x7674726F }; /* Enum BMDDeckControlStatusFlags - Deck Control status flags */ typedef uint32_t BMDDeckControlStatusFlags; enum _BMDDeckControlStatusFlags { bmdDeckControlStatusDeckConnected = 1 << 0, bmdDeckControlStatusRemoteMode = 1 << 1, bmdDeckControlStatusRecordInhibited = 1 << 2, bmdDeckControlStatusCassetteOut = 1 << 3 }; /* Enum BMDDeckControlExportModeOpsFlags - Export mode flags */ typedef uint32_t BMDDeckControlExportModeOpsFlags; enum _BMDDeckControlExportModeOpsFlags { bmdDeckControlExportModeInsertVideo = 1 << 0, bmdDeckControlExportModeInsertAudio1 = 1 << 1, bmdDeckControlExportModeInsertAudio2 = 1 << 2, bmdDeckControlExportModeInsertAudio3 = 1 << 3, bmdDeckControlExportModeInsertAudio4 = 1 << 4, bmdDeckControlExportModeInsertAudio5 = 1 << 5, bmdDeckControlExportModeInsertAudio6 = 1 << 6, bmdDeckControlExportModeInsertAudio7 = 1 << 7, bmdDeckControlExportModeInsertAudio8 = 1 << 8, bmdDeckControlExportModeInsertAudio9 = 1 << 9, bmdDeckControlExportModeInsertAudio10 = 1 << 10, bmdDeckControlExportModeInsertAudio11 = 1 << 11, bmdDeckControlExportModeInsertAudio12 = 1 << 12, bmdDeckControlExportModeInsertTimeCode = 1 << 13, bmdDeckControlExportModeInsertAssemble = 1 << 14, bmdDeckControlExportModeInsertPreview = 1 << 15, bmdDeckControlUseManualExport = 1 << 16 }; /* Enum BMDDeckControlError - Deck Control error */ typedef uint32_t BMDDeckControlError; enum _BMDDeckControlError { bmdDeckControlNoError = /* 'noer' */ 0x6E6F6572, bmdDeckControlModeError = /* 'moer' */ 0x6D6F6572, bmdDeckControlMissedInPointError = /* 'mier' */ 0x6D696572, bmdDeckControlDeckTimeoutError = /* 'dter' */ 0x64746572, bmdDeckControlCommandFailedError = /* 'cfer' */ 0x63666572, bmdDeckControlDeviceAlreadyOpenedError = /* 'dalo' */ 0x64616C6F, bmdDeckControlFailedToOpenDeviceError = /* 'fder' */ 0x66646572, bmdDeckControlInLocalModeError = /* 'lmer' */ 0x6C6D6572, bmdDeckControlEndOfTapeError = /* 'eter' */ 0x65746572, bmdDeckControlUserAbortError = /* 'uaer' */ 0x75616572, bmdDeckControlNoTapeInDeckError = /* 'nter' */ 0x6E746572, bmdDeckControlNoVideoFromCardError = /* 'nvfc' */ 0x6E766663, bmdDeckControlNoCommunicationError = /* 'ncom' */ 0x6E636F6D, bmdDeckControlBufferTooSmallError = /* 'btsm' */ 0x6274736D, bmdDeckControlBadChecksumError = /* 'chks' */ 0x63686B73, bmdDeckControlUnknownError = /* 'uner' */ 0x756E6572 }; // Forward Declarations class IDeckLinkDeckControlStatusCallback; class IDeckLinkDeckControl; /* Interface IDeckLinkDeckControlStatusCallback - Deck control state change callback. */ class IDeckLinkDeckControlStatusCallback : public IUnknown { public: virtual HRESULT TimecodeUpdate (/* in */ BMDTimecodeBCD currentTimecode) = 0; virtual HRESULT VTRControlStateChanged (/* in */ BMDDeckControlVTRControlState newState, /* in */ BMDDeckControlError error) = 0; virtual HRESULT DeckControlEventReceived (/* in */ BMDDeckControlEvent event, /* in */ BMDDeckControlError error) = 0; virtual HRESULT DeckControlStatusChanged (/* in */ BMDDeckControlStatusFlags flags, /* in */ uint32_t mask) = 0; protected: virtual ~IDeckLinkDeckControlStatusCallback () {} // call Release method to drop reference count }; /* Interface IDeckLinkDeckControl - Deck Control main interface */ class IDeckLinkDeckControl : public IUnknown { public: virtual HRESULT Open (/* in */ BMDTimeScale timeScale, /* in */ BMDTimeValue timeValue, /* in */ bool timecodeIsDropFrame, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT Close (/* in */ bool standbyOn) = 0; virtual HRESULT GetCurrentState (/* out */ BMDDeckControlMode *mode, /* out */ BMDDeckControlVTRControlState *vtrControlState, /* out */ BMDDeckControlStatusFlags *flags) = 0; virtual HRESULT SetStandby (/* in */ bool standbyOn) = 0; virtual HRESULT SendCommand (/* in */ uint8_t *inBuffer, /* in */ uint32_t inBufferSize, /* out */ uint8_t *outBuffer, /* out */ uint32_t *outDataSize, /* in */ uint32_t outBufferSize, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT Play (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT Stop (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT TogglePlayStop (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT Eject (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT GoToTimecode (/* in */ BMDTimecodeBCD timecode, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT FastForward (/* in */ bool viewTape, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT Rewind (/* in */ bool viewTape, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT StepForward (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT StepBack (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT Jog (/* in */ double rate, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT Shuttle (/* in */ double rate, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT GetTimecodeString (/* out */ const char **currentTimeCode, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT GetTimecode (/* out */ IDeckLinkTimecode **currentTimecode, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT GetTimecodeBCD (/* out */ BMDTimecodeBCD *currentTimecode, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT SetPreroll (/* in */ uint32_t prerollSeconds) = 0; virtual HRESULT GetPreroll (/* out */ uint32_t *prerollSeconds) = 0; virtual HRESULT SetExportOffset (/* in */ int32_t exportOffsetFields) = 0; virtual HRESULT GetExportOffset (/* out */ int32_t *exportOffsetFields) = 0; virtual HRESULT GetManualExportOffset (/* out */ int32_t *deckManualExportOffsetFields) = 0; virtual HRESULT SetCaptureOffset (/* in */ int32_t captureOffsetFields) = 0; virtual HRESULT GetCaptureOffset (/* out */ int32_t *captureOffsetFields) = 0; virtual HRESULT StartExport (/* in */ BMDTimecodeBCD inTimecode, /* in */ BMDTimecodeBCD outTimecode, /* in */ BMDDeckControlExportModeOpsFlags exportModeOps, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT StartCapture (/* in */ bool useVITC, /* in */ BMDTimecodeBCD inTimecode, /* in */ BMDTimecodeBCD outTimecode, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT GetDeviceID (/* out */ uint16_t *deviceId, /* out */ BMDDeckControlError *error) = 0; virtual HRESULT Abort (void) = 0; virtual HRESULT CrashRecordStart (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT CrashRecordStop (/* out */ BMDDeckControlError *error) = 0; virtual HRESULT SetCallback (/* in */ IDeckLinkDeckControlStatusCallback *callback) = 0; protected: virtual ~IDeckLinkDeckControl () {} // call Release method to drop reference count }; /* Functions */ extern "C" { } #endif /* defined(BMD_DECKLINKAPIDECKCONTROL_H) */ nageru-1.4.2/decklink/DeckLinkAPIDiscovery.h000066400000000000000000000045271301561706600206570ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPIDISCOVERY_H #define BMD_DECKLINKAPIDISCOVERY_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif // Type Declarations // Interface ID Declarations BMD_CONST REFIID IID_IDeckLink = /* C418FBDD-0587-48ED-8FE5-640F0A14AF91 */ {0xC4,0x18,0xFB,0xDD,0x05,0x87,0x48,0xED,0x8F,0xE5,0x64,0x0F,0x0A,0x14,0xAF,0x91}; // Forward Declarations class IDeckLink; /* Interface IDeckLink - represents a DeckLink device */ class IDeckLink : public IUnknown { public: virtual HRESULT GetModelName (/* out */ const char **modelName) = 0; virtual HRESULT GetDisplayName (/* out */ const char **displayName) = 0; protected: virtual ~IDeckLink () {} // call Release method to drop reference count }; /* Functions */ extern "C" { } #endif /* defined(BMD_DECKLINKAPIDISCOVERY_H) */ nageru-1.4.2/decklink/DeckLinkAPIDispatch.cpp000077500000000000000000000121731301561706600210010ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2009 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- **/ #include #include #include #include "DeckLinkAPI.h" #define kDeckLinkAPI_Name "libDeckLinkAPI.so" #define KDeckLinkPreviewAPI_Name "libDeckLinkPreviewAPI.so" typedef IDeckLinkIterator* (*CreateIteratorFunc)(void); typedef IDeckLinkAPIInformation* (*CreateAPIInformationFunc)(void); typedef IDeckLinkGLScreenPreviewHelper* (*CreateOpenGLScreenPreviewHelperFunc)(void); typedef IDeckLinkVideoConversion* (*CreateVideoConversionInstanceFunc)(void); typedef IDeckLinkDiscovery* (*CreateDeckLinkDiscoveryInstanceFunc)(void); static pthread_once_t gDeckLinkOnceControl = PTHREAD_ONCE_INIT; static pthread_once_t gPreviewOnceControl = PTHREAD_ONCE_INIT; static bool gLoadedDeckLinkAPI = false; static CreateIteratorFunc gCreateIteratorFunc = NULL; static CreateAPIInformationFunc gCreateAPIInformationFunc = NULL; static CreateOpenGLScreenPreviewHelperFunc gCreateOpenGLPreviewFunc = NULL; static CreateVideoConversionInstanceFunc gCreateVideoConversionFunc = NULL; static CreateDeckLinkDiscoveryInstanceFunc gCreateDeckLinkDiscoveryFunc = NULL; void InitDeckLinkAPI (void) { void *libraryHandle; libraryHandle = dlopen(kDeckLinkAPI_Name, RTLD_NOW|RTLD_GLOBAL); if (!libraryHandle) { fprintf(stderr, "%s\n", dlerror()); return; } gLoadedDeckLinkAPI = true; gCreateIteratorFunc = (CreateIteratorFunc)dlsym(libraryHandle, "CreateDeckLinkIteratorInstance_0002"); if (!gCreateIteratorFunc) fprintf(stderr, "%s\n", dlerror()); gCreateAPIInformationFunc = (CreateAPIInformationFunc)dlsym(libraryHandle, "CreateDeckLinkAPIInformationInstance_0001"); if (!gCreateAPIInformationFunc) fprintf(stderr, "%s\n", dlerror()); gCreateVideoConversionFunc = (CreateVideoConversionInstanceFunc)dlsym(libraryHandle, "CreateVideoConversionInstance_0001"); if (!gCreateVideoConversionFunc) fprintf(stderr, "%s\n", dlerror()); gCreateDeckLinkDiscoveryFunc = (CreateDeckLinkDiscoveryInstanceFunc)dlsym(libraryHandle, "CreateDeckLinkDiscoveryInstance_0001"); if (!gCreateDeckLinkDiscoveryFunc) fprintf(stderr, "%s\n", dlerror()); } void InitDeckLinkPreviewAPI (void) { void *libraryHandle; libraryHandle = dlopen(KDeckLinkPreviewAPI_Name, RTLD_NOW|RTLD_GLOBAL); if (!libraryHandle) { fprintf(stderr, "%s\n", dlerror()); return; } gCreateOpenGLPreviewFunc = (CreateOpenGLScreenPreviewHelperFunc)dlsym(libraryHandle, "CreateOpenGLScreenPreviewHelper_0001"); if (!gCreateOpenGLPreviewFunc) fprintf(stderr, "%s\n", dlerror()); } bool IsDeckLinkAPIPresent (void) { // If the DeckLink API dynamic library was successfully loaded, return this knowledge to the caller return gLoadedDeckLinkAPI; } IDeckLinkIterator* CreateDeckLinkIteratorInstance (void) { pthread_once(&gDeckLinkOnceControl, InitDeckLinkAPI); if (gCreateIteratorFunc == NULL) return NULL; return gCreateIteratorFunc(); } IDeckLinkAPIInformation* CreateDeckLinkAPIInformationInstance (void) { pthread_once(&gDeckLinkOnceControl, InitDeckLinkAPI); if (gCreateAPIInformationFunc == NULL) return NULL; return gCreateAPIInformationFunc(); } IDeckLinkGLScreenPreviewHelper* CreateOpenGLScreenPreviewHelper (void) { pthread_once(&gDeckLinkOnceControl, InitDeckLinkAPI); pthread_once(&gPreviewOnceControl, InitDeckLinkPreviewAPI); if (gCreateOpenGLPreviewFunc == NULL) return NULL; return gCreateOpenGLPreviewFunc(); } IDeckLinkVideoConversion* CreateVideoConversionInstance (void) { pthread_once(&gDeckLinkOnceControl, InitDeckLinkAPI); if (gCreateVideoConversionFunc == NULL) return NULL; return gCreateVideoConversionFunc(); } IDeckLinkDiscovery* CreateDeckLinkDiscoveryInstance (void) { pthread_once(&gDeckLinkOnceControl, InitDeckLinkAPI); if (gCreateDeckLinkDiscoveryFunc == NULL) return NULL; return gCreateDeckLinkDiscoveryFunc(); } nageru-1.4.2/decklink/DeckLinkAPIModes.h000066400000000000000000000224711301561706600177550ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPIMODES_H #define BMD_DECKLINKAPIMODES_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif // Type Declarations // Interface ID Declarations BMD_CONST REFIID IID_IDeckLinkDisplayModeIterator = /* 9C88499F-F601-4021-B80B-032E4EB41C35 */ {0x9C,0x88,0x49,0x9F,0xF6,0x01,0x40,0x21,0xB8,0x0B,0x03,0x2E,0x4E,0xB4,0x1C,0x35}; BMD_CONST REFIID IID_IDeckLinkDisplayMode = /* 3EB2C1AB-0A3D-4523-A3AD-F40D7FB14E78 */ {0x3E,0xB2,0xC1,0xAB,0x0A,0x3D,0x45,0x23,0xA3,0xAD,0xF4,0x0D,0x7F,0xB1,0x4E,0x78}; /* Enum BMDDisplayMode - Video display modes */ typedef uint32_t BMDDisplayMode; enum _BMDDisplayMode { /* SD Modes */ bmdModeNTSC = /* 'ntsc' */ 0x6E747363, bmdModeNTSC2398 = /* 'nt23' */ 0x6E743233, // 3:2 pulldown bmdModePAL = /* 'pal ' */ 0x70616C20, bmdModeNTSCp = /* 'ntsp' */ 0x6E747370, bmdModePALp = /* 'palp' */ 0x70616C70, /* HD 1080 Modes */ bmdModeHD1080p2398 = /* '23ps' */ 0x32337073, bmdModeHD1080p24 = /* '24ps' */ 0x32347073, bmdModeHD1080p25 = /* 'Hp25' */ 0x48703235, bmdModeHD1080p2997 = /* 'Hp29' */ 0x48703239, bmdModeHD1080p30 = /* 'Hp30' */ 0x48703330, bmdModeHD1080i50 = /* 'Hi50' */ 0x48693530, bmdModeHD1080i5994 = /* 'Hi59' */ 0x48693539, bmdModeHD1080i6000 = /* 'Hi60' */ 0x48693630, // N.B. This _really_ is 60.00 Hz. bmdModeHD1080p50 = /* 'Hp50' */ 0x48703530, bmdModeHD1080p5994 = /* 'Hp59' */ 0x48703539, bmdModeHD1080p6000 = /* 'Hp60' */ 0x48703630, // N.B. This _really_ is 60.00 Hz. /* HD 720 Modes */ bmdModeHD720p50 = /* 'hp50' */ 0x68703530, bmdModeHD720p5994 = /* 'hp59' */ 0x68703539, bmdModeHD720p60 = /* 'hp60' */ 0x68703630, /* 2k Modes */ bmdMode2k2398 = /* '2k23' */ 0x326B3233, bmdMode2k24 = /* '2k24' */ 0x326B3234, bmdMode2k25 = /* '2k25' */ 0x326B3235, /* DCI Modes (output only) */ bmdMode2kDCI2398 = /* '2d23' */ 0x32643233, bmdMode2kDCI24 = /* '2d24' */ 0x32643234, bmdMode2kDCI25 = /* '2d25' */ 0x32643235, /* 4k Modes */ bmdMode4K2160p2398 = /* '4k23' */ 0x346B3233, bmdMode4K2160p24 = /* '4k24' */ 0x346B3234, bmdMode4K2160p25 = /* '4k25' */ 0x346B3235, bmdMode4K2160p2997 = /* '4k29' */ 0x346B3239, bmdMode4K2160p30 = /* '4k30' */ 0x346B3330, bmdMode4K2160p50 = /* '4k50' */ 0x346B3530, bmdMode4K2160p5994 = /* '4k59' */ 0x346B3539, bmdMode4K2160p60 = /* '4k60' */ 0x346B3630, /* DCI Modes (output only) */ bmdMode4kDCI2398 = /* '4d23' */ 0x34643233, bmdMode4kDCI24 = /* '4d24' */ 0x34643234, bmdMode4kDCI25 = /* '4d25' */ 0x34643235, /* Special Modes */ bmdModeUnknown = /* 'iunk' */ 0x69756E6B }; /* Enum BMDFieldDominance - Video field dominance */ typedef uint32_t BMDFieldDominance; enum _BMDFieldDominance { bmdUnknownFieldDominance = 0, bmdLowerFieldFirst = /* 'lowr' */ 0x6C6F7772, bmdUpperFieldFirst = /* 'uppr' */ 0x75707072, bmdProgressiveFrame = /* 'prog' */ 0x70726F67, bmdProgressiveSegmentedFrame = /* 'psf ' */ 0x70736620 }; /* Enum BMDPixelFormat - Video pixel formats supported for output/input */ typedef uint32_t BMDPixelFormat; enum _BMDPixelFormat { bmdFormat8BitYUV = /* '2vuy' */ 0x32767579, bmdFormat10BitYUV = /* 'v210' */ 0x76323130, bmdFormat8BitARGB = 32, bmdFormat8BitBGRA = /* 'BGRA' */ 0x42475241, bmdFormat10BitRGB = /* 'r210' */ 0x72323130, // Big-endian RGB 10-bit per component with SMPTE video levels (64-960). Packed as 2:10:10:10 bmdFormat12BitRGB = /* 'R12B' */ 0x52313242, // Big-endian RGB 12-bit per component with full range (0-4095). Packed as 12-bit per component bmdFormat12BitRGBLE = /* 'R12L' */ 0x5231324C, // Little-endian RGB 12-bit per component with full range (0-4095). Packed as 12-bit per component bmdFormat10BitRGBXLE = /* 'R10l' */ 0x5231306C, // Little-endian 10-bit RGB with SMPTE video levels (64-940) bmdFormat10BitRGBX = /* 'R10b' */ 0x52313062, // Big-endian 10-bit RGB with SMPTE video levels (64-940) bmdFormatH265 = /* 'hev1' */ 0x68657631 // High Efficiency Video Coding (HEVC/h.265) }; /* Enum BMDDisplayModeFlags - Flags to describe the characteristics of an IDeckLinkDisplayMode. */ typedef uint32_t BMDDisplayModeFlags; enum _BMDDisplayModeFlags { bmdDisplayModeSupports3D = 1 << 0, bmdDisplayModeColorspaceRec601 = 1 << 1, bmdDisplayModeColorspaceRec709 = 1 << 2 }; // Forward Declarations class IDeckLinkDisplayModeIterator; class IDeckLinkDisplayMode; /* Interface IDeckLinkDisplayModeIterator - enumerates over supported input/output display modes. */ class IDeckLinkDisplayModeIterator : public IUnknown { public: virtual HRESULT Next (/* out */ IDeckLinkDisplayMode **deckLinkDisplayMode) = 0; protected: virtual ~IDeckLinkDisplayModeIterator () {} // call Release method to drop reference count }; /* Interface IDeckLinkDisplayMode - represents a display mode */ class IDeckLinkDisplayMode : public IUnknown { public: virtual HRESULT GetName (/* out */ const char **name) = 0; virtual BMDDisplayMode GetDisplayMode (void) = 0; virtual long GetWidth (void) = 0; virtual long GetHeight (void) = 0; virtual HRESULT GetFrameRate (/* out */ BMDTimeValue *frameDuration, /* out */ BMDTimeScale *timeScale) = 0; virtual BMDFieldDominance GetFieldDominance (void) = 0; virtual BMDDisplayModeFlags GetFlags (void) = 0; protected: virtual ~IDeckLinkDisplayMode () {} // call Release method to drop reference count }; /* Functions */ extern "C" { } #endif /* defined(BMD_DECKLINKAPIMODES_H) */ nageru-1.4.2/decklink/DeckLinkAPITypes.h000066400000000000000000000110551301561706600200060ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2015 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef BMD_DECKLINKAPITYPES_H #define BMD_DECKLINKAPITYPES_H #ifndef BMD_CONST #if defined(_MSC_VER) #define BMD_CONST __declspec(selectany) static const #else #define BMD_CONST static const #endif #endif // Type Declarations typedef int64_t BMDTimeValue; typedef int64_t BMDTimeScale; typedef uint32_t BMDTimecodeBCD; typedef uint32_t BMDTimecodeUserBits; // Interface ID Declarations BMD_CONST REFIID IID_IDeckLinkTimecode = /* BC6CFBD3-8317-4325-AC1C-1216391E9340 */ {0xBC,0x6C,0xFB,0xD3,0x83,0x17,0x43,0x25,0xAC,0x1C,0x12,0x16,0x39,0x1E,0x93,0x40}; /* Enum BMDTimecodeFlags - Timecode flags */ typedef uint32_t BMDTimecodeFlags; enum _BMDTimecodeFlags { bmdTimecodeFlagDefault = 0, bmdTimecodeIsDropFrame = 1 << 0, bmdTimecodeFieldMark = 1 << 1 }; /* Enum BMDVideoConnection - Video connection types */ typedef uint32_t BMDVideoConnection; enum _BMDVideoConnection { bmdVideoConnectionSDI = 1 << 0, bmdVideoConnectionHDMI = 1 << 1, bmdVideoConnectionOpticalSDI = 1 << 2, bmdVideoConnectionComponent = 1 << 3, bmdVideoConnectionComposite = 1 << 4, bmdVideoConnectionSVideo = 1 << 5 }; /* Enum BMDAudioConnection - Audio connection types */ typedef uint32_t BMDAudioConnection; enum _BMDAudioConnection { bmdAudioConnectionEmbedded = 1 << 0, bmdAudioConnectionAESEBU = 1 << 1, bmdAudioConnectionAnalog = 1 << 2, bmdAudioConnectionAnalogXLR = 1 << 3, bmdAudioConnectionAnalogRCA = 1 << 4, bmdAudioConnectionMicrophone = 1 << 5, bmdAudioConnectionHeadphones = 1 << 6 }; /* Enum BMDDeckControlConnection - Deck control connections */ typedef uint32_t BMDDeckControlConnection; enum _BMDDeckControlConnection { bmdDeckControlConnectionRS422Remote1 = 1 << 0, bmdDeckControlConnectionRS422Remote2 = 1 << 1 }; // Forward Declarations class IDeckLinkTimecode; /* Interface IDeckLinkTimecode - Used for video frame timecode representation. */ class IDeckLinkTimecode : public IUnknown { public: virtual BMDTimecodeBCD GetBCD (void) = 0; virtual HRESULT GetComponents (/* out */ uint8_t *hours, /* out */ uint8_t *minutes, /* out */ uint8_t *seconds, /* out */ uint8_t *frames) = 0; virtual HRESULT GetString (/* out */ const char **timecode) = 0; virtual BMDTimecodeFlags GetFlags (void) = 0; virtual HRESULT GetTimecodeUserBits (/* out */ BMDTimecodeUserBits *userBits) = 0; protected: virtual ~IDeckLinkTimecode () {} // call Release method to drop reference count }; /* Functions */ extern "C" { } #endif /* defined(BMD_DECKLINKAPITYPES_H) */ nageru-1.4.2/decklink/LinuxCOM.h000066400000000000000000000065411301561706600164050ustar00rootroot00000000000000/* -LICENSE-START- ** Copyright (c) 2009 Blackmagic Design ** ** Permission is hereby granted, free of charge, to any person or organization ** obtaining a copy of the software and accompanying documentation covered by ** this license (the "Software") to use, reproduce, display, distribute, ** execute, and transmit the Software, and to prepare derivative works of the ** Software, and to permit third-parties to whom the Software is furnished to ** do so, all subject to the following: ** ** The copyright notices in the Software and this entire statement, including ** the above license grant, this restriction and the following disclaimer, ** must be included in all copies of the Software, in whole or in part, and ** all derivative works of the Software, unless such copies or derivative ** works are solely in the form of machine-executable object code generated by ** a source language processor. ** ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ** FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT ** SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE ** FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ** ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER ** DEALINGS IN THE SOFTWARE. ** -LICENSE-END- */ #ifndef __LINUX_COM_H_ #define __LINUX_COM_H_ struct REFIID { unsigned char byte0; unsigned char byte1; unsigned char byte2; unsigned char byte3; unsigned char byte4; unsigned char byte5; unsigned char byte6; unsigned char byte7; unsigned char byte8; unsigned char byte9; unsigned char byte10; unsigned char byte11; unsigned char byte12; unsigned char byte13; unsigned char byte14; unsigned char byte15; }; typedef REFIID CFUUIDBytes; #define CFUUIDGetUUIDBytes(x) x typedef int HRESULT; typedef unsigned long ULONG; typedef void *LPVOID; #define SUCCEEDED(Status) ((HRESULT)(Status) >= 0) #define FAILED(Status) ((HRESULT)(Status)<0) #define IS_ERROR(Status) ((unsigned long)(Status) >> 31 == SEVERITY_ERROR) #define HRESULT_CODE(hr) ((hr) & 0xFFFF) #define HRESULT_FACILITY(hr) (((hr) >> 16) & 0x1fff) #define HRESULT_SEVERITY(hr) (((hr) >> 31) & 0x1) #define SEVERITY_SUCCESS 0 #define SEVERITY_ERROR 1 #define MAKE_HRESULT(sev,fac,code) ((HRESULT) (((unsigned long)(sev)<<31) | ((unsigned long)(fac)<<16) | ((unsigned long)(code))) ) #define S_OK ((HRESULT)0x00000000L) #define S_FALSE ((HRESULT)0x00000001L) #define E_UNEXPECTED ((HRESULT)0x8000FFFFL) #define E_NOTIMPL ((HRESULT)0x80000001L) #define E_OUTOFMEMORY ((HRESULT)0x80000002L) #define E_INVALIDARG ((HRESULT)0x80000003L) #define E_NOINTERFACE ((HRESULT)0x80000004L) #define E_POINTER ((HRESULT)0x80000005L) #define E_HANDLE ((HRESULT)0x80000006L) #define E_ABORT ((HRESULT)0x80000007L) #define E_FAIL ((HRESULT)0x80000008L) #define E_ACCESSDENIED ((HRESULT)0x80000009L) #define STDMETHODCALLTYPE #define IID_IUnknown (REFIID){0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0xC0,0x00,0x00,0x00,0x00,0x00,0x00,0x46} #define IUnknownUUID IID_IUnknown #ifdef __cplusplus class IUnknown { public: virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) = 0; virtual ULONG STDMETHODCALLTYPE AddRef(void) = 0; virtual ULONG STDMETHODCALLTYPE Release(void) = 0; }; #endif #endif nageru-1.4.2/decklink_capture.cpp000066400000000000000000000321551301561706600170250ustar00rootroot00000000000000#include "decklink_capture.h" #include #include #include #include #include #ifdef __SSE2__ #include #endif #include #include #include #include #include #include #include #include "bmusb/bmusb.h" #define FRAME_SIZE (8 << 20) // 8 MB. using namespace std; using namespace std::placeholders; using namespace bmusb; namespace { // TODO: Support stride. void memcpy_interleaved(uint8_t *dest1, uint8_t *dest2, const uint8_t *src, size_t n) { assert(n % 2 == 0); uint8_t *dptr1 = dest1; uint8_t *dptr2 = dest2; for (size_t i = 0; i < n; i += 2) { *dptr1++ = *src++; *dptr2++ = *src++; } } #ifdef __SSE2__ // Returns the number of bytes consumed. size_t memcpy_interleaved_fastpath(uint8_t *dest1, uint8_t *dest2, const uint8_t *src, size_t n) { const uint8_t *limit = src + n; size_t consumed = 0; // Align end to 32 bytes. limit = (const uint8_t *)(intptr_t(limit) & ~31); if (src >= limit) { return 0; } // Process [0,31] bytes, such that start gets aligned to 32 bytes. const uint8_t *aligned_src = (const uint8_t *)(intptr_t(src + 31) & ~31); if (aligned_src != src) { size_t n2 = aligned_src - src; memcpy_interleaved(dest1, dest2, src, n2); dest1 += n2 / 2; dest2 += n2 / 2; if (n2 % 2) { swap(dest1, dest2); } src = aligned_src; consumed += n2; } // Make the length a multiple of 64. if (((limit - src) % 64) != 0) { limit -= 32; } assert(((limit - src) % 64) == 0); #if __AVX2__ const __m256i * __restrict in = (const __m256i *)src; __m256i * __restrict out1 = (__m256i *)dest1; __m256i * __restrict out2 = (__m256i *)dest2; __m256i shuffle_cw = _mm256_set_epi8( 15, 13, 11, 9, 7, 5, 3, 1, 14, 12, 10, 8, 6, 4, 2, 0, 15, 13, 11, 9, 7, 5, 3, 1, 14, 12, 10, 8, 6, 4, 2, 0); while (in < (const __m256i *)limit) { // Note: For brevity, comments show lanes as if they were 2x64-bit (they're actually 2x128). __m256i data1 = _mm256_stream_load_si256(in); // AaBbCcDd EeFfGgHh __m256i data2 = _mm256_stream_load_si256(in + 1); // IiJjKkLl MmNnOoPp data1 = _mm256_shuffle_epi8(data1, shuffle_cw); // ABCDabcd EFGHefgh data2 = _mm256_shuffle_epi8(data2, shuffle_cw); // IJKLijkl MNOPmnop data1 = _mm256_permute4x64_epi64(data1, 0b11011000); // ABCDEFGH abcdefgh data2 = _mm256_permute4x64_epi64(data2, 0b11011000); // IJKLMNOP ijklmnop __m256i lo = _mm256_permute2x128_si256(data1, data2, 0b00100000); __m256i hi = _mm256_permute2x128_si256(data1, data2, 0b00110001); _mm256_storeu_si256(out1, lo); _mm256_storeu_si256(out2, hi); in += 2; ++out1; ++out2; consumed += 64; } #else const __m128i * __restrict in = (const __m128i *)src; __m128i * __restrict out1 = (__m128i *)dest1; __m128i * __restrict out2 = (__m128i *)dest2; __m128i mask_lower_byte = _mm_set1_epi16(0x00ff); while (in < (const __m128i *)limit) { __m128i data1 = _mm_load_si128(in); __m128i data2 = _mm_load_si128(in + 1); __m128i data1_lo = _mm_and_si128(data1, mask_lower_byte); __m128i data2_lo = _mm_and_si128(data2, mask_lower_byte); __m128i data1_hi = _mm_srli_epi16(data1, 8); __m128i data2_hi = _mm_srli_epi16(data2, 8); __m128i lo = _mm_packus_epi16(data1_lo, data2_lo); _mm_storeu_si128(out1, lo); __m128i hi = _mm_packus_epi16(data1_hi, data2_hi); _mm_storeu_si128(out2, hi); in += 2; ++out1; ++out2; consumed += 32; } #endif return consumed; } #endif // __SSE2__ } // namespace DeckLinkCapture::DeckLinkCapture(IDeckLink *card, int card_index) : card_index(card_index), card(card) { { const char *model_name; char buf[256]; if (card->GetModelName(&model_name) == S_OK) { snprintf(buf, sizeof(buf), "PCI card %d: %s", card_index, model_name); } else { snprintf(buf, sizeof(buf), "PCI card %d: Unknown DeckLink card", card_index); } description = buf; } if (card->QueryInterface(IID_IDeckLinkInput, (void**)&input) != S_OK) { fprintf(stderr, "Card %d has no inputs\n", card_index); exit(1); } IDeckLinkAttributes *attr; if (card->QueryInterface(IID_IDeckLinkAttributes, (void**)&attr) != S_OK) { fprintf(stderr, "Card %d has no attributes\n", card_index); exit(1); } // Get the list of available video inputs. int64_t video_input_mask; if (attr->GetInt(BMDDeckLinkVideoInputConnections, &video_input_mask) != S_OK) { fprintf(stderr, "Failed to enumerate video inputs for card %d\n", card_index); exit(1); } const vector> video_input_types = { { bmdVideoConnectionSDI, "SDI" }, { bmdVideoConnectionHDMI, "HDMI" }, { bmdVideoConnectionOpticalSDI, "Optical SDI" }, { bmdVideoConnectionComponent, "Component" }, { bmdVideoConnectionComposite, "Composite" }, { bmdVideoConnectionSVideo, "S-Video" } }; for (const auto &video_input : video_input_types) { if (video_input_mask & video_input.first) { video_inputs.emplace(video_input.first, video_input.second); } } // And then the available audio inputs. int64_t audio_input_mask; if (attr->GetInt(BMDDeckLinkAudioInputConnections, &audio_input_mask) != S_OK) { fprintf(stderr, "Failed to enumerate audio inputs for card %d\n", card_index); exit(1); } const vector> audio_input_types = { { bmdAudioConnectionEmbedded, "Embedded" }, { bmdAudioConnectionAESEBU, "AES/EBU" }, { bmdAudioConnectionAnalog, "Analog" }, { bmdAudioConnectionAnalogXLR, "Analog XLR" }, { bmdAudioConnectionAnalogRCA, "Analog RCA" }, { bmdAudioConnectionMicrophone, "Microphone" }, { bmdAudioConnectionHeadphones, "Headphones" } }; for (const auto &audio_input : audio_input_types) { if (audio_input_mask & audio_input.first) { audio_inputs.emplace(audio_input.first, audio_input.second); } } attr->Release(); /* Set up the video and audio sources. */ if (card->QueryInterface(IID_IDeckLinkConfiguration, (void**)&config) != S_OK) { fprintf(stderr, "Failed to get configuration interface for card %d\n", card_index); exit(1); } set_video_input(bmdVideoConnectionHDMI); set_audio_input(bmdAudioConnectionEmbedded); IDeckLinkDisplayModeIterator *mode_it; if (input->GetDisplayModeIterator(&mode_it) != S_OK) { fprintf(stderr, "Failed to enumerate display modes for card %d\n", card_index); exit(1); } for (IDeckLinkDisplayMode *mode_ptr; mode_it->Next(&mode_ptr) == S_OK; mode_ptr->Release()) { VideoMode mode; const char *mode_name; if (mode_ptr->GetName(&mode_name) != S_OK) { mode.name = "Unknown mode"; } else { mode.name = mode_name; } mode.autodetect = false; mode.width = mode_ptr->GetWidth(); mode.height = mode_ptr->GetHeight(); BMDTimeScale frame_rate_num; BMDTimeValue frame_rate_den; if (mode_ptr->GetFrameRate(&frame_rate_den, &frame_rate_num) != S_OK) { fprintf(stderr, "Could not get frame rate for mode '%s' on card %d\n", mode.name.c_str(), card_index); exit(1); } mode.frame_rate_num = frame_rate_num; mode.frame_rate_den = frame_rate_den; // TODO: Respect the TFF/BFF flag. mode.interlaced = (mode_ptr->GetFieldDominance() == bmdLowerFieldFirst || mode_ptr->GetFieldDominance() == bmdUpperFieldFirst); uint32_t id = mode_ptr->GetDisplayMode(); video_modes.insert(make_pair(id, mode)); } set_video_mode_no_restart(bmdModeHD720p5994); if (input->EnableAudioInput(48000, bmdAudioSampleType32bitInteger, 2) != S_OK) { fprintf(stderr, "Failed to enable audio input for card %d\n", card_index); exit(1); } input->SetCallback(this); } DeckLinkCapture::~DeckLinkCapture() { if (has_dequeue_callbacks) { dequeue_cleanup_callback(); } input->Release(); config->Release(); card->Release(); } HRESULT STDMETHODCALLTYPE DeckLinkCapture::QueryInterface(REFIID, LPVOID *) { return E_NOINTERFACE; } ULONG STDMETHODCALLTYPE DeckLinkCapture::AddRef(void) { return refcount.fetch_add(1) + 1; } ULONG STDMETHODCALLTYPE DeckLinkCapture::Release(void) { int new_ref = refcount.fetch_sub(1) - 1; if (new_ref == 0) delete this; return new_ref; } HRESULT STDMETHODCALLTYPE DeckLinkCapture::VideoInputFormatChanged( BMDVideoInputFormatChangedEvents, IDeckLinkDisplayMode* display_mode, BMDDetectedVideoInputFormatFlags) { if (display_mode->GetFrameRate(&frame_duration, &time_scale) != S_OK) { fprintf(stderr, "Could not get new frame rate\n"); exit(1); } return S_OK; } HRESULT STDMETHODCALLTYPE DeckLinkCapture::VideoInputFrameArrived( IDeckLinkVideoInputFrame *video_frame, IDeckLinkAudioInputPacket *audio_frame) { if (!done_init) { if (has_dequeue_callbacks) { dequeue_init_callback(); } done_init = true; } FrameAllocator::Frame current_video_frame, current_audio_frame; VideoFormat video_format; AudioFormat audio_format; video_format.frame_rate_nom = time_scale; video_format.frame_rate_den = frame_duration; if (video_frame) { video_format.has_signal = !(video_frame->GetFlags() & bmdFrameHasNoInputSource); int width = video_frame->GetWidth(); int height = video_frame->GetHeight(); const int stride = video_frame->GetRowBytes(); assert(stride == width * 2); current_video_frame = video_frame_allocator->alloc_frame(); if (current_video_frame.data != nullptr) { const uint8_t *frame_bytes; video_frame->GetBytes((void **)&frame_bytes); uint8_t *data = current_video_frame.data; uint8_t *data2 = current_video_frame.data2; size_t num_bytes = width * height * 2; #ifdef __SSE2__ size_t consumed = memcpy_interleaved_fastpath(data, data2, frame_bytes, num_bytes); frame_bytes += consumed; data += consumed / 2; data2 += consumed / 2; if (num_bytes % 2) { swap(data, data2); } current_video_frame.len += consumed; num_bytes -= consumed; #endif if (num_bytes > 0) { memcpy_interleaved(data, data2, frame_bytes, num_bytes); } current_video_frame.len += num_bytes; video_format.width = width; video_format.height = height; } } if (audio_frame) { int num_samples = audio_frame->GetSampleFrameCount(); current_audio_frame = audio_frame_allocator->alloc_frame(); if (current_audio_frame.data != nullptr) { const uint8_t *frame_bytes; audio_frame->GetBytes((void **)&frame_bytes); current_audio_frame.len = sizeof(int32_t) * 2 * num_samples; memcpy(current_audio_frame.data, frame_bytes, current_audio_frame.len); audio_format.bits_per_sample = 32; audio_format.num_channels = 2; } } if (current_video_frame.data != nullptr || current_audio_frame.data != nullptr) { // TODO: Put into a queue and put into a dequeue thread, if the // BlackMagic drivers don't already do that for us? frame_callback(timecode, current_video_frame, /*video_offset=*/0, video_format, current_audio_frame, /*audio_offset=*/0, audio_format); } timecode++; return S_OK; } void DeckLinkCapture::configure_card() { if (video_frame_allocator == nullptr) { owned_video_frame_allocator.reset(new MallocFrameAllocator(FRAME_SIZE, NUM_QUEUED_VIDEO_FRAMES)); set_video_frame_allocator(owned_video_frame_allocator.get()); } if (audio_frame_allocator == nullptr) { owned_audio_frame_allocator.reset(new MallocFrameAllocator(65536, NUM_QUEUED_AUDIO_FRAMES)); set_audio_frame_allocator(owned_audio_frame_allocator.get()); } } void DeckLinkCapture::start_bm_capture() { if (input->StartStreams() != S_OK) { fprintf(stderr, "StartStreams failed\n"); exit(1); } } void DeckLinkCapture::stop_dequeue_thread() { if (input->StopStreams() != S_OK) { fprintf(stderr, "StopStreams failed\n"); exit(1); } } void DeckLinkCapture::set_video_mode(uint32_t video_mode_id) { if (input->StopStreams() != S_OK) { fprintf(stderr, "StopStreams failed\n"); exit(1); } set_video_mode_no_restart(video_mode_id); if (input->StartStreams() != S_OK) { fprintf(stderr, "StartStreams failed\n"); exit(1); } } void DeckLinkCapture::set_video_mode_no_restart(uint32_t video_mode_id) { BMDDisplayModeSupport support; IDeckLinkDisplayMode *display_mode; if (input->DoesSupportVideoMode(video_mode_id, bmdFormat8BitYUV, /*flags=*/0, &support, &display_mode)) { fprintf(stderr, "Failed to query display mode for card %d\n", card_index); exit(1); } if (support == bmdDisplayModeNotSupported) { fprintf(stderr, "Card %d does not support display mode\n", card_index); exit(1); } if (display_mode->GetFrameRate(&frame_duration, &time_scale) != S_OK) { fprintf(stderr, "Could not get frame rate for card %d\n", card_index); exit(1); } if (input->EnableVideoInput(video_mode_id, bmdFormat8BitYUV, 0) != S_OK) { fprintf(stderr, "Failed to set video mode 0x%04x for card %d\n", video_mode_id, card_index); exit(1); } current_video_mode = video_mode_id; } void DeckLinkCapture::set_video_input(uint32_t video_input_id) { if (config->SetInt(bmdDeckLinkConfigVideoInputConnection, video_input_id) != S_OK) { fprintf(stderr, "Failed to set video input connection for card %d\n", card_index); exit(1); } current_video_input = video_input_id; } void DeckLinkCapture::set_audio_input(uint32_t audio_input_id) { if (config->SetInt(bmdDeckLinkConfigAudioInputConnection, audio_input_id) != S_OK) { fprintf(stderr, "Failed to set audio input connection for card %d\n", card_index); exit(1); } current_audio_input = audio_input_id; } nageru-1.4.2/decklink_capture.h000066400000000000000000000104041301561706600164630ustar00rootroot00000000000000#ifndef _DECKLINK_CAPTURE_H #define _DECKLINK_CAPTURE_H 1 #include #include #include #include #include #include #include #include "DeckLinkAPIModes.h" #include "DeckLinkAPITypes.h" #include "LinuxCOM.h" #include "bmusb/bmusb.h" class IDeckLink; class IDeckLinkConfiguration; // TODO: Adjust CaptureInterface to be a little less bmusb-centric. // There are too many member functions here that don't really do anything. class DeckLinkCapture : public bmusb::CaptureInterface, IDeckLinkInputCallback { public: DeckLinkCapture(IDeckLink *card, int card_index); // Takes ownership of . ~DeckLinkCapture(); // IDeckLinkInputCallback. HRESULT STDMETHODCALLTYPE QueryInterface(REFIID, LPVOID *) override; ULONG STDMETHODCALLTYPE AddRef() override; ULONG STDMETHODCALLTYPE Release() override; HRESULT STDMETHODCALLTYPE VideoInputFormatChanged( BMDVideoInputFormatChangedEvents, IDeckLinkDisplayMode*, BMDDetectedVideoInputFormatFlags) override; HRESULT STDMETHODCALLTYPE VideoInputFrameArrived( IDeckLinkVideoInputFrame *video_frame, IDeckLinkAudioInputPacket *audio_frame) override; // CaptureInterface. void set_video_frame_allocator(bmusb::FrameAllocator *allocator) override { video_frame_allocator = allocator; if (owned_video_frame_allocator.get() != allocator) { owned_video_frame_allocator.reset(); } } bmusb::FrameAllocator *get_video_frame_allocator() override { return video_frame_allocator; } // Does not take ownership. void set_audio_frame_allocator(bmusb::FrameAllocator *allocator) override { audio_frame_allocator = allocator; if (owned_audio_frame_allocator.get() != allocator) { owned_audio_frame_allocator.reset(); } } bmusb::FrameAllocator *get_audio_frame_allocator() override { return audio_frame_allocator; } void set_frame_callback(bmusb::frame_callback_t callback) override { frame_callback = callback; } void set_dequeue_thread_callbacks(std::function init, std::function cleanup) override { dequeue_init_callback = init; dequeue_cleanup_callback = cleanup; has_dequeue_callbacks = true; } std::string get_description() const override { return description; } void configure_card() override; void start_bm_capture() override; void stop_dequeue_thread() override; // TODO: Can the API communicate this to us somehow, for e.g. Thunderbolt cards? bool get_disconnected() const override { return false; } std::map get_available_video_modes() const override { return video_modes; } void set_video_mode(uint32_t video_mode_id) override; uint32_t get_current_video_mode() const override { return current_video_mode; } std::map get_available_video_inputs() const override { return video_inputs; } void set_video_input(uint32_t video_input_id) override; uint32_t get_current_video_input() const override { return current_video_input; } std::map get_available_audio_inputs() const override { return audio_inputs; } void set_audio_input(uint32_t audio_input_id) override; uint32_t get_current_audio_input() const override { return current_audio_input; } private: void set_video_mode_no_restart(uint32_t video_mode_id); std::atomic refcount{1}; bool done_init = false; std::string description; uint16_t timecode = 0; int card_index; bool has_dequeue_callbacks = false; std::function dequeue_init_callback = nullptr; std::function dequeue_cleanup_callback = nullptr; bmusb::FrameAllocator *video_frame_allocator = nullptr; bmusb::FrameAllocator *audio_frame_allocator = nullptr; std::unique_ptr owned_video_frame_allocator; std::unique_ptr owned_audio_frame_allocator; bmusb::frame_callback_t frame_callback = nullptr; IDeckLinkConfiguration *config = nullptr; IDeckLink *card = nullptr; IDeckLinkInput *input = nullptr; BMDTimeValue frame_duration; BMDTimeScale time_scale; std::map video_modes; BMDDisplayMode current_video_mode; std::map video_inputs; BMDVideoConnection current_video_input; std::map audio_inputs; BMDAudioConnection current_audio_input; }; #endif // !defined(_DECKLINK_CAPTURE_H) nageru-1.4.2/defs.h000066400000000000000000000031431301561706600140770ustar00rootroot00000000000000#ifndef _DEFS_H #define _DEFS_H #define OUTPUT_FREQUENCY 48000 // Currently needs to be exactly 48000, since bmusb outputs in that. #define MAX_FPS 60 #define WIDTH 1280 #define HEIGHT 720 #define FAKE_FPS 25 // Must be an integer. #define MAX_VIDEO_CARDS 16 #define MAX_ALSA_CARDS 16 #define MAX_BUSES 256 // Audio buses. // For deinterlacing. See also comments on InputState. #define FRAME_HISTORY_LENGTH 5 #define AUDIO_OUTPUT_CODEC_NAME "pcm_s32le" #define DEFAULT_AUDIO_OUTPUT_BIT_RATE 0 #define DEFAULT_X264_OUTPUT_BIT_RATE 4500 // 5 Mbit after making room for some audio and TCP overhead. #define LOCAL_DUMP_PREFIX "record-" #define LOCAL_DUMP_SUFFIX ".nut" #define DEFAULT_STREAM_MUX_NAME "nut" // Only for HTTP. Local dump guesses from LOCAL_DUMP_SUFFIX. #define MUX_OPTS { \ /* Make seekable .mov files. */ \ { "movflags", "empty_moov+frag_keyframe+default_base_moof" }, \ \ /* Make for somewhat less bursty stream output when using .mov. */ \ { "frag_duration", "125000" }, \ \ /* Keep nut muxer from using unlimited amounts of memory. */ \ { "write_index", "0" } \ } // In bytes. Beware, if too small, stream clients will start dropping data. // For mov, you want this at 10MB or so (for the reason mentioned above), // but for nut, there's no flushing, so such a large mux buffer would cause // the output to be very uneven. #define MUX_BUFFER_SIZE 10485760 // In number of frames. Comes in addition to any internal queues in x264 // (frame threading, lookahead, etc.). #define X264_QUEUE_LENGTH 50 #define X264_DEFAULT_PRESET "ultrafast" #define X264_DEFAULT_TUNE "film" #endif // !defined(_DEFS_H) nageru-1.4.2/disk_space_estimator.cpp000066400000000000000000000025621301561706600177110ustar00rootroot00000000000000#include "disk_space_estimator.h" #include #include #include #include #include "timebase.h" DiskSpaceEstimator::DiskSpaceEstimator(DiskSpaceEstimator::callback_t callback) : callback(callback) { } void DiskSpaceEstimator::report_write(const std::string &filename, uint64_t pts) { if (filename != last_filename) { last_filename = filename; measure_points.clear(); } // Reject points that are out-of-order (happens with B-frames). if (!measure_points.empty() && pts < measure_points.back().pts) { return; } // Remove too old points. while (measure_points.size() > 1 && measure_points.front().pts + window_length < pts) { measure_points.pop_front(); } struct stat st; if (stat(filename.c_str(), &st) == -1) { perror(filename.c_str()); return; } struct statfs fst; if (statfs(filename.c_str(), &fst) == -1) { perror(filename.c_str()); return; } if (!measure_points.empty()) { off_t free_bytes = off_t(fst.f_bavail) * fst.f_frsize; double bytes_per_second = double(st.st_size - measure_points.front().size) / (pts - measure_points.front().pts) * TIMEBASE; double seconds_left = free_bytes / bytes_per_second; callback(free_bytes, seconds_left); } measure_points.push_back({ pts, st.st_size }); } DiskSpaceEstimator *global_disk_space_estimator = nullptr; // Created in MainWindow::MainWindow(). nageru-1.4.2/disk_space_estimator.h000066400000000000000000000030341301561706600173510ustar00rootroot00000000000000#ifndef _DISK_SPACE_ESTIMATOR_H #define _DISK_SPACE_ESTIMATOR_H // A class responsible for measuring how much disk there is left when we // store our video to disk, and how much recording time that equates to. // It gets callbacks from the Mux writing the stream to disk (which also // knows which filesystem the file is going to), makes its calculations, // and calls back to the MainWindow, which shows it to the user. // // The bitrate is measured over a simple 30-second sliding window. #include #include #include #include #include #include "timebase.h" class DiskSpaceEstimator { public: typedef std::function callback_t; DiskSpaceEstimator(callback_t callback); // Report that a video frame with the given pts has just been written // to the given file, so the estimator should stat the file and see // by how much it grew since last time. Called by the Mux object // responsible for writing to the stream on disk. // // If the filename changed since last time, the estimation is reset. // is taken to be in TIMEBASE units (see timebase.h). void report_write(const std::string &filename, uint64_t pts); private: static constexpr int64_t window_length = 30 * TIMEBASE; callback_t callback; std::string last_filename; struct MeasurePoint { uint64_t pts; off_t size; }; std::deque measure_points; }; extern DiskSpaceEstimator *global_disk_space_estimator; #endif // !defined(_DISK_SPACE_ESTIMATOR_H) nageru-1.4.2/ebu_r128_proc.cc000066400000000000000000000151561301561706600156750ustar00rootroot00000000000000// ------------------------------------------------------------------------ // // Copyright (C) 2010-2011 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 2 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program; if not, write to the Free Software // Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. // // ------------------------------------------------------------------------ #include "ebu_r128_proc.h" #include #include float Ebu_r128_hist::_bin_power [100] = { 0.0f }; float Ebu_r128_proc::_chan_gain [5] = { 1.0f, 1.0f, 1.0f, 1.41f, 1.41f }; Ebu_r128_hist::Ebu_r128_hist (void) { _histc = new int [751]; initstat (); reset (); } Ebu_r128_hist::~Ebu_r128_hist (void) { delete[] _histc; } void Ebu_r128_hist::reset (void) { memset (_histc, 0, 751 * sizeof (float)); _count = 0; _error = 0; } void Ebu_r128_hist::initstat (void) { int i; if (_bin_power [0]) return; for (i = 0; i < 100; i++) { _bin_power [i] = powf (10.0f, i / 100.0f); } } void Ebu_r128_hist::addpoint (float v) { int k; k = (int) floorf (10 * v + 700.5f); if (k < 0) return; if (k > 750) { k = 750; _error++; } _histc [k]++; _count++; } float Ebu_r128_hist::integrate (int i) { int j, k, n; float s; j = i % 100; n = 0; s = 0; while (i <= 750) { k = _histc [i++]; n += k; s += k * _bin_power [j++]; if (j == 100) { j = 0; s /= 10.0f; } } return s / n; } void Ebu_r128_hist::calc_integ (float *vi, float *th) { int k; float s; if (_count < 50) { *vi = -200.0f; return; } s = integrate (0); // Original threshold was -8 dB below result of first integration // if (th) *th = 10 * log10f (s) - 8.0f; // k = (int)(floorf (100 * log10f (s) + 0.5f)) + 620; // Threshold redefined to -10 dB below result of first integration if (th) *th = 10 * log10f (s) - 10.0f; k = (int)(floorf (100 * log10f (s) + 0.5f)) + 600; if (k < 0) k = 0; s = integrate (k); *vi = 10 * log10f (s); } void Ebu_r128_hist::calc_range (float *v0, float *v1, float *th) { int i, j, k, n; float a, b, s; if (_count < 20) { *v0 = -200.0f; *v1 = -200.0f; return; } s = integrate (0); if (th) *th = 10 * log10f (s) - 20.0f; k = (int)(floorf (100 * log10f (s) + 0.5)) + 500; if (k < 0) k = 0; for (i = k, n = 0; i <= 750; i++) n += _histc [i]; a = 0.10f * n; b = 0.95f * n; for (i = k, s = 0; s < a; i++) s += _histc [i]; for (j = 750, s = n; s > b; j--) s -= _histc [j]; *v0 = (i - 701) / 10.0f; *v1 = (j - 699) / 10.0f; } Ebu_r128_proc::Ebu_r128_proc (void) { reset (); } Ebu_r128_proc::~Ebu_r128_proc (void) { } void Ebu_r128_proc::init (int nchan, float fsamp) { _nchan = nchan; _fsamp = fsamp; _fragm = (int) fsamp / 20; detect_init (_fsamp); reset (); } void Ebu_r128_proc::reset (void) { _integr = false; _frcnt = _fragm; _frpwr = 1e-30f; _wrind = 0; _div1 = 0; _div2 = 0; _loudness_M = -200.0f; _loudness_S = -200.0f; memset (_power, 0, 64 * sizeof (float)); integr_reset (); detect_reset (); } void Ebu_r128_proc::integr_reset (void) { _hist_M.reset (); _hist_S.reset (); _maxloudn_M = -200.0f; _maxloudn_S = -200.0f; _integrated = -200.0f; _integ_thr = -200.0f; _range_min = -200.0f; _range_max = -200.0f; _range_thr = -200.0f; _div1 = _div2 = 0; } void Ebu_r128_proc::process (int nfram, float *input []) { int i, k; for (i = 0; i < _nchan; i++) _ipp [i] = input [i]; while (nfram) { k = (_frcnt < nfram) ? _frcnt : nfram; _frpwr += detect_process (k); _frcnt -= k; if (_frcnt == 0) { _power [_wrind++] = _frpwr / _fragm; _frcnt = _fragm; _frpwr = 1e-30f; _wrind &= 63; _loudness_M = addfrags (8); _loudness_S = addfrags (60); if (_loudness_M > _maxloudn_M) _maxloudn_M = _loudness_M; if (_loudness_S > _maxloudn_S) _maxloudn_S = _loudness_S; if (_integr) { if (++_div1 == 2) { _hist_M.addpoint (_loudness_M); _div1 = 0; } if (++_div2 == 10) { _hist_S.addpoint (_loudness_S); _div2 = 0; _hist_M.calc_integ (&_integrated, &_integ_thr); _hist_S.calc_range (&_range_min, &_range_max, &_range_thr); } } } for (i = 0; i < _nchan; i++) _ipp [i] += k; nfram -= k; } } float Ebu_r128_proc::addfrags (int nfrag) { int i, k; float s; s = 0; k = (_wrind - nfrag) & 63; for (i = 0; i < nfrag; i++) s += _power [(i + k) & 63]; return -0.6976f + 10 * log10f (s / nfrag); } void Ebu_r128_proc::detect_init (float fsamp) { float a, b, c, d, r, u1, u2, w1, w2; r = 1 / tan (4712.3890f / fsamp); w1 = r / 1.12201f; w2 = r * 1.12201f; u1 = u2 = 1.4085f + 210.0f / fsamp; a = u1 * w1; b = w1 * w1; c = u2 * w2; d = w2 * w2; r = 1 + a + b; _a0 = (1 + c + d) / r; _a1 = (2 - 2 * d) / r; _a2 = (1 - c + d) / r; _b1 = (2 - 2 * b) / r; _b2 = (1 - a + b) / r; r = 48.0f / fsamp; a = 4.9886075f * r; b = 6.2298014f * r * r; r = 1 + a + b; a *= 2 / r; b *= 4 / r; _c3 = a + b; _c4 = b; r = 1.004995f / r; _a0 *= r; _a1 *= r; _a2 *= r; } void Ebu_r128_proc::detect_reset (void) { for (int i = 0; i < MAXCH; i++) _fst [i].reset (); } float Ebu_r128_proc::detect_process (int nfram) { int i, j; float si, sj; float x, y, z1, z2, z3, z4; float *p; Ebu_r128_fst *S; si = 0; for (i = 0, S = _fst; i < _nchan; i++, S++) { z1 = S->_z1; z2 = S->_z2; z3 = S->_z3; z4 = S->_z4; p = _ipp [i]; sj = 0; for (j = 0; j < nfram; j++) { x = p [j] - _b1 * z1 - _b2 * z2 + 1e-15f; y = _a0 * x + _a1 * z1 + _a2 * z2 - _c3 * z3 - _c4 * z4; z2 = z1; z1 = x; z4 += z3; z3 += y; sj += y * y; } if (_nchan == 1) si = 2 * sj; else si += _chan_gain [i] * sj; S->_z1 = z1; S->_z2 = z2; S->_z3 = z3; S->_z4 = z4; } return si; } nageru-1.4.2/ebu_r128_proc.h000066400000000000000000000101441301561706600155270ustar00rootroot00000000000000// ------------------------------------------------------------------------ // // Copyright (C) 2010-2011 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 2 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program; if not, write to the Free Software // Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. // // ------------------------------------------------------------------------ #ifndef __EBU_R128_PROC_H #define __EBU_R128_PROC_H #define MAXCH 5 class Ebu_r128_fst { private: friend class Ebu_r128_proc; void reset (void) { _z1 = _z2 = _z3 = _z4 = 0; } float _z1, _z2, _z3, _z4; }; class Ebu_r128_hist { private: Ebu_r128_hist (void); ~Ebu_r128_hist (void); friend class Ebu_r128_proc; void reset (void); void initstat (void); void addpoint (float v); float integrate (int ind); void calc_integ (float *vi, float *th); void calc_range (float *v0, float *v1, float *th); int *_histc; int _count; int _error; static float _bin_power [100]; }; class Ebu_r128_proc { public: Ebu_r128_proc (void); ~Ebu_r128_proc (void); void init (int nchan, float fsamp); void reset (void); void process (int nfram, float *input []); void integr_reset (void); void integr_pause (void) { _integr = false; } void integr_start (void) { _integr = true; } float loudness_M (void) const { return _loudness_M; } float maxloudn_M (void) const { return _maxloudn_M; } float loudness_S (void) const { return _loudness_S; } float maxloudn_S (void) const { return _maxloudn_S; } float integrated (void) const { return _integrated; } float integ_thr (void) const { return _integ_thr; } float range_min (void) const { return _range_min; } float range_max (void) const { return _range_max; } float range_thr (void) const { return _range_thr; } const int *histogram_M (void) const { return _hist_M._histc; } const int *histogram_S (void) const { return _hist_S._histc; } int hist_M_count (void) const { return _hist_M._count; } int hist_S_count (void) const { return _hist_S._count; } private: float addfrags (int nfrag); void detect_init (float fsamp); void detect_reset (void); float detect_process (int nfram); bool _integr; // Integration on/off. int _nchan; // Number of channels, 2 or 5. float _fsamp; // Sample rate. int _fragm; // Fragmenst size, 1/20 second. int _frcnt; // Number of samples remaining in current fragment. float _frpwr; // Power accumulated for current fragment. float _power [64]; // Array of fragment powers. int _wrind; // Write index into _frpwr int _div1; // M period counter, 200 ms; int _div2; // S period counter, 1s; float _loudness_M; float _maxloudn_M; float _loudness_S; float _maxloudn_S; float _integrated; float _integ_thr; float _range_min; float _range_max; float _range_thr; // Filter coefficients and states. float _a0, _a1, _a2; float _b1, _b2; float _c3, _c4; float *_ipp [MAXCH]; Ebu_r128_fst _fst [MAXCH]; Ebu_r128_hist _hist_M; Ebu_r128_hist _hist_S; // Default channel gains. static float _chan_gain [5]; }; #endif nageru-1.4.2/ellipsis_label.h000066400000000000000000000011431301561706600161370ustar00rootroot00000000000000#ifndef _ELLIPSIS_LABEL_H #define _ELLIPSIS_LABEL_H 1 #include class EllipsisLabel : public QLabel { Q_OBJECT public: EllipsisLabel(QWidget *parent) : QLabel(parent) {} void setFullText(const QString &s) { full_text = s; updateEllipsisText(); } protected: void resizeEvent(QResizeEvent *event) override { QLabel::resizeEvent(event); updateEllipsisText(); } private: void updateEllipsisText() { QFontMetrics metrics(this->font()); this->setText(metrics.elidedText(full_text, Qt::ElideRight, this->width())); } QString full_text; }; #endif // !defined(_ELLIPSIS_LABEL_H) nageru-1.4.2/ffmpeg_raii.cpp000066400000000000000000000023771301561706600157710ustar00rootroot00000000000000#include "ffmpeg_raii.h" extern "C" { #include #include #include #include } using namespace std; // AVFormatContext void avformat_close_input_unique(AVFormatContext *format_ctx) { avformat_close_input(&format_ctx); } AVFormatContextWithCloser avformat_open_input_unique( const char *pathname, AVInputFormat *fmt, AVDictionary **options) { AVFormatContext *format_ctx = nullptr; if (avformat_open_input(&format_ctx, pathname, fmt, options) != 0) { format_ctx = nullptr; } return AVFormatContextWithCloser(format_ctx, avformat_close_input_unique); } // AVCodecContext void avcodec_free_context_unique(AVCodecContext *codec_ctx) { avcodec_free_context(&codec_ctx); } AVCodecContextWithDeleter avcodec_alloc_context3_unique(const AVCodec *codec) { return AVCodecContextWithDeleter(avcodec_alloc_context3(codec), avcodec_free_context_unique); } // AVCodecParameters void avcodec_parameters_free_unique(AVCodecParameters *codec_par) { avcodec_parameters_free(&codec_par); } // AVFrame void av_frame_free_unique(AVFrame *frame) { av_frame_free(&frame); } AVFrameWithDeleter av_frame_alloc_unique() { return AVFrameWithDeleter(av_frame_alloc(), av_frame_free_unique); } nageru-1.4.2/ffmpeg_raii.h000066400000000000000000000031321301561706600154240ustar00rootroot00000000000000#ifndef _FFMPEG_RAII_H #define _FFMPEG_RAII_H 1 // Some helpers to make RAII versions of FFmpeg objects. // The cleanup functions don't interact all that well with unique_ptr, // so things get a bit messy and verbose, but overall it's worth it to ensure // we never leak things by accident in error paths. // // This does not cover any of the types that can actually be declared as // a unique_ptr with no helper functions for deleter. #include struct AVCodec; struct AVCodecContext; struct AVCodecParameters; struct AVDictionary; struct AVFormatContext; struct AVFrame; struct AVInputFormat; // AVFormatContext void avformat_close_input_unique(AVFormatContext *format_ctx); typedef std::unique_ptr AVFormatContextWithCloser; AVFormatContextWithCloser avformat_open_input_unique( const char *pathname, AVInputFormat *fmt, AVDictionary **options); // AVCodecContext void avcodec_free_context_unique(AVCodecContext *codec_ctx); typedef std::unique_ptr AVCodecContextWithDeleter; AVCodecContextWithDeleter avcodec_alloc_context3_unique(const AVCodec *codec); // AVCodecParameters void avcodec_parameters_free_unique(AVCodecParameters *codec_par); typedef std::unique_ptr AVCodecParametersWithDeleter; // AVFrame void av_frame_free_unique(AVFrame *frame); typedef std::unique_ptr AVFrameWithDeleter; AVFrameWithDeleter av_frame_alloc_unique(); #endif // !defined(_FFMPEG_RAII_H) nageru-1.4.2/filter.cpp000066400000000000000000000235071301561706600150040ustar00rootroot00000000000000#include #include #include #include #include #include #include "defs.h" #ifdef __SSE__ #include #endif #include "filter.h" using namespace std; #ifdef __SSE__ // For SSE, we set the denormals-as-zero flag instead. #define early_undenormalise(sample) #else // !defined(__SSE__) union uint_float { float f; unsigned int i; }; #define early_undenormalise(sample) { \ uint_float uf; \ uf.f = float(sample); \ if ((uf.i&0x60000000)==0) sample=0.0f; \ } #endif // !_defined(__SSE__) Filter::Filter() { omega = M_PI; resonance = 0.01f; A = 1.0f; init(FILTER_NONE, 1); update(); } void Filter::update() { /* uses coefficients grabbed from RBJs audio eq cookbook: http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt */ float sn, cs; float cutoff_freq = omega; cutoff_freq = min(cutoff_freq, (float)M_PI); cutoff_freq = max(cutoff_freq, 0.001f); calcSinCos(cutoff_freq, &sn, &cs); if (resonance <= 0) resonance = 0.001f; #ifdef __GNUC__ // Faster version of real_resonance = resonance ^ (1 / order). // pow(), at least on current GCC, is pretty slow. float real_resonance = resonance; switch (filter_order) { case 0: case 1: break; case 4: real_resonance = sqrt(real_resonance); // Fall through. case 2: real_resonance = sqrt(real_resonance); break; case 3: real_resonance = cbrt(real_resonance); break; default: assert(false); } #else float real_resonance = pow(resonance, 1.0f / filter_order); #endif float alpha = float(sn / (2 * real_resonance)); float a0 = 1 + alpha; a1 = -2 * cs; a2 = 1 - alpha; switch (filtertype) { case FILTER_NONE: a0 = b0 = 1.0f; a1 = a2 = b1 = b2 = 0.0; //identity filter break; case FILTER_LPF: b0 = (1 - cs) * 0.5f; b1 = 1 - cs; b2 = b0; // a1 = -2*cs; // a2 = 1 - alpha; break; case FILTER_HPF: b0 = (1 + cs) * 0.5f; b1 = -(1 + cs); b2 = b0; // a1 = -2*cs; // a2 = 1 - alpha; break; case FILTER_BPF: b0 = alpha; b1 = 0.0f; b2 = -alpha; // a1 = -2*cs; // a2 = 1 - alpha; break; case FILTER_NOTCH: b0 = 1.0f; b1 = -2*cs; b2 = 1.0f; // a1 = -2*cs; // a2 = 1 - alpha; break; case FILTER_APF: b0 = 1 - alpha; b1 = -2*cs; b2 = 1.0f; // a1 = -2*cs; // a2 = 1 - alpha; break; case FILTER_PEAKING_EQ: b0 = 1 + alpha * A; b1 = -2*cs; b2 = 1 - alpha * A; a0 = 1 + alpha / A; // a1 = -2*cs; a2 = 1 - alpha / A; break; case FILTER_LOW_SHELF: b0 = A * ((A + 1) - (A - 1)*cs + 2 * sqrt(A) * alpha); b1 = 2 * A * ((A - 1) - (A + 1)*cs ); b2 = A * ((A + 1) - (A - 1)*cs - 2 * sqrt(A) * alpha); a0 = (A + 1) + (A - 1)*cs + 2 * sqrt(A) * alpha ; a1 = -2 * ((A - 1) + (A + 1)*cs ); a2 = (A + 1) + (A - 1)*cs - 2 * sqrt(A) * alpha ; break; case FILTER_HIGH_SHELF: b0 = A * ((A + 1) + (A - 1)*cs + 2 * sqrt(A) * alpha); b1 = -2 * A * ((A - 1) + (A + 1)*cs ); b2 = A * ((A + 1) + (A - 1)*cs - 2 * sqrt(A) * alpha); a0 = (A + 1) - (A - 1)*cs + 2 * sqrt(A) * alpha ; a1 = 2 * ((A - 1) - (A + 1)*cs ); a2 = (A + 1) - (A - 1)*cs - 2 * sqrt(A) * alpha ; break; default: //unknown filter type assert(false); break; } const float invA0 = 1.0f / a0; b0 *= invA0; b1 *= invA0; b2 *= invA0; a1 *= invA0; a2 *= invA0; } #ifndef NDEBUG void Filter::debug() { // Feed this to gnuplot to get a graph of the frequency response. const float Fs2 = OUTPUT_FREQUENCY * 0.5f; printf("set xrange [2:%f]; ", Fs2); printf("set yrange [-80:20]; "); printf("set log x; "); printf("phasor(x) = cos(x*pi/%f)*{1,0} + sin(x*pi/%f)*{0,1}; ", Fs2, Fs2); printf("tfunc(x, b0, b1, b2, a0, a1, a2) = (b0 * phasor(x)**2 + b1 * phasor(x) + b2) / (a0 * phasor(x)**2 + a1 * phasor(x) + a2); "); printf("db(x) = 20*log10(x); "); printf("plot db(abs(tfunc(x, %f, %f, %f, %f, %f, %f))) title \"\"\n", b0, b1, b2, 1.0f, a1, a2); } #endif void Filter::init(FilterType type, int order) { filtertype = type; filter_order = order; if (filtertype == FILTER_NONE) filter_order = 0; if (filter_order == 0) filtertype = FILTER_NONE; //reset feedback buffer for (unsigned i = 0; i < filter_order; i++) { feedback[i].d0 = feedback[i].d1 = 0.0f; } } #ifdef __SSE__ void Filter::render_chunk(float *inout_buf, unsigned int n_samples) #else void Filter::render_chunk(float *inout_buf, unsigned int n_samples, unsigned stride) #endif { #ifdef __SSE__ const unsigned stride = 1; unsigned old_denormals_mode = _MM_GET_FLUSH_ZERO_MODE(); _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); #endif assert((n_samples & 3) == 0); // make sure n_samples is divisible by 4 // Apply the filter FILTER_ORDER times. for (unsigned j = 0; j < filter_order; j++) { float d0 = feedback[j].d0; float d1 = feedback[j].d1; float *inout_ptr = inout_buf; // Render n_samples mono samples. Unrolling manually by a // factor four seemingly helps a lot, perhaps because it // lets the CPU overlap arithmetic and memory operations // better, or perhaps simply because the loop overhead is // high. for (unsigned i = n_samples >> 2; i; i--) { float in, out; in = *inout_ptr; out = b0*in + d0; *inout_ptr = out; d0 = b1*in - a1*out + d1; d1 = b2*in - a2*out; inout_ptr += stride; in = *inout_ptr; out = b0*in + d0; *inout_ptr = out; d0 = b1*in - a1*out + d1; d1 = b2*in - a2*out; inout_ptr += stride; in = *inout_ptr; out = b0*in + d0; *inout_ptr = out; d0 = b1*in - a1*out + d1; d1 = b2*in - a2*out; inout_ptr += stride; in = *inout_ptr; out = b0*in + d0; *inout_ptr = out; d0 = b1*in - a1*out + d1; d1 = b2*in - a2*out; inout_ptr += stride; } early_undenormalise(d0); //do denormalization step early_undenormalise(d1); feedback[j].d0 = d0; feedback[j].d1 = d1; } #ifdef __SSE__ _MM_SET_FLUSH_ZERO_MODE(old_denormals_mode); #endif } void Filter::render(float *inout_buf, unsigned int buf_size, float cutoff, float resonance) { //render buf_size mono samples #ifdef __SSE__ assert(buf_size % 4 == 0); #endif if (filter_order == 0) return; this->set_linear_cutoff(cutoff); this->set_resonance(resonance); this->update(); this->render_chunk(inout_buf, buf_size); } void StereoFilter::init(FilterType type, int new_order) { #ifdef __SSE__ parm_filter.init(type, new_order); memset(feedback, 0, sizeof(feedback)); #else for (unsigned i = 0; i < 2; ++i) { filters[i].init(type, new_order); } #endif } void StereoFilter::render(float *inout_left_ptr, unsigned n_samples, float cutoff, float resonance, float dbgain_normalized) { #ifdef __SSE__ if (parm_filter.filtertype == FILTER_NONE || parm_filter.filter_order == 0) return; unsigned old_denormals_mode = _MM_GET_FLUSH_ZERO_MODE(); _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); parm_filter.set_linear_cutoff(cutoff); parm_filter.set_resonance(resonance); parm_filter.set_dbgain_normalized(dbgain_normalized); parm_filter.update(); __m128 b0 = _mm_set1_ps(parm_filter.b0); __m128 b1 = _mm_set1_ps(parm_filter.b1); __m128 b2 = _mm_set1_ps(parm_filter.b2); __m128 a1 = _mm_set1_ps(parm_filter.a1); __m128 a2 = _mm_set1_ps(parm_filter.a2); // Apply the filter FILTER_ORDER times. for (unsigned j = 0; j < parm_filter.filter_order; j++) { __m128 d0 = feedback[j].d0; __m128 d1 = feedback[j].d1; __m64 *inout_ptr = (__m64 *)inout_left_ptr; __m128 in = _mm_set1_ps(0.0f), out; for (unsigned i = n_samples; i; i--) { in = _mm_loadl_pi(in, inout_ptr); out = _mm_add_ps(_mm_mul_ps(b0, in), d0); _mm_storel_pi(inout_ptr, out); d0 = _mm_add_ps(_mm_sub_ps(_mm_mul_ps(b1, in), _mm_mul_ps(a1, out)), d1); d1 = _mm_sub_ps(_mm_mul_ps(b2, in), _mm_mul_ps(a2, out)); ++inout_ptr; } feedback[j].d0 = d0; feedback[j].d1 = d1; } _MM_SET_FLUSH_ZERO_MODE(old_denormals_mode); #else if (filters[0].filtertype == FILTER_NONE || filters[0].filter_order == 0) return; for (unsigned i = 0; i < 2; ++i) { filters[i].set_linear_cutoff(cutoff); filters[i].set_resonance(resonance); filters[i].update(); filters[i].render_chunk(inout_left_ptr, n_samples, 2); ++inout_left_ptr; } #endif } /* Find the transfer function for an IIR biquad. This is relatively basic signal processing, but for completeness, here's the rationale for the function: The basic system of an IIR biquad looks like this, for input x[n], output y[n] and constant filter coefficients [ab][0-2]: a2 y[n-2] + a1 y[n-1] + a0 y[n] = b2 x[n-2] + b1 x[n-1] + b0 x[n] Taking the discrete Fourier transform (DFT) of both sides (denoting by convention DFT{x[n]} by X[w], where w is the angular frequency, going from 0 to 2pi), yields, due to the linearity and shift properties of the DFT: a2 e^2jw Y[w] + a1 e^jw Y[w] + a0 Y[w] = b2 e^2jw X[w] + b1 e^jw X[w] + b0 Y[w] Simple factorization and reorganization yields Y[w] / X[w] = (b1 e^2jw + b1 e^jw + b0) / (a2 e^2jw + a1 e^jw + a0) and Y[w] / X[w] is by definition the filter's _transfer function_ (customarily denoted by H(w)), ie. the complex factor it applies to the frequency component w. The absolute value of the transfer function is the frequency response, ie. how much frequency w is boosted or weakened. (This derivation usually goes via the Z-transform and not the DFT, but the idea is exactly the same; the Z-transform is just a bit more general.) Sending a signal through first one filter and then through another one will naturally be equivalent to a filter with the transfer function equal to the pointwise multiplication of the two filters, so for N-order filters we need to raise the answer to the Nth power. */ complex Filter::evaluate_transfer_function(float omega) { complex z = exp(complex(0.0f, omega)); complex z2 = z * z; return pow((b0 * z2 + b1 * z + b2) / (1.0f * z2 + a1 * z + a2), filter_order); } nageru-1.4.2/filter.h000066400000000000000000000056141301561706600144500ustar00rootroot00000000000000// Filter class: // a cascaded biquad IIR filter // // Special cases for type=LPF/BPF/HPF: // // Butterworth filter: order=1, resonance=1/sqrt(2) // Linkwitz-Riley filter: order=2, resonance=1/2 #ifndef _FILTER_H #define _FILTER_H 1 #define _USE_MATH_DEFINES #include #include #ifdef __SSE__ #include #endif enum FilterType { FILTER_NONE = 0, FILTER_LPF, FILTER_HPF, FILTER_BPF, FILTER_NOTCH, FILTER_APF, // EQ filters. FILTER_PEAKING_EQ, FILTER_LOW_SHELF, FILTER_HIGH_SHELF, }; #define FILTER_MAX_ORDER 4 class Filter { friend class StereoFilter; friend class SplittingStereoFilter; public: Filter(); void init(FilterType type, int new_order); void update(); //update coefficients #ifndef NDEBUG void debug(); #endif std::complex evaluate_transfer_function(float omega); FilterType get_type() { return filtertype; } unsigned get_order() { return filter_order; } // cutoff is taken to be in the [0..pi> (see set_linear_cutoff, below). void render(float *inout_array, unsigned int buf_size, float cutoff, float resonance); // Set cutoff, from [0..pi> (where pi is the Nyquist frequency). // Overridden by render() if you use that. void set_linear_cutoff(float new_omega) { omega = new_omega; } void set_resonance(float new_resonance) { resonance = new_resonance; } // For EQ filters only. void set_dbgain_normalized(float db_gain_div_40) { A = pow(10.0f, db_gain_div_40); } #ifdef __SSE__ // We don't need the stride argument for SSE, as StereoFilter // has its own SSE implementations. void render_chunk(float *inout_buf, unsigned nSamples); #else void render_chunk(float *inout_buf, unsigned nSamples, unsigned stride = 1); #endif FilterType filtertype; private: float omega; //which is 2*Pi*frequency /SAMPLE_RATE float resonance; float A; // which is 10^(db_gain / 40) public: unsigned filter_order; private: float b0, b1, b2, a1, a2; //filter coefs struct FeedbackBuffer { float d0,d1; //feedback buffers } feedback[FILTER_MAX_ORDER]; void calcSinCos(float omega, float *sinVal, float *cosVal) { *sinVal = (float)sin(omega); *cosVal = (float)cos(omega); } }; class StereoFilter { public: void init(FilterType type, int new_order); void render(float *inout_left_ptr, unsigned n_samples, float cutoff, float resonance, float dbgain_normalized = 0.0f); #ifndef NDEBUG #ifdef __SSE__ void debug() { parm_filter.debug(); } #else void debug() { filters[0].debug(); } #endif #endif #ifdef __SSE__ FilterType get_type() { return parm_filter.get_type(); } #else FilterType get_type() { return filters[0].get_type(); } #endif private: #ifdef __SSE__ // We only use the filter to calculate coefficients; we don't actually // use its feedbacks. Filter parm_filter; struct SIMDFeedbackBuffer { __m128 d0, d1; } feedback[FILTER_MAX_ORDER]; #else Filter filters[2]; #endif }; #endif // !defined(_FILTER_H) nageru-1.4.2/flags.cpp000066400000000000000000000325661301561706600146200ustar00rootroot00000000000000#include "flags.h" #include #include #include #include #include using namespace std; Flags global_flags; // Long options that have no corresponding short option. enum LongOption { OPTION_MULTICHANNEL = 1000, OPTION_MIDI_MAPPING, OPTION_FAKE_CARDS_AUDIO, OPTION_HTTP_UNCOMPRESSED_VIDEO, OPTION_HTTP_X264_VIDEO, OPTION_X264_PRESET, OPTION_X264_TUNE, OPTION_X264_SPEEDCONTROL, OPTION_X264_SPEEDCONTROL_VERBOSE, OPTION_X264_BITRATE, OPTION_X264_VBV_BUFSIZE, OPTION_X264_VBV_MAX_BITRATE, OPTION_X264_PARAM, OPTION_HTTP_MUX, OPTION_HTTP_COARSE_TIMEBASE, OPTION_HTTP_AUDIO_CODEC, OPTION_HTTP_AUDIO_BITRATE, OPTION_FLAT_AUDIO, OPTION_GAIN_STAGING, OPTION_DISABLE_LOCUT, OPTION_ENABLE_LOCUT, OPTION_DISABLE_GAIN_STAGING_AUTO, OPTION_ENABLE_GAIN_STAGING_AUTO, OPTION_DISABLE_COMPRESSOR, OPTION_ENABLE_COMPRESSOR, OPTION_DISABLE_LIMITER, OPTION_ENABLE_LIMITER, OPTION_DISABLE_MAKEUP_GAIN_AUTO, OPTION_ENABLE_MAKEUP_GAIN_AUTO, OPTION_DISABLE_ALSA_OUTPUT, OPTION_NO_FLUSH_PBOS }; void usage() { fprintf(stderr, "Usage: nageru [OPTION]...\n"); fprintf(stderr, "\n"); fprintf(stderr, " -h, --help print usage information\n"); fprintf(stderr, " -c, --num-cards set number of input cards (default 2)\n"); fprintf(stderr, " -t, --theme=FILE choose theme (default theme.lua)\n"); fprintf(stderr, " -I, --theme-dir=DIR search for theme in this directory (can be given multiple times)\n"); fprintf(stderr, " -v, --va-display=SPEC VA-API device for H.264 encoding\n"); fprintf(stderr, " ($DISPLAY spec or /dev/dri/render* path)\n"); fprintf(stderr, " -m, --map-signal=SIGNAL,CARD set a default card mapping (can be given multiple times)\n"); fprintf(stderr, " -M, --input-mapping=FILE start with the given audio input mapping (implies --multichannel)\n"); fprintf(stderr, " --multichannel start in multichannel audio mapping mode\n"); fprintf(stderr, " --midi-mapping=FILE start with the given MIDI controller mapping (implies --multichannel)\n"); fprintf(stderr, " --fake-cards-audio make fake (disconnected) cards output a simple tone\n"); fprintf(stderr, " --http-uncompressed-video send uncompressed NV12 video to HTTP clients\n"); fprintf(stderr, " --http-x264-video send x264-compressed video to HTTP clients\n"); fprintf(stderr, " --x264-preset x264 quality preset (default " X264_DEFAULT_PRESET ")\n"); fprintf(stderr, " --x264-tune x264 tuning (default " X264_DEFAULT_TUNE ", can be blank)\n"); fprintf(stderr, " --x264-speedcontrol try to match x264 preset to available CPU speed\n"); fprintf(stderr, " --x264-speedcontrol-verbose output speedcontrol debugging statistics\n"); fprintf(stderr, " --x264-bitrate x264 bitrate (in kilobit/sec, default %d)\n", DEFAULT_X264_OUTPUT_BIT_RATE); fprintf(stderr, " --x264-vbv-bufsize x264 VBV size (in kilobits, 0 = one-frame VBV,\n"); fprintf(stderr, " default: same as --x264-bitrate, that is, one-second VBV)\n"); fprintf(stderr, " --x264-vbv-max-bitrate x264 local max bitrate (in kilobit/sec per --vbv-bufsize,\n"); fprintf(stderr, " 0 = no limit, default: same as --x264-bitrate, i.e., CBR)\n"); fprintf(stderr, " --x264-param=NAME[,VALUE] set any x264 parameter, for fine tuning\n"); fprintf(stderr, " --http-mux=NAME mux to use for HTTP streams (default " DEFAULT_STREAM_MUX_NAME ")\n"); fprintf(stderr, " --http-audio-codec=NAME audio codec to use for HTTP streams\n"); fprintf(stderr, " (default is to use the same as for the recording)\n"); fprintf(stderr, " --http-audio-bitrate=KBITS audio codec bit rate to use for HTTP streams\n"); fprintf(stderr, " (default is %d, ignored unless --http-audio-codec is set)\n", DEFAULT_AUDIO_OUTPUT_BIT_RATE / 1000); fprintf(stderr, " --http-coarse-timebase use less timebase for HTTP (recommended for muxers\n"); fprintf(stderr, " that handle large pts poorly, like e.g. MP4)\n"); fprintf(stderr, " --flat-audio start with most audio processing turned off\n"); fprintf(stderr, " (can be overridden by e.g. --enable-limiter)\n"); fprintf(stderr, " --gain-staging=DB set initial gain staging to the given value\n"); fprintf(stderr, " (--disable-gain-staging-auto)\n"); fprintf(stderr, " --disable-locut turn off locut filter (also --enable)\n"); fprintf(stderr, " --disable-gain-staging-auto turn off automatic gain staging (also --enable)\n"); fprintf(stderr, " --disable-compressor turn off regular compressor (also --enable)\n"); fprintf(stderr, " --disable-limiter turn off limiter (also --enable)\n"); fprintf(stderr, " --disable-makeup-gain-auto turn off auto-adjustment of final makeup gain (also --enable)\n"); fprintf(stderr, " --disable-alsa-output disable audio monitoring via ALSA\n"); fprintf(stderr, " --no-flush-pbos do not explicitly signal texture data uploads\n"); fprintf(stderr, " (will give display corruption, but makes it\n"); fprintf(stderr, " possible to run with apitrace in real time)\n"); } void parse_flags(int argc, char * const argv[]) { static const option long_options[] = { { "help", no_argument, 0, 'h' }, { "num-cards", required_argument, 0, 'c' }, { "theme", required_argument, 0, 't' }, { "theme-dir", required_argument, 0, 'I' }, { "map-signal", required_argument, 0, 'm' }, { "input-mapping", required_argument, 0, 'M' }, { "va-display", required_argument, 0, 'v' }, { "multichannel", no_argument, 0, OPTION_MULTICHANNEL }, { "midi-mapping", required_argument, 0, OPTION_MIDI_MAPPING }, { "fake-cards-audio", no_argument, 0, OPTION_FAKE_CARDS_AUDIO }, { "http-uncompressed-video", no_argument, 0, OPTION_HTTP_UNCOMPRESSED_VIDEO }, { "http-x264-video", no_argument, 0, OPTION_HTTP_X264_VIDEO }, { "x264-preset", required_argument, 0, OPTION_X264_PRESET }, { "x264-tune", required_argument, 0, OPTION_X264_TUNE }, { "x264-speedcontrol", no_argument, 0, OPTION_X264_SPEEDCONTROL }, { "x264-speedcontrol-verbose", no_argument, 0, OPTION_X264_SPEEDCONTROL_VERBOSE }, { "x264-bitrate", required_argument, 0, OPTION_X264_BITRATE }, { "x264-vbv-bufsize", required_argument, 0, OPTION_X264_VBV_BUFSIZE }, { "x264-vbv-max-bitrate", required_argument, 0, OPTION_X264_VBV_MAX_BITRATE }, { "x264-param", required_argument, 0, OPTION_X264_PARAM }, { "http-mux", required_argument, 0, OPTION_HTTP_MUX }, { "http-coarse-timebase", no_argument, 0, OPTION_HTTP_COARSE_TIMEBASE }, { "http-audio-codec", required_argument, 0, OPTION_HTTP_AUDIO_CODEC }, { "http-audio-bitrate", required_argument, 0, OPTION_HTTP_AUDIO_BITRATE }, { "flat-audio", no_argument, 0, OPTION_FLAT_AUDIO }, { "gain-staging", required_argument, 0, OPTION_GAIN_STAGING }, { "disable-locut", no_argument, 0, OPTION_DISABLE_LOCUT }, { "enable-locut", no_argument, 0, OPTION_ENABLE_LOCUT }, { "disable-gain-staging-auto", no_argument, 0, OPTION_DISABLE_GAIN_STAGING_AUTO }, { "enable-gain-staging-auto", no_argument, 0, OPTION_ENABLE_GAIN_STAGING_AUTO }, { "disable-compressor", no_argument, 0, OPTION_DISABLE_COMPRESSOR }, { "enable-compressor", no_argument, 0, OPTION_ENABLE_COMPRESSOR }, { "disable-limiter", no_argument, 0, OPTION_DISABLE_LIMITER }, { "enable-limiter", no_argument, 0, OPTION_ENABLE_LIMITER }, { "disable-makeup-gain-auto", no_argument, 0, OPTION_DISABLE_MAKEUP_GAIN_AUTO }, { "enable-makeup-gain-auto", no_argument, 0, OPTION_ENABLE_MAKEUP_GAIN_AUTO }, { "disable-alsa-output", no_argument, 0, OPTION_DISABLE_ALSA_OUTPUT }, { "no-flush-pbos", no_argument, 0, OPTION_NO_FLUSH_PBOS }, { 0, 0, 0, 0 } }; vector theme_dirs; for ( ;; ) { int option_index = 0; int c = getopt_long(argc, argv, "c:t:I:v:m:M:", long_options, &option_index); if (c == -1) { break; } switch (c) { case 'c': global_flags.num_cards = atoi(optarg); break; case 't': global_flags.theme_filename = optarg; break; case 'I': theme_dirs.push_back(optarg); break; case 'm': { char *ptr = strchr(optarg, ','); if (ptr == nullptr) { fprintf(stderr, "ERROR: Invalid argument '%s' to --map-signal (needs a signal and a card number, separated by comma)\n", optarg); exit(1); } *ptr = '\0'; const int signal_num = atoi(optarg); const int card_num = atoi(ptr + 1); if (global_flags.default_stream_mapping.count(signal_num)) { fprintf(stderr, "ERROR: Signal %d already mapped to card %d\n", signal_num, global_flags.default_stream_mapping[signal_num]); exit(1); } global_flags.default_stream_mapping[signal_num] = card_num; break; } case 'M': global_flags.input_mapping_filename = optarg; break; case OPTION_MULTICHANNEL: global_flags.multichannel_mapping_mode = true; break; case 'v': global_flags.va_display = optarg; break; case OPTION_MIDI_MAPPING: global_flags.midi_mapping_filename = optarg; global_flags.multichannel_mapping_mode = true; break; case OPTION_FAKE_CARDS_AUDIO: global_flags.fake_cards_audio = true; break; case OPTION_HTTP_UNCOMPRESSED_VIDEO: global_flags.uncompressed_video_to_http = true; break; case OPTION_HTTP_MUX: global_flags.stream_mux_name = optarg; break; case OPTION_HTTP_COARSE_TIMEBASE: global_flags.stream_coarse_timebase = true; break; case OPTION_HTTP_AUDIO_CODEC: global_flags.stream_audio_codec_name = optarg; break; case OPTION_HTTP_AUDIO_BITRATE: global_flags.stream_audio_codec_bitrate = atoi(optarg) * 1000; break; case OPTION_HTTP_X264_VIDEO: global_flags.x264_video_to_http = true; break; case OPTION_X264_PRESET: global_flags.x264_preset = optarg; break; case OPTION_X264_TUNE: global_flags.x264_tune = optarg; break; case OPTION_X264_SPEEDCONTROL: global_flags.x264_speedcontrol = true; break; case OPTION_X264_SPEEDCONTROL_VERBOSE: global_flags.x264_speedcontrol_verbose = true; break; case OPTION_X264_BITRATE: global_flags.x264_bitrate = atoi(optarg); break; case OPTION_X264_VBV_BUFSIZE: global_flags.x264_vbv_buffer_size = atoi(optarg); break; case OPTION_X264_VBV_MAX_BITRATE: global_flags.x264_vbv_max_bitrate = atoi(optarg); break; case OPTION_X264_PARAM: global_flags.x264_extra_param.push_back(optarg); break; case OPTION_FLAT_AUDIO: // If --flat-audio is given, turn off everything that messes with the sound, // except the final makeup gain. global_flags.locut_enabled = false; global_flags.gain_staging_auto = false; global_flags.compressor_enabled = false; global_flags.limiter_enabled = false; break; case OPTION_GAIN_STAGING: global_flags.initial_gain_staging_db = atof(optarg); global_flags.gain_staging_auto = false; break; case OPTION_DISABLE_LOCUT: global_flags.locut_enabled = false; break; case OPTION_ENABLE_LOCUT: global_flags.locut_enabled = true; break; case OPTION_DISABLE_GAIN_STAGING_AUTO: global_flags.gain_staging_auto = false; break; case OPTION_ENABLE_GAIN_STAGING_AUTO: global_flags.gain_staging_auto = true; break; case OPTION_DISABLE_COMPRESSOR: global_flags.compressor_enabled = false; break; case OPTION_ENABLE_COMPRESSOR: global_flags.compressor_enabled = true; break; case OPTION_DISABLE_LIMITER: global_flags.limiter_enabled = false; break; case OPTION_ENABLE_LIMITER: global_flags.limiter_enabled = true; break; case OPTION_DISABLE_MAKEUP_GAIN_AUTO: global_flags.final_makeup_gain_auto = false; break; case OPTION_ENABLE_MAKEUP_GAIN_AUTO: global_flags.final_makeup_gain_auto = true; break; case OPTION_DISABLE_ALSA_OUTPUT: global_flags.enable_alsa_output = false; break; case OPTION_NO_FLUSH_PBOS: global_flags.flush_pbos = false; break; case 'h': usage(); exit(0); default: fprintf(stderr, "Unknown option '%s'\n", argv[option_index]); fprintf(stderr, "\n"); usage(); exit(1); } } if (global_flags.uncompressed_video_to_http && global_flags.x264_video_to_http) { fprintf(stderr, "ERROR: --http-uncompressed-video and --http-x264-video are mutually incompatible\n"); exit(1); } if (global_flags.num_cards <= 0) { fprintf(stderr, "ERROR: --num-cards must be at least 1\n"); exit(1); } if (global_flags.x264_speedcontrol) { if (!global_flags.x264_preset.empty() && global_flags.x264_preset != "faster") { fprintf(stderr, "WARNING: --x264-preset is overridden by --x264-speedcontrol (implicitly uses \"faster\" as base preset)\n"); } global_flags.x264_preset = "faster"; } else if (global_flags.x264_preset.empty()) { global_flags.x264_preset = X264_DEFAULT_PRESET; } if (!theme_dirs.empty()) { global_flags.theme_dirs = theme_dirs; } for (pair mapping : global_flags.default_stream_mapping) { if (mapping.second >= global_flags.num_cards) { fprintf(stderr, "ERROR: Signal %d mapped to card %d, which doesn't exist (try adjusting --num-cards)\n", mapping.first, mapping.second); exit(1); } } } nageru-1.4.2/flags.h000066400000000000000000000035441301561706600142570ustar00rootroot00000000000000#ifndef _FLAGS_H #define _FLAGS_H #include #include #include #include "defs.h" struct Flags { int num_cards = 2; std::string va_display; bool fake_cards_audio = false; bool uncompressed_video_to_http = false; bool x264_video_to_http = false; std::vector theme_dirs { ".", "/usr/local/share/nageru" }; std::string theme_filename = "theme.lua"; bool locut_enabled = true; bool gain_staging_auto = true; float initial_gain_staging_db = 0.0f; bool compressor_enabled = true; bool limiter_enabled = true; bool final_makeup_gain_auto = true; bool flush_pbos = true; std::string stream_mux_name = DEFAULT_STREAM_MUX_NAME; bool stream_coarse_timebase = false; std::string stream_audio_codec_name; // Blank = use the same as for the recording. int stream_audio_codec_bitrate = DEFAULT_AUDIO_OUTPUT_BIT_RATE; // Ignored if stream_audio_codec_name is blank. std::string x264_preset; // Empty will be overridden by X264_DEFAULT_PRESET, unless speedcontrol is set. std::string x264_tune = X264_DEFAULT_TUNE; bool x264_speedcontrol = false; bool x264_speedcontrol_verbose = false; int x264_bitrate = DEFAULT_X264_OUTPUT_BIT_RATE; // In kilobit/sec. int x264_vbv_max_bitrate = -1; // In kilobits. 0 = no limit, -1 = same as (CBR). int x264_vbv_buffer_size = -1; // In kilobits. 0 = one-frame VBV, -1 = same as (one-second VBV). std::vector x264_extra_param; // In “key[,value]” format. bool enable_alsa_output = true; std::map default_stream_mapping; bool multichannel_mapping_mode = false; // Implicitly true if input_mapping_filename is nonempty. std::string input_mapping_filename; // Empty for none. std::string midi_mapping_filename; // Empty for none. }; extern Flags global_flags; void parse_flags(int argc, char * const argv[]); #endif // !defined(_FLAGS_H) nageru-1.4.2/glwidget.cpp000066400000000000000000000206251301561706600153230ustar00rootroot00000000000000#include "glwidget.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "audio_mixer.h" #include "context.h" #include "flags.h" #include "mainwindow.h" #include "mixer.h" #include "ref_counted_gl_sync.h" class QMouseEvent; #undef Success #include #include using namespace std; using namespace std::placeholders; GLWidget::GLWidget(QWidget *parent) : QGLWidget(parent, global_share_widget) { } void GLWidget::clean_context() { if (resource_pool != nullptr) { makeCurrent(); resource_pool->clean_context(); } } void GLWidget::initializeGL() { static once_flag flag; call_once(flag, [this]{ global_mixer = new Mixer(QGLFormat::toSurfaceFormat(format()), global_flags.num_cards); global_audio_mixer = global_mixer->get_audio_mixer(); global_mainwindow->mixer_created(global_mixer); global_mixer->start(); }); global_mixer->set_frame_ready_callback(output, [this]{ QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); }); if (output == Mixer::OUTPUT_LIVE) { global_mixer->set_transition_names_updated_callback(output, [this](const vector &names){ emit transition_names_updated(names); }); } if (output >= Mixer::OUTPUT_INPUT0) { global_mixer->set_name_updated_callback(output, [this](const string &name){ emit name_updated(output, name); }); global_mixer->set_color_updated_callback(output, [this](const string &color){ emit color_updated(output, color); }); int signal_num = global_mixer->get_channel_signal(output); if (signal_num != -1) { setContextMenuPolicy(Qt::CustomContextMenu); connect(this, &QWidget::customContextMenuRequested, bind(&GLWidget::show_context_menu, this, signal_num, _1)); } } glDisable(GL_BLEND); glDisable(GL_DEPTH_TEST); glDepthMask(GL_FALSE); } void GLWidget::resizeGL(int width, int height) { glViewport(0, 0, width, height); } void GLWidget::paintGL() { Mixer::DisplayFrame frame; if (!global_mixer->get_display_frame(output, &frame)) { glClearColor(0.0f, 1.0f, 0.0f, 1.0f); check_error(); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); check_error(); return; } check_error(); glWaitSync(frame.ready_fence.get(), /*flags=*/0, GL_TIMEOUT_IGNORED); check_error(); frame.setup_chain(); check_error(); frame.chain->render_to_screen(); check_error(); if (resource_pool == nullptr) { resource_pool = frame.chain->get_resource_pool(); } else { assert(resource_pool == frame.chain->get_resource_pool()); } } void GLWidget::mousePressEvent(QMouseEvent *event) { emit clicked(); } void GLWidget::show_context_menu(unsigned signal_num, const QPoint &pos) { QPoint global_pos = mapToGlobal(pos); QMenu menu; // Add a submenu for selecting input card, with an action for each card. QMenu card_submenu; QActionGroup card_group(&card_submenu); unsigned num_cards = global_mixer->get_num_cards(); unsigned current_card = global_mixer->map_signal(signal_num); for (unsigned card_index = 0; card_index < num_cards; ++card_index) { QString description(QString::fromStdString(global_mixer->get_card_description(card_index))); QAction *action = new QAction(description, &card_group); action->setCheckable(true); if (current_card == card_index) { action->setChecked(true); } action->setData(QList{"card", card_index}); card_submenu.addAction(action); } card_submenu.setTitle("Input source"); menu.addMenu(&card_submenu); // --- The choices in the next few options depend a lot on which card is active --- // Add a submenu for selecting video input, with an action for each input. QMenu video_input_submenu; QActionGroup video_input_group(&video_input_submenu); std::map video_inputs = global_mixer->get_available_video_inputs(current_card); uint32_t current_video_input = global_mixer->get_current_video_input(current_card); for (const auto &mode : video_inputs) { QString description(QString::fromStdString(mode.second)); QAction *action = new QAction(description, &video_input_group); action->setCheckable(true); if (mode.first == current_video_input) { action->setChecked(true); } action->setData(QList{"video_input", mode.first}); video_input_submenu.addAction(action); } video_input_submenu.setTitle("Video input"); menu.addMenu(&video_input_submenu); // The same for audio input. QMenu audio_input_submenu; QActionGroup audio_input_group(&audio_input_submenu); std::map audio_inputs = global_mixer->get_available_audio_inputs(current_card); uint32_t current_audio_input = global_mixer->get_current_audio_input(current_card); for (const auto &mode : audio_inputs) { QString description(QString::fromStdString(mode.second)); QAction *action = new QAction(description, &audio_input_group); action->setCheckable(true); if (mode.first == current_audio_input) { action->setChecked(true); } action->setData(QList{"audio_input", mode.first}); audio_input_submenu.addAction(action); } audio_input_submenu.setTitle("Audio input"); menu.addMenu(&audio_input_submenu); // The same for resolution. QMenu mode_submenu; QActionGroup mode_group(&mode_submenu); std::map video_modes = global_mixer->get_available_video_modes(current_card); uint32_t current_video_mode = global_mixer->get_current_video_mode(current_card); bool has_auto_mode = false; for (const auto &mode : video_modes) { QString description(QString::fromStdString(mode.second.name)); QAction *action = new QAction(description, &mode_group); action->setCheckable(true); if (mode.first == current_video_mode) { action->setChecked(true); } action->setData(QList{"video_mode", mode.first}); mode_submenu.addAction(action); // TODO: Relying on the 0 value here (from bmusb.h) is ugly, it should be a named constant. if (mode.first == 0) { has_auto_mode = true; } } // Add a “scan” menu if there's no “auto” mode. if (!has_auto_mode) { QAction *action = new QAction("Scan", &mode_group); action->setData(QList{"video_mode", 0}); mode_submenu.addSeparator(); mode_submenu.addAction(action); } mode_submenu.setTitle("Input mode"); menu.addMenu(&mode_submenu); // --- End of card-dependent choices --- // Add an audio source selector. QAction *audio_source_action = nullptr; if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { audio_source_action = new QAction("Use as audio source", &menu); audio_source_action->setCheckable(true); if (global_audio_mixer->get_simple_input() == signal_num) { audio_source_action->setChecked(true); audio_source_action->setEnabled(false); } menu.addAction(audio_source_action); } // And a master clock selector. QAction *master_clock_action = new QAction("Use as master clock", &menu); master_clock_action->setCheckable(true); if (global_mixer->get_master_clock() == signal_num) { master_clock_action->setChecked(true); master_clock_action->setEnabled(false); } menu.addAction(master_clock_action); // Show the menu and look at the result. QAction *selected_item = menu.exec(global_pos); if (audio_source_action != nullptr && selected_item == audio_source_action) { global_audio_mixer->set_simple_input(signal_num); } else if (selected_item == master_clock_action) { global_mixer->set_master_clock(signal_num); } else if (selected_item != nullptr) { QList selected = selected_item->data().toList(); if (selected[0].toString() == "video_mode") { uint32_t mode = selected[1].toUInt(nullptr); if (mode == 0 && !has_auto_mode) { global_mixer->start_mode_scanning(current_card); } else { global_mixer->set_video_mode(current_card, mode); } } else if (selected[0].toString() == "video_input") { uint32_t input = selected[1].toUInt(nullptr); global_mixer->set_video_input(current_card, input); } else if (selected[0].toString() == "audio_input") { uint32_t input = selected[1].toUInt(nullptr); global_mixer->set_audio_input(current_card, input); } else if (selected[0].toString() == "card") { unsigned card_index = selected[1].toUInt(nullptr); global_mixer->set_signal_mapping(signal_num, card_index); } else { assert(false); } } } nageru-1.4.2/glwidget.h000066400000000000000000000024331301561706600147650ustar00rootroot00000000000000#ifndef GLWIDGET_H #define GLWIDGET_H #include #include #include #include #include #include "mixer.h" class QMouseEvent; class QObject; class QPoint; class QWidget; namespace movit { class ResourcePool; } // namespace movit // Note: We use the older QGLWidget instead of QOpenGLWidget as it is // much faster (does not go through a separate offscreen rendering step). // // TODO: Consider if QOpenGLWindow could do what we want. class GLWidget : public QGLWidget { Q_OBJECT public: GLWidget(QWidget *parent = 0); void set_output(Mixer::Output output) { this->output = output; } void clean_context(); protected: void initializeGL() override; void resizeGL(int width, int height) override; void paintGL() override; void mousePressEvent(QMouseEvent *event) override; signals: void clicked(); void transition_names_updated(std::vector transition_names); void name_updated(Mixer::Output output, const std::string &name); void color_updated(Mixer::Output output, const std::string &color); private slots: void show_context_menu(unsigned signal_num, const QPoint &pos); private: Mixer::Output output; GLuint vao, program_num; GLuint position_vbo, texcoord_vbo; movit::ResourcePool *resource_pool = nullptr; }; #endif nageru-1.4.2/httpd.cpp000066400000000000000000000136301301561706600146360ustar00rootroot00000000000000#include "httpd.h" #include #include #include #include #include #include #include #include #include #include #include "defs.h" #include "metacube2.h" struct MHD_Connection; struct MHD_Response; using namespace std; HTTPD::HTTPD() { } HTTPD::~HTTPD() { if (mhd) { MHD_quiesce_daemon(mhd); for (Stream *stream : streams) { stream->stop(); } MHD_stop_daemon(mhd); } } void HTTPD::start(int port) { mhd = MHD_start_daemon(MHD_USE_THREAD_PER_CONNECTION | MHD_USE_POLL_INTERNALLY | MHD_USE_DUAL_STACK, port, nullptr, nullptr, &answer_to_connection_thunk, this, MHD_OPTION_NOTIFY_COMPLETED, nullptr, this, MHD_OPTION_END); if (mhd == nullptr) { fprintf(stderr, "Warning: Could not open HTTP server. (Port already in use?)\n"); } } void HTTPD::add_data(const char *buf, size_t size, bool keyframe) { unique_lock lock(streams_mutex); for (Stream *stream : streams) { stream->add_data(buf, size, keyframe ? Stream::DATA_TYPE_KEYFRAME : Stream::DATA_TYPE_OTHER); } } int HTTPD::answer_to_connection_thunk(void *cls, MHD_Connection *connection, const char *url, const char *method, const char *version, const char *upload_data, size_t *upload_data_size, void **con_cls) { HTTPD *httpd = (HTTPD *)cls; return httpd->answer_to_connection(connection, url, method, version, upload_data, upload_data_size, con_cls); } int HTTPD::answer_to_connection(MHD_Connection *connection, const char *url, const char *method, const char *version, const char *upload_data, size_t *upload_data_size, void **con_cls) { // See if the URL ends in “.metacube”. HTTPD::Stream::Framing framing; if (strstr(url, ".metacube") == url + strlen(url) - strlen(".metacube")) { framing = HTTPD::Stream::FRAMING_METACUBE; } else { framing = HTTPD::Stream::FRAMING_RAW; } HTTPD::Stream *stream = new HTTPD::Stream(this, framing); stream->add_data(header.data(), header.size(), Stream::DATA_TYPE_HEADER); { unique_lock lock(streams_mutex); streams.insert(stream); } *con_cls = stream; // Does not strictly have to be equal to MUX_BUFFER_SIZE. MHD_Response *response = MHD_create_response_from_callback( (size_t)-1, MUX_BUFFER_SIZE, &HTTPD::Stream::reader_callback_thunk, stream, &HTTPD::free_stream); // TODO: Content-type? if (framing == HTTPD::Stream::FRAMING_METACUBE) { MHD_add_response_header(response, "Content-encoding", "metacube"); } int ret = MHD_queue_response(connection, MHD_HTTP_OK, response); MHD_destroy_response(response); // Only decreases the refcount; actual free is after the request is done. return ret; } void HTTPD::free_stream(void *cls) { HTTPD::Stream *stream = (HTTPD::Stream *)cls; HTTPD *httpd = stream->get_parent(); { unique_lock lock(httpd->streams_mutex); delete stream; httpd->streams.erase(stream); } } ssize_t HTTPD::Stream::reader_callback_thunk(void *cls, uint64_t pos, char *buf, size_t max) { HTTPD::Stream *stream = (HTTPD::Stream *)cls; return stream->reader_callback(pos, buf, max); } ssize_t HTTPD::Stream::reader_callback(uint64_t pos, char *buf, size_t max) { unique_lock lock(buffer_mutex); has_buffered_data.wait(lock, [this]{ return should_quit || !buffered_data.empty(); }); if (should_quit) { return 0; } ssize_t ret = 0; while (max > 0 && !buffered_data.empty()) { const string &s = buffered_data.front(); assert(s.size() > used_of_buffered_data); size_t len = s.size() - used_of_buffered_data; if (max >= len) { // Consume the entire (rest of the) string. memcpy(buf, s.data() + used_of_buffered_data, len); buf += len; ret += len; max -= len; buffered_data.pop_front(); used_of_buffered_data = 0; } else { // We don't need the entire string; just use the first part of it. memcpy(buf, s.data() + used_of_buffered_data, max); buf += max; used_of_buffered_data += max; ret += max; max = 0; } } return ret; } void HTTPD::Stream::add_data(const char *buf, size_t buf_size, HTTPD::Stream::DataType data_type) { if (buf_size == 0) { return; } if (data_type == DATA_TYPE_KEYFRAME) { seen_keyframe = true; } else if (data_type == DATA_TYPE_OTHER && !seen_keyframe) { // Start sending only once we see a keyframe. return; } unique_lock lock(buffer_mutex); if (framing == FRAMING_METACUBE) { metacube2_block_header hdr; memcpy(hdr.sync, METACUBE2_SYNC, sizeof(hdr.sync)); hdr.size = htonl(buf_size); int flags = 0; if (data_type == DATA_TYPE_HEADER) { flags |= METACUBE_FLAGS_HEADER; } else if (data_type == DATA_TYPE_OTHER) { flags |= METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START; } hdr.flags = htons(flags); hdr.csum = htons(metacube2_compute_crc(&hdr)); buffered_data.emplace_back((char *)&hdr, sizeof(hdr)); } buffered_data.emplace_back(buf, buf_size); // Send a Metacube2 timestamp every keyframe. if (framing == FRAMING_METACUBE && data_type == DATA_TYPE_KEYFRAME) { timespec now; clock_gettime(CLOCK_REALTIME, &now); metacube2_timestamp_packet packet; packet.type = htobe64(METACUBE_METADATA_TYPE_ENCODER_TIMESTAMP); packet.tv_sec = htobe64(now.tv_sec); packet.tv_nsec = htobe64(now.tv_nsec); metacube2_block_header hdr; memcpy(hdr.sync, METACUBE2_SYNC, sizeof(hdr.sync)); hdr.size = htonl(sizeof(packet)); hdr.flags = htons(METACUBE_FLAGS_METADATA); hdr.csum = htons(metacube2_compute_crc(&hdr)); buffered_data.emplace_back((char *)&hdr, sizeof(hdr)); buffered_data.emplace_back((char *)&packet, sizeof(packet)); } has_buffered_data.notify_all(); } void HTTPD::Stream::stop() { unique_lock lock(buffer_mutex); should_quit = true; has_buffered_data.notify_all(); } nageru-1.4.2/httpd.h000066400000000000000000000043201301561706600142770ustar00rootroot00000000000000#ifndef _HTTPD_H #define _HTTPD_H // A class dealing with stream output to HTTP. #include #include #include #include #include #include #include #include struct MHD_Connection; struct MHD_Daemon; class HTTPD { public: HTTPD(); ~HTTPD(); // Should be called before start(). void set_header(const std::string &data) { header = data; } void start(int port); void add_data(const char *buf, size_t size, bool keyframe); private: static int answer_to_connection_thunk(void *cls, MHD_Connection *connection, const char *url, const char *method, const char *version, const char *upload_data, size_t *upload_data_size, void **con_cls); int answer_to_connection(MHD_Connection *connection, const char *url, const char *method, const char *version, const char *upload_data, size_t *upload_data_size, void **con_cls); static void free_stream(void *cls); class Stream { public: enum Framing { FRAMING_RAW, FRAMING_METACUBE }; Stream(HTTPD *parent, Framing framing) : parent(parent), framing(framing) {} static ssize_t reader_callback_thunk(void *cls, uint64_t pos, char *buf, size_t max); ssize_t reader_callback(uint64_t pos, char *buf, size_t max); enum DataType { DATA_TYPE_HEADER, DATA_TYPE_KEYFRAME, DATA_TYPE_OTHER }; void add_data(const char *buf, size_t size, DataType data_type); void stop(); HTTPD *get_parent() const { return parent; } private: HTTPD *parent; Framing framing; std::mutex buffer_mutex; bool should_quit = false; // Under . std::condition_variable has_buffered_data; std::deque buffered_data; // Protected by . size_t used_of_buffered_data = 0; // How many bytes of the first element of that is already used. Protected by . size_t seen_keyframe = false; }; MHD_Daemon *mhd = nullptr; std::mutex streams_mutex; std::set streams; // Not owned. std::string header; }; #endif // !defined(_HTTPD_H) nageru-1.4.2/image_input.cpp000066400000000000000000000212641301561706600160160ustar00rootroot00000000000000#include "image_input.h" #include #include #include #include #include #include extern "C" { #include #include #include #include #include #include #include #include #include } #include #include #include #include #include #include #include #include #include #include "ffmpeg_raii.h" #include "flags.h" #include "flat_input.h" struct SwsContext; using namespace std; namespace { string search_for_file(const string &filename) { // Look for the file in all theme_dirs until we find one; // that will be the permanent resolution of this file, whether // it is actually valid or not. // We store errors from all the attempts, and show them // once we know we can't find any of them. vector errors; for (const string &dir : global_flags.theme_dirs) { string pathname = dir + "/" + filename; if (access(pathname.c_str(), O_RDONLY) == 0) { return pathname; } else { char buf[512]; snprintf(buf, sizeof(buf), "%s: %s", pathname.c_str(), strerror(errno)); errors.push_back(buf); } } for (const string &error : errors) { fprintf(stderr, "%s\n", error.c_str()); } fprintf(stderr, "Couldn't find %s in any directory in --theme-dirs, exiting.\n", filename.c_str()); exit(1); } } // namespace ImageInput::ImageInput(const string &filename) : movit::FlatInput({movit::COLORSPACE_sRGB, movit::GAMMA_sRGB}, movit::FORMAT_RGBA_POSTMULTIPLIED_ALPHA, GL_UNSIGNED_BYTE, 1280, 720), // FIXME pathname(search_for_file(filename)), current_image(load_image(pathname)) { if (current_image == nullptr) { // Could happen even though search_for_file() returned. fprintf(stderr, "Couldn't load image, exiting.\n"); exit(1); } set_pixel_data(current_image->pixels.get()); } void ImageInput::set_gl_state(GLuint glsl_program_num, const string& prefix, unsigned *sampler_num) { // See if the background thread has given us a new version of our image. // Note: The old version might still be lying around in other ImageInputs // (in fact, it's likely), but at least the total amount of memory used // is bounded. Currently we don't even share textures between them, // so there's a fair amount of OpenGL memory waste anyway (the cache // is mostly there to save startup time, not RAM). { unique_lock lock(all_images_lock); if (all_images[pathname] != current_image) { current_image = all_images[pathname]; set_pixel_data(current_image->pixels.get()); } } movit::FlatInput::set_gl_state(glsl_program_num, prefix, sampler_num); } shared_ptr ImageInput::load_image(const string &pathname) { unique_lock lock(all_images_lock); // Held also during loading. if (all_images.count(pathname)) { return all_images[pathname]; } all_images[pathname] = load_image_raw(pathname); timespec first_modified = all_images[pathname]->last_modified; update_threads[pathname] = thread(bind(update_thread_func, pathname, first_modified)); return all_images[pathname]; } shared_ptr ImageInput::load_image_raw(const string &pathname) { // Note: Call before open, not after; otherwise, there's a race. // (There is now, too, but it tips the correct way. We could use fstat() // if we had the file descriptor.) struct stat buf; if (stat(pathname.c_str(), &buf) != 0) { fprintf(stderr, "%s: Error stat-ing file\n", pathname.c_str()); return nullptr; } timespec last_modified = buf.st_mtim; auto format_ctx = avformat_open_input_unique(pathname.c_str(), nullptr, nullptr); if (format_ctx == nullptr) { fprintf(stderr, "%s: Error opening file\n", pathname.c_str()); return nullptr; } if (avformat_find_stream_info(format_ctx.get(), nullptr) < 0) { fprintf(stderr, "%s: Error finding stream info\n", pathname.c_str()); return nullptr; } int stream_index = -1; for (unsigned i = 0; i < format_ctx->nb_streams; ++i) { if (format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { stream_index = i; break; } } if (stream_index == -1) { fprintf(stderr, "%s: No video stream found\n", pathname.c_str()); return nullptr; } const AVCodecParameters *codecpar = format_ctx->streams[stream_index]->codecpar; AVCodecContextWithDeleter codec_ctx = avcodec_alloc_context3_unique(nullptr); if (avcodec_parameters_to_context(codec_ctx.get(), codecpar) < 0) { fprintf(stderr, "%s: Cannot fill codec parameters\n", pathname.c_str()); return nullptr; } AVCodec *codec = avcodec_find_decoder(codecpar->codec_id); if (codec == nullptr) { fprintf(stderr, "%s: Cannot find decoder\n", pathname.c_str()); return nullptr; } if (avcodec_open2(codec_ctx.get(), codec, nullptr) < 0) { fprintf(stderr, "%s: Cannot open decoder\n", pathname.c_str()); return nullptr; } unique_ptr codec_ctx_cleanup( codec_ctx.get(), avcodec_close); // Read packets until we have a frame or there are none left. int frame_finished = 0; AVFrameWithDeleter frame = av_frame_alloc_unique(); bool eof = false; do { AVPacket pkt; unique_ptr pkt_cleanup( &pkt, av_packet_unref); av_init_packet(&pkt); pkt.data = nullptr; pkt.size = 0; if (av_read_frame(format_ctx.get(), &pkt) == 0) { if (pkt.stream_index != stream_index) { continue; } if (avcodec_send_packet(codec_ctx.get(), &pkt) < 0) { fprintf(stderr, "%s: Cannot send packet to codec.\n", pathname.c_str()); return nullptr; } } else { eof = true; // Or error, but ignore that for the time being. } int err = avcodec_receive_frame(codec_ctx.get(), frame.get()); if (err == 0) { frame_finished = true; break; } else if (err != AVERROR(EAGAIN)) { fprintf(stderr, "%s: Cannot receive frame from codec.\n", pathname.c_str()); return nullptr; } } while (!eof); if (!frame_finished) { fprintf(stderr, "%s: Decoder did not output frame.\n", pathname.c_str()); return nullptr; } // TODO: Scale down if needed! uint8_t *pic_data[4] = {nullptr}; unique_ptr pic_data_cleanup( &pic_data[0], av_freep); int linesizes[4]; if (av_image_alloc(pic_data, linesizes, frame->width, frame->height, AV_PIX_FMT_RGBA, 1) < 0) { fprintf(stderr, "%s: Could not allocate picture data\n", pathname.c_str()); return nullptr; } unique_ptr sws_ctx( sws_getContext(frame->width, frame->height, (AVPixelFormat)frame->format, frame->width, frame->height, AV_PIX_FMT_RGBA, SWS_BICUBIC, nullptr, nullptr, nullptr), sws_freeContext); if (sws_ctx == nullptr) { fprintf(stderr, "%s: Could not create scaler context\n", pathname.c_str()); return nullptr; } sws_scale(sws_ctx.get(), frame->data, frame->linesize, 0, frame->height, pic_data, linesizes); size_t len = frame->width * frame->height * 4; unique_ptr image_data(new uint8_t[len]); av_image_copy_to_buffer(image_data.get(), len, pic_data, linesizes, AV_PIX_FMT_RGBA, frame->width, frame->height, 1); shared_ptr image(new Image{move(image_data), last_modified}); return image; } // Fire up a thread to update the image every second. // We could do inotify, but this is good enough for now. void ImageInput::update_thread_func(const std::string &pathname, const timespec &first_modified) { timespec last_modified = first_modified; struct stat buf; for ( ;; ) { sleep(1); if (threads_should_quit) { return; } if (stat(pathname.c_str(), &buf) != 0) { fprintf(stderr, "%s: Couldn't check for new version, leaving the old in place.\n", pathname.c_str()); continue; } if (buf.st_mtim.tv_sec == last_modified.tv_sec && buf.st_mtim.tv_nsec == last_modified.tv_nsec) { // Not changed. continue; } shared_ptr image = load_image_raw(pathname); if (image == nullptr) { fprintf(stderr, "Couldn't load image, leaving the old in place.\n"); continue; } fprintf(stderr, "Loaded new version of %s from disk.\n", pathname.c_str()); unique_lock lock(all_images_lock); all_images[pathname] = image; last_modified = image->last_modified; } } void ImageInput::shutdown_updaters() { // TODO: Kick these out of the sleep before one second? threads_should_quit = true; for (auto &it : update_threads) { it.second.join(); } } mutex ImageInput::all_images_lock; map> ImageInput::all_images; map ImageInput::update_threads; volatile bool ImageInput::threads_should_quit = false; nageru-1.4.2/image_input.h000066400000000000000000000025701301561706600154620ustar00rootroot00000000000000#ifndef _IMAGE_INPUT_H #define _IMAGE_INPUT_H 1 #include #include #include #include #include #include #include #include #include #include // An output that takes its input from a static image, loaded with ffmpeg. // comes from a single 2D array with chunky pixels. The image is refreshed // from disk about every second. class ImageInput : public movit::FlatInput { public: ImageInput(const std::string &pathname); std::string effect_type_id() const override { return "ImageInput"; } void set_gl_state(GLuint glsl_program_num, const std::string& prefix, unsigned *sampler_num) override; static void shutdown_updaters(); private: struct Image { std::unique_ptr pixels; timespec last_modified; }; std::string pathname; std::shared_ptr current_image; static std::shared_ptr load_image(const std::string &pathname); static std::shared_ptr load_image_raw(const std::string &pathname); static void update_thread_func(const std::string &pathname, const timespec &first_modified); static std::mutex all_images_lock; static std::map> all_images; static std::map update_threads; static volatile bool threads_should_quit; }; #endif // !defined(_IMAGE_INPUT_H) nageru-1.4.2/input_mapping.cpp000066400000000000000000000144511301561706600163670ustar00rootroot00000000000000#include "input_mapping.h" #include #include #include #include #include #include #include #include "audio_mixer.h" #include "state.pb.h" using namespace std; using namespace google::protobuf; bool save_input_mapping_to_file(const map &devices, const InputMapping &input_mapping, const string &filename) { InputMappingProto mapping_proto; { map used_devices; for (const InputMapping::Bus &bus : input_mapping.buses) { if (!used_devices.count(bus.device)) { used_devices.emplace(bus.device, used_devices.size()); global_audio_mixer->serialize_device(bus.device, mapping_proto.add_device()); } BusProto *bus_proto = mapping_proto.add_bus(); bus_proto->set_name(bus.name); bus_proto->set_device_index(used_devices[bus.device]); bus_proto->set_source_channel_left(bus.source_channel[0]); bus_proto->set_source_channel_right(bus.source_channel[1]); } } // Save to disk. We use the text format because it's friendlier // for a user to look at and edit. int fd = open(filename.c_str(), O_WRONLY | O_TRUNC | O_CREAT, 0666); if (fd == -1) { perror(filename.c_str()); return false; } io::FileOutputStream output(fd); // Takes ownership of fd. if (!TextFormat::Print(mapping_proto, &output)) { // TODO: Don't overwrite the old file (if any) on error. output.Close(); return false; } output.Close(); return true; } bool load_input_mapping_from_file(const map &devices, const string &filename, InputMapping *new_mapping) { // Read and parse the protobuf from disk. int fd = open(filename.c_str(), O_RDONLY); if (fd == -1) { perror(filename.c_str()); return false; } io::FileInputStream input(fd); // Takes ownership of fd. InputMappingProto mapping_proto; if (!TextFormat::Parse(&input, &mapping_proto)) { input.Close(); return false; } input.Close(); // Map devices in the proto to our current ones: // Get a list of all active devices. set remaining_devices; for (const auto &device_spec_and_info : devices) { remaining_devices.insert(device_spec_and_info.first); } // Now look at every device in the serialized protobuf and try to map // it to one device we haven't taken yet. This isn't a full maximal matching, // but it's good enough for our uses. vector device_mapping; for (unsigned device_index = 0; device_index < unsigned(mapping_proto.device_size()); ++device_index) { const DeviceSpecProto &device_proto = mapping_proto.device(device_index); switch (device_proto.type()) { case DeviceSpecProto::SILENCE: device_mapping.push_back(DeviceSpec{InputSourceType::SILENCE, 0}); break; case DeviceSpecProto::CAPTURE_CARD: { // First see if there's a card that matches on both index and name. DeviceSpec spec{InputSourceType::CAPTURE_CARD, unsigned(device_proto.index())}; assert(devices.count(spec)); const DeviceInfo &dev = devices.find(spec)->second; if (remaining_devices.count(spec) && dev.display_name == device_proto.display_name()) { device_mapping.push_back(spec); remaining_devices.erase(spec); goto found_capture_card; } // Scan and see if there's a match on name alone. for (const DeviceSpec &spec : remaining_devices) { if (spec.type == InputSourceType::CAPTURE_CARD && dev.display_name == device_proto.display_name()) { device_mapping.push_back(spec); remaining_devices.erase(spec); goto found_capture_card; } } // OK, see if at least the index is free. if (remaining_devices.count(spec)) { device_mapping.push_back(spec); remaining_devices.erase(spec); goto found_capture_card; } // Give up. device_mapping.push_back(DeviceSpec{InputSourceType::SILENCE, 0}); found_capture_card: break; } case DeviceSpecProto::ALSA_INPUT: { // For ALSA, we don't really care about index, but we can use address // in its place. // First see if there's a card that matches on name, num_channels and address. for (const DeviceSpec &spec : remaining_devices) { assert(devices.count(spec)); const DeviceInfo &dev = devices.find(spec)->second; if (spec.type == InputSourceType::ALSA_INPUT && dev.alsa_name == device_proto.alsa_name() && dev.alsa_info == device_proto.alsa_info() && int(dev.num_channels) == device_proto.num_channels() && dev.alsa_address == device_proto.address()) { device_mapping.push_back(spec); remaining_devices.erase(spec); goto found_alsa_input; } } // Looser check: Ignore the address. for (const DeviceSpec &spec : remaining_devices) { assert(devices.count(spec)); const DeviceInfo &dev = devices.find(spec)->second; if (spec.type == InputSourceType::ALSA_INPUT && dev.alsa_name == device_proto.alsa_name() && dev.alsa_info == device_proto.alsa_info() && int(dev.num_channels) == device_proto.num_channels()) { device_mapping.push_back(spec); remaining_devices.erase(spec); goto found_alsa_input; } } // OK, so we couldn't map this to a device, but perhaps one is added // at some point in the future through hotplug. Create a dead card // matching this one; right now, it will give only silence, // but it could be replaced with something later. // // NOTE: There's a potential race condition here, if the card // gets inserted while we're doing the device remapping // (or perhaps more realistically, while we're reading the // input mapping from disk). DeviceSpec dead_card_spec; dead_card_spec = global_audio_mixer->create_dead_card( device_proto.alsa_name(), device_proto.alsa_info(), device_proto.num_channels()); device_mapping.push_back(dead_card_spec); found_alsa_input: break; } default: assert(false); } } for (const BusProto &bus_proto : mapping_proto.bus()) { if (bus_proto.device_index() < 0 || unsigned(bus_proto.device_index()) >= device_mapping.size()) { return false; } InputMapping::Bus bus; bus.name = bus_proto.name(); bus.device = device_mapping[bus_proto.device_index()]; bus.source_channel[0] = bus_proto.source_channel_left(); bus.source_channel[1] = bus_proto.source_channel_right(); new_mapping->buses.push_back(bus); } return true; } nageru-1.4.2/input_mapping.h000066400000000000000000000030351301561706600160300ustar00rootroot00000000000000#ifndef _INPUT_MAPPING_H #define _INPUT_MAPPING_H 1 #include #include #include #include enum class InputSourceType { SILENCE, CAPTURE_CARD, ALSA_INPUT }; struct DeviceSpec { InputSourceType type; unsigned index; bool operator== (const DeviceSpec &other) const { return type == other.type && index == other.index; } bool operator< (const DeviceSpec &other) const { if (type != other.type) return type < other.type; return index < other.index; } }; struct DeviceInfo { std::string display_name; unsigned num_channels; std::string alsa_name, alsa_info, alsa_address; // ALSA devices only, obviously. }; static inline uint64_t DeviceSpec_to_key(const DeviceSpec &device_spec) { return (uint64_t(device_spec.type) << 32) | device_spec.index; } static inline DeviceSpec key_to_DeviceSpec(uint64_t key) { return DeviceSpec{ InputSourceType(key >> 32), unsigned(key & 0xffffffff) }; } struct InputMapping { struct Bus { std::string name; DeviceSpec device; int source_channel[2] { -1, -1 }; // Left and right. -1 = none. }; std::vector buses; }; bool save_input_mapping_to_file(const std::map &devices, const InputMapping &mapping, const std::string &filename); bool load_input_mapping_from_file(const std::map &devices, const std::string &filename, InputMapping *mapping); #endif // !defined(_INPUT_MAPPING_H) nageru-1.4.2/input_mapping_dialog.cpp000066400000000000000000000262111301561706600177030ustar00rootroot00000000000000#include "input_mapping_dialog.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "alsa_pool.h" #include "defs.h" #include "post_to_main_thread.h" #include "ui_input_mapping.h" using namespace std; using namespace std::placeholders; InputMappingDialog::InputMappingDialog() : ui(new Ui::InputMappingDialog), mapping(global_audio_mixer->get_input_mapping()), old_mapping(mapping), devices(global_audio_mixer->get_devices()) { for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { bus_settings.push_back(global_audio_mixer->get_bus_settings(bus_index)); } ui->setupUi(this); ui->table->setSelectionBehavior(QAbstractItemView::SelectRows); ui->table->setSelectionMode(QAbstractItemView::SingleSelection); // Makes implementing moving easier for now. fill_ui_from_mapping(mapping); connect(ui->table, &QTableWidget::cellChanged, this, &InputMappingDialog::cell_changed); connect(ui->ok_cancel_buttons, &QDialogButtonBox::accepted, this, &InputMappingDialog::ok_clicked); connect(ui->ok_cancel_buttons, &QDialogButtonBox::rejected, this, &InputMappingDialog::cancel_clicked); connect(ui->add_button, &QPushButton::clicked, this, &InputMappingDialog::add_clicked); connect(ui->remove_button, &QPushButton::clicked, this, &InputMappingDialog::remove_clicked); connect(ui->up_button, &QPushButton::clicked, bind(&InputMappingDialog::updown_clicked, this, -1)); connect(ui->down_button, &QPushButton::clicked, bind(&InputMappingDialog::updown_clicked, this, 1)); connect(ui->save_button, &QPushButton::clicked, this, &InputMappingDialog::save_clicked); connect(ui->load_button, &QPushButton::clicked, this, &InputMappingDialog::load_clicked); update_button_state(); connect(ui->table, &QTableWidget::itemSelectionChanged, this, &InputMappingDialog::update_button_state); saved_callback = global_audio_mixer->get_state_changed_callback(); global_audio_mixer->set_state_changed_callback([this]{ post_to_main_thread([this]{ devices = global_audio_mixer->get_devices(); for (unsigned row = 0; row < mapping.buses.size(); ++row) { fill_row_from_bus(row, mapping.buses[row]); } }); }); } InputMappingDialog::~InputMappingDialog() { global_audio_mixer->set_state_changed_callback(saved_callback); } void InputMappingDialog::fill_ui_from_mapping(const InputMapping &mapping) { ui->table->verticalHeader()->hide(); ui->table->horizontalHeader()->setSectionResizeMode(1, QHeaderView::ResizeToContents); ui->table->horizontalHeader()->setSectionResizeMode(2, QHeaderView::ResizeToContents); ui->table->horizontalHeader()->setSectionResizeMode(3, QHeaderView::ResizeToContents); ui->table->horizontalHeader()->setSectionsClickable(false); ui->table->setRowCount(mapping.buses.size()); for (unsigned row = 0; row < mapping.buses.size(); ++row) { fill_row_from_bus(row, mapping.buses[row]); } } void InputMappingDialog::fill_row_from_bus(unsigned row, const InputMapping::Bus &bus) { QString name(QString::fromStdString(bus.name)); ui->table->setItem(row, 0, new QTableWidgetItem(name)); // Card choices. If there's already a combobox here, we try to modify // the elements in-place, so that the UI doesn't go away under the user's feet // if they are in the process of choosing an item. QComboBox *card_combo = static_cast(ui->table->cellWidget(row, 1)); if (card_combo == nullptr) { card_combo = new QComboBox; } unsigned current_index = 0; if (card_combo->count() == 0) { card_combo->addItem(QString("(none) ")); } for (const auto &spec_and_info : devices) { QString label(QString::fromStdString(spec_and_info.second.display_name)); if (spec_and_info.first.type == InputSourceType::ALSA_INPUT) { ALSAPool::Device::State state = global_audio_mixer->get_alsa_card_state(spec_and_info.first.index); if (state == ALSAPool::Device::State::EMPTY) { continue; } else if (state == ALSAPool::Device::State::STARTING) { label += " (busy)"; } else if (state == ALSAPool::Device::State::DEAD) { label += " (dead)"; } } ++current_index; if (unsigned(card_combo->count()) > current_index) { card_combo->setItemText(current_index, label + " "); card_combo->setItemData(current_index, qulonglong(DeviceSpec_to_key(spec_and_info.first))); } else { card_combo->addItem( label + " ", qulonglong(DeviceSpec_to_key(spec_and_info.first))); } if (bus.device == spec_and_info.first) { card_combo->setCurrentIndex(current_index); } } // Remove any excess items from earlier. (This is only for paranoia; // they should be held, so it shouldn't matter.) while (unsigned(card_combo->count()) > current_index + 1) { card_combo->removeItem(current_index + 1); } connect(card_combo, static_cast(&QComboBox::currentIndexChanged), bind(&InputMappingDialog::card_selected, this, card_combo, row, _1)); ui->table->setCellWidget(row, 1, card_combo); setup_channel_choices_from_bus(row, bus); } void InputMappingDialog::setup_channel_choices_from_bus(unsigned row, const InputMapping::Bus &bus) { // Left and right channel. // TODO: If there's already a widget here, modify it instead of creating a new one, // as we do with card choices. for (unsigned channel = 0; channel < 2; ++channel) { QComboBox *channel_combo = new QComboBox; channel_combo->addItem(QString("(none)")); if (bus.device.type == InputSourceType::CAPTURE_CARD || bus.device.type == InputSourceType::ALSA_INPUT) { auto device_it = devices.find(bus.device); assert(device_it != devices.end()); unsigned num_device_channels = device_it->second.num_channels; for (unsigned source = 0; source < num_device_channels; ++source) { char buf[256]; snprintf(buf, sizeof(buf), "Channel %u ", source + 1); channel_combo->addItem(QString(buf)); } channel_combo->setCurrentIndex(bus.source_channel[channel] + 1); } else { channel_combo->setCurrentIndex(0); } connect(channel_combo, static_cast(&QComboBox::currentIndexChanged), bind(&InputMappingDialog::channel_selected, this, row, channel, _1)); ui->table->setCellWidget(row, 2 + channel, channel_combo); } } void InputMappingDialog::ok_clicked() { global_audio_mixer->set_state_changed_callback(saved_callback); global_audio_mixer->set_input_mapping(mapping); for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { global_audio_mixer->set_bus_settings(bus_index, bus_settings[bus_index]); global_audio_mixer->reset_peak(bus_index); } accept(); } void InputMappingDialog::cancel_clicked() { global_audio_mixer->set_state_changed_callback(saved_callback); global_audio_mixer->set_input_mapping(old_mapping); reject(); } void InputMappingDialog::cell_changed(int row, int column) { if (column != 0) { // Spurious; only really the name column should fire these. return; } mapping.buses[row].name = ui->table->item(row, column)->text().toStdString(); } void InputMappingDialog::card_selected(QComboBox *card_combo, unsigned row, int index) { uint64_t key = card_combo->itemData(index).toULongLong(); mapping.buses[row].device = key_to_DeviceSpec(key); setup_channel_choices_from_bus(row, mapping.buses[row]); } void InputMappingDialog::channel_selected(unsigned row, unsigned channel, int index) { mapping.buses[row].source_channel[channel] = index - 1; } void InputMappingDialog::add_clicked() { QTableWidgetSelectionRange all(0, 0, ui->table->rowCount() - 1, ui->table->columnCount() - 1); ui->table->setRangeSelected(all, false); InputMapping::Bus new_bus; new_bus.name = "New input"; new_bus.device.type = InputSourceType::SILENCE; mapping.buses.push_back(new_bus); bus_settings.push_back(AudioMixer::get_default_bus_settings()); ui->table->setRowCount(mapping.buses.size()); unsigned row = mapping.buses.size() - 1; fill_row_from_bus(row, new_bus); ui->table->editItem(ui->table->item(row, 0)); // Start editing the name. update_button_state(); } void InputMappingDialog::remove_clicked() { assert(ui->table->rowCount() != 0); set> rows_to_delete; // Need to remove in reverse order. for (const QTableWidgetSelectionRange &range : ui->table->selectedRanges()) { for (int row = range.topRow(); row <= range.bottomRow(); ++row) { rows_to_delete.insert(row); } } if (rows_to_delete.empty()) { rows_to_delete.insert(ui->table->rowCount() - 1); } for (int row : rows_to_delete) { ui->table->removeRow(row); mapping.buses.erase(mapping.buses.begin() + row); bus_settings.erase(bus_settings.begin() + row); } update_button_state(); } void InputMappingDialog::updown_clicked(int direction) { assert(ui->table->selectedRanges().size() == 1); const QTableWidgetSelectionRange &range = ui->table->selectedRanges()[0]; int a_row = range.bottomRow(); int b_row = range.bottomRow() + direction; swap(mapping.buses[a_row], mapping.buses[b_row]); swap(bus_settings[a_row], bus_settings[b_row]); fill_row_from_bus(a_row, mapping.buses[a_row]); fill_row_from_bus(b_row, mapping.buses[b_row]); QTableWidgetSelectionRange a_sel(a_row, 0, a_row, ui->table->columnCount() - 1); QTableWidgetSelectionRange b_sel(b_row, 0, b_row, ui->table->columnCount() - 1); ui->table->setRangeSelected(a_sel, false); ui->table->setRangeSelected(b_sel, true); } void InputMappingDialog::save_clicked() { QString filename = QFileDialog::getSaveFileName(this, "Save input mapping", QString(), tr("Mapping files (*.mapping)")); if (!filename.endsWith(".mapping")) { filename += ".mapping"; } if (!save_input_mapping_to_file(devices, mapping, filename.toStdString())) { QMessageBox box; box.setText("Could not save mapping to '" + filename + "'. Check that you have the right permissions and try again."); box.exec(); } } void InputMappingDialog::load_clicked() { QString filename = QFileDialog::getOpenFileName(this, "Load input mapping", QString(), tr("Mapping files (*.mapping)")); InputMapping new_mapping; if (!load_input_mapping_from_file(devices, filename.toStdString(), &new_mapping)) { QMessageBox box; box.setText("Could not load mapping from '" + filename + "'. Check that the file exists, has the right permissions and is valid."); box.exec(); return; } mapping = new_mapping; bus_settings.clear(); for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { bus_settings.push_back(global_audio_mixer->get_bus_settings(bus_index)); } devices = global_audio_mixer->get_devices(); // New dead cards may have been made. fill_ui_from_mapping(mapping); } void InputMappingDialog::update_button_state() { ui->add_button->setDisabled(mapping.buses.size() >= MAX_BUSES); ui->remove_button->setDisabled(mapping.buses.size() == 0); ui->up_button->setDisabled( ui->table->selectedRanges().empty() || ui->table->selectedRanges()[0].bottomRow() == 0); ui->down_button->setDisabled( ui->table->selectedRanges().empty() || ui->table->selectedRanges()[0].bottomRow() == ui->table->rowCount() - 1); } nageru-1.4.2/input_mapping_dialog.h000066400000000000000000000031741301561706600173530ustar00rootroot00000000000000#ifndef _INPUT_MAPPING_DIALOG_H #define _INPUT_MAPPING_DIALOG_H #include #include #include #include #include "audio_mixer.h" #include "input_mapping.h" class QObject; namespace Ui { class InputMappingDialog; } // namespace Ui class QComboBox; class InputMappingDialog : public QDialog { Q_OBJECT public: InputMappingDialog(); ~InputMappingDialog(); private: void fill_ui_from_mapping(const InputMapping &mapping); void fill_row_from_bus(unsigned row, const InputMapping::Bus &bus); void setup_channel_choices_from_bus(unsigned row, const InputMapping::Bus &bus); void cell_changed(int row, int column); void card_selected(QComboBox *card_combo, unsigned row, int index); void channel_selected(unsigned row, unsigned channel, int index); void ok_clicked(); void cancel_clicked(); void add_clicked(); void remove_clicked(); void updown_clicked(int direction); void save_clicked(); void load_clicked(); void update_button_state(); Ui::InputMappingDialog *ui; InputMapping mapping; // Under edit. Will be committed on OK. // The old mapping. Will be re-committed on cancel, so that we // unhold all the unused devices (otherwise they would be // held forever). InputMapping old_mapping; // One for each bus in the mapping. Edited along with the mapping, // so that old volumes etc. are being kept in place for buses that // existed before. std::vector bus_settings; std::map devices; // Needs no lock, accessed only on the UI thread. AudioMixer::state_changed_callback_t saved_callback; }; #endif // !defined(_INPUT_MAPPING_DIALOG_H) nageru-1.4.2/input_state.h000066400000000000000000000014461301561706600155210ustar00rootroot00000000000000#ifndef _INPUT_STATE_H #define _INPUT_STATE_H 1 #include "defs.h" #include "ref_counted_frame.h" struct BufferedFrame { RefCountedFrame frame; unsigned field_number; }; // Encapsulates the state of all inputs at any given instant. // In particular, this is captured by Theme::get_chain(), // so that it can hold on to all the frames it needs for rendering. struct InputState { // For each card, the last five frames (or fields), with 0 being the // most recent one. Note that we only need the actual history if we have // interlaced output (for deinterlacing), so if we detect progressive input, // we immediately clear out all history and all entries will point to the same // frame. BufferedFrame buffered_frames[MAX_VIDEO_CARDS][FRAME_HISTORY_LENGTH]; }; #endif // !defined(_INPUT_STATE_H) nageru-1.4.2/lrameter.cpp000066400000000000000000000056201301561706600153260ustar00rootroot00000000000000#include "lrameter.h" #include #include #include #include #include "vu_common.h" class QPaintEvent; class QResizeEvent; using namespace std; LRAMeter::LRAMeter(QWidget *parent) : QWidget(parent) { } void LRAMeter::resizeEvent(QResizeEvent *event) { recalculate_pixmaps(); } void LRAMeter::paintEvent(QPaintEvent *event) { QPainter painter(this); float level_lufs; float range_low_lufs; float range_high_lufs; { unique_lock lock(level_mutex); level_lufs = this->level_lufs; range_low_lufs = this->range_low_lufs; range_high_lufs = this->range_high_lufs; } float level_lu = level_lufs - ref_level_lufs; float range_low_lu = range_low_lufs - ref_level_lufs; float range_high_lu = range_high_lufs - ref_level_lufs; int range_low_pos = lrint(lufs_to_pos(range_low_lu, height())); int range_high_pos = lrint(lufs_to_pos(range_high_lu, height())); QRect top_off_rect(0, 0, width(), range_high_pos); QRect on_rect(0, range_high_pos, width(), range_low_pos - range_high_pos); QRect bottom_off_rect(0, range_low_pos, width(), height() - range_low_pos); painter.drawPixmap(top_off_rect, off_pixmap, top_off_rect); painter.drawPixmap(on_rect, on_pixmap, on_rect); painter.drawPixmap(bottom_off_rect, off_pixmap, bottom_off_rect); // Draw the target area (+/-1 LU is allowed EBU range). // It turns green when we're within. int min_y = lrint(lufs_to_pos(1.0f, height())); int max_y = lrint(lufs_to_pos(-1.0f, height())); // FIXME: This outlining isn't so pretty. { QPen pen(Qt::black); pen.setWidth(5); painter.setPen(pen); painter.drawRect(2, min_y, width() - 5, max_y - min_y); } { QPen pen; if (level_lu >= -1.0f && level_lu <= 1.0f) { pen.setColor(Qt::green); } else { pen.setColor(Qt::red); } pen.setWidth(3); painter.setPen(pen); painter.drawRect(2, min_y, width() - 5, max_y - min_y); } // Draw the integrated loudness meter, in the same color as the target area. int y = lrint(lufs_to_pos(level_lu, height())); { QPen pen(Qt::black); pen.setWidth(5); painter.setPen(pen); painter.drawRect(2, y, width() - 5, 1); } { QPen pen; if (level_lu >= -1.0f && level_lu <= 1.0f) { pen.setColor(Qt::green); } else { pen.setColor(Qt::red); } pen.setWidth(3); painter.setPen(pen); painter.drawRect(2, y, width() - 5, 1); } } void LRAMeter::recalculate_pixmaps() { const int margin = 5; on_pixmap = QPixmap(width(), height()); QPainter on_painter(&on_pixmap); on_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(on_painter, width(), height(), margin, 2.0, true, min_level, max_level, /*flip=*/false); off_pixmap = QPixmap(width(), height()); QPainter off_painter(&off_pixmap); off_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(off_painter, width(), height(), margin, 2.0, false, min_level, max_level, /*flip=*/false); } nageru-1.4.2/lrameter.h000066400000000000000000000025641301561706600147770ustar00rootroot00000000000000#ifndef LRAMETER_H #define LRAMETER_H #include #include #include #include #include #include "vu_common.h" class QObject; class QPaintEvent; class QResizeEvent; class LRAMeter : public QWidget { Q_OBJECT public: LRAMeter(QWidget *parent); void set_levels(float level_lufs, float range_low_lufs, float range_high_lufs) { std::unique_lock lock(level_mutex); this->level_lufs = level_lufs; this->range_low_lufs = range_low_lufs; this->range_high_lufs = range_high_lufs; QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); } double lufs_to_pos(float level_lu, int height) { return ::lufs_to_pos(level_lu, height, min_level, max_level); } void set_min_level(float min_level) { this->min_level = min_level; recalculate_pixmaps(); } void set_max_level(float max_level) { this->max_level = max_level; recalculate_pixmaps(); } void set_ref_level(float ref_level_lufs) { this->ref_level_lufs = ref_level_lufs; } private: void resizeEvent(QResizeEvent *event) override; void paintEvent(QPaintEvent *event) override; void recalculate_pixmaps(); std::mutex level_mutex; float level_lufs = -HUGE_VAL; float range_low_lufs = -HUGE_VAL; float range_high_lufs = -HUGE_VAL; float min_level = -18.0f, max_level = 9.0f, ref_level_lufs = -23.0f; QPixmap on_pixmap, off_pixmap; }; #endif nageru-1.4.2/main.cpp000066400000000000000000000043751301561706600144450ustar00rootroot00000000000000extern "C" { #include } #include #include #include #include #include // IWYU pragma: keep #include #include #include #include #include #include #include "context.h" #include "flags.h" #include "image_input.h" #include "mainwindow.h" #include "mixer.h" int main(int argc, char *argv[]) { parse_flags(argc, argv); if (global_flags.va_display.empty() || global_flags.va_display[0] != '/') { // We normally use EGL for zerocopy, but if we use VA against DRM // instead of against X11, we turn it off, and then don't need EGL. setenv("QT_XCB_GL_INTEGRATION", "xcb_egl", 0); using_egl = true; } setlinebuf(stdout); av_register_all(); QCoreApplication::setAttribute(Qt::AA_ShareOpenGLContexts, true); QApplication app(argc, argv); QSurfaceFormat fmt; fmt.setDepthBufferSize(0); fmt.setStencilBufferSize(0); fmt.setProfile(QSurfaceFormat::CoreProfile); fmt.setMajorVersion(3); fmt.setMinorVersion(1); // Turn off vsync, since Qt generally gives us at most frame rate // (display frequency) / (number of QGLWidgets active). fmt.setSwapInterval(0); QSurfaceFormat::setDefaultFormat(fmt); QGLFormat::setDefaultFormat(QGLFormat::fromSurfaceFormat(fmt)); global_share_widget = new QGLWidget(); MainWindow mainWindow; mainWindow.resize(QSize(1500, 850)); mainWindow.show(); app.installEventFilter(&mainWindow); // For white balance color picking. // Even on an otherwise unloaded system, it would seem writing the recording // to disk (potentially terabytes of data as time goes by) causes Nageru // to be pushed out of RAM. If we have the right privileges, simply lock us // into memory for better realtime behavior. if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) { perror("mlockall()"); fprintf(stderr, "Failed to lock Nageru into RAM. You probably want to\n"); fprintf(stderr, "increase \"memlock\" for your user in limits.conf\n"); fprintf(stderr, "for better realtime behavior.\n"); uses_mlock = false; } else { uses_mlock = true; } int rc = app.exec(); global_mixer->quit(); mainWindow.mixer_shutting_down(); delete global_mixer; ImageInput::shutdown_updaters(); return rc; } nageru-1.4.2/mainwindow.cpp000066400000000000000000001327541301561706600157000ustar00rootroot00000000000000#include "mainwindow.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "aboutdialog.h" #include "alsa_pool.h" #include "clickable_label.h" #include "correlation_meter.h" #include "disk_space_estimator.h" #include "ellipsis_label.h" #include "flags.h" #include "glwidget.h" #include "input_mapping.h" #include "input_mapping_dialog.h" #include "lrameter.h" #include "midi_mapping.pb.h" #include "midi_mapping_dialog.h" #include "mixer.h" #include "nonlinear_fader.h" #include "post_to_main_thread.h" #include "ui_audio_expanded_view.h" #include "ui_audio_miniview.h" #include "ui_display.h" #include "ui_mainwindow.h" #include "vumeter.h" using namespace std; using namespace std::chrono; using namespace std::placeholders; Q_DECLARE_METATYPE(std::string); Q_DECLARE_METATYPE(std::vector); MainWindow *global_mainwindow = nullptr; // -0.1 dBFS is EBU peak limit. We use it consistently, even for the bus meters // (which don't calculate interpolate peak, and in general don't follow EBU recommendations). constexpr float peak_limit_dbfs = -0.1f; namespace { void schedule_cut_signal(int ignored) { global_mixer->schedule_cut(); } void quit_signal(int ignored) { global_mainwindow->close(); } void slave_knob(QDial *master, QDial *slave) { QWidget::connect(master, &QDial::valueChanged, [slave](int value){ slave->blockSignals(true); slave->setValue(value); slave->blockSignals(false); }); QWidget::connect(slave, &QDial::valueChanged, [master](int value){ master->setValue(value); }); } void slave_checkbox(QCheckBox *master, QCheckBox *slave) { QWidget::connect(master, &QCheckBox::stateChanged, [slave](int state){ slave->blockSignals(true); slave->setCheckState(Qt::CheckState(state)); slave->blockSignals(false); }); QWidget::connect(slave, &QCheckBox::stateChanged, [master](int state){ master->setCheckState(Qt::CheckState(state)); }); } void slave_fader(NonLinearFader *master, NonLinearFader *slave) { QWidget::connect(master, &NonLinearFader::dbValueChanged, [slave](double value) { slave->blockSignals(true); slave->setDbValue(value); slave->blockSignals(false); }); QWidget::connect(slave, &NonLinearFader::dbValueChanged, [master](double value){ master->setDbValue(value); }); } constexpr unsigned DB_NO_FLAGS = 0x0; constexpr unsigned DB_WITH_SIGN = 0x1; constexpr unsigned DB_BARE = 0x2; string format_db(double db, unsigned flags) { string text; if (flags & DB_WITH_SIGN) { if (isfinite(db)) { char buf[256]; snprintf(buf, sizeof(buf), "%+.1f", db); text = buf; } else if (db < 0.0) { text = "-∞"; } else { // Should never happen, really. text = "+∞"; } } else { if (isfinite(db)) { char buf[256]; snprintf(buf, sizeof(buf), "%.1f", db); text = buf; } else if (db < 0.0) { text = "-∞"; } else { // Should never happen, really. text = "∞"; } } if (!(flags & DB_BARE)) { text += " dB"; } return text; } void set_peak_label(QLabel *peak_label, float peak_db) { peak_label->setText(QString::fromStdString(format_db(peak_db, DB_BARE))); if (peak_db > peak_limit_dbfs) { peak_label->setStyleSheet("QLabel { background-color: red; color: white; }"); } else { peak_label->setStyleSheet(""); } } } // namespace MainWindow::MainWindow() : ui(new Ui::MainWindow), midi_mapper(this) { global_mainwindow = this; ui->setupUi(this); global_disk_space_estimator = new DiskSpaceEstimator(bind(&MainWindow::report_disk_space, this, _1, _2)); disk_free_label = new QLabel(this); disk_free_label->setStyleSheet("QLabel {padding-right: 5px;}"); ui->menuBar->setCornerWidget(disk_free_label); QActionGroup *audio_mapping_group = new QActionGroup(this); ui->simple_audio_mode->setActionGroup(audio_mapping_group); ui->multichannel_audio_mode->setActionGroup(audio_mapping_group); ui->me_live->set_output(Mixer::OUTPUT_LIVE); ui->me_preview->set_output(Mixer::OUTPUT_PREVIEW); // The menus. connect(ui->cut_action, &QAction::triggered, this, &MainWindow::cut_triggered); connect(ui->exit_action, &QAction::triggered, this, &MainWindow::exit_triggered); connect(ui->about_action, &QAction::triggered, this, &MainWindow::about_triggered); connect(ui->simple_audio_mode, &QAction::triggered, this, &MainWindow::simple_audio_mode_triggered); connect(ui->multichannel_audio_mode, &QAction::triggered, this, &MainWindow::multichannel_audio_mode_triggered); connect(ui->input_mapping_action, &QAction::triggered, this, &MainWindow::input_mapping_triggered); connect(ui->midi_mapping_action, &QAction::triggered, this, &MainWindow::midi_mapping_triggered); if (global_flags.x264_video_to_http) { connect(ui->x264_bitrate_action, &QAction::triggered, this, &MainWindow::x264_bitrate_triggered); } else { ui->x264_bitrate_action->setEnabled(false); } // Hook up the transition buttons. (Keyboard shortcuts are set in set_transition_names().) // TODO: Make them dynamic. connect(ui->transition_btn1, &QPushButton::clicked, bind(&MainWindow::transition_clicked, this, 0)); connect(ui->transition_btn2, &QPushButton::clicked, bind(&MainWindow::transition_clicked, this, 1)); connect(ui->transition_btn3, &QPushButton::clicked, bind(&MainWindow::transition_clicked, this, 2)); // Aiee... transition_btn1 = ui->transition_btn1; transition_btn2 = ui->transition_btn2; transition_btn3 = ui->transition_btn3; qRegisterMetaType("std::string"); qRegisterMetaType>("std::vector"); connect(ui->me_live, &GLWidget::transition_names_updated, this, &MainWindow::set_transition_names); qRegisterMetaType("Mixer::Output"); // Hook up the prev/next buttons on the audio views. connect(ui->compact_prev_page, &QAbstractButton::clicked, bind(&QStackedWidget::setCurrentIndex, ui->audio_views, 1)); connect(ui->compact_next_page, &QAbstractButton::clicked, bind(&QStackedWidget::setCurrentIndex, ui->audio_views, 1)); connect(ui->full_prev_page, &QAbstractButton::clicked, bind(&QStackedWidget::setCurrentIndex, ui->audio_views, 0)); connect(ui->full_next_page, &QAbstractButton::clicked, bind(&QStackedWidget::setCurrentIndex, ui->audio_views, 0)); // And bind the same to PgUp/PgDown. auto switch_page = [this]{ if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL) { ui->audio_views->setCurrentIndex(1 - ui->audio_views->currentIndex()); } }; connect(new QShortcut(QKeySequence::MoveToNextPage, this), &QShortcut::activated, switch_page); connect(new QShortcut(QKeySequence::MoveToPreviousPage, this), &QShortcut::activated, switch_page); last_audio_level_callback = steady_clock::now() - seconds(1); if (!global_flags.midi_mapping_filename.empty()) { MIDIMappingProto midi_mapping; if (!load_midi_mapping_from_file(global_flags.midi_mapping_filename, &midi_mapping)) { fprintf(stderr, "Couldn't load MIDI mapping '%s'; exiting.\n", global_flags.midi_mapping_filename.c_str()); exit(1); } midi_mapper.set_midi_mapping(midi_mapping); } midi_mapper.refresh_highlights(); midi_mapper.refresh_lights(); } void MainWindow::resizeEvent(QResizeEvent* event) { QMainWindow::resizeEvent(event); // Ask for a relayout, but only after the event loop is done doing relayout // on everything else. QMetaObject::invokeMethod(this, "relayout", Qt::QueuedConnection); } void MainWindow::mixer_created(Mixer *mixer) { // Make the previews. unsigned num_previews = mixer->get_num_channels(); for (unsigned i = 0; i < num_previews; ++i) { Mixer::Output output = Mixer::Output(Mixer::OUTPUT_INPUT0 + i); QWidget *preview = new QWidget(this); Ui::Display *ui_display = new Ui::Display; ui_display->setupUi(preview); ui_display->label->setText(mixer->get_channel_name(output).c_str()); ui_display->display->set_output(output); ui->preview_displays->insertWidget(previews.size(), preview, 1); previews.push_back(ui_display); // Hook up the click. connect(ui_display->display, &GLWidget::clicked, bind(&MainWindow::channel_clicked, this, i)); // Let the theme update the text whenever the resolution or color changed. connect(ui_display->display, &GLWidget::name_updated, this, &MainWindow::update_channel_name); connect(ui_display->display, &GLWidget::color_updated, this, &MainWindow::update_channel_color); // Hook up the keyboard key. QShortcut *shortcut = new QShortcut(QKeySequence(Qt::Key_1 + i), this); connect(shortcut, &QShortcut::activated, bind(&MainWindow::channel_clicked, this, i)); // Hook up the white balance button (irrelevant if invisible). ui_display->wb_button->setVisible(mixer->get_supports_set_wb(output)); connect(ui_display->wb_button, &QPushButton::clicked, bind(&MainWindow::wb_button_clicked, this, i)); } global_audio_mixer->set_state_changed_callback(bind(&MainWindow::audio_state_changed, this)); slave_knob(ui->locut_cutoff_knob, ui->locut_cutoff_knob_2); slave_knob(ui->limiter_threshold_knob, ui->limiter_threshold_knob_2); slave_knob(ui->makeup_gain_knob, ui->makeup_gain_knob_2); slave_checkbox(ui->makeup_gain_auto_checkbox, ui->makeup_gain_auto_checkbox_2); slave_checkbox(ui->limiter_enabled, ui->limiter_enabled_2); reset_audio_mapping_ui(); // TODO: Fetch all of the values these for completeness, // not just the enable knobs implied by flags. ui->limiter_enabled->setChecked(global_audio_mixer->get_limiter_enabled()); ui->makeup_gain_auto_checkbox->setChecked(global_audio_mixer->get_final_makeup_gain_auto()); // Controls used only for simple audio fetch their state from the first bus. constexpr unsigned simple_bus_index = 0; if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { ui->locut_enabled->setChecked(global_audio_mixer->get_locut_enabled(simple_bus_index)); ui->gainstaging_knob->setValue(global_audio_mixer->get_gain_staging_db(simple_bus_index)); ui->gainstaging_auto_checkbox->setChecked(global_audio_mixer->get_gain_staging_auto(simple_bus_index)); ui->compressor_enabled->setChecked(global_audio_mixer->get_compressor_enabled(simple_bus_index)); ui->compressor_threshold_db_display->setText( QString::fromStdString(format_db(mixer->get_audio_mixer()->get_compressor_threshold_dbfs(simple_bus_index), DB_WITH_SIGN))); } connect(ui->locut_enabled, &QCheckBox::stateChanged, [this](int state){ global_audio_mixer->set_locut_enabled(simple_bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui->gainstaging_knob, &QAbstractSlider::valueChanged, bind(&MainWindow::gain_staging_knob_changed, this, simple_bus_index, _1)); connect(ui->gainstaging_auto_checkbox, &QCheckBox::stateChanged, [this, simple_bus_index](int state){ global_audio_mixer->set_gain_staging_auto(simple_bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui->compressor_threshold_knob, &QDial::valueChanged, bind(&MainWindow::compressor_threshold_knob_changed, this, simple_bus_index, _1)); connect(ui->compressor_enabled, &QCheckBox::stateChanged, [this, simple_bus_index](int state){ global_audio_mixer->set_compressor_enabled(simple_bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); // Global mastering controls. QString limiter_threshold_label( QString::fromStdString(format_db(mixer->get_audio_mixer()->get_limiter_threshold_dbfs(), DB_WITH_SIGN))); ui->limiter_threshold_db_display->setText(limiter_threshold_label); ui->limiter_threshold_db_display_2->setText(limiter_threshold_label); connect(ui->locut_cutoff_knob, &QDial::valueChanged, this, &MainWindow::cutoff_knob_changed); cutoff_knob_changed(ui->locut_cutoff_knob->value()); connect(ui->makeup_gain_knob, &QAbstractSlider::valueChanged, this, &MainWindow::final_makeup_gain_knob_changed); connect(ui->makeup_gain_auto_checkbox, &QCheckBox::stateChanged, [this](int state){ global_audio_mixer->set_final_makeup_gain_auto(state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui->limiter_threshold_knob, &QDial::valueChanged, this, &MainWindow::limiter_threshold_knob_changed); connect(ui->limiter_enabled, &QCheckBox::stateChanged, [this](int state){ global_audio_mixer->set_limiter_enabled(state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui->reset_meters_button, &QPushButton::clicked, this, &MainWindow::reset_meters_button_clicked); // Even though we have a reset button right next to it, the fact that // the expanded audio view labels are clickable makes it natural to // click this one as well. connect(ui->peak_display, &ClickableLabel::clicked, this, &MainWindow::reset_meters_button_clicked); mixer->get_audio_mixer()->set_audio_level_callback(bind(&MainWindow::audio_level_callback, this, _1, _2, _3, _4, _5, _6, _7, _8)); midi_mapper.refresh_highlights(); midi_mapper.refresh_lights(); midi_mapper.start_thread(); struct sigaction act; memset(&act, 0, sizeof(act)); act.sa_handler = schedule_cut_signal; act.sa_flags = SA_RESTART; sigaction(SIGHUP, &act, nullptr); // Mostly for debugging. Don't override SIGINT, that's so evil if // shutdown isn't instant. memset(&act, 0, sizeof(act)); act.sa_handler = quit_signal; act.sa_flags = SA_RESTART; sigaction(SIGUSR1, &act, nullptr); } void MainWindow::reset_audio_mapping_ui() { bool simple = (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE); ui->simple_audio_mode->setChecked(simple); ui->multichannel_audio_mode->setChecked(!simple); ui->input_mapping_action->setEnabled(!simple); ui->midi_mapping_action->setEnabled(!simple); ui->locut_enabled->setVisible(simple); ui->gainstaging_label->setVisible(simple); ui->gainstaging_knob->setVisible(simple); ui->gainstaging_db_display->setVisible(simple); ui->gainstaging_auto_checkbox->setVisible(simple); ui->compressor_threshold_label->setVisible(simple); ui->compressor_threshold_knob->setVisible(simple); ui->compressor_threshold_db_display->setVisible(simple); ui->compressor_enabled->setVisible(simple); setup_audio_miniview(); setup_audio_expanded_view(); if (simple) { ui->audio_views->setCurrentIndex(0); } ui->compact_header->setVisible(!simple); midi_mapper.refresh_highlights(); midi_mapper.refresh_lights(); } void MainWindow::setup_audio_miniview() { // Remove any existing channels. for (QLayoutItem *item; (item = ui->faders->takeAt(0)) != nullptr; ) { delete item->widget(); delete item; } audio_miniviews.clear(); if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { return; } // Set up brand new ones from the input mapping. InputMapping mapping = global_audio_mixer->get_input_mapping(); audio_miniviews.resize(mapping.buses.size()); for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { QWidget *channel = new QWidget(this); Ui::AudioMiniView *ui_audio_miniview = new Ui::AudioMiniView; ui_audio_miniview->setupUi(channel); ui_audio_miniview->bus_desc_label->setFullText( QString::fromStdString(mapping.buses[bus_index].name)); audio_miniviews[bus_index] = ui_audio_miniview; // Set up the peak meter. VUMeter *peak_meter = ui_audio_miniview->peak_meter; peak_meter->set_min_level(-30.0f); peak_meter->set_max_level(0.0f); peak_meter->set_ref_level(0.0f); ui_audio_miniview->fader->setDbValue(global_audio_mixer->get_fader_volume(bus_index)); ui->faders->addWidget(channel); connect(ui_audio_miniview->fader, &NonLinearFader::dbValueChanged, bind(&MainWindow::mini_fader_changed, this, bus_index, _1)); connect(ui_audio_miniview->peak_display_label, &ClickableLabel::clicked, [bus_index]() { global_audio_mixer->reset_peak(bus_index); }); } } void MainWindow::setup_audio_expanded_view() { // Remove any existing channels. for (QLayoutItem *item; (item = ui->buses->takeAt(0)) != nullptr; ) { delete item->widget(); delete item; } audio_expanded_views.clear(); if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { return; } // Set up brand new ones from the input mapping. InputMapping mapping = global_audio_mixer->get_input_mapping(); audio_expanded_views.resize(mapping.buses.size()); for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { QWidget *channel = new QWidget(this); Ui::AudioExpandedView *ui_audio_expanded_view = new Ui::AudioExpandedView; ui_audio_expanded_view->setupUi(channel); ui_audio_expanded_view->bus_desc_label->setFullText( QString::fromStdString(mapping.buses[bus_index].name)); audio_expanded_views[bus_index] = ui_audio_expanded_view; update_eq_label(bus_index, EQ_BAND_TREBLE, global_audio_mixer->get_eq(bus_index, EQ_BAND_TREBLE)); update_eq_label(bus_index, EQ_BAND_MID, global_audio_mixer->get_eq(bus_index, EQ_BAND_MID)); update_eq_label(bus_index, EQ_BAND_BASS, global_audio_mixer->get_eq(bus_index, EQ_BAND_BASS)); ui_audio_expanded_view->fader->setDbValue(global_audio_mixer->get_fader_volume(bus_index)); ui_audio_expanded_view->mute_button->setChecked(global_audio_mixer->get_mute(bus_index) ? Qt::Checked : Qt::Unchecked); connect(ui_audio_expanded_view->mute_button, &QPushButton::toggled, bind(&MainWindow::mute_button_toggled, this, bus_index, _1)); ui->buses->addWidget(channel); ui_audio_expanded_view->locut_enabled->setChecked(global_audio_mixer->get_locut_enabled(bus_index)); connect(ui_audio_expanded_view->locut_enabled, &QCheckBox::stateChanged, [this, bus_index](int state){ global_audio_mixer->set_locut_enabled(bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui_audio_expanded_view->treble_knob, &QDial::valueChanged, bind(&MainWindow::eq_knob_changed, this, bus_index, EQ_BAND_TREBLE, _1)); connect(ui_audio_expanded_view->mid_knob, &QDial::valueChanged, bind(&MainWindow::eq_knob_changed, this, bus_index, EQ_BAND_MID, _1)); connect(ui_audio_expanded_view->bass_knob, &QDial::valueChanged, bind(&MainWindow::eq_knob_changed, this, bus_index, EQ_BAND_BASS, _1)); ui_audio_expanded_view->gainstaging_knob->setValue(global_audio_mixer->get_gain_staging_db(bus_index)); ui_audio_expanded_view->gainstaging_auto_checkbox->setChecked(global_audio_mixer->get_gain_staging_auto(bus_index)); ui_audio_expanded_view->compressor_enabled->setChecked(global_audio_mixer->get_compressor_enabled(bus_index)); connect(ui_audio_expanded_view->gainstaging_knob, &QAbstractSlider::valueChanged, bind(&MainWindow::gain_staging_knob_changed, this, bus_index, _1)); connect(ui_audio_expanded_view->gainstaging_auto_checkbox, &QCheckBox::stateChanged, [this, bus_index](int state){ global_audio_mixer->set_gain_staging_auto(bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); connect(ui_audio_expanded_view->compressor_threshold_knob, &QDial::valueChanged, bind(&MainWindow::compressor_threshold_knob_changed, this, bus_index, _1)); connect(ui_audio_expanded_view->compressor_enabled, &QCheckBox::stateChanged, [this, bus_index](int state){ global_audio_mixer->set_compressor_enabled(bus_index, state == Qt::Checked); midi_mapper.refresh_lights(); }); slave_fader(audio_miniviews[bus_index]->fader, ui_audio_expanded_view->fader); // Set up the peak meter. VUMeter *peak_meter = ui_audio_expanded_view->peak_meter; peak_meter->set_min_level(-30.0f); peak_meter->set_max_level(0.0f); peak_meter->set_ref_level(0.0f); connect(ui_audio_expanded_view->peak_display_label, &ClickableLabel::clicked, [this, bus_index]() { global_audio_mixer->reset_peak(bus_index); midi_mapper.refresh_lights(); }); } update_cutoff_labels(global_audio_mixer->get_locut_cutoff()); } void MainWindow::mixer_shutting_down() { ui->me_live->clean_context(); ui->me_preview->clean_context(); for (Ui::Display *display : previews) { display->display->clean_context(); } } void MainWindow::cut_triggered() { global_mixer->schedule_cut(); } void MainWindow::x264_bitrate_triggered() { bool ok; int new_bitrate = QInputDialog::getInt(this, "Change x264 bitrate", "Choose new bitrate for x264 HTTP output (from 100–100,000 kbit/sec):", global_flags.x264_bitrate, /*min=*/100, /*max=*/100000, /*step=*/100, &ok); if (ok && new_bitrate >= 100 && new_bitrate <= 100000) { global_flags.x264_bitrate = new_bitrate; global_mixer->change_x264_bitrate(new_bitrate); } } void MainWindow::exit_triggered() { close(); } void MainWindow::about_triggered() { AboutDialog().exec(); } void MainWindow::simple_audio_mode_triggered() { if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { return; } unsigned card_index = global_audio_mixer->get_simple_input(); if (card_index == numeric_limits::max()) { QMessageBox::StandardButton reply = QMessageBox::question(this, "Mapping too complex", "The current audio mapping is too complicated to be representable in simple mode, " "and will be discarded if you proceed. Really go to simple audio mode?", QMessageBox::Yes | QMessageBox::No); if (reply == QMessageBox::No) { ui->simple_audio_mode->setChecked(false); ui->multichannel_audio_mode->setChecked(true); return; } card_index = 0; } global_audio_mixer->set_simple_input(/*card_index=*/card_index); reset_audio_mapping_ui(); } void MainWindow::multichannel_audio_mode_triggered() { if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL) { return; } // Take the generated input mapping from the simple input, // and set it as a normal multichannel mapping, which causes // the mode to go to multichannel. global_audio_mixer->set_input_mapping(global_audio_mixer->get_input_mapping()); reset_audio_mapping_ui(); } void MainWindow::input_mapping_triggered() { if (InputMappingDialog().exec() == QDialog::Accepted) { setup_audio_miniview(); setup_audio_expanded_view(); } midi_mapper.refresh_highlights(); midi_mapper.refresh_lights(); } void MainWindow::midi_mapping_triggered() { MIDIMappingDialog(&midi_mapper).exec(); } void MainWindow::gain_staging_knob_changed(unsigned bus_index, int value) { if (bus_index == 0) { ui->gainstaging_auto_checkbox->setCheckState(Qt::Unchecked); } if (bus_index < audio_expanded_views.size()) { audio_expanded_views[bus_index]->gainstaging_auto_checkbox->setCheckState(Qt::Unchecked); } float gain_db = value * 0.1f; global_audio_mixer->set_gain_staging_db(bus_index, gain_db); // The label will be updated by the audio level callback. } void MainWindow::final_makeup_gain_knob_changed(int value) { ui->makeup_gain_auto_checkbox->setCheckState(Qt::Unchecked); float gain_db = value * 0.1f; global_audio_mixer->set_final_makeup_gain_db(gain_db); // The label will be updated by the audio level callback. } void MainWindow::cutoff_knob_changed(int value) { float octaves = value * 0.1f; float cutoff_hz = 20.0 * pow(2.0, octaves); global_audio_mixer->set_locut_cutoff(cutoff_hz); update_cutoff_labels(cutoff_hz); } void MainWindow::update_cutoff_labels(float cutoff_hz) { char buf[256]; snprintf(buf, sizeof(buf), "%ld Hz", lrintf(cutoff_hz)); ui->locut_cutoff_display->setText(buf); ui->locut_cutoff_display_2->setText(buf); for (unsigned bus_index = 0; bus_index < audio_expanded_views.size(); ++bus_index) { audio_expanded_views[bus_index]->locut_enabled->setText( QString("Lo-cut: ") + buf); } } void MainWindow::report_disk_space(off_t free_bytes, double estimated_seconds_left) { char time_str[256]; if (estimated_seconds_left < 60.0) { strcpy(time_str, "Less than a minute"); } else if (estimated_seconds_left < 1800.0) { // Less than half an hour: Xm Ys (red). int s = lrintf(estimated_seconds_left); int m = s / 60; s %= 60; snprintf(time_str, sizeof(time_str), "%dm %ds", m, s); } else if (estimated_seconds_left < 3600.0) { // Less than an hour: Xm. int m = lrintf(estimated_seconds_left / 60.0); snprintf(time_str, sizeof(time_str), "%dm", m); } else if (estimated_seconds_left < 36000.0) { // Less than ten hours: Xh Ym. int m = lrintf(estimated_seconds_left / 60.0); int h = m / 60; m %= 60; snprintf(time_str, sizeof(time_str), "%dh %dm", h, m); } else { // More than ten hours: Xh. int h = lrintf(estimated_seconds_left / 3600.0); snprintf(time_str, sizeof(time_str), "%dh", h); } char buf[256]; snprintf(buf, sizeof(buf), "Disk free: %'.0f MB (approx. %s)", free_bytes / 1048576.0, time_str); std::string label = buf; post_to_main_thread([this, label]{ disk_free_label->setText(QString::fromStdString(label)); ui->menuBar->setCornerWidget(disk_free_label); // Need to set this again for the sizing to get right. }); } void MainWindow::eq_knob_changed(unsigned bus_index, EQBand band, int value) { float gain_db = value * 0.1f; global_audio_mixer->set_eq(bus_index, band, gain_db); update_eq_label(bus_index, band, gain_db); } void MainWindow::update_eq_label(unsigned bus_index, EQBand band, float gain_db) { Ui::AudioExpandedView *view = audio_expanded_views[bus_index]; string db_string = format_db(gain_db, DB_WITH_SIGN); switch (band) { case EQ_BAND_TREBLE: view->treble_label->setText(QString::fromStdString("Treble: " + db_string)); break; case EQ_BAND_MID: view->mid_label->setText(QString::fromStdString("Mid: " + db_string)); break; case EQ_BAND_BASS: view->bass_label->setText(QString::fromStdString("Bass: " + db_string)); break; default: assert(false); } } void MainWindow::limiter_threshold_knob_changed(int value) { float threshold_dbfs = value * 0.1f; global_audio_mixer->set_limiter_threshold_dbfs(threshold_dbfs); ui->limiter_threshold_db_display->setText( QString::fromStdString(format_db(threshold_dbfs, DB_WITH_SIGN))); ui->limiter_threshold_db_display_2->setText( QString::fromStdString(format_db(threshold_dbfs, DB_WITH_SIGN))); } void MainWindow::compressor_threshold_knob_changed(unsigned bus_index, int value) { float threshold_dbfs = value * 0.1f; global_audio_mixer->set_compressor_threshold_dbfs(bus_index, threshold_dbfs); QString label(QString::fromStdString(format_db(threshold_dbfs, DB_WITH_SIGN))); if (bus_index == 0) { ui->compressor_threshold_db_display->setText(label); } if (bus_index < audio_expanded_views.size()) { audio_expanded_views[bus_index]->compressor_threshold_db_display->setText(label); } } void MainWindow::mini_fader_changed(int bus, double volume_db) { QString label(QString::fromStdString(format_db(volume_db, DB_WITH_SIGN))); audio_miniviews[bus]->fader_label->setText(label); audio_expanded_views[bus]->fader_label->setText(label); global_audio_mixer->set_fader_volume(bus, volume_db); } void MainWindow::mute_button_toggled(int bus, bool checked) { global_audio_mixer->set_mute(bus, checked); midi_mapper.refresh_lights(); } void MainWindow::reset_meters_button_clicked() { global_audio_mixer->reset_meters(); ui->peak_display->setText(QString::fromStdString(format_db(-HUGE_VAL, DB_WITH_SIGN | DB_BARE))); ui->peak_display->setStyleSheet(""); } void MainWindow::audio_level_callback(float level_lufs, float peak_db, vector bus_levels, float global_level_lufs, float range_low_lufs, float range_high_lufs, float final_makeup_gain_db, float correlation) { steady_clock::time_point now = steady_clock::now(); // The meters are somewhat inefficient to update. Only update them // every 100 ms or so (we get updates every 5–20 ms). Note that this // means that the digital peak meters are ever so slightly too low // (each update won't be a faithful representation of the highest peak // since the previous update, since there are frames we won't draw), // but the _peak_ of the peak meters will be correct (it's tracked in // AudioMixer, not here), and that's much more important. double last_update_age = duration(now - last_audio_level_callback).count(); if (last_update_age < 0.100) { return; } last_audio_level_callback = now; post_to_main_thread([=]() { ui->vu_meter->set_level(level_lufs); for (unsigned bus_index = 0; bus_index < bus_levels.size(); ++bus_index) { if (bus_index < audio_miniviews.size()) { const AudioMixer::BusLevel &level = bus_levels[bus_index]; Ui::AudioMiniView *miniview = audio_miniviews[bus_index]; miniview->peak_meter->set_level( level.current_level_dbfs[0], level.current_level_dbfs[1]); miniview->peak_meter->set_peak( level.peak_level_dbfs[0], level.peak_level_dbfs[1]); set_peak_label(miniview->peak_display_label, level.historic_peak_dbfs); Ui::AudioExpandedView *view = audio_expanded_views[bus_index]; view->peak_meter->set_level( level.current_level_dbfs[0], level.current_level_dbfs[1]); view->peak_meter->set_peak( level.peak_level_dbfs[0], level.peak_level_dbfs[1]); view->reduction_meter->set_reduction_db(level.compressor_attenuation_db); view->gainstaging_knob->blockSignals(true); view->gainstaging_knob->setValue(lrintf(level.gain_staging_db * 10.0f)); view->gainstaging_knob->blockSignals(false); view->gainstaging_db_display->setText( QString("Gain: ") + QString::fromStdString(format_db(level.gain_staging_db, DB_WITH_SIGN))); set_peak_label(view->peak_display_label, level.historic_peak_dbfs); midi_mapper.set_has_peaked(bus_index, level.historic_peak_dbfs >= -0.1f); } } ui->lra_meter->set_levels(global_level_lufs, range_low_lufs, range_high_lufs); ui->correlation_meter->set_correlation(correlation); ui->peak_display->setText(QString::fromStdString(format_db(peak_db, DB_BARE))); set_peak_label(ui->peak_display, peak_db); // NOTE: Will be invisible when using multitrack audio. ui->gainstaging_knob->blockSignals(true); ui->gainstaging_knob->setValue(lrintf(bus_levels[0].gain_staging_db * 10.0f)); ui->gainstaging_knob->blockSignals(false); ui->gainstaging_db_display->setText( QString::fromStdString(format_db(bus_levels[0].gain_staging_db, DB_WITH_SIGN))); ui->makeup_gain_knob->blockSignals(true); ui->makeup_gain_knob->setValue(lrintf(final_makeup_gain_db * 10.0f)); ui->makeup_gain_knob->blockSignals(false); ui->makeup_gain_db_display->setText( QString::fromStdString(format_db(final_makeup_gain_db, DB_WITH_SIGN))); ui->makeup_gain_db_display_2->setText( QString::fromStdString(format_db(final_makeup_gain_db, DB_WITH_SIGN))); // Peak labels could have changed. midi_mapper.refresh_lights(); }); } void MainWindow::relayout() { int height = ui->vertical_layout->geometry().height(); double remaining_height = height; // Allocate the height; the most important part is to keep the main displays // at 16:9 if at all possible. double me_width = ui->me_preview->width(); double me_height = me_width * 9.0 / 16.0 + ui->label_preview->height() + ui->preview_vertical_layout->spacing(); // TODO: Scale the widths when we need to do this. if (me_height / double(height) > 0.8) { me_height = height * 0.8; } remaining_height -= me_height + ui->vertical_layout->spacing(); // Space between the M/E displays and the audio strip. remaining_height -= ui->vertical_layout->spacing(); // The label above the audio strip. double compact_label_height = ui->compact_label->minimumHeight() + ui->compact_audio_layout->spacing(); remaining_height -= compact_label_height; // The previews will be constrained by the remaining height, and the width. double preview_label_height = previews[0]->title_bar->geometry().height() + previews[0]->main_vertical_layout->spacing(); int preview_total_width = ui->preview_displays->geometry().width() - (previews.size() - 1) * ui->preview_displays->spacing(); double preview_height = min(remaining_height - preview_label_height, (preview_total_width / double(previews.size())) * 9.0 / 16.0); remaining_height -= preview_height + preview_label_height + ui->vertical_layout->spacing(); ui->vertical_layout->setStretch(0, lrintf(me_height)); ui->vertical_layout->setStretch(1, lrintf(compact_label_height) + lrintf(remaining_height) + lrintf(preview_height + preview_label_height)); // Audio strip and previews together. ui->compact_audio_layout->setStretch(0, lrintf(compact_label_height)); ui->compact_audio_layout->setStretch(1, lrintf(remaining_height)); // Audio strip. ui->compact_audio_layout->setStretch(2, lrintf(preview_height + preview_label_height)); // Set the widths for the previews. double preview_width = preview_height * 16.0 / 9.0; for (unsigned i = 0; i < previews.size(); ++i) { ui->preview_displays->setStretch(i, lrintf(preview_width)); } // The preview horizontal spacer. double remaining_preview_width = preview_total_width - previews.size() * preview_width; ui->preview_displays->setStretch(previews.size(), lrintf(remaining_preview_width)); } void MainWindow::set_locut(float value) { set_relative_value(ui->locut_cutoff_knob, value); } void MainWindow::set_limiter_threshold(float value) { set_relative_value(ui->limiter_threshold_knob, value); } void MainWindow::set_makeup_gain(float value) { set_relative_value(ui->makeup_gain_knob, value); } void MainWindow::set_treble(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::treble_knob, value); } void MainWindow::set_mid(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::mid_knob, value); } void MainWindow::set_bass(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::bass_knob, value); } void MainWindow::set_gain(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::gainstaging_knob, value); } void MainWindow::set_compressor_threshold(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::compressor_threshold_knob, value); } void MainWindow::set_fader(unsigned bus_idx, float value) { set_relative_value_if_exists(bus_idx, &Ui::AudioExpandedView::fader, value); } void MainWindow::toggle_mute(unsigned bus_idx) { click_button_if_exists(bus_idx, &Ui::AudioExpandedView::mute_button); } void MainWindow::toggle_locut(unsigned bus_idx) { click_button_if_exists(bus_idx, &Ui::AudioExpandedView::locut_enabled); } void MainWindow::toggle_auto_gain_staging(unsigned bus_idx) { click_button_if_exists(bus_idx, &Ui::AudioExpandedView::gainstaging_auto_checkbox); } void MainWindow::toggle_compressor(unsigned bus_idx) { click_button_if_exists(bus_idx, &Ui::AudioExpandedView::compressor_enabled); } void MainWindow::clear_peak(unsigned bus_idx) { post_to_main_thread([=]{ if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL) { global_audio_mixer->reset_peak(bus_idx); midi_mapper.set_has_peaked(bus_idx, false); midi_mapper.refresh_lights(); } }); } void MainWindow::clear_all_highlights() { post_to_main_thread([this]{ highlight_locut(false); highlight_limiter_threshold(false); highlight_makeup_gain(false); highlight_toggle_limiter(false); highlight_toggle_auto_makeup_gain(false); for (unsigned bus_idx = 0; bus_idx < audio_expanded_views.size(); ++bus_idx) { highlight_treble(bus_idx, false); highlight_mid(bus_idx, false); highlight_bass(bus_idx, false); highlight_gain(bus_idx, false); highlight_compressor_threshold(bus_idx, false); highlight_fader(bus_idx, false); highlight_mute(bus_idx, false); highlight_toggle_locut(bus_idx, false); highlight_toggle_auto_gain_staging(bus_idx, false); highlight_toggle_compressor(bus_idx, false); } }); } void MainWindow::toggle_limiter() { if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL) { ui->limiter_enabled->click(); } } void MainWindow::toggle_auto_makeup_gain() { if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL) { ui->makeup_gain_auto_checkbox->click(); } } void MainWindow::highlight_locut(bool highlight) { post_to_main_thread([this, highlight]{ highlight_control(ui->locut_cutoff_knob, highlight); highlight_control(ui->locut_cutoff_knob_2, highlight); }); } void MainWindow::highlight_limiter_threshold(bool highlight) { post_to_main_thread([this, highlight]{ highlight_control(ui->limiter_threshold_knob, highlight); highlight_control(ui->limiter_threshold_knob_2, highlight); }); } void MainWindow::highlight_makeup_gain(bool highlight) { post_to_main_thread([this, highlight]{ highlight_control(ui->makeup_gain_knob, highlight); highlight_control(ui->makeup_gain_knob_2, highlight); }); } void MainWindow::highlight_treble(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::treble_knob, highlight); } void MainWindow::highlight_mid(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::mid_knob, highlight); } void MainWindow::highlight_bass(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::bass_knob, highlight); } void MainWindow::highlight_gain(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::gainstaging_knob, highlight); } void MainWindow::highlight_compressor_threshold(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::compressor_threshold_knob, highlight); } void MainWindow::highlight_fader(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::fader, highlight); } void MainWindow::highlight_mute(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::mute_button, highlight, /*is_mute_btton=*/true); } void MainWindow::highlight_toggle_locut(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::locut_enabled, highlight); } void MainWindow::highlight_toggle_auto_gain_staging(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::gainstaging_auto_checkbox, highlight); } void MainWindow::highlight_toggle_compressor(unsigned bus_idx, bool highlight) { highlight_control_if_exists(bus_idx, &Ui::AudioExpandedView::compressor_enabled, highlight); } void MainWindow::highlight_toggle_limiter(bool highlight) { post_to_main_thread([this, highlight]{ highlight_control(ui->limiter_enabled, highlight); highlight_control(ui->limiter_enabled_2, highlight); }); } void MainWindow::highlight_toggle_auto_makeup_gain(bool highlight) { post_to_main_thread([this, highlight]{ highlight_control(ui->makeup_gain_auto_checkbox, highlight); highlight_control(ui->makeup_gain_auto_checkbox_2, highlight); }); } template void MainWindow::set_relative_value(T *control, float value) { post_to_main_thread([control, value]{ control->setValue(lrintf(control->minimum() + value * (control->maximum() - control->minimum()))); }); } template void MainWindow::set_relative_value_if_exists(unsigned bus_idx, T *(Ui_AudioExpandedView::*control), float value) { if (global_audio_mixer != nullptr && global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL && bus_idx < audio_expanded_views.size()) { set_relative_value(audio_expanded_views[bus_idx]->*control, value); } } template void MainWindow::click_button_if_exists(unsigned bus_idx, T *(Ui_AudioExpandedView::*control)) { post_to_main_thread([this, bus_idx, control]{ if (global_audio_mixer != nullptr && global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::MULTICHANNEL && bus_idx < audio_expanded_views.size()) { (audio_expanded_views[bus_idx]->*control)->click(); } }); } template void MainWindow::highlight_control(T *control, bool highlight) { if (control == nullptr) { return; } if (global_audio_mixer == nullptr || global_audio_mixer->get_mapping_mode() != AudioMixer::MappingMode::MULTICHANNEL) { highlight = false; } if (highlight) { control->setStyleSheet("background: rgb(0,255,0,80)"); } else { control->setStyleSheet(""); } } template void MainWindow::highlight_mute_control(T *control, bool highlight) { if (control == nullptr) { return; } if (global_audio_mixer == nullptr || global_audio_mixer->get_mapping_mode() != AudioMixer::MappingMode::MULTICHANNEL) { highlight = false; } if (highlight) { control->setStyleSheet("QPushButton { background: rgb(0,255,0,80); } QPushButton:checked { background: rgba(255,80,0,140); }"); } else { control->setStyleSheet("QPushButton:checked { background: rgba(255,0,0,80); }"); } } template void MainWindow::highlight_control_if_exists(unsigned bus_idx, T *(Ui_AudioExpandedView::*control), bool highlight, bool is_mute_button) { post_to_main_thread([this, bus_idx, control, highlight, is_mute_button]{ if (bus_idx < audio_expanded_views.size()) { if (is_mute_button) { highlight_mute_control(audio_expanded_views[bus_idx]->*control, highlight); } else { highlight_control(audio_expanded_views[bus_idx]->*control, highlight); } } }); } void MainWindow::set_transition_names(vector transition_names) { if (transition_names.size() < 1 || transition_names[0].empty()) { transition_btn1->setText(QString("")); } else { transition_btn1->setText(QString::fromStdString(transition_names[0] + " (J)")); ui->transition_btn1->setShortcut(QKeySequence("J")); } if (transition_names.size() < 2 || transition_names[1].empty()) { transition_btn2->setText(QString("")); } else { transition_btn2->setText(QString::fromStdString(transition_names[1] + " (K)")); ui->transition_btn2->setShortcut(QKeySequence("K")); } if (transition_names.size() < 3 || transition_names[2].empty()) { transition_btn3->setText(QString("")); } else { transition_btn3->setText(QString::fromStdString(transition_names[2] + " (L)")); ui->transition_btn3->setShortcut(QKeySequence("L")); } } void MainWindow::update_channel_name(Mixer::Output output, const string &name) { if (output >= Mixer::OUTPUT_INPUT0) { unsigned channel = output - Mixer::OUTPUT_INPUT0; previews[channel]->label->setText(name.c_str()); } } void MainWindow::update_channel_color(Mixer::Output output, const string &color) { if (output >= Mixer::OUTPUT_INPUT0) { unsigned channel = output - Mixer::OUTPUT_INPUT0; previews[channel]->frame->setStyleSheet(QString::fromStdString("background-color:" + color)); } } void MainWindow::transition_clicked(int transition_number) { global_mixer->transition_clicked(transition_number); } void MainWindow::channel_clicked(int channel_number) { if (current_wb_pick_display == channel_number) { // The picking was already done from eventFilter(), since we don't get // the mouse pointer here. } else { global_mixer->channel_clicked(channel_number); } } void MainWindow::wb_button_clicked(int channel_number) { current_wb_pick_display = channel_number; QApplication::setOverrideCursor(Qt::CrossCursor); } bool MainWindow::eventFilter(QObject *watched, QEvent *event) { if (current_wb_pick_display != -1 && event->type() == QEvent::MouseButtonRelease && watched->isWidgetType()) { QApplication::restoreOverrideCursor(); if (watched == previews[current_wb_pick_display]->display) { const QMouseEvent *mouse_event = (QMouseEvent *)event; set_white_balance(current_wb_pick_display, mouse_event->x(), mouse_event->y()); } else { // The user clicked on something else, give up. // (The click goes through, which might not be ideal, but, yes.) current_wb_pick_display = -1; } } return false; } namespace { double srgb_to_linear(double x) { if (x < 0.04045) { return x / 12.92; } else { return pow((x + 0.055) / 1.055, 2.4); } } } // namespace void MainWindow::set_white_balance(int channel_number, int x, int y) { // Set the white balance to neutral for the grab. It's probably going to // flicker a bit, but hopefully this display is not live anyway. global_mixer->set_wb(Mixer::OUTPUT_INPUT0 + channel_number, 0.5, 0.5, 0.5); previews[channel_number]->display->updateGL(); QRgb reference_color = previews[channel_number]->display->grabFrameBuffer().pixel(x, y); double r = srgb_to_linear(qRed(reference_color) / 255.0); double g = srgb_to_linear(qGreen(reference_color) / 255.0); double b = srgb_to_linear(qBlue(reference_color) / 255.0); global_mixer->set_wb(Mixer::OUTPUT_INPUT0 + channel_number, r, g, b); previews[channel_number]->display->updateGL(); } void MainWindow::audio_state_changed() { post_to_main_thread([this]{ if (global_audio_mixer->get_mapping_mode() == AudioMixer::MappingMode::SIMPLE) { return; } InputMapping mapping = global_audio_mixer->get_input_mapping(); for (unsigned bus_index = 0; bus_index < mapping.buses.size(); ++bus_index) { const InputMapping::Bus &bus = mapping.buses[bus_index]; string suffix; if (bus.device.type == InputSourceType::ALSA_INPUT) { ALSAPool::Device::State state = global_audio_mixer->get_alsa_card_state(bus.device.index); if (state == ALSAPool::Device::State::STARTING) { suffix = " (busy)"; } else if (state == ALSAPool::Device::State::DEAD) { suffix = " (dead)"; } } audio_miniviews[bus_index]->bus_desc_label->setFullText( QString::fromStdString(bus.name + suffix)); audio_expanded_views[bus_index]->bus_desc_label->setFullText( QString::fromStdString(bus.name + suffix)); } }); } nageru-1.4.2/mainwindow.h000066400000000000000000000133161301561706600153350ustar00rootroot00000000000000#ifndef MAINWINDOW_H #define MAINWINDOW_H #include #include #include #include #include #include #include #include "audio_mixer.h" #include "midi_mapper.h" #include "mixer.h" class QEvent; class QObject; class QResizeEvent; class Ui_AudioExpandedView; namespace Ui { class AudioExpandedView; class AudioMiniView; class Display; class MainWindow; } // namespace Ui class QLabel; class QPushButton; class MainWindow : public QMainWindow, public ControllerReceiver { Q_OBJECT public: MainWindow(); void resizeEvent(QResizeEvent *event) override; void mixer_created(Mixer *mixer); // Used to release FBOs on the global ResourcePool. Call after the // mixer has been shut down but not destroyed yet. void mixer_shutting_down(); public slots: void cut_triggered(); void x264_bitrate_triggered(); void exit_triggered(); void about_triggered(); void simple_audio_mode_triggered(); void multichannel_audio_mode_triggered(); void input_mapping_triggered(); void midi_mapping_triggered(); void transition_clicked(int transition_number); void channel_clicked(int channel_number); void wb_button_clicked(int channel_number); void set_transition_names(std::vector transition_names); void update_channel_name(Mixer::Output output, const std::string &name); void update_channel_color(Mixer::Output output, const std::string &color); void gain_staging_knob_changed(unsigned bus_index, int value); void final_makeup_gain_knob_changed(int value); void cutoff_knob_changed(int value); void eq_knob_changed(unsigned bus_index, EQBand band, int value); void limiter_threshold_knob_changed(int value); void compressor_threshold_knob_changed(unsigned bus_index, int value); void mini_fader_changed(int bus, double db_volume); void mute_button_toggled(int bus, bool checked); void reset_meters_button_clicked(); void relayout(); // ControllerReceiver interface. void set_locut(float value) override; void set_limiter_threshold(float value) override; void set_makeup_gain(float value) override; void set_treble(unsigned bus_idx, float value) override; void set_mid(unsigned bus_idx, float value) override; void set_bass(unsigned bus_idx, float value) override; void set_gain(unsigned bus_idx, float value) override; void set_compressor_threshold(unsigned bus_idx, float value) override; void set_fader(unsigned bus_idx, float value) override; void toggle_mute(unsigned bus_idx) override; void toggle_locut(unsigned bus_idx) override; void toggle_auto_gain_staging(unsigned bus_idx) override; void toggle_compressor(unsigned bus_idx) override; void clear_peak(unsigned bus_idx) override; void toggle_limiter() override; void toggle_auto_makeup_gain() override; void clear_all_highlights() override; void highlight_locut(bool highlight) override; void highlight_limiter_threshold(bool highlight) override; void highlight_makeup_gain(bool highlight) override; void highlight_treble(unsigned bus_idx, bool highlight) override; void highlight_mid(unsigned bus_idx, bool highlight) override; void highlight_bass(unsigned bus_idx, bool highlight) override; void highlight_gain(unsigned bus_idx, bool highlight) override; void highlight_compressor_threshold(unsigned bus_idx, bool highlight) override; void highlight_fader(unsigned bus_idx, bool highlight) override; void highlight_mute(unsigned bus_idx, bool highlight) override; void highlight_toggle_locut(unsigned bus_idx, bool highlight) override; void highlight_toggle_auto_gain_staging(unsigned bus_idx, bool highlight) override; void highlight_toggle_compressor(unsigned bus_idx, bool highlight) override; void highlight_clear_peak(unsigned bus_idx, bool highlight) override {} // We don't mark this currently. void highlight_toggle_limiter(bool highlight) override; void highlight_toggle_auto_makeup_gain(bool highlight) override; // Raw receivers are not used. void controller_changed(unsigned controller) override {} void note_on(unsigned note) override {} private: void reset_audio_mapping_ui(); void setup_audio_miniview(); void setup_audio_expanded_view(); bool eventFilter(QObject *watched, QEvent *event) override; void set_white_balance(int channel_number, int x, int y); void update_cutoff_labels(float cutoff_hz); void update_eq_label(unsigned bus_index, EQBand band, float gain_db); // Called from DiskSpaceEstimator. void report_disk_space(off_t free_bytes, double estimated_seconds_left); // Called from the mixer. void audio_level_callback(float level_lufs, float peak_db, std::vector bus_levels, float global_level_lufs, float range_low_lufs, float range_high_lufs, float final_makeup_gain_db, float correlation); std::chrono::steady_clock::time_point last_audio_level_callback; void audio_state_changed(); template void set_relative_value(T *control, float value); template void set_relative_value_if_exists(unsigned bus_idx, T *Ui_AudioExpandedView::*control, float value); template void click_button_if_exists(unsigned bus_idx, T *Ui_AudioExpandedView::*control); template void highlight_control(T *control, bool highlight); template void highlight_mute_control(T *control, bool highlight); template void highlight_control_if_exists(unsigned bus_idx, T *(Ui_AudioExpandedView::*control), bool highlight, bool is_mute_button = false); Ui::MainWindow *ui; QLabel *disk_free_label; QPushButton *transition_btn1, *transition_btn2, *transition_btn3; std::vector previews; std::vector audio_miniviews; std::vector audio_expanded_views; int current_wb_pick_display = -1; MIDIMapper midi_mapper; }; extern MainWindow *global_mainwindow; #endif nageru-1.4.2/metacube2.cpp000066400000000000000000000027111301561706600153600ustar00rootroot00000000000000/* * Implementation of Metacube2 utility functions. * * Note: This file is meant to compile as both C and C++, for easier inclusion * in other projects. */ #include "metacube2.h" #include #include /* * https://www.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf * recommends this for messages as short as ours (see table at page 34). */ #define METACUBE2_CRC_POLYNOMIAL 0x8FDB /* Semi-random starting value to make sure all-zero won't pass. */ #define METACUBE2_CRC_START 0x1234 /* This code is based on code generated by pycrc. */ uint16_t metacube2_compute_crc(const struct metacube2_block_header *hdr) { static const int data_len = sizeof(hdr->size) + sizeof(hdr->flags); const uint8_t *data = (uint8_t *)&hdr->size; uint16_t crc = METACUBE2_CRC_START; int i, j; for (i = 0; i < data_len; ++i) { uint8_t c = data[i]; for (j = 0; j < 8; j++) { int bit = crc & 0x8000; crc = (crc << 1) | ((c >> (7 - j)) & 0x01); if (bit) { crc ^= METACUBE2_CRC_POLYNOMIAL; } } } /* Finalize. */ for (i = 0; i < 16; i++) { int bit = crc & 0x8000; crc = crc << 1; if (bit) { crc ^= METACUBE2_CRC_POLYNOMIAL; } } /* * Invert the checksum for metadata packets, so that clients that * don't understand metadata will ignore it as broken. There will * probably be logging, but apart from that, it's harmless. */ if (ntohs(hdr->flags) & METACUBE_FLAGS_METADATA) { crc ^= 0xffff; } return crc; } nageru-1.4.2/metacube2.h000066400000000000000000000033071301561706600150270ustar00rootroot00000000000000#ifndef _METACUBE2_H #define _METACUBE2_H /* * Definitions for the Metacube2 protocol, used to communicate with Cubemap. * * Note: This file is meant to compile as both C and C++, for easier inclusion * in other projects. */ #include #define METACUBE2_SYNC "cube!map" /* 8 bytes long. */ #define METACUBE_FLAGS_HEADER 0x1 #define METACUBE_FLAGS_NOT_SUITABLE_FOR_STREAM_START 0x2 /* * Metadata packets; should not be counted as data, but rather * parsed (or ignored if you don't understand them). * * Metadata packets start with a uint64_t (network byte order) * that describe the type; the rest is defined by the type. */ #define METACUBE_FLAGS_METADATA 0x4 struct metacube2_block_header { char sync[8]; /* METACUBE2_SYNC */ uint32_t size; /* Network byte order. Does not include header. */ uint16_t flags; /* Network byte order. METACUBE_FLAGS_*. */ uint16_t csum; /* Network byte order. CRC16 of size and flags. If METACUBE_FLAGS_METADATA is set, inverted so that older clients will ignore it as broken. */ }; uint16_t metacube2_compute_crc(const struct metacube2_block_header *hdr); /* * The only currently defined metadata type. Set by the encoder, * and can be measured for latency purposes (e.g., if the network * can't keep up, the latency will tend to increase. */ #define METACUBE_METADATA_TYPE_ENCODER_TIMESTAMP 0x1 struct metacube2_timestamp_packet { uint64_t type; /* METACUBE_METADATA_TYPE_ENCODER_TIMESTAMP, in network byte order. */ /* * Time since the UTC epoch. Basically a struct timespec. * Both are in network byte order. */ uint64_t tv_sec; uint64_t tv_nsec; }; #endif /* !defined(_METACUBE_H) */ nageru-1.4.2/midi_mapper.cpp000066400000000000000000000601431301561706600160020ustar00rootroot00000000000000#include "midi_mapper.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "audio_mixer.h" #include "midi_mapping.pb.h" using namespace google::protobuf; using namespace std; using namespace std::placeholders; namespace { double map_controller_to_float(int val) { // Slightly hackish mapping so that we can represent exactly 0.0, 0.5 and 1.0. if (val <= 0) { return 0.0; } else if (val >= 127) { return 1.0; } else { return (val + 0.5) / 127.0; } } } // namespace MIDIMapper::MIDIMapper(ControllerReceiver *receiver) : receiver(receiver), mapping_proto(new MIDIMappingProto) { should_quit_fd = eventfd(/*initval=*/0, /*flags=*/0); assert(should_quit_fd != -1); } MIDIMapper::~MIDIMapper() { should_quit = true; const uint64_t one = 1; if (write(should_quit_fd, &one, sizeof(one)) != sizeof(one)) { perror("write(should_quit_fd)"); exit(1); } midi_thread.join(); close(should_quit_fd); } bool load_midi_mapping_from_file(const string &filename, MIDIMappingProto *new_mapping) { // Read and parse the protobuf from disk. int fd = open(filename.c_str(), O_RDONLY); if (fd == -1) { perror(filename.c_str()); return false; } io::FileInputStream input(fd); // Takes ownership of fd. if (!TextFormat::Parse(&input, new_mapping)) { input.Close(); return false; } input.Close(); return true; } bool save_midi_mapping_to_file(const MIDIMappingProto &mapping_proto, const string &filename) { // Save to disk. We use the text format because it's friendlier // for a user to look at and edit. int fd = open(filename.c_str(), O_WRONLY | O_TRUNC | O_CREAT, 0666); if (fd == -1) { perror(filename.c_str()); return false; } io::FileOutputStream output(fd); // Takes ownership of fd. if (!TextFormat::Print(mapping_proto, &output)) { // TODO: Don't overwrite the old file (if any) on error. output.Close(); return false; } output.Close(); return true; } void MIDIMapper::set_midi_mapping(const MIDIMappingProto &new_mapping) { lock_guard lock(mu); if (mapping_proto) { mapping_proto->CopyFrom(new_mapping); } else { mapping_proto.reset(new MIDIMappingProto(new_mapping)); } num_controller_banks = min(max(mapping_proto->num_controller_banks(), 1), 5); current_controller_bank = 0; receiver->clear_all_highlights(); update_highlights(); } void MIDIMapper::start_thread() { midi_thread = thread(&MIDIMapper::thread_func, this); } const MIDIMappingProto &MIDIMapper::get_current_mapping() const { lock_guard lock(mu); return *mapping_proto; } ControllerReceiver *MIDIMapper::set_receiver(ControllerReceiver *new_receiver) { lock_guard lock(mu); swap(receiver, new_receiver); return new_receiver; // Now old receiver. } #define RETURN_ON_ERROR(msg, expr) do { \ int err = (expr); \ if (err < 0) { \ fprintf(stderr, msg ": %s\n", snd_strerror(err)); \ return; \ } \ } while (false) #define WARN_ON_ERROR(msg, expr) do { \ int err = (expr); \ if (err < 0) { \ fprintf(stderr, msg ": %s\n", snd_strerror(err)); \ } \ } while (false) void MIDIMapper::thread_func() { snd_seq_t *seq; int err; RETURN_ON_ERROR("snd_seq_open", snd_seq_open(&seq, "default", SND_SEQ_OPEN_DUPLEX, 0)); RETURN_ON_ERROR("snd_seq_nonblock", snd_seq_nonblock(seq, 1)); RETURN_ON_ERROR("snd_seq_client_name", snd_seq_set_client_name(seq, "nageru")); RETURN_ON_ERROR("snd_seq_create_simple_port", snd_seq_create_simple_port(seq, "nageru", SND_SEQ_PORT_CAP_READ | SND_SEQ_PORT_CAP_SUBS_READ | SND_SEQ_PORT_CAP_WRITE | SND_SEQ_PORT_CAP_SUBS_WRITE, SND_SEQ_PORT_TYPE_MIDI_GENERIC | SND_SEQ_PORT_TYPE_APPLICATION)); int queue_id = snd_seq_alloc_queue(seq); RETURN_ON_ERROR("snd_seq_create_queue", queue_id); RETURN_ON_ERROR("snd_seq_start_queue", snd_seq_start_queue(seq, queue_id, nullptr)); // The sequencer object is now ready to be used from other threads. { lock_guard lock(mu); alsa_seq = seq; alsa_queue_id = queue_id; } // Listen to the announce port (0:1), which will tell us about new ports. RETURN_ON_ERROR("snd_seq_connect_from", snd_seq_connect_from(seq, 0, /*client=*/0, /*port=*/1)); // Now go through all ports and subscribe to them. snd_seq_client_info_t *cinfo; snd_seq_client_info_alloca(&cinfo); snd_seq_client_info_set_client(cinfo, -1); while (snd_seq_query_next_client(seq, cinfo) >= 0) { int client = snd_seq_client_info_get_client(cinfo); snd_seq_port_info_t *pinfo; snd_seq_port_info_alloca(&pinfo); snd_seq_port_info_set_client(pinfo, client); snd_seq_port_info_set_port(pinfo, -1); while (snd_seq_query_next_port(seq, pinfo) >= 0) { constexpr int mask = SND_SEQ_PORT_CAP_READ | SND_SEQ_PORT_CAP_SUBS_READ; if ((snd_seq_port_info_get_capability(pinfo) & mask) == mask) { lock_guard lock(mu); subscribe_to_port_lock_held(seq, *snd_seq_port_info_get_addr(pinfo)); } } } int num_alsa_fds = snd_seq_poll_descriptors_count(seq, POLLIN); unique_ptr fds(new pollfd[num_alsa_fds + 1]); while (!should_quit) { snd_seq_poll_descriptors(seq, fds.get(), num_alsa_fds, POLLIN); fds[num_alsa_fds].fd = should_quit_fd; fds[num_alsa_fds].events = POLLIN; fds[num_alsa_fds].revents = 0; err = poll(fds.get(), num_alsa_fds + 1, -1); if (err == 0 || (err == -1 && errno == EINTR)) { continue; } if (err == -1) { perror("poll"); break; } if (fds[num_alsa_fds].revents) { // Activity on should_quit_fd. break; } // Seemingly we can get multiple events in a single poll, // and if we don't handle them all, poll will _not_ alert us! while (!should_quit) { snd_seq_event_t *event; err = snd_seq_event_input(seq, &event); if (err < 0) { if (err == -EINTR) continue; if (err == -EAGAIN) break; fprintf(stderr, "snd_seq_event_input: %s\n", snd_strerror(err)); return; } if (event) { handle_event(seq, event); } } } } void MIDIMapper::handle_event(snd_seq_t *seq, snd_seq_event_t *event) { if (event->source.client == snd_seq_client_id(seq)) { // Ignore events we sent out ourselves. return; } lock_guard lock(mu); switch (event->type) { case SND_SEQ_EVENT_CONTROLLER: { const int controller = event->data.control.param; const float value = map_controller_to_float(event->data.control.value); receiver->controller_changed(controller); // Global controllers. match_controller(controller, MIDIMappingBusProto::kLocutFieldNumber, MIDIMappingProto::kLocutBankFieldNumber, value, bind(&ControllerReceiver::set_locut, receiver, _2)); match_controller(controller, MIDIMappingBusProto::kLimiterThresholdFieldNumber, MIDIMappingProto::kLimiterThresholdBankFieldNumber, value, bind(&ControllerReceiver::set_limiter_threshold, receiver, _2)); match_controller(controller, MIDIMappingBusProto::kMakeupGainFieldNumber, MIDIMappingProto::kMakeupGainBankFieldNumber, value, bind(&ControllerReceiver::set_makeup_gain, receiver, _2)); // Bus controllers. match_controller(controller, MIDIMappingBusProto::kTrebleFieldNumber, MIDIMappingProto::kTrebleBankFieldNumber, value, bind(&ControllerReceiver::set_treble, receiver, _1, _2)); match_controller(controller, MIDIMappingBusProto::kMidFieldNumber, MIDIMappingProto::kMidBankFieldNumber, value, bind(&ControllerReceiver::set_mid, receiver, _1, _2)); match_controller(controller, MIDIMappingBusProto::kBassFieldNumber, MIDIMappingProto::kBassBankFieldNumber, value, bind(&ControllerReceiver::set_bass, receiver, _1, _2)); match_controller(controller, MIDIMappingBusProto::kGainFieldNumber, MIDIMappingProto::kGainBankFieldNumber, value, bind(&ControllerReceiver::set_gain, receiver, _1, _2)); match_controller(controller, MIDIMappingBusProto::kCompressorThresholdFieldNumber, MIDIMappingProto::kCompressorThresholdBankFieldNumber, value, bind(&ControllerReceiver::set_compressor_threshold, receiver, _1, _2)); match_controller(controller, MIDIMappingBusProto::kFaderFieldNumber, MIDIMappingProto::kFaderBankFieldNumber, value, bind(&ControllerReceiver::set_fader, receiver, _1, _2)); break; } case SND_SEQ_EVENT_NOTEON: { const int note = event->data.note.note; receiver->note_on(note); for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); if (bus_mapping.has_prev_bank() && bus_mapping.prev_bank().note_number() == note) { current_controller_bank = (current_controller_bank + num_controller_banks - 1) % num_controller_banks; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_next_bank() && bus_mapping.next_bank().note_number() == note) { current_controller_bank = (current_controller_bank + 1) % num_controller_banks; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_select_bank_1() && bus_mapping.select_bank_1().note_number() == note) { current_controller_bank = 0; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_select_bank_2() && bus_mapping.select_bank_2().note_number() == note && num_controller_banks >= 2) { current_controller_bank = 1; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_select_bank_3() && bus_mapping.select_bank_3().note_number() == note && num_controller_banks >= 3) { current_controller_bank = 2; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_select_bank_4() && bus_mapping.select_bank_4().note_number() == note && num_controller_banks >= 4) { current_controller_bank = 3; update_highlights(); update_lights_lock_held(); } if (bus_mapping.has_select_bank_5() && bus_mapping.select_bank_5().note_number() == note && num_controller_banks >= 5) { current_controller_bank = 4; update_highlights(); update_lights_lock_held(); } } match_button(note, MIDIMappingBusProto::kToggleLocutFieldNumber, MIDIMappingProto::kToggleLocutBankFieldNumber, bind(&ControllerReceiver::toggle_locut, receiver, _1)); match_button(note, MIDIMappingBusProto::kToggleAutoGainStagingFieldNumber, MIDIMappingProto::kToggleAutoGainStagingBankFieldNumber, bind(&ControllerReceiver::toggle_auto_gain_staging, receiver, _1)); match_button(note, MIDIMappingBusProto::kToggleCompressorFieldNumber, MIDIMappingProto::kToggleCompressorBankFieldNumber, bind(&ControllerReceiver::toggle_compressor, receiver, _1)); match_button(note, MIDIMappingBusProto::kClearPeakFieldNumber, MIDIMappingProto::kClearPeakBankFieldNumber, bind(&ControllerReceiver::clear_peak, receiver, _1)); match_button(note, MIDIMappingBusProto::kToggleMuteFieldNumber, MIDIMappingProto::kClearPeakBankFieldNumber, bind(&ControllerReceiver::toggle_mute, receiver, _1)); match_button(note, MIDIMappingBusProto::kToggleLimiterFieldNumber, MIDIMappingProto::kToggleLimiterBankFieldNumber, bind(&ControllerReceiver::toggle_limiter, receiver)); match_button(note, MIDIMappingBusProto::kToggleAutoMakeupGainFieldNumber, MIDIMappingProto::kToggleAutoMakeupGainBankFieldNumber, bind(&ControllerReceiver::toggle_auto_makeup_gain, receiver)); break; } case SND_SEQ_EVENT_PORT_START: subscribe_to_port_lock_held(seq, event->data.addr); break; case SND_SEQ_EVENT_PORT_EXIT: printf("MIDI port %d:%d went away.\n", event->data.addr.client, event->data.addr.port); break; case SND_SEQ_EVENT_PORT_SUBSCRIBED: if (event->data.connect.sender.client != 0 && // Ignore system senders. event->data.connect.sender.client != snd_seq_client_id(seq) && event->data.connect.dest.client == snd_seq_client_id(seq)) { ++num_subscribed_ports; update_highlights(); } break; case SND_SEQ_EVENT_PORT_UNSUBSCRIBED: if (event->data.connect.sender.client != 0 && // Ignore system senders. event->data.connect.sender.client != snd_seq_client_id(seq) && event->data.connect.dest.client == snd_seq_client_id(seq)) { --num_subscribed_ports; update_highlights(); } break; case SND_SEQ_EVENT_NOTEOFF: case SND_SEQ_EVENT_CLIENT_START: case SND_SEQ_EVENT_CLIENT_EXIT: case SND_SEQ_EVENT_CLIENT_CHANGE: case SND_SEQ_EVENT_PORT_CHANGE: break; default: printf("Ignoring MIDI event of unknown type %d.\n", event->type); } } void MIDIMapper::subscribe_to_port_lock_held(snd_seq_t *seq, const snd_seq_addr_t &addr) { // Client 0 is basically the system; ignore it. if (addr.client == 0) { return; } int err = snd_seq_connect_from(seq, 0, addr.client, addr.port); if (err < 0) { // Just print out a warning (i.e., don't die); it could // very well just be e.g. another application. printf("Couldn't subscribe to MIDI port %d:%d (%s).\n", addr.client, addr.port, snd_strerror(err)); } else { printf("Subscribed to MIDI port %d:%d.\n", addr.client, addr.port); } // For sending data back. err = snd_seq_connect_to(seq, 0, addr.client, addr.port); if (err < 0) { printf("Couldn't subscribe MIDI port %d:%d (%s) to us.\n", addr.client, addr.port, snd_strerror(err)); } else { printf("Subscribed MIDI port %d:%d to us.\n", addr.client, addr.port); } current_light_status.clear(); // The current state of the device is unknown. update_lights_lock_held(); } void MIDIMapper::match_controller(int controller, int field_number, int bank_field_number, float value, function func) { if (bank_mismatch(bank_field_number)) { return; } for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { continue; } const MIDIControllerProto &controller_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); if (controller_proto.controller_number() == controller) { func(bus_idx, value); } } } void MIDIMapper::match_button(int note, int field_number, int bank_field_number, function func) { if (bank_mismatch(bank_field_number)) { return; } for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { continue; } const MIDIButtonProto &button_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); if (button_proto.note_number() == note) { func(bus_idx); } } } bool MIDIMapper::has_active_controller(unsigned bus_idx, int field_number, int bank_field_number) { if (bank_mismatch(bank_field_number)) { return false; } const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); return bus_reflection->HasField(bus_mapping, descriptor); } bool MIDIMapper::bank_mismatch(int bank_field_number) { const FieldDescriptor *bank_descriptor = mapping_proto->GetDescriptor()->FindFieldByNumber(bank_field_number); const Reflection *reflection = mapping_proto->GetReflection(); return (reflection->HasField(*mapping_proto, bank_descriptor) && reflection->GetInt32(*mapping_proto, bank_descriptor) != current_controller_bank); } void MIDIMapper::refresh_highlights() { receiver->clear_all_highlights(); update_highlights(); } void MIDIMapper::refresh_lights() { lock_guard lock(mu); update_lights_lock_held(); } void MIDIMapper::update_highlights() { if (num_subscribed_ports.load() == 0) { receiver->clear_all_highlights(); return; } // Global controllers. bool highlight_locut = false; bool highlight_limiter_threshold = false; bool highlight_makeup_gain = false; bool highlight_toggle_limiter = false; bool highlight_toggle_auto_makeup_gain = false; for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { if (has_active_controller( bus_idx, MIDIMappingBusProto::kLocutFieldNumber, MIDIMappingProto::kLocutBankFieldNumber)) { highlight_locut = true; } if (has_active_controller( bus_idx, MIDIMappingBusProto::kLimiterThresholdFieldNumber, MIDIMappingProto::kLimiterThresholdBankFieldNumber)) { highlight_limiter_threshold = true; } if (has_active_controller( bus_idx, MIDIMappingBusProto::kMakeupGainFieldNumber, MIDIMappingProto::kMakeupGainBankFieldNumber)) { highlight_makeup_gain = true; } if (has_active_controller( bus_idx, MIDIMappingBusProto::kToggleLimiterFieldNumber, MIDIMappingProto::kToggleLimiterBankFieldNumber)) { highlight_toggle_limiter = true; } if (has_active_controller( bus_idx, MIDIMappingBusProto::kToggleAutoMakeupGainFieldNumber, MIDIMappingProto::kToggleAutoMakeupGainBankFieldNumber)) { highlight_toggle_auto_makeup_gain = true; } } receiver->highlight_locut(highlight_locut); receiver->highlight_limiter_threshold(highlight_limiter_threshold); receiver->highlight_makeup_gain(highlight_makeup_gain); receiver->highlight_toggle_limiter(highlight_toggle_limiter); receiver->highlight_toggle_auto_makeup_gain(highlight_toggle_auto_makeup_gain); // Per-bus controllers. for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { receiver->highlight_treble(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kTrebleFieldNumber, MIDIMappingProto::kTrebleBankFieldNumber)); receiver->highlight_mid(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kMidFieldNumber, MIDIMappingProto::kMidBankFieldNumber)); receiver->highlight_bass(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kBassFieldNumber, MIDIMappingProto::kBassBankFieldNumber)); receiver->highlight_gain(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kGainFieldNumber, MIDIMappingProto::kGainBankFieldNumber)); receiver->highlight_compressor_threshold(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kCompressorThresholdFieldNumber, MIDIMappingProto::kCompressorThresholdBankFieldNumber)); receiver->highlight_fader(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kFaderFieldNumber, MIDIMappingProto::kFaderBankFieldNumber)); receiver->highlight_mute(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kToggleMuteFieldNumber, MIDIMappingProto::kToggleMuteBankFieldNumber)); receiver->highlight_toggle_locut(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kToggleLocutFieldNumber, MIDIMappingProto::kToggleLocutBankFieldNumber)); receiver->highlight_toggle_auto_gain_staging(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kToggleAutoGainStagingFieldNumber, MIDIMappingProto::kToggleAutoGainStagingBankFieldNumber)); receiver->highlight_toggle_compressor(bus_idx, has_active_controller( bus_idx, MIDIMappingBusProto::kToggleCompressorFieldNumber, MIDIMappingProto::kToggleCompressorBankFieldNumber)); } } void MIDIMapper::update_lights_lock_held() { if (alsa_seq == nullptr || global_audio_mixer == nullptr) { return; } set active_lights; // Desired state. if (current_controller_bank == 0) { activate_lights_all_buses(MIDIMappingBusProto::kBank1IsSelectedFieldNumber, &active_lights); } if (current_controller_bank == 1) { activate_lights_all_buses(MIDIMappingBusProto::kBank2IsSelectedFieldNumber, &active_lights); } if (current_controller_bank == 2) { activate_lights_all_buses(MIDIMappingBusProto::kBank3IsSelectedFieldNumber, &active_lights); } if (current_controller_bank == 3) { activate_lights_all_buses(MIDIMappingBusProto::kBank4IsSelectedFieldNumber, &active_lights); } if (current_controller_bank == 4) { activate_lights_all_buses(MIDIMappingBusProto::kBank5IsSelectedFieldNumber, &active_lights); } if (global_audio_mixer->get_limiter_enabled()) { activate_lights_all_buses(MIDIMappingBusProto::kLimiterIsOnFieldNumber, &active_lights); } if (global_audio_mixer->get_final_makeup_gain_auto()) { activate_lights_all_buses(MIDIMappingBusProto::kAutoMakeupGainIsOnFieldNumber, &active_lights); } unsigned num_buses = min(global_audio_mixer->num_buses(), mapping_proto->bus_mapping_size()); for (unsigned bus_idx = 0; bus_idx < num_buses; ++bus_idx) { if (global_audio_mixer->get_mute(bus_idx)) { activate_lights(bus_idx, MIDIMappingBusProto::kIsMutedFieldNumber, &active_lights); } if (global_audio_mixer->get_locut_enabled(bus_idx)) { activate_lights(bus_idx, MIDIMappingBusProto::kLocutIsOnFieldNumber, &active_lights); } if (global_audio_mixer->get_gain_staging_auto(bus_idx)) { activate_lights(bus_idx, MIDIMappingBusProto::kAutoGainStagingIsOnFieldNumber, &active_lights); } if (global_audio_mixer->get_compressor_enabled(bus_idx)) { activate_lights(bus_idx, MIDIMappingBusProto::kCompressorIsOnFieldNumber, &active_lights); } if (has_peaked[bus_idx]) { activate_lights(bus_idx, MIDIMappingBusProto::kHasPeakedFieldNumber, &active_lights); } } unsigned num_events = 0; for (unsigned note_num = 1; note_num <= 127; ++note_num) { bool active = active_lights.count(note_num); if (current_light_status.count(note_num) && current_light_status[note_num] == active) { // Already known to be in the desired state. continue; } snd_seq_event_t ev; snd_seq_ev_clear(&ev); // Some devices drop events if we throw them onto them // too quickly. Add a 1 ms delay for each. snd_seq_real_time_t tm{0, num_events++ * 1000000}; snd_seq_ev_schedule_real(&ev, alsa_queue_id, true, &tm); snd_seq_ev_set_source(&ev, 0); snd_seq_ev_set_subs(&ev); // For some reason, not all devices respond to note off. // Use note-on with velocity of 0 (which is equivalent) instead. snd_seq_ev_set_noteon(&ev, /*channel=*/0, note_num, active ? 127 : 0); WARN_ON_ERROR("snd_seq_event_output", snd_seq_event_output(alsa_seq, &ev)); current_light_status[note_num] = active; } WARN_ON_ERROR("snd_seq_drain_output", snd_seq_drain_output(alsa_seq)); } void MIDIMapper::activate_lights(unsigned bus_idx, int field_number, set *active_lights) { const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { return; } const MIDILightProto &light_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); active_lights->insert(light_proto.note_number()); } void MIDIMapper::activate_lights_all_buses(int field_number, set *active_lights) { for (size_t bus_idx = 0; bus_idx < size_t(mapping_proto->bus_mapping_size()); ++bus_idx) { const MIDIMappingBusProto &bus_mapping = mapping_proto->bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { continue; } const MIDILightProto &light_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); active_lights->insert(light_proto.note_number()); } } nageru-1.4.2/midi_mapper.h000066400000000000000000000122661301561706600154520ustar00rootroot00000000000000#ifndef _MIDI_MAPPER_H #define _MIDI_MAPPER_H 1 // MIDIMapper is a class that listens for incoming MIDI messages from // mixer controllers (ie., it is not meant to be used with regular // instruments), interprets them according to a device-specific, user-defined // mapping, and calls back into a receiver (typically the MainWindow). // This way, it is possible to control audio functionality using physical // pots and faders instead of the mouse. #include #include #include #include #include #include #include #include #include "defs.h" class MIDIMappingProto; typedef struct snd_seq_addr snd_seq_addr_t; typedef struct snd_seq_event snd_seq_event_t; typedef struct _snd_seq snd_seq_t; // Interface for receiving interpreted controller messages. class ControllerReceiver { public: // All values are [0.0, 1.0]. virtual void set_locut(float value) = 0; virtual void set_limiter_threshold(float value) = 0; virtual void set_makeup_gain(float value) = 0; virtual void set_treble(unsigned bus_idx, float value) = 0; virtual void set_mid(unsigned bus_idx, float value) = 0; virtual void set_bass(unsigned bus_idx, float value) = 0; virtual void set_gain(unsigned bus_idx, float value) = 0; virtual void set_compressor_threshold(unsigned bus_idx, float value) = 0; virtual void set_fader(unsigned bus_idx, float value) = 0; virtual void toggle_mute(unsigned bus_idx) = 0; virtual void toggle_locut(unsigned bus_idx) = 0; virtual void toggle_auto_gain_staging(unsigned bus_idx) = 0; virtual void toggle_compressor(unsigned bus_idx) = 0; virtual void clear_peak(unsigned bus_idx) = 0; virtual void toggle_limiter() = 0; virtual void toggle_auto_makeup_gain() = 0; // Signals to highlight controls to mark them to the user // as MIDI-controllable (or not). virtual void clear_all_highlights() = 0; virtual void highlight_locut(bool highlight) = 0; virtual void highlight_limiter_threshold(bool highlight) = 0; virtual void highlight_makeup_gain(bool highlight) = 0; virtual void highlight_treble(unsigned bus_idx, bool highlight) = 0; virtual void highlight_mid(unsigned bus_idx, bool highlight) = 0; virtual void highlight_bass(unsigned bus_idx, bool highlight) = 0; virtual void highlight_gain(unsigned bus_idx, bool highlight) = 0; virtual void highlight_compressor_threshold(unsigned bus_idx, bool highlight) = 0; virtual void highlight_fader(unsigned bus_idx, bool highlight) = 0; virtual void highlight_mute(unsigned bus_idx, bool highlight) = 0; virtual void highlight_toggle_locut(unsigned bus_idx, bool highlight) = 0; virtual void highlight_toggle_auto_gain_staging(unsigned bus_idx, bool highlight) = 0; virtual void highlight_toggle_compressor(unsigned bus_idx, bool highlight) = 0; virtual void highlight_clear_peak(unsigned bus_idx, bool highlight) = 0; virtual void highlight_toggle_limiter(bool highlight) = 0; virtual void highlight_toggle_auto_makeup_gain(bool highlight) = 0; // Raw events; used for the editor dialog only. virtual void controller_changed(unsigned controller) = 0; virtual void note_on(unsigned note) = 0; }; class MIDIMapper { public: MIDIMapper(ControllerReceiver *receiver); virtual ~MIDIMapper(); void set_midi_mapping(const MIDIMappingProto &new_mapping); void start_thread(); const MIDIMappingProto &get_current_mapping() const; // Overwrites and returns the previous value. ControllerReceiver *set_receiver(ControllerReceiver *new_receiver); void refresh_highlights(); void refresh_lights(); void set_has_peaked(unsigned bus_idx, bool has_peaked) { this->has_peaked[bus_idx] = has_peaked; } private: void thread_func(); void handle_event(snd_seq_t *seq, snd_seq_event_t *event); void subscribe_to_port_lock_held(snd_seq_t *seq, const snd_seq_addr_t &addr); void match_controller(int controller, int field_number, int bank_field_number, float value, std::function func); void match_button(int note, int field_number, int bank_field_number, std::function func); bool has_active_controller(unsigned bus_idx, int field_number, int bank_field_number); // Also works for buttons. bool bank_mismatch(int bank_field_number); void update_highlights(); void update_lights_lock_held(); void activate_lights(unsigned bus_idx, int field_number, std::set *active_lights); void activate_lights_all_buses(int field_number, std::set *active_lights); std::atomic should_quit{false}; int should_quit_fd; std::atomic has_peaked[MAX_BUSES] {{ false }}; mutable std::mutex mu; ControllerReceiver *receiver; // Under . std::unique_ptr mapping_proto; // Under . int num_controller_banks; // Under . std::atomic current_controller_bank{0}; std::atomic num_subscribed_ports{0}; std::thread midi_thread; std::map current_light_status; // Keyed by note number. Under . snd_seq_t *alsa_seq{nullptr}; // Under . int alsa_queue_id{-1}; // Under . }; bool load_midi_mapping_from_file(const std::string &filename, MIDIMappingProto *new_mapping); bool save_midi_mapping_to_file(const MIDIMappingProto &mapping_proto, const std::string &filename); #endif // !defined(_MIDI_MAPPER_H) nageru-1.4.2/midi_mapping.proto000066400000000000000000000106641301561706600165350ustar00rootroot00000000000000// Mappings from MIDI controllers to the UI. (We don't really build // a more complicated data structure than this in Nageru itself either; // we just edit and match directly against the protobuf.) syntax = "proto2"; // A single, given controller mapping. message MIDIControllerProto { required int32 controller_number = 1; // TODO: Add flags like invert here if/when we need them. } message MIDIButtonProto { required int32 note_number = 1; } message MIDILightProto { required int32 note_number = 1; } // All the mappings for a given a bus. message MIDIMappingBusProto { // TODO: If we need support for lots of buses (i.e., more than the typical eight // on a mixer), add a system for bus banks, like we have for controller banks. // optional int32 bus_bank = 1; optional MIDIControllerProto treble = 2; optional MIDIControllerProto mid = 3; optional MIDIControllerProto bass = 4; optional MIDIControllerProto gain = 5; optional MIDIControllerProto compressor_threshold = 6; optional MIDIControllerProto fader = 7; optional MIDIButtonProto toggle_mute = 8; optional MIDIButtonProto toggle_locut = 9; optional MIDIButtonProto toggle_auto_gain_staging = 10; optional MIDIButtonProto toggle_compressor = 11; optional MIDIButtonProto clear_peak = 12; // These are really global (controller bank change affects all buss), // but it's not uncommon that we'd want one button per bus to switch banks. // E.g., if the user binds the “mute” button to “next bank”, they'd want every // mute button on the mixer to do that, so they need one mapping per bus. optional MIDIButtonProto prev_bank = 13; optional MIDIButtonProto next_bank = 14; optional MIDIButtonProto select_bank_1 = 15; optional MIDIButtonProto select_bank_2 = 16; optional MIDIButtonProto select_bank_3 = 17; optional MIDIButtonProto select_bank_4 = 18; optional MIDIButtonProto select_bank_5 = 19; optional MIDIButtonProto toggle_limiter = 20; optional MIDIButtonProto toggle_auto_makeup_gain = 21; // These are also global (they belong to the master bus), and unlike // the bank change commands, one would usually have only one of each, // but there's no reason to limit them to one each, and the editor UI // becomes simpler if they are the treated the same way as the bank // commands. optional MIDIControllerProto locut = 22; optional MIDIControllerProto limiter_threshold = 23; optional MIDIControllerProto makeup_gain = 24; // Per-bus lights. optional MIDILightProto is_muted = 25; optional MIDILightProto locut_is_on = 26; optional MIDILightProto auto_gain_staging_is_on = 27; optional MIDILightProto compressor_is_on = 28; optional MIDILightProto has_peaked = 29; // Global lights. Same logic as above for why they're in this proto. optional MIDILightProto bank_1_is_selected = 30; optional MIDILightProto bank_2_is_selected = 31; optional MIDILightProto bank_3_is_selected = 32; optional MIDILightProto bank_4_is_selected = 33; optional MIDILightProto bank_5_is_selected = 34; optional MIDILightProto limiter_is_on = 35; optional MIDILightProto auto_makeup_gain_is_on = 36; } // The top-level protobuf, containing all the bus mappings, as well as // more global settings. // // Since a typical mixer will have fewer physical controls than what Nageru // could use, Nageru supports so-called controller banks. A mapping can // optionally belong to a bank, and if so, that mapping is only active when // that bank is selected. The user can then select the current bank using // other mappings, typically by having some mixer button assigned to // “next bank”. This yields effective multiplexing of lesser-used controls. message MIDIMappingProto { optional int32 num_controller_banks = 1 [default = 0]; // Max 5. // Bus controller banks. optional int32 treble_bank = 2; optional int32 mid_bank = 3; optional int32 bass_bank = 4; optional int32 gain_bank = 5; optional int32 compressor_threshold_bank = 6; optional int32 fader_bank = 7; // Bus button banks. optional int32 toggle_mute_bank = 8; optional int32 toggle_locut_bank = 9; optional int32 toggle_auto_gain_staging_bank = 10; optional int32 toggle_compressor_bank = 11; optional int32 clear_peak_bank = 12; // Global controller banks. optional int32 locut_bank = 13; optional int32 limiter_threshold_bank = 14; optional int32 makeup_gain_bank = 15; // Global buttons. optional int32 toggle_limiter_bank = 16; optional int32 toggle_auto_makeup_gain_bank = 17; repeated MIDIMappingBusProto bus_mapping = 18; } nageru-1.4.2/midi_mapping_dialog.cpp000066400000000000000000000560461301561706600174770ustar00rootroot00000000000000#include "midi_mapping_dialog.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "midi_mapper.h" #include "midi_mapping.pb.h" #include "post_to_main_thread.h" #include "ui_midi_mapping.h" class QObject; using namespace google::protobuf; using namespace std; vector per_bus_controllers = { { "Treble", MIDIMappingBusProto::kTrebleFieldNumber, MIDIMappingProto::kTrebleBankFieldNumber }, { "Mid", MIDIMappingBusProto::kMidFieldNumber, MIDIMappingProto::kMidBankFieldNumber }, { "Bass", MIDIMappingBusProto::kBassFieldNumber, MIDIMappingProto::kBassBankFieldNumber }, { "Gain", MIDIMappingBusProto::kGainFieldNumber, MIDIMappingProto::kGainBankFieldNumber }, { "Compressor threshold", MIDIMappingBusProto::kCompressorThresholdFieldNumber, MIDIMappingProto::kCompressorThresholdBankFieldNumber}, { "Fader", MIDIMappingBusProto::kFaderFieldNumber, MIDIMappingProto::kFaderBankFieldNumber } }; vector per_bus_buttons = { { "Toggle mute", MIDIMappingBusProto::kToggleMuteFieldNumber, MIDIMappingProto::kToggleMuteBankFieldNumber }, { "Toggle locut", MIDIMappingBusProto::kToggleLocutFieldNumber, MIDIMappingProto::kToggleLocutBankFieldNumber }, { "Togle auto gain staging", MIDIMappingBusProto::kToggleAutoGainStagingFieldNumber, MIDIMappingProto::kToggleAutoGainStagingBankFieldNumber }, { "Togle compressor", MIDIMappingBusProto::kToggleCompressorFieldNumber, MIDIMappingProto::kToggleCompressorBankFieldNumber }, { "Clear peak", MIDIMappingBusProto::kClearPeakFieldNumber, MIDIMappingProto::kClearPeakBankFieldNumber } }; vector per_bus_lights = { { "Is muted", MIDIMappingBusProto::kIsMutedFieldNumber, 0 }, { "Locut is on", MIDIMappingBusProto::kLocutIsOnFieldNumber, 0 }, { "Auto gain staging is on", MIDIMappingBusProto::kAutoGainStagingIsOnFieldNumber, 0 }, { "Compressor is on", MIDIMappingBusProto::kCompressorIsOnFieldNumber, 0 }, { "Bus has peaked", MIDIMappingBusProto::kHasPeakedFieldNumber, 0 } }; vector global_controllers = { { "Locut cutoff", MIDIMappingBusProto::kLocutFieldNumber, MIDIMappingProto::kLocutBankFieldNumber }, { "Limiter threshold", MIDIMappingBusProto::kLimiterThresholdFieldNumber, MIDIMappingProto::kLimiterThresholdBankFieldNumber }, { "Makeup gain", MIDIMappingBusProto::kMakeupGainFieldNumber, MIDIMappingProto::kMakeupGainBankFieldNumber } }; vector global_buttons = { { "Previous bank", MIDIMappingBusProto::kPrevBankFieldNumber, 0 }, { "Next bank", MIDIMappingBusProto::kNextBankFieldNumber, 0 }, { "Select bank 1", MIDIMappingBusProto::kSelectBank1FieldNumber, 0 }, { "Select bank 2", MIDIMappingBusProto::kSelectBank2FieldNumber, 0 }, { "Select bank 3", MIDIMappingBusProto::kSelectBank3FieldNumber, 0 }, { "Select bank 4", MIDIMappingBusProto::kSelectBank4FieldNumber, 0 }, { "Select bank 5", MIDIMappingBusProto::kSelectBank5FieldNumber, 0 }, { "Toggle limiter", MIDIMappingBusProto::kToggleLimiterFieldNumber, MIDIMappingProto::kToggleLimiterBankFieldNumber }, { "Toggle auto makeup gain", MIDIMappingBusProto::kToggleAutoMakeupGainFieldNumber, MIDIMappingProto::kToggleAutoMakeupGainBankFieldNumber } }; vector global_lights = { { "Bank 1 is selected", MIDIMappingBusProto::kBank1IsSelectedFieldNumber, 0 }, { "Bank 2 is selected", MIDIMappingBusProto::kBank2IsSelectedFieldNumber, 0 }, { "Bank 3 is selected", MIDIMappingBusProto::kBank3IsSelectedFieldNumber, 0 }, { "Bank 4 is selected", MIDIMappingBusProto::kBank4IsSelectedFieldNumber, 0 }, { "Bank 5 is selected", MIDIMappingBusProto::kBank5IsSelectedFieldNumber, 0 }, { "Limiter is on", MIDIMappingBusProto::kLimiterIsOnFieldNumber, 0 }, { "Auto makeup gain is on", MIDIMappingBusProto::kAutoMakeupGainIsOnFieldNumber, 0 }, }; namespace { int get_bank(const MIDIMappingProto &mapping_proto, int bank_field_number, int default_value) { const FieldDescriptor *bank_descriptor = mapping_proto.GetDescriptor()->FindFieldByNumber(bank_field_number); const Reflection *reflection = mapping_proto.GetReflection(); if (!reflection->HasField(mapping_proto, bank_descriptor)) { return default_value; } return reflection->GetInt32(mapping_proto, bank_descriptor); } int get_controller_mapping(const MIDIMappingProto &mapping_proto, size_t bus_idx, int field_number, int default_value) { if (bus_idx >= size_t(mapping_proto.bus_mapping_size())) { return default_value; } const MIDIMappingBusProto &bus_mapping = mapping_proto.bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { return default_value; } const MIDIControllerProto &controller_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); return controller_proto.controller_number(); } int get_button_mapping(const MIDIMappingProto &mapping_proto, size_t bus_idx, int field_number, int default_value) { if (bus_idx >= size_t(mapping_proto.bus_mapping_size())) { return default_value; } const MIDIMappingBusProto &bus_mapping = mapping_proto.bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { return default_value; } const MIDIButtonProto &bus_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); return bus_proto.note_number(); } int get_light_mapping(const MIDIMappingProto &mapping_proto, size_t bus_idx, int field_number, int default_value) { if (bus_idx >= size_t(mapping_proto.bus_mapping_size())) { return default_value; } const MIDIMappingBusProto &bus_mapping = mapping_proto.bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping.GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping.GetReflection(); if (!bus_reflection->HasField(bus_mapping, descriptor)) { return default_value; } const MIDILightProto &bus_proto = static_cast(bus_reflection->GetMessage(bus_mapping, descriptor)); return bus_proto.note_number(); } } // namespace MIDIMappingDialog::MIDIMappingDialog(MIDIMapper *mapper) : ui(new Ui::MIDIMappingDialog), mapper(mapper) { ui->setupUi(this); const MIDIMappingProto mapping_proto = mapper->get_current_mapping(); // Take a copy. old_receiver = mapper->set_receiver(this); QStringList labels; labels << ""; labels << "Controller bank"; for (unsigned bus_idx = 0; bus_idx < num_buses; ++bus_idx) { char buf[256]; snprintf(buf, sizeof(buf), "Bus %d", bus_idx + 1); labels << buf; } labels << ""; ui->treeWidget->setColumnCount(num_buses + 3); ui->treeWidget->setHeaderLabels(labels); add_controls("Per-bus controllers", ControlType::CONTROLLER, SpinnerGroup::PER_BUS_CONTROLLERS, mapping_proto, per_bus_controllers); add_controls("Per-bus buttons", ControlType::BUTTON, SpinnerGroup::PER_BUS_BUTTONS, mapping_proto, per_bus_buttons); add_controls("Per-bus lights", ControlType::LIGHT, SpinnerGroup::PER_BUS_LIGHTS, mapping_proto, per_bus_lights); add_controls("Global controllers", ControlType::CONTROLLER, SpinnerGroup::GLOBAL_CONTROLLERS, mapping_proto, global_controllers); add_controls("Global buttons", ControlType::BUTTON, SpinnerGroup::GLOBAL_BUTTONS, mapping_proto, global_buttons); add_controls("Global lights", ControlType::LIGHT, SpinnerGroup::GLOBAL_LIGHTS, mapping_proto, global_lights); fill_controls_from_mapping(mapping_proto); // Auto-resize every column but the last. for (unsigned column_idx = 0; column_idx < num_buses + 3; ++column_idx) { ui->treeWidget->resizeColumnToContents(column_idx); } connect(ui->guess_bus_button, &QPushButton::clicked, bind(&MIDIMappingDialog::guess_clicked, this, false)); connect(ui->guess_group_button, &QPushButton::clicked, bind(&MIDIMappingDialog::guess_clicked, this, true)); connect(ui->ok_cancel_buttons, &QDialogButtonBox::accepted, this, &MIDIMappingDialog::ok_clicked); connect(ui->ok_cancel_buttons, &QDialogButtonBox::rejected, this, &MIDIMappingDialog::cancel_clicked); connect(ui->save_button, &QPushButton::clicked, this, &MIDIMappingDialog::save_clicked); connect(ui->load_button, &QPushButton::clicked, this, &MIDIMappingDialog::load_clicked); update_guess_button_state(); } MIDIMappingDialog::~MIDIMappingDialog() { mapper->set_receiver(old_receiver); mapper->refresh_highlights(); } bool MIDIMappingDialog::eventFilter(QObject *obj, QEvent *event) { if (event->type() == QEvent::FocusIn || event->type() == QEvent::FocusOut) { // We ignore the guess buttons themselves; it should be allowed // to navigate from a spinner to focus on a button (to click it). if (obj != ui->guess_bus_button && obj != ui->guess_group_button) { update_guess_button_state(); } } return false; } void MIDIMappingDialog::guess_clicked(bool limit_to_group) { FocusInfo focus = find_focus(); if (focus.bus_idx == -1) { // The guess button probably took the focus away from us. focus = last_focus; } assert(focus.bus_idx != -1); // The button should have been disabled. pair bus_and_offset = guess_offset(focus.bus_idx, limit_to_group ? focus.spinner_group : SpinnerGroup::ALL_GROUPS); const int source_bus_idx = bus_and_offset.first; const int offset = bus_and_offset.second; assert(source_bus_idx != -1); // The button should have been disabled. for (const auto &field_number_and_spinner : spinners[focus.bus_idx]) { int field_number = field_number_and_spinner.first; QSpinBox *spinner = field_number_and_spinner.second.spinner; SpinnerGroup this_spinner_group = field_number_and_spinner.second.group; if (limit_to_group && this_spinner_group != focus.spinner_group) { continue; } assert(spinners[source_bus_idx].count(field_number)); QSpinBox *source_spinner = spinners[source_bus_idx][field_number].spinner; assert(spinners[source_bus_idx][field_number].group == this_spinner_group); if (source_spinner->value() != -1) { spinner->setValue(source_spinner->value() + offset); } } // See if we can find a “next” bus to move the focus to. const int next_bus_idx = focus.bus_idx + (focus.bus_idx - source_bus_idx); // Note: Could become e.g. -1. for (const InstantiatedSpinner &is : controller_spinners) { if (int(is.bus_idx) == next_bus_idx && is.field_number == focus.field_number) { is.spinner->setFocus(); } } for (const InstantiatedSpinner &is : button_spinners) { if (int(is.bus_idx) == next_bus_idx && is.field_number == focus.field_number) { is.spinner->setFocus(); } } for (const InstantiatedSpinner &is : light_spinners) { if (int(is.bus_idx) == next_bus_idx && is.field_number == focus.field_number) { is.spinner->setFocus(); } } } void MIDIMappingDialog::ok_clicked() { unique_ptr new_mapping = construct_mapping_proto_from_ui(); mapper->set_midi_mapping(*new_mapping); mapper->set_receiver(old_receiver); accept(); } void MIDIMappingDialog::cancel_clicked() { mapper->set_receiver(old_receiver); reject(); } void MIDIMappingDialog::save_clicked() { unique_ptr new_mapping = construct_mapping_proto_from_ui(); QString filename = QFileDialog::getSaveFileName(this, "Save MIDI mapping", QString(), tr("Mapping files (*.midimapping)")); if (!filename.endsWith(".midimapping")) { filename += ".midimapping"; } if (!save_midi_mapping_to_file(*new_mapping, filename.toStdString())) { QMessageBox box; box.setText("Could not save mapping to '" + filename + "'. Check that you have the right permissions and try again."); box.exec(); } } void MIDIMappingDialog::load_clicked() { QString filename = QFileDialog::getOpenFileName(this, "Load MIDI mapping", QString(), tr("Mapping files (*.midimapping)")); MIDIMappingProto new_mapping; if (!load_midi_mapping_from_file(filename.toStdString(), &new_mapping)) { QMessageBox box; box.setText("Could not load mapping from '" + filename + "'. Check that the file exists, has the right permissions and is valid."); box.exec(); return; } fill_controls_from_mapping(new_mapping); } namespace { template T *get_mutable_bus_message(MIDIMappingProto *mapping_proto, unsigned bus_idx, int field_number) { while (size_t(mapping_proto->bus_mapping_size()) <= bus_idx) { mapping_proto->add_bus_mapping(); } MIDIMappingBusProto *bus_mapping = mapping_proto->mutable_bus_mapping(bus_idx); const FieldDescriptor *descriptor = bus_mapping->GetDescriptor()->FindFieldByNumber(field_number); const Reflection *bus_reflection = bus_mapping->GetReflection(); return static_cast(bus_reflection->MutableMessage(bus_mapping, descriptor)); } } // namespace unique_ptr MIDIMappingDialog::construct_mapping_proto_from_ui() { unique_ptr mapping_proto(new MIDIMappingProto); for (const InstantiatedSpinner &is : controller_spinners) { const int val = is.spinner->value(); if (val == -1) { continue; } MIDIControllerProto *controller_proto = get_mutable_bus_message(mapping_proto.get(), is.bus_idx, is.field_number); controller_proto->set_controller_number(val); } for (const InstantiatedSpinner &is : button_spinners) { const int val = is.spinner->value(); if (val == -1) { continue; } MIDIButtonProto *button_proto = get_mutable_bus_message(mapping_proto.get(), is.bus_idx, is.field_number); button_proto->set_note_number(val); } for (const InstantiatedSpinner &is : light_spinners) { const int val = is.spinner->value(); if (val == -1) { continue; } MIDILightProto *light_proto = get_mutable_bus_message(mapping_proto.get(), is.bus_idx, is.field_number); light_proto->set_note_number(val); } int highest_bank_used = 0; // 1-indexed. for (const InstantiatedComboBox &ic : bank_combo_boxes) { const int val = ic.combo_box->currentIndex(); highest_bank_used = std::max(highest_bank_used, val); if (val == 0) { continue; } const FieldDescriptor *descriptor = mapping_proto->GetDescriptor()->FindFieldByNumber(ic.field_number); const Reflection *bus_reflection = mapping_proto->GetReflection(); bus_reflection->SetInt32(mapping_proto.get(), descriptor, val - 1); } mapping_proto->set_num_controller_banks(highest_bank_used); return mapping_proto; } void MIDIMappingDialog::add_bank_selector(QTreeWidgetItem *item, const MIDIMappingProto &mapping_proto, int bank_field_number) { if (bank_field_number == 0) { return; } QComboBox *bank_selector = new QComboBox(this); bank_selector->addItems(QStringList() << "" << "Bank 1" << "Bank 2" << "Bank 3" << "Bank 4" << "Bank 5"); bank_selector->setAutoFillBackground(true); bank_combo_boxes.push_back(InstantiatedComboBox{ bank_selector, bank_field_number }); ui->treeWidget->setItemWidget(item, 1, bank_selector); } void MIDIMappingDialog::add_controls(const string &heading, MIDIMappingDialog::ControlType control_type, MIDIMappingDialog::SpinnerGroup spinner_group, const MIDIMappingProto &mapping_proto, const vector &controls) { QTreeWidgetItem *heading_item = new QTreeWidgetItem(ui->treeWidget); heading_item->setText(0, QString::fromStdString(heading)); heading_item->setFirstColumnSpanned(true); heading_item->setExpanded(true); for (const Control &control : controls) { QTreeWidgetItem *item = new QTreeWidgetItem(heading_item); heading_item->addChild(item); add_bank_selector(item, mapping_proto, control.bank_field_number); item->setText(0, QString::fromStdString(control.label + " ")); for (unsigned bus_idx = 0; bus_idx < num_buses; ++bus_idx) { QSpinBox *spinner = new QSpinBox(this); spinner->setRange(-1, 127); spinner->setAutoFillBackground(true); spinner->setSpecialValueText("\u200d"); // Zero-width joiner (ie., empty). spinner->installEventFilter(this); // So we know when the focus changes. ui->treeWidget->setItemWidget(item, bus_idx + 2, spinner); if (control_type == ControlType::CONTROLLER) { controller_spinners.push_back(InstantiatedSpinner{ spinner, bus_idx, spinner_group, control.field_number }); } else if (control_type == ControlType::BUTTON) { button_spinners.push_back(InstantiatedSpinner{ spinner, bus_idx, spinner_group, control.field_number }); } else { assert(control_type == ControlType::LIGHT); light_spinners.push_back(InstantiatedSpinner{ spinner, bus_idx, spinner_group, control.field_number }); } spinners[bus_idx][control.field_number] = SpinnerAndGroup{ spinner, spinner_group }; connect(spinner, static_cast(&QSpinBox::valueChanged), bind(&MIDIMappingDialog::update_guess_button_state, this)); } } ui->treeWidget->addTopLevelItem(heading_item); } void MIDIMappingDialog::fill_controls_from_mapping(const MIDIMappingProto &mapping_proto) { for (const InstantiatedSpinner &is : controller_spinners) { is.spinner->setValue(get_controller_mapping(mapping_proto, is.bus_idx, is.field_number, -1)); } for (const InstantiatedSpinner &is : button_spinners) { is.spinner->setValue(get_button_mapping(mapping_proto, is.bus_idx, is.field_number, -1)); } for (const InstantiatedSpinner &is : light_spinners) { is.spinner->setValue(get_light_mapping(mapping_proto, is.bus_idx, is.field_number, -1)); } for (const InstantiatedComboBox &ic : bank_combo_boxes) { ic.combo_box->setCurrentIndex(get_bank(mapping_proto, ic.field_number, -1) + 1); } } void MIDIMappingDialog::controller_changed(unsigned controller) { post_to_main_thread([=]{ for (const InstantiatedSpinner &is : controller_spinners) { if (is.spinner->hasFocus()) { is.spinner->setValue(controller); is.spinner->selectAll(); } } }); } void MIDIMappingDialog::note_on(unsigned note) { post_to_main_thread([=]{ for (const InstantiatedSpinner &is : button_spinners) { if (is.spinner->hasFocus()) { is.spinner->setValue(note); is.spinner->selectAll(); } } for (const InstantiatedSpinner &is : light_spinners) { if (is.spinner->hasFocus()) { is.spinner->setValue(note); is.spinner->selectAll(); } } }); } pair MIDIMappingDialog::guess_offset(unsigned bus_idx, MIDIMappingDialog::SpinnerGroup spinner_group) { constexpr pair not_found(-1, 0); if (bus_is_empty(bus_idx, spinner_group)) { return not_found; } // See if we can find a non-empty bus to source from (prefer from the left). unsigned source_bus_idx; if (bus_idx > 0 && !bus_is_empty(bus_idx - 1, spinner_group)) { source_bus_idx = bus_idx - 1; } else if (bus_idx < num_buses - 1 && !bus_is_empty(bus_idx + 1, spinner_group)) { source_bus_idx = bus_idx + 1; } else { return not_found; } // See if we can find a consistent offset. bool found_offset = false; int offset = 0; int minimum_allowed_offset = numeric_limits::min(); int maximum_allowed_offset = numeric_limits::max(); for (const auto &field_number_and_spinner : spinners[bus_idx]) { int field_number = field_number_and_spinner.first; QSpinBox *spinner = field_number_and_spinner.second.spinner; SpinnerGroup this_spinner_group = field_number_and_spinner.second.group; assert(spinners[source_bus_idx].count(field_number)); QSpinBox *source_spinner = spinners[source_bus_idx][field_number].spinner; assert(spinners[source_bus_idx][field_number].group == this_spinner_group); if (spinner_group != SpinnerGroup::ALL_GROUPS && spinner_group != this_spinner_group) { continue; } if (spinner->value() == -1) { if (source_spinner->value() != -1) { // If the source value is e.g. 3, offset can't be less than -2 or larger than 124. // Otherwise, we'd extrapolate values outside [1..127]. minimum_allowed_offset = max(minimum_allowed_offset, 1 - source_spinner->value()); maximum_allowed_offset = min(maximum_allowed_offset, 127 - source_spinner->value()); } continue; } if (source_spinner->value() == -1) { // The bus has a controller set that the source bus doesn't set. return not_found; } int candidate_offset = spinner->value() - source_spinner->value(); if (!found_offset) { offset = candidate_offset; found_offset = true; } else if (candidate_offset != offset) { return not_found; } } if (!found_offset) { // Given that the bus wasn't empty, this shouldn't happen. assert(false); return not_found; } if (offset < minimum_allowed_offset || offset > maximum_allowed_offset) { return not_found; } return make_pair(source_bus_idx, offset); } bool MIDIMappingDialog::bus_is_empty(unsigned bus_idx, SpinnerGroup spinner_group) { for (const auto &field_number_and_spinner : spinners[bus_idx]) { QSpinBox *spinner = field_number_and_spinner.second.spinner; SpinnerGroup this_spinner_group = field_number_and_spinner.second.group; if (spinner_group != SpinnerGroup::ALL_GROUPS && spinner_group != this_spinner_group) { continue; } if (spinner->value() != -1) { return false; } } return true; } void MIDIMappingDialog::update_guess_button_state() { FocusInfo focus = find_focus(); if (focus.bus_idx < 0) { return; } { pair bus_and_offset = guess_offset(focus.bus_idx, SpinnerGroup::ALL_GROUPS); ui->guess_bus_button->setEnabled(bus_and_offset.first != -1); } { pair bus_and_offset = guess_offset(focus.bus_idx, focus.spinner_group); ui->guess_group_button->setEnabled(bus_and_offset.first != -1); } last_focus = focus; } MIDIMappingDialog::FocusInfo MIDIMappingDialog::find_focus() const { for (const InstantiatedSpinner &is : controller_spinners) { if (is.spinner->hasFocus()) { return FocusInfo{ int(is.bus_idx), is.spinner_group, is.field_number }; } } for (const InstantiatedSpinner &is : button_spinners) { if (is.spinner->hasFocus()) { return FocusInfo{ int(is.bus_idx), is.spinner_group, is.field_number }; } } for (const InstantiatedSpinner &is : light_spinners) { if (is.spinner->hasFocus()) { return FocusInfo{ int(is.bus_idx), is.spinner_group, is.field_number }; } } return FocusInfo{ -1, SpinnerGroup::ALL_GROUPS, -1 }; } nageru-1.4.2/midi_mapping_dialog.h000066400000000000000000000130151301561706600171310ustar00rootroot00000000000000#ifndef _MIDI_MAPPING_DIALOG_H #define _MIDI_MAPPING_DIALOG_H #include #include #include #include #include #include #include #include #include "midi_mapper.h" class QEvent; class QObject; namespace Ui { class MIDIMappingDialog; } // namespace Ui class MIDIMappingProto; class QComboBox; class QSpinBox; class QTreeWidgetItem; class MIDIMappingDialog : public QDialog, public ControllerReceiver { Q_OBJECT public: MIDIMappingDialog(MIDIMapper *mapper); ~MIDIMappingDialog(); bool eventFilter(QObject *obj, QEvent *event) override; // For use in midi_mapping_dialog.cpp only. struct Control { std::string label; int field_number; // In MIDIMappingBusProto. int bank_field_number; // In MIDIMappingProto. }; // ControllerReceiver interface. We only implement the raw events. // All values are [0.0, 1.0]. void set_locut(float value) override {} void set_limiter_threshold(float value) override {} void set_makeup_gain(float value) override {} void set_treble(unsigned bus_idx, float value) override {} void set_mid(unsigned bus_idx, float value) override {} void set_bass(unsigned bus_idx, float value) override {} void set_gain(unsigned bus_idx, float value) override {} void set_compressor_threshold(unsigned bus_idx, float value) override {} void set_fader(unsigned bus_idx, float value) override {} void toggle_mute(unsigned bus_idx) override {} void toggle_locut(unsigned bus_idx) override {} void toggle_auto_gain_staging(unsigned bus_idx) override {} void toggle_compressor(unsigned bus_idx) override {} void clear_peak(unsigned bus_idx) override {} void toggle_limiter() override {} void toggle_auto_makeup_gain() override {} void clear_all_highlights() override {} void highlight_locut(bool highlight) override {} void highlight_limiter_threshold(bool highlight) override {} void highlight_makeup_gain(bool highlight) override {} void highlight_treble(unsigned bus_idx, bool highlight) override {} void highlight_mid(unsigned bus_idx, bool highlight) override {} void highlight_bass(unsigned bus_idx, bool highlight) override {} void highlight_gain(unsigned bus_idx, bool highlight) override {} void highlight_compressor_threshold(unsigned bus_idx, bool highlight) override {} void highlight_fader(unsigned bus_idx, bool highlight) override {} void highlight_mute(unsigned bus_idx, bool highlight) override {} void highlight_toggle_locut(unsigned bus_idx, bool highlight) override {} void highlight_toggle_auto_gain_staging(unsigned bus_idx, bool highlight) override {} void highlight_toggle_compressor(unsigned bus_idx, bool highlight) override {} void highlight_clear_peak(unsigned bus_idx, bool highlight) override {} void highlight_toggle_limiter(bool highlight) override {} void highlight_toggle_auto_makeup_gain(bool highlight) override {} // Raw events; used for the editor dialog only. void controller_changed(unsigned controller) override; void note_on(unsigned note) override; public slots: void guess_clicked(bool limit_to_group); void ok_clicked(); void cancel_clicked(); void save_clicked(); void load_clicked(); private: static constexpr unsigned num_buses = 8; // Each spinner belongs to exactly one group, corresponding to the // subheadings in the UI. This is so that we can extrapolate data // across only single groups if need be. enum class SpinnerGroup { ALL_GROUPS = -1, PER_BUS_CONTROLLERS, PER_BUS_BUTTONS, PER_BUS_LIGHTS, GLOBAL_CONTROLLERS, GLOBAL_BUTTONS, GLOBAL_LIGHTS }; void add_bank_selector(QTreeWidgetItem *item, const MIDIMappingProto &mapping_proto, int bank_field_number); enum class ControlType { CONTROLLER, BUTTON, LIGHT }; void add_controls(const std::string &heading, ControlType control_type, SpinnerGroup spinner_group, const MIDIMappingProto &mapping_proto, const std::vector &controls); void fill_controls_from_mapping(const MIDIMappingProto &mapping_proto); // Tries to find a source bus and an offset to it that would give // a consistent offset for the rest of the mappings in this bus. // Returns -1 for the bus if no consistent offset can be found. std::pair guess_offset(unsigned bus_idx, SpinnerGroup spinner_group); bool bus_is_empty(unsigned bus_idx, SpinnerGroup spinner_group); void update_guess_button_state(); struct FocusInfo { int bus_idx; // -1 for none. SpinnerGroup spinner_group; int field_number; }; FocusInfo find_focus() const; std::unique_ptr construct_mapping_proto_from_ui(); Ui::MIDIMappingDialog *ui; MIDIMapper *mapper; ControllerReceiver *old_receiver; FocusInfo last_focus{-1, SpinnerGroup::ALL_GROUPS, -1}; // All controllers actually laid out on the grid (we need to store them // so that we can move values back and forth between the controls and // the protobuf on save/load). struct InstantiatedSpinner { QSpinBox *spinner; unsigned bus_idx; SpinnerGroup spinner_group; int field_number; // In MIDIMappingBusProto. }; struct InstantiatedComboBox { QComboBox *combo_box; int field_number; // In MIDIMappingProto. }; std::vector controller_spinners; std::vector button_spinners; std::vector light_spinners; std::vector bank_combo_boxes; // Keyed on bus index, then field number. struct SpinnerAndGroup { QSpinBox *spinner; SpinnerGroup group; }; std::map> spinners; }; #endif // !defined(_MIDI_MAPPING_DIALOG_H) nageru-1.4.2/mixer.cpp000066400000000000000000001117601301561706600146420ustar00rootroot00000000000000#undef Success #include "mixer.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "DeckLinkAPI.h" #include "LinuxCOM.h" #include "alsa_output.h" #include "bmusb/bmusb.h" #include "bmusb/fake_capture.h" #include "context.h" #include "decklink_capture.h" #include "defs.h" #include "disk_space_estimator.h" #include "flags.h" #include "input_mapping.h" #include "pbo_frame_allocator.h" #include "ref_counted_gl_sync.h" #include "resampling_queue.h" #include "timebase.h" #include "video_encoder.h" class IDeckLink; class QOpenGLContext; using namespace movit; using namespace std; using namespace std::chrono; using namespace std::placeholders; using namespace bmusb; Mixer *global_mixer = nullptr; bool uses_mlock = false; namespace { void insert_new_frame(RefCountedFrame frame, unsigned field_num, bool interlaced, unsigned card_index, InputState *input_state) { if (interlaced) { for (unsigned frame_num = FRAME_HISTORY_LENGTH; frame_num --> 1; ) { // :-) input_state->buffered_frames[card_index][frame_num] = input_state->buffered_frames[card_index][frame_num - 1]; } input_state->buffered_frames[card_index][0] = { frame, field_num }; } else { for (unsigned frame_num = 0; frame_num < FRAME_HISTORY_LENGTH; ++frame_num) { input_state->buffered_frames[card_index][frame_num] = { frame, field_num }; } } } } // namespace void QueueLengthPolicy::update_policy(int queue_length) { if (queue_length < 0) { // Starvation. if (been_at_safe_point_since_last_starvation && safe_queue_length < 5) { ++safe_queue_length; fprintf(stderr, "Card %u: Starvation, increasing safe limit to %u frames\n", card_index, safe_queue_length); } frames_with_at_least_one = 0; been_at_safe_point_since_last_starvation = false; return; } if (queue_length > 0) { if (queue_length >= int(safe_queue_length)) { been_at_safe_point_since_last_starvation = true; } if (++frames_with_at_least_one >= 1000 && safe_queue_length > 0) { --safe_queue_length; fprintf(stderr, "Card %u: Spare frames for more than 1000 frames, reducing safe limit to %u frames\n", card_index, safe_queue_length); frames_with_at_least_one = 0; } } else { frames_with_at_least_one = 0; } } Mixer::Mixer(const QSurfaceFormat &format, unsigned num_cards) : httpd(), num_cards(num_cards), mixer_surface(create_surface(format)), h264_encoder_surface(create_surface(format)), audio_mixer(num_cards) { CHECK(init_movit(MOVIT_SHADER_DIR, MOVIT_DEBUG_OFF)); check_error(); // Since we allow non-bouncing 4:2:2 YCbCrInputs, effective subpixel precision // will be halved when sampling them, and we need to compensate here. movit_texel_subpixel_precision /= 2.0; resource_pool.reset(new ResourcePool); theme.reset(new Theme(global_flags.theme_filename, global_flags.theme_dirs, resource_pool.get(), num_cards)); for (unsigned i = 0; i < NUM_OUTPUTS; ++i) { output_channel[i].parent = this; output_channel[i].channel = i; } ImageFormat inout_format; inout_format.color_space = COLORSPACE_sRGB; inout_format.gamma_curve = GAMMA_sRGB; // Display chain; shows the live output produced by the main chain (its RGBA version). display_chain.reset(new EffectChain(WIDTH, HEIGHT, resource_pool.get())); check_error(); display_input = new FlatInput(inout_format, FORMAT_RGB, GL_UNSIGNED_BYTE, WIDTH, HEIGHT); // FIXME: GL_UNSIGNED_BYTE is really wrong. display_chain->add_input(display_input); display_chain->add_output(inout_format, OUTPUT_ALPHA_FORMAT_POSTMULTIPLIED); display_chain->set_dither_bits(0); // Don't bother. display_chain->finalize(); video_encoder.reset(new VideoEncoder(resource_pool.get(), h264_encoder_surface, global_flags.va_display, WIDTH, HEIGHT, &httpd, global_disk_space_estimator)); // Start listening for clients only once VideoEncoder has written its header, if any. httpd.start(9095); // First try initializing the then PCI devices, then USB, then // fill up with fake cards until we have the desired number of cards. unsigned num_pci_devices = 0; unsigned card_index = 0; { IDeckLinkIterator *decklink_iterator = CreateDeckLinkIteratorInstance(); if (decklink_iterator != nullptr) { for ( ; card_index < num_cards; ++card_index) { IDeckLink *decklink; if (decklink_iterator->Next(&decklink) != S_OK) { break; } configure_card(card_index, new DeckLinkCapture(decklink, card_index), /*is_fake_capture=*/false); ++num_pci_devices; } decklink_iterator->Release(); fprintf(stderr, "Found %u DeckLink PCI card(s).\n", num_pci_devices); } else { fprintf(stderr, "DeckLink drivers not found. Probing for USB cards only.\n"); } } unsigned num_usb_devices = BMUSBCapture::num_cards(); for (unsigned usb_card_index = 0; usb_card_index < num_usb_devices && card_index < num_cards; ++usb_card_index, ++card_index) { BMUSBCapture *capture = new BMUSBCapture(usb_card_index); capture->set_card_disconnected_callback(bind(&Mixer::bm_hotplug_remove, this, card_index)); configure_card(card_index, capture, /*is_fake_capture=*/false); } fprintf(stderr, "Found %u USB card(s).\n", num_usb_devices); unsigned num_fake_cards = 0; for ( ; card_index < num_cards; ++card_index, ++num_fake_cards) { FakeCapture *capture = new FakeCapture(WIDTH, HEIGHT, FAKE_FPS, OUTPUT_FREQUENCY, card_index, global_flags.fake_cards_audio); configure_card(card_index, capture, /*is_fake_capture=*/true); } if (num_fake_cards > 0) { fprintf(stderr, "Initialized %u fake cards.\n", num_fake_cards); } BMUSBCapture::set_card_connected_callback(bind(&Mixer::bm_hotplug_add, this, _1)); BMUSBCapture::start_bm_thread(); for (card_index = 0; card_index < num_cards; ++card_index) { cards[card_index].queue_length_policy.reset(card_index); cards[card_index].capture->start_bm_capture(); } // Set up stuff for NV12 conversion. // Cb/Cr shader. string cbcr_vert_shader = "#version 130 \n" " \n" "in vec2 position; \n" "in vec2 texcoord; \n" "out vec2 tc0; \n" "uniform vec2 foo_chroma_offset_0; \n" " \n" "void main() \n" "{ \n" " // The result of glOrtho(0.0, 1.0, 0.0, 1.0, 0.0, 1.0) is: \n" " // \n" " // 2.000 0.000 0.000 -1.000 \n" " // 0.000 2.000 0.000 -1.000 \n" " // 0.000 0.000 -2.000 -1.000 \n" " // 0.000 0.000 0.000 1.000 \n" " gl_Position = vec4(2.0 * position.x - 1.0, 2.0 * position.y - 1.0, -1.0, 1.0); \n" " vec2 flipped_tc = texcoord; \n" " tc0 = flipped_tc + foo_chroma_offset_0; \n" "} \n"; string cbcr_frag_shader = "#version 130 \n" "in vec2 tc0; \n" "uniform sampler2D cbcr_tex; \n" "out vec4 FragColor; \n" "void main() { \n" " FragColor = texture(cbcr_tex, tc0); \n" "} \n"; vector frag_shader_outputs; cbcr_program_num = resource_pool->compile_glsl_program(cbcr_vert_shader, cbcr_frag_shader, frag_shader_outputs); float vertices[] = { 0.0f, 2.0f, 0.0f, 0.0f, 2.0f, 0.0f }; cbcr_vbo = generate_vbo(2, GL_FLOAT, sizeof(vertices), vertices); cbcr_position_attribute_index = glGetAttribLocation(cbcr_program_num, "position"); cbcr_texcoord_attribute_index = glGetAttribLocation(cbcr_program_num, "texcoord"); if (global_flags.enable_alsa_output) { alsa.reset(new ALSAOutput(OUTPUT_FREQUENCY, /*num_channels=*/2)); } } Mixer::~Mixer() { resource_pool->release_glsl_program(cbcr_program_num); glDeleteBuffers(1, &cbcr_vbo); BMUSBCapture::stop_bm_thread(); for (unsigned card_index = 0; card_index < num_cards; ++card_index) { { unique_lock lock(bmusb_mutex); cards[card_index].should_quit = true; // Unblock thread. cards[card_index].new_frames_changed.notify_all(); } cards[card_index].capture->stop_dequeue_thread(); } video_encoder.reset(nullptr); } void Mixer::configure_card(unsigned card_index, CaptureInterface *capture, bool is_fake_capture) { printf("Configuring card %d...\n", card_index); CaptureCard *card = &cards[card_index]; if (card->capture != nullptr) { card->capture->stop_dequeue_thread(); delete card->capture; } card->capture = capture; card->is_fake_capture = is_fake_capture; card->capture->set_frame_callback(bind(&Mixer::bm_frame, this, card_index, _1, _2, _3, _4, _5, _6, _7)); if (card->frame_allocator == nullptr) { card->frame_allocator.reset(new PBOFrameAllocator(8 << 20, WIDTH, HEIGHT)); // 8 MB. } card->capture->set_video_frame_allocator(card->frame_allocator.get()); if (card->surface == nullptr) { card->surface = create_surface_with_same_format(mixer_surface); } while (!card->new_frames.empty()) card->new_frames.pop(); card->fractional_samples = 0; card->last_timecode = -1; card->capture->configure_card(); DeviceSpec device{InputSourceType::CAPTURE_CARD, card_index}; audio_mixer.reset_resampler(device); audio_mixer.set_display_name(device, card->capture->get_description()); audio_mixer.trigger_state_changed_callback(); } namespace { int unwrap_timecode(uint16_t current_wrapped, int last) { uint16_t last_wrapped = last & 0xffff; if (current_wrapped > last_wrapped) { return (last & ~0xffff) | current_wrapped; } else { return 0x10000 + ((last & ~0xffff) | current_wrapped); } } } // namespace void Mixer::bm_frame(unsigned card_index, uint16_t timecode, FrameAllocator::Frame video_frame, size_t video_offset, VideoFormat video_format, FrameAllocator::Frame audio_frame, size_t audio_offset, AudioFormat audio_format) { DeviceSpec device{InputSourceType::CAPTURE_CARD, card_index}; CaptureCard *card = &cards[card_index]; if (is_mode_scanning[card_index]) { if (video_format.has_signal) { // Found a stable signal, so stop scanning. is_mode_scanning[card_index] = false; } else { static constexpr double switch_time_s = 0.5; // Should be enough time for the signal to stabilize. steady_clock::time_point now = steady_clock::now(); double sec_since_last_switch = duration(steady_clock::now() - last_mode_scan_change[card_index]).count(); if (sec_since_last_switch > switch_time_s) { // It isn't this mode; try the next one. mode_scanlist_index[card_index]++; mode_scanlist_index[card_index] %= mode_scanlist[card_index].size(); cards[card_index].capture->set_video_mode(mode_scanlist[card_index][mode_scanlist_index[card_index]]); last_mode_scan_change[card_index] = now; } } } int64_t frame_length = int64_t(TIMEBASE) * video_format.frame_rate_den / video_format.frame_rate_nom; assert(frame_length > 0); size_t num_samples = (audio_frame.len > audio_offset) ? (audio_frame.len - audio_offset) / audio_format.num_channels / (audio_format.bits_per_sample / 8) : 0; if (num_samples > OUTPUT_FREQUENCY / 10) { printf("Card %d: Dropping frame with implausible audio length (len=%d, offset=%d) [timecode=0x%04x video_len=%d video_offset=%d video_format=%x)\n", card_index, int(audio_frame.len), int(audio_offset), timecode, int(video_frame.len), int(video_offset), video_format.id); if (video_frame.owner) { video_frame.owner->release_frame(video_frame); } if (audio_frame.owner) { audio_frame.owner->release_frame(audio_frame); } return; } int dropped_frames = 0; if (card->last_timecode != -1) { dropped_frames = unwrap_timecode(timecode, card->last_timecode) - card->last_timecode - 1; } // Number of samples per frame if we need to insert silence. // (Could be nonintegral, but resampling will save us then.) const int silence_samples = OUTPUT_FREQUENCY * video_format.frame_rate_den / video_format.frame_rate_nom; if (dropped_frames > MAX_FPS * 2) { fprintf(stderr, "Card %d lost more than two seconds (or time code jumping around; from 0x%04x to 0x%04x), resetting resampler\n", card_index, card->last_timecode, timecode); audio_mixer.reset_resampler(device); dropped_frames = 0; } else if (dropped_frames > 0) { // Insert silence as needed. fprintf(stderr, "Card %d dropped %d frame(s) (before timecode 0x%04x), inserting silence.\n", card_index, dropped_frames, timecode); bool success; do { success = audio_mixer.add_silence(device, silence_samples, dropped_frames, frame_length); } while (!success); } audio_mixer.add_audio(device, audio_frame.data + audio_offset, num_samples, audio_format, frame_length); // Done with the audio, so release it. if (audio_frame.owner) { audio_frame.owner->release_frame(audio_frame); } card->last_timecode = timecode; size_t expected_length = video_format.width * (video_format.height + video_format.extra_lines_top + video_format.extra_lines_bottom) * 2; if (video_frame.len - video_offset == 0 || video_frame.len - video_offset != expected_length) { if (video_frame.len != 0) { printf("Card %d: Dropping video frame with wrong length (%ld; expected %ld)\n", card_index, video_frame.len - video_offset, expected_length); } if (video_frame.owner) { video_frame.owner->release_frame(video_frame); } // Still send on the information that we _had_ a frame, even though it's corrupted, // so that pts can go up accordingly. { unique_lock lock(bmusb_mutex); CaptureCard::NewFrame new_frame; new_frame.frame = RefCountedFrame(FrameAllocator::Frame()); new_frame.length = frame_length; new_frame.interlaced = false; new_frame.dropped_frames = dropped_frames; card->new_frames.push(move(new_frame)); card->new_frames_changed.notify_all(); } return; } PBOFrameAllocator::Userdata *userdata = (PBOFrameAllocator::Userdata *)video_frame.userdata; unsigned num_fields = video_format.interlaced ? 2 : 1; steady_clock::time_point frame_upload_start; if (video_format.interlaced) { // Send the two fields along as separate frames; the other side will need to add // a deinterlacer to actually get this right. assert(video_format.height % 2 == 0); video_format.height /= 2; assert(frame_length % 2 == 0); frame_length /= 2; num_fields = 2; frame_upload_start = steady_clock::now(); } userdata->last_interlaced = video_format.interlaced; userdata->last_has_signal = video_format.has_signal; userdata->last_is_connected = video_format.is_connected; userdata->last_frame_rate_nom = video_format.frame_rate_nom; userdata->last_frame_rate_den = video_format.frame_rate_den; RefCountedFrame frame(video_frame); // Upload the textures. size_t cbcr_width = video_format.width / 2; size_t cbcr_offset = video_offset / 2; size_t y_offset = video_frame.size / 2 + video_offset / 2; for (unsigned field = 0; field < num_fields; ++field) { // Put the actual texture upload in a lambda that is executed in the main thread. // It is entirely possible to do this in the same thread (and it might even be // faster, depending on the GPU and driver), but it appears to be trickling // driver bugs very easily. // // Note that this means we must hold on to the actual frame data in // until the upload command is run, but we hold on to much longer than that // (in fact, all the way until we no longer use the texture in rendering). auto upload_func = [field, video_format, y_offset, cbcr_offset, cbcr_width, userdata]() { unsigned field_start_line = (field == 1) ? video_format.second_field_start : video_format.extra_lines_top + field * (video_format.height + 22); if (userdata->tex_y[field] == 0 || userdata->tex_cbcr[field] == 0 || video_format.width != userdata->last_width[field] || video_format.height != userdata->last_height[field]) { // We changed resolution since last use of this texture, so we need to create // a new object. Note that this each card has its own PBOFrameAllocator, // we don't need to worry about these flip-flopping between resolutions. glBindTexture(GL_TEXTURE_2D, userdata->tex_cbcr[field]); check_error(); glTexImage2D(GL_TEXTURE_2D, 0, GL_RG8, cbcr_width, video_format.height, 0, GL_RG, GL_UNSIGNED_BYTE, nullptr); check_error(); glBindTexture(GL_TEXTURE_2D, userdata->tex_y[field]); check_error(); glTexImage2D(GL_TEXTURE_2D, 0, GL_R8, video_format.width, video_format.height, 0, GL_RED, GL_UNSIGNED_BYTE, nullptr); check_error(); userdata->last_width[field] = video_format.width; userdata->last_height[field] = video_format.height; } GLuint pbo = userdata->pbo; check_error(); glBindBuffer(GL_PIXEL_UNPACK_BUFFER, pbo); check_error(); size_t field_y_start = y_offset + video_format.width * field_start_line; size_t field_cbcr_start = cbcr_offset + cbcr_width * field_start_line * sizeof(uint16_t); if (global_flags.flush_pbos) { glFlushMappedBufferRange(GL_PIXEL_UNPACK_BUFFER, field_y_start, video_format.width * video_format.height); check_error(); glFlushMappedBufferRange(GL_PIXEL_UNPACK_BUFFER, field_cbcr_start, cbcr_width * video_format.height * sizeof(uint16_t)); check_error(); } glBindTexture(GL_TEXTURE_2D, userdata->tex_cbcr[field]); check_error(); glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, cbcr_width, video_format.height, GL_RG, GL_UNSIGNED_BYTE, BUFFER_OFFSET(field_cbcr_start)); check_error(); glBindTexture(GL_TEXTURE_2D, userdata->tex_y[field]); check_error(); glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, video_format.width, video_format.height, GL_RED, GL_UNSIGNED_BYTE, BUFFER_OFFSET(field_y_start)); check_error(); glBindTexture(GL_TEXTURE_2D, 0); check_error(); glBindBuffer(GL_PIXEL_UNPACK_BUFFER, 0); check_error(); }; if (field == 1) { // Don't upload the second field as fast as we can; wait until // the field time has approximately passed. (Otherwise, we could // get timing jitter against the other sources, and possibly also // against the video display, although the latter is not as critical.) // This requires our system clock to be reasonably close to the // video clock, but that's not an unreasonable assumption. steady_clock::time_point second_field_start = frame_upload_start + nanoseconds(frame_length * 1000000000 / TIMEBASE); this_thread::sleep_until(second_field_start); } { unique_lock lock(bmusb_mutex); CaptureCard::NewFrame new_frame; new_frame.frame = frame; new_frame.length = frame_length; new_frame.field = field; new_frame.interlaced = video_format.interlaced; new_frame.upload_func = upload_func; new_frame.dropped_frames = dropped_frames; card->new_frames.push(move(new_frame)); card->new_frames_changed.notify_all(); } } } void Mixer::bm_hotplug_add(libusb_device *dev) { lock_guard lock(hotplug_mutex); hotplugged_cards.push_back(dev); } void Mixer::bm_hotplug_remove(unsigned card_index) { cards[card_index].new_frames_changed.notify_all(); } void Mixer::thread_func() { eglBindAPI(EGL_OPENGL_API); QOpenGLContext *context = create_context(mixer_surface); if (!make_current(context, mixer_surface)) { printf("oops\n"); exit(1); } steady_clock::time_point start, now; start = steady_clock::now(); int frame = 0; int stats_dropped_frames = 0; while (!should_quit) { CaptureCard::NewFrame new_frames[MAX_VIDEO_CARDS]; bool has_new_frame[MAX_VIDEO_CARDS] = { false }; int num_samples[MAX_VIDEO_CARDS] = { 0 }; unsigned master_card_index = theme->map_signal(master_clock_channel); assert(master_card_index < num_cards); get_one_frame_from_each_card(master_card_index, new_frames, has_new_frame, num_samples); schedule_audio_resampling_tasks(new_frames[master_card_index].dropped_frames, num_samples[master_card_index], new_frames[master_card_index].length); stats_dropped_frames += new_frames[master_card_index].dropped_frames; handle_hotplugged_cards(); for (unsigned card_index = 0; card_index < num_cards; ++card_index) { if (card_index == master_card_index || !has_new_frame[card_index]) { continue; } if (new_frames[card_index].frame->len == 0) { ++new_frames[card_index].dropped_frames; } if (new_frames[card_index].dropped_frames > 0) { printf("Card %u dropped %d frames before this\n", card_index, int(new_frames[card_index].dropped_frames)); } } // If the first card is reporting a corrupted or otherwise dropped frame, // just increase the pts (skipping over this frame) and don't try to compute anything new. if (new_frames[master_card_index].frame->len == 0) { ++stats_dropped_frames; pts_int += new_frames[master_card_index].length; continue; } for (unsigned card_index = 0; card_index < num_cards; ++card_index) { if (!has_new_frame[card_index] || new_frames[card_index].frame->len == 0) continue; CaptureCard::NewFrame *new_frame = &new_frames[card_index]; assert(new_frame->frame != nullptr); insert_new_frame(new_frame->frame, new_frame->field, new_frame->interlaced, card_index, &input_state); check_error(); // The new texture might need uploading before use. if (new_frame->upload_func) { new_frame->upload_func(); new_frame->upload_func = nullptr; } } int64_t frame_duration = new_frames[master_card_index].length; render_one_frame(frame_duration); ++frame; pts_int += frame_duration; now = steady_clock::now(); double elapsed = duration(now - start).count(); if (frame % 100 == 0) { printf("%d frames (%d dropped) in %.3f seconds = %.1f fps (%.1f ms/frame)", frame, stats_dropped_frames, elapsed, frame / elapsed, 1e3 * elapsed / frame); // chain->print_phase_timing(); // Check our memory usage, to see if we are close to our mlockall() // limit (if at all set). rusage used; if (getrusage(RUSAGE_SELF, &used) == -1) { perror("getrusage(RUSAGE_SELF)"); assert(false); } if (uses_mlock) { rlimit limit; if (getrlimit(RLIMIT_MEMLOCK, &limit) == -1) { perror("getrlimit(RLIMIT_MEMLOCK)"); assert(false); } printf(", using %ld / %ld MB lockable memory (%.1f%%)", long(used.ru_maxrss / 1024), long(limit.rlim_cur / 1048576), float(100.0 * (used.ru_maxrss * 1024.0) / limit.rlim_cur)); } else { printf(", using %ld MB memory (not locked)", long(used.ru_maxrss / 1024)); } printf("\n"); } if (should_cut.exchange(false)) { // Test and clear. video_encoder->do_cut(frame); } #if 0 // Reset every 100 frames, so that local variations in frame times // (especially for the first few frames, when the shaders are // compiled etc.) don't make it hard to measure for the entire // remaining duration of the program. if (frame == 10000) { frame = 0; start = now; } #endif check_error(); } resource_pool->clean_context(); } void Mixer::get_one_frame_from_each_card(unsigned master_card_index, CaptureCard::NewFrame new_frames[MAX_VIDEO_CARDS], bool has_new_frame[MAX_VIDEO_CARDS], int num_samples[MAX_VIDEO_CARDS]) { start: // The first card is the master timer, so wait for it to have a new frame. // TODO: Add a timeout. unique_lock lock(bmusb_mutex); cards[master_card_index].new_frames_changed.wait(lock, [this, master_card_index]{ return !cards[master_card_index].new_frames.empty() || cards[master_card_index].capture->get_disconnected(); }); if (cards[master_card_index].new_frames.empty()) { // We were woken up, but not due to a new frame. Deal with it // and then restart. assert(cards[master_card_index].capture->get_disconnected()); handle_hotplugged_cards(); goto start; } for (unsigned card_index = 0; card_index < num_cards; ++card_index) { CaptureCard *card = &cards[card_index]; if (card->new_frames.empty()) { assert(card_index != master_card_index); card->queue_length_policy.update_policy(-1); continue; } new_frames[card_index] = move(card->new_frames.front()); has_new_frame[card_index] = true; card->new_frames.pop(); card->new_frames_changed.notify_all(); int num_samples_times_timebase = OUTPUT_FREQUENCY * new_frames[card_index].length + card->fractional_samples; num_samples[card_index] = num_samples_times_timebase / TIMEBASE; card->fractional_samples = num_samples_times_timebase % TIMEBASE; assert(num_samples[card_index] >= 0); if (card_index == master_card_index) { // We don't use the queue length policy for the master card, // but we will if it stops being the master. Thus, clear out // the policy in case we switch in the future. card->queue_length_policy.reset(card_index); } else { // If we have excess frames compared to the policy for this card, // drop frames from the head. card->queue_length_policy.update_policy(card->new_frames.size()); while (card->new_frames.size() > card->queue_length_policy.get_safe_queue_length()) { card->new_frames.pop(); } } } } void Mixer::handle_hotplugged_cards() { // Check for cards that have been disconnected since last frame. for (unsigned card_index = 0; card_index < num_cards; ++card_index) { CaptureCard *card = &cards[card_index]; if (card->capture->get_disconnected()) { fprintf(stderr, "Card %u went away, replacing with a fake card.\n", card_index); FakeCapture *capture = new FakeCapture(WIDTH, HEIGHT, FAKE_FPS, OUTPUT_FREQUENCY, card_index, global_flags.fake_cards_audio); configure_card(card_index, capture, /*is_fake_capture=*/true); card->queue_length_policy.reset(card_index); card->capture->start_bm_capture(); } } // Check for cards that have been connected since last frame. vector hotplugged_cards_copy; { lock_guard lock(hotplug_mutex); swap(hotplugged_cards, hotplugged_cards_copy); } for (libusb_device *new_dev : hotplugged_cards_copy) { // Look for a fake capture card where we can stick this in. int free_card_index = -1; for (unsigned card_index = 0; card_index < num_cards; ++card_index) { if (cards[card_index].is_fake_capture) { free_card_index = int(card_index); break; } } if (free_card_index == -1) { fprintf(stderr, "New card plugged in, but no free slots -- ignoring.\n"); libusb_unref_device(new_dev); } else { // BMUSBCapture takes ownership. fprintf(stderr, "New card plugged in, choosing slot %d.\n", free_card_index); CaptureCard *card = &cards[free_card_index]; BMUSBCapture *capture = new BMUSBCapture(free_card_index, new_dev); configure_card(free_card_index, capture, /*is_fake_capture=*/false); card->queue_length_policy.reset(free_card_index); capture->set_card_disconnected_callback(bind(&Mixer::bm_hotplug_remove, this, free_card_index)); capture->start_bm_capture(); } } } void Mixer::schedule_audio_resampling_tasks(unsigned dropped_frames, int num_samples_per_frame, int length_per_frame) { // Resample the audio as needed, including from previously dropped frames. assert(num_cards > 0); for (unsigned frame_num = 0; frame_num < dropped_frames + 1; ++frame_num) { const bool dropped_frame = (frame_num != dropped_frames); { // Signal to the audio thread to process this frame. // Note that if the frame is a dropped frame, we signal that // we don't want to use this frame as base for adjusting // the resampler rate. The reason for this is that the timing // of these frames is often way too late; they typically don't // “arrive” before we synthesize them. Thus, we could end up // in a situation where we have inserted e.g. five audio frames // into the queue before we then start pulling five of them // back out. This makes ResamplingQueue overestimate the delay, // causing undue resampler changes. (We _do_ use the last, // non-dropped frame; perhaps we should just discard that as well, // since dropped frames are expected to be rare, and it might be // better to just wait until we have a slightly more normal situation). unique_lock lock(audio_mutex); bool adjust_rate = !dropped_frame; audio_task_queue.push(AudioTask{pts_int, num_samples_per_frame, adjust_rate}); audio_task_queue_changed.notify_one(); } if (dropped_frame) { // For dropped frames, increase the pts. Note that if the format changed // in the meantime, we have no way of detecting that; we just have to // assume the frame length is always the same. pts_int += length_per_frame; } } } void Mixer::render_one_frame(int64_t duration) { // Get the main chain from the theme, and set its state immediately. Theme::Chain theme_main_chain = theme->get_chain(0, pts(), WIDTH, HEIGHT, input_state); EffectChain *chain = theme_main_chain.chain; theme_main_chain.setup_chain(); //theme_main_chain.chain->enable_phase_timing(true); GLuint y_tex, cbcr_tex; bool got_frame = video_encoder->begin_frame(&y_tex, &cbcr_tex); assert(got_frame); // Render main chain. GLuint cbcr_full_tex = resource_pool->create_2d_texture(GL_RG8, WIDTH, HEIGHT); GLuint rgba_tex = resource_pool->create_2d_texture(GL_RGB565, WIDTH, HEIGHT); // Saves texture bandwidth, although dithering gets messed up. GLuint fbo = resource_pool->create_fbo(y_tex, cbcr_full_tex, rgba_tex); check_error(); chain->render_to_fbo(fbo, WIDTH, HEIGHT); resource_pool->release_fbo(fbo); subsample_chroma(cbcr_full_tex, cbcr_tex); resource_pool->release_2d_texture(cbcr_full_tex); // Set the right state for rgba_tex. glBindFramebuffer(GL_FRAMEBUFFER, 0); glBindTexture(GL_TEXTURE_2D, rgba_tex); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); const int64_t av_delay = TIMEBASE / 10; // Corresponds to the fixed delay in resampling_queue.h. TODO: Make less hard-coded. RefCountedGLsync fence = video_encoder->end_frame(pts_int + av_delay, duration, theme_main_chain.input_frames); // The live frame just shows the RGBA texture we just rendered. // It owns rgba_tex now. DisplayFrame live_frame; live_frame.chain = display_chain.get(); live_frame.setup_chain = [this, rgba_tex]{ display_input->set_texture_num(rgba_tex); }; live_frame.ready_fence = fence; live_frame.input_frames = {}; live_frame.temp_textures = { rgba_tex }; output_channel[OUTPUT_LIVE].output_frame(live_frame); // Set up preview and any additional channels. for (int i = 1; i < theme->get_num_channels() + 2; ++i) { DisplayFrame display_frame; Theme::Chain chain = theme->get_chain(i, pts(), WIDTH, HEIGHT, input_state); // FIXME: dimensions display_frame.chain = chain.chain; display_frame.setup_chain = chain.setup_chain; display_frame.ready_fence = fence; display_frame.input_frames = chain.input_frames; display_frame.temp_textures = {}; output_channel[i].output_frame(display_frame); } } void Mixer::audio_thread_func() { while (!should_quit) { AudioTask task; { unique_lock lock(audio_mutex); audio_task_queue_changed.wait(lock, [this]{ return should_quit || !audio_task_queue.empty(); }); if (should_quit) { return; } task = audio_task_queue.front(); audio_task_queue.pop(); } ResamplingQueue::RateAdjustmentPolicy rate_adjustment_policy = task.adjust_rate ? ResamplingQueue::ADJUST_RATE : ResamplingQueue::DO_NOT_ADJUST_RATE; vector samples_out = audio_mixer.get_output( double(task.pts_int) / TIMEBASE, task.num_samples, rate_adjustment_policy); // Send the samples to the sound card, then add them to the output. if (alsa) { alsa->write(samples_out); } video_encoder->add_audio(task.pts_int, move(samples_out)); } } void Mixer::subsample_chroma(GLuint src_tex, GLuint dst_tex) { GLuint vao; glGenVertexArrays(1, &vao); check_error(); glBindVertexArray(vao); check_error(); // Extract Cb/Cr. GLuint fbo = resource_pool->create_fbo(dst_tex); glBindFramebuffer(GL_FRAMEBUFFER, fbo); glViewport(0, 0, WIDTH/2, HEIGHT/2); check_error(); glUseProgram(cbcr_program_num); check_error(); glActiveTexture(GL_TEXTURE0); check_error(); glBindTexture(GL_TEXTURE_2D, src_tex); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); check_error(); float chroma_offset_0[] = { -0.5f / WIDTH, 0.0f }; set_uniform_vec2(cbcr_program_num, "foo", "chroma_offset_0", chroma_offset_0); glBindBuffer(GL_ARRAY_BUFFER, cbcr_vbo); check_error(); for (GLint attr_index : { cbcr_position_attribute_index, cbcr_texcoord_attribute_index }) { glEnableVertexAttribArray(attr_index); check_error(); glVertexAttribPointer(attr_index, 2, GL_FLOAT, GL_FALSE, 0, BUFFER_OFFSET(0)); check_error(); } glDrawArrays(GL_TRIANGLES, 0, 3); check_error(); for (GLint attr_index : { cbcr_position_attribute_index, cbcr_texcoord_attribute_index }) { glDisableVertexAttribArray(attr_index); check_error(); } glUseProgram(0); check_error(); glBindFramebuffer(GL_FRAMEBUFFER, 0); check_error(); resource_pool->release_fbo(fbo); glDeleteVertexArrays(1, &vao); } void Mixer::release_display_frame(DisplayFrame *frame) { for (GLuint texnum : frame->temp_textures) { resource_pool->release_2d_texture(texnum); } frame->temp_textures.clear(); frame->ready_fence.reset(); frame->input_frames.clear(); } void Mixer::start() { mixer_thread = thread(&Mixer::thread_func, this); audio_thread = thread(&Mixer::audio_thread_func, this); } void Mixer::quit() { should_quit = true; audio_task_queue_changed.notify_one(); mixer_thread.join(); audio_thread.join(); } void Mixer::transition_clicked(int transition_num) { theme->transition_clicked(transition_num, pts()); } void Mixer::channel_clicked(int preview_num) { theme->channel_clicked(preview_num); } void Mixer::start_mode_scanning(unsigned card_index) { assert(card_index < num_cards); if (is_mode_scanning[card_index]) { return; } is_mode_scanning[card_index] = true; mode_scanlist[card_index].clear(); for (const auto &mode : cards[card_index].capture->get_available_video_modes()) { mode_scanlist[card_index].push_back(mode.first); } assert(!mode_scanlist[card_index].empty()); mode_scanlist_index[card_index] = 0; cards[card_index].capture->set_video_mode(mode_scanlist[card_index][0]); last_mode_scan_change[card_index] = steady_clock::now(); } Mixer::OutputChannel::~OutputChannel() { if (has_current_frame) { parent->release_display_frame(¤t_frame); } if (has_ready_frame) { parent->release_display_frame(&ready_frame); } } void Mixer::OutputChannel::output_frame(DisplayFrame frame) { // Store this frame for display. Remove the ready frame if any // (it was seemingly never used). { unique_lock lock(frame_mutex); if (has_ready_frame) { parent->release_display_frame(&ready_frame); } ready_frame = frame; has_ready_frame = true; } if (new_frame_ready_callback) { new_frame_ready_callback(); } // Reduce the number of callbacks by filtering duplicates. The reason // why we bother doing this is that Qt seemingly can get into a state // where its builds up an essentially unbounded queue of signals, // consuming more and more memory, and there's no good way of collapsing // user-defined signals or limiting the length of the queue. if (transition_names_updated_callback) { vector transition_names = global_mixer->get_transition_names(); bool changed = false; if (transition_names.size() != last_transition_names.size()) { changed = true; } else { for (unsigned i = 0; i < transition_names.size(); ++i) { if (transition_names[i] != last_transition_names[i]) { changed = true; break; } } } if (changed) { transition_names_updated_callback(transition_names); last_transition_names = transition_names; } } if (name_updated_callback) { string name = global_mixer->get_channel_name(channel); if (name != last_name) { name_updated_callback(name); last_name = name; } } if (color_updated_callback) { string color = global_mixer->get_channel_color(channel); if (color != last_color) { color_updated_callback(color); last_color = color; } } } bool Mixer::OutputChannel::get_display_frame(DisplayFrame *frame) { unique_lock lock(frame_mutex); if (!has_current_frame && !has_ready_frame) { return false; } if (has_current_frame && has_ready_frame) { // We have a new ready frame. Toss the current one. parent->release_display_frame(¤t_frame); has_current_frame = false; } if (has_ready_frame) { assert(!has_current_frame); current_frame = ready_frame; ready_frame.ready_fence.reset(); // Drop the refcount. ready_frame.input_frames.clear(); // Drop the refcounts. has_current_frame = true; has_ready_frame = false; } *frame = current_frame; return true; } void Mixer::OutputChannel::set_frame_ready_callback(Mixer::new_frame_ready_callback_t callback) { new_frame_ready_callback = callback; } void Mixer::OutputChannel::set_transition_names_updated_callback(Mixer::transition_names_updated_callback_t callback) { transition_names_updated_callback = callback; } void Mixer::OutputChannel::set_name_updated_callback(Mixer::name_updated_callback_t callback) { name_updated_callback = callback; } void Mixer::OutputChannel::set_color_updated_callback(Mixer::color_updated_callback_t callback) { color_updated_callback = callback; } mutex RefCountedGLsync::fence_lock; nageru-1.4.2/mixer.h000066400000000000000000000327201301561706600143050ustar00rootroot00000000000000#ifndef _MIXER_H #define _MIXER_H 1 // The actual video mixer, running in its own separate background thread. #include #include #undef Success #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "audio_mixer.h" #include "bmusb/bmusb.h" #include "defs.h" #include "httpd.h" #include "input_state.h" #include "libusb.h" #include "pbo_frame_allocator.h" #include "ref_counted_frame.h" #include "ref_counted_gl_sync.h" #include "theme.h" #include "timebase.h" #include "video_encoder.h" class ALSAOutput; class QSurface; namespace movit { class Effect; class EffectChain; class FlatInput; class ResourcePool; } // namespace movit class QSurfaceFormat; // For any card that's not the master (where we pick out the frames as they // come, as fast as we can process), there's going to be a queue. The question // is when we should drop frames from that queue (apart from the obvious // dropping if the 16-frame queue should become full), especially given that // the frame rate could be lower or higher than the master (either subtly or // dramatically). We have two (conflicting) demands: // // 1. We want to avoid starving the queue. // 2. We don't want to add more delay than is needed. // // Our general strategy is to drop as many frames as we can (helping for #2) // that we think is safe for #1 given jitter. To this end, we set a lower floor N, // where we assume that if we have N frames in the queue, we're always safe from // starvation. (Typically, N will be 0 or 1. It starts off at 0.) If we have // more than N frames in the queue after reading out the one we need, we head-drop // them to reduce the queue. // // N is reduced as follows: If the queue has had at least one spare frame for // at least 50 (master) frames (ie., it's been too conservative for a second), // we reduce N by 1 and reset the timers. TODO: Only do this if N ever actually // touched the limit. // // Whenever the queue is starved (we needed a frame but there was none), // and we've been at N since the last starvation, N was obviously too low, // so we increment it. We will never set N above 5, though. class QueueLengthPolicy { public: QueueLengthPolicy() {} void reset(unsigned card_index) { this->card_index = card_index; safe_queue_length = 0; frames_with_at_least_one = 0; been_at_safe_point_since_last_starvation = false; } void update_policy(int queue_length); // Give in -1 for starvation. unsigned get_safe_queue_length() const { return safe_queue_length; } private: unsigned card_index; // For debugging only. unsigned safe_queue_length = 0; // Called N in the comments. unsigned frames_with_at_least_one = 0; bool been_at_safe_point_since_last_starvation = false; }; class Mixer { public: // The surface format is used for offscreen destinations for OpenGL contexts we need. Mixer(const QSurfaceFormat &format, unsigned num_cards); ~Mixer(); void start(); void quit(); void transition_clicked(int transition_num); void channel_clicked(int preview_num); enum Output { OUTPUT_LIVE = 0, OUTPUT_PREVIEW, OUTPUT_INPUT0, // 1, 2, 3, up to 15 follow numerically. NUM_OUTPUTS = 18 }; struct DisplayFrame { // The chain for rendering this frame. To render a display frame, // first wait for , then call // to wire up all the inputs, and then finally call // chain->render_to_screen() or similar. movit::EffectChain *chain; std::function setup_chain; // Asserted when all the inputs are ready; you cannot render the chain // before this. RefCountedGLsync ready_fence; // Holds on to all the input frames needed for this display frame, // so they are not released while still rendering. std::vector input_frames; // Textures that should be released back to the resource pool // when this frame disappears, if any. // TODO: Refcount these as well? std::vector temp_textures; }; // Implicitly frees the previous one if there's a new frame available. bool get_display_frame(Output output, DisplayFrame *frame) { return output_channel[output].get_display_frame(frame); } typedef std::function new_frame_ready_callback_t; void set_frame_ready_callback(Output output, new_frame_ready_callback_t callback) { output_channel[output].set_frame_ready_callback(callback); } // TODO: Should this really be per-channel? Shouldn't it just be called for e.g. the live output? typedef std::function &)> transition_names_updated_callback_t; void set_transition_names_updated_callback(Output output, transition_names_updated_callback_t callback) { output_channel[output].set_transition_names_updated_callback(callback); } typedef std::function name_updated_callback_t; void set_name_updated_callback(Output output, name_updated_callback_t callback) { output_channel[output].set_name_updated_callback(callback); } typedef std::function color_updated_callback_t; void set_color_updated_callback(Output output, color_updated_callback_t callback) { output_channel[output].set_color_updated_callback(callback); } std::vector get_transition_names() { return theme->get_transition_names(pts()); } unsigned get_num_channels() const { return theme->get_num_channels(); } std::string get_channel_name(unsigned channel) const { return theme->get_channel_name(channel); } std::string get_channel_color(unsigned channel) const { return theme->get_channel_color(channel); } int get_channel_signal(unsigned channel) const { return theme->get_channel_signal(channel); } int map_signal(unsigned channel) { return theme->map_signal(channel); } unsigned get_master_clock() const { return master_clock_channel; } void set_master_clock(unsigned channel) { master_clock_channel = channel; } void set_signal_mapping(int signal, int card) { return theme->set_signal_mapping(signal, card); } bool get_supports_set_wb(unsigned channel) const { return theme->get_supports_set_wb(channel); } void set_wb(unsigned channel, double r, double g, double b) const { theme->set_wb(channel, r, g, b); } // Note: You can also get this through the global variable global_audio_mixer. AudioMixer *get_audio_mixer() { return &audio_mixer; } const AudioMixer *get_audio_mixer() const { return &audio_mixer; } void schedule_cut() { should_cut = true; } unsigned get_num_cards() const { return num_cards; } std::string get_card_description(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_description(); } std::map get_available_video_modes(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_available_video_modes(); } uint32_t get_current_video_mode(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_current_video_mode(); } void set_video_mode(unsigned card_index, uint32_t mode) { assert(card_index < num_cards); cards[card_index].capture->set_video_mode(mode); } void start_mode_scanning(unsigned card_index); std::map get_available_video_inputs(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_available_video_inputs(); } uint32_t get_current_video_input(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_current_video_input(); } void set_video_input(unsigned card_index, uint32_t input) { assert(card_index < num_cards); cards[card_index].capture->set_video_input(input); } std::map get_available_audio_inputs(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_available_audio_inputs(); } uint32_t get_current_audio_input(unsigned card_index) const { assert(card_index < num_cards); return cards[card_index].capture->get_current_audio_input(); } void set_audio_input(unsigned card_index, uint32_t input) { assert(card_index < num_cards); cards[card_index].capture->set_audio_input(input); } void change_x264_bitrate(unsigned rate_kbit) { video_encoder->change_x264_bitrate(rate_kbit); } private: void configure_card(unsigned card_index, bmusb::CaptureInterface *capture, bool is_fake_capture); void bm_frame(unsigned card_index, uint16_t timecode, bmusb::FrameAllocator::Frame video_frame, size_t video_offset, bmusb::VideoFormat video_format, bmusb::FrameAllocator::Frame audio_frame, size_t audio_offset, bmusb::AudioFormat audio_format); void bm_hotplug_add(libusb_device *dev); void bm_hotplug_remove(unsigned card_index); void place_rectangle(movit::Effect *resample_effect, movit::Effect *padding_effect, float x0, float y0, float x1, float y1); void thread_func(); void handle_hotplugged_cards(); void schedule_audio_resampling_tasks(unsigned dropped_frames, int num_samples_per_frame, int length_per_frame); void render_one_frame(int64_t duration); void audio_thread_func(); void subsample_chroma(GLuint src_tex, GLuint dst_dst); void release_display_frame(DisplayFrame *frame); double pts() { return double(pts_int) / TIMEBASE; } HTTPD httpd; unsigned num_cards; QSurface *mixer_surface, *h264_encoder_surface; std::unique_ptr resource_pool; std::unique_ptr theme; std::atomic audio_source_channel{0}; std::atomic master_clock_channel{0}; std::unique_ptr display_chain; GLuint cbcr_program_num; // Owned by . GLuint cbcr_vbo; // Holds position and texcoord data. GLuint cbcr_position_attribute_index, cbcr_texcoord_attribute_index; std::unique_ptr video_encoder; // Effects part of . Owned by . movit::FlatInput *display_input; int64_t pts_int = 0; // In TIMEBASE units. std::mutex bmusb_mutex; bool has_bmusb_thread = false; struct CaptureCard { bmusb::CaptureInterface *capture = nullptr; bool is_fake_capture; std::unique_ptr frame_allocator; // Stuff for the OpenGL context (for texture uploading). QSurface *surface = nullptr; struct NewFrame { RefCountedFrame frame; int64_t length; // In TIMEBASE units. bool interlaced; unsigned field; // Which field (0 or 1) of the frame to use. Always 0 for progressive. std::function upload_func; // Needs to be called to actually upload the texture to OpenGL. unsigned dropped_frames = 0; // Number of dropped frames before this one. }; std::queue new_frames; bool should_quit = false; std::condition_variable new_frames_changed; // Set whenever new_frames (or should_quit) is changed. QueueLengthPolicy queue_length_policy; // Refers to the "new_frames" queue. // Accumulated errors in number of 1/TIMEBASE samples. If OUTPUT_FREQUENCY divided by // frame rate is integer, will always stay zero. unsigned fractional_samples = 0; int last_timecode = -1; // Unwrapped. }; CaptureCard cards[MAX_VIDEO_CARDS]; // protected by AudioMixer audio_mixer; // Same as global_audio_mixer (see audio_mixer.h). void get_one_frame_from_each_card(unsigned master_card_index, CaptureCard::NewFrame new_frames[MAX_VIDEO_CARDS], bool has_new_frame[MAX_VIDEO_CARDS], int num_samples[MAX_VIDEO_CARDS]); InputState input_state; // Cards we have been noticed about being hotplugged, but haven't tried adding yet. // Protected by its own mutex. std::mutex hotplug_mutex; std::vector hotplugged_cards; class OutputChannel { public: ~OutputChannel(); void output_frame(DisplayFrame frame); bool get_display_frame(DisplayFrame *frame); void set_frame_ready_callback(new_frame_ready_callback_t callback); void set_transition_names_updated_callback(transition_names_updated_callback_t callback); void set_name_updated_callback(name_updated_callback_t callback); void set_color_updated_callback(color_updated_callback_t callback); private: friend class Mixer; unsigned channel; Mixer *parent = nullptr; // Not owned. std::mutex frame_mutex; DisplayFrame current_frame, ready_frame; // protected by bool has_current_frame = false, has_ready_frame = false; // protected by new_frame_ready_callback_t new_frame_ready_callback; transition_names_updated_callback_t transition_names_updated_callback; name_updated_callback_t name_updated_callback; color_updated_callback_t color_updated_callback; std::vector last_transition_names; std::string last_name, last_color; }; OutputChannel output_channel[NUM_OUTPUTS]; std::thread mixer_thread; std::thread audio_thread; std::atomic should_quit{false}; std::atomic should_cut{false}; std::unique_ptr alsa; struct AudioTask { int64_t pts_int; int num_samples; bool adjust_rate; }; std::mutex audio_mutex; std::condition_variable audio_task_queue_changed; std::queue audio_task_queue; // Under audio_mutex. // For mode scanning. bool is_mode_scanning[MAX_VIDEO_CARDS]{ false }; std::vector mode_scanlist[MAX_VIDEO_CARDS]; unsigned mode_scanlist_index[MAX_VIDEO_CARDS]{ 0 }; std::chrono::steady_clock::time_point last_mode_scan_change[MAX_VIDEO_CARDS]; }; extern Mixer *global_mixer; extern bool uses_mlock; #endif // !defined(_MIXER_H) nageru-1.4.2/mux.cpp000066400000000000000000000142261301561706600143260ustar00rootroot00000000000000#include "mux.h" #include #include #include #include #include #include #include #include #include #include extern "C" { #include #include #include #include #include #include #include } #include "defs.h" #include "timebase.h" using namespace std; struct PacketBefore { PacketBefore(const AVFormatContext *ctx) : ctx(ctx) {} bool operator() (const AVPacket *a, const AVPacket *b) const { int64_t a_dts = (a->dts == AV_NOPTS_VALUE ? a->pts : a->dts); int64_t b_dts = (b->dts == AV_NOPTS_VALUE ? b->pts : b->dts); AVRational a_timebase = ctx->streams[a->stream_index]->time_base; AVRational b_timebase = ctx->streams[b->stream_index]->time_base; if (av_compare_ts(a_dts, a_timebase, b_dts, b_timebase) != 0) { return av_compare_ts(a_dts, a_timebase, b_dts, b_timebase) < 0; } else { return av_compare_ts(a->pts, a_timebase, b->pts, b_timebase) < 0; } } const AVFormatContext * const ctx; }; Mux::Mux(AVFormatContext *avctx, int width, int height, Codec video_codec, const string &video_extradata, const AVCodecParameters *audio_codecpar, int time_base, std::function write_callback) : avctx(avctx), write_callback(write_callback) { avstream_video = avformat_new_stream(avctx, nullptr); if (avstream_video == nullptr) { fprintf(stderr, "avformat_new_stream() failed\n"); exit(1); } avstream_video->time_base = AVRational{1, time_base}; avstream_video->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; if (video_codec == CODEC_H264) { avstream_video->codecpar->codec_id = AV_CODEC_ID_H264; } else { assert(video_codec == CODEC_NV12); avstream_video->codecpar->codec_id = AV_CODEC_ID_RAWVIDEO; avstream_video->codecpar->codec_tag = avcodec_pix_fmt_to_codec_tag(AV_PIX_FMT_NV12); } avstream_video->codecpar->width = width; avstream_video->codecpar->height = height; // Colorspace details. Closely correspond to settings in EffectChain_finalize, // as noted in each comment. // Note that the H.264 stream also contains this information and depending on the // mux, this might simply get ignored. See sps_rbsp(). avstream_video->codecpar->color_primaries = AVCOL_PRI_BT709; // RGB colorspace (inout_format.color_space). avstream_video->codecpar->color_trc = AVCOL_TRC_UNSPECIFIED; // Gamma curve (inout_format.gamma_curve). avstream_video->codecpar->color_space = AVCOL_SPC_SMPTE170M; // YUV colorspace (output_ycbcr_format.luma_coefficients). avstream_video->codecpar->color_range = AVCOL_RANGE_MPEG; // Full vs. limited range (output_ycbcr_format.full_range). avstream_video->codecpar->chroma_location = AVCHROMA_LOC_LEFT; // Chroma sample location. See chroma_offset_0[] in Mixer::subsample_chroma(). avstream_video->codecpar->field_order = AV_FIELD_PROGRESSIVE; if (!video_extradata.empty()) { avstream_video->codecpar->extradata = (uint8_t *)av_malloc(video_extradata.size()); avstream_video->codecpar->extradata_size = video_extradata.size(); memcpy(avstream_video->codecpar->extradata, video_extradata.data(), video_extradata.size()); } avstream_audio = avformat_new_stream(avctx, nullptr); if (avstream_audio == nullptr) { fprintf(stderr, "avformat_new_stream() failed\n"); exit(1); } avstream_audio->time_base = AVRational{1, time_base}; if (avcodec_parameters_copy(avstream_audio->codecpar, audio_codecpar) < 0) { fprintf(stderr, "avcodec_parameters_copy() failed\n"); exit(1); } AVDictionary *options = NULL; vector> opts = MUX_OPTS; for (pair opt : opts) { av_dict_set(&options, opt.first.c_str(), opt.second.c_str(), 0); } if (avformat_write_header(avctx, &options) < 0) { fprintf(stderr, "avformat_write_header() failed\n"); exit(1); } // Make sure the header is written before the constructor exits. avio_flush(avctx->pb); } Mux::~Mux() { av_write_trailer(avctx); if (!(avctx->oformat->flags & AVFMT_NOFILE) && !(avctx->flags & AVFMT_FLAG_CUSTOM_IO)) { avio_closep(&avctx->pb); } avformat_free_context(avctx); } void Mux::add_packet(const AVPacket &pkt, int64_t pts, int64_t dts) { AVPacket pkt_copy; if (av_copy_packet(&pkt_copy, &pkt) < 0) { fprintf(stderr, "av_copy_packet() failed\n"); exit(1); } if (pkt.stream_index == 0) { pkt_copy.pts = av_rescale_q(pts, AVRational{1, TIMEBASE}, avstream_video->time_base); pkt_copy.dts = av_rescale_q(dts, AVRational{1, TIMEBASE}, avstream_video->time_base); pkt_copy.duration = av_rescale_q(pkt.duration, AVRational{1, TIMEBASE}, avstream_video->time_base); } else if (pkt.stream_index == 1) { pkt_copy.pts = av_rescale_q(pts, AVRational{1, TIMEBASE}, avstream_audio->time_base); pkt_copy.dts = av_rescale_q(dts, AVRational{1, TIMEBASE}, avstream_audio->time_base); pkt_copy.duration = av_rescale_q(pkt.duration, AVRational{1, TIMEBASE}, avstream_audio->time_base); } else { assert(false); } { lock_guard lock(mu); if (plug_count > 0) { plugged_packets.push_back(av_packet_clone(&pkt_copy)); } else { write_packet_or_die(pkt_copy); } } av_packet_unref(&pkt_copy); // Note: This will be wrong in the case of plugged packets, but that only happens // for network streams, not for files, and write callbacks are only really relevant // for files. (We don't want to do this from write_packet_or_die, as it only has // the rescaled pts, which is unsuitable for callback.) if (pkt.stream_index == 0 && write_callback != nullptr) { write_callback(pts); } } void Mux::write_packet_or_die(const AVPacket &pkt) { if (av_interleaved_write_frame(avctx, const_cast(&pkt)) < 0) { fprintf(stderr, "av_interleaved_write_frame() failed\n"); exit(1); } avio_flush(avctx->pb); } void Mux::plug() { lock_guard lock(mu); ++plug_count; } void Mux::unplug() { lock_guard lock(mu); if (--plug_count > 0) { return; } assert(plug_count >= 0); sort(plugged_packets.begin(), plugged_packets.end(), PacketBefore(avctx)); for (AVPacket *pkt : plugged_packets) { write_packet_or_die(*pkt); av_packet_free(&pkt); } plugged_packets.clear(); } nageru-1.4.2/mux.h000066400000000000000000000033161301561706600137710ustar00rootroot00000000000000#ifndef _MUX_H #define _MUX_H 1 // Wrapper around an AVFormat mux. extern "C" { #include #include } #include #include #include #include #include class Mux { public: enum Codec { CODEC_H264, CODEC_NV12, // Uncompressed 4:2:0. }; // Takes ownership of avctx. will be called every time // a write has been made to the video stream (id 0), with the pts of // the just-written frame. (write_callback can be nullptr.) Mux(AVFormatContext *avctx, int width, int height, Codec video_codec, const std::string &video_extradata, const AVCodecParameters *audio_codecpar, int time_base, std::function write_callback); ~Mux(); void add_packet(const AVPacket &pkt, int64_t pts, int64_t dts); // As long as the mux is plugged, it will not actually write anything to disk, // just queue the packets. Once it is unplugged, the packets are reordered by pts // and written. This is primarily useful if you might have two different encoders // writing to the mux at the same time (because one is shutting down), so that // pts might otherwise come out-of-order. // // You can plug and unplug multiple times; only when the plug count reaches zero, // something will actually happen. void plug(); void unplug(); private: void write_packet_or_die(const AVPacket &pkt); // Must be called with held. std::mutex mu; AVFormatContext *avctx; // Protected by . int plug_count = 0; // Protected by . std::vector plugged_packets; // Protected by . AVStream *avstream_video, *avstream_audio; std::function write_callback; }; #endif // !defined(_MUX_H) nageru-1.4.2/nonlinear_fader.cpp000066400000000000000000000056141301561706600166440ustar00rootroot00000000000000#include "nonlinear_fader.h" #include #include #include #include #include #include #include #include #include #include #include "piecewise_interpolator.h" class QPaintEvent; class QWidget; using namespace std; namespace { PiecewiseInterpolator interpolator({ // The main area is from +6 to -12 dB (18 dB), and we use half the slider range for it. // Adjust slightly so that the MIDI controller value of 106 becomes exactly 0.0 dB // (cf. map_controller_to_float()); otherwise, we'd miss ever so slightly, which is // really frustrating. { 6.0, 1.0 }, { -12.0, 1.0 - (1.0 - 106.5/127.0) * 3.0 }, // About 0.492. // -12 to -21 is half the range (9 dB). Halve. { -21.0, 0.325 }, // -21 to -30 (9 dB) gets the same range as the previous one. { -30.0, 0.25 }, // -30 to -48 (18 dB) gets half of half. { -48.0, 0.125 }, // -48 to -84 (36 dB) gets half of half of half. { -84.0, 0.0 }, }); } // namespace NonLinearFader::NonLinearFader(QWidget *parent) : QSlider(parent) { update_slider_position(); } void NonLinearFader::setDbValue(double db) { db_value = db; update_slider_position(); emit dbValueChanged(db); } void NonLinearFader::paintEvent(QPaintEvent *event) { QStyleOptionSlider opt; this->initStyleOption(&opt); QRect gr = this->style()->subControlRect(QStyle::CC_Slider, &opt, QStyle::SC_SliderGroove, this); QRect sr = this->style()->subControlRect(QStyle::CC_Slider, &opt, QStyle::SC_SliderHandle, this); // FIXME: Where does the slider_length / 2 come from? I can't really find it // in the Qt code, but it seems to match up with reality. int slider_length = sr.height(); int slider_max = gr.top() + (slider_length / 2); int slider_min = gr.bottom() + (slider_length / 2) - slider_length + 1; QPainter p(this); // Draw some ticks every 6 dB. // FIXME: Find a way to make the slider wider, so that we have more space for tickmarks // and some dB numbering. int x_margin = 5; p.setPen(Qt::darkGray); for (int db = -84; db <= 6; db += 6) { int y = slider_min + lrint(interpolator.db_to_fraction(db) * (slider_max - slider_min)); p.drawLine(QPoint(0, y), QPoint(gr.left() - x_margin, y)); p.drawLine(QPoint(gr.right() + x_margin, y), QPoint(width() - 1, y)); } QSlider::paintEvent(event); } void NonLinearFader::sliderChange(SliderChange change) { QSlider::sliderChange(change); if (change == QAbstractSlider::SliderValueChange && !inhibit_updates) { if (value() == 0) { db_value = -HUGE_VAL; } else { double frac = double(value() - minimum()) / (maximum() - minimum()); db_value = interpolator.fraction_to_db(frac); } emit dbValueChanged(db_value); } } void NonLinearFader::update_slider_position() { inhibit_updates = true; double val = interpolator.db_to_fraction(db_value) * (maximum() - minimum()) + minimum(); setValue(lrint(val)); inhibit_updates = false; } nageru-1.4.2/nonlinear_fader.h000066400000000000000000000011151301561706600163010ustar00rootroot00000000000000#ifndef _NONLINEAR_FADER_H #define _NONLINEAR_FADER_H 1 #include #include #include class QObject; class QPaintEvent; class QWidget; class NonLinearFader : public QSlider { Q_OBJECT public: NonLinearFader(QWidget *parent); void setDbValue(double db); signals: void dbValueChanged(double db); protected: void paintEvent(QPaintEvent *event) override; void sliderChange(SliderChange change) override; private: void update_slider_position(); bool inhibit_updates = false; double db_value = 0.0; }; #endif // !defined(_NONLINEAR_FADER_H) nageru-1.4.2/patches/000077500000000000000000000000001301561706600144335ustar00rootroot00000000000000nageru-1.4.2/patches/zita-resampler-sse.diff000066400000000000000000000332431301561706600210210ustar00rootroot00000000000000diff -ur orig/zita-resampler-1.3.0/libs/resampler.cc zita-resampler-1.3.0/libs/resampler.cc --- orig/zita-resampler-1.3.0/libs/resampler.cc 2012-10-26 22:58:55.000000000 +0200 +++ zita-resampler-1.3.0/libs/resampler.cc 2016-09-05 00:30:34.520191288 +0200 @@ -24,6 +24,10 @@ #include #include +#ifdef __SSE2__ +#include +#endif + static unsigned int gcd (unsigned int a, unsigned int b) { @@ -47,6 +51,118 @@ return 1; } +#ifdef __SSE2__ + +static inline float calc_mono_sample_sse (unsigned int hl, + const float *c1, + const float *c2, + const float *q1, + const float *q2) +{ + unsigned int i; + __m128 denorm, s, w1, w2, shuf; + + denorm = _mm_set1_ps (1e-20f); + s = denorm; + for (i = 0; i < hl; i += 4) + { + q2 -= 4; + + // s += *q1 * c1 [i]; + w1 = _mm_loadu_ps (&c1 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1), w1)); + + // s += *q2 * c2 [i]; + w2 = _mm_loadu_ps (&c2 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (0, 1, 2, 3)))); + + q1 += 4; + } + s = _mm_sub_ps (s, denorm); + + // Add all the elements of s together into one. Adapted from + // http://stackoverflow.com/questions/6996764/fastest-way-to-do-horizontal-float-vector-sum-on-x86 + shuf = _mm_shuffle_ps (s, s, _MM_SHUFFLE (2, 3, 0, 1)); + s = _mm_add_ps (s, shuf); + s = _mm_add_ss (s, _mm_movehl_ps (shuf, s)); + return _mm_cvtss_f32 (s); +} + +// Note: This writes four floats instead of two (the last two are garbage). +// The caller will need to make sure there is room for all four. +static inline void calc_stereo_sample_sse (unsigned int hl, + const float *c1, + const float *c2, + const float *q1, + const float *q2, + float *out_data) +{ + unsigned int i; + __m128 denorm, s, w1, w2; + + denorm = _mm_set1_ps (1e-20f); + s = denorm; + for (i = 0; i < hl; i += 4) + { + q2 -= 8; + + // s += *q1 * c1 [i]; + w1 = _mm_loadu_ps (&c1 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1), _mm_unpacklo_ps (w1, w1))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1 + 4), _mm_unpackhi_ps (w1, w1))); + + // s += *q2 * c2 [i]; + w2 = _mm_loadu_ps (&c2 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2 + 4), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (0, 0, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (2, 2, 3, 3)))); + + q1 += 8; + } + s = _mm_sub_ps (s, denorm); + s = _mm_add_ps (s, _mm_shuffle_ps (s, s, _MM_SHUFFLE (1, 0, 3, 2))); + + _mm_storeu_ps (out_data, s); +} + +static inline void calc_quad_sample_sse (int hl, + int nchan, + const float *c1, + const float *c2, + const float *q1, + const float *q2, + float *out_data) +{ + int i; + __m128 denorm, s, w1, w2; + + denorm = _mm_set1_ps (1e-20f); + s = denorm; + for (i = 0; i < hl; i += 4) + { + q2 -= 4 * nchan; + + // s += *p1 * _c1 [i]; + w1 = _mm_loadu_ps (&c1 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1), _mm_shuffle_ps (w1, w1, _MM_SHUFFLE (0, 0, 0, 0)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1 + nchan), _mm_shuffle_ps (w1, w1, _MM_SHUFFLE (1, 1, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1 + 2 * nchan), _mm_shuffle_ps (w1, w1, _MM_SHUFFLE (2, 2, 2, 2)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q1 + 3 * nchan), _mm_shuffle_ps (w1, w1, _MM_SHUFFLE (3, 3, 3, 3)))); + + // s += *p2 * _c2 [i]; + w2 = _mm_loadu_ps (&c2 [i]); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2 + 3 * nchan), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (0, 0, 0, 0)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2 + 2 * nchan), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (1, 1, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2 + nchan), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (2, 2, 2, 2)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (q2), _mm_shuffle_ps (w2, w2, _MM_SHUFFLE (3, 3, 3, 3)))); + + q1 += 4 * nchan; + } + s = _mm_sub_ps (s, denorm); + + _mm_storeu_ps (out_data, s); +} +#endif + Resampler::Resampler (void) : _table (0), @@ -213,18 +329,42 @@ { float *c1 = _table->_ctab + hl * ph; float *c2 = _table->_ctab + hl * (np - ph); - for (c = 0; c < _nchan; c++) +#ifdef __SSE2__ + if ((hl % 4) == 0 && _nchan == 1) + { + *out_data++ = calc_mono_sample_sse (hl, c1, c2, p1, p2); + } + else if ((hl % 4) == 0 && _nchan == 2) { - float *q1 = p1 + c; - float *q2 = p2 + c; - float s = 1e-20f; - for (i = 0; i < hl; i++) + if (out_count >= 2) + { + calc_stereo_sample_sse (hl, c1, c2, p1, p2, out_data); + } + else + { + float tmp[4]; + calc_stereo_sample_sse (hl, c1, c2, p1, p2, tmp); + out_data[0] = tmp[0]; + out_data[1] = tmp[1]; + } + out_data += 2; + } + else +#endif + { + for (c = 0; c < _nchan; c++) { - q2 -= _nchan; - s += *q1 * c1 [i] + *q2 * c2 [i]; - q1 += _nchan; + float *q1 = p1 + c; + float *q2 = p2 + c; + float s = 1e-20f; + for (i = 0; i < hl; i++) + { + q2 -= _nchan; + s += *q1 * c1 [i] + *q2 * c2 [i]; + q1 += _nchan; + } + *out_data++ = s - 1e-20f; } - *out_data++ = s - 1e-20f; } } else diff -ur orig/zita-resampler-1.3.0/libs/vresampler.cc zita-resampler-1.3.0/libs/vresampler.cc --- orig/zita-resampler-1.3.0/libs/vresampler.cc 2012-10-26 22:58:55.000000000 +0200 +++ zita-resampler-1.3.0/libs/vresampler.cc 2016-09-05 00:33:53.907511211 +0200 @@ -25,6 +25,152 @@ #include +#ifdef __SSE2__ + +#include + +static inline float calc_mono_sample_sse (int hl, + float b, + const float *p1, + const float *p2, + const float *q1, + const float *q2) +{ + int i; + __m128 denorm, bs, s, c1, c2, w1, w2, shuf; + + denorm = _mm_set1_ps (1e-25f); + bs = _mm_set1_ps (b); + s = denorm; + for (i = 0; i < hl; i += 4) + { + p2 -= 4; + + // _c1 [i] = q1 [i] + b * (q1 [i + hl] - q1 [i]); + w1 = _mm_loadu_ps (&q1 [i]); + w2 = _mm_loadu_ps (&q1 [i + hl]); + c1 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // _c2 [i] = q2 [i] + b * (q2 [i - hl] - q2 [i]); + w1 = _mm_loadu_ps (&q2 [i]); + w2 = _mm_loadu_ps (&q2 [i - hl]); + c2 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // s += *p1 * _c1 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1), c1)); + + // s += *p2 * _c2 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (0, 1, 2, 3)))); + + p1 += 4; + } + s = _mm_sub_ps (s, denorm); + + // Add all the elements of s together into one. Adapted from + // http://stackoverflow.com/questions/6996764/fastest-way-to-do-horizontal-float-vector-sum-on-x86 + shuf = _mm_shuffle_ps (s, s, _MM_SHUFFLE (2, 3, 0, 1)); + s = _mm_add_ps (s, shuf); + s = _mm_add_ss (s, _mm_movehl_ps (shuf, s)); + return _mm_cvtss_f32 (s); +} + +// Note: This writes four floats instead of two (the last two are garbage). +// The caller will need to make sure there is room for all four. +static inline void calc_stereo_sample_sse (int hl, + float b, + const float *p1, + const float *p2, + const float *q1, + const float *q2, + float *out_data) +{ + int i; + __m128 denorm, bs, s, c1, c2, w1, w2; + + denorm = _mm_set1_ps (1e-25f); + bs = _mm_set1_ps (b); + s = denorm; + for (i = 0; i < hl; i += 4) + { + p2 -= 8; + + // _c1 [i] = q1 [i] + b * (q1 [i + hl] - q1 [i]); + w1 = _mm_loadu_ps (&q1 [i]); + w2 = _mm_loadu_ps (&q1 [i + hl]); + c1 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // _c2 [i] = q2 [i] + b * (q2 [i - hl] - q2 [i]); + w1 = _mm_loadu_ps (&q2 [i]); + w2 = _mm_loadu_ps (&q2 [i - hl]); + c2 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // s += *p1 * _c1 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1), _mm_unpacklo_ps (c1, c1))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1 + 4), _mm_unpackhi_ps (c1, c1))); + + // s += *p2 * _c2 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2 + 4), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (0, 0, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (2, 2, 3, 3)))); + + p1 += 8; + } + s = _mm_sub_ps (s, denorm); + s = _mm_add_ps (s, _mm_shuffle_ps (s, s, _MM_SHUFFLE (1, 0, 3, 2))); + + _mm_storeu_ps (out_data, s); +} + +static inline void calc_quad_sample_sse (int hl, + int nchan, + float b, + const float *p1, + const float *p2, + const float *q1, + const float *q2, + float *out_data) +{ + int i; + __m128 denorm, bs, s, c1, c2, w1, w2; + + denorm = _mm_set1_ps (1e-25f); + bs = _mm_set1_ps (b); + s = denorm; + for (i = 0; i < hl; i += 4) + { + p2 -= 4 * nchan; + + // _c1 [i] = q1 [i] + b * (q1 [i + hl] - q1 [i]); + w1 = _mm_loadu_ps (&q1 [i]); + w2 = _mm_loadu_ps (&q1 [i + hl]); + c1 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // _c2 [i] = q2 [i] + b * (q2 [i - hl] - q2 [i]); + w1 = _mm_loadu_ps (&q2 [i]); + w2 = _mm_loadu_ps (&q2 [i - hl]); + c2 = _mm_add_ps (w1, _mm_mul_ps(bs, _mm_sub_ps (w2, w1))); + + // s += *p1 * _c1 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1), _mm_shuffle_ps (c1, c1, _MM_SHUFFLE (0, 0, 0, 0)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1 + nchan), _mm_shuffle_ps (c1, c1, _MM_SHUFFLE (1, 1, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1 + 2 * nchan), _mm_shuffle_ps (c1, c1, _MM_SHUFFLE (2, 2, 2, 2)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p1 + 3 * nchan), _mm_shuffle_ps (c1, c1, _MM_SHUFFLE (3, 3, 3, 3)))); + + // s += *p2 * _c2 [i]; + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2 + 3 * nchan), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (0, 0, 0, 0)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2 + 2 * nchan), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (1, 1, 1, 1)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2 + nchan), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (2, 2, 2, 2)))); + s = _mm_add_ps (s, _mm_mul_ps (_mm_loadu_ps (p2), _mm_shuffle_ps (c2, c2, _MM_SHUFFLE (3, 3, 3, 3)))); + + p1 += 4 * nchan; + } + s = _mm_sub_ps (s, denorm); + + _mm_storeu_ps (out_data, s); +} + +#endif + + VResampler::VResampler (void) : _table (0), _nchan (0), @@ -163,7 +309,7 @@ int VResampler::process (void) { - unsigned int k, np, in, nr, n, c; + unsigned int j, k, np, in, nr, n, c; int i, hl, nz; double ph, dp, dd; float a, b, *p1, *p2, *q1, *q2; @@ -212,23 +358,55 @@ a = 1.0f - b; q1 = _table->_ctab + hl * k; q2 = _table->_ctab + hl * (np - k); - for (i = 0; i < hl; i++) +#ifdef __SSE2__ + if ((hl % 4) == 0 && _nchan == 1) + { + *out_data++ = calc_mono_sample_sse (hl, b, p1, p2, q1, q2); + } + else if ((hl % 4) == 0 && _nchan == 2) { - _c1 [i] = a * q1 [i] + b * q1 [i + hl]; - _c2 [i] = a * q2 [i] + b * q2 [i - hl]; + if (out_count >= 2) + { + calc_stereo_sample_sse (hl, b, p1, p2, q1, q2, out_data); + } + else + { + float tmp[4]; + calc_stereo_sample_sse (hl, b, p1, p2, q1, q2, tmp); + out_data[0] = tmp[0]; + out_data[1] = tmp[1]; + } + out_data += 2; + } + else if ((hl % 4) == 0 && (_nchan % 4) == 0) + { + for (j = 0; j < _nchan; j += 4) + { + calc_quad_sample_sse (hl, _nchan, b, p1 + j, p2 + j, q1, q2, out_data + j); + } + out_data += _nchan; } - for (c = 0; c < _nchan; c++) + else +#endif { - q1 = p1 + c; - q2 = p2 + c; - a = 1e-25f; for (i = 0; i < hl; i++) { - q2 -= _nchan; - a += *q1 * _c1 [i] + *q2 * _c2 [i]; - q1 += _nchan; + _c1 [i] = a * q1 [i] + b * q1 [i + hl]; + _c2 [i] = a * q2 [i] + b * q2 [i - hl]; + } + for (c = 0; c < _nchan; c++) + { + q1 = p1 + c; + q2 = p2 + c; + a = 1e-25f; + for (i = 0; i < hl; i++) + { + q2 -= _nchan; + a += *q1 * _c1 [i] + *q2 * _c2 [i]; + q1 += _nchan; + } + *out_data++ = a - 1e-25f; } - *out_data++ = a - 1e-25f; } } else nageru-1.4.2/pbo_frame_allocator.cpp000066400000000000000000000120161301561706600175020ustar00rootroot00000000000000#include "pbo_frame_allocator.h" #include #include #include #include #include #include using namespace std; PBOFrameAllocator::PBOFrameAllocator(size_t frame_size, GLuint width, GLuint height, size_t num_queued_frames, GLenum buffer, GLenum permissions, GLenum map_bits) : buffer(buffer) { userdata.reset(new Userdata[num_queued_frames]); for (size_t i = 0; i < num_queued_frames; ++i) { GLuint pbo; glGenBuffers(1, &pbo); check_error(); glBindBuffer(buffer, pbo); check_error(); glBufferStorage(buffer, frame_size, NULL, permissions | GL_MAP_PERSISTENT_BIT); check_error(); Frame frame; frame.data = (uint8_t *)glMapBufferRange(buffer, 0, frame_size, permissions | map_bits | GL_MAP_PERSISTENT_BIT); frame.data2 = frame.data + frame_size / 2; check_error(); frame.size = frame_size; frame.userdata = &userdata[i]; userdata[i].pbo = pbo; frame.owner = this; frame.interleaved = true; // Create textures. We don't allocate any data for the second field at this point // (just create the texture state with the samplers), since our default assumed // resolution is progressive. glGenTextures(2, userdata[i].tex_y); check_error(); glGenTextures(2, userdata[i].tex_cbcr); check_error(); userdata[i].last_width[0] = width; userdata[i].last_height[0] = height; userdata[i].last_width[1] = 0; userdata[i].last_height[1] = 0; userdata[i].last_interlaced = false; userdata[i].last_has_signal = false; userdata[i].last_is_connected = false; for (unsigned field = 0; field < 2; ++field) { glBindTexture(GL_TEXTURE_2D, userdata[i].tex_y[field]); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); check_error(); if (field == 0) { glTexImage2D(GL_TEXTURE_2D, 0, GL_R8, width, height, 0, GL_RED, GL_UNSIGNED_BYTE, NULL); check_error(); } glBindTexture(GL_TEXTURE_2D, userdata[i].tex_cbcr[field]); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); check_error(); if (field == 0) { glTexImage2D(GL_TEXTURE_2D, 0, GL_RG8, width / 2, height, 0, GL_RG, GL_UNSIGNED_BYTE, NULL); check_error(); } } freelist.push(frame); } glBindBuffer(buffer, 0); check_error(); glBindTexture(GL_TEXTURE_2D, 0); check_error(); } PBOFrameAllocator::~PBOFrameAllocator() { while (!freelist.empty()) { Frame frame = freelist.front(); freelist.pop(); GLuint pbo = ((Userdata *)frame.userdata)->pbo; glBindBuffer(buffer, pbo); check_error(); glUnmapBuffer(buffer); check_error(); glBindBuffer(buffer, 0); check_error(); glDeleteBuffers(1, &pbo); check_error(); glDeleteTextures(2, ((Userdata *)frame.userdata)->tex_y); check_error(); glDeleteTextures(2, ((Userdata *)frame.userdata)->tex_cbcr); check_error(); } } //static int sumsum = 0; bmusb::FrameAllocator::Frame PBOFrameAllocator::alloc_frame() { Frame vf; unique_lock lock(freelist_mutex); // Meh. if (freelist.empty()) { printf("Frame overrun (no more spare PBO frames), dropping frame!\n"); } else { //fprintf(stderr, "freelist has %d allocated\n", ++sumsum); vf = freelist.front(); freelist.pop(); // Meh. } vf.len = 0; vf.overflow = 0; return vf; } void PBOFrameAllocator::release_frame(Frame frame) { if (frame.overflow > 0) { printf("%d bytes overflow after last (PBO) frame\n", int(frame.overflow)); } #if 0 // Poison the page. (Note that this might be bogus if you don't have an OpenGL context.) memset(frame.data, 0, frame.size); Userdata *userdata = (Userdata *)frame.userdata; for (unsigned field = 0; field < 2; ++field) { glBindTexture(GL_TEXTURE_2D, userdata->tex_y[field]); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); check_error(); glTexImage2D(GL_TEXTURE_2D, 0, GL_R8, userdata->last_width[field], userdata->last_height[field], 0, GL_RED, GL_UNSIGNED_BYTE, NULL); check_error(); glBindTexture(GL_TEXTURE_2D, userdata->tex_cbcr[field]); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); check_error(); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); check_error(); glTexImage2D(GL_TEXTURE_2D, 0, GL_RG8, userdata->last_width[field] / 2, userdata->last_height[field], 0, GL_RED, GL_UNSIGNED_BYTE, NULL); check_error(); } #endif unique_lock lock(freelist_mutex); freelist.push(frame); //--sumsum; } nageru-1.4.2/pbo_frame_allocator.h000066400000000000000000000026501301561706600171520ustar00rootroot00000000000000#ifndef _PBO_FRAME_ALLOCATOR #define _PBO_FRAME_ALLOCATOR 1 #include #include #include #include #include #include #include "bmusb/bmusb.h" // An allocator that allocates straight into OpenGL pinned memory. // Meant for video frames only. We use a queue rather than a stack, // since we want to maximize pipelineability. class PBOFrameAllocator : public bmusb::FrameAllocator { public: // Note: You need to have an OpenGL context when calling // the constructor. PBOFrameAllocator(size_t frame_size, GLuint width, GLuint height, size_t num_queued_frames = 16, // FIXME: should be 6 GLenum buffer = GL_PIXEL_UNPACK_BUFFER_ARB, GLenum permissions = GL_MAP_WRITE_BIT, GLenum map_bits = GL_MAP_FLUSH_EXPLICIT_BIT); ~PBOFrameAllocator() override; Frame alloc_frame() override; void release_frame(Frame frame) override; struct Userdata { GLuint pbo; // The second set is only used for the second field of interlaced inputs. GLuint tex_y[2], tex_cbcr[2]; GLuint last_width[2], last_height[2]; bool last_interlaced, last_has_signal, last_is_connected; unsigned last_frame_rate_nom, last_frame_rate_den; }; private: std::mutex freelist_mutex; std::queue freelist; GLenum buffer; std::unique_ptr userdata; }; #endif // !defined(_PBO_FRAME_ALLOCATOR) nageru-1.4.2/piecewise_interpolator.cpp000066400000000000000000000024121301561706600202660ustar00rootroot00000000000000#include "piecewise_interpolator.h" #include double PiecewiseInterpolator::fraction_to_db(double db) const { if (db >= control_points[0].fraction) { return control_points[0].db_value; } if (db <= control_points.back().fraction) { return control_points.back().db_value; } for (unsigned i = 1; i < control_points.size(); ++i) { const double x0 = control_points[i].fraction; const double x1 = control_points[i - 1].fraction; const double y0 = control_points[i].db_value; const double y1 = control_points[i - 1].db_value; if (db >= x0 && db <= x1) { const double t = (db - x0) / (x1 - x0); return y0 + t * (y1 - y0); } } assert(false); } double PiecewiseInterpolator::db_to_fraction(double x) const { if (x >= control_points[0].db_value) { return control_points[0].fraction; } if (x <= control_points.back().db_value) { return control_points.back().fraction; } for (unsigned i = 1; i < control_points.size(); ++i) { const double x0 = control_points[i].db_value; const double x1 = control_points[i - 1].db_value; const double y0 = control_points[i].fraction; const double y1 = control_points[i - 1].fraction; if (x >= x0 && x <= x1) { const double t = (x - x0) / (x1 - x0); return y0 + t * (y1 - y0); } } assert(false); } nageru-1.4.2/piecewise_interpolator.h000066400000000000000000000013421301561706600177340ustar00rootroot00000000000000#ifndef _PIECEWISE_INTERPOLATOR_H #define _PIECEWISE_INTERPOLATOR_H // A class to do piecewise linear interpolation of one scale to another // (and back). Typically used to implement nonlinear dB mappings for sliders // or meters, thus the nomenclature. #include class PiecewiseInterpolator { public: // Both dB and fraction values must go from high to low. struct ControlPoint { double db_value; double fraction; }; PiecewiseInterpolator(const std::vector &control_points) : control_points(control_points) {} double fraction_to_db(double db) const; double db_to_fraction(double x) const; private: const std::vector control_points; }; #endif // !defined(_PIECEWISE_INTERPOLATOR_H) nageru-1.4.2/post_to_main_thread.h000066400000000000000000000007031301561706600171770ustar00rootroot00000000000000#ifndef _POST_TO_MAIN_THREAD_H #define _POST_TO_MAIN_THREAD_H 1 #include #include #include // http://stackoverflow.com/questions/21646467/how-to-execute-a-functor-in-a-given-thread-in-qt-gcd-style template static inline void post_to_main_thread(F &&fun) { QObject signalSource; QObject::connect(&signalSource, &QObject::destroyed, qApp, std::move(fun)); } #endif // !defined(_POST_TO_MAIN_THREAD_H) nageru-1.4.2/quicksync_encoder.cpp000066400000000000000000002354341301561706600172330ustar00rootroot00000000000000#include "quicksync_encoder.h" #include // Must be above the Xlib includes. #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include extern "C" { #include #include #include #include } // namespace #include "audio_encoder.h" #include "context.h" #include "defs.h" #include "disk_space_estimator.h" #include "ffmpeg_raii.h" #include "flags.h" #include "mux.h" #include "ref_counted_frame.h" #include "timebase.h" #include "x264_encoder.h" using namespace std; using namespace std::placeholders; class QOpenGLContext; class QSurface; #define CHECK_VASTATUS(va_status, func) \ if (va_status != VA_STATUS_SUCCESS) { \ fprintf(stderr, "%s:%d (%s) failed with %d\n", __func__, __LINE__, func, va_status); \ exit(1); \ } #define BUFFER_OFFSET(i) ((char *)NULL + (i)) //#include "loadsurface.h" #define NAL_REF_IDC_NONE 0 #define NAL_REF_IDC_LOW 1 #define NAL_REF_IDC_MEDIUM 2 #define NAL_REF_IDC_HIGH 3 #define NAL_NON_IDR 1 #define NAL_IDR 5 #define NAL_SPS 7 #define NAL_PPS 8 #define NAL_SEI 6 #define SLICE_TYPE_P 0 #define SLICE_TYPE_B 1 #define SLICE_TYPE_I 2 #define IS_P_SLICE(type) (SLICE_TYPE_P == (type)) #define IS_B_SLICE(type) (SLICE_TYPE_B == (type)) #define IS_I_SLICE(type) (SLICE_TYPE_I == (type)) #define ENTROPY_MODE_CAVLC 0 #define ENTROPY_MODE_CABAC 1 #define PROFILE_IDC_BASELINE 66 #define PROFILE_IDC_MAIN 77 #define PROFILE_IDC_HIGH 100 #define BITSTREAM_ALLOCATE_STEPPING 4096 #define SURFACE_NUM 16 /* 16 surfaces for source YUV */ #define MAX_NUM_REF1 16 // Seemingly a hardware-fixed value, not related to SURFACE_NUM #define MAX_NUM_REF2 32 // Seemingly a hardware-fixed value, not related to SURFACE_NUM static constexpr unsigned int MaxFrameNum = (2<<16); static constexpr unsigned int MaxPicOrderCntLsb = (2<<8); static constexpr unsigned int Log2MaxFrameNum = 16; static constexpr unsigned int Log2MaxPicOrderCntLsb = 8; static constexpr int rc_default_modes[] = { // Priority list of modes. VA_RC_VBR, VA_RC_CQP, VA_RC_VBR_CONSTRAINED, VA_RC_CBR, VA_RC_VCM, VA_RC_NONE, }; /* thread to save coded data */ #define SRC_SURFACE_FREE 0 #define SRC_SURFACE_IN_ENCODING 1 struct __bitstream { unsigned int *buffer; int bit_offset; int max_size_in_dword; }; typedef struct __bitstream bitstream; using namespace std; // H.264 video comes out in encoding order (e.g. with two B-frames: // 0, 3, 1, 2, 6, 4, 5, etc.), but uncompressed video needs to // come in the right order. Since we do everything, including waiting // for the frames to come out of OpenGL, in encoding order, we need // a reordering buffer for uncompressed frames so that they come out // correctly. We go the super-lazy way of not making it understand // anything about the true order (which introduces some extra latency, // though); we know that for N B-frames we need at most (N-1) frames // in the reorder buffer, and can just sort on that. // // The class also deals with keeping a freelist as needed. class FrameReorderer { public: FrameReorderer(unsigned queue_length, int width, int height); struct Frame { int64_t pts, duration; uint8_t *data; // Invert to get the smallest pts first. bool operator< (const Frame &other) const { return pts > other.pts; } }; // Returns the next frame to insert with its pts, if any. Otherwise -1 and nullptr. // Does _not_ take ownership of data; a copy is taken if needed. // The returned pointer is valid until the next call to reorder_frame, or destruction. // As a special case, if queue_length == 0, will just return pts and data (no reordering needed). Frame reorder_frame(int64_t pts, int64_t duration, uint8_t *data); // The same as reorder_frame, but without inserting anything. Used to empty the queue. Frame get_first_frame(); bool empty() const { return frames.empty(); } private: unsigned queue_length; int width, height; priority_queue frames; stack freelist; // Includes the last value returned from reorder_frame. // Owns all the pointers. Normally, freelist and frames could do this themselves, // except priority_queue doesn't work well with movable-only types. vector> owner; }; FrameReorderer::FrameReorderer(unsigned queue_length, int width, int height) : queue_length(queue_length), width(width), height(height) { for (unsigned i = 0; i < queue_length; ++i) { owner.emplace_back(new uint8_t[width * height * 2]); freelist.push(owner.back().get()); } } FrameReorderer::Frame FrameReorderer::reorder_frame(int64_t pts, int64_t duration, uint8_t *data) { if (queue_length == 0) { return Frame{pts, duration, data}; } assert(!freelist.empty()); uint8_t *storage = freelist.top(); freelist.pop(); memcpy(storage, data, width * height * 2); frames.push(Frame{pts, duration, storage}); if (frames.size() >= queue_length) { return get_first_frame(); } else { return Frame{-1, -1, nullptr}; } } FrameReorderer::Frame FrameReorderer::get_first_frame() { assert(!frames.empty()); Frame storage = frames.top(); frames.pop(); freelist.push(storage.data); return storage; } class QuickSyncEncoderImpl { public: QuickSyncEncoderImpl(const std::string &filename, movit::ResourcePool *resource_pool, QSurface *surface, const string &va_display, int width, int height, AVOutputFormat *oformat, X264Encoder *x264_encoder, DiskSpaceEstimator *disk_space_estimator); ~QuickSyncEncoderImpl(); void add_audio(int64_t pts, vector audio); bool begin_frame(GLuint *y_tex, GLuint *cbcr_tex); RefCountedGLsync end_frame(int64_t pts, int64_t duration, const vector &input_frames); void shutdown(); void release_gl_resources(); void set_stream_mux(Mux *mux) { stream_mux = mux; } // So we never get negative dts. int64_t global_delay() const { return int64_t(ip_period - 1) * (TIMEBASE / MAX_FPS); } private: struct storage_task { unsigned long long display_order; int frame_type; vector audio; int64_t pts, dts, duration; }; struct PendingFrame { RefCountedGLsync fence; vector input_frames; int64_t pts, duration; }; void open_output_file(const std::string &filename); void encode_thread_func(); void encode_remaining_frames_as_p(int encoding_frame_num, int gop_start_display_frame_num, int64_t last_dts); void add_packet_for_uncompressed_frame(int64_t pts, int64_t duration, const uint8_t *data); void encode_frame(PendingFrame frame, int encoding_frame_num, int display_frame_num, int gop_start_display_frame_num, int frame_type, int64_t pts, int64_t dts, int64_t duration); void storage_task_thread(); void storage_task_enqueue(storage_task task); void save_codeddata(storage_task task); int render_packedsequence(); int render_packedpicture(); void render_packedslice(); int render_sequence(); int render_picture(int frame_type, int display_frame_num, int gop_start_display_frame_num); void sps_rbsp(bitstream *bs); void pps_rbsp(bitstream *bs); int build_packed_pic_buffer(unsigned char **header_buffer); int render_slice(int encoding_frame_num, int display_frame_num, int gop_start_display_frame_num, int frame_type); void slice_header(bitstream *bs); int build_packed_seq_buffer(unsigned char **header_buffer); int build_packed_slice_buffer(unsigned char **header_buffer); int init_va(const string &va_display); int deinit_va(); void enable_zerocopy_if_possible(); VADisplay va_open_display(const string &va_display); void va_close_display(VADisplay va_dpy); int setup_encode(); void release_encode(); void update_ReferenceFrames(int frame_type); int update_RefPicList(int frame_type); bool is_shutdown = false; bool has_released_gl_resources = false; bool use_zerocopy; int drm_fd = -1; thread encode_thread, storage_thread; mutex storage_task_queue_mutex; condition_variable storage_task_queue_changed; int srcsurface_status[SURFACE_NUM]; // protected by storage_task_queue_mutex queue storage_task_queue; // protected by storage_task_queue_mutex bool storage_thread_should_quit = false; // protected by storage_task_queue_mutex mutex frame_queue_mutex; condition_variable frame_queue_nonempty; bool encode_thread_should_quit = false; // under frame_queue_mutex int current_storage_frame; map pending_video_frames; // under frame_queue_mutex movit::ResourcePool *resource_pool; QSurface *surface; unique_ptr file_audio_encoder; unique_ptr reorderer; X264Encoder *x264_encoder; // nullptr if not using x264. Mux* stream_mux = nullptr; // To HTTP. unique_ptr file_mux; // To local disk. Display *x11_display = nullptr; // Encoder parameters VADisplay va_dpy; VAProfile h264_profile = (VAProfile)~0; VAConfigAttrib config_attrib[VAConfigAttribTypeMax]; int config_attrib_num = 0, enc_packed_header_idx; struct GLSurface { VASurfaceID src_surface, ref_surface; VABufferID coded_buf; VAImage surface_image; GLuint y_tex, cbcr_tex; // Only if use_zerocopy == true. EGLImage y_egl_image, cbcr_egl_image; // Only if use_zerocopy == false. GLuint pbo; uint8_t *y_ptr, *cbcr_ptr; size_t y_offset, cbcr_offset; }; GLSurface gl_surfaces[SURFACE_NUM]; VAConfigID config_id; VAContextID context_id; VAEncSequenceParameterBufferH264 seq_param; VAEncPictureParameterBufferH264 pic_param; VAEncSliceParameterBufferH264 slice_param; VAPictureH264 CurrentCurrPic; VAPictureH264 ReferenceFrames[MAX_NUM_REF1], RefPicList0_P[MAX_NUM_REF2], RefPicList0_B[MAX_NUM_REF2], RefPicList1_B[MAX_NUM_REF2]; // Static quality settings. static constexpr unsigned int frame_bitrate = 15000000 / 60; // Doesn't really matter; only initial_qp does. static constexpr unsigned int num_ref_frames = 2; static constexpr int initial_qp = 15; static constexpr int minimal_qp = 0; static constexpr int intra_period = 30; static constexpr int intra_idr_period = MAX_FPS; // About a second; more at lower frame rates. Not ideal. // Quality settings that are meant to be static, but might be overridden // by the profile. int constraint_set_flag = 0; int h264_packedheader = 0; /* support pack header? */ int h264_maxref = (1<<16|1); int h264_entropy_mode = 1; /* cabac */ int ip_period = 3; int rc_mode = -1; unsigned int current_frame_num = 0; unsigned int numShortTerm = 0; int frame_width; int frame_height; int frame_width_mbaligned; int frame_height_mbaligned; DiskSpaceEstimator *disk_space_estimator; }; // Supposedly vaRenderPicture() is supposed to destroy the buffer implicitly, // but if we don't delete it here, we get leaks. The GStreamer implementation // does the same. static void render_picture_and_delete(VADisplay dpy, VAContextID context, VABufferID *buffers, int num_buffers) { VAStatus va_status = vaRenderPicture(dpy, context, buffers, num_buffers); CHECK_VASTATUS(va_status, "vaRenderPicture"); for (int i = 0; i < num_buffers; ++i) { va_status = vaDestroyBuffer(dpy, buffers[i]); CHECK_VASTATUS(va_status, "vaDestroyBuffer"); } } static unsigned int va_swap32(unsigned int val) { unsigned char *pval = (unsigned char *)&val; return ((pval[0] << 24) | (pval[1] << 16) | (pval[2] << 8) | (pval[3] << 0)); } static void bitstream_start(bitstream *bs) { bs->max_size_in_dword = BITSTREAM_ALLOCATE_STEPPING; bs->buffer = (unsigned int *)calloc(bs->max_size_in_dword * sizeof(int), 1); bs->bit_offset = 0; } static void bitstream_end(bitstream *bs) { int pos = (bs->bit_offset >> 5); int bit_offset = (bs->bit_offset & 0x1f); int bit_left = 32 - bit_offset; if (bit_offset) { bs->buffer[pos] = va_swap32((bs->buffer[pos] << bit_left)); } } static void bitstream_put_ui(bitstream *bs, unsigned int val, int size_in_bits) { int pos = (bs->bit_offset >> 5); int bit_offset = (bs->bit_offset & 0x1f); int bit_left = 32 - bit_offset; if (!size_in_bits) return; bs->bit_offset += size_in_bits; if (bit_left > size_in_bits) { bs->buffer[pos] = (bs->buffer[pos] << size_in_bits | val); } else { size_in_bits -= bit_left; if (bit_left >= 32) { bs->buffer[pos] = (val >> size_in_bits); } else { bs->buffer[pos] = (bs->buffer[pos] << bit_left) | (val >> size_in_bits); } bs->buffer[pos] = va_swap32(bs->buffer[pos]); if (pos + 1 == bs->max_size_in_dword) { bs->max_size_in_dword += BITSTREAM_ALLOCATE_STEPPING; bs->buffer = (unsigned int *)realloc(bs->buffer, bs->max_size_in_dword * sizeof(unsigned int)); } bs->buffer[pos + 1] = val; } } static void bitstream_put_ue(bitstream *bs, unsigned int val) { int size_in_bits = 0; int tmp_val = ++val; while (tmp_val) { tmp_val >>= 1; size_in_bits++; } bitstream_put_ui(bs, 0, size_in_bits - 1); // leading zero bitstream_put_ui(bs, val, size_in_bits); } static void bitstream_put_se(bitstream *bs, int val) { unsigned int new_val; if (val <= 0) new_val = -2 * val; else new_val = 2 * val - 1; bitstream_put_ue(bs, new_val); } static void bitstream_byte_aligning(bitstream *bs, int bit) { int bit_offset = (bs->bit_offset & 0x7); int bit_left = 8 - bit_offset; int new_val; if (!bit_offset) return; assert(bit == 0 || bit == 1); if (bit) new_val = (1 << bit_left) - 1; else new_val = 0; bitstream_put_ui(bs, new_val, bit_left); } static void rbsp_trailing_bits(bitstream *bs) { bitstream_put_ui(bs, 1, 1); bitstream_byte_aligning(bs, 0); } static void nal_start_code_prefix(bitstream *bs) { bitstream_put_ui(bs, 0x00000001, 32); } static void nal_header(bitstream *bs, int nal_ref_idc, int nal_unit_type) { bitstream_put_ui(bs, 0, 1); /* forbidden_zero_bit: 0 */ bitstream_put_ui(bs, nal_ref_idc, 2); bitstream_put_ui(bs, nal_unit_type, 5); } void QuickSyncEncoderImpl::sps_rbsp(bitstream *bs) { int profile_idc = PROFILE_IDC_BASELINE; if (h264_profile == VAProfileH264High) profile_idc = PROFILE_IDC_HIGH; else if (h264_profile == VAProfileH264Main) profile_idc = PROFILE_IDC_MAIN; bitstream_put_ui(bs, profile_idc, 8); /* profile_idc */ bitstream_put_ui(bs, !!(constraint_set_flag & 1), 1); /* constraint_set0_flag */ bitstream_put_ui(bs, !!(constraint_set_flag & 2), 1); /* constraint_set1_flag */ bitstream_put_ui(bs, !!(constraint_set_flag & 4), 1); /* constraint_set2_flag */ bitstream_put_ui(bs, !!(constraint_set_flag & 8), 1); /* constraint_set3_flag */ bitstream_put_ui(bs, 0, 4); /* reserved_zero_4bits */ bitstream_put_ui(bs, seq_param.level_idc, 8); /* level_idc */ bitstream_put_ue(bs, seq_param.seq_parameter_set_id); /* seq_parameter_set_id */ if ( profile_idc == PROFILE_IDC_HIGH) { bitstream_put_ue(bs, 1); /* chroma_format_idc = 1, 4:2:0 */ bitstream_put_ue(bs, 0); /* bit_depth_luma_minus8 */ bitstream_put_ue(bs, 0); /* bit_depth_chroma_minus8 */ bitstream_put_ui(bs, 0, 1); /* qpprime_y_zero_transform_bypass_flag */ bitstream_put_ui(bs, 0, 1); /* seq_scaling_matrix_present_flag */ } bitstream_put_ue(bs, seq_param.seq_fields.bits.log2_max_frame_num_minus4); /* log2_max_frame_num_minus4 */ bitstream_put_ue(bs, seq_param.seq_fields.bits.pic_order_cnt_type); /* pic_order_cnt_type */ if (seq_param.seq_fields.bits.pic_order_cnt_type == 0) bitstream_put_ue(bs, seq_param.seq_fields.bits.log2_max_pic_order_cnt_lsb_minus4); /* log2_max_pic_order_cnt_lsb_minus4 */ else { assert(0); } bitstream_put_ue(bs, seq_param.max_num_ref_frames); /* num_ref_frames */ bitstream_put_ui(bs, 0, 1); /* gaps_in_frame_num_value_allowed_flag */ bitstream_put_ue(bs, seq_param.picture_width_in_mbs - 1); /* pic_width_in_mbs_minus1 */ bitstream_put_ue(bs, seq_param.picture_height_in_mbs - 1); /* pic_height_in_map_units_minus1 */ bitstream_put_ui(bs, seq_param.seq_fields.bits.frame_mbs_only_flag, 1); /* frame_mbs_only_flag */ if (!seq_param.seq_fields.bits.frame_mbs_only_flag) { assert(0); } bitstream_put_ui(bs, seq_param.seq_fields.bits.direct_8x8_inference_flag, 1); /* direct_8x8_inference_flag */ bitstream_put_ui(bs, seq_param.frame_cropping_flag, 1); /* frame_cropping_flag */ if (seq_param.frame_cropping_flag) { bitstream_put_ue(bs, seq_param.frame_crop_left_offset); /* frame_crop_left_offset */ bitstream_put_ue(bs, seq_param.frame_crop_right_offset); /* frame_crop_right_offset */ bitstream_put_ue(bs, seq_param.frame_crop_top_offset); /* frame_crop_top_offset */ bitstream_put_ue(bs, seq_param.frame_crop_bottom_offset); /* frame_crop_bottom_offset */ } //if ( frame_bit_rate < 0 ) { //TODO EW: the vui header isn't correct if ( false ) { bitstream_put_ui(bs, 0, 1); /* vui_parameters_present_flag */ } else { bitstream_put_ui(bs, 1, 1); /* vui_parameters_present_flag */ bitstream_put_ui(bs, 0, 1); /* aspect_ratio_info_present_flag */ bitstream_put_ui(bs, 0, 1); /* overscan_info_present_flag */ bitstream_put_ui(bs, 1, 1); /* video_signal_type_present_flag */ { bitstream_put_ui(bs, 5, 3); /* video_format (5 = Unspecified) */ bitstream_put_ui(bs, 0, 1); /* video_full_range_flag */ bitstream_put_ui(bs, 1, 1); /* colour_description_present_flag */ { bitstream_put_ui(bs, 1, 8); /* colour_primaries (1 = BT.709) */ bitstream_put_ui(bs, 2, 8); /* transfer_characteristics (2 = unspecified, since we use sRGB) */ bitstream_put_ui(bs, 6, 8); /* matrix_coefficients (6 = BT.601/SMPTE 170M) */ } } bitstream_put_ui(bs, 0, 1); /* chroma_loc_info_present_flag */ bitstream_put_ui(bs, 1, 1); /* timing_info_present_flag */ { bitstream_put_ui(bs, 1, 32); // FPS bitstream_put_ui(bs, TIMEBASE * 2, 32); // FPS bitstream_put_ui(bs, 1, 1); } bitstream_put_ui(bs, 1, 1); /* nal_hrd_parameters_present_flag */ { // hrd_parameters bitstream_put_ue(bs, 0); /* cpb_cnt_minus1 */ bitstream_put_ui(bs, 4, 4); /* bit_rate_scale */ bitstream_put_ui(bs, 6, 4); /* cpb_size_scale */ bitstream_put_ue(bs, frame_bitrate - 1); /* bit_rate_value_minus1[0] */ bitstream_put_ue(bs, frame_bitrate*8 - 1); /* cpb_size_value_minus1[0] */ bitstream_put_ui(bs, 1, 1); /* cbr_flag[0] */ bitstream_put_ui(bs, 23, 5); /* initial_cpb_removal_delay_length_minus1 */ bitstream_put_ui(bs, 23, 5); /* cpb_removal_delay_length_minus1 */ bitstream_put_ui(bs, 23, 5); /* dpb_output_delay_length_minus1 */ bitstream_put_ui(bs, 23, 5); /* time_offset_length */ } bitstream_put_ui(bs, 0, 1); /* vcl_hrd_parameters_present_flag */ bitstream_put_ui(bs, 0, 1); /* low_delay_hrd_flag */ bitstream_put_ui(bs, 0, 1); /* pic_struct_present_flag */ bitstream_put_ui(bs, 0, 1); /* bitstream_restriction_flag */ } rbsp_trailing_bits(bs); /* rbsp_trailing_bits */ } void QuickSyncEncoderImpl::pps_rbsp(bitstream *bs) { bitstream_put_ue(bs, pic_param.pic_parameter_set_id); /* pic_parameter_set_id */ bitstream_put_ue(bs, pic_param.seq_parameter_set_id); /* seq_parameter_set_id */ bitstream_put_ui(bs, pic_param.pic_fields.bits.entropy_coding_mode_flag, 1); /* entropy_coding_mode_flag */ bitstream_put_ui(bs, 0, 1); /* pic_order_present_flag: 0 */ bitstream_put_ue(bs, 0); /* num_slice_groups_minus1 */ bitstream_put_ue(bs, pic_param.num_ref_idx_l0_active_minus1); /* num_ref_idx_l0_active_minus1 */ bitstream_put_ue(bs, pic_param.num_ref_idx_l1_active_minus1); /* num_ref_idx_l1_active_minus1 1 */ bitstream_put_ui(bs, pic_param.pic_fields.bits.weighted_pred_flag, 1); /* weighted_pred_flag: 0 */ bitstream_put_ui(bs, pic_param.pic_fields.bits.weighted_bipred_idc, 2); /* weighted_bipred_idc: 0 */ bitstream_put_se(bs, pic_param.pic_init_qp - 26); /* pic_init_qp_minus26 */ bitstream_put_se(bs, 0); /* pic_init_qs_minus26 */ bitstream_put_se(bs, 0); /* chroma_qp_index_offset */ bitstream_put_ui(bs, pic_param.pic_fields.bits.deblocking_filter_control_present_flag, 1); /* deblocking_filter_control_present_flag */ bitstream_put_ui(bs, 0, 1); /* constrained_intra_pred_flag */ bitstream_put_ui(bs, 0, 1); /* redundant_pic_cnt_present_flag */ /* more_rbsp_data */ bitstream_put_ui(bs, pic_param.pic_fields.bits.transform_8x8_mode_flag, 1); /*transform_8x8_mode_flag */ bitstream_put_ui(bs, 0, 1); /* pic_scaling_matrix_present_flag */ bitstream_put_se(bs, pic_param.second_chroma_qp_index_offset ); /*second_chroma_qp_index_offset */ rbsp_trailing_bits(bs); } void QuickSyncEncoderImpl::slice_header(bitstream *bs) { int first_mb_in_slice = slice_param.macroblock_address; bitstream_put_ue(bs, first_mb_in_slice); /* first_mb_in_slice: 0 */ bitstream_put_ue(bs, slice_param.slice_type); /* slice_type */ bitstream_put_ue(bs, slice_param.pic_parameter_set_id); /* pic_parameter_set_id: 0 */ bitstream_put_ui(bs, pic_param.frame_num, seq_param.seq_fields.bits.log2_max_frame_num_minus4 + 4); /* frame_num */ /* frame_mbs_only_flag == 1 */ if (!seq_param.seq_fields.bits.frame_mbs_only_flag) { /* FIXME: */ assert(0); } if (pic_param.pic_fields.bits.idr_pic_flag) bitstream_put_ue(bs, slice_param.idr_pic_id); /* idr_pic_id: 0 */ if (seq_param.seq_fields.bits.pic_order_cnt_type == 0) { bitstream_put_ui(bs, pic_param.CurrPic.TopFieldOrderCnt, seq_param.seq_fields.bits.log2_max_pic_order_cnt_lsb_minus4 + 4); /* pic_order_present_flag == 0 */ } else { /* FIXME: */ assert(0); } /* redundant_pic_cnt_present_flag == 0 */ /* slice type */ if (IS_P_SLICE(slice_param.slice_type)) { bitstream_put_ui(bs, slice_param.num_ref_idx_active_override_flag, 1); /* num_ref_idx_active_override_flag: */ if (slice_param.num_ref_idx_active_override_flag) bitstream_put_ue(bs, slice_param.num_ref_idx_l0_active_minus1); /* ref_pic_list_reordering */ bitstream_put_ui(bs, 0, 1); /* ref_pic_list_reordering_flag_l0: 0 */ } else if (IS_B_SLICE(slice_param.slice_type)) { bitstream_put_ui(bs, slice_param.direct_spatial_mv_pred_flag, 1); /* direct_spatial_mv_pred: 1 */ bitstream_put_ui(bs, slice_param.num_ref_idx_active_override_flag, 1); /* num_ref_idx_active_override_flag: */ if (slice_param.num_ref_idx_active_override_flag) { bitstream_put_ue(bs, slice_param.num_ref_idx_l0_active_minus1); bitstream_put_ue(bs, slice_param.num_ref_idx_l1_active_minus1); } /* ref_pic_list_reordering */ bitstream_put_ui(bs, 0, 1); /* ref_pic_list_reordering_flag_l0: 0 */ bitstream_put_ui(bs, 0, 1); /* ref_pic_list_reordering_flag_l1: 0 */ } if ((pic_param.pic_fields.bits.weighted_pred_flag && IS_P_SLICE(slice_param.slice_type)) || ((pic_param.pic_fields.bits.weighted_bipred_idc == 1) && IS_B_SLICE(slice_param.slice_type))) { /* FIXME: fill weight/offset table */ assert(0); } /* dec_ref_pic_marking */ if (pic_param.pic_fields.bits.reference_pic_flag) { /* nal_ref_idc != 0 */ unsigned char no_output_of_prior_pics_flag = 0; unsigned char long_term_reference_flag = 0; unsigned char adaptive_ref_pic_marking_mode_flag = 0; if (pic_param.pic_fields.bits.idr_pic_flag) { bitstream_put_ui(bs, no_output_of_prior_pics_flag, 1); /* no_output_of_prior_pics_flag: 0 */ bitstream_put_ui(bs, long_term_reference_flag, 1); /* long_term_reference_flag: 0 */ } else { bitstream_put_ui(bs, adaptive_ref_pic_marking_mode_flag, 1); /* adaptive_ref_pic_marking_mode_flag: 0 */ } } if (pic_param.pic_fields.bits.entropy_coding_mode_flag && !IS_I_SLICE(slice_param.slice_type)) bitstream_put_ue(bs, slice_param.cabac_init_idc); /* cabac_init_idc: 0 */ bitstream_put_se(bs, slice_param.slice_qp_delta); /* slice_qp_delta: 0 */ /* ignore for SP/SI */ if (pic_param.pic_fields.bits.deblocking_filter_control_present_flag) { bitstream_put_ue(bs, slice_param.disable_deblocking_filter_idc); /* disable_deblocking_filter_idc: 0 */ if (slice_param.disable_deblocking_filter_idc != 1) { bitstream_put_se(bs, slice_param.slice_alpha_c0_offset_div2); /* slice_alpha_c0_offset_div2: 2 */ bitstream_put_se(bs, slice_param.slice_beta_offset_div2); /* slice_beta_offset_div2: 2 */ } } if (pic_param.pic_fields.bits.entropy_coding_mode_flag) { bitstream_byte_aligning(bs, 1); } } int QuickSyncEncoderImpl::build_packed_pic_buffer(unsigned char **header_buffer) { bitstream bs; bitstream_start(&bs); nal_start_code_prefix(&bs); nal_header(&bs, NAL_REF_IDC_HIGH, NAL_PPS); pps_rbsp(&bs); bitstream_end(&bs); *header_buffer = (unsigned char *)bs.buffer; return bs.bit_offset; } int QuickSyncEncoderImpl::build_packed_seq_buffer(unsigned char **header_buffer) { bitstream bs; bitstream_start(&bs); nal_start_code_prefix(&bs); nal_header(&bs, NAL_REF_IDC_HIGH, NAL_SPS); sps_rbsp(&bs); bitstream_end(&bs); *header_buffer = (unsigned char *)bs.buffer; return bs.bit_offset; } int QuickSyncEncoderImpl::build_packed_slice_buffer(unsigned char **header_buffer) { bitstream bs; int is_idr = !!pic_param.pic_fields.bits.idr_pic_flag; int is_ref = !!pic_param.pic_fields.bits.reference_pic_flag; bitstream_start(&bs); nal_start_code_prefix(&bs); if (IS_I_SLICE(slice_param.slice_type)) { nal_header(&bs, NAL_REF_IDC_HIGH, is_idr ? NAL_IDR : NAL_NON_IDR); } else if (IS_P_SLICE(slice_param.slice_type)) { nal_header(&bs, NAL_REF_IDC_MEDIUM, NAL_NON_IDR); } else { assert(IS_B_SLICE(slice_param.slice_type)); nal_header(&bs, is_ref ? NAL_REF_IDC_LOW : NAL_REF_IDC_NONE, NAL_NON_IDR); } slice_header(&bs); bitstream_end(&bs); *header_buffer = (unsigned char *)bs.buffer; return bs.bit_offset; } /* Assume frame sequence is: Frame#0, #1, #2, ..., #M, ..., #X, ... (encoding order) 1) period between Frame #X and Frame #N = #X - #N 2) 0 means infinite for intra_period/intra_idr_period, and 0 is invalid for ip_period 3) intra_idr_period % intra_period (intra_period > 0) and intra_period % ip_period must be 0 4) intra_period and intra_idr_period take precedence over ip_period 5) if ip_period > 1, intra_period and intra_idr_period are not the strict periods of I/IDR frames, see bellow examples ------------------------------------------------------------------- intra_period intra_idr_period ip_period frame sequence (intra_period/intra_idr_period/ip_period) 0 ignored 1 IDRPPPPPPP ... (No IDR/I any more) 0 ignored >=2 IDR(PBB)(PBB)... (No IDR/I any more) 1 0 ignored IDRIIIIIII... (No IDR any more) 1 1 ignored IDR IDR IDR IDR... 1 >=2 ignored IDRII IDRII IDR... (1/3/ignore) >=2 0 1 IDRPPP IPPP I... (3/0/1) >=2 0 >=2 IDR(PBB)(PBB)(IBB) (6/0/3) (PBB)(IBB)(PBB)(IBB)... >=2 >=2 1 IDRPPPPP IPPPPP IPPPPP (6/18/1) IDRPPPPP IPPPPP IPPPPP... >=2 >=2 >=2 {IDR(PBB)(PBB)(IBB)(PBB)(IBB)(PBB)} (6/18/3) {IDR(PBB)(PBB)(IBB)(PBB)(IBB)(PBB)}... {IDR(PBB)(PBB)(IBB)(PBB)} (6/12/3) {IDR(PBB)(PBB)(IBB)(PBB)}... {IDR(PBB)(PBB)} (6/6/3) {IDR(PBB)(PBB)}. */ // General pts/dts strategy: // // Getting pts and dts right with variable frame rate (VFR) and B-frames can be a // bit tricky. We assume first of all that the frame rate never goes _above_ // MAX_FPS, which gives us a frame period N. The decoder can always decode // in at least this speed, as long at dts <= pts (the frame is not attempted // presented before it is decoded). Furthermore, we never have longer chains of // B-frames than a fixed constant C. (In a B-frame chain, we say that the base // I/P-frame has order O=0, the B-frame depending on it directly has order O=1, // etc. The last frame in the chain, which no B-frames depend on, is the “tip” // frame, with an order O <= C.) // // Many strategies are possible, but we establish these rules: // // - Tip frames have dts = pts - (C-O)*N. // - Non-tip frames have dts = dts_last + N. // // An example, with C=2 and N=10 and the data flow showed with arrows: // // I B P B B P // pts: 30 40 50 60 70 80 // ↓ ↓ ↓ // dts: 10 30 20 60 50←40 // | | ↑ ↑ // `--|--' | // `----------' // // To show that this works fine also with irregular spacings, let's say that // the third frame is delayed a bit (something earlier was dropped). Now the // situation looks like this: // // I B P B B P // pts: 30 40 80 90 100 110 // ↓ ↓ ↓ // dts: 10 30 20 90 50←40 // | | ↑ ↑ // `--|--' | // `----------' // // The resetting on every tip frame makes sure dts never ends up lagging a lot // behind pts, and the subtraction of (C-O)*N makes sure pts <= dts. // // In the output of this function, if is >= 0, it means to reset the // dts from the current pts minus , while if it's -1, the frame is not // a tip frame and should be given a dts based on the previous one. #define FRAME_P 0 #define FRAME_B 1 #define FRAME_I 2 #define FRAME_IDR 7 void encoding2display_order( int encoding_order, int intra_period, int intra_idr_period, int ip_period, int *displaying_order, int *frame_type, int *pts_lag) { int encoding_order_gop = 0; *pts_lag = 0; if (intra_period == 1) { /* all are I/IDR frames */ *displaying_order = encoding_order; if (intra_idr_period == 0) *frame_type = (encoding_order == 0)?FRAME_IDR:FRAME_I; else *frame_type = (encoding_order % intra_idr_period == 0)?FRAME_IDR:FRAME_I; return; } if (intra_period == 0) intra_idr_period = 0; if (ip_period == 1) { // No B-frames, sequence is like IDR PPPPP IPPPPP. encoding_order_gop = (intra_idr_period == 0) ? encoding_order : (encoding_order % intra_idr_period); *displaying_order = encoding_order; if (encoding_order_gop == 0) { /* the first frame */ *frame_type = FRAME_IDR; } else if (intra_period != 0 && /* have I frames */ encoding_order_gop >= 2 && (encoding_order_gop % intra_period == 0)) { *frame_type = FRAME_I; } else { *frame_type = FRAME_P; } return; } // We have B-frames. Sequence is like IDR (PBB)(PBB)(IBB)(PBB). encoding_order_gop = (intra_idr_period == 0) ? encoding_order : (encoding_order % (intra_idr_period + 1)); *pts_lag = -1; // Most frames are not tip frames. if (encoding_order_gop == 0) { /* the first frame */ *frame_type = FRAME_IDR; *displaying_order = encoding_order; // IDR frames are a special case; I honestly can't find the logic behind // why this is the right thing, but it seems to line up nicely in practice :-) *pts_lag = TIMEBASE / MAX_FPS; } else if (((encoding_order_gop - 1) % ip_period) != 0) { /* B frames */ *frame_type = FRAME_B; *displaying_order = encoding_order - 1; if ((encoding_order_gop % ip_period) == 0) { *pts_lag = 0; // Last B-frame. } } else if (intra_period != 0 && /* have I frames */ encoding_order_gop >= 2 && ((encoding_order_gop - 1) / ip_period % (intra_period / ip_period)) == 0) { *frame_type = FRAME_I; *displaying_order = encoding_order + ip_period - 1; } else { *frame_type = FRAME_P; *displaying_order = encoding_order + ip_period - 1; } } static const char *rc_to_string(int rc_mode) { switch (rc_mode) { case VA_RC_NONE: return "NONE"; case VA_RC_CBR: return "CBR"; case VA_RC_VBR: return "VBR"; case VA_RC_VCM: return "VCM"; case VA_RC_CQP: return "CQP"; case VA_RC_VBR_CONSTRAINED: return "VBR_CONSTRAINED"; default: return "Unknown"; } } void QuickSyncEncoderImpl::enable_zerocopy_if_possible() { if (global_flags.uncompressed_video_to_http) { fprintf(stderr, "Disabling zerocopy H.264 encoding due to --http-uncompressed-video.\n"); use_zerocopy = false; } else if (global_flags.x264_video_to_http) { fprintf(stderr, "Disabling zerocopy H.264 encoding due to --http-x264-video.\n"); use_zerocopy = false; } else { use_zerocopy = true; } } VADisplay QuickSyncEncoderImpl::va_open_display(const string &va_display) { if (va_display.empty()) { x11_display = XOpenDisplay(NULL); if (!x11_display) { fprintf(stderr, "error: can't connect to X server!\n"); return NULL; } enable_zerocopy_if_possible(); return vaGetDisplay(x11_display); } else if (va_display[0] != '/') { x11_display = XOpenDisplay(va_display.c_str()); if (!x11_display) { fprintf(stderr, "error: can't connect to X server!\n"); return NULL; } enable_zerocopy_if_possible(); return vaGetDisplay(x11_display); } else { drm_fd = open(va_display.c_str(), O_RDWR); if (drm_fd == -1) { perror(va_display.c_str()); return NULL; } use_zerocopy = false; return vaGetDisplayDRM(drm_fd); } } void QuickSyncEncoderImpl::va_close_display(VADisplay va_dpy) { if (x11_display) { XCloseDisplay(x11_display); x11_display = nullptr; } if (drm_fd != -1) { close(drm_fd); } } int QuickSyncEncoderImpl::init_va(const string &va_display) { VAProfile profile_list[]={VAProfileH264High, VAProfileH264Main, VAProfileH264Baseline, VAProfileH264ConstrainedBaseline}; VAEntrypoint *entrypoints; int num_entrypoints, slice_entrypoint; int support_encode = 0; int major_ver, minor_ver; VAStatus va_status; unsigned int i; va_dpy = va_open_display(va_display); va_status = vaInitialize(va_dpy, &major_ver, &minor_ver); CHECK_VASTATUS(va_status, "vaInitialize"); num_entrypoints = vaMaxNumEntrypoints(va_dpy); entrypoints = (VAEntrypoint *)malloc(num_entrypoints * sizeof(*entrypoints)); if (!entrypoints) { fprintf(stderr, "error: failed to initialize VA entrypoints array\n"); exit(1); } /* use the highest profile */ for (i = 0; i < sizeof(profile_list)/sizeof(profile_list[0]); i++) { if ((h264_profile != ~0) && h264_profile != profile_list[i]) continue; h264_profile = profile_list[i]; vaQueryConfigEntrypoints(va_dpy, h264_profile, entrypoints, &num_entrypoints); for (slice_entrypoint = 0; slice_entrypoint < num_entrypoints; slice_entrypoint++) { if (entrypoints[slice_entrypoint] == VAEntrypointEncSlice) { support_encode = 1; break; } } if (support_encode == 1) break; } if (support_encode == 0) { printf("Can't find VAEntrypointEncSlice for H264 profiles. If you are using a non-Intel GPU\n"); printf("but have one in your system, try launching Nageru with --va-display /dev/dri/renderD128\n"); printf("to use VA-API against DRM instead of X11.\n"); exit(1); } else { switch (h264_profile) { case VAProfileH264Baseline: ip_period = 1; constraint_set_flag |= (1 << 0); /* Annex A.2.1 */ h264_entropy_mode = 0; break; case VAProfileH264ConstrainedBaseline: constraint_set_flag |= (1 << 0 | 1 << 1); /* Annex A.2.2 */ ip_period = 1; break; case VAProfileH264Main: constraint_set_flag |= (1 << 1); /* Annex A.2.2 */ break; case VAProfileH264High: constraint_set_flag |= (1 << 3); /* Annex A.2.4 */ break; default: h264_profile = VAProfileH264Baseline; ip_period = 1; constraint_set_flag |= (1 << 0); /* Annex A.2.1 */ break; } } VAConfigAttrib attrib[VAConfigAttribTypeMax]; /* find out the format for the render target, and rate control mode */ for (i = 0; i < VAConfigAttribTypeMax; i++) attrib[i].type = (VAConfigAttribType)i; va_status = vaGetConfigAttributes(va_dpy, h264_profile, VAEntrypointEncSlice, &attrib[0], VAConfigAttribTypeMax); CHECK_VASTATUS(va_status, "vaGetConfigAttributes"); /* check the interested configattrib */ if ((attrib[VAConfigAttribRTFormat].value & VA_RT_FORMAT_YUV420) == 0) { printf("Not find desired YUV420 RT format\n"); exit(1); } else { config_attrib[config_attrib_num].type = VAConfigAttribRTFormat; config_attrib[config_attrib_num].value = VA_RT_FORMAT_YUV420; config_attrib_num++; } if (attrib[VAConfigAttribRateControl].value != VA_ATTRIB_NOT_SUPPORTED) { int tmp = attrib[VAConfigAttribRateControl].value; if (rc_mode == -1 || !(rc_mode & tmp)) { if (rc_mode != -1) { printf("Warning: Don't support the specified RateControl mode: %s!!!, switch to ", rc_to_string(rc_mode)); } for (i = 0; i < sizeof(rc_default_modes) / sizeof(rc_default_modes[0]); i++) { if (rc_default_modes[i] & tmp) { rc_mode = rc_default_modes[i]; break; } } } config_attrib[config_attrib_num].type = VAConfigAttribRateControl; config_attrib[config_attrib_num].value = rc_mode; config_attrib_num++; } if (attrib[VAConfigAttribEncPackedHeaders].value != VA_ATTRIB_NOT_SUPPORTED) { int tmp = attrib[VAConfigAttribEncPackedHeaders].value; h264_packedheader = 1; config_attrib[config_attrib_num].type = VAConfigAttribEncPackedHeaders; config_attrib[config_attrib_num].value = VA_ENC_PACKED_HEADER_NONE; if (tmp & VA_ENC_PACKED_HEADER_SEQUENCE) { config_attrib[config_attrib_num].value |= VA_ENC_PACKED_HEADER_SEQUENCE; } if (tmp & VA_ENC_PACKED_HEADER_PICTURE) { config_attrib[config_attrib_num].value |= VA_ENC_PACKED_HEADER_PICTURE; } if (tmp & VA_ENC_PACKED_HEADER_SLICE) { config_attrib[config_attrib_num].value |= VA_ENC_PACKED_HEADER_SLICE; } if (tmp & VA_ENC_PACKED_HEADER_MISC) { config_attrib[config_attrib_num].value |= VA_ENC_PACKED_HEADER_MISC; } enc_packed_header_idx = config_attrib_num; config_attrib_num++; } if (attrib[VAConfigAttribEncInterlaced].value != VA_ATTRIB_NOT_SUPPORTED) { config_attrib[config_attrib_num].type = VAConfigAttribEncInterlaced; config_attrib[config_attrib_num].value = VA_ENC_PACKED_HEADER_NONE; config_attrib_num++; } if (attrib[VAConfigAttribEncMaxRefFrames].value != VA_ATTRIB_NOT_SUPPORTED) { h264_maxref = attrib[VAConfigAttribEncMaxRefFrames].value; } free(entrypoints); return 0; } int QuickSyncEncoderImpl::setup_encode() { VAStatus va_status; VASurfaceID *tmp_surfaceid; int codedbuf_size, i; VASurfaceID src_surface[SURFACE_NUM]; VASurfaceID ref_surface[SURFACE_NUM]; va_status = vaCreateConfig(va_dpy, h264_profile, VAEntrypointEncSlice, &config_attrib[0], config_attrib_num, &config_id); CHECK_VASTATUS(va_status, "vaCreateConfig"); /* create source surfaces */ va_status = vaCreateSurfaces(va_dpy, VA_RT_FORMAT_YUV420, frame_width_mbaligned, frame_height_mbaligned, &src_surface[0], SURFACE_NUM, NULL, 0); CHECK_VASTATUS(va_status, "vaCreateSurfaces"); /* create reference surfaces */ va_status = vaCreateSurfaces(va_dpy, VA_RT_FORMAT_YUV420, frame_width_mbaligned, frame_height_mbaligned, &ref_surface[0], SURFACE_NUM, NULL, 0); CHECK_VASTATUS(va_status, "vaCreateSurfaces"); tmp_surfaceid = (VASurfaceID *)calloc(2 * SURFACE_NUM, sizeof(VASurfaceID)); memcpy(tmp_surfaceid, src_surface, SURFACE_NUM * sizeof(VASurfaceID)); memcpy(tmp_surfaceid + SURFACE_NUM, ref_surface, SURFACE_NUM * sizeof(VASurfaceID)); /* Create a context for this encode pipe */ va_status = vaCreateContext(va_dpy, config_id, frame_width_mbaligned, frame_height_mbaligned, VA_PROGRESSIVE, tmp_surfaceid, 2 * SURFACE_NUM, &context_id); CHECK_VASTATUS(va_status, "vaCreateContext"); free(tmp_surfaceid); codedbuf_size = (frame_width_mbaligned * frame_height_mbaligned * 400) / (16*16); for (i = 0; i < SURFACE_NUM; i++) { /* create coded buffer once for all * other VA buffers which won't be used again after vaRenderPicture. * so APP can always vaCreateBuffer for every frame * but coded buffer need to be mapped and accessed after vaRenderPicture/vaEndPicture * so VA won't maintain the coded buffer */ va_status = vaCreateBuffer(va_dpy, context_id, VAEncCodedBufferType, codedbuf_size, 1, NULL, &gl_surfaces[i].coded_buf); CHECK_VASTATUS(va_status, "vaCreateBuffer"); } /* create OpenGL objects */ //glGenFramebuffers(SURFACE_NUM, fbos); for (i = 0; i < SURFACE_NUM; i++) { if (use_zerocopy) { gl_surfaces[i].y_tex = resource_pool->create_2d_texture(GL_R8, 1, 1); gl_surfaces[i].cbcr_tex = resource_pool->create_2d_texture(GL_RG8, 1, 1); } else { gl_surfaces[i].y_tex = resource_pool->create_2d_texture(GL_R8, frame_width, frame_height); gl_surfaces[i].cbcr_tex = resource_pool->create_2d_texture(GL_RG8, frame_width / 2, frame_height / 2); // Generate a PBO to read into. It doesn't necessarily fit 1:1 with the VA-API // buffers, due to potentially differing pitch. glGenBuffers(1, &gl_surfaces[i].pbo); glBindBuffer(GL_PIXEL_PACK_BUFFER, gl_surfaces[i].pbo); glBufferStorage(GL_PIXEL_PACK_BUFFER, frame_width * frame_height * 2, nullptr, GL_MAP_READ_BIT | GL_MAP_WRITE_BIT | GL_MAP_PERSISTENT_BIT); uint8_t *ptr = (uint8_t *)glMapBufferRange(GL_PIXEL_PACK_BUFFER, 0, frame_width * frame_height * 2, GL_MAP_READ_BIT | GL_MAP_PERSISTENT_BIT); gl_surfaces[i].y_offset = 0; gl_surfaces[i].cbcr_offset = frame_width * frame_height; gl_surfaces[i].y_ptr = ptr + gl_surfaces[i].y_offset; gl_surfaces[i].cbcr_ptr = ptr + gl_surfaces[i].cbcr_offset; glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); } } for (i = 0; i < SURFACE_NUM; i++) { gl_surfaces[i].src_surface = src_surface[i]; gl_surfaces[i].ref_surface = ref_surface[i]; } return 0; } // Given a list like 1 9 3 0 2 8 4 and a pivot element 3, will produce // // 2 1 0 [3] 4 8 9 template static void sort_two(T *begin, T *end, const T &pivot, const C &less_than) { T *middle = partition(begin, end, [&](const T &elem) { return less_than(elem, pivot); }); sort(begin, middle, [&](const T &a, const T &b) { return less_than(b, a); }); sort(middle, end, less_than); } void QuickSyncEncoderImpl::update_ReferenceFrames(int frame_type) { int i; if (frame_type == FRAME_B) return; CurrentCurrPic.flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE; numShortTerm++; if (numShortTerm > num_ref_frames) numShortTerm = num_ref_frames; for (i=numShortTerm-1; i>0; i--) ReferenceFrames[i] = ReferenceFrames[i-1]; ReferenceFrames[0] = CurrentCurrPic; current_frame_num++; if (current_frame_num > MaxFrameNum) current_frame_num = 0; } int QuickSyncEncoderImpl::update_RefPicList(int frame_type) { const auto descending_by_frame_idx = [](const VAPictureH264 &a, const VAPictureH264 &b) { return a.frame_idx > b.frame_idx; }; const auto ascending_by_top_field_order_cnt = [](const VAPictureH264 &a, const VAPictureH264 &b) { return a.TopFieldOrderCnt < b.TopFieldOrderCnt; }; const auto descending_by_top_field_order_cnt = [](const VAPictureH264 &a, const VAPictureH264 &b) { return a.TopFieldOrderCnt > b.TopFieldOrderCnt; }; if (frame_type == FRAME_P) { memcpy(RefPicList0_P, ReferenceFrames, numShortTerm * sizeof(VAPictureH264)); sort(&RefPicList0_P[0], &RefPicList0_P[numShortTerm], descending_by_frame_idx); } else if (frame_type == FRAME_B) { memcpy(RefPicList0_B, ReferenceFrames, numShortTerm * sizeof(VAPictureH264)); sort_two(&RefPicList0_B[0], &RefPicList0_B[numShortTerm], CurrentCurrPic, ascending_by_top_field_order_cnt); memcpy(RefPicList1_B, ReferenceFrames, numShortTerm * sizeof(VAPictureH264)); sort_two(&RefPicList1_B[0], &RefPicList1_B[numShortTerm], CurrentCurrPic, descending_by_top_field_order_cnt); } return 0; } int QuickSyncEncoderImpl::render_sequence() { VABufferID seq_param_buf, rc_param_buf, render_id[2]; VAStatus va_status; VAEncMiscParameterBuffer *misc_param; VAEncMiscParameterRateControl *misc_rate_ctrl; seq_param.level_idc = 41 /*SH_LEVEL_3*/; seq_param.picture_width_in_mbs = frame_width_mbaligned / 16; seq_param.picture_height_in_mbs = frame_height_mbaligned / 16; seq_param.bits_per_second = frame_bitrate; seq_param.intra_period = intra_period; seq_param.intra_idr_period = intra_idr_period; seq_param.ip_period = ip_period; seq_param.max_num_ref_frames = num_ref_frames; seq_param.seq_fields.bits.frame_mbs_only_flag = 1; seq_param.time_scale = TIMEBASE * 2; seq_param.num_units_in_tick = 1; /* Tc = num_units_in_tick / scale */ seq_param.seq_fields.bits.log2_max_pic_order_cnt_lsb_minus4 = Log2MaxPicOrderCntLsb - 4; seq_param.seq_fields.bits.log2_max_frame_num_minus4 = Log2MaxFrameNum - 4;; seq_param.seq_fields.bits.frame_mbs_only_flag = 1; seq_param.seq_fields.bits.chroma_format_idc = 1; seq_param.seq_fields.bits.direct_8x8_inference_flag = 1; if (frame_width != frame_width_mbaligned || frame_height != frame_height_mbaligned) { seq_param.frame_cropping_flag = 1; seq_param.frame_crop_left_offset = 0; seq_param.frame_crop_right_offset = (frame_width_mbaligned - frame_width)/2; seq_param.frame_crop_top_offset = 0; seq_param.frame_crop_bottom_offset = (frame_height_mbaligned - frame_height)/2; } va_status = vaCreateBuffer(va_dpy, context_id, VAEncSequenceParameterBufferType, sizeof(seq_param), 1, &seq_param, &seq_param_buf); CHECK_VASTATUS(va_status, "vaCreateBuffer"); va_status = vaCreateBuffer(va_dpy, context_id, VAEncMiscParameterBufferType, sizeof(VAEncMiscParameterBuffer) + sizeof(VAEncMiscParameterRateControl), 1, NULL, &rc_param_buf); CHECK_VASTATUS(va_status, "vaCreateBuffer"); vaMapBuffer(va_dpy, rc_param_buf, (void **)&misc_param); misc_param->type = VAEncMiscParameterTypeRateControl; misc_rate_ctrl = (VAEncMiscParameterRateControl *)misc_param->data; memset(misc_rate_ctrl, 0, sizeof(*misc_rate_ctrl)); misc_rate_ctrl->bits_per_second = frame_bitrate; misc_rate_ctrl->target_percentage = 66; misc_rate_ctrl->window_size = 1000; misc_rate_ctrl->initial_qp = initial_qp; misc_rate_ctrl->min_qp = minimal_qp; misc_rate_ctrl->basic_unit_size = 0; vaUnmapBuffer(va_dpy, rc_param_buf); render_id[0] = seq_param_buf; render_id[1] = rc_param_buf; render_picture_and_delete(va_dpy, context_id, &render_id[0], 2); return 0; } static int calc_poc(int pic_order_cnt_lsb, int frame_type) { static int PicOrderCntMsb_ref = 0, pic_order_cnt_lsb_ref = 0; int prevPicOrderCntMsb, prevPicOrderCntLsb; int PicOrderCntMsb, TopFieldOrderCnt; if (frame_type == FRAME_IDR) prevPicOrderCntMsb = prevPicOrderCntLsb = 0; else { prevPicOrderCntMsb = PicOrderCntMsb_ref; prevPicOrderCntLsb = pic_order_cnt_lsb_ref; } if ((pic_order_cnt_lsb < prevPicOrderCntLsb) && ((prevPicOrderCntLsb - pic_order_cnt_lsb) >= (int)(MaxPicOrderCntLsb / 2))) PicOrderCntMsb = prevPicOrderCntMsb + MaxPicOrderCntLsb; else if ((pic_order_cnt_lsb > prevPicOrderCntLsb) && ((pic_order_cnt_lsb - prevPicOrderCntLsb) > (int)(MaxPicOrderCntLsb / 2))) PicOrderCntMsb = prevPicOrderCntMsb - MaxPicOrderCntLsb; else PicOrderCntMsb = prevPicOrderCntMsb; TopFieldOrderCnt = PicOrderCntMsb + pic_order_cnt_lsb; if (frame_type != FRAME_B) { PicOrderCntMsb_ref = PicOrderCntMsb; pic_order_cnt_lsb_ref = pic_order_cnt_lsb; } return TopFieldOrderCnt; } int QuickSyncEncoderImpl::render_picture(int frame_type, int display_frame_num, int gop_start_display_frame_num) { VABufferID pic_param_buf; VAStatus va_status; int i = 0; pic_param.CurrPic.picture_id = gl_surfaces[display_frame_num % SURFACE_NUM].ref_surface; pic_param.CurrPic.frame_idx = current_frame_num; pic_param.CurrPic.flags = 0; pic_param.CurrPic.TopFieldOrderCnt = calc_poc((display_frame_num - gop_start_display_frame_num) % MaxPicOrderCntLsb, frame_type); pic_param.CurrPic.BottomFieldOrderCnt = pic_param.CurrPic.TopFieldOrderCnt; CurrentCurrPic = pic_param.CurrPic; memcpy(pic_param.ReferenceFrames, ReferenceFrames, numShortTerm*sizeof(VAPictureH264)); for (i = numShortTerm; i < MAX_NUM_REF1; i++) { pic_param.ReferenceFrames[i].picture_id = VA_INVALID_SURFACE; pic_param.ReferenceFrames[i].flags = VA_PICTURE_H264_INVALID; } pic_param.pic_fields.bits.idr_pic_flag = (frame_type == FRAME_IDR); pic_param.pic_fields.bits.reference_pic_flag = (frame_type != FRAME_B); pic_param.pic_fields.bits.entropy_coding_mode_flag = h264_entropy_mode; pic_param.pic_fields.bits.deblocking_filter_control_present_flag = 1; pic_param.frame_num = current_frame_num; pic_param.coded_buf = gl_surfaces[display_frame_num % SURFACE_NUM].coded_buf; pic_param.last_picture = false; // FIXME pic_param.pic_init_qp = initial_qp; va_status = vaCreateBuffer(va_dpy, context_id, VAEncPictureParameterBufferType, sizeof(pic_param), 1, &pic_param, &pic_param_buf); CHECK_VASTATUS(va_status, "vaCreateBuffer"); render_picture_and_delete(va_dpy, context_id, &pic_param_buf, 1); return 0; } int QuickSyncEncoderImpl::render_packedsequence() { VAEncPackedHeaderParameterBuffer packedheader_param_buffer; VABufferID packedseq_para_bufid, packedseq_data_bufid, render_id[2]; unsigned int length_in_bits; unsigned char *packedseq_buffer = NULL; VAStatus va_status; length_in_bits = build_packed_seq_buffer(&packedseq_buffer); packedheader_param_buffer.type = VAEncPackedHeaderSequence; packedheader_param_buffer.bit_length = length_in_bits; /*length_in_bits*/ packedheader_param_buffer.has_emulation_bytes = 0; va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderParameterBufferType, sizeof(packedheader_param_buffer), 1, &packedheader_param_buffer, &packedseq_para_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderDataBufferType, (length_in_bits + 7) / 8, 1, packedseq_buffer, &packedseq_data_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); render_id[0] = packedseq_para_bufid; render_id[1] = packedseq_data_bufid; render_picture_and_delete(va_dpy, context_id, render_id, 2); free(packedseq_buffer); return 0; } int QuickSyncEncoderImpl::render_packedpicture() { VAEncPackedHeaderParameterBuffer packedheader_param_buffer; VABufferID packedpic_para_bufid, packedpic_data_bufid, render_id[2]; unsigned int length_in_bits; unsigned char *packedpic_buffer = NULL; VAStatus va_status; length_in_bits = build_packed_pic_buffer(&packedpic_buffer); packedheader_param_buffer.type = VAEncPackedHeaderPicture; packedheader_param_buffer.bit_length = length_in_bits; packedheader_param_buffer.has_emulation_bytes = 0; va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderParameterBufferType, sizeof(packedheader_param_buffer), 1, &packedheader_param_buffer, &packedpic_para_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderDataBufferType, (length_in_bits + 7) / 8, 1, packedpic_buffer, &packedpic_data_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); render_id[0] = packedpic_para_bufid; render_id[1] = packedpic_data_bufid; render_picture_and_delete(va_dpy, context_id, render_id, 2); free(packedpic_buffer); return 0; } void QuickSyncEncoderImpl::render_packedslice() { VAEncPackedHeaderParameterBuffer packedheader_param_buffer; VABufferID packedslice_para_bufid, packedslice_data_bufid, render_id[2]; unsigned int length_in_bits; unsigned char *packedslice_buffer = NULL; VAStatus va_status; length_in_bits = build_packed_slice_buffer(&packedslice_buffer); packedheader_param_buffer.type = VAEncPackedHeaderSlice; packedheader_param_buffer.bit_length = length_in_bits; packedheader_param_buffer.has_emulation_bytes = 0; va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderParameterBufferType, sizeof(packedheader_param_buffer), 1, &packedheader_param_buffer, &packedslice_para_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); va_status = vaCreateBuffer(va_dpy, context_id, VAEncPackedHeaderDataBufferType, (length_in_bits + 7) / 8, 1, packedslice_buffer, &packedslice_data_bufid); CHECK_VASTATUS(va_status, "vaCreateBuffer"); render_id[0] = packedslice_para_bufid; render_id[1] = packedslice_data_bufid; render_picture_and_delete(va_dpy, context_id, render_id, 2); free(packedslice_buffer); } int QuickSyncEncoderImpl::render_slice(int encoding_frame_num, int display_frame_num, int gop_start_display_frame_num, int frame_type) { VABufferID slice_param_buf; VAStatus va_status; int i; update_RefPicList(frame_type); /* one frame, one slice */ slice_param.macroblock_address = 0; slice_param.num_macroblocks = frame_width_mbaligned * frame_height_mbaligned/(16*16); /* Measured by MB */ slice_param.slice_type = (frame_type == FRAME_IDR)?2:frame_type; if (frame_type == FRAME_IDR) { if (encoding_frame_num != 0) ++slice_param.idr_pic_id; } else if (frame_type == FRAME_P) { int refpiclist0_max = h264_maxref & 0xffff; memcpy(slice_param.RefPicList0, RefPicList0_P, refpiclist0_max*sizeof(VAPictureH264)); for (i = refpiclist0_max; i < MAX_NUM_REF2; i++) { slice_param.RefPicList0[i].picture_id = VA_INVALID_SURFACE; slice_param.RefPicList0[i].flags = VA_PICTURE_H264_INVALID; } } else if (frame_type == FRAME_B) { int refpiclist0_max = h264_maxref & 0xffff; int refpiclist1_max = (h264_maxref >> 16) & 0xffff; memcpy(slice_param.RefPicList0, RefPicList0_B, refpiclist0_max*sizeof(VAPictureH264)); for (i = refpiclist0_max; i < MAX_NUM_REF2; i++) { slice_param.RefPicList0[i].picture_id = VA_INVALID_SURFACE; slice_param.RefPicList0[i].flags = VA_PICTURE_H264_INVALID; } memcpy(slice_param.RefPicList1, RefPicList1_B, refpiclist1_max*sizeof(VAPictureH264)); for (i = refpiclist1_max; i < MAX_NUM_REF2; i++) { slice_param.RefPicList1[i].picture_id = VA_INVALID_SURFACE; slice_param.RefPicList1[i].flags = VA_PICTURE_H264_INVALID; } } slice_param.slice_alpha_c0_offset_div2 = 0; slice_param.slice_beta_offset_div2 = 0; slice_param.direct_spatial_mv_pred_flag = 1; slice_param.pic_order_cnt_lsb = (display_frame_num - gop_start_display_frame_num) % MaxPicOrderCntLsb; if (h264_packedheader && config_attrib[enc_packed_header_idx].value & VA_ENC_PACKED_HEADER_SLICE) render_packedslice(); va_status = vaCreateBuffer(va_dpy, context_id, VAEncSliceParameterBufferType, sizeof(slice_param), 1, &slice_param, &slice_param_buf); CHECK_VASTATUS(va_status, "vaCreateBuffer"); render_picture_and_delete(va_dpy, context_id, &slice_param_buf, 1); return 0; } void QuickSyncEncoderImpl::save_codeddata(storage_task task) { VACodedBufferSegment *buf_list = NULL; VAStatus va_status; string data; va_status = vaMapBuffer(va_dpy, gl_surfaces[task.display_order % SURFACE_NUM].coded_buf, (void **)(&buf_list)); CHECK_VASTATUS(va_status, "vaMapBuffer"); while (buf_list != NULL) { data.append(reinterpret_cast(buf_list->buf), buf_list->size); buf_list = (VACodedBufferSegment *) buf_list->next; } vaUnmapBuffer(va_dpy, gl_surfaces[task.display_order % SURFACE_NUM].coded_buf); { // Add video. AVPacket pkt; memset(&pkt, 0, sizeof(pkt)); pkt.buf = nullptr; pkt.data = reinterpret_cast(&data[0]); pkt.size = data.size(); pkt.stream_index = 0; if (task.frame_type == FRAME_IDR) { pkt.flags = AV_PKT_FLAG_KEY; } else { pkt.flags = 0; } pkt.duration = task.duration; if (file_mux) { file_mux->add_packet(pkt, task.pts + global_delay(), task.dts + global_delay()); } if (!global_flags.uncompressed_video_to_http && !global_flags.x264_video_to_http) { stream_mux->add_packet(pkt, task.pts + global_delay(), task.dts + global_delay()); } } } // this is weird. but it seems to put a new frame onto the queue void QuickSyncEncoderImpl::storage_task_enqueue(storage_task task) { unique_lock lock(storage_task_queue_mutex); storage_task_queue.push(move(task)); storage_task_queue_changed.notify_all(); } void QuickSyncEncoderImpl::storage_task_thread() { for ( ;; ) { storage_task current; { // wait until there's an encoded frame unique_lock lock(storage_task_queue_mutex); storage_task_queue_changed.wait(lock, [this]{ return storage_thread_should_quit || !storage_task_queue.empty(); }); if (storage_thread_should_quit && storage_task_queue.empty()) return; current = move(storage_task_queue.front()); storage_task_queue.pop(); } VAStatus va_status; // waits for data, then saves it to disk. va_status = vaSyncSurface(va_dpy, gl_surfaces[current.display_order % SURFACE_NUM].src_surface); CHECK_VASTATUS(va_status, "vaSyncSurface"); save_codeddata(move(current)); { unique_lock lock(storage_task_queue_mutex); srcsurface_status[current.display_order % SURFACE_NUM] = SRC_SURFACE_FREE; storage_task_queue_changed.notify_all(); } } } void QuickSyncEncoderImpl::release_encode() { for (unsigned i = 0; i < SURFACE_NUM; i++) { vaDestroyBuffer(va_dpy, gl_surfaces[i].coded_buf); vaDestroySurfaces(va_dpy, &gl_surfaces[i].src_surface, 1); vaDestroySurfaces(va_dpy, &gl_surfaces[i].ref_surface, 1); } vaDestroyContext(va_dpy, context_id); vaDestroyConfig(va_dpy, config_id); } void QuickSyncEncoderImpl::release_gl_resources() { assert(is_shutdown); if (has_released_gl_resources) { return; } for (unsigned i = 0; i < SURFACE_NUM; i++) { if (!use_zerocopy) { glBindBuffer(GL_PIXEL_PACK_BUFFER, gl_surfaces[i].pbo); glUnmapBuffer(GL_PIXEL_PACK_BUFFER); glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); glDeleteBuffers(1, &gl_surfaces[i].pbo); } resource_pool->release_2d_texture(gl_surfaces[i].y_tex); resource_pool->release_2d_texture(gl_surfaces[i].cbcr_tex); } has_released_gl_resources = true; } int QuickSyncEncoderImpl::deinit_va() { vaTerminate(va_dpy); va_close_display(va_dpy); return 0; } QuickSyncEncoderImpl::QuickSyncEncoderImpl(const std::string &filename, movit::ResourcePool *resource_pool, QSurface *surface, const string &va_display, int width, int height, AVOutputFormat *oformat, X264Encoder *x264_encoder, DiskSpaceEstimator *disk_space_estimator) : current_storage_frame(0), resource_pool(resource_pool), surface(surface), x264_encoder(x264_encoder), frame_width(width), frame_height(height), disk_space_estimator(disk_space_estimator) { file_audio_encoder.reset(new AudioEncoder(AUDIO_OUTPUT_CODEC_NAME, DEFAULT_AUDIO_OUTPUT_BIT_RATE, oformat)); open_output_file(filename); file_audio_encoder->add_mux(file_mux.get()); frame_width_mbaligned = (frame_width + 15) & (~15); frame_height_mbaligned = (frame_height + 15) & (~15); //print_input(); if (global_flags.uncompressed_video_to_http || global_flags.x264_video_to_http) { reorderer.reset(new FrameReorderer(ip_period - 1, frame_width, frame_height)); } if (global_flags.x264_video_to_http) { assert(x264_encoder != nullptr); } else { assert(x264_encoder == nullptr); } init_va(va_display); setup_encode(); // No frames are ready yet. memset(srcsurface_status, SRC_SURFACE_FREE, sizeof(srcsurface_status)); memset(&seq_param, 0, sizeof(seq_param)); memset(&pic_param, 0, sizeof(pic_param)); memset(&slice_param, 0, sizeof(slice_param)); storage_thread = thread(&QuickSyncEncoderImpl::storage_task_thread, this); encode_thread = thread([this]{ //SDL_GL_MakeCurrent(window, context); QOpenGLContext *context = create_context(this->surface); eglBindAPI(EGL_OPENGL_API); if (!make_current(context, this->surface)) { printf("display=%p surface=%p context=%p curr=%p err=%d\n", eglGetCurrentDisplay(), this->surface, context, eglGetCurrentContext(), eglGetError()); exit(1); } encode_thread_func(); delete_context(context); }); } QuickSyncEncoderImpl::~QuickSyncEncoderImpl() { shutdown(); release_gl_resources(); } bool QuickSyncEncoderImpl::begin_frame(GLuint *y_tex, GLuint *cbcr_tex) { assert(!is_shutdown); { // Wait until this frame slot is done encoding. unique_lock lock(storage_task_queue_mutex); if (srcsurface_status[current_storage_frame % SURFACE_NUM] != SRC_SURFACE_FREE) { fprintf(stderr, "Warning: Slot %d (for frame %d) is still encoding, rendering has to wait for H.264 encoder\n", current_storage_frame % SURFACE_NUM, current_storage_frame); } storage_task_queue_changed.wait(lock, [this]{ return storage_thread_should_quit || (srcsurface_status[current_storage_frame % SURFACE_NUM] == SRC_SURFACE_FREE); }); srcsurface_status[current_storage_frame % SURFACE_NUM] = SRC_SURFACE_IN_ENCODING; if (storage_thread_should_quit) return false; } //*fbo = fbos[current_storage_frame % SURFACE_NUM]; GLSurface *surf = &gl_surfaces[current_storage_frame % SURFACE_NUM]; *y_tex = surf->y_tex; *cbcr_tex = surf->cbcr_tex; VAStatus va_status = vaDeriveImage(va_dpy, surf->src_surface, &surf->surface_image); CHECK_VASTATUS(va_status, "vaDeriveImage"); if (use_zerocopy) { VABufferInfo buf_info; buf_info.mem_type = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME; // or VA_SURFACE_ATTRIB_MEM_TYPE_KERNEL_DRM? va_status = vaAcquireBufferHandle(va_dpy, surf->surface_image.buf, &buf_info); CHECK_VASTATUS(va_status, "vaAcquireBufferHandle"); // Create Y image. surf->y_egl_image = EGL_NO_IMAGE_KHR; EGLint y_attribs[] = { EGL_WIDTH, frame_width, EGL_HEIGHT, frame_height, EGL_LINUX_DRM_FOURCC_EXT, fourcc_code('R', '8', ' ', ' '), EGL_DMA_BUF_PLANE0_FD_EXT, EGLint(buf_info.handle), EGL_DMA_BUF_PLANE0_OFFSET_EXT, EGLint(surf->surface_image.offsets[0]), EGL_DMA_BUF_PLANE0_PITCH_EXT, EGLint(surf->surface_image.pitches[0]), EGL_NONE }; surf->y_egl_image = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, y_attribs); assert(surf->y_egl_image != EGL_NO_IMAGE_KHR); // Associate Y image to a texture. glBindTexture(GL_TEXTURE_2D, *y_tex); glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, surf->y_egl_image); // Create CbCr image. surf->cbcr_egl_image = EGL_NO_IMAGE_KHR; EGLint cbcr_attribs[] = { EGL_WIDTH, frame_width, EGL_HEIGHT, frame_height, EGL_LINUX_DRM_FOURCC_EXT, fourcc_code('G', 'R', '8', '8'), EGL_DMA_BUF_PLANE0_FD_EXT, EGLint(buf_info.handle), EGL_DMA_BUF_PLANE0_OFFSET_EXT, EGLint(surf->surface_image.offsets[1]), EGL_DMA_BUF_PLANE0_PITCH_EXT, EGLint(surf->surface_image.pitches[1]), EGL_NONE }; surf->cbcr_egl_image = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, cbcr_attribs); assert(surf->cbcr_egl_image != EGL_NO_IMAGE_KHR); // Associate CbCr image to a texture. glBindTexture(GL_TEXTURE_2D, *cbcr_tex); glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, surf->cbcr_egl_image); } return true; } void QuickSyncEncoderImpl::add_audio(int64_t pts, vector audio) { assert(!is_shutdown); file_audio_encoder->encode_audio(audio, pts + global_delay()); } RefCountedGLsync QuickSyncEncoderImpl::end_frame(int64_t pts, int64_t duration, const vector &input_frames) { assert(!is_shutdown); if (!use_zerocopy) { GLSurface *surf = &gl_surfaces[current_storage_frame % SURFACE_NUM]; glPixelStorei(GL_PACK_ROW_LENGTH, 0); check_error(); glBindBuffer(GL_PIXEL_PACK_BUFFER, surf->pbo); check_error(); glBindTexture(GL_TEXTURE_2D, surf->y_tex); check_error(); glGetTexImage(GL_TEXTURE_2D, 0, GL_RED, GL_UNSIGNED_BYTE, BUFFER_OFFSET(surf->y_offset)); check_error(); glBindTexture(GL_TEXTURE_2D, surf->cbcr_tex); check_error(); glGetTexImage(GL_TEXTURE_2D, 0, GL_RG, GL_UNSIGNED_BYTE, BUFFER_OFFSET(surf->cbcr_offset)); check_error(); glBindTexture(GL_TEXTURE_2D, 0); check_error(); glBindBuffer(GL_PIXEL_PACK_BUFFER, 0); check_error(); glMemoryBarrier(GL_TEXTURE_UPDATE_BARRIER_BIT | GL_CLIENT_MAPPED_BUFFER_BARRIER_BIT); check_error(); } RefCountedGLsync fence = RefCountedGLsync(GL_SYNC_GPU_COMMANDS_COMPLETE, /*flags=*/0); check_error(); glFlush(); // Make the H.264 thread see the fence as soon as possible. check_error(); { unique_lock lock(frame_queue_mutex); pending_video_frames[current_storage_frame] = PendingFrame{ fence, input_frames, pts, duration }; ++current_storage_frame; } frame_queue_nonempty.notify_all(); return fence; } void QuickSyncEncoderImpl::shutdown() { if (is_shutdown) { return; } { unique_lock lock(frame_queue_mutex); encode_thread_should_quit = true; frame_queue_nonempty.notify_all(); } encode_thread.join(); { unique_lock lock(storage_task_queue_mutex); storage_thread_should_quit = true; frame_queue_nonempty.notify_all(); storage_task_queue_changed.notify_all(); } storage_thread.join(); // Encode any leftover audio in the queues, and also any delayed frames. file_audio_encoder->encode_last_audio(); release_encode(); deinit_va(); file_mux.reset(); is_shutdown = true; } void QuickSyncEncoderImpl::open_output_file(const std::string &filename) { AVFormatContext *avctx = avformat_alloc_context(); avctx->oformat = av_guess_format(NULL, filename.c_str(), NULL); assert(filename.size() < sizeof(avctx->filename) - 1); strcpy(avctx->filename, filename.c_str()); string url = "file:" + filename; int ret = avio_open2(&avctx->pb, url.c_str(), AVIO_FLAG_WRITE, &avctx->interrupt_callback, NULL); if (ret < 0) { char tmp[AV_ERROR_MAX_STRING_SIZE]; fprintf(stderr, "%s: avio_open2() failed: %s\n", filename.c_str(), av_make_error_string(tmp, sizeof(tmp), ret)); exit(1); } string video_extradata = ""; // FIXME: See other comment about global headers. AVCodecParametersWithDeleter audio_codecpar = file_audio_encoder->get_codec_parameters(); file_mux.reset(new Mux(avctx, frame_width, frame_height, Mux::CODEC_H264, video_extradata, audio_codecpar.get(), TIMEBASE, std::bind(&DiskSpaceEstimator::report_write, disk_space_estimator, filename, _1))); } void QuickSyncEncoderImpl::encode_thread_func() { int64_t last_dts = -1; int gop_start_display_frame_num = 0; for (int encoding_frame_num = 0; ; ++encoding_frame_num) { PendingFrame frame; int pts_lag; int frame_type, display_frame_num; encoding2display_order(encoding_frame_num, intra_period, intra_idr_period, ip_period, &display_frame_num, &frame_type, &pts_lag); if (frame_type == FRAME_IDR) { numShortTerm = 0; current_frame_num = 0; gop_start_display_frame_num = display_frame_num; } { unique_lock lock(frame_queue_mutex); frame_queue_nonempty.wait(lock, [this, display_frame_num]{ return encode_thread_should_quit || pending_video_frames.count(display_frame_num) != 0; }); if (encode_thread_should_quit && pending_video_frames.count(display_frame_num) == 0) { // We have queued frames that were supposed to be B-frames, // but will be no P-frame to encode them against. Encode them all // as P-frames instead. Note that this happens under the mutex, // but nobody else uses it at this point, since we're shutting down, // so there's no contention. encode_remaining_frames_as_p(encoding_frame_num, gop_start_display_frame_num, last_dts); return; } else { frame = move(pending_video_frames[display_frame_num]); pending_video_frames.erase(display_frame_num); } } // Determine the dts of this frame. int64_t dts; if (pts_lag == -1) { assert(last_dts != -1); dts = last_dts + (TIMEBASE / MAX_FPS); } else { dts = frame.pts - pts_lag; } last_dts = dts; encode_frame(frame, encoding_frame_num, display_frame_num, gop_start_display_frame_num, frame_type, frame.pts, dts, frame.duration); } } void QuickSyncEncoderImpl::encode_remaining_frames_as_p(int encoding_frame_num, int gop_start_display_frame_num, int64_t last_dts) { if (pending_video_frames.empty()) { return; } for (auto &pending_frame : pending_video_frames) { int display_frame_num = pending_frame.first; assert(display_frame_num > 0); PendingFrame frame = move(pending_frame.second); int64_t dts = last_dts + (TIMEBASE / MAX_FPS); printf("Finalizing encode: Encoding leftover frame %d as P-frame instead of B-frame.\n", display_frame_num); encode_frame(frame, encoding_frame_num++, display_frame_num, gop_start_display_frame_num, FRAME_P, frame.pts, dts, frame.duration); last_dts = dts; } if (global_flags.uncompressed_video_to_http || global_flags.x264_video_to_http) { // Add frames left in reorderer. while (!reorderer->empty()) { FrameReorderer::Frame output_frame = reorderer->get_first_frame(); if (global_flags.uncompressed_video_to_http) { add_packet_for_uncompressed_frame(output_frame.pts, output_frame.duration, output_frame.data); } else { assert(global_flags.x264_video_to_http); x264_encoder->add_frame(output_frame.pts, output_frame.duration, output_frame.data); } } } } void QuickSyncEncoderImpl::add_packet_for_uncompressed_frame(int64_t pts, int64_t duration, const uint8_t *data) { AVPacket pkt; memset(&pkt, 0, sizeof(pkt)); pkt.buf = nullptr; pkt.data = const_cast(data); pkt.size = frame_width * frame_height * 2; pkt.stream_index = 0; pkt.flags = AV_PKT_FLAG_KEY; pkt.duration = duration; stream_mux->add_packet(pkt, pts, pts); } namespace { void memcpy_with_pitch(uint8_t *dst, const uint8_t *src, size_t src_width, size_t dst_pitch, size_t height) { if (src_width == dst_pitch) { memcpy(dst, src, src_width * height); } else { for (size_t y = 0; y < height; ++y) { const uint8_t *sptr = src + y * src_width; uint8_t *dptr = dst + y * dst_pitch; memcpy(dptr, sptr, src_width); } } } } // namespace void QuickSyncEncoderImpl::encode_frame(QuickSyncEncoderImpl::PendingFrame frame, int encoding_frame_num, int display_frame_num, int gop_start_display_frame_num, int frame_type, int64_t pts, int64_t dts, int64_t duration) { // Wait for the GPU to be done with the frame. GLenum sync_status; do { sync_status = glClientWaitSync(frame.fence.get(), 0, 1000000000); check_error(); } while (sync_status == GL_TIMEOUT_EXPIRED); assert(sync_status != GL_WAIT_FAILED); // Release back any input frames we needed to render this frame. frame.input_frames.clear(); GLSurface *surf = &gl_surfaces[display_frame_num % SURFACE_NUM]; VAStatus va_status; if (use_zerocopy) { eglDestroyImageKHR(eglGetCurrentDisplay(), surf->y_egl_image); eglDestroyImageKHR(eglGetCurrentDisplay(), surf->cbcr_egl_image); va_status = vaReleaseBufferHandle(va_dpy, surf->surface_image.buf); CHECK_VASTATUS(va_status, "vaReleaseBufferHandle"); } else { unsigned char *surface_p = nullptr; vaMapBuffer(va_dpy, surf->surface_image.buf, (void **)&surface_p); unsigned char *va_y_ptr = (unsigned char *)surface_p + surf->surface_image.offsets[0]; memcpy_with_pitch(va_y_ptr, surf->y_ptr, frame_width, surf->surface_image.pitches[0], frame_height); unsigned char *va_cbcr_ptr = (unsigned char *)surface_p + surf->surface_image.offsets[1]; memcpy_with_pitch(va_cbcr_ptr, surf->cbcr_ptr, (frame_width / 2) * sizeof(uint16_t), surf->surface_image.pitches[1], frame_height / 2); va_status = vaUnmapBuffer(va_dpy, surf->surface_image.buf); CHECK_VASTATUS(va_status, "vaUnmapBuffer"); if (global_flags.uncompressed_video_to_http || global_flags.x264_video_to_http) { // Add uncompressed video. (Note that pts == dts here.) // Delay needs to match audio. FrameReorderer::Frame output_frame = reorderer->reorder_frame(pts + global_delay(), duration, reinterpret_cast(surf->y_ptr)); if (output_frame.data != nullptr) { if (global_flags.uncompressed_video_to_http) { add_packet_for_uncompressed_frame(output_frame.pts, output_frame.duration, output_frame.data); } else { assert(global_flags.x264_video_to_http); x264_encoder->add_frame(output_frame.pts, output_frame.duration, output_frame.data); } } } } va_status = vaDestroyImage(va_dpy, surf->surface_image.image_id); CHECK_VASTATUS(va_status, "vaDestroyImage"); // Schedule the frame for encoding. VASurfaceID va_surface = surf->src_surface; va_status = vaBeginPicture(va_dpy, context_id, va_surface); CHECK_VASTATUS(va_status, "vaBeginPicture"); if (frame_type == FRAME_IDR) { // FIXME: If the mux wants global headers, we should not put the // SPS/PPS before each IDR frame, but rather put it into the // codec extradata (formatted differently?). render_sequence(); render_picture(frame_type, display_frame_num, gop_start_display_frame_num); if (h264_packedheader) { render_packedsequence(); render_packedpicture(); } } else { //render_sequence(); render_picture(frame_type, display_frame_num, gop_start_display_frame_num); } render_slice(encoding_frame_num, display_frame_num, gop_start_display_frame_num, frame_type); va_status = vaEndPicture(va_dpy, context_id); CHECK_VASTATUS(va_status, "vaEndPicture"); // so now the data is done encoding (well, async job kicked off)... // we send that to the storage thread storage_task tmp; tmp.display_order = display_frame_num; tmp.frame_type = frame_type; tmp.pts = pts; tmp.dts = dts; tmp.duration = duration; storage_task_enqueue(move(tmp)); update_ReferenceFrames(frame_type); } // Proxy object. QuickSyncEncoder::QuickSyncEncoder(const std::string &filename, movit::ResourcePool *resource_pool, QSurface *surface, const string &va_display, int width, int height, AVOutputFormat *oformat, X264Encoder *x264_encoder, DiskSpaceEstimator *disk_space_estimator) : impl(new QuickSyncEncoderImpl(filename, resource_pool, surface, va_display, width, height, oformat, x264_encoder, disk_space_estimator)) {} // Must be defined here because unique_ptr<> destructor needs to know the impl. QuickSyncEncoder::~QuickSyncEncoder() {} void QuickSyncEncoder::add_audio(int64_t pts, vector audio) { impl->add_audio(pts, audio); } bool QuickSyncEncoder::begin_frame(GLuint *y_tex, GLuint *cbcr_tex) { return impl->begin_frame(y_tex, cbcr_tex); } RefCountedGLsync QuickSyncEncoder::end_frame(int64_t pts, int64_t duration, const vector &input_frames) { return impl->end_frame(pts, duration, input_frames); } void QuickSyncEncoder::shutdown() { impl->shutdown(); } void QuickSyncEncoder::set_stream_mux(Mux *mux) { impl->set_stream_mux(mux); } int64_t QuickSyncEncoder::global_delay() const { return impl->global_delay(); } nageru-1.4.2/quicksync_encoder.h000066400000000000000000000055141301561706600166720ustar00rootroot00000000000000// Hardware H.264 encoding via VAAPI. Heavily modified based on example // code by Intel. Intel's original copyright and license is reproduced below: // // Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved. // // Permission is hereby granted, free of charge, to any person obtaining a // copy of this software and associated documentation files (the // "Software"), to deal in the Software without restriction, including // without limitation the rights to use, copy, modify, merge, publish, // distribute, sub license, and/or sell copies of the Software, and to // permit persons to whom the Software is furnished to do so, subject to // the following conditions: // // The above copyright notice and this permission notice (including the // next paragraph) shall be included in all copies or substantial portions // of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. // IN NO EVENT SHALL PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR // ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, // TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE // SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. #ifndef _H264ENCODE_H #define _H264ENCODE_H #include #include #include #include #include #include extern "C" { #include } #include "ref_counted_gl_sync.h" class DiskSpaceEstimator; class Mux; class QSurface; class QuickSyncEncoderImpl; class RefCountedFrame; class X264Encoder; namespace movit { class ResourcePool; } // namespace movit // This is just a pimpl, because including anything X11-related in a .h file // tends to trip up Qt. All the real logic is in QuickSyncEncoderImpl, defined in the // .cpp file. class QuickSyncEncoder { public: QuickSyncEncoder(const std::string &filename, movit::ResourcePool *resource_pool, QSurface *surface, const std::string &va_display, int width, int height, AVOutputFormat *oformat, X264Encoder *x264_encoder, DiskSpaceEstimator *disk_space_estimator); ~QuickSyncEncoder(); void set_stream_mux(Mux *mux); // Does not take ownership. Must be called unless x264 is used for the stream. void add_audio(int64_t pts, std::vector audio); bool begin_frame(GLuint *y_tex, GLuint *cbcr_tex); RefCountedGLsync end_frame(int64_t pts, int64_t duration, const std::vector &input_frames); void shutdown(); // Blocking. Does not require an OpenGL context. void release_gl_resources(); // Requires an OpenGL context. Must be run after shutdown. int64_t global_delay() const; // So we never get negative dts. private: std::unique_ptr impl; }; #endif nageru-1.4.2/ref.raw000066400000000000000000002400001301561706600142670ustar00rootroot00000000000000˶!X~7q7vd8Ye9櫸f_I9չg9CisL!:k1:en";o;~6}Cɻ;<%i0M㺊CGTا=6RY_*>/L>䘯nE==̑>2|X*upڽ-Gw=>B|fq`=>:C>ٽSg==` }>UқrTֽBH=]>Dm.)w=;> >ݲA#׽z_=6=ڀ>_-lDUؽX=W>ۯNߜB= > >U뮾ؽd=^=>RVؽqz]= >zlYBp`=$>&Ք>b!,ٽӹi==K>aB ٽb=ؕ>J&}=E >Z>Yfٽn=w=H>1GCٽbg=X_> ;-g=>>ٽ%Ws=d=>Vg}ٽ%l=>/Iei O=ц>Vd>sٽx=O=~j>ѯ ٽp=j>w믾yp 6=^>> iڽ |=9=K>Pp-ٽ u==>毾*|P =>j>Kڽƀ="=\+>O`{(ڽDz=q>8⯾Vlv @=r>>nڽ&=3 = >*;yaڽy~=C>ݯŸg h=fP>m> ڽp==>֡ ڽwŁ=v>ٯ#I| ƿ=,>H>B\ڽ<==Š>0eu ڽq=>ԯn4 =>m>t*۽ ==>#-r ۽qY=Dw>"Яà;I =>>﮾a۽S==u>p @۽d=>˯0Lb=ո>vj> 뮾}۽V=v=L>%eF nw۽J݊=u>Iǯk{==>c>c殾۽֐=T=">< ܭ۽=>¯/Ä=c>yc>ᮾܽ=i1=>  ۽P=Mo>*)=7>ޛ>@ݮ9ܽH=^ =lɏ>_ gܽԃ= >bc\$=>X>خnܽ{==>бXNܽ$=e>)2Ü=ؑ>dҜ><Ԯ ܽ==j>hMm˂ܽ'ޕ=ߝ>8p=\>J>Ϯܽԛ=K=7>mTܽ=Y>ѭUϨB=8t>i>lˮ ݽ=h=A>ޤLQ(ܽB'=ў>e5=}?>8>Ǯ^>ݽ^=_;=Γ>ݽ#E=4H>== >>®qݽn9=& =ꗔ>C9MRݽ^=8>Bjd=ѕ>#>ݽR==H_>zݽs=R3>5sEy=?>>Z ݽ.f=9=$>eा/'=ݽG=o>4ݤ}B=[>D >5< ޽u=s=>-ݽs=>)` =>?> IIݽAR==6>⣾rS۽= >7 %8=Eѕ>ѳ>cDٽa= =]'>CX׽=9>)DW-=x>1>Sս==>bI%pӽơ=>咦,=m> >^0vѽ젥==F>J|iϽ=ӧ>v嚾6* 5=>K>LͽX=4=>+˽]==>vjGLG=3>>>Jɽ==s!>~i,Ƚ= ݒ> nݵ3,c=纋>>X?Oƽ|=w=->U㽭Ľٜ=>2/kܽ=+lj>M:>lu½X==?>Kv6޽b=8>Pm׽=ه>>)U9)=i߷=V>7ٽ5@=m>ԔzDҽ=m>A>dےǻ==s><%XԽ =>cIߋͽH3=? >{>#5g,=;G=㕂>\<Ͻȶ=ۄ>`,ɽ~=/>M>Eze==w>釾ʽ5=Z>û4 ĽҪ=FW>n(>7ފױO=Ԭ=}>:mƽ\GԔ=8>I$?/=}> >sO˦Aۗ=<(=9z>-a½xp=> =my>S>ͅʅrd==v>:,㳽=~>:=u>{>Xs=="s> ~C= z>||=VXr>w> po=T[=Lo>yae1= v>GGtx]=rn>s>R'}p{ =ӟ=96l>#udu=r>{Itī=lk>Vo>mxfp=T=gh>oq|/P=7n>DwW1pq=h>l>s=ޚ=re>cm\>݊=ej>r*lB=d>Jh>soe3k=q=` b>ii0ܗg=\f>Xn6h󣠽 U=8Ya>Ed>c.k1= =^>Ȁe|T=b>EjRd\=^>N`>[f_=j=}[>8a@ʞ<5Hx=KH_>Qe`誙Ա=Z>F]>bۑ؈=zY=eX>v]U=[>a\Nyl=W>Y>#^uCP= =:U>:,Zۄ=I(X>] Y.='7V>Z·ȅ= Ȍ=6R>OVDr =RT>YlUwӏ*=WQ>+R>١V#8@=S=O>lRt{=>@Q>5UQXˆ=BN>^O>NRQć渂=SV=K>1jOQb~*~=M>QZNv=6K>L>N\31=,)=H>[K_A4N5{=J>MJ,Dž=5H>H>HC3݁K@x=IG>4EJ~GPăUn=-E>ҕG9E|== C>2EX!Ku=D>*F^1DEC|=MB>NB>D|8y==$@>AJz/Xr=@>C@R|x=h?>*?>}@=Tx-v={=G=>>m0v7fo=o=>\?b=v5t=<>g<>8 =s$s= pw=5s:>S;yqul=q:>*<:qMp=v9>m9>u9Gop=ns=7>x_8tÑmJi=7>S8mo7F|lLl=I6> 6>M[6Vfkm={o=K4>FM5Vo`if=4>>5ka4Ctgh=D14>3>d36?gj=k=/2>H2!sjECec=1>QI2`1bd&e=Oz1>X/0>/+c5g=g=/>P/me8a`=.>C/+m.׾]ha=k.>S->@,_*_~d=c=V,>f,u`@]]=k,>,Z+ Y]=p',>Ճ*>]);[$a=;`=;*>)Z[y[Y^[=2E)>f(1(G}TrZ=R)>'>h&_W40^=\='>s&0_WUQ%X=&>%%P V=&>%>#S?[=qX=%>A#RfQ KU=#> ##KR=m$>4V"> ORX=aXU=">8!{NNutR=%!>) g eGxO=!>>3 6LbjU=/Q= >Њj1J6yJO=>VVGY>@~HR=VN=>F?FL=>J+?H=y>@>E&O=iJ=w@>RnA;mCJ=}f> 3;ME=>>.uAL=ˈG=>K=?AG=+>!/CQ7!B=8>>4E>>I=4D==>iG9<D=o>3o>=>C>":"G=U@=3>*,.6Q9A=>/X;=ݐ>[>7UD===o>g?s26 ?=ؠ>{ Nr6,j8=I>6U> h4A=:=>u 6.2U<=G > ZR (N5= > >$, >1G>=A[7=ao > V>+@/Ѧ9=U > `:%0>2= >R > $. <= B4== >e#',6= >6"!8/=ؠ >=v>ot+.V9=!41= >0$[$)TX4=t>jsx>,=fx>:>>!(6=d1.=2>Ar!&1=@> CUYO)=xW>*>'%3=9+=B>#M/=>B/j&=>>>NC"pO1=8M(=>1 a,=<=C#=8,>=3rm.=;k%=Ƕ>)=j=?] =!>_l=;,="=d=SvS2p'==:N)=&==R=>}EP|)==i=+~$=g}=sI 5A=ME=+J=_6~&=3=|=>m"={=Ok‘;y C=[=Q=U潐b$=K==)꽕 T='=rfU3=='i=^S!=h== {3\==t!" K=:=n=`z޽moa== =#Af;n==Yݽh==R=|ڽ =^=V=e߽ T=\=VٽJ޽.=A= =ֽv ax===޷۽O 7M='a= ֽڽI ==b= :ӽ G =F ==ؽ4='=:cҽ׽C7 ==D=؜ϽS[=- == Խߎ='=ν~ӽ=|=>:=̽UgI=T= = ѽLy>M=,=B˽=нT3Me===,Ƚd ===ͽN# =(=ǽ̽弨 =ʏ=K=)Žݝ==.=A:ʽx =k=dĽ;ɽDἧm<,==$Q =.=(=tƽ=a=^ ~Žpܼ_<==т =<=ýgAL=!=Ž½%^ؼU<ŋ=L=}F>=r-<G=lBܼ==Pu{b%Լ9:#ļK&/t<=b>RF<>@=Յ= лV<<=t=s﮼F<orƒ$ʊ=-Mvʦy_<|H=DETBɁ<+P=I3=;ڈX}?MPzNf)Zf-<p9=667Dn<@=o#=-Nf<6<*=);bb<Æ6=&34Aj<== =+R{zrf =!"SF@It|<=#SK$FA: }$<=<k1>7;<>E<<C+5U<>*۸+)<c<缰4+<=<)<%&).L<<{<<̾ <=<W:075<I<<α"0ucW<=hj3*T"<=4 H@2,Y_$hjW>@![u-%H<"=4<}*a&Z+<(=+b"@0V_<2=b =$<%u=pؔ r><5<'=l피c;<%=ZzɊw2 AރH@<4=%W0=EL6M=:=0!񥼴Y=$%/𢼘Q=#;|]M=J=sN.E]Fk<*gM=E1^h<M= 1|.2U=e7!q;Ἴn#||<\==U[sy<ְ\=?> >x综—w<^=$\=?{=侼¿*{y<`=[|]=>C翼Cpm=Uk=?J7Wʼׇ<珈<:l=(KՌ̼kʼ\w=Tq-ּ1BӼڋ=\ͺ߼8ۼ<=^J]2ܼ\`<=gU=N_Рݼz~ܟ<| ֲ<`=\=(Ϝ<< =wpҴz<ݜ=gڻ֯<7=o=7 <r & <5=,pk0k<=M=7~|<~,<=τ0o<= *S'p<=-=(*n3<΄=6JU=pؼ,=6<ƃ=ƽ뼣V<=bR˽X^䪼W2=z=%=ټ-='C<=Uǽ\.<=q̽fZ<իX3=No=4= Uڼ‘.=P<=ȽYAЧ<=]ͽX^fǬZY4=e=D=*ۼ /=L_<=ɽ⠼'<=ܴνcG-\5=^=V=7!ݼ0=zo<=Bʽ<=Ͻj̰P`6=MW=Sj=ER޼K{1=l<=!{ԽA~:=J=H=Ľ*o5=<9=нF#_< =ս)Eo;=K=T=4ŽFfYp6=<=xѽ kT<=ֽ½lΕ<=L==ƽXr7==<-= ҽO[.𧼚J<~= ؽý-==0P=Q=bǽ_u8=!Խ8٨B<#=$9ٽĽkB>=SU===eȽ#|y9=p>ȀR=+h<=a-<=YNjIOS=NC=X彰kLS=>[Ѽ]Y==9=V彃x>S=1"<p=@T<Hn<^=%q_Ѽ#Y=h=[9=V{;S=¼<0\=8qr${ѼFY==Z8=WpQ2S=#¼<[=}q&CѼY==G8=X}0S=<~x=>_?,¼<[=q'ὕѼ+Y==Z8=Xk.S==vC\a¼̛= 9=^d#S=U὘ѼLY=!=c9=Di)S=f<=?OüIüfὗѼ6Y=:=n:=oNES=$=\XüZS=P0= olļK=m=6̽SeH<5=j=Zн<5==F޽tӼ4K3==c.̽ S~UL =.=%ϽEU<Ą==@ӽ ¼<=i=ὧؼNA ==)罔ϽU==^c⽳ؼOfe =n=Q置+н3U<==ӽhüĽ<5=,=Xm⽡ؼOi ==U5нU=&=+{8ټ{ QF =F=нXCVg=:мټDReD =w =ZCҽVɼrUT =5$=eҽ`WG<=y;=սʼ|Tӹ =.=~ӽW^<=G=սl_˼A彪Wڼ8T =a2=ӽxWT<͸=K=%ս˼=D=S彟bڼT =16=p,ӽW<=VP=o.սп˼ ӽW}<=T=7սJ˼<0== m彞sڼ U =;=ɛpGӽWl<)=V=b;ս̼!< ,=,=u&yڼQU ===9PӽnWC\<= Y=?ս6̼~A=轔aӽWA:<=\=OHսxM̼ <7=b=9ڼHU- =B=-vjӽW#)<=^=Lսe̼@<==×ڼNXU) =OD=7VsӽZW<=w`=Pս}̼_<=`=cڼgU =E=?#|ӽWH<=7b=Tս̼1<=aͽǽѶSaG^=m=[c=lr伈&=-=B1彟'==>n=Ҵ1g5=l^==,&̡(0L=i>FF뽸+=4>=뽱#z%&=S=2K=v-CEN=0>I콐ߏ+=@>F ='%:>)=%U==,5XgM=z>G!]+=:>Y=-&ً)=U=Ba=V@-fM=`I>@y+=>'6=::e&')=TV==W-(N=f>Q\P2 .B,=k%>K=&B*=2V==g).gN=>!%v- ,=fp>V=fh'(*=W= =gLy.xN='">P ,=>=gF'*=^W==_.ZGO=j>4Bs 7-=>y=7H([X+=W=YM=BJ/a.O=ɳ>H g-=R>d =j*:(,+=%X=#=/eP=>J !-=T>=()b,=*lX=f{=H0LgP=/F>˶ B0.=J>x0=Y)o,=X==Жt0#nP=>ddW# ă.=^6>=5 *.,=l!Y=Ϊ=m07 =)Q=~> p.=>5W=܊|z*+-=A|Y=8C=m3jZ1) Q=h#>X 3+/=?>=#*!-=aY== 1$ Q=m>J1j K/=>o=b)]+-=3Z=6u=>/2+ MR=> /=h>\=We+BF.='Z==2 R=>]3E)0=">ǩ=3?,A.=Z=ި=n3N CS=oM>h2 ~0=~>?=@,n/=H[=EC=ɽp3! tS=>Hq-!n0=Q>=&#-d/=[=/=, ]3Pv S=>ZW*1=>l=$-/=G\=y=\F4 h;T=?/>װG1=>j=G].$0=`\=>=4f T=z>igQl61=";>='z.0=\=\=5{ T=>l\u1=)>>-=1.ڃ0=\==#5< T=ކ>yn8fu|.1=d@>B=K.@0=\=]='5T=>z1=L>=.0=\==ty95$T=X>)zؕ1=L>Ĩ=kf.0=\==:53T=~>yř'1=L>w=.G0=\='=:5BT={>ys֏1=L>;=.0=\=m=(9;5QZT=#y>x<1=L>=Z.|0=\==f;5`T=kv>xq1=M>œ=c,.x0= v\="= O<5oT=s>wa1=9M>= .ct0=Kj\==#<5CT= q>#wiR1=JM>~=-/p0=^\==Q%=5`T=`n>vɱ[C1=NM>o=/k0=%S\=6=u'i>5uT=k> v.Ƶy41=KM>[=2/Tg0=G\=p=)>5T=i>u%1=JM>Qz=N*/b0=Z<\==+?5{T=lf>u1=6M>;u=>5/o^0=1\==-'@5ȨT=c>ste1=5M>+p=0K@/!Z0=%\==/@5DT=)a>sx0=@M> k=jK/U0=\=D=1:A5'CT=^>s0=ZM>f=cW/Q0=g\==04B5qT=[> sNw0=|M>Xa=5b/ r@0=M>\=0m/:F0=[==7B5nT=V>p¹0=M>W= yw/D@0=[=j=,9B5"ɣT=AT>o0=M>5S=# /):0=[==:B51ǢT=Q>nS`0=M>N=r ߋ/30=[==<%C5@T=5O>mp b0=rM>I=/-0='[=.=S=`C5OqT=L>lbgy0=KM>2E=/'0=Q[=N=>C5^>T=8J>kYi0=>M>@=m/ 0=S[==@C5mIT=G>j]Y0=FM>;=C/0=*[==AiD5}7T=KE>iH0=XM>d7=/^0=[==]CD5H;T=B>hh80=A>=6//=߄[=_=P955"3T=g@>h 9(0=J?>=//=v[=3=8 35řT==>Qg0==>=b/@/=qh[= =805bT=;>fl0=:>=[//=MZ[==7.5 T=-9>eC/=8>=-//=XL[= =6I,5T=6>d^ /=e6> =p]/q/=>[==J6)5lT=o4>'dq$&/=04>M=//=0[==5'5T=2>Uc,/=2>=//=q#[== 5%5RpT=/>bV =2B/=/>B=y/-/= [==4J$5ɐT=c->a p8w/=->=Z//=[==4"5$LT=+>)at>/=+>== o0e/= Z=*=t3 5h T=(>\`KD/=e)>= 0X/=Z=N=2Y5T=q&>t_7Ipy/=?'>&=ަ0T/=tZ=y=25Y!T="$>}^e(Oj/=&%>=+f0w/=sZ==H185,T=!>]aTZ/= #>U=r!0}/=Z=Վ=o05q8T=>q\W!YK/= >ݘ=x'0t/=Z= =/5CAT=Y>l[a$^Pu=-.0l/=Z=B=.5iODT=>gZH'd2-/=><= 40-c/=Z==-5ZYT=>WY*Fi/=>=G+;0~Z/=:Z==- 5KfT=>EX,]n/=>w=u~A0Q/=Z=|=9, 5}qT=j>,W/cs#/=>o=xG0,I/=yZ=x=]+A5|T=1>V2\x.=>^g=rMN0@/=FmZ=t=}*c5"^T=> U5V}.=>_=lT08/=q`Z=/q=)5T= >T8|.=>V=g[0//=zSZ=m=(4kRT= >S;.=>ON=-ab0'/=+R?.= >E=J[i0/=8Z=vf='4mT=U>IQEC.= >,==lUp0/=x+Z=b=&4xߏT=>Z[0.=>4=Ow0 /=+Z=M_=C&'4T= >WZxR.=>4,=I,0&/=Z=[=%U4AT=>TY~.=>#=D0.=Z=YQXl.=>u=>>݌0>.=Y=T=$$4T=>=NWNZ.=>Y=8t0.=GY=1Q=d#4k>T=>+KV[H.=>M =20;.=Y=M="C4 {T=>H_UE7.= >X=:-A0.=jY=1J=!4xT=r>ET%.=>s='0C.=/Y=F= 4* @uT=W>BRX.=3>=$"0.=Y=2C=' 4+qT=2> ?Q!.=K>=0.=Y=?=47nT=><<Q-=c>=D0.=-Y=S<=4BVjT= >t9+P;[-=|>s=0,.=Y=8=R4RNhfT=>6qO--=>=w0.=Y=Y5=4Z9bT=<>f4N(-=>=H 0.=1|Y=1=bU4e]T=r>!2jNm]-=>=q~0.=pY=.=4lqYT=>/N-=>=^ 0.=5eY=7+=P4|LUT=:>-Mcd-=>/=n0!.=YY='=4rPT=>w+M{-=@>[=S0v.=sNY=$=d4LT=>O)L{k-=m>=09m.=BCY='+LI[-=>=v0c.=8Y==4*DT=>$KJ-=>1= 0Z.=,Y==q4!@T=L>"sK:-=>O=1Q.=1!Y==4 R[ K *-=?>M=i< 1H.=Y==48T=>Jsp-=>F=1@.= Y==4,5T=H>gI-=>3==-17.=X==vj4F1T=>IK,=>P{=N1..=nX=u =4=Z.T=>=:H,=Z>wt=#1~&.=X==4++T=|>#G,=>m=(1(.=X==C48'T=>E",=>g=-1.=4X==p4$T=h>,Dh,=Y>`=]31 .=X==4!T=> ZC ,=>Z=a81B.=%X==4<#hT=x>cB ?,=>vS=v>1-=vX== $4-8T=>aSA,=b>L=D1-=ѨX==\)4M8!T=>2@z,=>;F=I1v-=8X=="4BT=> ?zk,=,>?=&N1q-=ՓX=V=4LKT=>S=\,=>*9= T1-=rX==4VuT=->p2=1Y1-=5X==G4a T=>&;\=,=s>H,=2^1-=tX==04Lk T=R>09 -,=>%={c1r-=|jX=>=i 94uT=|>8$,=\>=ؔLi1V-=I`X== g4T=y>7R'G,=>h=bn1(-=UX== Ȭ4T=Iv>6*+=N>=׋t1-=KX=:=` 4S=r>6-+=ɾ> =d"z1ģ-=@AX== |4IS=o>)51D+=E>=R1j-=I7X== a4S=l>|Z45i+=ƻ>=N1-=-X=0=s q4,S=oi>Dd39+=I>p={T1Q-=%$X==4GS=zf>o2=+=ʸ>7=u 1w-=gX==4 S=1e>6-zҶ+=>/=kX11*/=W= ^=j0h9O= >|uo-=>j=D-K.= T==o;Eb=:>% ۼ;%=n==a,׽>X=(x=߽Ǽ&<<}=޽<мҼO<=<=N߽$Ӽ<Ք= =RN dȼ =F]=sмru<<=佗AؼIۼd)<=^&彦mܼ=:MݼS<"==R潂<Ѽ}6ټdݼC <ə= dȟݼv8<= =M@EѼ(< void release_refcounted_frame(bmusb::FrameAllocator::Frame *frame) { if (frame->owner) { frame->owner->release_frame(*frame); } delete frame; } nageru-1.4.2/ref_counted_frame.h000066400000000000000000000015221301561706600166240ustar00rootroot00000000000000#ifndef _REF_COUNTED_FRAME_H #define _REF_COUNTED_FRAME_H 1 // A wrapper around FrameAllocator::Frame that is automatically refcounted; // when the refcount goes to zero, the frame is given back to the allocator. // // Note that the important point isn't really the pointer to the Frame itself, // it's the resources it's representing that need to go back to the allocator. #include #include "bmusb/bmusb.h" void release_refcounted_frame(bmusb::FrameAllocator::Frame *frame); typedef std::shared_ptr RefCountedFrameBase; class RefCountedFrame : public RefCountedFrameBase { public: RefCountedFrame() {} RefCountedFrame(const bmusb::FrameAllocator::Frame &frame) : RefCountedFrameBase(new bmusb::FrameAllocator::Frame(frame), release_refcounted_frame) {} }; #endif // !defined(_REF_COUNTED_FRAME_H) nageru-1.4.2/ref_counted_gl_sync.h000066400000000000000000000021421301561706600171670ustar00rootroot00000000000000#ifndef _REF_COUNTED_GL_SYNC_H #define _REF_COUNTED_GL_SYNC_H 1 // A wrapper around GLsync (OpenGL fences) that is automatically refcounted. // Useful since we sometimes want to use the same fence two entirely different // places. (We could set two fences at the same time, but they are not an // unlimited hardware resource, so it would be a bit wasteful.) #include #include #include typedef std::shared_ptr<__GLsync> RefCountedGLsyncBase; class RefCountedGLsync : public RefCountedGLsyncBase { public: RefCountedGLsync() {} RefCountedGLsync(GLenum condition, GLbitfield flags) : RefCountedGLsyncBase(locked_glFenceSync(condition, flags), glDeleteSync) {} private: // These are to work around apitrace bug #446. static GLsync locked_glFenceSync(GLenum condition, GLbitfield flags) { std::lock_guard lock(fence_lock); return glFenceSync(condition, flags); } static void locked_glDeleteSync(GLsync sync) { std::lock_guard lock(fence_lock); glDeleteSync(sync); } static std::mutex fence_lock; }; #endif // !defined(_REF_COUNTED_GL_SYNC_H) nageru-1.4.2/resampling_queue.cpp000066400000000000000000000137161301561706600170650ustar00rootroot00000000000000// Parts of the code is adapted from Adriaensen's project Zita-ajbridge // (as of November 2015), although it has been heavily reworked for this use // case. Original copyright follows: // // Copyright (C) 2012-2015 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 3 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program. If not, see . #include "resampling_queue.h" #include #include #include #include #include #include #include using namespace std; ResamplingQueue::ResamplingQueue(unsigned card_num, unsigned freq_in, unsigned freq_out, unsigned num_channels) : card_num(card_num), freq_in(freq_in), freq_out(freq_out), num_channels(num_channels), ratio(double(freq_out) / double(freq_in)) { vresampler.setup(ratio, num_channels, /*hlen=*/32); // Prime the resampler so there's no more delay. vresampler.inp_count = vresampler.inpsize() / 2 - 1; vresampler.out_count = 1048576; vresampler.process (); } void ResamplingQueue::add_input_samples(double pts, const float *samples, ssize_t num_samples) { if (num_samples == 0) { return; } if (first_input) { // Synthesize a fake length. last_input_len = double(num_samples) / freq_in; first_input = false; } else { last_input_len = pts - last_input_pts; } last_input_pts = pts; k_a0 = k_a1; k_a1 += num_samples; buffer.insert(buffer.end(), samples, samples + num_samples * num_channels); } bool ResamplingQueue::get_output_samples(double pts, float *samples, ssize_t num_samples, ResamplingQueue::RateAdjustmentPolicy rate_adjustment_policy) { assert(num_samples > 0); if (first_input) { // No data yet, just return zeros. memset(samples, 0, num_samples * num_channels * sizeof(float)); return true; } double rcorr = -1.0; if (rate_adjustment_policy == ADJUST_RATE) { double last_output_len; if (first_output) { // Synthesize a fake length. last_output_len = double(num_samples) / freq_out; } else { last_output_len = pts - last_output_pts; } last_output_pts = pts; // Using the time point since just before the last call to add_input_samples() as a base, // estimate actual delay based on activity since then, measured in number of input samples: double actual_delay = 0.0; assert(last_input_len != 0); actual_delay += (k_a1 - k_a0) * last_output_len / last_input_len; // Inserted samples since k_a0, rescaled for the different time periods. actual_delay += k_a0 - total_consumed_samples; // Samples inserted before k_a0 but not consumed yet. actual_delay += vresampler.inpdist(); // Delay in the resampler itself. double err = actual_delay - expected_delay; if (first_output && err < 0.0) { // Before the very first block, insert artificial delay based on our initial estimate, // so that we don't need a long period to stabilize at the beginning. int delay_samples_to_add = lrintf(-err); for (ssize_t i = 0; i < delay_samples_to_add * num_channels; ++i) { buffer.push_front(0.0f); } total_consumed_samples -= delay_samples_to_add; // Equivalent to increasing k_a0 and k_a1. err += delay_samples_to_add; } first_output = false; // Compute loop filter coefficients for the two filters. We need to compute them // every time, since they depend on the number of samples the user asked for. // // The loop bandwidth is at 0.02 Hz; we trust the initial estimate quite well, // and our jitter is pretty large since none of the threads involved run at // real-time priority. double loop_bandwidth_hz = 0.02; // Set filters. The first filter much wider than the first one (20x as wide). double w = (2.0 * M_PI) * loop_bandwidth_hz * num_samples / freq_out; double w0 = 1.0 - exp(-20.0 * w); double w1 = w * 1.5 / num_samples / ratio; double w2 = w / 1.5; // Filter through the loop filter to find the correction ratio. z1 += w0 * (w1 * err - z1); z2 += w0 * (z1 - z2); z3 += w2 * z2; rcorr = 1.0 - z2 - z3; if (rcorr > 1.05) rcorr = 1.05; if (rcorr < 0.95) rcorr = 0.95; assert(!isnan(rcorr)); vresampler.set_rratio(rcorr); } else { assert(rate_adjustment_policy == DO_NOT_ADJUST_RATE); }; // Finally actually resample, consuming exactly output samples. vresampler.out_data = samples; vresampler.out_count = num_samples; while (vresampler.out_count > 0) { if (buffer.empty()) { // This should never happen unless delay is set way too low, // or we're dropping a lot of data. fprintf(stderr, "Card %u: PANIC: Out of input samples to resample, still need %d output samples! (correction factor is %f)\n", card_num, int(vresampler.out_count), rcorr); memset(vresampler.out_data, 0, vresampler.out_count * num_channels * sizeof(float)); return false; } float inbuf[1024]; size_t num_input_samples = sizeof(inbuf) / (sizeof(float) * num_channels); if (num_input_samples * num_channels > buffer.size()) { num_input_samples = buffer.size() / num_channels; } copy(buffer.begin(), buffer.begin() + num_input_samples * num_channels, inbuf); vresampler.inp_count = num_input_samples; vresampler.inp_data = inbuf; int err = vresampler.process(); assert(err == 0); size_t consumed_samples = num_input_samples - vresampler.inp_count; total_consumed_samples += consumed_samples; buffer.erase(buffer.begin(), buffer.begin() + consumed_samples * num_channels); } return true; } nageru-1.4.2/resampling_queue.h000066400000000000000000000105001301561706600165160ustar00rootroot00000000000000#ifndef _RESAMPLING_QUEUE_H #define _RESAMPLING_QUEUE_H 1 // Takes in samples from an input source, possibly with jitter, and outputs a fixed number // of samples every iteration. Used to a) change sample rates if needed, and b) deal with // input sources that don't have audio locked to video. For every input video // frame, you call add_input_samples() with the pts (measured in seconds) of the video frame, // taken to be the start point of the frame's audio. When you want to _output_ a finished // frame with audio, you get_output_samples() with the number of samples you want, and will // get exactly that number of samples back. If the input and output clocks are not in sync, // the audio will be stretched for you. (If they are _very_ out of sync, this will come through // as a pitch shift.) Of course, the process introduces some delay; you specify a target delay // (typically measured in milliseconds, although more is fine) and the algorithm works to // provide exactly that. // // A/V sync is a much harder problem than one would intuitively assume. This implementation // is based on a 2012 paper by Fons Adriaensen, “Controlling adaptive resampling” // (http://kokkinizita.linuxaudio.org/papers/adapt-resamp.pdf). The paper gives an algorithm // that converges to jitter of <100 ns; the basic idea is to measure the _rate_ the input // queue fills and is drained (as opposed to the length of the queue itself), and smoothly // adjust the resampling rate so that it reaches steady state at the desired delay. // // Parts of the code is adapted from Adriaensen's project Zita-ajbridge (based on the same // algorithm), although it has been heavily reworked for this use case. Original copyright follows: // // Copyright (C) 2012-2015 Fons Adriaensen // // This program is free software; you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation; either version 3 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with this program. If not, see . #include #include #include #include #include "defs.h" class ResamplingQueue { public: // card_num is for debugging outputs only. ResamplingQueue(unsigned card_num, unsigned freq_in, unsigned freq_out, unsigned num_channels = 2); // If policy is DO_NOT_ADJUST_RATE, the resampling rate will not be changed. // This is primarily useful if you have an extraordinary situation, such as // dropped frames. enum RateAdjustmentPolicy { DO_NOT_ADJUST_RATE, ADJUST_RATE }; // Note: pts is always in seconds. void add_input_samples(double pts, const float *samples, ssize_t num_samples); // Returns false if underrun. bool get_output_samples(double pts, float *samples, ssize_t num_samples, RateAdjustmentPolicy rate_adjustment_policy); private: void init_loop_filter(double bandwidth_hz); VResampler vresampler; unsigned card_num; unsigned freq_in, freq_out, num_channels; bool first_input = true, first_output = true; double last_input_pts; // Start of last input block, in seconds. double last_output_pts; ssize_t k_a0 = 0; // Total amount of samples inserted _before_ the last call to add_input_samples(). ssize_t k_a1 = 0; // Total amount of samples inserted _after_ the last call to add_input_samples(). ssize_t total_consumed_samples = 0; // Duration of last input block, in seconds. double last_input_len; // Filter state for the loop filter. double z1 = 0.0, z2 = 0.0, z3 = 0.0; // Ratio between the two frequencies. double ratio; // How much delay we are expected to have, in input samples. // If actual delay drifts too much away from this, we will start // changing the resampling ratio to compensate. double expected_delay = OUTPUT_FREQUENCY * 0.1; // 100 ms. // Input samples not yet fed into the resampler. // TODO: Use a circular buffer instead, for efficiency. std::deque buffer; }; #endif // !defined(_RESAMPLING_QUEUE_H) nageru-1.4.2/simple.lua000066400000000000000000000142001301561706600147750ustar00rootroot00000000000000-- The theme is what decides what's actually shown on screen, what kind of -- transitions are available (if any), and what kind of inputs there are, -- if any. In general, it drives the entire display logic by creating Movit -- chains, setting their parameters and then deciding which to show when. -- -- Themes are written in Lua, which reflects a simplified form of the Movit API -- where all the low-level details (such as texture formats) are handled by the -- C++ side and you generally just build chains. -- -- This is a much simpler theme than the default theme; it only allows you to -- switch between inputs and set white balance, no transitions or the likes. -- Thus, it should be simpler to understand. local input_neutral_color = {{0.5, 0.5, 0.5}, {0.5, 0.5, 0.5}} local live_signal_num = 0 local preview_signal_num = 1 -- A chain to show a single input, with white balance. In a real example, -- we'd probably want to support deinterlacing and high-quality scaling -- (if the input isn't exactly what we want). However, we don't want these -- things always on, so we'd need to generate more chains for the various -- cases. In such a simple example, just having two is fine. function make_simple_chain(hq) local chain = EffectChain.new(16, 9) local input = chain:add_live_input(false, false) -- No deinterlacing, no bounce override. input:connect_signal(0) -- First input card. Can be changed whenever you want. local wb_effect = chain:add_effect(WhiteBalanceEffect.new()) chain:finalize(hq) return { chain = chain, input = input, wb_effect = wb_effect, } end -- We only make two chains; one for the live view and one for the previews. -- (Since they have different outputs, you cannot mix and match them.) local simple_hq_chain = make_simple_chain(true) local simple_lq_chain = make_simple_chain(false) -- API ENTRY POINT -- Returns the number of outputs in addition to the live (0) and preview (1). -- Called only once, at the start of the program. function num_channels() return 2 end -- API ENTRY POINT -- Returns the name for each additional channel (starting from 2). -- Called at the start of the program, and then each frame for live -- channels in case they change resolution. function channel_name(channel) if channel == 2 then return "First input" elseif channel == 3 then return "Second input" end end -- API ENTRY POINT -- Returns, given a channel number, which signal it corresponds to (starting from 0). -- Should return -1 if the channel does not correspond to a simple signal. -- (The information is used for whether right-click on the channel should bring up -- an input selector or not.) -- Called once for each channel, at the start of the program. -- Will never be called for live (0) or preview (1). function channel_signal(channel) if channel == 2 then return 0 elseif channel == 3 then return 1 else return -1 end end -- API ENTRY POINT -- Called every frame. Returns the color (if any) to paint around the given -- channel. Returns a CSS color (typically to mark live and preview signals); -- "transparent" is allowed. -- Will never be called for live (0) or preview (1). function channel_color(channel) return "transparent" end -- API ENTRY POINT -- Returns if a given channel supports setting white balance (starting from 2). -- Called only once for each channel, at the start of the program. function supports_set_wb(channel) return channel == 2 or channel == 3 end -- API ENTRY POINT -- Gets called with a new gray point when the white balance is changing. -- The color is in linear light (not sRGB gamma). function set_wb(channel, red, green, blue) if channel == 2 then input_neutral_color[1] = { red, green, blue } elseif channel == 3 then input_neutral_color[2] = { red, green, blue } end end -- API ENTRY POINT -- Called every frame. function get_transitions(t) if live_signal_num == preview_signal_num then -- No transitions possible. return {} else return {"Cut"} end end -- API ENTRY POINT -- Called when the user clicks a transition button. For our case, -- we only do cuts, so we ignore the parameters; just switch live and preview. function transition_clicked(num, t) local temp = live_signal_num live_signal_num = preview_signal_num preview_signal_num = temp end -- API ENTRY POINT function channel_clicked(num) preview_signal_num = num end -- API ENTRY POINT -- Called every frame. Get the chain for displaying at input , -- where 0 is live, 1 is preview, 2 is the first channel to display -- in the bottom bar, and so on up to num_channels()+1. t is the -- current time in seconds. width and height are the dimensions of -- the output, although you can ignore them if you don't need them -- (they're useful if you want to e.g. know what to resample by). -- -- is basically an exposed InputState, which you can use to -- query for information about the signals at the point of the current -- frame. In particular, you can call get_width() and get_height() -- for any signal number, and use that to e.g. assist in chain selection. -- -- You should return two objects; the chain itself, and then a -- function (taking no parameters) that is run just before rendering. -- The function needs to call connect_signal on any inputs, so that -- it gets updated video data for the given frame. (You are allowed -- to switch which input your input is getting from between frames, -- but not calling connect_signal results in undefined behavior.) -- If you want to change any parameters in the chain, this is also -- the right place. -- -- NOTE: The chain returned must be finalized with the Y'CbCr flag -- if and only if num==0. function get_chain(num, t, width, height, signals) local chain, signal_num if num == 0 then -- Live (right pane). chain = simple_hq_chain signal_num = live_signal_num elseif num == 1 then -- Preview (left pane). chain = simple_lq_chain signal_num = preview_signal_num else -- One of the two previews (bottom panes). chain = simple_lq_chain signal_num = num - 2 end prepare = function() chain.input:connect_signal(signal_num) local color = input_neutral_color[signal_num + 1] chain.wb_effect:set_vec3("neutral_color", color[1], color[2], color[3]) end return chain.chain, prepare end nageru-1.4.2/state.proto000066400000000000000000000023171301561706600152140ustar00rootroot00000000000000// Used to serialize state between runs. Currently only audio input mappings, // but in theory we could do the entire mix, video inputs, etc. syntax = "proto2"; // Similar to DeviceSpec, but only devices that are used are stored, // and contains additional information that will help us try to map // to the right device even if the devices have moved around. message DeviceSpecProto { // Members from DeviceSpec itself. enum InputSourceType { SILENCE = 0; CAPTURE_CARD = 1; ALSA_INPUT = 2; }; optional InputSourceType type = 1; optional int32 index = 2; // Additional information. optional string display_name = 3; optional string alsa_name = 4; // Only for ALSA devices. optional string alsa_info = 5; // Only for ALSA devices. optional int32 num_channels = 6; // Only for ALSA devices. optional string address = 7; // Only for ALSA devices. } // Corresponds to InputMapping::Bus. message BusProto { optional string name = 1; optional int32 device_index = 2; // Index into the "devices" array. optional int32 source_channel_left = 3; optional int32 source_channel_right = 4; } // Corresponds to InputMapping. message InputMappingProto { repeated DeviceSpecProto device = 1; repeated BusProto bus = 2; } nageru-1.4.2/stereocompressor.cpp000066400000000000000000000112471301561706600171330ustar00rootroot00000000000000#include "stereocompressor.h" #include #include #include using namespace std; namespace { // Implement a less accurate but faster pow(x, y). We use the standard identity // // x^y = exp(y * ln(x)) // // with the ranges: // // x in 1..(1/threshold) // y in -1..0 // // Assume threshold goes from 0 to -40 dB. That means 1/threshold = 100, // so input to ln(x) can be 1..100. Worst case for end accuracy is y=-1. // To get a good minimax approximation (not the least wrt. continuity // at x=1), I had to make a piecewise linear function for the two ranges: // // with(numapprox): // f1 := minimax(ln, 1..6, [3, 3], x -> 1/x, 'maxerror'); // f2 := minimax(ln, 6..100, [3, 3], x -> 1/x, 'maxerror'); // f := x -> piecewise(x < 6, f1(x), f2(x)); // // (Continuity: Error is down to the 1e-6 range for x=1, difference between // f1 and f2 range at the crossover point is in the 1e-5 range. The cutoff // point at x=6 is chosen to get maxerror pretty close between f1 and f2.) // // Maximum output of ln(x) here is of course ln(100) ~= 4.605. So we can find // an approximation for exp over the range -4.605..0, where we care mostly // about the relative error: // // g := minimax(exp, -ln(100)..0, [3, 3], x -> 1/exp(x), 'maxerror'); // // We can find the worst-case error in dB from this through a simple plot: // // dbdiff := (x, y) -> abs(20 * log10(x / y)); // plot(dbdiff(g(-f(x)), 1/x), x=1..100); // // which readily shows the error never to be above ~0.001 dB or so // (actually 0.00119 dB, for the case of x=100). y=-1 remains the worst case, // it would seem. // // If we cared even more about speed, we could probably fuse y into // the coefficients for ln_nom and postgain into the coefficients for ln_den. // But if so, we should probably rather just SIMD the entire thing instead. inline float fastpow(float x, float y) { float ln_nom, ln_den; if (x < 6.0f) { ln_nom = -0.059237648f + (-0.0165117771f + (0.06818859075f + 0.007560968243f * x) * x) * x; ln_den = 0.0202509098f + (0.08419174188f + (0.03647189417f + 0.001642577975f * x) * x) * x; } else { ln_nom = -0.005430534f + (0.00633589178f + (0.0006319155549f + 0.4789541675e-5f * x) * x) * x; ln_den = 0.0064785099f + (0.003219629109f + (0.0001531823694f + 0.6884656640e-6f * x) * x) * x; } float v = y * ln_nom / ln_den; float exp_nom = 0.2195097621f + (0.08546059868f + (0.01208501759f + 0.0006173448113f * v) * v) * v; float exp_den = 0.2194980791f + (-0.1343051968f + (0.03556072737f - 0.006174398513f * v) * v) * v; return exp_nom / exp_den; } inline float compressor_knee(float x, float threshold, float inv_threshold, float inv_ratio_minus_one, float postgain) { assert(inv_ratio_minus_one <= 0.0f); if (x > threshold) { return postgain * fastpow(x * inv_threshold, inv_ratio_minus_one); } else { return postgain; } } } // namespace void StereoCompressor::process(float *buf, size_t num_samples, float threshold, float ratio, float attack_time, float release_time, float makeup_gain) { float attack_increment = float(pow(2.0f, 1.0f / (attack_time * sample_rate + 1))); if (attack_time == 0.0f) attack_increment = 100000; // For instant attack reaction. const float release_increment = float(pow(2.0f, -1.0f / (release_time * sample_rate + 1))); const float peak_increment = float(pow(2.0f, -1.0f / (0.003f * sample_rate + 1))); float inv_ratio_minus_one = 1.0f / ratio - 1.0f; if (ratio > 63) inv_ratio_minus_one = -1.0f; // Infinite ratio. float inv_threshold = 1.0f / threshold; float *left_ptr = buf; float *right_ptr = buf + 1; if (inv_ratio_minus_one >= 0.0) { for (size_t i = 0; i < num_samples; ++i) { *left_ptr *= makeup_gain; left_ptr += 2; *right_ptr *= makeup_gain; right_ptr += 2; } return; } float peak_level = this->peak_level; float compr_level = this->compr_level; for (size_t i = 0; i < num_samples; ++i) { if (fabs(*left_ptr) > peak_level) peak_level = float(fabs(*left_ptr)); if (fabs(*right_ptr) > peak_level) peak_level = float(fabs(*right_ptr)); if (peak_level > compr_level) { compr_level = min(compr_level * attack_increment, peak_level); } else { compr_level = max(compr_level * release_increment, 0.0001f); } float scalefactor_with_gain = compressor_knee(compr_level, threshold, inv_threshold, inv_ratio_minus_one, makeup_gain); *left_ptr *= scalefactor_with_gain; left_ptr += 2; *right_ptr *= scalefactor_with_gain; right_ptr += 2; peak_level = max(peak_level * peak_increment, 0.0001f); } // Store attenuation level for debug/visualization. scalefactor = compressor_knee(compr_level, threshold, inv_threshold, inv_ratio_minus_one, 1.0f); this->peak_level = peak_level; this->compr_level = compr_level; } nageru-1.4.2/stereocompressor.h000066400000000000000000000024761301561706600166040ustar00rootroot00000000000000#ifndef _STEREOCOMPRESSOR_H #define _STEREOCOMPRESSOR_H 1 #include // A simple compressor based on absolute values, with independent // attack/release times. There is no sidechain or lookahead, but the // peak value is shared between both channels. // // The compressor was originally written by, and is copyrighted by, Rune Holm. // It has been adapted and relicensed under GPLv3 (or, at your option, // any later version) for Nageru, so that its license matches the rest of the code. class StereoCompressor { public: StereoCompressor(float sample_rate) : sample_rate(sample_rate) { reset(); } void reset() { peak_level = compr_level = 0.1f; scalefactor = 0.0f; } // Process interleaved stereo data in-place. // Attack and release times are in seconds. void process(float *buf, size_t num_samples, float threshold, float ratio, float attack_time, float release_time, float makeup_gain); // Last level estimated (after attack/decay applied). float get_level() { return compr_level; } // Last attenuation factor applied, e.g. if 5x compression is currently applied, // this number will be 0.2. float get_attenuation() { return scalefactor; } private: float sample_rate; float peak_level; float compr_level; float scalefactor; }; #endif /* !defined(_STEREOCOMPRESSOR_H) */ nageru-1.4.2/theme.cpp000066400000000000000000000735141301561706600146240ustar00rootroot00000000000000#include "theme.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "defs.h" #include "deinterlace_effect.h" #include "flags.h" #include "image_input.h" #include "input.h" #include "input_state.h" #include "pbo_frame_allocator.h" class Mixer; namespace movit { class ResourcePool; } // namespace movit using namespace std; using namespace movit; extern Mixer *global_mixer; namespace { // Contains basically the same data as InputState, but does not hold on to // a reference to the frames. This is important so that we can release them // without having to wait for Lua's GC. struct InputStateInfo { InputStateInfo(const InputState& input_state); unsigned last_width[MAX_VIDEO_CARDS], last_height[MAX_VIDEO_CARDS]; bool last_interlaced[MAX_VIDEO_CARDS], last_has_signal[MAX_VIDEO_CARDS], last_is_connected[MAX_VIDEO_CARDS]; unsigned last_frame_rate_nom[MAX_VIDEO_CARDS], last_frame_rate_den[MAX_VIDEO_CARDS]; }; InputStateInfo::InputStateInfo(const InputState &input_state) { for (unsigned signal_num = 0; signal_num < MAX_VIDEO_CARDS; ++signal_num) { BufferedFrame frame = input_state.buffered_frames[signal_num][0]; if (frame.frame == nullptr) { last_width[signal_num] = last_height[signal_num] = 0; last_interlaced[signal_num] = false; last_has_signal[signal_num] = false; last_is_connected[signal_num] = false; continue; } const PBOFrameAllocator::Userdata *userdata = (const PBOFrameAllocator::Userdata *)frame.frame->userdata; last_width[signal_num] = userdata->last_width[frame.field_number]; last_height[signal_num] = userdata->last_height[frame.field_number]; last_interlaced[signal_num] = userdata->last_interlaced; last_has_signal[signal_num] = userdata->last_has_signal; last_is_connected[signal_num] = userdata->last_is_connected; last_frame_rate_nom[signal_num] = userdata->last_frame_rate_nom; last_frame_rate_den[signal_num] = userdata->last_frame_rate_den; } } class LuaRefWithDeleter { public: LuaRefWithDeleter(mutex *m, lua_State *L, int ref) : m(m), L(L), ref(ref) {} ~LuaRefWithDeleter() { unique_lock lock(*m); luaL_unref(L, LUA_REGISTRYINDEX, ref); } int get() const { return ref; } private: LuaRefWithDeleter(const LuaRefWithDeleter &) = delete; mutex *m; lua_State *L; int ref; }; template int wrap_lua_object(lua_State* L, const char *class_name, Args&&... args) { // Construct the C++ object and put it on the stack. void *mem = lua_newuserdata(L, sizeof(T)); new(mem) T(forward(args)...); // Look up the metatable named , and set it on the new object. luaL_getmetatable(L, class_name); lua_setmetatable(L, -2); return 1; } // Like wrap_lua_object, but the object is not owned by Lua; ie. it's not freed // by Lua GC. This is typically the case for Effects, which are owned by EffectChain // and expected to be destructed by it. The object will be of type T** instead of T* // when exposed to Lua. // // Note that we currently leak if you allocate an Effect in this way and never call // add_effect. We should see if there's a way to e.g. set __gc on it at construction time // and then release that once add_effect() takes ownership. template int wrap_lua_object_nonowned(lua_State* L, const char *class_name, Args&&... args) { // Construct the pointer ot the C++ object and put it on the stack. T **obj = (T **)lua_newuserdata(L, sizeof(T *)); *obj = new T(forward(args)...); // Look up the metatable named , and set it on the new object. luaL_getmetatable(L, class_name); lua_setmetatable(L, -2); return 1; } Theme *get_theme_updata(lua_State* L) { luaL_checktype(L, lua_upvalueindex(1), LUA_TLIGHTUSERDATA); return (Theme *)lua_touserdata(L, lua_upvalueindex(1)); } Effect *get_effect(lua_State *L, int idx) { if (luaL_testudata(L, idx, "WhiteBalanceEffect") || luaL_testudata(L, idx, "ResampleEffect") || luaL_testudata(L, idx, "PaddingEffect") || luaL_testudata(L, idx, "IntegralPaddingEffect") || luaL_testudata(L, idx, "OverlayEffect") || luaL_testudata(L, idx, "ResizeEffect") || luaL_testudata(L, idx, "MultiplyEffect") || luaL_testudata(L, idx, "MixEffect") || luaL_testudata(L, idx, "ImageInput")) { return *(Effect **)lua_touserdata(L, idx); } luaL_error(L, "Error: Index #%d was not an Effect type\n", idx); return nullptr; } InputStateInfo *get_input_state_info(lua_State *L, int idx) { if (luaL_testudata(L, idx, "InputStateInfo")) { return (InputStateInfo *)lua_touserdata(L, idx); } luaL_error(L, "Error: Index #%d was not InputStateInfo\n", idx); return nullptr; } bool checkbool(lua_State* L, int idx) { luaL_checktype(L, idx, LUA_TBOOLEAN); return lua_toboolean(L, idx); } string checkstdstring(lua_State *L, int index) { size_t len; const char* cstr = lua_tolstring(L, index, &len); return string(cstr, len); } int EffectChain_new(lua_State* L) { assert(lua_gettop(L) == 2); Theme *theme = get_theme_updata(L); int aspect_w = luaL_checknumber(L, 1); int aspect_h = luaL_checknumber(L, 2); return wrap_lua_object(L, "EffectChain", aspect_w, aspect_h, theme->get_resource_pool()); } int EffectChain_gc(lua_State* L) { assert(lua_gettop(L) == 1); EffectChain *chain = (EffectChain *)luaL_checkudata(L, 1, "EffectChain"); chain->~EffectChain(); return 0; } int EffectChain_add_live_input(lua_State* L) { assert(lua_gettop(L) == 3); Theme *theme = get_theme_updata(L); EffectChain *chain = (EffectChain *)luaL_checkudata(L, 1, "EffectChain"); bool override_bounce = checkbool(L, 2); bool deinterlace = checkbool(L, 3); return wrap_lua_object(L, "LiveInputWrapper", theme, chain, override_bounce, deinterlace); } int EffectChain_add_effect(lua_State* L) { assert(lua_gettop(L) >= 2); EffectChain *chain = (EffectChain *)luaL_checkudata(L, 1, "EffectChain"); // TODO: Better error reporting. Effect *effect = get_effect(L, 2); if (lua_gettop(L) == 2) { if (effect->num_inputs() == 0) { chain->add_input((Input *)effect); } else { chain->add_effect(effect); } } else { vector inputs; for (int idx = 3; idx <= lua_gettop(L); ++idx) { if (luaL_testudata(L, idx, "LiveInputWrapper")) { LiveInputWrapper *input = (LiveInputWrapper *)lua_touserdata(L, idx); inputs.push_back(input->get_effect()); } else { inputs.push_back(get_effect(L, idx)); } } chain->add_effect(effect, inputs); } lua_settop(L, 2); // Return the effect itself. // Make sure Lua doesn't garbage-collect it away. lua_pushvalue(L, -1); luaL_ref(L, LUA_REGISTRYINDEX); // TODO: leak? return 1; } int EffectChain_finalize(lua_State* L) { assert(lua_gettop(L) == 2); EffectChain *chain = (EffectChain *)luaL_checkudata(L, 1, "EffectChain"); bool is_main_chain = checkbool(L, 2); // Add outputs as needed. // NOTE: If you change any details about the output format, you will need to // also update what's given to the muxer (HTTPD::Mux constructor) and // what's put in the H.264 stream (sps_rbsp()). ImageFormat inout_format; inout_format.color_space = COLORSPACE_REC_709; // Output gamma is tricky. We should output Rec. 709 for TV, except that // we expect to run with web players and others that don't really care and // just output with no conversion. So that means we'll need to output sRGB, // even though H.264 has no setting for that (we use “unspecified”). inout_format.gamma_curve = GAMMA_sRGB; if (is_main_chain) { YCbCrFormat output_ycbcr_format; // We actually output 4:2:0 in the end, but chroma subsampling // happens in a pass not run by Movit (see Mixer::subsample_chroma()). output_ycbcr_format.chroma_subsampling_x = 1; output_ycbcr_format.chroma_subsampling_y = 1; // Rec. 709 would be the sane thing to do, but it seems many players // (e.g. MPlayer and VLC) just default to BT.601 coefficients no matter // what (see discussions in e.g. https://trac.ffmpeg.org/ticket/4978). // We _do_ set the right flags, though, so that a player that works // properly doesn't have to guess. output_ycbcr_format.luma_coefficients = YCBCR_REC_601; output_ycbcr_format.full_range = false; output_ycbcr_format.num_levels = 256; chain->add_ycbcr_output(inout_format, OUTPUT_ALPHA_FORMAT_POSTMULTIPLIED, output_ycbcr_format, YCBCR_OUTPUT_SPLIT_Y_AND_CBCR); chain->set_dither_bits(8); chain->set_output_origin(OUTPUT_ORIGIN_TOP_LEFT); } chain->add_output(inout_format, OUTPUT_ALPHA_FORMAT_POSTMULTIPLIED); chain->finalize(); return 0; } int LiveInputWrapper_connect_signal(lua_State* L) { assert(lua_gettop(L) == 2); LiveInputWrapper *input = (LiveInputWrapper *)luaL_checkudata(L, 1, "LiveInputWrapper"); int signal_num = luaL_checknumber(L, 2); input->connect_signal(signal_num); return 0; } int ImageInput_new(lua_State* L) { assert(lua_gettop(L) == 1); string filename = checkstdstring(L, 1); return wrap_lua_object_nonowned(L, "ImageInput", filename); } int WhiteBalanceEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "WhiteBalanceEffect"); } int ResampleEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "ResampleEffect"); } int PaddingEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "PaddingEffect"); } int IntegralPaddingEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "IntegralPaddingEffect"); } int OverlayEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "OverlayEffect"); } int ResizeEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "ResizeEffect"); } int MultiplyEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "MultiplyEffect"); } int MixEffect_new(lua_State* L) { assert(lua_gettop(L) == 0); return wrap_lua_object_nonowned(L, "MixEffect"); } int InputStateInfo_get_width(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushnumber(L, input_state_info->last_width[signal_num]); return 1; } int InputStateInfo_get_height(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushnumber(L, input_state_info->last_height[signal_num]); return 1; } int InputStateInfo_get_interlaced(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushboolean(L, input_state_info->last_interlaced[signal_num]); return 1; } int InputStateInfo_get_has_signal(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushboolean(L, input_state_info->last_has_signal[signal_num]); return 1; } int InputStateInfo_get_is_connected(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushboolean(L, input_state_info->last_is_connected[signal_num]); return 1; } int InputStateInfo_get_frame_rate_nom(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushnumber(L, input_state_info->last_frame_rate_nom[signal_num]); return 1; } int InputStateInfo_get_frame_rate_den(lua_State* L) { assert(lua_gettop(L) == 2); InputStateInfo *input_state_info = get_input_state_info(L, 1); Theme *theme = get_theme_updata(L); int signal_num = theme->map_signal(luaL_checknumber(L, 2)); lua_pushnumber(L, input_state_info->last_frame_rate_den[signal_num]); return 1; } int Effect_set_float(lua_State *L) { assert(lua_gettop(L) == 3); Effect *effect = (Effect *)get_effect(L, 1); string key = checkstdstring(L, 2); float value = luaL_checknumber(L, 3); if (!effect->set_float(key, value)) { luaL_error(L, "Effect refused set_float(\"%s\", %d) (invalid key?)", key.c_str(), int(value)); } return 0; } int Effect_set_int(lua_State *L) { assert(lua_gettop(L) == 3); Effect *effect = (Effect *)get_effect(L, 1); string key = checkstdstring(L, 2); float value = luaL_checknumber(L, 3); if (!effect->set_int(key, value)) { luaL_error(L, "Effect refused set_int(\"%s\", %d) (invalid key?)", key.c_str(), int(value)); } return 0; } int Effect_set_vec3(lua_State *L) { assert(lua_gettop(L) == 5); Effect *effect = (Effect *)get_effect(L, 1); string key = checkstdstring(L, 2); float v[3]; v[0] = luaL_checknumber(L, 3); v[1] = luaL_checknumber(L, 4); v[2] = luaL_checknumber(L, 5); if (!effect->set_vec3(key, v)) { luaL_error(L, "Effect refused set_vec3(\"%s\", %f, %f, %f) (invalid key?)", key.c_str(), v[0], v[1], v[2]); } return 0; } int Effect_set_vec4(lua_State *L) { assert(lua_gettop(L) == 6); Effect *effect = (Effect *)get_effect(L, 1); string key = checkstdstring(L, 2); float v[4]; v[0] = luaL_checknumber(L, 3); v[1] = luaL_checknumber(L, 4); v[2] = luaL_checknumber(L, 5); v[3] = luaL_checknumber(L, 6); if (!effect->set_vec4(key, v)) { luaL_error(L, "Effect refused set_vec4(\"%s\", %f, %f, %f, %f) (invalid key?)", key.c_str(), v[0], v[1], v[2], v[3]); } return 0; } const luaL_Reg EffectChain_funcs[] = { { "new", EffectChain_new }, { "__gc", EffectChain_gc }, { "add_live_input", EffectChain_add_live_input }, { "add_effect", EffectChain_add_effect }, { "finalize", EffectChain_finalize }, { NULL, NULL } }; const luaL_Reg LiveInputWrapper_funcs[] = { { "connect_signal", LiveInputWrapper_connect_signal }, { NULL, NULL } }; const luaL_Reg ImageInput_funcs[] = { { "new", ImageInput_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg WhiteBalanceEffect_funcs[] = { { "new", WhiteBalanceEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg ResampleEffect_funcs[] = { { "new", ResampleEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg PaddingEffect_funcs[] = { { "new", PaddingEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg IntegralPaddingEffect_funcs[] = { { "new", IntegralPaddingEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg OverlayEffect_funcs[] = { { "new", OverlayEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg ResizeEffect_funcs[] = { { "new", ResizeEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg MultiplyEffect_funcs[] = { { "new", MultiplyEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg MixEffect_funcs[] = { { "new", MixEffect_new }, { "set_float", Effect_set_float }, { "set_int", Effect_set_int }, { "set_vec3", Effect_set_vec3 }, { "set_vec4", Effect_set_vec4 }, { NULL, NULL } }; const luaL_Reg InputStateInfo_funcs[] = { { "get_width", InputStateInfo_get_width }, { "get_height", InputStateInfo_get_height }, { "get_interlaced", InputStateInfo_get_interlaced }, { "get_has_signal", InputStateInfo_get_has_signal }, { "get_is_connected", InputStateInfo_get_is_connected }, { "get_frame_rate_nom", InputStateInfo_get_frame_rate_nom }, { "get_frame_rate_den", InputStateInfo_get_frame_rate_den }, { NULL, NULL } }; } // namespace LiveInputWrapper::LiveInputWrapper(Theme *theme, EffectChain *chain, bool override_bounce, bool deinterlace) : theme(theme), deinterlace(deinterlace) { ImageFormat inout_format; inout_format.color_space = COLORSPACE_sRGB; // Gamma curve depends on the input signal, and we don't really get any // indications. A camera would be expected to do Rec. 709, but // I haven't checked if any do in practice. However, computers _do_ output // in sRGB gamma (ie., they don't convert from sRGB to Rec. 709), and // I wouldn't really be surprised if most non-professional cameras do, too. // So we pick sRGB as the least evil here. inout_format.gamma_curve = GAMMA_sRGB; // The Blackmagic driver docs claim that the device outputs Y'CbCr // according to Rec. 601, but practical testing indicates it definitely // is Rec. 709 (at least up to errors attributable to rounding errors). // Perhaps 601 was only to indicate the subsampling positions, not the // colorspace itself? Tested with a Lenovo X1 gen 3 as input. YCbCrFormat input_ycbcr_format; input_ycbcr_format.chroma_subsampling_x = 2; input_ycbcr_format.chroma_subsampling_y = 1; input_ycbcr_format.cb_x_position = 0.0; input_ycbcr_format.cr_x_position = 0.0; input_ycbcr_format.cb_y_position = 0.5; input_ycbcr_format.cr_y_position = 0.5; input_ycbcr_format.luma_coefficients = YCBCR_REC_709; input_ycbcr_format.full_range = false; unsigned num_inputs; if (deinterlace) { deinterlace_effect = new movit::DeinterlaceEffect(); // As per the comments in deinterlace_effect.h, we turn this off. // The most likely interlaced input for us is either a camera // (where it's fine to turn it off) or a laptop (where it _should_ // be turned off). CHECK(deinterlace_effect->set_int("enable_spatial_interlacing_check", 0)); num_inputs = deinterlace_effect->num_inputs(); assert(num_inputs == FRAME_HISTORY_LENGTH); } else { num_inputs = 1; } for (unsigned i = 0; i < num_inputs; ++i) { if (override_bounce) { inputs.push_back(new NonBouncingYCbCrInput(inout_format, input_ycbcr_format, WIDTH, HEIGHT, YCBCR_INPUT_SPLIT_Y_AND_CBCR)); } else { inputs.push_back(new YCbCrInput(inout_format, input_ycbcr_format, WIDTH, HEIGHT, YCBCR_INPUT_SPLIT_Y_AND_CBCR)); } chain->add_input(inputs.back()); } if (deinterlace) { vector reverse_inputs(inputs.rbegin(), inputs.rend()); chain->add_effect(deinterlace_effect, reverse_inputs); } } void LiveInputWrapper::connect_signal(int signal_num) { if (global_mixer == nullptr) { // No data yet. return; } signal_num = theme->map_signal(signal_num); BufferedFrame first_frame = theme->input_state->buffered_frames[signal_num][0]; if (first_frame.frame == nullptr) { // No data yet. return; } unsigned width, height; { const PBOFrameAllocator::Userdata *userdata = (const PBOFrameAllocator::Userdata *)first_frame.frame->userdata; width = userdata->last_width[first_frame.field_number]; height = userdata->last_height[first_frame.field_number]; } BufferedFrame last_good_frame = first_frame; for (unsigned i = 0; i < inputs.size(); ++i) { BufferedFrame frame = theme->input_state->buffered_frames[signal_num][i]; if (frame.frame == nullptr) { // Not enough data; reuse last frame (well, field). // This is suboptimal, but we have nothing better. frame = last_good_frame; } const PBOFrameAllocator::Userdata *userdata = (const PBOFrameAllocator::Userdata *)frame.frame->userdata; if (userdata->last_width[frame.field_number] != width || userdata->last_height[frame.field_number] != height) { // Resolution changed; reuse last frame/field. frame = last_good_frame; userdata = (const PBOFrameAllocator::Userdata *)frame.frame->userdata; } inputs[i]->set_texture_num(0, userdata->tex_y[frame.field_number]); inputs[i]->set_texture_num(1, userdata->tex_cbcr[frame.field_number]); inputs[i]->set_width(userdata->last_width[frame.field_number]); inputs[i]->set_height(userdata->last_height[frame.field_number]); last_good_frame = frame; } if (deinterlace) { BufferedFrame frame = theme->input_state->buffered_frames[signal_num][0]; CHECK(deinterlace_effect->set_int("current_field_position", frame.field_number)); } } namespace { int call_num_channels(lua_State *L) { lua_getglobal(L, "num_channels"); if (lua_pcall(L, 0, 1, 0) != 0) { fprintf(stderr, "error running function `num_channels': %s\n", lua_tostring(L, -1)); exit(1); } int num_channels = luaL_checknumber(L, 1); lua_pop(L, 1); assert(lua_gettop(L) == 0); return num_channels; } } // namespace Theme::Theme(const string &filename, const vector &search_dirs, ResourcePool *resource_pool, unsigned num_cards) : resource_pool(resource_pool), num_cards(num_cards), signal_to_card_mapping(global_flags.default_stream_mapping) { L = luaL_newstate(); luaL_openlibs(L); register_class("EffectChain", EffectChain_funcs); register_class("LiveInputWrapper", LiveInputWrapper_funcs); register_class("ImageInput", ImageInput_funcs); register_class("WhiteBalanceEffect", WhiteBalanceEffect_funcs); register_class("ResampleEffect", ResampleEffect_funcs); register_class("PaddingEffect", PaddingEffect_funcs); register_class("IntegralPaddingEffect", IntegralPaddingEffect_funcs); register_class("OverlayEffect", OverlayEffect_funcs); register_class("ResizeEffect", ResizeEffect_funcs); register_class("MultiplyEffect", MultiplyEffect_funcs); register_class("MixEffect", MixEffect_funcs); register_class("InputStateInfo", InputStateInfo_funcs); // Run script. Search through all directories until we find a file that will load // (as in, does not return LUA_ERRFILE); then run it. We store load errors // from all the attempts, and show them once we know we can't find any of them. lua_settop(L, 0); vector errors; bool success = false; for (size_t i = 0; i < search_dirs.size(); ++i) { string path = search_dirs[i] + "/" + filename; int err = luaL_loadfile(L, path.c_str()); if (err == 0) { // Success; actually call the code. if (lua_pcall(L, 0, LUA_MULTRET, 0)) { fprintf(stderr, "Error when running %s: %s\n", path.c_str(), lua_tostring(L, -1)); exit(1); } success = true; break; } errors.push_back(lua_tostring(L, -1)); lua_pop(L, 1); if (err != LUA_ERRFILE) { // The file actually loaded, but failed to parse somehow. Abort; don't try the next one. break; } } if (!success) { for (const string &error : errors) { fprintf(stderr, "%s\n", error.c_str()); } exit(1); } assert(lua_gettop(L) == 0); // Ask it for the number of channels. num_channels = call_num_channels(L); } Theme::~Theme() { lua_close(L); } void Theme::register_class(const char *class_name, const luaL_Reg *funcs) { assert(lua_gettop(L) == 0); luaL_newmetatable(L, class_name); // mt = {} lua_pushlightuserdata(L, this); luaL_setfuncs(L, funcs, 1); // for (name,f in funcs) { mt[name] = f, with upvalue {theme} } lua_pushvalue(L, -1); lua_setfield(L, -2, "__index"); // mt.__index = mt lua_setglobal(L, class_name); // ClassName = mt assert(lua_gettop(L) == 0); } Theme::Chain Theme::get_chain(unsigned num, float t, unsigned width, unsigned height, InputState input_state) { Chain chain; unique_lock lock(m); assert(lua_gettop(L) == 0); lua_getglobal(L, "get_chain"); /* function to be called */ lua_pushnumber(L, num); lua_pushnumber(L, t); lua_pushnumber(L, width); lua_pushnumber(L, height); wrap_lua_object(L, "InputStateInfo", input_state); if (lua_pcall(L, 5, 2, 0) != 0) { fprintf(stderr, "error running function `get_chain': %s\n", lua_tostring(L, -1)); exit(1); } chain.chain = (EffectChain *)luaL_testudata(L, -2, "EffectChain"); if (chain.chain == nullptr) { fprintf(stderr, "get_chain() for chain number %d did not return an EffectChain\n", num); exit(1); } if (!lua_isfunction(L, -1)) { fprintf(stderr, "Argument #-1 should be a function\n"); exit(1); } lua_pushvalue(L, -1); shared_ptr funcref(new LuaRefWithDeleter(&m, L, luaL_ref(L, LUA_REGISTRYINDEX))); lua_pop(L, 2); assert(lua_gettop(L) == 0); chain.setup_chain = [this, funcref, input_state]{ unique_lock lock(m); this->input_state = &input_state; // Set up state, including connecting signals. lua_rawgeti(L, LUA_REGISTRYINDEX, funcref->get()); if (lua_pcall(L, 0, 0, 0) != 0) { fprintf(stderr, "error running chain setup callback: %s\n", lua_tostring(L, -1)); exit(1); } assert(lua_gettop(L) == 0); }; // TODO: Can we do better, e.g. by running setup_chain() and seeing what it references? // Actually, setup_chain does maybe hold all the references we need now anyway? for (unsigned card_index = 0; card_index < num_cards; ++card_index) { for (unsigned frame_num = 0; frame_num < FRAME_HISTORY_LENGTH; ++frame_num) { chain.input_frames.push_back(input_state.buffered_frames[card_index][frame_num].frame); } } return chain; } string Theme::get_channel_name(unsigned channel) { unique_lock lock(m); lua_getglobal(L, "channel_name"); lua_pushnumber(L, channel); if (lua_pcall(L, 1, 1, 0) != 0) { fprintf(stderr, "error running function `channel_name': %s\n", lua_tostring(L, -1)); exit(1); } const char *ret = lua_tostring(L, -1); if (ret == nullptr) { fprintf(stderr, "function `channel_name' returned nil for channel %d\n", channel); exit(1); } string retstr = ret; lua_pop(L, 1); assert(lua_gettop(L) == 0); return retstr; } int Theme::get_channel_signal(unsigned channel) { unique_lock lock(m); lua_getglobal(L, "channel_signal"); lua_pushnumber(L, channel); if (lua_pcall(L, 1, 1, 0) != 0) { fprintf(stderr, "error running function `channel_signal': %s\n", lua_tostring(L, -1)); exit(1); } int ret = luaL_checknumber(L, 1); lua_pop(L, 1); assert(lua_gettop(L) == 0); return ret; } std::string Theme::get_channel_color(unsigned channel) { unique_lock lock(m); lua_getglobal(L, "channel_color"); lua_pushnumber(L, channel); if (lua_pcall(L, 1, 1, 0) != 0) { fprintf(stderr, "error running function `channel_color': %s\n", lua_tostring(L, -1)); exit(1); } const char *ret = lua_tostring(L, -1); if (ret == nullptr) { fprintf(stderr, "function `channel_color' returned nil for channel %d\n", channel); exit(1); } string retstr = ret; lua_pop(L, 1); assert(lua_gettop(L) == 0); return retstr; } bool Theme::get_supports_set_wb(unsigned channel) { unique_lock lock(m); lua_getglobal(L, "supports_set_wb"); lua_pushnumber(L, channel); if (lua_pcall(L, 1, 1, 0) != 0) { fprintf(stderr, "error running function `supports_set_wb': %s\n", lua_tostring(L, -1)); exit(1); } bool ret = checkbool(L, -1); lua_pop(L, 1); assert(lua_gettop(L) == 0); return ret; } void Theme::set_wb(unsigned channel, double r, double g, double b) { unique_lock lock(m); lua_getglobal(L, "set_wb"); lua_pushnumber(L, channel); lua_pushnumber(L, r); lua_pushnumber(L, g); lua_pushnumber(L, b); if (lua_pcall(L, 4, 0, 0) != 0) { fprintf(stderr, "error running function `set_wb': %s\n", lua_tostring(L, -1)); exit(1); } assert(lua_gettop(L) == 0); } vector Theme::get_transition_names(float t) { unique_lock lock(m); lua_getglobal(L, "get_transitions"); lua_pushnumber(L, t); if (lua_pcall(L, 1, 1, 0) != 0) { fprintf(stderr, "error running function `get_transitions': %s\n", lua_tostring(L, -1)); exit(1); } vector ret; lua_pushnil(L); while (lua_next(L, -2) != 0) { ret.push_back(lua_tostring(L, -1)); lua_pop(L, 1); } lua_pop(L, 1); assert(lua_gettop(L) == 0); return ret; } int Theme::map_signal(int signal_num) { unique_lock lock(map_m); if (signal_to_card_mapping.count(signal_num)) { return signal_to_card_mapping[signal_num]; } if (signal_num >= int(num_cards)) { fprintf(stderr, "WARNING: Theme asked for input %d, but we only have %u card(s).\n", signal_num, num_cards); fprintf(stderr, "Mapping to card %d instead.\n", signal_num % num_cards); } signal_to_card_mapping[signal_num] = signal_num % num_cards; return signal_num % num_cards; } void Theme::set_signal_mapping(int signal_num, int card_num) { unique_lock lock(map_m); assert(card_num < int(num_cards)); signal_to_card_mapping[signal_num] = card_num; } void Theme::transition_clicked(int transition_num, float t) { unique_lock lock(m); lua_getglobal(L, "transition_clicked"); lua_pushnumber(L, transition_num); lua_pushnumber(L, t); if (lua_pcall(L, 2, 0, 0) != 0) { fprintf(stderr, "error running function `transition_clicked': %s\n", lua_tostring(L, -1)); exit(1); } assert(lua_gettop(L) == 0); } void Theme::channel_clicked(int preview_num) { unique_lock lock(m); lua_getglobal(L, "channel_clicked"); lua_pushnumber(L, preview_num); if (lua_pcall(L, 1, 0, 0) != 0) { fprintf(stderr, "error running function `channel_clicked': %s\n", lua_tostring(L, -1)); exit(1); } assert(lua_gettop(L) == 0); } nageru-1.4.2/theme.h000066400000000000000000000057031301561706600142640ustar00rootroot00000000000000#ifndef _THEME_H #define _THEME_H 1 #include #include #include #include #include #include #include #include #include "ref_counted_frame.h" struct InputState; namespace movit { class ResourcePool; class Effect; class EffectChain; struct ImageFormat; struct YCbCrFormat; } // namespace movit class NonBouncingYCbCrInput : public movit::YCbCrInput { public: NonBouncingYCbCrInput(const movit::ImageFormat &image_format, const movit::YCbCrFormat &ycbcr_format, unsigned width, unsigned height, movit::YCbCrInputSplitting ycbcr_input_splitting = movit::YCBCR_INPUT_PLANAR) : movit::YCbCrInput(image_format, ycbcr_format, width, height, ycbcr_input_splitting) {} bool override_disable_bounce() const override { return true; } }; class Theme { public: Theme(const std::string &filename, const std::vector &search_dirs, movit::ResourcePool *resource_pool, unsigned num_cards); ~Theme(); struct Chain { movit::EffectChain *chain; std::function setup_chain; // May have duplicates. std::vector input_frames; }; Chain get_chain(unsigned num, float t, unsigned width, unsigned height, InputState input_state); int get_num_channels() const { return num_channels; } int map_signal(int signal_num); void set_signal_mapping(int signal_num, int card_num); std::string get_channel_name(unsigned channel); int get_channel_signal(unsigned channel); bool get_supports_set_wb(unsigned channel); void set_wb(unsigned channel, double r, double g, double b); std::string get_channel_color(unsigned channel); std::vector get_transition_names(float t); void connect_signal(movit::YCbCrInput *input, int signal_num); void transition_clicked(int transition_num, float t); void channel_clicked(int preview_num); movit::ResourcePool *get_resource_pool() const { return resource_pool; } private: void register_class(const char *class_name, const luaL_Reg *funcs); std::mutex m; lua_State *L; // Protected by . const InputState *input_state; // Protected by . Only set temporarily, during chain setup. movit::ResourcePool *resource_pool; int num_channels; unsigned num_cards; std::mutex map_m; std::map signal_to_card_mapping; // Protected by . friend class LiveInputWrapper; }; class LiveInputWrapper { public: LiveInputWrapper(Theme *theme, movit::EffectChain *chain, bool override_bounce, bool deinterlace); void connect_signal(int signal_num); movit::Effect *get_effect() const { if (deinterlace) { return deinterlace_effect; } else { return inputs[0]; } } private: Theme *theme; // Not owned by us. std::vector inputs; // Multiple ones if deinterlacing. Owned by the chain. movit::Effect *deinterlace_effect = nullptr; // Owned by the chain. bool deinterlace; }; #endif // !defined(_THEME_H) nageru-1.4.2/theme.lua000066400000000000000000000704451301561706600146230ustar00rootroot00000000000000-- The theme is what decides what's actually shown on screen, what kind of -- transitions are available (if any), and what kind of inputs there are, -- if any. In general, it drives the entire display logic by creating Movit -- chains, setting their parameters and then deciding which to show when. -- -- Themes are written in Lua, which reflects a simplified form of the Movit API -- where all the low-level details (such as texture formats) are handled by the -- C++ side and you generally just build chains. local transition_start = -2.0 local transition_end = -1.0 local transition_type = 0 local transition_src_signal = 0 local transition_dst_signal = 0 local neutral_colors = { {0.5, 0.5, 0.5}, -- Input 0. {0.5, 0.5, 0.5} -- Input 1. } local live_signal_num = 0 local preview_signal_num = 1 -- Valid values for live_signal_num and preview_signal_num. local INPUT0_SIGNAL_NUM = 0 local INPUT1_SIGNAL_NUM = 1 local SBS_SIGNAL_NUM = 2 local STATIC_SIGNAL_NUM = 3 -- Valid values for transition_type. (Cuts are done directly, so they need no entry.) local NO_TRANSITION = 0 local ZOOM_TRANSITION = 1 -- Also for slides. local FADE_TRANSITION = 2 -- Last width/height/frame rate for each channel, if we have it. -- Note that unlike the values we get from Nageru, the resolution is per -- frame and not per field, since we deinterlace. local last_resolution = {} -- Utility function to help creating many similar chains that can differ -- in a free set of chosen parameters. function make_cartesian_product(parms, callback) return make_cartesian_product_internal(parms, callback, 1, {}) end function make_cartesian_product_internal(parms, callback, index, args) if index > #parms then return callback(unpack(args)) end local ret = {} for _, value in ipairs(parms[index]) do args[index] = value ret[value] = make_cartesian_product_internal(parms, callback, index + 1, args) end return ret end function make_sbs_input(chain, signal, deint, hq) local input = chain:add_live_input(not deint, deint) -- Override bounce only if not deinterlacing. input:connect_signal(signal) local resample_effect = nil local resize_effect = nil if (hq) then resample_effect = chain:add_effect(ResampleEffect.new()) else resize_effect = chain:add_effect(ResizeEffect.new()) end local wb_effect = chain:add_effect(WhiteBalanceEffect.new()) local padding_effect = chain:add_effect(IntegralPaddingEffect.new()) return { input = input, wb_effect = wb_effect, resample_effect = resample_effect, resize_effect = resize_effect, padding_effect = padding_effect } end -- The main live chain. function make_sbs_chain(input0_type, input1_type, hq) local chain = EffectChain.new(16, 9) local input0 = make_sbs_input(chain, INPUT0_SIGNAL_NUM, input0_type == "livedeint", hq) local input1 = make_sbs_input(chain, INPUT1_SIGNAL_NUM, input1_type == "livedeint", hq) input0.padding_effect:set_vec4("border_color", 0.0, 0.0, 0.0, 1.0) input1.padding_effect:set_vec4("border_color", 0.0, 0.0, 0.0, 0.0) chain:add_effect(OverlayEffect.new(), input0.padding_effect, input1.padding_effect) chain:finalize(hq) return { chain = chain, input0 = input0, input1 = input1 } end -- Make all possible combinations of side-by-side chains. local sbs_chains = make_cartesian_product({ {"live", "livedeint"}, -- input0_type {"live", "livedeint"}, -- input1_type {true, false} -- hq }, function(input0_type, input1_type, hq) return make_sbs_chain(input0_type, input1_type, hq) end) function make_fade_input(chain, signal, live, deint, scale) local input, wb_effect, resample_effect, last if live then input = chain:add_live_input(false, deint) input:connect_signal(signal) last = input else input = chain:add_effect(ImageInput.new("bg.jpeg")) last = input end -- If we cared about this for the non-main inputs, we would have -- checked hq here and invoked ResizeEffect instead. if scale then resample_effect = chain:add_effect(ResampleEffect.new()) last = resample_effect end -- Make sure to put the white balance after the scaling (usually more efficient). if live then wb_effect = chain:add_effect(WhiteBalanceEffect.new()) last = wb_effect end return { input = input, wb_effect = wb_effect, resample_effect = resample_effect, last = last } end -- A chain to fade between two inputs, of which either can be a picture -- or a live input. In practice only used live, but we still support the -- hq parameter. function make_fade_chain(input0_live, input0_deint, input0_scale, input1_live, input1_deint, input1_scale, hq) local chain = EffectChain.new(16, 9) local input0 = make_fade_input(chain, INPUT0_SIGNAL_NUM, input0_live, input0_deint, input0_scale) local input1 = make_fade_input(chain, INPUT1_SIGNAL_NUM, input1_live, input1_deint, input1_scale) local mix_effect = chain:add_effect(MixEffect.new(), input0.last, input1.last) chain:finalize(hq) return { chain = chain, input0 = input0, input1 = input1, mix_effect = mix_effect } end -- Chains to fade between two inputs, in various configurations. local fade_chains = make_cartesian_product({ {"static", "live", "livedeint"}, -- input0_type {true, false}, -- input0_scale {"static", "live", "livedeint"}, -- input1_type {true, false}, -- input1_scale {true} -- hq }, function(input0_type, input0_scale, input1_type, input1_scale, hq) local input0_live = (input0_type ~= "static") local input1_live = (input1_type ~= "static") local input0_deint = (input0_type == "livedeint") local input1_deint = (input1_type == "livedeint") return make_fade_chain(input0_live, input0_deint, input0_scale, input1_live, input1_deint, input1_scale, hq) end) -- A chain to show a single input on screen. function make_simple_chain(input_deint, input_scale, hq) local chain = EffectChain.new(16, 9) local input = chain:add_live_input(false, input_deint) input:connect_signal(0) -- First input card. Can be changed whenever you want. local resample_effect, resize_effect if scale then if hq then resample_effect = chain:add_effect(ResampleEffect.new()) else resize_effect = chain:add_effect(ResizeEffect.new()) end end local wb_effect = chain:add_effect(WhiteBalanceEffect.new()) chain:finalize(hq) return { chain = chain, input = input, wb_effect = wb_effect, resample_effect = resample_effect, resize_effect = resize_effect } end -- Make all possible combinations of single-input chains. local simple_chains = make_cartesian_product({ {"live", "livedeint"}, -- input_type {true, false}, -- input_scale {true, false} -- hq }, function(input_type, input_scale, hq) local input_deint = (input_type == "livedeint") return make_simple_chain(input_deint, input_scale, hq) end) -- A chain to show a single static picture on screen (HQ version). local static_chain_hq = EffectChain.new(16, 9) local static_chain_hq_input = static_chain_hq:add_effect(ImageInput.new("bg.jpeg")) static_chain_hq:finalize(true) -- A chain to show a single static picture on screen (LQ version). local static_chain_lq = EffectChain.new(16, 9) local static_chain_lq_input = static_chain_lq:add_effect(ImageInput.new("bg.jpeg")) static_chain_lq:finalize(false) -- Used for indexing into the tables of chains. function get_input_type(signals, signal_num) if signal_num == STATIC_SIGNAL_NUM then return "static" elseif signals:get_interlaced(signal_num) then return "livedeint" else return "live" end end function needs_scale(signals, signal_num, width, height) if signal_num == STATIC_SIGNAL_NUM then -- We assume this is already correctly scaled at load time. return false end assert(is_plain_signal(signal_num)) return (signals:get_width(signal_num) ~= width or signals:get_height(signal_num) ~= height) end function set_scale_parameters_if_needed(chain_or_input, width, height) if chain_or_input.resample_effect then chain_or_input.resample_effect:set_int("width", width) chain_or_input.resample_effect:set_int("height", height) elseif chain_or_input.resize_effect then chain_or_input.resize_effect:set_int("width", width) chain_or_input.resize_effect:set_int("height", height) end end -- API ENTRY POINT -- Returns the number of outputs in addition to the live (0) and preview (1). -- Called only once, at the start of the program. function num_channels() return 4 end function is_plain_signal(num) return num == INPUT0_SIGNAL_NUM or num == INPUT1_SIGNAL_NUM end -- Helper function to write e.g. “720p60”. The difference between this -- and get_channel_resolution_raw() is that this one also can say that -- there's no signal. function get_channel_resolution(signal_num) res = last_resolution[signal_num] if (not res) or not res.is_connected then return "disconnected" end if res.height <= 0 then return "no signal" end if not res.has_signal then if res.height == 525 then -- Special mode for the USB3 cards. return "no signal" end return get_channel_resolution_raw(res) .. ", no signal" else return get_channel_resolution_raw(res) end end -- Helper function to write e.g. “60” or “59.94”. function get_frame_rate(res) local nom = res.frame_rate_nom local den = res.frame_rate_den if nom % den == 0 then return nom / den else return string.format("%.2f", nom / den) end end -- Helper function to write e.g. “720p60”. function get_channel_resolution_raw(res) if res.interlaced then return res.height .. "i" .. get_frame_rate(res) else return res.height .. "p" .. get_frame_rate(res) end end -- API ENTRY POINT -- Returns the name for each additional channel (starting from 2). -- Called at the start of the program, and then each frame for live -- channels in case they change resolution. function channel_name(channel) local signal_num = channel - 2 if is_plain_signal(signal_num) then return "Input " .. (signal_num + 1) .. " (" .. get_channel_resolution(signal_num) .. ")" elseif signal_num == SBS_SIGNAL_NUM then return "Side-by-side" elseif signal_num == STATIC_SIGNAL_NUM then return "Static picture" end end -- API ENTRY POINT -- Returns, given a channel number, which signal it corresponds to (starting from 0). -- Should return -1 if the channel does not correspond to a simple signal. -- (The information is used for whether right-click on the channel should bring up -- an input selector or not.) -- Called once for each channel, at the start of the program. -- Will never be called for live (0) or preview (1). function channel_signal(channel) if channel == 2 then return 0 elseif channel == 3 then return 1 else return -1 end end -- API ENTRY POINT -- Called every frame. Returns the color (if any) to paint around the given -- channel. Returns a CSS color (typically to mark live and preview signals); -- "transparent" is allowed. -- Will never be called for live (0) or preview (1). function channel_color(channel) if transition_type ~= NO_TRANSITION then if channel_involved_in(channel, transition_src_signal) or channel_involved_in(channel, transition_dst_signal) then return "#f00" end else if channel_involved_in(channel, live_signal_num) then return "#f00" end end if channel_involved_in(channel, preview_signal_num) then return "#0f0" end return "transparent" end function channel_involved_in(channel, signal_num) if is_plain_signal(signal_num) then return channel == (signal_num + 2) end if signal_num == SBS_SIGNAL_NUM then return (channel == 2 or channel == 3) end if signal_num == STATIC_SIGNAL_NUM then return (channel == 5) end return false end -- API ENTRY POINT -- Returns if a given channel supports setting white balance (starting from 2). -- Called only once for each channel, at the start of the program. function supports_set_wb(channel) return is_plain_signal(channel - 2) end -- API ENTRY POINT -- Gets called with a new gray point when the white balance is changing. -- The color is in linear light (not sRGB gamma). function set_wb(channel, red, green, blue) if is_plain_signal(channel - 2) then neutral_colors[channel - 2 + 1] = { red, green, blue } end end function finish_transitions(t) if transition_type ~= NO_TRANSITION and t >= transition_end then live_signal_num = transition_dst_signal transition_type = NO_TRANSITION end end function in_transition(t) return t >= transition_start and t <= transition_end end -- API ENTRY POINT -- Called every frame. function get_transitions(t) if in_transition(t) then -- Transition already in progress, the only thing we can do is really -- cut to the preview. (TODO: Make an “abort” and/or “finish”, too?) return {"Cut"} end finish_transitions(t) if live_signal_num == preview_signal_num then -- No transitions possible. return {} end if (is_plain_signal(live_signal_num) or live_signal_num == STATIC_SIGNAL_NUM) and (is_plain_signal(preview_signal_num) or preview_signal_num == STATIC_SIGNAL_NUM) then return {"Cut", "", "Fade"} end -- Various zooms. if live_signal_num == SBS_SIGNAL_NUM and is_plain_signal(preview_signal_num) then return {"Cut", "Zoom in"} elseif is_plain_signal(live_signal_num) and preview_signal_num == SBS_SIGNAL_NUM then return {"Cut", "Zoom out"} end return {"Cut"} end function swap_preview_live() local temp = live_signal_num live_signal_num = preview_signal_num preview_signal_num = temp end function start_transition(type_, t, duration) transition_start = t transition_end = t + duration transition_type = type_ transition_src_signal = live_signal_num transition_dst_signal = preview_signal_num swap_preview_live() end -- API ENTRY POINT -- Called when the user clicks a transition button. function transition_clicked(num, t) if num == 0 then -- Cut. if in_transition(t) then -- Ongoing transition; finish it immediately before the cut. finish_transitions(transition_end) end swap_preview_live() elseif num == 1 then -- Zoom. finish_transitions(t) if live_signal_num == preview_signal_num then -- Nothing to do. return end if is_plain_signal(live_signal_num) and is_plain_signal(preview_signal_num) then -- We can't zoom between these. Just make a cut. io.write("Cutting from " .. live_signal_num .. " to " .. live_signal_num .. "\n") swap_preview_live() return end if (live_signal_num == SBS_SIGNAL_NUM and is_plain_signal(preview_signal_num)) or (preview_signal_num == SBS_SIGNAL_NUM and is_plain_signal(live_signal_num)) then start_transition(ZOOM_TRANSITION, t, 1.0) end elseif num == 2 then finish_transitions(t) -- Fade. if (live_signal_num ~= preview_signal_num) and (is_plain_signal(live_signal_num) or live_signal_num == STATIC_SIGNAL_NUM) and (is_plain_signal(preview_signal_num) or preview_signal_num == STATIC_SIGNAL_NUM) then start_transition(FADE_TRANSITION, t, 1.0) else -- Fades involving SBS are ignored (we have no chain for it). end end end -- API ENTRY POINT function channel_clicked(num) preview_signal_num = num end function get_fade_chain(signals, t, width, height, input_resolution) local input0_type = get_input_type(signals, transition_src_signal) local input0_scale = needs_scale(signals, transition_src_signal, width, height) local input1_type = get_input_type(signals, transition_dst_signal) local input1_scale = needs_scale(signals, transition_dst_signal, width, height) local chain = fade_chains[input0_type][input0_scale][input1_type][input1_scale][true] prepare = function() if input0_type == "live" or input0_type == "livedeint" then chain.input0.input:connect_signal(transition_src_signal) set_neutral_color_from_signal(chain.input0.wb_effect, transition_src_signal) end set_scale_parameters_if_needed(chain.input0, width, height) if input1_type == "live" or input1_type == "livedeint" then chain.input1.input:connect_signal(transition_dst_signal) set_neutral_color_from_signal(chain.input1.wb_effect, transition_dst_signal) end set_scale_parameters_if_needed(chain.input1, width, height) local tt = calc_fade_progress(t, transition_start, transition_end) chain.mix_effect:set_float("strength_first", 1.0 - tt) chain.mix_effect:set_float("strength_second", tt) end return chain.chain, prepare end -- SBS code (live_signal_num == SBS_SIGNAL_NUM, or in a transition to/from it). function get_sbs_chain(signals, t, width, height, input_resolution) local input0_type = get_input_type(signals, INPUT0_SIGNAL_NUM) local input1_type = get_input_type(signals, INPUT1_SIGNAL_NUM) return sbs_chains[input0_type][input1_type][true] end -- API ENTRY POINT -- Called every frame. Get the chain for displaying at input , -- where 0 is live, 1 is preview, 2 is the first channel to display -- in the bottom bar, and so on up to num_channels()+1. t is the -- current time in seconds. width and height are the dimensions of -- the output, although you can ignore them if you don't need them -- (they're useful if you want to e.g. know what to resample by). -- -- is basically an exposed InputState, which you can use to -- query for information about the signals at the point of the current -- frame. In particular, you can call get_width() and get_height() -- for any signal number, and use that to e.g. assist in chain selection. -- -- You should return two objects; the chain itself, and then a -- function (taking no parameters) that is run just before rendering. -- The function needs to call connect_signal on any inputs, so that -- it gets updated video data for the given frame. (You are allowed -- to switch which input your input is getting from between frames, -- but not calling connect_signal results in undefined behavior.) -- If you want to change any parameters in the chain, this is also -- the right place. -- -- NOTE: The chain returned must be finalized with the Y'CbCr flag -- if and only if num==0. function get_chain(num, t, width, height, signals) local input_resolution = {} for signal_num=0,1 do local res = { width = signals:get_width(signal_num), height = signals:get_height(signal_num), interlaced = signals:get_interlaced(signal_num), is_connected = signals:get_is_connected(signal_num), has_signal = signals:get_has_signal(signal_num), frame_rate_nom = signals:get_frame_rate_nom(signal_num), frame_rate_den = signals:get_frame_rate_den(signal_num) } if res.interlaced then -- Convert height from frame height to field height. -- (Needed for e.g. place_rectangle.) res.height = res.height * 2 -- Show field rate instead of frame rate; really for cosmetics only -- (and actually contrary to EBU recommendations, although in line -- with typical user expectations). res.frame_rate_nom = res.frame_rate_nom * 2 end input_resolution[signal_num] = res end last_resolution = input_resolution if num == 0 then -- Live. finish_transitions(t) if transition_type == ZOOM_TRANSITION then -- Transition in or out of SBS. local chain = get_sbs_chain(signals, t, width, height, input_resolution) prepare = function() prepare_sbs_chain(chain, calc_zoom_progress(t), transition_type, transition_src_signal, transition_dst_signal, width, height, input_resolution) end return chain.chain, prepare elseif transition_type == NO_TRANSITION and live_signal_num == SBS_SIGNAL_NUM then -- Static SBS view. local chain = get_sbs_chain(signals, t, width, height, input_resolution) prepare = function() prepare_sbs_chain(chain, 0.0, NO_TRANSITION, 0, SBS_SIGNAL_NUM, width, height, input_resolution) end return chain.chain, prepare elseif transition_type == FADE_TRANSITION then return get_fade_chain(signals, t, width, height, input_resolution) elseif is_plain_signal(live_signal_num) then local input_type = get_input_type(signals, live_signal_num) local input_scale = needs_scale(signals, live_signal_num, width, height) local chain = simple_chains[input_type][input_scale][true] prepare = function() chain.input:connect_signal(live_signal_num) set_scale_parameters_if_needed(chain, width, height) set_neutral_color_from_signal(chain.wb_effect, live_signal_num) end return chain.chain, prepare elseif live_signal_num == STATIC_SIGNAL_NUM then -- Static picture. prepare = function() end return static_chain_hq, prepare else assert(false) end end if num == 1 then -- Preview. num = preview_signal_num + 2 end -- Individual preview inputs. if is_plain_signal(num - 2) then local signal_num = num - 2 local input_type = get_input_type(signals, signal_num) local input_scale = needs_scale(signals, signal_num, width, height) local chain = simple_chains[input_type][input_scale][false] prepare = function() chain.input:connect_signal(signal_num) set_scale_parameters_if_needed(chain, width, height) set_neutral_color(chain.wb_effect, neutral_colors[signal_num + 1]) end return chain.chain, prepare end if num == SBS_SIGNAL_NUM + 2 then local input0_type = get_input_type(signals, INPUT0_SIGNAL_NUM) local input1_type = get_input_type(signals, INPUT1_SIGNAL_NUM) local chain = sbs_chains[input0_type][input1_type][false] prepare = function() prepare_sbs_chain(chain, 0.0, NO_TRANSITION, 0, SBS_SIGNAL_NUM, width, height, input_resolution) end return chain.chain, prepare end if num == STATIC_SIGNAL_NUM + 2 then prepare = function() end return static_chain_lq, prepare end end function place_rectangle(resample_effect, resize_effect, padding_effect, x0, y0, x1, y1, screen_width, screen_height, input_width, input_height) local srcx0 = 0.0 local srcx1 = 1.0 local srcy0 = 0.0 local srcy1 = 1.0 padding_effect:set_int("width", screen_width) padding_effect:set_int("height", screen_height) -- Cull. if x0 > screen_width or x1 < 0.0 or y0 > screen_height or y1 < 0.0 then if resample_effect ~= nil then resample_effect:set_int("width", 1) resample_effect:set_int("height", 1) resample_effect:set_float("zoom_x", screen_width) resample_effect:set_float("zoom_y", screen_height) else resize_effect:set_int("width", 1) resize_effect:set_int("height", 1) end padding_effect:set_int("left", screen_width + 100) padding_effect:set_int("top", screen_height + 100) return end -- Clip. if x0 < 0 then srcx0 = -x0 / (x1 - x0) x0 = 0 end if y0 < 0 then srcy0 = -y0 / (y1 - y0) y0 = 0 end if x1 > screen_width then srcx1 = (screen_width - x0) / (x1 - x0) x1 = screen_width end if y1 > screen_height then srcy1 = (screen_height - y0) / (y1 - y0) y1 = screen_height end if resample_effect ~= nil then -- High-quality resampling. local x_subpixel_offset = x0 - math.floor(x0) local y_subpixel_offset = y0 - math.floor(y0) -- Resampling must be to an integral number of pixels. Round up, -- and then add an extra pixel so we have some leeway for the border. local width = math.ceil(x1 - x0) + 1 local height = math.ceil(y1 - y0) + 1 resample_effect:set_int("width", width) resample_effect:set_int("height", height) -- Correct the discrepancy with zoom. (This will leave a small -- excess edge of pixels and subpixels, which we'll correct for soon.) local zoom_x = (x1 - x0) / (width * (srcx1 - srcx0)) local zoom_y = (y1 - y0) / (height * (srcy1 - srcy0)) resample_effect:set_float("zoom_x", zoom_x) resample_effect:set_float("zoom_y", zoom_y) resample_effect:set_float("zoom_center_x", 0.0) resample_effect:set_float("zoom_center_y", 0.0) -- Padding must also be to a whole-pixel offset. padding_effect:set_int("left", math.floor(x0)) padding_effect:set_int("top", math.floor(y0)) -- Correct _that_ discrepancy by subpixel offset in the resampling. resample_effect:set_float("left", srcx0 * input_width - x_subpixel_offset / zoom_x) resample_effect:set_float("top", srcy0 * input_height - y_subpixel_offset / zoom_y) -- Finally, adjust the border so it is exactly where we want it. padding_effect:set_float("border_offset_left", x_subpixel_offset) padding_effect:set_float("border_offset_right", x1 - (math.floor(x0) + width)) padding_effect:set_float("border_offset_top", y_subpixel_offset) padding_effect:set_float("border_offset_bottom", y1 - (math.floor(y0) + height)) else -- Lower-quality simple resizing. local width = round(x1 - x0) local height = round(y1 - y0) resize_effect:set_int("width", width) resize_effect:set_int("height", height) -- Padding must also be to a whole-pixel offset. padding_effect:set_int("left", math.floor(x0)) padding_effect:set_int("top", math.floor(y0)) end end -- This is broken, of course (even for positive numbers), but Lua doesn't give us access to real rounding. function round(x) return math.floor(x + 0.5) end function lerp(a, b, t) return a + (b - a) * t end function lerp_pos(a, b, t) return { x0 = lerp(a.x0, b.x0, t), y0 = lerp(a.y0, b.y0, t), x1 = lerp(a.x1, b.x1, t), y1 = lerp(a.y1, b.y1, t) } end function pos_from_top_left(x, y, width, height, screen_width, screen_height) local xs = screen_width / 1280.0 local ys = screen_height / 720.0 return { x0 = round(xs * x), y0 = round(ys * y), x1 = round(xs * (x + width)), y1 = round(ys * (y + height)) } end function prepare_sbs_chain(chain, t, transition_type, src_signal, dst_signal, screen_width, screen_height, input_resolution) chain.input0.input:connect_signal(0) chain.input1.input:connect_signal(1) set_neutral_color(chain.input0.wb_effect, neutral_colors[1]) set_neutral_color(chain.input1.wb_effect, neutral_colors[2]) -- First input is positioned (16,48) from top-left. -- Second input is positioned (16,48) from the bottom-right. local pos0 = pos_from_top_left(16, 48, 848, 477, screen_width, screen_height) local pos1 = pos_from_top_left(1280 - 384 - 16, 720 - 216 - 48, 384, 216, screen_width, screen_height) local pos_fs = { x0 = 0, y0 = 0, x1 = screen_width, y1 = screen_height } local affine_param if transition_type == NO_TRANSITION then -- Static SBS view. affine_param = { sx = 1.0, sy = 1.0, tx = 0.0, ty = 0.0 } -- Identity. else -- Zooming to/from SBS view into or out of a single view. assert(transition_type == ZOOM_TRANSITION) local signal, real_t if src_signal == SBS_SIGNAL_NUM then signal = dst_signal real_t = t else assert(dst_signal == SBS_SIGNAL_NUM) signal = src_signal real_t = 1.0 - t end if signal == INPUT0_SIGNAL_NUM then affine_param = find_affine_param(pos0, lerp_pos(pos0, pos_fs, real_t)) elseif signal == INPUT1_SIGNAL_NUM then affine_param = find_affine_param(pos1, lerp_pos(pos1, pos_fs, real_t)) end end -- NOTE: input_resolution is not 1-indexed, unlike usual Lua arrays. place_rectangle_with_affine(chain.input0.resample_effect, chain.input0.resize_effect, chain.input0.padding_effect, pos0, affine_param, screen_width, screen_height, input_resolution[0].width, input_resolution[0].height) place_rectangle_with_affine(chain.input1.resample_effect, chain.input1.resize_effect, chain.input1.padding_effect, pos1, affine_param, screen_width, screen_height, input_resolution[1].width, input_resolution[1].height) end -- Find the transformation that changes the first rectangle to the second one. function find_affine_param(a, b) local sx = (b.x1 - b.x0) / (a.x1 - a.x0) local sy = (b.y1 - b.y0) / (a.y1 - a.y0) return { sx = sx, sy = sy, tx = b.x0 - a.x0 * sx, ty = b.y0 - a.y0 * sy } end function place_rectangle_with_affine(resample_effect, resize_effect, padding_effect, pos, aff, screen_width, screen_height, input_width, input_height) local x0 = pos.x0 * aff.sx + aff.tx local x1 = pos.x1 * aff.sx + aff.tx local y0 = pos.y0 * aff.sy + aff.ty local y1 = pos.y1 * aff.sy + aff.ty place_rectangle(resample_effect, resize_effect, padding_effect, x0, y0, x1, y1, screen_width, screen_height, input_width, input_height) end function set_neutral_color(effect, color) effect:set_vec3("neutral_color", color[1], color[2], color[3]) end function set_neutral_color_from_signal(effect, signal) if is_plain_signal(signal) then set_neutral_color(effect, neutral_colors[signal - INPUT0_SIGNAL_NUM + 1]) end end function calc_zoom_progress(t) if t < transition_start then return 0.0 elseif t > transition_end then return 1.0 else local tt = (t - transition_start) / (transition_end - transition_start) -- Smooth it a bit. return math.sin(tt * 3.14159265358 * 0.5) end end function calc_fade_progress(t, transition_start, transition_end) local tt = (t - transition_start) / (transition_end - transition_start) if tt < 0.0 then return 0.0 elseif tt > 1.0 then return 1.0 end -- Make the fade look maybe a tad more natural, by pumping it -- through a sigmoid function. tt = 10.0 * tt - 5.0 tt = 1.0 / (1.0 + math.exp(-tt)) return tt end nageru-1.4.2/timebase.h000066400000000000000000000016351301561706600147530ustar00rootroot00000000000000#ifndef _TIMEBASE_H #define _TIMEBASE_H 1 // Common timebase that allows us to represent one frame exactly in all the // relevant frame rates: // // Timebase: 1/60000 // Frame at 50fps: 1200/60000 // Frame at 60fps: 1000/60000 // Frame at 59.94fps: 1001/60000 // // If we also wanted to represent one sample at 48000 Hz, we'd need // to go to 300000. Also supporting one sample at 44100 Hz would mean // going to 44100000; probably a bit excessive. #define TIMEBASE 60000 // Some muxes, like MP4 (or at least avformat's implementation of it), // are not too fond of values above 2^31. At timebase 60000, that's only // about ten hours or so, so we define a coarser timebase that doesn't // get 59.94 precisely (so there will be a marginal amount of pts jitter), // but can do at least 50 and 60 precisely, and months of streaming. #define COARSE_TIMEBASE 300 #endif // !defined(_TIMEBASE_H) nageru-1.4.2/ui_aboutdialog.ui000066400000000000000000000146511301561706600163410ustar00rootroot00000000000000 AboutDialog 0 0 684 544 About Nageru <p><b>Nageru 1.4.2</b></p> <p>Realtime video mixer</p> true <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd"> <html><head><meta name="qrichtext" content="1" /></head><body> <p> Nageru is Copyright (C) 2015 Steinar H. Gunderson &lt;steinar+nageru@gunderson.no&gt;<br /> Portions Copyright (C) 2003 Rune Holm.<br /> Portions Copyright (C) 2010-2011 Fons Adriaensen &lt;fons@linuxaudio.org&gt;<br /> Portions Copyright (C) 2012-2015 Fons Adriaensen &lt;fons@linuxaudio.org&gt;<br /> Portions Copyright (C) 2008-2015 Fons Adriaensen &lt;fons@linuxaudio.org&gt;<br /> Portions Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved.</p> <p>This program is free software: you can redistribute it and/or modify<br /> it under the terms of the GNU General Public License as published by<br /> the Free Software Foundation, either version 3 of the License, or<br /> (at your option) any later version.</p> <p>This program is distributed in the hope that it will be useful,<br /> but WITHOUT ANY WARRANTY; without even the implied warranty of<br /> MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the<br /> GNU General Public License for more details.</p> <p>You should have received a copy of the GNU General Public License<br /> along with this program. If not, see &lt;<a href="http://www.gnu.org/licenses/"><span style=" text-decoration: underline; color:#0000ff;">http://www.gnu.org/licenses/</span></a>&gt;.</p> <p><br />Portions of h264encode.h and h264encode.cpp:</p> <p>Copyright (c) 2007-2013 Intel Corporation. All Rights Reserved.</p> <p>Permission is hereby granted, free of charge, to any person obtaining a<br /> copy of this software and associated documentation files (the<br /> &quot;Software&quot;), to deal in the Software without restriction, including<br /> without limitation the rights to use, copy, modify, merge, publish,<br /> distribute, sub license, and/or sell copies of the Software, and to<br /> permit persons to whom the Software is furnished to do so, subject to<br /> the following conditions:</p> <p>The above copyright notice and this permission notice (including the<br /> next paragraph) shall be included in all copies or substantial portions<br /> of the Software.</p> <p>THE SOFTWARE IS PROVIDED &quot;AS IS&quot;, WITHOUT WARRANTY OF ANY KIND, EXPRESS<br /> OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF<br /> MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.<br /> IN NO EVENT SHALL PRECISION INSIGHT AND/OR ITS SUPPLIERS BE LIABLE FOR<br /> ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,<br /> TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE<br /> SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</p> <p><br />All files in decklink/:</p> <p>Copyright (c) 2009 Blackmagic Design<br /> Copyright (c) 2015 Blackmagic Design</p> <p>Permission is hereby granted, free of charge, to any person or organization obtaining a copy of the software and accompanying documentation covered by this license (the "Software") to use, reproduce, display, distribute, execute, and transmit the Software, and to prepare derivative works of the Software, and to permit third-parties to whom the Software is furnished to do so, all subject to the following:</p> <p>The copyright notices in the Software and this entire statement, including the above license grant, this restriction and the following disclaimer, must be included in all copies of the Software, in whole or in part, and all derivative works of the Software, unless such copies or derivative works are solely in the form of machine-executable object code generated by a source language processor.</p> <p>THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</p> </body></html> Qt::Horizontal QDialogButtonBox::Ok button_box accepted() Dialog accept() 248 254 157 274 button_box rejected() Dialog reject() 316 260 286 274 nageru-1.4.2/ui_audio_expanded_view.ui000066400000000000000000000357061301561706600200560ustar00rootroot00000000000000 AudioExpandedView 0 0 312 434 AudioExpandedView DejaVu Sans 75 true true Bus name Qt::AlignCenter Qt::Vertical 20 40 8 Lo-cut 31 31 -150 150 60.000000000000000 true Treble: +0.0 dB 31 31 -150 150 60.000000000000000 true Mid: +0.0 dB 31 31 -150 150 60.000000000000000 true Bass: +0.0 dB Qt::Horizontal 8 Auto gain staging true 31 31 -300 300 60.000000000000000 true Gain: +0.0 dB Qt::Horizontal Qt::Horizontal 40 20 Compressor true Qt::Horizontal 40 20 Threshold Qt::AlignCenter 64 64 -400 0 -260 30.000000000000000 true -10.0 dB Qt::AlignCenter 0 Reduction Qt::AlignCenter 16777215 16777215 Qt::Vertical 20 40 20 16777215 60 0 -40.0 Qt::AlignCenter 0 0 0 0 40 22 8 QPushButton:checked { background: rgba(255,0,0,80); } Mute true 1000 10 100 Qt::Vertical 60 0 +0.0 dB Qt::AlignCenter VUMeter QWidget

vumeter.h
1 ClickableLabel QLabel
clickable_label.h
 NonLinearFader QSlider
nonlinear_fader.h
 EllipsisLabel QLabel
ellipsis_label.h
 CompressionReductionMeter QWidget
compression_reduction_meter.h
1 nageru-1.4.2/ui_audio_miniview.ui000066400000000000000000000266021301561706600170560ustar00rootroot00000000000000 AudioMiniView 0 0 139 300 0 0 139 0 139 16777215 Form 0 0 0 0 0 1 true QFrame::Panel QFrame::Plain 0 6 0 0 0 0 0 6 1 0 0 0 Channel description Qt::AlignCenter 0 0 4 0 0 0 1 16 0 1 0 0 0 255 255 255 5 239 111 255 255 255 5 239 111 5 239 111 5 239 111 true 30 0 -0.0 Qt::AlignCenter 4 0 0 1000 10 100 Qt::Vertical QSlider::NoTicks 30 0 0 30 0 0 +0.0 dB Qt::AlignCenter Qt::Vertical VUMeter QWidget
vumeter.h
1 EllipsisLabel QLabel
ellipsis_label.h
 NonLinearFader QSlider
nonlinear_fader.h
 ClickableLabel QLabel
clickable_label.h
 nageru-1.4.2/ui_display.ui000066400000000000000000000060271301561706600155120ustar00rootroot00000000000000 Display 0 0 606 433 Form 0 0 0 0 0 1 true QFrame::Box QFrame::Plain 0 3 3 3 3 0 0 1 0 0 24 TextLabel Qt::AlignCenter Set WB GLWidget QWidget
glwidget.h
 nageru-1.4.2/ui_input_mapping.ui000066400000000000000000000107511301561706600167160ustar00rootroot00000000000000 InputMappingDialog 0 0 879 583 Input mapping Device Left input Right input 0 0 30 30 .. 0 0 30 30 .. 30 30 .. 30 30 .. Qt::Horizontal 40 20 &Save… &Load… Qt::Horizontal 40 20 Qt::Horizontal QDialogButtonBox::Cancel|QDialogButtonBox::Ok nageru-1.4.2/ui_mainwindow.ui000066400000000000000000001460261301561706600162250ustar00rootroot00000000000000 MainWindow 0 0 1089 664 Nageru true 0 0 0 1 1 Preview Qt::AlignCenter 25 0 20 0 0 115 0 16777215 16777215 Cut 0 0 115 0 16777215 16777215 Fade 0 0 115 0 16777215 16777215 Wipe 0 1 1 16 9 16 9 Live Qt::AlignCenter 0 0 0 0 14 255 255 255 239 0 4 255 255 255 239 0 4 239 0 4 239 0 4 true 0 4 0 0 1 16 0 1 0 0 0 255 255 255 5 239 111 255 255 255 5 239 111 5 239 111 5 239 111 true 30 0 -0.0 Qt::AlignCenter 3 0 0 0 24 0 255 255 255 0 239 219 255 255 255 0 239 219 0 239 219 0 239 219 true 30 20 RST false 0 0 0 0 0 8 0 0 0 0 0 0 Compact audio view Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter true 15 15 ... true Qt::LeftArrow 15 15 ... true Qt::RightArrow 6 0 QFrame::NoFrame Qt::ScrollBarAlwaysOff true 0 0 505 236 0 0 0 0 0 QLayout::SetFixedSize Qt::Horizontal 40 20 0 64 64 16777215 64 -400 0 -260 30.000000000000000 true Auto true Compr. threshold Gain staging Qt::AlignCenter -26.0 dB Qt::AlignCenter 64 64 16777215 64 -300 300 60.000000000000000 true -0.0 dB Qt::AlignCenter -14.0 dB Qt::AlignCenter Enabled true Auto true 64 64 16777215 64 -400 0 -140 30.000000000000000 true 120 Hz Qt::AlignCenter Enabled true -0.0 dB Qt::AlignCenter Enabled true Lo-cut (24dB/oct) Qt::Vertical QSizePolicy::Expanding 20 40 Qt::Vertical 20 40 64 64 16777215 64 -150 150 30.000000000000000 true Limiter threshold Qt::AlignCenter Makeup gain Qt::AlignCenter 0 0 64 64 16777215 64 60 26 Qt::Horizontal QSizePolicy::Preferred 0 40 0 0 0 0 0 Full audio view Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter true 15 15 ... true Qt::LeftArrow 15 15 ... true Qt::RightArrow 6 0 QFrame::NoFrame Qt::ScrollBarAlwaysOff true 0 0 728 282 0 0 0 0 0 QLayout::SetFixedSize Qt::Horizontal 723 20 0 120 Hz Qt::AlignCenter -0.0 dB Qt::AlignCenter -14.0 dB Qt::AlignCenter 0 0 64 64 16777215 64 60 26 Enabled true 64 64 16777215 64 -400 0 -140 30.000000000000000 true Auto true Qt::Vertical 20 40 Lo-cut (24dB/oct) Qt::Vertical QSizePolicy::Expanding 20 40 64 64 16777215 64 -150 150 30.000000000000000 true Limiter threshold Qt::AlignCenter Makeup gain Qt::AlignCenter 0 0 1089 23 &Video &Help &Audio &Exit &Begin new video segment &About Nageru… Change &x264 bitrate… &Input mapping… true true Simple true Multichannel Setup MIDI controller… VUMeter QWidget
vumeter.h
1 ClickableLabel QLabel
clickable_label.h
 GLWidget QWidget
glwidget.h
 LRAMeter QWidget
lrameter.h
1 CorrelationMeter QWidget
correlation_meter.h
1 nageru-1.4.2/ui_midi_mapping.ui000066400000000000000000000053221301561706600164770ustar00rootroot00000000000000 MIDIMappingDialog 0 0 879 583 MIDI controller setup 1 Add or change a mapping by clicking in the cell, then moving the corresponding control on your MIDI device. Guess &bus Guess &group Qt::Horizontal 40 20 &Save… &Load… Qt::Horizontal 40 20 Qt::Horizontal QDialogButtonBox::Cancel|QDialogButtonBox::Ok nageru-1.4.2/video_encoder.cpp000066400000000000000000000143471301561706600163260ustar00rootroot00000000000000#include "video_encoder.h" #include #include #include #include #include #include extern "C" { #include } #include "audio_encoder.h" #include "defs.h" #include "ffmpeg_raii.h" #include "flags.h" #include "httpd.h" #include "mux.h" #include "quicksync_encoder.h" #include "timebase.h" #include "x264_encoder.h" class RefCountedFrame; using namespace std; using namespace movit; namespace { string generate_local_dump_filename(int frame) { time_t now = time(NULL); tm now_tm; localtime_r(&now, &now_tm); char timestamp[256]; strftime(timestamp, sizeof(timestamp), "%F-%T%z", &now_tm); // Use the frame number to disambiguate between two cuts starting // on the same second. char filename[256]; snprintf(filename, sizeof(filename), "%s%s-f%02d%s", LOCAL_DUMP_PREFIX, timestamp, frame % 100, LOCAL_DUMP_SUFFIX); return filename; } } // namespace VideoEncoder::VideoEncoder(ResourcePool *resource_pool, QSurface *surface, const std::string &va_display, int width, int height, HTTPD *httpd, DiskSpaceEstimator *disk_space_estimator) : resource_pool(resource_pool), surface(surface), va_display(va_display), width(width), height(height), httpd(httpd), disk_space_estimator(disk_space_estimator) { oformat = av_guess_format(global_flags.stream_mux_name.c_str(), nullptr, nullptr); assert(oformat != nullptr); if (global_flags.stream_audio_codec_name.empty()) { stream_audio_encoder.reset(new AudioEncoder(AUDIO_OUTPUT_CODEC_NAME, DEFAULT_AUDIO_OUTPUT_BIT_RATE, oformat)); } else { stream_audio_encoder.reset(new AudioEncoder(global_flags.stream_audio_codec_name, global_flags.stream_audio_codec_bitrate, oformat)); } if (global_flags.x264_video_to_http) { x264_encoder.reset(new X264Encoder(oformat)); } string filename = generate_local_dump_filename(/*frame=*/0); quicksync_encoder.reset(new QuickSyncEncoder(filename, resource_pool, surface, va_display, width, height, oformat, x264_encoder.get(), disk_space_estimator)); open_output_stream(); stream_audio_encoder->add_mux(stream_mux.get()); quicksync_encoder->set_stream_mux(stream_mux.get()); if (global_flags.x264_video_to_http) { x264_encoder->set_mux(stream_mux.get()); } } VideoEncoder::~VideoEncoder() { quicksync_encoder.reset(nullptr); while (quicksync_encoders_in_shutdown.load() > 0) { usleep(10000); } } void VideoEncoder::do_cut(int frame) { string filename = generate_local_dump_filename(frame); printf("Starting new recording: %s\n", filename.c_str()); // Do the shutdown of the old encoder in a separate thread, since it can // take some time (it needs to wait for all the frames in the queue to be // done encoding, for one) and we are running on the main mixer thread. // However, since this means both encoders could be sending packets at // the same time, it means pts could come out of order to the stream mux, // and we need to plug it until the shutdown is complete. stream_mux->plug(); lock_guard lock(qs_mu); QuickSyncEncoder *old_encoder = quicksync_encoder.release(); // When we go C++14, we can use move capture instead. thread([old_encoder, this]{ old_encoder->shutdown(); stream_mux->unplug(); // We cannot delete the encoder here, as this thread has no OpenGL context. // We'll deal with it in begin_frame(). lock_guard lock(qs_mu); qs_needing_cleanup.emplace_back(old_encoder); }).detach(); quicksync_encoder.reset(new QuickSyncEncoder(filename, resource_pool, surface, va_display, width, height, oformat, x264_encoder.get(), disk_space_estimator)); quicksync_encoder->set_stream_mux(stream_mux.get()); } void VideoEncoder::change_x264_bitrate(unsigned rate_kbit) { x264_encoder->change_bitrate(rate_kbit); } void VideoEncoder::add_audio(int64_t pts, std::vector audio) { lock_guard lock(qs_mu); quicksync_encoder->add_audio(pts, audio); stream_audio_encoder->encode_audio(audio, pts + quicksync_encoder->global_delay()); } bool VideoEncoder::begin_frame(GLuint *y_tex, GLuint *cbcr_tex) { lock_guard lock(qs_mu); qs_needing_cleanup.clear(); // Since we have an OpenGL context here, and are called regularly. return quicksync_encoder->begin_frame(y_tex, cbcr_tex); } RefCountedGLsync VideoEncoder::end_frame(int64_t pts, int64_t duration, const std::vector &input_frames) { lock_guard lock(qs_mu); return quicksync_encoder->end_frame(pts, duration, input_frames); } void VideoEncoder::open_output_stream() { AVFormatContext *avctx = avformat_alloc_context(); avctx->oformat = oformat; uint8_t *buf = (uint8_t *)av_malloc(MUX_BUFFER_SIZE); avctx->pb = avio_alloc_context(buf, MUX_BUFFER_SIZE, 1, this, nullptr, nullptr, nullptr); avctx->pb->write_data_type = &VideoEncoder::write_packet2_thunk; avctx->pb->ignore_boundary_point = 1; Mux::Codec video_codec; if (global_flags.uncompressed_video_to_http) { video_codec = Mux::CODEC_NV12; } else { video_codec = Mux::CODEC_H264; } avctx->flags = AVFMT_FLAG_CUSTOM_IO; string video_extradata; if (global_flags.x264_video_to_http) { video_extradata = x264_encoder->get_global_headers(); } int time_base = global_flags.stream_coarse_timebase ? COARSE_TIMEBASE : TIMEBASE; stream_mux.reset(new Mux(avctx, width, height, video_codec, video_extradata, stream_audio_encoder->get_codec_parameters().get(), time_base, /*write_callback=*/nullptr)); } int VideoEncoder::write_packet2_thunk(void *opaque, uint8_t *buf, int buf_size, AVIODataMarkerType type, int64_t time) { VideoEncoder *video_encoder = (VideoEncoder *)opaque; return video_encoder->write_packet2(buf, buf_size, type, time); } int VideoEncoder::write_packet2(uint8_t *buf, int buf_size, AVIODataMarkerType type, int64_t time) { if (type == AVIO_DATA_MARKER_SYNC_POINT || type == AVIO_DATA_MARKER_BOUNDARY_POINT) { seen_sync_markers = true; } else if (type == AVIO_DATA_MARKER_UNKNOWN && !seen_sync_markers) { // We don't know if this is a keyframe or not (the muxer could // avoid marking it), so we just have to make the best of it. type = AVIO_DATA_MARKER_SYNC_POINT; } if (type == AVIO_DATA_MARKER_HEADER) { stream_mux_header.append((char *)buf, buf_size); httpd->set_header(stream_mux_header); } else { httpd->add_data((char *)buf, buf_size, type == AVIO_DATA_MARKER_SYNC_POINT); } return buf_size; } nageru-1.4.2/video_encoder.h000066400000000000000000000045141301561706600157660ustar00rootroot00000000000000// A class to orchestrate the concept of video encoding. Will keep track of // the muxes to stream and disk, the QuickSyncEncoder, and also the X264Encoder // (for the stream) if there is one. #ifndef _VIDEO_ENCODER_H #define _VIDEO_ENCODER_H #include #include #include #include #include #include #include #include extern "C" { #include #include } #include "ref_counted_gl_sync.h" class AudioEncoder; class DiskSpaceEstimator; class HTTPD; class Mux; class QSurface; class QuickSyncEncoder; class RefCountedFrame; class X264Encoder; namespace movit { class ResourcePool; } // namespace movit class VideoEncoder { public: VideoEncoder(movit::ResourcePool *resource_pool, QSurface *surface, const std::string &va_display, int width, int height, HTTPD *httpd, DiskSpaceEstimator *disk_space_estimator); ~VideoEncoder(); void add_audio(int64_t pts, std::vector audio); bool begin_frame(GLuint *y_tex, GLuint *cbcr_tex); RefCountedGLsync end_frame(int64_t pts, int64_t duration, const std::vector &input_frames); // Does a cut of the disk stream immediately ("frame" is used for the filename only). void do_cut(int frame); void change_x264_bitrate(unsigned rate_kbit); private: void open_output_stream(); static int write_packet2_thunk(void *opaque, uint8_t *buf, int buf_size, AVIODataMarkerType type, int64_t time); int write_packet2(uint8_t *buf, int buf_size, AVIODataMarkerType type, int64_t time); AVOutputFormat *oformat; std::mutex qs_mu; std::unique_ptr quicksync_encoder; // Under . movit::ResourcePool *resource_pool; QSurface *surface; std::string va_display; int width, height; HTTPD *httpd; DiskSpaceEstimator *disk_space_estimator; bool seen_sync_markers = false; std::unique_ptr stream_mux; // To HTTP. std::unique_ptr stream_audio_encoder; std::unique_ptr x264_encoder; // nullptr if not using x264. std::string stream_mux_header; std::atomic quicksync_encoders_in_shutdown{0}; // Encoders that are shutdown, but need to call release_gl_resources() // (or be deleted) from some thread with an OpenGL context. std::vector> qs_needing_cleanup; // Under . }; #endif nageru-1.4.2/vu_common.cpp000066400000000000000000000043501301561706600155140ustar00rootroot00000000000000#include "vu_common.h" #include #include #include #include using namespace std; double lufs_to_pos(float level_lu, int height, float min_level, float max_level) { // Note: “max” is the loudest level, but y=0 is top of screen. // Handle -inf. if (level_lu < min_level) { return height - 1; } double y = height * (level_lu - max_level) / (min_level - max_level); y = max(y, 0); y = min(y, height - 1); // If we are big enough, snap to pixel grid instead of antialiasing // the edges; the unevenness will be less noticeable than the blurriness. double height_per_level = height / (max_level - min_level) - 2.0; if (height_per_level >= 10.0) { y = round(y); } return y; } void draw_vu_meter(QPainter &painter, int width, int height, int horizontal_margin, double segment_margin, bool is_on, float min_level, float max_level, bool flip, int y_offset) { painter.fillRect(horizontal_margin, y_offset, width - 2 * horizontal_margin, height, Qt::black); for (int y = 0; y < height; ++y) { // Find coverage of “on” rectangles in this pixel row. double coverage = 0.0; for (int level = floor(min_level); level <= ceil(max_level); ++level) { double min_y = lufs_to_pos(level + 1.0, height, min_level, max_level) + segment_margin * 0.5; double max_y = lufs_to_pos(level, height, min_level, max_level) - segment_margin * 0.5; min_y = std::max(min_y, y); min_y = std::min(min_y, y + 1); max_y = std::max(max_y, y); max_y = std::min(max_y, y + 1); coverage += max_y - min_y; } double on_r, on_g, on_b; if (is_on) { double t = double(y) / height; if (t <= 0.5) { on_r = 1.0; on_g = 2.0 * t; on_b = 0.0; } else { on_r = 1.0 - 2.0 * (t - 0.5); on_g = 1.0; on_b = 0.0; } } else { on_r = on_g = on_b = 0.05; } // Correct for coverage and do a simple gamma correction. int r = lrintf(255 * pow(on_r * coverage, 1.0 / 2.2)); int g = lrintf(255 * pow(on_g * coverage, 1.0 / 2.2)); int b = lrintf(255 * pow(on_b * coverage, 1.0 / 2.2)); int draw_y = flip ? (height - y - 1) : y; painter.fillRect(horizontal_margin, draw_y + y_offset, width - 2 * horizontal_margin, 1, QColor(r, g, b)); } } nageru-1.4.2/vu_common.h000066400000000000000000000005521301561706600151610ustar00rootroot00000000000000#ifndef _VU_COMMON_H #define _VU_COMMON_H 1 class QPainter; double lufs_to_pos(float level_lu, int height, float min_level, float max_level); void draw_vu_meter(QPainter &painter, int width, int height, int horizontal_margin, double segment_margin, bool is_on, float min_level, float max_level, bool flip, int y_offset = 0); #endif // !defined(_VU_COMMON_H) nageru-1.4.2/vumeter.cpp000066400000000000000000000043301301561706600151770ustar00rootroot00000000000000#include "vumeter.h" #include #include #include "vu_common.h" class QPaintEvent; class QResizeEvent; using namespace std; VUMeter::VUMeter(QWidget *parent) : QWidget(parent) { } void VUMeter::resizeEvent(QResizeEvent *event) { recalculate_pixmaps(); } void VUMeter::paintEvent(QPaintEvent *event) { QPainter painter(this); float level_lufs[2], peak_lufs[2]; { unique_lock lock(level_mutex); level_lufs[0] = this->level_lufs[0]; level_lufs[1] = this->level_lufs[1]; peak_lufs[0] = this->peak_lufs[0]; peak_lufs[1] = this->peak_lufs[1]; } int mid = width() / 2; for (unsigned channel = 0; channel < 2; ++channel) { int left = (channel == 0) ? 0 : mid; int right = (channel == 0) ? mid : width(); float level_lu = level_lufs[channel] - ref_level_lufs; int on_pos = lrint(lufs_to_pos(level_lu, height())); QRect off_rect(left, 0, right - left, on_pos); QRect on_rect(left, on_pos, right - left, height() - on_pos); painter.drawPixmap(off_rect, off_pixmap, off_rect); painter.drawPixmap(on_rect, on_pixmap, on_rect); float peak_lu = peak_lufs[channel] - ref_level_lufs; if (peak_lu >= min_level && peak_lu <= max_level) { int peak_pos = lrint(lufs_to_pos(peak_lu, height())); QRect peak_rect(left, peak_pos - 1, right, 2); painter.drawPixmap(peak_rect, full_on_pixmap, peak_rect); } } } void VUMeter::recalculate_pixmaps() { full_on_pixmap = QPixmap(width(), height()); QPainter full_on_painter(&full_on_pixmap); full_on_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(full_on_painter, width(), height(), 0, 0.0, true, min_level, max_level, /*flip=*/false); float margin = 0.5 * (width() - 20); on_pixmap = QPixmap(width(), height()); QPainter on_painter(&on_pixmap); on_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(on_painter, width(), height(), margin, 2.0, true, min_level, max_level, /*flip=*/false); off_pixmap = QPixmap(width(), height()); QPainter off_painter(&off_pixmap); off_painter.fillRect(0, 0, width(), height(), parentWidget()->palette().window()); draw_vu_meter(off_painter, width(), height(), margin, 2.0, false, min_level, max_level, /*flip=*/false); } nageru-1.4.2/vumeter.h000066400000000000000000000033411301561706600146450ustar00rootroot00000000000000#ifndef VUMETER_H #define VUMETER_H #include #include #include #include #include #include "vu_common.h" class QObject; class QPaintEvent; class QResizeEvent; class VUMeter : public QWidget { Q_OBJECT public: VUMeter(QWidget *parent); void set_level(float level_lufs) { set_level(level_lufs, level_lufs); } void set_level(float level_lufs_left, float level_lufs_right) { std::unique_lock lock(level_mutex); this->level_lufs[0] = level_lufs_left; this->level_lufs[1] = level_lufs_right; QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); } void set_peak(float peak_lufs) { set_peak(peak_lufs, peak_lufs); } void set_peak(float peak_lufs_left, float peak_lufs_right) { std::unique_lock lock(level_mutex); this->peak_lufs[0] = peak_lufs_left; this->peak_lufs[1] = peak_lufs_right; QMetaObject::invokeMethod(this, "update", Qt::AutoConnection); } double lufs_to_pos(float level_lu, int height) { return ::lufs_to_pos(level_lu, height, min_level, max_level); } void set_min_level(float min_level) { this->min_level = min_level; recalculate_pixmaps(); } void set_max_level(float max_level) { this->max_level = max_level; recalculate_pixmaps(); } void set_ref_level(float ref_level_lufs) { this->ref_level_lufs = ref_level_lufs; } private: void resizeEvent(QResizeEvent *event) override; void paintEvent(QPaintEvent *event) override; void recalculate_pixmaps(); std::mutex level_mutex; float level_lufs[2] { -HUGE_VALF, -HUGE_VALF }; float peak_lufs[2] { -HUGE_VALF, -HUGE_VALF }; float min_level = -18.0f, max_level = 9.0f, ref_level_lufs = -23.0f; QPixmap full_on_pixmap, on_pixmap, off_pixmap; }; #endif nageru-1.4.2/x264_encoder.cpp000066400000000000000000000213141301561706600157130ustar00rootroot00000000000000#include "x264_encoder.h" #include #include #include #include #include #include #include "defs.h" #include "flags.h" #include "mux.h" #include "timebase.h" #include "x264_speed_control.h" extern "C" { #include #include } using namespace std; namespace { void update_vbv_settings(x264_param_t *param) { if (global_flags.x264_vbv_buffer_size < 0) { param->rc.i_vbv_buffer_size = param->rc.i_bitrate; // One-second VBV. } else { param->rc.i_vbv_buffer_size = global_flags.x264_vbv_buffer_size; } if (global_flags.x264_vbv_max_bitrate < 0) { param->rc.i_vbv_max_bitrate = param->rc.i_bitrate; // CBR. } else { param->rc.i_vbv_max_bitrate = global_flags.x264_vbv_max_bitrate; } } } // namespace X264Encoder::X264Encoder(AVOutputFormat *oformat) : wants_global_headers(oformat->flags & AVFMT_GLOBALHEADER) { frame_pool.reset(new uint8_t[WIDTH * HEIGHT * 2 * X264_QUEUE_LENGTH]); for (unsigned i = 0; i < X264_QUEUE_LENGTH; ++i) { free_frames.push(frame_pool.get() + i * (WIDTH * HEIGHT * 2)); } encoder_thread = thread(&X264Encoder::encoder_thread_func, this); } X264Encoder::~X264Encoder() { should_quit = true; queued_frames_nonempty.notify_all(); encoder_thread.join(); } void X264Encoder::add_frame(int64_t pts, int64_t duration, const uint8_t *data) { QueuedFrame qf; qf.pts = pts; qf.duration = duration; { lock_guard lock(mu); if (free_frames.empty()) { fprintf(stderr, "WARNING: x264 queue full, dropping frame with pts %ld\n", pts); return; } qf.data = free_frames.front(); free_frames.pop(); } memcpy(qf.data, data, WIDTH * HEIGHT * 2); { lock_guard lock(mu); queued_frames.push(qf); queued_frames_nonempty.notify_all(); } } void X264Encoder::init_x264() { x264_param_t param; x264_param_default_preset(¶m, global_flags.x264_preset.c_str(), global_flags.x264_tune.c_str()); param.i_width = WIDTH; param.i_height = HEIGHT; param.i_csp = X264_CSP_NV12; param.b_vfr_input = 1; param.i_timebase_num = 1; param.i_timebase_den = TIMEBASE; param.i_keyint_max = 50; // About one second. if (global_flags.x264_speedcontrol) { param.i_frame_reference = 16; // Because speedcontrol is never allowed to change this above what we set at start. } // NOTE: These should be in sync with the ones in h264encode.cpp (sbs_rbsp()). param.vui.i_vidformat = 5; // Unspecified. param.vui.b_fullrange = 0; param.vui.i_colorprim = 1; // BT.709. param.vui.i_transfer = 2; // Unspecified (since we use sRGB). param.vui.i_colmatrix = 6; // BT.601/SMPTE 170M. param.rc.i_rc_method = X264_RC_ABR; param.rc.i_bitrate = global_flags.x264_bitrate; update_vbv_settings(¶m); if (param.rc.i_vbv_max_bitrate > 0) { // If the user wants VBV control to cap the max rate, it is // also reasonable to assume that they are fine with the stream // constantly being around that rate even for very low-complexity // content; the obvious and extreme example being a static // black picture. // // One would think it's fine to have low-complexity content use // less bitrate, but it seems to cause problems in practice; // e.g. VLC seems to often drop the stream (similar to a buffer // underrun) in such cases, but only when streaming from Nageru, // not when reading a dump of the same stream from disk. // I'm not 100% sure whether it's in VLC (possibly some buffering // in the HTTP layer), in microhttpd or somewhere in Nageru itself, // but it's a typical case of problems that can arise. Similarly, // TCP's congestion control is not always fond of the rate staying // low for a while and then rising quickly -- a variation on the same // problem. // // We solve this by simply asking x264 to fill in dummy bits // in these cases, so that the bitrate stays reasonable constant. // It's a waste of bandwidth, but it makes things go much more // smoothly in these cases. (We don't do it if VBV control is off // in general, not the least because it makes no sense and x264 // thus ignores the parameter.) param.rc.b_filler = 1; } // Occasionally players have problem with extremely low quantizers; // be on the safe side. Shouldn't affect quality in any meaningful way. param.rc.i_qp_min = 5; for (const string &str : global_flags.x264_extra_param) { const size_t pos = str.find(','); if (pos == string::npos) { if (x264_param_parse(¶m, str.c_str(), nullptr) != 0) { fprintf(stderr, "ERROR: x264 rejected parameter '%s'\n", str.c_str()); } } else { const string key = str.substr(0, pos); const string value = str.substr(pos + 1); if (x264_param_parse(¶m, key.c_str(), value.c_str()) != 0) { fprintf(stderr, "ERROR: x264 rejected parameter '%s' set to '%s'\n", key.c_str(), value.c_str()); } } } x264_param_apply_profile(¶m, "high"); param.b_repeat_headers = !wants_global_headers; x264 = x264_encoder_open(¶m); if (x264 == nullptr) { fprintf(stderr, "ERROR: x264 initialization failed.\n"); exit(1); } if (global_flags.x264_speedcontrol) { speed_control.reset(new X264SpeedControl(x264, /*f_speed=*/1.0f, X264_QUEUE_LENGTH, /*f_buffer_init=*/1.0f)); } if (wants_global_headers) { x264_nal_t *nal; int num_nal; x264_encoder_headers(x264, &nal, &num_nal); for (int i = 0; i < num_nal; ++i) { if (nal[i].i_type == NAL_SEI) { // Don't put the SEI in extradata; make it part of the first frame instead. buffered_sei += string((const char *)nal[i].p_payload, nal[i].i_payload); } else { global_headers += string((const char *)nal[i].p_payload, nal[i].i_payload); } } } } void X264Encoder::encoder_thread_func() { if (nice(5) == -1) { // Note that x264 further nices some of its threads. perror("nice()"); // No exit; it's not fatal. } init_x264(); x264_init_done = true; bool frames_left; do { QueuedFrame qf; // Wait for a queued frame, then dequeue it. { unique_lock lock(mu); queued_frames_nonempty.wait(lock, [this]() { return !queued_frames.empty() || should_quit; }); if (!queued_frames.empty()) { qf = queued_frames.front(); queued_frames.pop(); } else { qf.pts = -1; qf.duration = -1; qf.data = nullptr; } frames_left = !queued_frames.empty(); } encode_frame(qf); { lock_guard lock(mu); free_frames.push(qf.data); } // We should quit only if the should_quit flag is set _and_ we have nothing // in either queue. } while (!should_quit || frames_left || x264_encoder_delayed_frames(x264) > 0); x264_encoder_close(x264); } void X264Encoder::encode_frame(X264Encoder::QueuedFrame qf) { x264_nal_t *nal = nullptr; int num_nal = 0; x264_picture_t pic; x264_picture_t *input_pic = nullptr; if (qf.data) { x264_picture_init(&pic); pic.i_pts = qf.pts; pic.img.i_csp = X264_CSP_NV12; pic.img.i_plane = 2; pic.img.plane[0] = qf.data; pic.img.i_stride[0] = WIDTH; pic.img.plane[1] = qf.data + WIDTH * HEIGHT; pic.img.i_stride[1] = WIDTH / 2 * sizeof(uint16_t); pic.opaque = reinterpret_cast(intptr_t(qf.duration)); input_pic = &pic; } // See if we have a new bitrate to change to. unsigned new_rate = new_bitrate_kbit.exchange(0); // Read and clear. if (new_rate != 0) { if (speed_control) { speed_control->set_config_override_function([new_rate](x264_param_t *param) { param->rc.i_bitrate = new_rate; update_vbv_settings(param); }); } else { x264_param_t param; x264_encoder_parameters(x264, ¶m); param.rc.i_bitrate = new_rate; update_vbv_settings(¶m); x264_encoder_reconfig(x264, ¶m); } } if (speed_control) { speed_control->before_frame(float(free_frames.size()) / X264_QUEUE_LENGTH, X264_QUEUE_LENGTH, 1e6 * qf.duration / TIMEBASE); } x264_encoder_encode(x264, &nal, &num_nal, input_pic, &pic); if (speed_control) { speed_control->after_frame(); } if (num_nal == 0) return; // We really need one AVPacket for the entire frame, it seems, // so combine it all. size_t num_bytes = buffered_sei.size(); for (int i = 0; i < num_nal; ++i) { num_bytes += nal[i].i_payload; } unique_ptr data(new uint8_t[num_bytes]); uint8_t *ptr = data.get(); if (!buffered_sei.empty()) { memcpy(ptr, buffered_sei.data(), buffered_sei.size()); ptr += buffered_sei.size(); buffered_sei.clear(); } for (int i = 0; i < num_nal; ++i) { memcpy(ptr, nal[i].p_payload, nal[i].i_payload); ptr += nal[i].i_payload; } AVPacket pkt; memset(&pkt, 0, sizeof(pkt)); pkt.buf = nullptr; pkt.data = data.get(); pkt.size = num_bytes; pkt.stream_index = 0; if (pic.b_keyframe) { pkt.flags = AV_PKT_FLAG_KEY; } else { pkt.flags = 0; } pkt.duration = reinterpret_cast(pic.opaque); mux->add_packet(pkt, pic.i_pts, pic.i_dts); } nageru-1.4.2/x264_encoder.h000066400000000000000000000060441301561706600153630ustar00rootroot00000000000000// A wrapper around x264, to encode video in higher quality than Quick Sync // can give us. We maintain a queue of uncompressed Y'CbCr frames (of 50 frames, // so a little under 100 MB at 720p), then have a separate thread pull out // those threads as fast as we can to give it to x264 for encoding. // // TODO: We use x264's “speedcontrol” patch if available, so that quality is // automatically scaled up or down to content and available CPU time. // // The encoding threads are niced down because mixing is more important than // encoding; if we lose frames in mixing, we'll lose frames to disk _and_ // to the stream, as where if we lose frames in encoding, we'll lose frames // to the stream only, so the latter is strictly better. More importantly, // this allows speedcontrol (when implemented) to do its thing without // disturbing the mixer. #ifndef _X264ENCODE_H #define _X264ENCODE_H 1 #include #include #include #include #include #include #include #include #include #include extern "C" { #include } class Mux; class X264SpeedControl; class X264Encoder { public: X264Encoder(AVOutputFormat *oformat); // Does not take ownership. // Called after the last frame. Will block; once this returns, // the last data is flushed. ~X264Encoder(); // Must be called before first frame. Does not take ownership. void set_mux(Mux *mux) { this->mux = mux; } // is taken to be raw NV12 data of WIDTHxHEIGHT resolution. // Does not block. void add_frame(int64_t pts, int64_t duration, const uint8_t *data); std::string get_global_headers() const { while (!x264_init_done) { sched_yield(); } return global_headers; } void change_bitrate(unsigned rate_kbit) { new_bitrate_kbit = rate_kbit; } private: struct QueuedFrame { int64_t pts, duration; uint8_t *data; }; void encoder_thread_func(); void init_x264(); void encode_frame(QueuedFrame qf); // One big memory chunk of all 50 (or whatever) frames, allocated in // the constructor. All data functions just use pointers into this // pool. std::unique_ptr frame_pool; Mux *mux = nullptr; bool wants_global_headers; std::string global_headers; std::string buffered_sei; // Will be output before first frame, if any. std::thread encoder_thread; std::atomic x264_init_done{false}; std::atomic should_quit{false}; x264_t *x264; std::unique_ptr speed_control; std::atomic new_bitrate_kbit{0}; // 0 for no change. // Protects everything below it. std::mutex mu; // Frames that are not being encoded or waiting to be encoded, // so that add_frame() can use new ones. std::queue free_frames; // Frames that are waiting to be encoded (ie., add_frame() has been // called, but they are not picked up for encoding yet). std::queue queued_frames; // Whenever the state of changes. std::condition_variable queued_frames_nonempty; }; #endif // !defined(_X264ENCODE_H) nageru-1.4.2/x264_speed_control.cpp000066400000000000000000000301621301561706600171350ustar00rootroot00000000000000#include "x264_speed_control.h" #include #include #include #include #include #include #include #include #include "flags.h" using namespace std; using namespace std::chrono; X264SpeedControl::X264SpeedControl(x264_t *x264, float f_speed, int i_buffer_size, float f_buffer_init) : x264(x264), f_speed(f_speed) { x264_param_t param; x264_encoder_parameters(x264, ¶m); float fps = (float)param.i_fps_num / param.i_fps_den; uspf = 1e6 / fps; set_buffer_size(i_buffer_size); buffer_fill = buffer_size * f_buffer_init; buffer_fill = max(buffer_fill, uspf); buffer_fill = min(buffer_fill, buffer_size); timestamp = steady_clock::now(); preset = -1; cplx_num = 3e3; //FIXME estimate initial complexity cplx_den = .1; stat.min_buffer = buffer_size; stat.max_buffer = 0; stat.avg_preset = 0.0; stat.den = 0; } X264SpeedControl::~X264SpeedControl() { fprintf(stderr, "speedcontrol: avg preset=%.3f buffer min=%.3f max=%.3f\n", stat.avg_preset / stat.den, (float)stat.min_buffer / buffer_size, (float)stat.max_buffer / buffer_size ); // x264_log( x264, X264_LOG_INFO, "speedcontrol: avg cplx=%.5f\n", cplx_num / cplx_den ); } typedef struct { float time; // relative encoding time, compared to the other presets int subme; int me; int refs; int mix; int trellis; int partitions; int badapt; int bframes; int direct; int merange; } sc_preset_t; // The actual presets, including the equivalent commandline options. Note that // all presets are benchmarked with --weightp 1 --mbtree --rc-lookahead 20 // on top of the given settings (equivalent settings to the "faster" preset). // Timings and SSIM measurements were done on a quadcore Haswell i5 3.2 GHz // on the first 1000 frames of "Tears of Steel" in 1080p. // // Note that the two first and the two last are also used for extrapolation // should the desired time be outside the range. Thus, it is disadvantageous if // they are chosen so that the timings are too close to each other. #define SC_PRESETS 26 static const sc_preset_t presets[SC_PRESETS] = { #define I4 X264_ANALYSE_I4x4 #define I8 X264_ANALYSE_I8x8 #define P4 X264_ANALYSE_PSUB8x8 #define P8 X264_ANALYSE_PSUB16x16 #define B8 X264_ANALYSE_BSUB16x16 // Preset 0: 14.179db, --preset superfast --b-adapt 0 --bframes 0 { .time= 1.000, .subme=1, .me=X264_ME_DIA, .refs=1, .mix=0, .trellis=0, .partitions=I8|I4, .badapt=0, .bframes=0, .direct=0, .merange=16 }, // Preset 1: 14.459db, --preset superfast { .time= 1.283, .subme=1, .me=X264_ME_DIA, .refs=1, .mix=0, .trellis=0, .partitions=I8|I4, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 2: 14.761db, --preset superfast --subme 2 { .time= 1.603, .subme=2, .me=X264_ME_DIA, .refs=1, .mix=0, .trellis=0, .partitions=I8|I4, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 3: 15.543db, --preset veryfast { .time= 1.843, .subme=2, .me=X264_ME_HEX, .refs=1, .mix=0, .trellis=0, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 4: 15.716db, --preset veryfast --subme 3 { .time= 2.452, .subme=3, .me=X264_ME_HEX, .refs=1, .mix=0, .trellis=0, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 5: 15.786db, --preset veryfast --subme 3 --ref 2 { .time= 2.733, .subme=3, .me=X264_ME_HEX, .refs=2, .mix=0, .trellis=0, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 6: 15.813db, --preset veryfast --subme 4 --ref 2 { .time= 3.085, .subme=4, .me=X264_ME_HEX, .refs=2, .mix=0, .trellis=0, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 7: 15.849db, --preset faster { .time= 3.101, .subme=4, .me=X264_ME_HEX, .refs=2, .mix=0, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 8: 15.857db, --preset faster --mixed-refs { .time= 3.284, .subme=4, .me=X264_ME_HEX, .refs=2, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 9: 15.869db, --preset faster --mixed-refs --subme 5 { .time= 3.587, .subme=5, .me=X264_ME_HEX, .refs=2, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 10: 16.051db, --preset fast { .time= 3.947, .subme=6, .me=X264_ME_HEX, .refs=2, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 11: 16.356db, --preset fast --subme 7 { .time= 4.041, .subme=7, .me=X264_ME_HEX, .refs=2, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 12: 16.418db, --preset fast --subme 7 --ref 3 { .time= 4.406, .subme=7, .me=X264_ME_HEX, .refs=3, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 13: 16.460db, --preset medium { .time= 4.707, .subme=7, .me=X264_ME_HEX, .refs=3, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 14: 16.517db, --preset medium --subme 8 { .time= 5.133, .subme=8, .me=X264_ME_HEX, .refs=3, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 15: 16.523db, --preset medium --subme 8 --me umh { .time= 6.050, .subme=8, .me=X264_ME_UMH, .refs=3, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=1, .bframes=3, .direct=1, .merange=16 }, // Preset 16: 16.543db, --preset medium --subme 8 --me umh --direct auto --b-adapt 2 { .time= 6.849, .subme=8, .me=X264_ME_UMH, .refs=3, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 17: 16.613db, --preset slow { .time= 8.042, .subme=8, .me=X264_ME_UMH, .refs=5, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 18: 16.641db, --preset slow --subme 9 { .time= 8.972, .subme=9, .me=X264_ME_UMH, .refs=5, .mix=1, .trellis=1, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 19: 16.895db, --preset slow --subme 9 --trellis 2 { .time=10.073, .subme=9, .me=X264_ME_UMH, .refs=5, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 20: 16.918db, --preset slow --subme 9 --trellis 2 --ref 6 { .time=11.147, .subme=9, .me=X264_ME_UMH, .refs=6, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 21: 16.934db, --preset slow --subme 9 --trellis 2 --ref 7 { .time=12.267, .subme=9, .me=X264_ME_UMH, .refs=7, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 22: 16.948db, --preset slower { .time=13.829, .subme=9, .me=X264_ME_UMH, .refs=8, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8|P4, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 23: 17.058db, --preset slower --subme 10 { .time=14.831, .subme=10, .me=X264_ME_UMH, .refs=8, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8|P4, .badapt=2, .bframes=3, .direct=3, .merange=16 }, // Preset 24: 17.268db, --preset slower --subme 10 --bframes 8 { .time=18.705, .subme=10, .me=X264_ME_UMH, .refs=8, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8|P4, .badapt=2, .bframes=8, .direct=3, .merange=16 }, // Preset 25: 17.297db, --preset veryslow { .time=31.419, .subme=10, .me=X264_ME_UMH, .refs=16, .mix=1, .trellis=2, .partitions=I8|I4|P8|B8|P4, .badapt=2, .bframes=8, .direct=3, .merange=24 }, #undef I4 #undef I8 #undef P4 #undef P8 #undef B8 }; void X264SpeedControl::before_frame(float new_buffer_fill, int new_buffer_size, float new_uspf) { if (new_uspf > 0.0) { uspf = new_uspf; } if (new_buffer_size) { set_buffer_size(new_buffer_size); } buffer_fill = buffer_size * new_buffer_fill; steady_clock::time_point t; // update buffer state after encoding and outputting the previous frame(s) if (first) { t = timestamp = steady_clock::now(); first = false; } else { t = steady_clock::now(); } auto delta_t = t - timestamp; timestamp = t; // update the time predictor if (preset >= 0) { int cpu_time = duration_cast(cpu_time_last_frame).count(); cplx_num *= cplx_decay; cplx_den *= cplx_decay; cplx_num += cpu_time / presets[preset].time; ++cplx_den; stat.avg_preset += preset; ++stat.den; } stat.min_buffer = min(buffer_fill, stat.min_buffer); stat.max_buffer = max(buffer_fill, stat.max_buffer); if (buffer_fill >= buffer_size) { // oops, cpu was idle // not really an error, but we'll warn for debugging purposes static int64_t idle_t = 0; static steady_clock::time_point print_interval; static bool first = false; idle_t += buffer_fill - buffer_size; if (first || duration(t - print_interval).count() > 0.1) { //fprintf(stderr, "speedcontrol idle (%.6f sec)\n", idle_t/1e6); print_interval = t; idle_t = 0; first = false; } buffer_fill = buffer_size; } else if (buffer_fill <= 0) { // oops, we're late // fprintf(stderr, "speedcontrol underflow (%.6f sec)\n", buffer_fill/1e6); } { // Pick the preset that should return the buffer to 3/4-full within a time // specified by compensation_period. // // NOTE: This doesn't actually do that, at least assuming the same target is // chosen for every frame; exactly what it does is unclear to me. It seems // to consistently undershoot a bit, so it needs to be saved by the second // predictor below. However, fixing the formula seems to yield somewhat less // stable results in practice; in particular, once the buffer is half-full // or so, it would give us a negative target. Perhaps increasing // compensation_period would be a good idea, but initial (very brief) tests // did not yield good results. float target = uspf / f_speed * (buffer_fill + compensation_period) / (buffer_size*3/4 + compensation_period); float cplx = cplx_num / cplx_den; float set, t0, t1; float filled = (float) buffer_fill / buffer_size; int i; t0 = presets[0].time * cplx; for (i = 1; ; i++) { t1 = presets[i].time * cplx; if (t1 >= target || i == SC_PRESETS - 1) break; t0 = t1; } // exponential interpolation between states set = i-1 + (log(target) - log(t0)) / (log(t1) - log(t0)); set = max(set, -5); set = min(set, (SC_PRESETS-1) + 5); // Even if our time estimations in the SC_PRESETS array are off // this will push us towards our target fullness float s1 = set; set += (40 * (filled-0.75)); float s2 = (40 * (filled-0.75)); set = min(max(set, 0), SC_PRESETS - 1); apply_preset(dither_preset(set)); if (global_flags.x264_speedcontrol_verbose) { static float cpu, wall, tgt, den; const float decay = 1-1/100.; cpu = cpu*decay + duration_cast(cpu_time_last_frame).count(); wall = wall*decay + duration_cast(delta_t).count(); tgt = tgt*decay + target; den = den*decay + 1; fprintf(stderr, "speed: %.2f+%.2f %d[%.5f] (t/c/w: %6.0f/%6.0f/%6.0f = %.4f) fps=%.2f\r", s1, s2, preset, (float)buffer_fill / buffer_size, tgt/den, cpu/den, wall/den, cpu/wall, 1e6*den/wall ); } } } void X264SpeedControl::after_frame() { cpu_time_last_frame = steady_clock::now() - timestamp; } void X264SpeedControl::set_buffer_size(int new_buffer_size) { new_buffer_size = max(3, new_buffer_size); buffer_size = new_buffer_size * uspf; cplx_decay = 1 - 1./new_buffer_size; compensation_period = buffer_size/4; } int X264SpeedControl::dither_preset(float f) { int i = f; if (f < 0) { i--; } dither += f - i; if (dither >= 1.0) { dither--; i++; } return i; } void X264SpeedControl::apply_preset(int new_preset) { new_preset = max(new_preset, 0); new_preset = min(new_preset, SC_PRESETS - 1); const sc_preset_t *s = &presets[new_preset]; x264_param_t p; x264_encoder_parameters(x264, &p); p.i_frame_reference = s->refs; p.i_bframe_adaptive = s->badapt; p.i_bframe = s->bframes; p.analyse.inter = s->partitions; p.analyse.i_subpel_refine = s->subme; p.analyse.i_me_method = s->me; p.analyse.i_trellis = s->trellis; p.analyse.b_mixed_references = s->mix; p.analyse.i_direct_mv_pred = s->direct; p.analyse.i_me_range = s->merange; if (override_func) { override_func(&p); } x264_encoder_reconfig(x264, &p); preset = new_preset; } nageru-1.4.2/x264_speed_control.h000066400000000000000000000141621301561706600166040ustar00rootroot00000000000000// The x264 speed control tries to encode video at maximum possible quality // without skipping frames (at the expense of higher encoding latency and // less even output rates, although VBV is still respected). It does this // by continuously (every frame) changing the x264 quality settings such that // it uses maximum amount of CPU, but no more. // // Speed control works by maintaining a queue of frames, with the confusing // nomenclature “full” meaning that there are no queues in the frame. // (Conversely, if the queue is “empty” and a new frame comes in, we need to // drop that frame.) It tries to keep the buffer 3/4 “full” by using a table // of measured relative speeds for the different presets, and choosing one that it // thinks will return the buffer to that state over time. However, since // different frames take different times to encode regardless of preset, it // also tries to maintain a running average of how long the typical frame will // take to encode at the fastest preset (the so-called “complexity”), by dividing // the actual time by the relative time for the preset used. // // Frame timings is a complex topic in its own sright, since usually, multiple // frames are encoded in parallel. X264SpeedControl only supports the timing // method that the original patch calls “alternate timing”; one simply measures // the time the last x264_encoder_encode() call took. (The other alternative given // is to measure the time between successive x264_encoder_encode() calls.) // Unless using the zerocopy presets (which activate slice threading), the function // actually returns not when the given frame is done encoding, but when one a few // frames back is done encoding. So it doesn't actually measure the time of any // given one frame, but it measures something correlated to it, at least as long as // you are near 100% CPU utilization (ie., the encoded frame doesn't linger in the // buffers already when x264_encoder_encode() is called). // // The code has a long history; it was originally part of Avail Media's x264 // branch, used in their encoder appliances, and then a snapshot of that was // released. (Given that x264 is licensed under GPLv2 or newer, this means that // we can also treat the patch as GPLv2 or newer if we want, which we do. // As far as I know, it is copyright Avail Media, although no specific copyright // notice was posted on the patch.) // // From there, it was incorporated in OBE's x264 tree (x264-obe) and some bugs // were fixed. I started working on it for the purposes of Nageru, fixing various // issues, adding VFR support and redoing the timings entirely based on more // modern presets (the patch was made before several important x264 features, // such as weighted P-frames). Finally, I took it out of x264 and put it into // Nageru (it does not actually use any hooks into the codec itself), so that // one does not need to patch x264 to use it in Nageru. It still could do with // some cleanup, but it's much, much better than just using a static preset. #include #include #include extern "C" { #include "x264.h" } class X264SpeedControl { public: // x264: Encoding object we are using; must be opened. Assumed to be // set to the "faster" preset, and with 16 reference frames. // f_speed: Relative encoding speed, usually 1.0. // i_buffer_size: Number of frames in the buffer. // f_buffer_init: Relative fullness of buffer at start // (0.0 = assumed to be frames in buffer, // 1.0 = no frames in buffer) X264SpeedControl(x264_t *x264, float f_speed, int i_buffer_size, float f_buffer_init); ~X264SpeedControl(); // You need to call before_frame() immediately before each call to // x264_encoder_encode(), and after_frame() immediately after. // // new_buffer_fill: Buffer fullness, in microseconds (_not_ a relative // number, unlike f_buffer_init in the constructor). // new_buffer_size: If > 0, new number of frames in the buffer, // ie. the buffer size has changed. (It is harmless to set this // even if the buffer hasn't actually changed.) // f_uspf: If > 0, new microseconds per frame, ie. the frame rate has // changed. (Of course, with VFR, it can be impossible to truly know // the frame rate of the coming frames, but it is a reasonable // assumption that the next second or so is likely to be the same // frame rate as the last frame.) void before_frame(float new_buffer_fill, int new_buffer_size, float f_uspf); void after_frame(); // x264 seemingly has an issue where x264_encoder_reconfig() is not reflected // immediately in x264_encoder_parameters(). Since speed control keeps calling // those two all the time, any changes you make outside X264SpeedControl // could be overridden. Thus, to make changes to encoder parameters, you should // instead set a function here, which will be called every time parameters // are modified. void set_config_override_function(std::function override_func) { this->override_func = override_func; } private: void set_buffer_size(int new_buffer_size); int dither_preset(float f); void apply_preset(int new_preset); // Not owned by us. x264_t *x264; float f_speed; // all times that are not std::chrono::* are in usec std::chrono::steady_clock::time_point timestamp; // when was speedcontrol last invoked std::chrono::steady_clock::duration cpu_time_last_frame{std::chrono::seconds{0}}; // time spent encoding the previous frame int64_t buffer_size; // assumed application-side buffer of frames to be streamed (measured in microseconds), int64_t buffer_fill; // where full = we don't have to hurry int64_t compensation_period; // how quickly we try to return to the target buffer fullness float uspf; // microseconds per frame int preset = -1; // which setting was used in the previous frame float cplx_num = 3e3; // rolling average of estimated spf for preset #0. FIXME estimate initial complexity float cplx_den = .1; float cplx_decay; float dither = 0.0f; bool first = true; struct { int64_t min_buffer, max_buffer; double avg_preset; int den; } stat; std::function override_func = nullptr; };