diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..94a9ed0 --- /dev/null +++ b/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/README.md b/README.md index da00e35..b47e10c 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,10 @@ -# Searx package for YunoHost +Searx for YunoHost +================== + +[Yunohost project](https://yunohost.org/#/) Searx is a privacy-respecting, hackable metasearch engine. For more information ,visit [Searx homepage](https://asciimoo.github.io/searx/). This package is the [YunoHost](https://yunohost.org) application that ease -Searx installation on your YunoHost server. \ No newline at end of file +Searx installation on your YunoHost server. diff --git a/check_process b/check_process index bec608b..1e20307 100644 --- a/check_process +++ b/check_process @@ -3,7 +3,7 @@ ; Manifest domain="domain.tld" (DOMAIN) path="/path" (PATH) - is_public="Yes" (PUBLIC|public=Yes|private=No) + is_public=1 (PUBLIC|public=1|private=0) ; Checks pkg_linter=1 setup_sub_dir=1 @@ -14,13 +14,8 @@ upgrade=1 backup_restore=1 multi_instance=0 - wrong_user=1 - wrong_path=1 incorrect_path=1 - corrupt_source=0 - fail_download_source=0 port_already_use=0 - final_path_already_use=0 ;;; Levels Level 1=auto Level 2=auto diff --git a/conf/app.src b/conf/app.src new file mode 100644 index 0000000..12382fb --- /dev/null +++ b/conf/app.src @@ -0,0 +1,3 @@ +SOURCE_URL=https://github.com/asciimoo/searx/archive/v0.11.0.tar.gz +SOURCE_SUM=dd6e595e34c3133fe27fa619725df78a +ARCH_FORMAT=tar.gz \ No newline at end of file diff --git a/conf/nginx.conf b/conf/nginx.conf index 286b4d0..303ba80 100644 --- a/conf/nginx.conf +++ b/conf/nginx.conf @@ -1,17 +1,18 @@ -location YNH_WWW_LOCATION { - if ($scheme = http) { - rewrite ^ https://$server_name$request_uri? permanent; - } - try_files $uri @searx; +#noroot location = __PATH__ { rewrite ^ __PATH__/; } + +location __PATH__ { + if ($scheme = http) { + rewrite ^ https://$server_name$request_uri? permanent; + } + try_files $uri @searx; } location @searx { - uwsgi_param SCRIPT_NAME ''; - include uwsgi_params; - uwsgi_modifier1 30; - uwsgi_pass unix:///run/uwsgi/app/searx/socket; + uwsgi_param SCRIPT_NAME '__PATH_NO_ROOT__'; + include uwsgi_params; + uwsgi_modifier1 30; + uwsgi_pass unix:///run/uwsgi/app/searx/socket; - # Include SSOWAT user panel. - include conf.d/yunohost_panel.conf.inc; + # Include SSOWAT user panel. + include conf.d/yunohost_panel.conf.inc; } - diff --git a/conf/nginx.conf-noroot b/conf/nginx.conf-noroot deleted file mode 100644 index 1ebc2b6..0000000 --- a/conf/nginx.conf-noroot +++ /dev/null @@ -1,18 +0,0 @@ -location = YNH_WWW_LOCATION { rewrite ^ YNH_WWW_LOCATION/; } - -location YNH_WWW_LOCATION { - if ($scheme = http) { - rewrite ^ https://$server_name$request_uri? permanent; - } - try_files $uri @searx; -} - -location @searx { - uwsgi_param SCRIPT_NAME 'YNH_WWW_LOCATION'; - include uwsgi_params; - uwsgi_modifier1 30; - uwsgi_pass unix:///run/uwsgi/app/searx/socket; - - # Include SSOWAT user panel. - include conf.d/yunohost_panel.conf.inc; -} diff --git a/conf/searx.ini b/conf/searx.ini index 2910d02..8329d22 100644 --- a/conf/searx.ini +++ b/conf/searx.ini @@ -1,7 +1,7 @@ [uwsgi] # Who will run the code -uid = searx -gid = searx +uid = __APP__ +gid = __APP__ # Number of workers workers = 4 @@ -15,15 +15,15 @@ master = true plugin = python # Application base folder -base = /opt/yunohost/searx +base = __FINALPATH__ # Module to import -module = searx.webapp +module = __APP__.webapp # Virtualenv and python path -virtualenv = /opt/yunohost/searx/ -pythonpath = /opt/yunohost/searx/ -chdir = /opt/yunohost/searx/searx/ +virtualenv = __FINALPATH__ +pythonpath = __FINALPATH__ +chdir = __FINALPATH__/searx/ # The variable holding flask application callable = app diff --git a/conf/settings.yml b/conf/settings.yml index 651a30b..015f937 100644 --- a/conf/settings.yml +++ b/conf/settings.yml @@ -1,29 +1,39 @@ general: debug : False # Debug mode, only for development - instance_name : Searx + instance_name : "YunoSearx" # displayed name search: safe_search : 0 # Filter results. 0: None, 1: Moderate, 2: Strict autocomplete : "" # Existing autocomplete backends: "dbpedia", "duckduckgo", "google", "startpage", "wikipedia" - leave blank to turn it off by default + language : "all" server: port : 8888 bind_address : "127.0.0.1" # address to listen on - secret_key : "ultrasecretkey" # change this! - base_url : False # Set custom base_url. Possible values: False or "https://your.custom.host/location/" + secret_key : "__SECRETKEY__" # change this! !!! + base_url : __BASEURL__ # Set custom base_url. Possible values: False or "https://your.custom.host/location/" image_proxy : False # Proxying image results through searx + http_protocol_version : "1.0" # 1.0 and 1.1 are supported ui: themes_path : "" # Custom ui themes path - leave it blank if you didn't change default_theme : oscar # ui theme default_locale : "" # Default interface locale - leave blank to detect from browser information or use codes from the 'locales' config section +# searx supports result proxification using an external service: https://github.com/asciimoo/morty +# uncomment below section if you have running morty proxy +#result_proxy: +# url : http://127.0.0.1:3000/ +# key : your_morty_proxy_key + outgoing: # communication with search engines request_timeout : 2.0 # seconds useragent_suffix : "" # suffix of searx_useragent, could contain informations like an email address to the administrator + pool_connections : 100 # Number of different hosts + pool_maxsize : 10 # Number of simultaneous requests by host # uncomment below section if you want to use a proxy # see http://docs.python-requests.org/en/latest/user/advanced/#proxies -# SOCKS proxies are not supported : see https://github.com/kennethreitz/requests/pull/478 +# SOCKS proxies are also supported: see http://docs.python-requests.org/en/master/user/advanced/#socks # proxies : # http : http://127.0.0.1:8080 # https: http://127.0.0.1:8080 @@ -34,11 +44,29 @@ outgoing: # communication with search engines # - 1.1.1.2 engines: + - name : arch linux wiki + engine : archlinux + shortcut : al + + - name : archive is + engine : xpath + search_url : https://archive.is/{query} + url_xpath : (//div[@class="TEXT-BLOCK"]/a)/@href + title_xpath : (//div[@class="TEXT-BLOCK"]/a) + content_xpath : //div[@class="TEXT-BLOCK"]/ul/li + categories : general + timeout : 7.0 + disabled : True + shortcut : ai + + - name : base + engine : base + shortcut : bs + - name : wikipedia engine : mediawiki shortcut : wp base_url : 'https://{language}.wikipedia.org/' - number_of_results : 1 - name : bing engine : bing @@ -52,9 +80,38 @@ engines: engine : bing_news shortcut : bin - - name : btdigg - engine : btdigg - shortcut : bt + - name : bitbucket + engine : xpath + paging : True + search_url : https://bitbucket.org/repo/all/{pageno}?name={query} + url_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]/@href + title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"] + content_xpath : //article[@class="repo-summary"]/p + categories : it + timeout : 4.0 + disabled : True + shortcut : bb + + - name : ccc-tv + engine : xpath + paging : False + search_url : https://media.ccc.de/search/?q={query} + url_xpath : //div[@class="caption"]/h3/a/@href + title_xpath : //div[@class="caption"]/h3/a/text() + content_xpath : //div[@class="caption"]/h4/@title + categories : videos + disabled : True + shortcut : c3tv + + - name : crossref + engine : json_engine + paging : True + search_url : http://search.crossref.org/dois?q={query}&page={pageno} + url_query : doi + title_query : title + content_query : fullCitation + categories : science + shortcut : cr - name : currency engine : currency_convert @@ -73,18 +130,53 @@ engines: - name : ddg definitions engine : duckduckgo_definitions shortcut : ddd + weight : 2 + disabled : True + + - name : digbt + engine : digbt + shortcut : dbt + timeout : 6.0 + disabled : True - name : digg engine : digg shortcut : dg + - name : erowid + engine : xpath + paging : True + first_page_num : 0 + page_size : 30 + search_url : https://www.erowid.org/search.php?q={query}&s={pageno} + url_xpath : //dl[@class="results-list"]/dt[@class="result-title"]/a/@href + title_xpath : //dl[@class="results-list"]/dt[@class="result-title"]/a/text() + content_xpath : //dl[@class="results-list"]/dd[@class="result-details"] + categories : general + shortcut : ew + disabled : True + - name : wikidata engine : wikidata shortcut : wd + weight : 2 - name : duckduckgo engine : duckduckgo shortcut : ddg + disabled : True + + - name : etymonline + engine : xpath + paging : True + search_url : http://etymonline.com/?search={query}&p={pageno} + url_xpath : //dt/a[1]/@href + title_xpath : //dt + content_xpath : //dd + suggestion_xpath : //a[@class="crossreference"] + first_page_num : 0 + shortcut : et + disabled : True # api-key required: http://www.faroo.com/hp/api/api.html#key # - name : faroo @@ -101,6 +193,11 @@ engines: shortcut : 1x disabled : True + - name : fdroid + engine : fdroid + shortcut : fd + disabled : True + - name : flickr categories : images shortcut : fl @@ -111,11 +208,29 @@ engines: # Or you can use the html non-stable engine, activated by default engine : flickr_noapi + - name : frinkiac + engine : frinkiac + shortcut : frk + disabled : True + - name : gigablast engine : gigablast shortcut : gb + timeout : 3.0 disabled: True + - name : gitlab + engine : xpath + paging : True + search_url : https://gitlab.com/search?page={pageno}&search={query} + url_xpath : //li[@class="project-row"]//a[@class="project"]/@href + title_xpath : //li[@class="project-row"]//span[contains(@class, "project-full-name")] + content_xpath : //li[@class="project-row"]//div[@class="description"]/p + categories : it + shortcut : gl + timeout : 5.0 + disabled : True + - name : github engine : github shortcut : gh @@ -132,6 +247,20 @@ engines: engine : google_news shortcut : gon + - name : google scholar + engine : xpath + paging : True + search_url : https://scholar.google.com/scholar?start={pageno}&q={query}&hl=en&as_sdt=0,5&as_vis=1 + results_xpath : //div[@class="gs_r"]/div[@class="gs_ri"] + url_xpath : .//h3/a/@href + title_xpath : .//h3/a + content_xpath : .//div[@class="gs_rs"] + suggestion_xpath : //div[@id="gs_qsuggest"]/ul/li + page_size : 10 + first_page_num : 0 + categories : science + shortcut : gos + - name : google play apps engine : xpath search_url : https://play.google.com/store/search?q={query}&c=apps @@ -146,8 +275,8 @@ engines: engine : xpath search_url : https://play.google.com/store/search?q={query}&c=movies url_xpath : //a[@class="title"]/@href - title_xpath : //a[@class="title"] - content_xpath : //a[@class="subtitle"] + title_xpath : //a[@class="title"]/@title + content_xpath : //a[contains(@class, "subtitle")] categories : videos shortcut : gpm disabled : True @@ -162,14 +291,109 @@ engines: shortcut : gps disabled : True + - name : geektimes + engine : xpath + paging : True + search_url : https://geektimes.ru/search/page{pageno}/?q={query} + url_xpath : //div[@class="search_results"]//a[@class="post__title_link"]/@href + title_xpath : //div[@class="search_results"]//a[@class="post__title_link"] + content_xpath : //div[@class="search_results"]//div[contains(@class, "content")] + categories : it + timeout : 4.0 + disabled : True + shortcut : gt + + - name : habrahabr + engine : xpath + paging : True + search_url : https://habrahabr.ru/search/page{pageno}/?q={query} + url_xpath : //div[@class="search_results"]//a[contains(@class, "post__title_link")]/@href + title_xpath : //div[@class="search_results"]//a[contains(@class, "post__title_link")] + content_xpath : //div[@class="search_results"]//div[contains(@class, "content")] + categories : it + timeout : 4.0 + disabled : True + shortcut : habr + + - name : hoogle + engine : json_engine + paging : True + search_url : https://www.haskell.org/hoogle/?mode=json&hoogle={query}&start={pageno} + results_query : results + url_query : location + title_query : self + content_query : docs + page_size : 20 + categories : it + shortcut : ho + + - name : ina + engine : ina + shortcut : in + timeout : 6.0 + disabled : True + + - name: kickass + engine : kickass + shortcut : kc + timeout : 4.0 + disabled : True + + - name : lobste.rs + engine : xpath + search_url : https://lobste.rs/search?utf8=%E2%9C%93&q={query}&what=stories&order=relevance + results_xpath : //li[contains(@class, "story")] + url_xpath : .//span[@class="link"]/a/@href + title_xpath : .//span[@class="link"]/a + content_xpath : .//a[@class="domain"] + categories : it + shortcut : lo + + - name : microsoft academic + engine : json_engine + paging : True + search_url : https://academic.microsoft.com/api/search/GetEntityResults?query=%40{query}%40&filters=&offset={pageno}&limit=8&correlationId=undefined + results_query : results + url_query : u + title_query : dn + content_query : d + page_size : 8 + first_page_num : 0 + categories : science + shortcut : ma + - name : mixcloud engine : mixcloud shortcut : mc + - name : nyaa + engine : nyaa + shortcut : nt + disabled : True + - name : openstreetmap engine : openstreetmap shortcut : osm + - name : openrepos + engine : xpath + paging : True + search_url : https://openrepos.net/search/node/{query}?page={pageno} + url_xpath : //li[@class="search-result"]//h3[@class="title"]/a/@href + title_xpath : //li[@class="search-result"]//h3[@class="title"]/a + content_xpath : //li[@class="search-result"]//div[@class="search-snippet-info"]//p[@class="search-snippet"] + categories : files + timeout : 4.0 + disabled : True + shortcut : or + + - name : pdbe + engine : pdbe + shortcut : pdb +# Hide obsolete PDB entries. +# Default is not to hide obsolete structures +# hide_obsolete : False + - name : photon engine : photon shortcut : ph @@ -177,7 +401,8 @@ engines: - name : piratebay engine : piratebay shortcut : tpb - disabled : True + url: https://pirateproxy.red/ + timeout : 3.0 - name : qwant engine : qwant @@ -200,9 +425,17 @@ engines: shortcut : qws categories : social media - - name : kickass - engine : kickass - shortcut : ka + - name : reddit + engine : reddit + shortcut : re + page_size : 25 + timeout : 10.0 + disabled : True + + - name : scanr structures + shortcut: scs + engine : scanr_structures + disabled : True - name : soundcloud engine : soundcloud @@ -221,6 +454,14 @@ engines: shortcut : scc disabled : True +# - name : searx +# engine : searx_engine +# shortcut : se +# instance_urls : +# - http://127.0.0.1:8888/ +# - ... +# disabled : True + - name : spotify engine : spotify shortcut : stf @@ -240,8 +481,8 @@ engines: - name : ixquick engine : startpage - base_url : 'https://www.ixquick.com/' - search_url : 'https://www.ixquick.com/do/search' + base_url : 'https://www.ixquick.eu/' + search_url : 'https://www.ixquick.eu/do/search' shortcut : iq timeout : 6.0 disabled : True @@ -251,6 +492,12 @@ engines: shortcut : sw disabled : True + - name : tokyotoshokan + engine : tokyotoshokan + shortcut : tt + timeout : 6.0 + disabled : True + - name : twitter engine : twitter shortcut : tw @@ -266,9 +513,9 @@ engines: # - name : urbandictionary # engine : xpath # search_url : http://www.urbandictionary.com/define.php?term={query} -# url_xpath : //div[@class="word"]//a/@href -# title_xpath : //div[@class="word"]//a -# content_xpath : //div[@class="definition"] +# url_xpath : //*[@class="word"]/@href +# title_xpath : //*[@class="def-header"] +# content_xpath : //*[@class="meaning"] # shortcut : ud - name : yahoo @@ -301,6 +548,47 @@ engines: engine : vimeo shortcut : vm + - name : wolframalpha + shortcut : wa + # You can use the engine using the official stable API, but you need an API key + # See : http://products.wolframalpha.com/api/ + # engine : wolframalpha_api + # api_key: '' # required! + engine : wolframalpha_noapi + timeout: 6.0 + categories : science + + - name : seedpeer + engine : seedpeer + shortcut: speu + categories: files, music, videos + disabled: True + + - name : dictzone + engine : dictzone + shortcut : dc + + - name : mymemory translated + engine : translated + shortcut : tl + timeout : 5.0 + disabled : True + # You can use without an API key, but you are limited to 1000 words/day + # See : http://mymemory.translated.net/doc/usagelimits.php + # api_key : '' + + - name : voat + engine: xpath + shortcut: vo + categories: social media + search_url : https://voat.co/search?q={query} + url_xpath : //p[contains(@class, "title")]/a/@href + title_xpath : //p[contains(@class, "title")]/a + content_xpath : //span[@class="domain"] + timeout : 10.0 + disabled : True + + #The blekko technology and team have joined IBM Watson! -> https://blekko.com/ # - name : blekko images # engine : blekko_images @@ -314,18 +602,31 @@ engines: # number_of_results : 5 # timeout : 3.0 +# Doku engine lets you access to any Doku wiki instance: +# A public one or a privete/corporate one. +# - name : ubuntuwiki +# engine : doku +# shortcut : uw +# base_url : 'http://doc.ubuntu-fr.org' + locales: en : English - de : Deutsch - he : עברית - hu : Magyar - fr : Français - es : Español - it : Italiano - nl : Nederlands + bg : Български (Bulgarian) + cs : Čeština (Czech) + de : Deutsch (German) + el_GR : Ελληνικά (Greek_Greece) + eo : Esperanto (Esperanto) + es : Español (Spanish) + fr : Français (French) + he : עברית (Hebrew) + hu : Magyar (Hungarian) + it : Italiano (Italian) ja : 日本語 (Japanese) - tr : Türkçe - pt : Português - ru : Russian - ro : Romanian + nl : Nederlands (Dutch) + pt : Português (Portuguese) + pt_BR : Português (Portuguese_Brazil) + ro : Română (Romanian) + ru : Русский (Russian) + sv : Svenska (Swedish) + tr : Türkçe (Turkish) zh : 中文 (Chinese) diff --git a/manifest.json b/manifest.json index baedef6..643c7dd 100644 --- a/manifest.json +++ b/manifest.json @@ -1,59 +1,60 @@ { - "name": "Searx", - "id": "searx", - "packaging_format": 1, - "requirements": { - "yunohost": ">> 2.3.15" - }, - "description": { - "en": "A privacy-respecting, hackable metasearch engine", - "fr": "Un méta-moteur de recherche respectueux de la vie privée et bidouillable", - "de": "Eine Meta-Suchmaschine, die den Privatsphäre wahrt und 'hackable' ist." - }, - "url": "https://asciimoo.github.io/searx/", - "maintainer": { - "name": "opi", - "email": "opi@zeropi.net" - }, - "multi_instance": "false", - "services": [ - "nginx", - "uwsgi" - ], - "arguments": { - "install" : [ - { - "name": "domain", - "type": "domain", - "ask": { - "en": "Choose a domain for Searx", - "fr": "Choisissez un domaine pour Searx", - "de": "Wählen einen Domain-Namen für Searx" - }, - "example": "domain.org" - }, - { - "name": "path", - "type": "path", - "ask": { - "en": "Choose a path for Searx", - "fr": "Choisissez un chemin pour Searx", - "de": "Wählen einen Pfad für Searx" - }, - "example": "/searx", - "default": "/searx" - }, - { - "name": "is_public", - "ask": { - "en": "Is it a public Searx site ?", - "fr": "Est-ce un site public ?", - "de": "Ist es eine öffentliche Applikation ?" - }, - "choices": ["Yes", "No"], - "default": "Yes" - } - - ] - } + "name": "Searx", + "id": "searx", + "packaging_format": 1, + "requirements": { + "yunohost": ">= 2.4" + }, + "description": { + "en": "A privacy-respecting, hackable metasearch engine", + "fr": "Un méta-moteur de recherche respectueux de la vie privée et bidouillable", + "de": "Eine Meta-Suchmaschine, die den Privatsphäre wahrt und 'hackable' ist." + }, + "version": "0.11", + "url": "https://asciimoo.github.io/searx/", + "license": "AGPLv3", + "maintainer": { + "name": "opi", + "email": "opi@zeropi.net" + }, + "multi_instance": false, + "services": [ + "nginx", + "uwsgi" + ], + "arguments": { + "install" : [ + { + "name": "domain", + "type": "domain", + "ask": { + "en": "Choose a domain for Searx", + "fr": "Choisissez un domaine pour Searx", + "de": "Wählen einen Domain-Namen für Searx" + }, + "example": "domain.org" + }, + { + "name": "path", + "type": "path", + "ask": { + "en": "Choose a path for Searx", + "fr": "Choisissez un chemin pour Searx", + "de": "Wählen einen Pfad für Searx" + }, + "example": "/searx", + "default": "/searx" + }, + { + "name": "is_public", + "type": "boolean", + "ask": { + "en": "Is it a public Searx site ?", + "fr": "Est-ce un site public ?", + "de": "Ist es eine öffentliche Applikation ?" + }, + "default": true + } + ] + } } diff --git a/scripts/_common.sh b/scripts/_common.sh new file mode 100755 index 0000000..26ee343 --- /dev/null +++ b/scripts/_common.sh @@ -0,0 +1,380 @@ +#!/bin/bash + +#================================================= +#================================================= +# TESTING +#================================================= +#================================================= + +ynh_setup_source () { + src_url=$(cat ../conf/app.src | grep SOURCE_URL | cut -d= -f2-) + src_checksum=$(cat ../conf/app.src | grep SOURCE_SUM | cut -d= -f2-) + arch_format=$(cat ../conf/app.src | grep ARCH_FORMAT | cut -d= -f2-) + local_source="/opt/yunohost-apps-src/$YNH_APP_ID/source.$arch_format" + + if test -e "$local_source" + then # Use the local source file if it is present + cp $local_source source.$arch_format + else # If not, download the source + wget -nv -O source.$arch_format $src_url + fi + + # Check the control sum + echo "$src_checksum source.$arch_format" \ + | md5sum -c --status || ynh_die "Corrupt source" + + # Extract source into the app dir + sudo mkdir -p "$final_path" + if [ $(echo "$arch_format" | tr '[:upper:]' '[:lower:]') = "zip" ] + then # Zip format + # Using of a temp directory, because unzip doesn't manage --strip-components + temp_dir=$(mktemp -d) + unzip -quo source.zip -d "$temp_dir" + sudo cp -a $temp_dir/*/. "$final_path" + ynh_secure_remove "$temp_dir" + elif [ $(echo "$arch_format" | tr '[:upper:]' '[:lower:]') = "tar.gz" ]; then + sudo tar -x -f source.tar.gz -C "$final_path" --strip-components 1 + else + ynh_die "Format d'archive non reconnu." + fi + + # Apply patches + if test -f ../sources/patches/*.patch; then + (cd "$DEST" \ + && for p in ${PKG_DIR}/patches/*.patch; do \ + sudo patch -p1 < $p; done) \ + || ynh_die "Unable to apply patches" + fi + + # Add supplementary files + if test -e "../sources/extra_files"; then + sudo cp -a ../sources/extra_files/. "$final_path" + fi +} + +ynh_nginx_config () { + finalnginxconf="/etc/nginx/conf.d/$domain.d/$app.conf" + ynh_compare_checksum_config "$finalnginxconf" 1 + sudo cp ../conf/nginx.conf "$finalnginxconf" + + # To avoid a break by set -u, use a void substitution ${var:-}. If the variable is not set, it's simply set with an empty variable. + # Substitute in a nginx config file only if the variable is not empty + if test -n "${path_url:-}"; then + ynh_replace_string "__PATH__" "$path_url" "$finalnginxconf" + fi + if test -n "${domain:-}"; then + ynh_replace_string "__DOMAIN__" "$domain" "$finalnginxconf" + fi + if test -n "${port:-}"; then + ynh_replace_string "__PORT__" "$port" "$finalnginxconf" + fi + if test -n "${app:-}"; then + ynh_replace_string "__NAME__" "$app" "$finalnginxconf" + fi + if test -n "${final_path:-}"; then + ynh_replace_string "__FINALPATH__" "$final_path" "$finalnginxconf" + fi + ynh_store_checksum_config "$finalnginxconf" + + sudo systemctl reload nginx +} + +ynh_remove_nginx_config () { + ynh_secure_remove "/etc/nginx/conf.d/$domain.d/$app.conf" + sudo systemctl reload nginx +} + +ynh_store_checksum_config () { + config_file_checksum=checksum_${1//[\/ ]/_} # Replace all '/' and ' ' by '_' + ynh_app_setting_set $app $config_file_checksum $(sudo md5sum "$1" | cut -d' ' -f1) +} + +ynh_compare_checksum_config () { + current_config_file=$1 + compress_backup=${2:-0} # If $2 is empty, compress_backup will set at 0 + config_file_checksum=checksum_${current_config_file//[\/ ]/_} # Replace all '/' and ' ' by '_' + checksum_value=$(ynh_app_setting_get $app $config_file_checksum) + if [ -n "$checksum_value" ] + then # Proceed only if a value was stocked into the app config + if ! echo "$checksum_value $current_config_file" | sudo md5sum -c --status + then # If the checksum is now different + backup_config_file="$current_config_file.backup.$(date '+%d.%m.%y_%Hh%M,%Ss')" + if [ $compress_backup -eq 1 ] + then + sudo tar --create --gzip --file "$backup_config_file.tar.gz" "$current_config_file" # Backup the current config file and compress + backup_config_file="$backup_config_file.tar.gz" + else + sudo cp -a "$current_config_file" "$backup_config_file" # Backup the current config file + fi + echo "Config file $current_config_file has been manually modified since the installation or last upgrade. So it has been duplicated in $backup_config_file" >&2 + echo "$backup_config_file" # Return the name of the backup file + fi + fi +} + +#================================================= +#================================================= + +#================================================= +# CHECKING +#================================================= + +CHECK_DOMAINPATH () { # Vérifie la disponibilité du path et du domaine. + sudo yunohost app checkurl $domain$path_url -a $app +} + +#================================================= +# DISPLAYING +#================================================= + +WARNING () { # Écrit sur le canal d'erreur pour passer en warning. + $@ >&2 +} + +QUIET () { # Redirige la sortie standard dans /dev/null + $@ > /dev/null +} + +#================================================= +# BACKUP +#================================================= + +BACKUP_FAIL_UPGRADE () { + WARNING echo "Upgrade failed." + app_bck=${app//_/-} # Replace all '_' by '-' + if sudo yunohost backup list | grep -q $app_bck-pre-upgrade$backup_number; then # Vérifie l'existence de l'archive avant de supprimer l'application et de restaurer + sudo yunohost app remove $app # Supprime l'application avant de la restaurer. + sudo yunohost backup restore --ignore-hooks $app_bck-pre-upgrade$backup_number --apps $app --force # Restore the backup if upgrade failed + ynh_die "The app was restored to the way it was before the failed upgrade." + fi +} + +BACKUP_BEFORE_UPGRADE () { # Backup the current version of the app, restore it if the upgrade fails + backup_number=1 + old_backup_number=2 + app_bck=${app//_/-} # Replace all '_' by '-' + if sudo yunohost backup list | grep -q $app_bck-pre-upgrade1; then # Vérifie l'existence d'une archive déjà numéroté à 1. + backup_number=2 # Et passe le numéro de l'archive à 2 + old_backup_number=1 + fi + + sudo yunohost backup create --ignore-hooks --apps $app --name $app_bck-pre-upgrade$backup_number # Créer un backup différent de celui existant. + if [ "$?" -eq 0 ]; then # Si le backup est un succès, supprime l'archive précédente. + if sudo yunohost backup list | grep -q $app_bck-pre-upgrade$old_backup_number; then # Vérifie l'existence de l'ancienne archive avant de la supprimer, pour éviter une erreur. + QUIET sudo yunohost backup delete $app_bck-pre-upgrade$old_backup_number + fi + else # Si le backup a échoué + ynh_die "Backup failed, the upgrade process was aborted." + fi +} + +HUMAN_SIZE () { # Transforme une taille en Ko en une taille lisible pour un humain + human=$(numfmt --to=iec --from-unit=1K $1) + echo $human +} + +CHECK_SIZE () { # Vérifie avant chaque backup que l'espace est suffisant + file_to_analyse=$1 + backup_size=$(sudo du --summarize "$file_to_analyse" | cut -f1) + free_space=$(sudo df --output=avail "/home/yunohost.backup" | sed 1d) + + if [ $free_space -le $backup_size ] + then + WARNING echo "Espace insuffisant pour sauvegarder $file_to_analyse." + WARNING echo "Espace disponible: $(HUMAN_SIZE $free_space)" + ynh_die "Espace nécessaire: $(HUMAN_SIZE $backup_size)" + fi +} + +#================================================= +#================================================= +# FUTUR YNH HELPERS +#================================================= +# Importer ce fichier de fonction avant celui des helpers officiel +# Ainsi, les officiels prendront le pas sur ceux-ci le cas échéant +#================================================= + +# Normalize the url path syntax +# Handle the slash at the beginning of path and its absence at ending +# Return a normalized url path +# +# example: url_path=$(ynh_normalize_url_path $url_path) +# ynh_normalize_url_path example -> /example +# ynh_normalize_url_path /example -> /example +# ynh_normalize_url_path /example/ -> /example +# ynh_normalize_url_path / -> / +# +# usage: ynh_normalize_url_path path_to_normalize +# | arg: url_path_to_normalize - URL path to normalize before using it +ynh_normalize_url_path () { + path_url=$1 + test -n "$path_url" || ynh_die "ynh_normalize_url_path expect a URL path as first argument and received nothing." + if [ "${path_url:0:1}" != "/" ]; then # If the first character is not a / + path_url="/$path_url" # Add / at begin of path variable + fi + if [ "${path_url:${#path_url}-1}" == "/" ] && [ ${#path_url} -gt 1 ]; then # If the last character is a / and that not the only character. + path_url="${path_url:0:${#path_url}-1}" # Delete the last character + fi + echo $path_url +} + +# Manage a fail of the script +# +# Print a warning to inform that the script was failed +# Execute the ynh_clean_setup function if used in the app script +# +# usage of ynh_clean_setup function +# This function provide a way to clean some residual of installation that not managed by remove script. +# To use it, simply add in your script: +# ynh_clean_setup () { +# instructions... +# } +# This function is optionnal. +# +# Usage: ynh_exit_properly is used only by the helper ynh_abort_if_errors. +# You must not use it directly. +ynh_exit_properly () { + exit_code=$? + if [ "$exit_code" -eq 0 ]; then + exit 0 # Exit without error if the script ended correctly + fi + + trap '' EXIT # Ignore new exit signals + set +eu # Do not exit anymore if a command fail or if a variable is empty + + echo -e "!!\n $app's script has encountered an error. Its execution was cancelled.\n!!" >&2 + + if type -t ynh_clean_setup > /dev/null; then # Check if the function exist in the app script. + ynh_clean_setup # Call the function to do specific cleaning for the app. + fi + + ynh_die # Exit with error status +} + +# Exit if an error occurs during the execution of the script. +# +# Stop immediatly the execution if an error occured or if a empty variable is used. +# The execution of the script is derivate to ynh_exit_properly function before exit. +# +# Usage: ynh_abort_if_errors +ynh_abort_if_errors () { + set -eu # Exit if a command fail, and if a variable is used unset. + trap ynh_exit_properly EXIT # Capturing exit signals on shell script +} + +# Define and install dependencies with a equivs control file +# This helper can/should only be called once per app +# +# usage: ynh_install_app_dependencies dep [dep [...]] +# | arg: dep - the package name to install in dependence +ynh_install_app_dependencies () { + dependencies=$@ + manifest_path="../manifest.json" + if [ ! -e "$manifest_path" ]; then + manifest_path="../settings/manifest.json" # Into the restore script, the manifest is not at the same place + fi + version=$(sudo python3 -c "import sys, json;print(json.load(open(\"$manifest_path\"))['version'])") # Retrieve the version number in the manifest file. + dep_app=${app//_/-} # Replace all '_' by '-' + + if ynh_package_is_installed "${dep_app}-ynh-deps"; then + echo "A package named ${dep_app}-ynh-deps is already installed" >&2 + else + cat > ./${dep_app}-ynh-deps.control << EOF # Make a control file for equivs-build +Section: misc +Priority: optional +Package: ${dep_app}-ynh-deps +Version: ${version} +Depends: ${dependencies// /, } +Architecture: all +Description: Fake package for ${app} (YunoHost app) dependencies + This meta-package is only responsible of installing its dependencies. +EOF + ynh_package_install_from_equivs ./${dep_app}-ynh-deps.control \ + || ynh_die "Unable to install dependencies" # Install the fake package and its dependencies + ynh_app_setting_set $app apt_dependencies $dependencies + fi +} + +# Remove fake package and its dependencies +# +# Dependencies will removed only if no other package need them. +# +# usage: ynh_remove_app_dependencies +ynh_remove_app_dependencies () { + dep_app=${app//_/-} # Replace all '_' by '-' + ynh_package_autoremove ${dep_app}-ynh-deps # Remove the fake package and its dependencies if they not still used. +} + +# Create a system user +# +# usage: ynh_system_user_create user_name [home_dir] +# | arg: user_name - Name of the system user that will be create +# | arg: home_dir - Path of the home dir for the user. Usually the final path of the app. If this argument is omitted, the user will be created without home +ynh_system_user_create () { + if ! ynh_system_user_exists "$1" # Check if the user exists on the system + then # If the user doesn't exist + if [ $# -ge 2 ]; then # If a home dir is mentioned + user_home_dir="-d $2" + else + user_home_dir="--no-create-home" + fi + sudo useradd $user_home_dir --system --user-group $1 --shell /usr/sbin/nologin || ynh_die "Unable to create $1 system account" + fi +} + +# Delete a system user +# +# usage: ynh_system_user_delete user_name +# | arg: user_name - Name of the system user that will be create +ynh_system_user_delete () { + if ynh_system_user_exists "$1" # Check if the user exists on the system + then + echo "Remove the user $1" >&2 + sudo userdel $1 + else + echo "The user $1 was not found" >&2 + fi +} + +# Substitute/replace a string by another in a file +# +# usage: ynh_replace_string match_string replace_string target_file +# | arg: match_string - String to be searched and replaced in the file +# | arg: replace_string - String that will replace matches +# | arg: target_file - File in which the string will be replaced. +ynh_replace_string () { + delimit=@ + match_string=${1//${delimit}/"\\${delimit}"} # Escape the delimiter if it's in the string. + replace_string=${2//${delimit}/"\\${delimit}"} + workfile=$3 + + sudo sed --in-place "s${delimit}${match_string}${delimit}${replace_string}${delimit}g" "$workfile" +} + +# Remove a file or a directory securely +# +# usage: ynh_secure_remove path_to_remove +# | arg: path_to_remove - File or directory to remove +ynh_secure_remove () { + path_to_remove=$1 + forbidden_path=" \ + /var/www \ + /home/yunohost.app" + + if [[ "$forbidden_path" =~ "$path_to_remove" \ + # Match all paths or subpaths in $forbidden_path + || "$path_to_remove" =~ ^/[[:alnum:]]+$ \ + # Match all first level paths from / (Like /var, /root, etc...) + || "${path_to_remove:${#path_to_remove}-1}" = "/" ]] + # Match if the path finishes by /. Because it seems there is an empty variable + then + echo "Avoid deleting $path_to_remove." >&2 + else + if [ -e "$path_to_remove" ] + then + sudo rm -R "$path_to_remove" + else + echo "$path_to_remove wasn't deleted because it doesn't exist." >&2 + fi + fi +} diff --git a/scripts/backup b/scripts/backup index d4e95d4..ff5e734 100755 --- a/scripts/backup +++ b/scripts/backup @@ -1,21 +1,55 @@ #!/bin/bash -app="searx" -# causes the shell to exit if any subcommand or pipeline returns a non-zero status -set -e +#================================================= +# GENERIC STARTING +#================================================= +# MANAGE FAILURE OF THE SCRIPT +#================================================= -# Source YNH helpers -. /usr/share/yunohost/helpers +# Exit on command errors and treat unset variables as an error +set -eu -# Retrieve arguments -domain=$(ynh_app_setting_get "$app" domain) -path=$(ynh_app_setting_get "$app" path) -is_public=$(ynh_app_setting_get "$app" is_public) +#================================================= +# IMPORT GENERIC HELPERS +#================================================= -# Backup sources & data -ynh_backup "/opt/yunohost/$app" "sources" +if [ ! -e _common.sh ]; then + # Rapatrie le fichier de fonctions si il n'est pas dans le dossier courant + sudo cp ../settings/scripts/_common.sh ./_common.sh + sudo chmod a+rx _common.sh +fi +source _common.sh +source /usr/share/yunohost/helpers -# Copy Nginx conf -ynh_backup "/etc/uwsgi/apps-available/$app.ini" "conf/searx.ini" -ynh_backup "/opt/yunohost/$app/searx/settings.yml" "conf/settings.yml" -ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "conf/nginx.conf" +#================================================= +# LOAD SETTINGS +#================================================= + +# Récupère les infos de l'application. +app=$YNH_APP_INSTANCE_NAME + +final_path=$(ynh_app_setting_get $app final_path) +domain=$(ynh_app_setting_get $app domain) + +#================================================= +# STANDARD BACKUP STEPS +#================================================= +# BACKUP OF THE MAIN DIR OF THE APP +#================================================= + +CHECK_SIZE "$final_path" +ynh_backup "$final_path" "sources" + +#================================================= +# BACKUP OF THE NGINX CONFIGURATION +#================================================= + +ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf" "nginx.conf" + +#================================================= +# SPECIFIC BACKUP +#================================================= +# BACKUP OF THE UWSGI CONFIG +#================================================= + +ynh_backup "/etc/uwsgi/apps-available/$app.ini" "uwsgi_conf" diff --git a/scripts/install b/scripts/install index 4caba1e..1175414 100644 --- a/scripts/install +++ b/scripts/install @@ -1,113 +1,147 @@ #!/bin/bash -app="searx" -# causes the shell to exit if any subcommand or pipeline returns a non-zero status -set -e +#================================================= +# GENERIC STARTING +#================================================= +# IMPORT GENERIC HELPERS +#================================================= -# Source app helpers -. /usr/share/yunohost/helpers +source _common.sh +source /usr/share/yunohost/helpers + +#================================================= +# MANAGE FAILURE OF THE SCRIPT +#================================================= + +ynh_abort_if_errors # Active trap pour arrêter le script si une erreur est détectée. + +#================================================= +# RETRIEVE ARGUMENTS FROM THE MANIFEST +#================================================= -# Retrieve arguments domain=$YNH_APP_ARG_DOMAIN -path=$YNH_APP_ARG_PATH +path_url=$YNH_APP_ARG_PATH is_public=$YNH_APP_ARG_IS_PUBLIC -# Force path to start with a / -if [ "${path:0:1}" != "/" ]; then - path="/$path" -fi +app=$YNH_APP_INSTANCE_NAME -# Remove trailing slash to path -path=${path%/} -#force location to be / or /foo -location=${path:-/} +#================================================= +# CHECK IF THE APP CAN BE INSTALLED WITH THIS ARGS +#================================================= -# Check domain/path availability -sudo yunohost app checkurl $domain$path -a $app \ - || (echo "Path not available: $domain$path" && exit 1) +path_url=$(ynh_normalize_url_path $path_url) # Vérifie et corrige la syntaxe du path. +CHECK_DOMAINPATH # Vérifie la disponibilité du path et du domaine. +# Check final_path +final_path="/opt/yunohost/$app" +test ! -e "$final_path" || ynh_die "This path already contains a folder" -# Save specific settings -sudo yunohost app setting searx is_public -v $is_public +#================================================= +# STORE SETTINGS FROM MANIFEST +#================================================= -# Check depends installation -sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev python-lxml uwsgi uwsgi-plugin-python -y +ynh_app_setting_set $app domain $domain +ynh_app_setting_set $app path $path_url +ynh_app_setting_set $app is_public $is_public -# Check Swap -if [ $(sudo swapon -s | wc -l) = 1 ]; +#================================================= +# STANDARD MODIFICATIONS +#================================================= +# INSTALL DEPENDENCIES +#================================================= + +ynh_install_app_dependencies git build-essential libxslt-dev python-dev python-virtualenv virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev python-lxml uwsgi uwsgi-plugin-python + +#================================================= +# NGINX CONFIGURATION +#================================================= + +ynh_nginx_config +if [ "$path_url" = "/" ] then - # It is NOT possible to setup a swap file on a tmpfs filesystem - mount | grep /tmp | grep tmpfs > /dev/null 2>&1 - if [ $? = 1 ]; - then - tmp_swap_file=/tmp/searx_swapfile - else - tmp_swap_file=/var/cache/searx_swapfile - fi - sudo dd if=/dev/zero of=$tmp_swap_file bs=1M count=256 - sudo chmod 600 $tmp_swap_file - sudo mkswap $tmp_swap_file - sudo swapon $tmp_swap_file -fi - -final_path=/opt/yunohost/$app - -# Init virtualenv -if [ ! -d $final_path ]; -then - sudo mkdir -p $final_path -fi -wget https://github.com/asciimoo/searx/archive/v0.10.0.tar.gz -tar xvf v0.10.0.tar.gz && cp -rf searx-0.10.0/* ../sources -sudo cp -r ../sources/* $final_path -sudo virtualenv --system-site-packages $final_path -sudo bash -c "source $final_path/bin/activate && pip install -r $final_path/requirements-ynh.txt" - -# Disable swapfile -if [[ -v "$tmp_swap_file" ]]; -then - sudo swapoff $tmp_swap_file - sudo rm -f $tmp_swap_file -fi - -#Configuration Searx -sudo cp ../conf/settings.yml $final_path/searx/ -sudo sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" $final_path/searx/settings.yml -if [ "$path" != "/" ]; -then - sudo sed -i -e "s@ynhbaseurl@https://$domain$path/@g" $final_path/searx/settings.yml + ynh_replace_string "__PATH_NO_ROOT__" "" /etc/nginx/conf.d/$domain.d/$app.conf + sudo sed --in-place '/#noroot*/d' /etc/nginx/conf.d/$domain.d/$app.conf else - sudo sed -i -e "s@ynhbaseurl@False@g" $final_path/searx/settings.yml + ynh_replace_string "#noroot" "" /etc/nginx/conf.d/$domain.d/$app.conf + ynh_replace_string "__PATH_NO_ROOT__" "$path_url" /etc/nginx/conf.d/$domain.d/$app.conf fi +ynh_store_checksum_config "/etc/nginx/conf.d/$domain.d/$app.conf" -# Set permissions to searx directory -sudo useradd searx -d $final_path -sudo chown searx:searx -R $final_path +#================================================= +# CREATE DEDICATED USER +#================================================= + +ynh_system_user_create $app # Créer un utilisateur système dédié à l'app + +#================================================= +# DOWNLOAD, CHECK AND UNPACK SOURCE +#================================================= + +ynh_app_setting_set $app final_path $final_path +ynh_setup_source # Télécharge la source, décompresse et copie dans $final_path + +#================================================= +# SPECIFIC SETUP +#================================================= +# INSTALL SEARX IN A VIRTUALENV +#================================================= + +sudo virtualenv --system-site-packages "$final_path" +sudo bash -c "source $final_path/bin/activate && pip install --requirement $final_path/requirements-ynh.txt" + +#================================================= +# CONFIGURE SEARX +#================================================= + +sudo cp ../conf/settings.yml "$final_path/searx/" + +# Generate a secret key +ynh_replace_string "__SECRETKEY__" "$(ynh_string_random)" "$final_path/searx/settings.yml" + +# Modify the base_url parameter, if it's installed in a subpath +if [ "$path_url" != "/" ] +then + ynh_replace_string "__BASEURL__" "https://${domain}${path_url}/" "$final_path/searx/settings.yml" +else + ynh_replace_string "__BASEURL__" "False" "$final_path/searx/settings.yml" +fi +ynh_store_checksum_config "$final_path/searx/settings.yml" + +#================================================= +# SET PERMISSIONS ON SEARX DIRECTORY +#================================================= + +sudo chown $app: --recursive "$final_path" + +#================================================= +# CONFIGURE UWSGI FOR SEARX +#================================================= -# Copy uwsgi config sudo cp ../conf/searx.ini /etc/uwsgi/apps-available/$app.ini -sudo ln -s /etc/uwsgi/apps-available/searx.ini /etc/uwsgi/apps-enabled/$app.ini +ynh_replace_string "__APP__" "$app" /etc/uwsgi/apps-available/$app.ini +ynh_replace_string "__FINALPATH__" "$final_path" /etc/uwsgi/apps-available/$app.ini +sudo ln -s /etc/uwsgi/apps-available/$app.ini /etc/uwsgi/apps-enabled/$app.ini -# Modify Nginx configuration file and copy it to Nginx conf directory -sed -i "s@YNH_WWW_LOCATION@$location@g" ../conf/nginx.conf* -sed -i "s@YNH_WWW_PATH@$path@g" ../conf/nginx.conf* -if [ "$path" != "/" ]; +#================================================= +# ENABLE UWSGI SERVICE IN ADMIN PANEL +#================================================= + +sudo systemctl restart uwsgi +# Ajoute le service au monitoring de Yunohost. +sudo yunohost service add uwsgi --log "/var/log/uwsgi/app/$app.log" + +#================================================= +# GENERIC FINALISATION +#================================================= +# SETUP SSOWAT +#================================================= + +if [ $is_public -eq 1 ] then - sudo cp ../conf/nginx.conf-noroot /etc/nginx/conf.d/$domain.d/$app.conf -else - sudo cp ../conf/nginx.conf /etc/nginx/conf.d/$domain.d/$app.conf + ynh_app_setting_set $app skipped_uris "/" fi -# Fix permission -#sudo find $final_path/ -type d -exec chmod 2755 {} \; -#sudo find $final_path/ -type f -exec chmod g+r,o+r {} \; - -## Reload Nginx and regenerate SSOwat conf -sudo service nginx reload -sudo service uwsgi restart -sudo yunohost service add uwsgi -l /var/log/uwsgi/app/$app.log - -if [ "$is_public" = "Yes" ]; -then - sudo yunohost app setting $app unprotected_uris -v "/" -fi +#================================================= +# RELOAD NGINX +#================================================= +sudo systemctl reload nginx diff --git a/scripts/remove b/scripts/remove index 495cf5e..8319fdd 100644 --- a/scripts/remove +++ b/scripts/remove @@ -1,30 +1,74 @@ #!/bin/bash -app="searx" -# Source app helpers -. /usr/share/yunohost/helpers +#================================================= +# GENERIC STARTING +#================================================= +# IMPORT GENERIC HELPERS +#================================================= -# Retrieve arguments -domain=$(ynh_app_setting_get "$app" domain) +source _common.sh +source /usr/share/yunohost/helpers -# Remove files -sudo rm -Rf /opt/yunohost/$app -sudo rm -f /etc/uwsgi/apps-enabled/$app.ini -sudo rm -f /etc/uwsgi/apps-available/$app.ini -sudo rm -f /etc/nginx/conf.d/$domain.d/$app.conf +#================================================= +# LOAD SETTINGS +#================================================= -# Stop uwsgi -sudo service uwsgi stop -sudo killall uwsgi +app=$YNH_APP_INSTANCE_NAME -# Remove user (kill attached process before) -if id -u searx > /dev/null 2>&1; then - sudo killall -KILL -u searx - sudo userdel --force searx +domain=$(ynh_app_setting_get $app domain) + +#================================================= +# STANDARD REMOVE +#================================================= +# STOP AND REMOVE SERVICE +#================================================= + +# Suppression de la configuration uwsgi +sudo systemctl stop uwsgi +if [ -h "/etc/uwsgi/apps-enabled/$app.ini" ]; then + echo "Delete uwsgi config" + ynh_secure_remove "/etc/uwsgi/apps-enabled/$app.ini" +fi +ynh_secure_remove "/etc/uwsgi/apps-available/$app.ini" + +#================================================= +# REMOVE DEPENDENCIES +#================================================= + +ynh_remove_app_dependencies + +#================================================= +# REMOVE THE MAIN DIR OF THE APP +#================================================= + +ynh_secure_remove "/opt/yunohost/$app" + +#================================================= +# REMOVE THE NGINX CONFIGURATION +#================================================= + +ynh_remove_nginx_config # Suppression de la configuration nginx + +#================================================= +# DISABLE SERVICE IN ADMIN PANEL +#================================================= + +if [ -e /etc/init.d/uwsgi ] +then + # Redémarre le service uwsgi si il n'est pas désinstallé. + sudo systemctl start uwsgi +else + if sudo yunohost service status | grep -q uwsgi # Test l'existence du service dans Yunohost + then + echo "Remove uwsgi service" + sudo yunohost service remove uwsgi + fi fi -# Restart uwsgi -sudo service uwsgi start +#================================================= +# GENERIC FINALISATION +#================================================= +# REMOVE DEDICATED USER +#================================================= -# Reload Nginx -sudo service nginx reload +ynh_system_user_delete $app diff --git a/scripts/restore b/scripts/restore index f5ce5d7..a7ed97f 100755 --- a/scripts/restore +++ b/scripts/restore @@ -1,103 +1,96 @@ #!/bin/bash -app="searx" -# causes the shell to exit if any subcommand or pipeline returns a non-zero status -set -e +#================================================= +# GENERIC STARTING +#================================================= +# MANAGE FAILURE OF THE SCRIPT +#================================================= -# Source YNH helpers -. /usr/share/yunohost/helpers +# Exit on command errors and treat unset variables as an error +set -eu -# Retrieve arguments -domain=$(ynh_app_setting_get "$app" domain) -path=$(ynh_app_setting_get "$app" path) -is_public=$(ynh_app_setting_get "$app" is_public) +#================================================= +# IMPORT GENERIC HELPERS +#================================================= -# Remove trailing slash to path -path=${path%/} -#force location to be / or /foo -location=${path:-/} - -# Check domain/path availability -sudo yunohost app checkurl $domain$path -a $app -if [[ ! $? -eq 0 ]]; then - echo "There is already an app on this URL : $domain$path" | sudo tee /dev/stderr - exit 1 +if [ ! -e _common.sh ]; then + # Rapatrie le fichier de fonctions si il n'est pas dans le dossier courant + sudo cp ../settings/scripts/_common.sh ./_common.sh + sudo chmod a+rx _common.sh fi +source _common.sh +source /usr/share/yunohost/helpers -final_path=/opt/yunohost/$app -if [ -d $final_path ]; then - echo "There is already a directory: $final_path " | sudo tee /dev/stderr - exit 1 +#================================================= +# LOAD SETTINGS +#================================================= + +app=$YNH_APP_INSTANCE_NAME + +final_path=$(ynh_app_setting_get $app final_path) +domain=$(ynh_app_setting_get $app domain) +path_url=$(ynh_app_setting_get $app path) + +#================================================= +# CHECK IF THE APP CAN BE RESTORED +#================================================= + +sudo yunohost app checkurl "${domain}${path_url}" -a "$app" \ + || ynh_die "Path not available: ${domain}${path_url}" +test ! -d $final_path \ + || ynh_die "There is already a directory: $final_path " + +#================================================= +# STANDARD RESTORE STEPS +#================================================= +# RESTORE OF THE NGINX CONFIGURATION +#================================================= + +conf=/etc/nginx/conf.d/$domain.d/$app.conf +if [ -f $conf ]; then + ynh_die "There is already a nginx conf file at this path: $conf " fi +sudo cp -a ./nginx.conf $conf -nginx_conf=/etc/nginx/conf.d/$domain.d/$app.conf -if [ -f $nginx_conf ]; then - echo "There is already a nginx conf file at this path: $nginx_conf " | sudo tee /dev/stderr - exit 1 -fi +#================================================= +# RESTORE OF THE MAIN DIR OF THE APP +#================================================= -# Run install script -# sudo bash -c "./install $domain $path $is_public $app" +sudo mkdir -p $final_path +sudo cp -a ./sources/. $final_path -# Save specific settings -sudo yunohost app setting $app is_public -v $is_public +#================================================= +# INSTALL DEPENDENCIES +#================================================= -# Check depends installation -sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev python-lxml uwsgi uwsgi-plugin-python -y +ynh_install_app_dependencies libxslt-dev virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev python-lxml uwsgi uwsgi-plugin-python -# Check Swap -if [ $(sudo swapon -s | wc -l) = 1 ]; -then - # It is NOT possible to setup a swap file on a tmpfs filesystem - mount | grep /tmp | grep tmpfs > /dev/null 2>&1 - if [ $? = 1 ]; - then - tmp_swap_file=/tmp/searx_swapfile - else - tmp_swap_file=/var/cache/searx_swapfile - fi - sudo dd if=/dev/zero of=$tmp_swap_file bs=1M count=256 - sudo chmod 600 $tmp_swap_file - sudo mkswap $tmp_swap_file - sudo swapon $tmp_swap_file -fi +#================================================= +# RECREATE OF THE DEDICATED USER +#================================================= -# Restore sources -final_path=/opt/yunohost/$app -sudo cp -a "./sources" $final_path +ynh_system_user_create $app # Recreate the dedicated user, if not exist -# Init virtualenv -sudo virtualenv --system-site-packages $final_path -sudo bash -c "source $final_path/bin/activate && pip install -r $final_path/requirements-ynh.txt" +#================================================= +# SPECIFIC RESTORE +#================================================= +# RESTORE USER RIGHTS +#================================================= -# Disable swapfile -if [[ -v "$tmp_swap_file" ]]; -then - sudo swapoff $tmp_swap_file - sudo rm -f $tmp_swap_file -fi +sudo chown $app: -R $final_path -#Configuration Searx -sudo cp "./conf/settings.yml" "$final_path/searx/settings.yml" +#================================================= +# RESTORE THE UWSGI CONFIG +#================================================= -# Set permissions to searx directory -sudo useradd searx -d $final_path -sudo chown searx:searx -R $final_path +sudo cp -a ./uwsgi_conf /etc/uwsgi/apps-available/$app.ini +sudo ln -s /etc/uwsgi/apps-available/$app.ini /etc/uwsgi/apps-enabled/$app.ini -# Copy uwsgi config -sudo cp "./conf/searx.ini" "/etc/uwsgi/apps-available/$app.ini" -sudo ln -s "/etc/uwsgi/apps-available/$app.ini" /etc/uwsgi/apps-enabled/ - -# Nginx conf -sudo cp "conf/nginx.conf" $nginx_conf - -if [ "$is_public" = "Yes" ]; -then - sudo yunohost app setting searx unprotected_uris -v "/" -fi - -## Reload services -sudo service nginx reload -sudo service uwsgi restart -sudo yunohost service add uwsgi -l /var/log/uwsgi/app/$app.log +#================================================= +# GENERIC FINALISATION +#================================================= +# RELOAD NGINX AND UWSGI +#================================================= +sudo systemctl restart uwsgi +sudo systemctl reload nginx diff --git a/scripts/upgrade b/scripts/upgrade index 9ae81ca..c71f3da 100644 --- a/scripts/upgrade +++ b/scripts/upgrade @@ -1,94 +1,145 @@ #!/bin/bash -app="searx" -# causes the shell to exit if any subcommand or pipeline returns a non-zero status -set -e +#================================================= +# GENERIC STARTING +#================================================= +# IMPORT GENERIC HELPERS +#================================================= -# Source app helpers -. /usr/share/yunohost/helpers +source _common.sh +source /usr/share/yunohost/helpers -# Retrieve arguments -domain=$(ynh_app_setting_get "$app" domain) -path=$(ynh_app_setting_get "$app" path) -is_public=$(ynh_app_setting_get "$app" is_public) +#================================================= +# LOAD SETTINGS +#================================================= -# Remove trailing slash to path -path=${path%/} -#force location to be / or /foo -location=${path:-/} +app=$YNH_APP_INSTANCE_NAME -# Check depends installation -sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev python-lxml uwsgi uwsgi-plugin-python -y +domain=$(ynh_app_setting_get $app domain) +path_url=$(ynh_app_setting_get $app path) +is_public=$(ynh_app_setting_get $app is_public) +final_path=$(ynh_app_setting_get $app final_path) -# Check Swap -if [ $(sudo swapon -s | wc -l) = 1 ]; -then - mount | grep /tmp | grep tmpfs > /dev/null 2>&1 - if [ $? = 1 ]; - then - tmp_swap_file=/tmp/searx_swapfile - else - tmp_swap_file=/var/cache/ - fi - sudo dd if=/dev/zero of=$tmp_swap_file bs=1M count=256 - sudo chmod 600 $tmp_swap_file - sudo mkswap $tmp_swap_file - sudo swapon $tmp_swap_file +#================================================= +# FIX OLD THINGS +#================================================= + +if [ "$is_public" = "Yes" ]; then + ynh_app_setting_set $app is_public 1 # Fixe is_public en booléen + is_public=1 +elif [ "$is_public" = "No" ]; then + ynh_app_setting_set $app is_public 0 + is_public=0 fi -final_path=/opt/yunohost/$app -wget https://github.com/asciimoo/searx/archive/v0.10.0.tar.gz -tar xvf v0.10.0.tar.gz && cp -rf searx-0.10.0/* ../sources -sudo cp -rf ../sources/* $final_path -sudo bash -c "source $final_path/bin/activate && pip install -r $final_path/requirements-ynh.txt --upgrade" - -# Disable swapfile -if [[ -v "$tmp_swap_file" ]]; -then - sudo swapoff $tmp_swap_file - sudo rm -f $tmp_swap_file +if [ -z $final_path ]; then # Si final_path n'est pas renseigné dans app setting + final_path="/opt/yunohost/$app" + ynh_app_setting_set $app final_path $final_path fi -#Configuration Searx -sudo cp ../conf/settings.yml $final_path/searx/ -sudo sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" $final_path/searx/settings.yml -if [ "$path" != "/" ]; +#================================================= +# BACKUP BEFORE UPGRADE THEN ACTIVE TRAP +#================================================= + +BACKUP_BEFORE_UPGRADE # Backup the current version of the app +ynh_clean_setup () { + BACKUP_FAIL_UPGRADE # restore it if the upgrade fails +} +ynh_abort_if_errors # Active trap pour arrêter le script si une erreur est détectée. + +#================================================= +# CHECK THE PATH +#================================================= + +path_url=$(ynh_normalize_url_path $path_url) # Vérifie et corrige la syntaxe du path. + +#================================================= +# STANDARD UPGRADE STEPS +#================================================= +# DOWNLOAD, CHECK AND UNPACK SOURCE +#================================================= + +ynh_setup_source # Télécharge la source, décompresse et copie dans $final_path + +#================================================= +# NGINX CONFIGURATION +#================================================= + +ynh_nginx_config +if [ "$path_url" = "/" ] then - sudo sed -i -e "s@ynhbaseurl@https://$domain$path/@g" $final_path/searx/settings.yml + ynh_replace_string "__PATH_NO_ROOT__" "" /etc/nginx/conf.d/$domain.d/$app.conf + sudo sed --in-place '/#noroot*/d' /etc/nginx/conf.d/$domain.d/$app.conf else - sudo sed -i -e "s@ynhbaseurl@False@g" $final_path/searx/settings.yml + ynh_replace_string "#noroot" "" /etc/nginx/conf.d/$domain.d/$app.conf + ynh_replace_string "__PATH_NO_ROOT__" "$path_url" /etc/nginx/conf.d/$domain.d/$app.conf fi +ynh_store_checksum_config "/etc/nginx/conf.d/$domain.d/$app.conf" -# Set permissions to searx directory -if ! id -u searx > /dev/null 2>&1; then - sudo useradd searx -d $final_path +#================================================= +# CREATE DEDICATED USER +#================================================= + +ynh_system_user_create $app # Create the dedicated user, if not exist + +#================================================= +# SPECIFIC UPGRADE +#================================================= +# UPGRADE SEARX IN ITS VIRTUALENV +#================================================= + +sudo virtualenv --system-site-packages "$final_path" +sudo bash -c "source $final_path/bin/activate && pip install --requirement $final_path/requirements-ynh.txt --upgrade" + +#================================================= +# CONFIGURE SEARX +#================================================= + +# Créé un backup du fichier de config si il a été modifié. +ynh_compare_checksum_config "$final_path/searx/settings.yml" + +sudo cp ../conf/settings.yml "$final_path/searx/" + +# Generate a secret key +ynh_replace_string "__SECRETKEY__" "$(ynh_string_random)" "$final_path/searx/settings.yml" + +# Modify the base_url parameter, if it's installed in a subpath +if [ "$path_url" != "/" ] +then + ynh_replace_string "__BASEURL__" "https://${domain}${path_url}/" "$final_path/searx/settings.yml" +else + ynh_replace_string "__BASEURL__" "False" "$final_path/searx/settings.yml" fi -sudo chown searx:searx -R $final_path +ynh_store_checksum_config "$final_path/searx/settings.yml" + +#================================================= +# GENERIC FINALISATION +#================================================= +# SECURING FILES AND DIRECTORIES +#================================================= + +sudo chown $app: --recursive "$final_path" + +#================================================= +# CONFIGURE UWSGI FOR SEARX +#================================================= -# Copy uwsgi config sudo cp ../conf/searx.ini /etc/uwsgi/apps-available/$app.ini +ynh_replace_string "__APP__" "$app" /etc/uwsgi/apps-available/$app.ini +ynh_replace_string "__FINALPATH__" "$final_path" /etc/uwsgi/apps-available/$app.ini +sudo systemctl restart uwsgi -# Modify Nginx configuration file and copy it to Nginx conf directory -sed -i "s@YNH_WWW_LOCATION@$location@g" ../conf/nginx.conf* -sed -i "s@YNH_WWW_PATH@$path@g" ../conf/nginx.conf* -if [ "$path" != "/" ]; +#================================================= +# SETUP SSOWAT +#================================================= + +if [ $is_public -eq 1 ] then - sudo cp ../conf/nginx.conf-noroot /etc/nginx/conf.d/$domain.d/$app.conf -else - sudo cp ../conf/nginx.conf /etc/nginx/conf.d/$domain.d/$app.conf + ynh_app_setting_set $app skipped_uris "/" fi -# Fix permission -#sudo find $final_path/ -type d -exec chmod 2755 {} \; -#sudo find $final_path/ -type f -exec chmod g+r,o+r {} \; - -if [ "$is_public" = "Yes" ]; -then - sudo yunohost app setting searx skipped_uris -d - sudo yunohost app setting searx unprotected_uris -v "/" -fi - -## Reload services -sudo service nginx reload -sudo service uwsgi restart +#================================================= +# RELOAD NGINX +#================================================= +sudo systemctl reload nginx diff --git a/sources/requirements-ynh.txt b/sources/extra_files/requirements-ynh.txt similarity index 50% rename from sources/requirements-ynh.txt rename to sources/extra_files/requirements-ynh.txt index f901007..a821ba1 100644 --- a/sources/requirements-ynh.txt +++ b/sources/extra_files/requirements-ynh.txt @@ -1,14 +1,14 @@ -certifi==2016.2.28 -flask==0.11.1 +certifi==2016.9.26 +flask==0.12 flask-babel==0.11.1 -# lxml==3.5.0 +# lxml==3.7.1 markupsafe==0.23 -ndg-httpsclient==0.4.1 +ndg-httpsclient==0.4.2 pyasn1==0.1.9 pyasn1-modules==0.0.8 pygments==2.1.3 -pyopenssl==0.15.1 +pyopenssl==16.2.0 python-dateutil==2.5.3 pyyaml==3.11 -requests[socks]==2.10.0 +requests[socks]==2.12.4 cffi>=1.6 \ No newline at end of file