diff --git a/00_Prepare_everything/index.html b/00_Prepare_everything/index.html old mode 100755 new mode 100644 index 0ae93cd..86f230c --- a/00_Prepare_everything/index.html +++ b/00_Prepare_everything/index.html @@ -153,10 +153,9 @@

Before You Start

-

These documents will guide you through the process of creating your own Extractor -service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube and SoundCloud. -The whole documentation consists of this page, which explains the general concept of the NewPipeExtractor, as well as our -Jdoc setup.

+

These documents will guide you through the process of understanding or creating your own Extractor +service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube, SoundCloud and MediaCCC. +The whole documentation consists of this page and Jdoc setup, which explains the general concept of the NewPipeExtractor.

IMPORTANT!!! This is likely to be the worst documentation you have ever read, so do not hesitate to report if you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community diff --git a/01_Concept_of_the_extractor/index.html b/01_Concept_of_the_extractor/index.html old mode 100755 new mode 100644 index d006114..d79983f --- a/01_Concept_of_the_extractor/index.html +++ b/01_Concept_of_the_extractor/index.html @@ -75,7 +75,7 @@

  • Collector/Extractor Pattern for Lists
  • -
  • InfoItems Encapsulated in Pages
  • +
  • ListExtractor
  • @@ -196,15 +196,16 @@ try {

    Collector/Extractor Pattern for Lists

    Information can be represented as a list. In NewPipe, a list is represented by a InfoItemsCollector. -A InfoItemCollector will collect and assemble a list of InfoItem. -For each item that should be extracted, a new Extractor must be created, and given to the InfoItemCollector via commit().

    +A InfoItemsCollector will collect and assemble a list of InfoItem. +For each item that should be extracted, a new Extractor must be created, and given to the InfoItemsCollector via commit().

    InfoItemsCollector_objectdiagram.svg

    -

    If you are implementing a list for your service you need to extend InfoItem containing the extracted information -and implement an InfoItemExtractor, -that will return the data of one InfoItem.

    +

    If you are implementing a list in your service you need to implement an InfoItemExtractor, +that will be able to retreve data for one and only one InfoItem. This extractor will then be comitted to the InfoItemsCollector that can collect the type of InfoItems you want to generate.

    A common implementation would look like this:

    -
    private MyInfoItemCollector collectInfoItemsFromElement(Element e) {
    -    MyInfoItemCollector collector = new MyInfoItemCollector(getServiceId());
    +
    private SomeInfoItemCollector collectInfoItemsFromElement(Element e) {
    +    // See *Some* as something like Stream or Channel
    +    // e.g. StreamInfoItemsCollector, and ChannelInfoItemsCollector are provided by NP
    +    SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId());
     
         for(final Element li : element.children()) {
             collector.commit(new InfoItemExtractor() {
    @@ -225,15 +226,21 @@ that will return the data of one InfoItem.

    -

    InfoItems Encapsulated in Pages

    +

    ListExtractor

    +

    There is more to know about lists:

    +
      +
    1. When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail, and its creator. Such info can be called list header.

      -

      When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down.

      -

      This is why a list in NewPipe lists are chopped down into smaller lists called InfoItemsPages. Each page has its own URL, and needs to be extracted separately.

      -

      Additional metadata about the list and extracting multiple pages can be handled by a -ListExtractor, -and its ListExtractor.InfoItemsPage.

      -

      For extracting list header information it behaves like a regular extractor. For handling InfoItemsPages it adds methods +

    2. +
    3. +

      When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down.

      +
    4. +
    +

    Both of these Problems are fixed by the ListExtractor which takes care about extracting additional metadata about the liast, +and by chopping down lists into several pages, so called InfoItemsPages. +Each page has its own URL, and needs to be extracted separately.

    +

    For extracting list header information a ListExtractor behaves like a regular extractor. For handling InfoItemsPages it adds methods such as:

    The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular web page, but all the others as an AJAX request.

    +

    An InfoItemsPage itself has two constructors which take these parameters: +- The InfoitemsCollector of the list that the page should represent +- A nextPageUrl which represents the url of the following page (may be null if not page follows). +- Optionally errors which is a list of Exceptions that may have happened during extracton.

    +

    Here is a simplified reference implementation of a list extractor that only extracts pages, but not metadata:

    +
    class MyListExtractor extends ListExtractor {
    +    ...
    +    private Document document;
    +
    +    ...
    +
    +    public InfoItemsPage<SomeInfoItem> getPage(pageUrl)
    +        throws ExtractionException {
    +        SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId());
    +        document = myFunctionToGetThePageHTMLWhatever(pageUrl);
    +
    +        //remember this part from the simple list extraction
    +        for(final Element li : document.children()) {
    +            collector.commit(new InfoItemExtractor() {
    +                @Override
    +                public String getName() throws ParsingException {
    +                    ...
    +                }
    +
    +                @Override
    +                public String getUrl() throws ParsingException {
    +                    ...
    +                }
    +                ...
    +        }
    +        return new InfoItemsPage<SomeInfoItem>(collector, myFunctionToGetTheNextPageUrl(document));
    +    }
    +
    +    public InfoItemsPage<SomeInfoItem> getInitialPage() {
    +        //document here got initialzied by the fetch() function.
    +        return getPage(getTheCurrentPageUrl(document));
    +    }
    +    ... 
    +}
    +
    diff --git a/02_Concept_of_LinkHandler/index.html b/02_Concept_of_LinkHandler/index.html old mode 100755 new mode 100644 diff --git a/03_Implement_a_service/index.html b/03_Implement_a_service/index.html old mode 100755 new mode 100644 diff --git a/04_Run_changes_in_App/index.html b/04_Run_changes_in_App/index.html old mode 100755 new mode 100644 diff --git a/05_releasing/index.html b/05_releasing/index.html old mode 100755 new mode 100644 diff --git a/06_documentation/index.html b/06_documentation/index.html old mode 100755 new mode 100644 diff --git a/07_maintainers_view/index.html b/07_maintainers_view/index.html old mode 100755 new mode 100644 diff --git a/404.html b/404.html old mode 100755 new mode 100644 diff --git a/css/github.min.css b/css/github.min.css old mode 100755 new mode 100644 index 3fadc2d..47e68ba --- a/css/github.min.css +++ b/css/github.min.css @@ -1,62 +1,62 @@ -.hljs { - display:block; - overflow-x:auto; - padding:1em; - color:#333; - background:#f8f8f8 -} -.hljs-comment,.hljs-quote { - color:#998; - font-style:italic -} -.hljs-keyword,.hljs-selector-tag,.hljs-subst { - color:#333; - font-weight:bold -} -.hljs-number,.hljs-literal,.hljs-variable,.hljs-template-variable,.hljs-tag .hljs-attr { - color:#008080 -} -.hljs-string,.hljs-doctag { - color:#d14 -} -.hljs-title,.hljs-section,.hljs-selector-id { - color:#900; - font-weight:bold -} -.hljs-subst { - font-weight:normal -} -.hljs-type,.hljs-class .hljs-title { - color:#458; - font-weight:bold -} -.hljs-tag,.hljs-name,.hljs-attribute { - color:#000080; - font-weight:normal -} -.hljs-regexp,.hljs-link { - color:#009926 -} -.hljs-symbol,.hljs-bullet { - color:#990073 -} -.hljs-built_in,.hljs-builtin-name { - color:#0086b3 -} -.hljs-meta { - color:#999; - font-weight:bold -} -.hljs-deletion { - background:#fdd -} -.hljs-addition { - background:#dfd -} -.hljs-emphasis { - font-style:italic -} -.hljs-strong { - font-weight:bold -} - +.hljs { + display:block; + overflow-x:auto; + padding:1em; + color:#333; + background:#f8f8f8 +} +.hljs-comment,.hljs-quote { + color:#998; + font-style:italic +} +.hljs-keyword,.hljs-selector-tag,.hljs-subst { + color:#333; + font-weight:bold +} +.hljs-number,.hljs-literal,.hljs-variable,.hljs-template-variable,.hljs-tag .hljs-attr { + color:#008080 +} +.hljs-string,.hljs-doctag { + color:#d14 +} +.hljs-title,.hljs-section,.hljs-selector-id { + color:#900; + font-weight:bold +} +.hljs-subst { + font-weight:normal +} +.hljs-type,.hljs-class .hljs-title { + color:#458; + font-weight:bold +} +.hljs-tag,.hljs-name,.hljs-attribute { + color:#000080; + font-weight:normal +} +.hljs-regexp,.hljs-link { + color:#009926 +} +.hljs-symbol,.hljs-bullet { + color:#990073 +} +.hljs-built_in,.hljs-builtin-name { + color:#0086b3 +} +.hljs-meta { + color:#999; + font-weight:bold +} +.hljs-deletion { + background:#fdd +} +.hljs-addition { + background:#dfd +} +.hljs-emphasis { + font-style:italic +} +.hljs-strong { + font-weight:bold +} + diff --git a/css/highlight.css b/css/highlight.css old mode 100755 new mode 100644 index ed74bb1..4e68981 --- a/css/highlight.css +++ b/css/highlight.css @@ -1,115 +1,115 @@ -.codehilite code, .codehilite pre{color:#3F3F3F;background-color:#F7F7F7; -overflow: auto; -box-sizing: border-box; - - padding: 0.01em 16px; - padding-top: 0.01em; - padding-right-value: 16px; - padding-bottom: 0.01em; - padding-left-value: 16px; - padding-left-ltr-source: physical; - padding-left-rtl-source: physical; - padding-right-ltr-source: physical; - padding-right-rtl-source: physical; - -border-radius: 16px !important; - border-top-left-radius: 16px; - border-top-right-radius: 16px; - border-bottom-right-radius: 16px; - border-bottom-left-radius: 16px; - -border: 1px solid #CCC !important; - border-top-width: 1px; - border-right-width-value: 1px; - border-right-width-ltr-source: physical; - border-right-width-rtl-source: physical; - border-bottom-width: 1px; - border-left-width-value: 1px; - border-left-width-ltr-source: physical; - border-left-width-rtl-source: physical; - border-top-style: solid; - border-right-style-value: solid; - border-right-style-ltr-source: physical; - border-right-style-rtl-source: physical; - border-bottom-style: solid; - border-left-style-value: solid; - border-left-style-ltr-source: physical; - border-left-style-rtl-source: physical; - border-top-color: #CCC; - border-right-color-value: #CCC; - border-right-color-ltr-source: physical; - border-right-color-rtl-source: physical; - border-bottom-color: #CCC; - border-left-color-value: #CCC; - border-left-color-ltr-source: physical; - border-left-color-rtl-source: physical; - -moz-border-top-colors: none; - -moz-border-right-colors: none; - -moz-border-bottom-colors: none; - -moz-border-left-colors: none; - border-image-source: none; - border-image-slice: 100% 100% 100% 100%; - border-image-width: 1 1 1 1; - border-image-outset: 0 0 0 0; - border-image-repeat: stretch stretch;} -.codehilite .hll { background-color: #ffffcc } -.codehilite .c { color: #999988; font-style: italic } /* Comment */ -.codehilite .err { color: #a61717; background-color: #e3d2d2 } /* Error */ -.codehilite .k { color: #000000; font-weight: bold } /* Keyword */ -.codehilite .o { color: #000000; font-weight: bold } /* Operator */ -.codehilite .cm { color: #999988; font-style: italic } /* Comment.Multiline */ -.codehilite .cp { color: #999999; font-weight: bold; font-style: italic } /* Comment.Preproc */ -.codehilite .c1 { color: #999988; font-style: italic } /* Comment.Single */ -.codehilite .cs { color: #999999; font-weight: bold; font-style: italic } /* Comment.Special */ -.codehilite .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */ -.codehilite .ge { color: #000000; font-style: italic } /* Generic.Emph */ -.codehilite .gr { color: #aa0000 } /* Generic.Error */ -.codehilite .gh { color: #999999 } /* Generic.Heading */ -.codehilite .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */ -.codehilite .go { color: #888888 } /* Generic.Output */ -.codehilite .gp { color: #555555 } /* Generic.Prompt */ -.codehilite .gs { font-weight: bold } /* Generic.Strong */ -.codehilite .gu { color: #aaaaaa } /* Generic.Subheading */ -.codehilite .gt { color: #aa0000 } /* Generic.Traceback */ -.codehilite .kc { color: #000000; font-weight: bold } /* Keyword.Constant */ -.codehilite .kd { color: #000000; font-weight: bold } /* Keyword.Declaration */ -.codehilite .kn { color: #000000; font-weight: bold } /* Keyword.Namespace */ -.codehilite .kp { color: #000000; font-weight: bold } /* Keyword.Pseudo */ -.codehilite .kr { color: #000000; font-weight: bold } /* Keyword.Reserved */ -.codehilite .kt { color: #445588; font-weight: bold } /* Keyword.Type */ -.codehilite .m { color: #009999 } /* Literal.Number */ -.codehilite .s { color: #d01040 } /* Literal.String */ -.codehilite .na { color: #008080 } /* Name.Attribute */ -.codehilite .nb { color: #0086B3 } /* Name.Builtin */ -.codehilite .nc { color: #445588; font-weight: bold } /* Name.Class */ -.codehilite .no { color: #008080 } /* Name.Constant */ -.codehilite .nd { color: #3c5d5d; font-weight: bold } /* Name.Decorator */ -.codehilite .ni { color: #800080 } /* Name.Entity */ -.codehilite .ne { color: #990000; font-weight: bold } /* Name.Exception */ -.codehilite .nf { color: #990000; font-weight: bold } /* Name.Function */ -.codehilite .nl { color: #990000; font-weight: bold } /* Name.Label */ -.codehilite .nn { color: #555555 } /* Name.Namespace */ -.codehilite .nt { color: #000080 } /* Name.Tag */ -.codehilite .nv { color: #008080 } /* Name.Variable */ -.codehilite .ow { color: #000000; font-weight: bold } /* Operator.Word */ -.codehilite .w { color: #bbbbbb } /* Text.Whitespace */ -.codehilite .mf { color: #009999 } /* Literal.Number.Float */ -.codehilite .mh { color: #009999 } /* Literal.Number.Hex */ -.codehilite .mi { color: #009999 } /* Literal.Number.Integer */ -.codehilite .mo { color: #009999 } /* Literal.Number.Oct */ -.codehilite .sb { color: #d01040 } /* Literal.String.Backtick */ -.codehilite .sc { color: #d01040 } /* Literal.String.Char */ -.codehilite .sd { color: #d01040 } /* Literal.String.Doc */ -.codehilite .s2 { color: #d01040 } /* Literal.String.Double */ -.codehilite .se { color: #d01040 } /* Literal.String.Escape */ -.codehilite .sh { color: #d01040 } /* Literal.String.Heredoc */ -.codehilite .si { color: #d01040 } /* Literal.String.Interpol */ -.codehilite .sx { color: #d01040 } /* Literal.String.Other */ -.codehilite .sr { color: #009926 } /* Literal.String.Regex */ -.codehilite .s1 { color: #d01040 } /* Literal.String.Single */ -.codehilite .ss { color: #990073 } /* Literal.String.Symbol */ -.codehilite .bp { color: #999999 } /* Name.Builtin.Pseudo */ -.codehilite .vc { color: #008080 } /* Name.Variable.Class */ -.codehilite .vg { color: #008080 } /* Name.Variable.Global */ -.codehilite .vi { color: #008080 } /* Name.Variable.Instance */ -.codehilite .il { color: #009999 } /* Literal.Number.Integer.Long */ +.codehilite code, .codehilite pre{color:#3F3F3F;background-color:#F7F7F7; +overflow: auto; +box-sizing: border-box; + + padding: 0.01em 16px; + padding-top: 0.01em; + padding-right-value: 16px; + padding-bottom: 0.01em; + padding-left-value: 16px; + padding-left-ltr-source: physical; + padding-left-rtl-source: physical; + padding-right-ltr-source: physical; + padding-right-rtl-source: physical; + +border-radius: 16px !important; + border-top-left-radius: 16px; + border-top-right-radius: 16px; + border-bottom-right-radius: 16px; + border-bottom-left-radius: 16px; + +border: 1px solid #CCC !important; + border-top-width: 1px; + border-right-width-value: 1px; + border-right-width-ltr-source: physical; + border-right-width-rtl-source: physical; + border-bottom-width: 1px; + border-left-width-value: 1px; + border-left-width-ltr-source: physical; + border-left-width-rtl-source: physical; + border-top-style: solid; + border-right-style-value: solid; + border-right-style-ltr-source: physical; + border-right-style-rtl-source: physical; + border-bottom-style: solid; + border-left-style-value: solid; + border-left-style-ltr-source: physical; + border-left-style-rtl-source: physical; + border-top-color: #CCC; + border-right-color-value: #CCC; + border-right-color-ltr-source: physical; + border-right-color-rtl-source: physical; + border-bottom-color: #CCC; + border-left-color-value: #CCC; + border-left-color-ltr-source: physical; + border-left-color-rtl-source: physical; + -moz-border-top-colors: none; + -moz-border-right-colors: none; + -moz-border-bottom-colors: none; + -moz-border-left-colors: none; + border-image-source: none; + border-image-slice: 100% 100% 100% 100%; + border-image-width: 1 1 1 1; + border-image-outset: 0 0 0 0; + border-image-repeat: stretch stretch;} +.codehilite .hll { background-color: #ffffcc } +.codehilite .c { color: #999988; font-style: italic } /* Comment */ +.codehilite .err { color: #a61717; background-color: #e3d2d2 } /* Error */ +.codehilite .k { color: #000000; font-weight: bold } /* Keyword */ +.codehilite .o { color: #000000; font-weight: bold } /* Operator */ +.codehilite .cm { color: #999988; font-style: italic } /* Comment.Multiline */ +.codehilite .cp { color: #999999; font-weight: bold; font-style: italic } /* Comment.Preproc */ +.codehilite .c1 { color: #999988; font-style: italic } /* Comment.Single */ +.codehilite .cs { color: #999999; font-weight: bold; font-style: italic } /* Comment.Special */ +.codehilite .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */ +.codehilite .ge { color: #000000; font-style: italic } /* Generic.Emph */ +.codehilite .gr { color: #aa0000 } /* Generic.Error */ +.codehilite .gh { color: #999999 } /* Generic.Heading */ +.codehilite .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */ +.codehilite .go { color: #888888 } /* Generic.Output */ +.codehilite .gp { color: #555555 } /* Generic.Prompt */ +.codehilite .gs { font-weight: bold } /* Generic.Strong */ +.codehilite .gu { color: #aaaaaa } /* Generic.Subheading */ +.codehilite .gt { color: #aa0000 } /* Generic.Traceback */ +.codehilite .kc { color: #000000; font-weight: bold } /* Keyword.Constant */ +.codehilite .kd { color: #000000; font-weight: bold } /* Keyword.Declaration */ +.codehilite .kn { color: #000000; font-weight: bold } /* Keyword.Namespace */ +.codehilite .kp { color: #000000; font-weight: bold } /* Keyword.Pseudo */ +.codehilite .kr { color: #000000; font-weight: bold } /* Keyword.Reserved */ +.codehilite .kt { color: #445588; font-weight: bold } /* Keyword.Type */ +.codehilite .m { color: #009999 } /* Literal.Number */ +.codehilite .s { color: #d01040 } /* Literal.String */ +.codehilite .na { color: #008080 } /* Name.Attribute */ +.codehilite .nb { color: #0086B3 } /* Name.Builtin */ +.codehilite .nc { color: #445588; font-weight: bold } /* Name.Class */ +.codehilite .no { color: #008080 } /* Name.Constant */ +.codehilite .nd { color: #3c5d5d; font-weight: bold } /* Name.Decorator */ +.codehilite .ni { color: #800080 } /* Name.Entity */ +.codehilite .ne { color: #990000; font-weight: bold } /* Name.Exception */ +.codehilite .nf { color: #990000; font-weight: bold } /* Name.Function */ +.codehilite .nl { color: #990000; font-weight: bold } /* Name.Label */ +.codehilite .nn { color: #555555 } /* Name.Namespace */ +.codehilite .nt { color: #000080 } /* Name.Tag */ +.codehilite .nv { color: #008080 } /* Name.Variable */ +.codehilite .ow { color: #000000; font-weight: bold } /* Operator.Word */ +.codehilite .w { color: #bbbbbb } /* Text.Whitespace */ +.codehilite .mf { color: #009999 } /* Literal.Number.Float */ +.codehilite .mh { color: #009999 } /* Literal.Number.Hex */ +.codehilite .mi { color: #009999 } /* Literal.Number.Integer */ +.codehilite .mo { color: #009999 } /* Literal.Number.Oct */ +.codehilite .sb { color: #d01040 } /* Literal.String.Backtick */ +.codehilite .sc { color: #d01040 } /* Literal.String.Char */ +.codehilite .sd { color: #d01040 } /* Literal.String.Doc */ +.codehilite .s2 { color: #d01040 } /* Literal.String.Double */ +.codehilite .se { color: #d01040 } /* Literal.String.Escape */ +.codehilite .sh { color: #d01040 } /* Literal.String.Heredoc */ +.codehilite .si { color: #d01040 } /* Literal.String.Interpol */ +.codehilite .sx { color: #d01040 } /* Literal.String.Other */ +.codehilite .sr { color: #009926 } /* Literal.String.Regex */ +.codehilite .s1 { color: #d01040 } /* Literal.String.Single */ +.codehilite .ss { color: #990073 } /* Literal.String.Symbol */ +.codehilite .bp { color: #999999 } /* Name.Builtin.Pseudo */ +.codehilite .vc { color: #008080 } /* Name.Variable.Class */ +.codehilite .vg { color: #008080 } /* Name.Variable.Global */ +.codehilite .vi { color: #008080 } /* Name.Variable.Instance */ +.codehilite .il { color: #009999 } /* Literal.Number.Integer.Long */ diff --git a/css/local_fonts.css b/css/local_fonts.css old mode 100755 new mode 100644 index 4a4963f..4139f27 --- a/css/local_fonts.css +++ b/css/local_fonts.css @@ -1,60 +1,60 @@ -/* - Stylesheet to load all local fonts except Font-Awesome because that is done by the theme. - Add all new fonts below. - */ - -@font-face { - font-family: 'Inconsolata'; - font-style: normal; - font-weight: 400; - src: local('Inconsolata'), local('Inconsolata-Regular'), url(../fonts/Inconsolata-Regular.ttf) format('truetype'); -} - -@font-face { - font-family: 'Inconsolata'; - font-style: normal; - font-weight: 700; - src: local('Inconsolata Bold'), local('Inconsolata-Bold'), url(../fonts/Inconsolata-Bold.ttf) format('truetype'); -} - -@font-face { - font-family: 'Lato'; - font-style: normal; - font-weight: 400; - src: local('Lato Regular'), local('Lato-Regular'), url(../fonts/Lato-Regular.ttf) format('truetype'); -} - -@font-face { - font-family: 'Lato'; - font-style: normal; - font-weight: 700; - src: local('Lato Bold'), local('Lato-Bold'), url(../fonts/Lato-Bold.ttf) format('truetype'); -} - -@font-face { - font-family: 'Lato'; - font-style: italic; - font-weight: 400; - src: local('Lato Italic'), local('Lato-Italic'), url(../fonts/Lato-Italic.ttf) format('truetype'); -} - -@font-face { - font-family: 'Lato'; - font-style: italic; - font-weight: 700; - src: local('Lato Bold Italic'), local('Lato-BoldItalic'), url(../fonts/Lato-BoldItalic.ttf) format('truetype'); -} - -@font-face { - font-family: 'Roboto Slab'; - font-style: normal; - font-weight: 400; - src: local('Roboto Slab Regular'), local('RobotoSlab-Regular'), url(../fonts/RobotoSlab-Regular.ttf) format('truetype'); -} - -@font-face { - font-family: 'Roboto Slab'; - font-style: normal; - font-weight: 700; - src: local('Roboto Slab Bold'), local('RobotoSlab-Bold'), url(../fonts/RobotoSlab-Bold.ttf) format('truetype'); -} +/* + Stylesheet to load all local fonts except Font-Awesome because that is done by the theme. + Add all new fonts below. + */ + +@font-face { + font-family: 'Inconsolata'; + font-style: normal; + font-weight: 400; + src: local('Inconsolata'), local('Inconsolata-Regular'), url(../fonts/Inconsolata-Regular.ttf) format('truetype'); +} + +@font-face { + font-family: 'Inconsolata'; + font-style: normal; + font-weight: 700; + src: local('Inconsolata Bold'), local('Inconsolata-Bold'), url(../fonts/Inconsolata-Bold.ttf) format('truetype'); +} + +@font-face { + font-family: 'Lato'; + font-style: normal; + font-weight: 400; + src: local('Lato Regular'), local('Lato-Regular'), url(../fonts/Lato-Regular.ttf) format('truetype'); +} + +@font-face { + font-family: 'Lato'; + font-style: normal; + font-weight: 700; + src: local('Lato Bold'), local('Lato-Bold'), url(../fonts/Lato-Bold.ttf) format('truetype'); +} + +@font-face { + font-family: 'Lato'; + font-style: italic; + font-weight: 400; + src: local('Lato Italic'), local('Lato-Italic'), url(../fonts/Lato-Italic.ttf) format('truetype'); +} + +@font-face { + font-family: 'Lato'; + font-style: italic; + font-weight: 700; + src: local('Lato Bold Italic'), local('Lato-BoldItalic'), url(../fonts/Lato-BoldItalic.ttf) format('truetype'); +} + +@font-face { + font-family: 'Roboto Slab'; + font-style: normal; + font-weight: 400; + src: local('Roboto Slab Regular'), local('RobotoSlab-Regular'), url(../fonts/RobotoSlab-Regular.ttf) format('truetype'); +} + +@font-face { + font-family: 'Roboto Slab'; + font-style: normal; + font-weight: 700; + src: local('Roboto Slab Bold'), local('RobotoSlab-Bold'), url(../fonts/RobotoSlab-Bold.ttf) format('truetype'); +} diff --git a/css/theme.css b/css/theme.css old mode 100755 new mode 100644 diff --git a/css/theme_child.css b/css/theme_child.css old mode 100755 new mode 100644 index bf8041f..0cabce0 --- a/css/theme_child.css +++ b/css/theme_child.css @@ -1,3 +1,3 @@ -.wy-nav-top i { - line-height: 50px; +.wy-nav-top i { + line-height: 50px; } \ No newline at end of file diff --git a/css/theme_extra.css b/css/theme_extra.css old mode 100755 new mode 100644 diff --git a/fonts/Inconsolata-Bold.ttf b/fonts/Inconsolata-Bold.ttf old mode 100755 new mode 100644 diff --git a/fonts/Inconsolata-Regular.ttf b/fonts/Inconsolata-Regular.ttf old mode 100755 new mode 100644 diff --git a/fonts/Lato-Bold.ttf b/fonts/Lato-Bold.ttf old mode 100755 new mode 100644 diff --git a/fonts/Lato-BoldItalic.ttf b/fonts/Lato-BoldItalic.ttf old mode 100755 new mode 100644 diff --git a/fonts/Lato-Italic.ttf b/fonts/Lato-Italic.ttf old mode 100755 new mode 100644 diff --git a/fonts/Lato-Regular.ttf b/fonts/Lato-Regular.ttf old mode 100755 new mode 100644 diff --git a/fonts/RobotoSlab-Bold.ttf b/fonts/RobotoSlab-Bold.ttf old mode 100755 new mode 100644 diff --git a/fonts/RobotoSlab-Regular.ttf b/fonts/RobotoSlab-Regular.ttf old mode 100755 new mode 100644 diff --git a/fonts/fontawesome-webfont.eot b/fonts/fontawesome-webfont.eot old mode 100755 new mode 100644 diff --git a/fonts/fontawesome-webfont.svg b/fonts/fontawesome-webfont.svg old mode 100755 new mode 100644 diff --git a/fonts/fontawesome-webfont.ttf b/fonts/fontawesome-webfont.ttf old mode 100755 new mode 100644 diff --git a/fonts/fontawesome-webfont.woff b/fonts/fontawesome-webfont.woff old mode 100755 new mode 100644 diff --git a/img/InfoItemsCollector_objectdiagram.svg b/img/InfoItemsCollector_objectdiagram.svg old mode 100755 new mode 100644 index 0f515cb..d661de9 --- a/img/InfoItemsCollector_objectdiagram.svg +++ b/img/InfoItemsCollector_objectdiagram.svg @@ -1,39 +1,39 @@ - - - - - - - - :InfoItemsCollector - - - - - - - - itemExtractor1:InfoItemExtractor - - - - - - - - itemExtractor2:InfoItemExtractor - - - - - - - - itemExtractor3:InfoItemExtractor - - - - - - - + + + + + + + + :InfoItemsCollector + + + + + + + + itemExtractor1:InfoItemExtractor + + + + + + + + itemExtractor2:InfoItemExtractor + + + + + + + + itemExtractor3:InfoItemExtractor + + + + + + + diff --git a/img/check_path.png b/img/check_path.png old mode 100755 new mode 100644 diff --git a/img/could_not_decrypt.png b/img/could_not_decrypt.png old mode 100755 new mode 100644 diff --git a/img/draft_name.png b/img/draft_name.png old mode 100755 new mode 100644 diff --git a/img/favicon.ico b/img/favicon.ico old mode 100755 new mode 100644 diff --git a/img/feature_branch.svg b/img/feature_branch.svg old mode 100755 new mode 100644 index 77c0d5d..7858eec --- a/img/feature_branch.svg +++ b/img/feature_branch.svg @@ -1,149 +1,149 @@ - - - - - - image/svg+xml - - - - - - - - - - dev - - - - - - - - - - - feature_xyz - - - - - - - - - - - - - - + + + + + + image/svg+xml + + + + + + + + + + dev + + + + + + + + + + + feature_xyz + + + + + + + + + + + + + + diff --git a/img/hotfix_branch.svg b/img/hotfix_branch.svg old mode 100755 new mode 100644 index 119c55b..a0517f1 --- a/img/hotfix_branch.svg +++ b/img/hotfix_branch.svg @@ -1,160 +1,160 @@ - - - - - - image/svg+xml - - - - - - - - - - master - - - - - - - - - - - hotfix - - - - - - - - - - - - - - - - - + + + + + + image/svg+xml + + + + + + + + + + master + + + + + + + + + + + hotfix + + + + + + + + + + + + + + + + + diff --git a/img/jitpack_fail.png b/img/jitpack_fail.png old mode 100755 new mode 100644 diff --git a/img/kde_in_a_nutshell.jpg b/img/kde_in_a_nutshell.jpg old mode 100755 new mode 100644 diff --git a/img/merge_into_dev.svg b/img/merge_into_dev.svg old mode 100755 new mode 100644 index 78235d4..f837f1b --- a/img/merge_into_dev.svg +++ b/img/merge_into_dev.svg @@ -1,210 +1,210 @@ - - - - - - image/svg+xml - - - - - - - - - - dev - - - - - - - - - - - feature_xyz - - - - - - - - - - - - - - - - - - - - - PULL REQUEST - Do QA/Codereview here - - - - - - - - - + + + + + + image/svg+xml + + + + + + + + + + dev + + + + + + + + + + + feature_xyz + + + + + + + + + + + + + + + + + + + + + PULL REQUEST + Do QA/Codereview here + + + + + + + + + diff --git a/img/onedoes.jpg b/img/onedoes.jpg old mode 100755 new mode 100644 diff --git a/img/prepare_tests_passed.png b/img/prepare_tests_passed.png old mode 100755 new mode 100644 diff --git a/img/rebase_back_hotfix.svg b/img/rebase_back_hotfix.svg old mode 100755 new mode 100644 index 2aad4a3..c968d5d --- a/img/rebase_back_hotfix.svg +++ b/img/rebase_back_hotfix.svg @@ -1,310 +1,310 @@ - - - - - - image/svg+xml - - - - - - - - - - dev - - - - - - - - - - - - - - - - - - - - - - - hotfix - - - master - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - REBASE - - - - - - - - - + + + + + + image/svg+xml + + + + + + + + + + dev + + + + + + + + + + + + + + + + + + + + + + + hotfix + + + master + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + REBASE + + + + + + + + + diff --git a/img/rebase_back_release.svg b/img/rebase_back_release.svg old mode 100755 new mode 100644 index 6679e02..1410a18 --- a/img/rebase_back_release.svg +++ b/img/rebase_back_release.svg @@ -1,338 +1,338 @@ - - - - - - image/svg+xml - - - - - - - - - - dev - - - - - - - - - - - - - - - - - - - - - - - release_x.y.z - - - master - - - - - - - - - - - - - - - - quickfix - - - - - - - - - - - - - - - - - - - - - - - - - - - - REBASE - - - - - - + + + + + + image/svg+xml + + + + + + + + + + dev + + + + + + + + + + + + + + + + + + + + + + + release_x.y.z + + + master + + + + + + + + + + + + + + + + quickfix + + + + + + + + + + + + + + + + + + + + + + + + + + + + REBASE + + + + + + diff --git a/img/release_branch.svg b/img/release_branch.svg old mode 100755 new mode 100644 index d1561ae..d0803f4 --- a/img/release_branch.svg +++ b/img/release_branch.svg @@ -1,270 +1,270 @@ - - - - - - image/svg+xml - - - - - - - - - - dev - - - - - - - - - - - - - - - - - - - - - - - release_x.y.z - - - master - - - - - - - - - - - - - - - - - - - - quickfix - - - PR from release_x.y.z - to master - - - - - - - - + + + + + + image/svg+xml + + + + + + + + + + dev + + + + + + + + + + + + + + + + + + + + + + + release_x.y.z + + + master + + + + + + + + + + + + + + + + + + + + quickfix + + + PR from release_x.y.z + to master + + + + + + + + diff --git a/img/select_gradle.png b/img/select_gradle.png old mode 100755 new mode 100644 diff --git a/img/select_gradle_wrapper.png b/img/select_gradle_wrapper.png old mode 100755 new mode 100644 diff --git a/img/sync_ok.png b/img/sync_ok.png old mode 100755 new mode 100644 diff --git a/img/termux_files.png b/img/termux_files.png old mode 100755 new mode 100644 diff --git a/index.html b/index.html old mode 100755 new mode 100644 index d5b710a..6e7001f --- a/index.html +++ b/index.html @@ -198,5 +198,5 @@ It focuses on making it possible for the creator of a scraper for a streaming se diff --git a/js/highlight.min.js b/js/highlight.min.js old mode 100755 new mode 100644 index 4cc1f9c..0a10a57 --- a/js/highlight.min.js +++ b/js/highlight.min.js @@ -1,3 +1,3 @@ -/*! highlight.js v9.12.0 | BSD3 License | git.io/hljslicense */ -!function(e){var t="object"==typeof window&&window||"object"==typeof self&&self;"undefined"!=typeof exports?e(exports):t&&(t.hljs=e({}),"function"==typeof define&&define.amd&&define([],function(){return t.hljs}))}(function(e){function t(e){return e.replace(/&/g,"&").replace(//g,">")}function r(e){return e.nodeName.toLowerCase()}function a(e,t){var r=e&&e.exec(t);return r&&0===r.index}function n(e){return E.test(e)}function i(e){var t,r,a,i,s=e.className+" ";if(s+=e.parentNode?e.parentNode.className:"",r=M.exec(s))return w(r[1])?r[1]:"no-highlight";for(s=s.split(/\s+/),t=0,a=s.length;a>t;t++)if(i=s[t],n(i)||w(i))return i}function s(e){var t,r={},a=Array.prototype.slice.call(arguments,1);for(t in e)r[t]=e[t];return a.forEach(function(e){for(t in e)r[t]=e[t]}),r}function c(e){var t=[];return function a(e,n){for(var i=e.firstChild;i;i=i.nextSibling)3===i.nodeType?n+=i.nodeValue.length:1===i.nodeType&&(t.push({event:"start",offset:n,node:i}),n=a(i,n),r(i).match(/br|hr|img|input/)||t.push({event:"stop",offset:n,node:i}));return n}(e,0),t}function o(e,a,n){function i(){return e.length&&a.length?e[0].offset!==a[0].offset?e[0].offset"}function c(e){u+=""}function o(e){("start"===e.event?s:c)(e.node)}for(var l=0,u="",d=[];e.length||a.length;){var b=i();if(u+=t(n.substring(l,b[0].offset)),l=b[0].offset,b===e){d.reverse().forEach(c);do o(b.splice(0,1)[0]),b=i();while(b===e&&b.length&&b[0].offset===l);d.reverse().forEach(s)}else"start"===b[0].event?d.push(b[0].node):d.pop(),o(b.splice(0,1)[0])}return u+t(n.substr(l))}function l(e){return e.v&&!e.cached_variants&&(e.cached_variants=e.v.map(function(t){return s(e,{v:null},t)})),e.cached_variants||e.eW&&[s(e)]||[e]}function u(e){function t(e){return e&&e.source||e}function r(r,a){return new RegExp(t(r),"m"+(e.cI?"i":"")+(a?"g":""))}function a(n,i){if(!n.compiled){if(n.compiled=!0,n.k=n.k||n.bK,n.k){var s={},c=function(t,r){e.cI&&(r=r.toLowerCase()),r.split(" ").forEach(function(e){var r=e.split("|");s[r[0]]=[t,r[1]?Number(r[1]):1]})};"string"==typeof n.k?c("keyword",n.k):k(n.k).forEach(function(e){c(e,n.k[e])}),n.k=s}n.lR=r(n.l||/\w+/,!0),i&&(n.bK&&(n.b="\\b("+n.bK.split(" ").join("|")+")\\b"),n.b||(n.b=/\B|\b/),n.bR=r(n.b),n.e||n.eW||(n.e=/\B|\b/),n.e&&(n.eR=r(n.e)),n.tE=t(n.e)||"",n.eW&&i.tE&&(n.tE+=(n.e?"|":"")+i.tE)),n.i&&(n.iR=r(n.i)),null==n.r&&(n.r=1),n.c||(n.c=[]),n.c=Array.prototype.concat.apply([],n.c.map(function(e){return l("self"===e?n:e)})),n.c.forEach(function(e){a(e,n)}),n.starts&&a(n.starts,i);var o=n.c.map(function(e){return e.bK?"\\.?("+e.b+")\\.?":e.b}).concat([n.tE,n.i]).map(t).filter(Boolean);n.t=o.length?r(o.join("|"),!0):{exec:function(){return null}}}}a(e)}function d(e,r,n,i){function s(e,t){var r,n;for(r=0,n=t.c.length;n>r;r++)if(a(t.c[r].bR,e))return t.c[r]}function c(e,t){if(a(e.eR,t)){for(;e.endsParent&&e.parent;)e=e.parent;return e}return e.eW?c(e.parent,t):void 0}function o(e,t){return!n&&a(t.iR,e)}function l(e,t){var r=v.cI?t[0].toLowerCase():t[0];return e.k.hasOwnProperty(r)&&e.k[r]}function p(e,t,r,a){var n=a?"":L.classPrefix,i='',i+t+s}function m(){var e,r,a,n;if(!N.k)return t(E);for(n="",r=0,N.lR.lastIndex=0,a=N.lR.exec(E);a;)n+=t(E.substring(r,a.index)),e=l(N,a),e?(M+=e[1],n+=p(e[0],t(a[0]))):n+=t(a[0]),r=N.lR.lastIndex,a=N.lR.exec(E);return n+t(E.substr(r))}function f(){var e="string"==typeof N.sL;if(e&&!x[N.sL])return t(E);var r=e?d(N.sL,E,!0,k[N.sL]):b(E,N.sL.length?N.sL:void 0);return N.r>0&&(M+=r.r),e&&(k[N.sL]=r.top),p(r.language,r.value,!1,!0)}function g(){C+=null!=N.sL?f():m(),E=""}function _(e){C+=e.cN?p(e.cN,"",!0):"",N=Object.create(e,{parent:{value:N}})}function h(e,t){if(E+=e,null==t)return g(),0;var r=s(t,N);if(r)return r.skip?E+=t:(r.eB&&(E+=t),g(),r.rB||r.eB||(E=t)),_(r,t),r.rB?0:t.length;var a=c(N,t);if(a){var n=N;n.skip?E+=t:(n.rE||n.eE||(E+=t),g(),n.eE&&(E=t));do N.cN&&(C+=R),N.skip||(M+=N.r),N=N.parent;while(N!==a.parent);return a.starts&&_(a.starts,""),n.rE?0:t.length}if(o(t,N))throw new Error('Illegal lexeme "'+t+'" for mode "'+(N.cN||"")+'"');return E+=t,t.length||1}var v=w(e);if(!v)throw new Error('Unknown language: "'+e+'"');u(v);var y,N=i||v,k={},C="";for(y=N;y!==v;y=y.parent)y.cN&&(C=p(y.cN,"",!0)+C);var E="",M=0;try{for(var B,S,$=0;;){if(N.t.lastIndex=$,B=N.t.exec(r),!B)break;S=h(r.substring($,B.index),B[0]),$=B.index+S}for(h(r.substr($)),y=N;y.parent;y=y.parent)y.cN&&(C+=R);return{r:M,value:C,language:e,top:N}}catch(A){if(A.message&&-1!==A.message.indexOf("Illegal"))return{r:0,value:t(r)};throw A}}function b(e,r){r=r||L.languages||k(x);var a={r:0,value:t(e)},n=a;return r.filter(w).forEach(function(t){var r=d(t,e,!1);r.language=t,r.r>n.r&&(n=r),r.r>a.r&&(n=a,a=r)}),n.language&&(a.second_best=n),a}function p(e){return L.tabReplace||L.useBR?e.replace(B,function(e,t){return L.useBR&&"\n"===e?"
    ":L.tabReplace?t.replace(/\t/g,L.tabReplace):""}):e}function m(e,t,r){var a=t?C[t]:r,n=[e.trim()];return e.match(/\bhljs\b/)||n.push("hljs"),-1===e.indexOf(a)&&n.push(a),n.join(" ").trim()}function f(e){var t,r,a,s,l,u=i(e);n(u)||(L.useBR?(t=document.createElementNS("http://www.w3.org/1999/xhtml","div"),t.innerHTML=e.innerHTML.replace(/\n/g,"").replace(//g,"\n")):t=e,l=t.textContent,a=u?d(u,l,!0):b(l),r=c(t),r.length&&(s=document.createElementNS("http://www.w3.org/1999/xhtml","div"),s.innerHTML=a.value,a.value=o(r,c(s),l)),a.value=p(a.value),e.innerHTML=a.value,e.className=m(e.className,u,a.language),e.result={language:a.language,re:a.r},a.second_best&&(e.second_best={language:a.second_best.language,re:a.second_best.r}))}function g(e){L=s(L,e)}function _(){if(!_.called){_.called=!0;var e=document.querySelectorAll("pre code");N.forEach.call(e,f)}}function h(){addEventListener("DOMContentLoaded",_,!1),addEventListener("load",_,!1)}function v(t,r){var a=x[t]=r(e);a.aliases&&a.aliases.forEach(function(e){C[e]=t})}function y(){return k(x)}function w(e){return e=(e||"").toLowerCase(),x[e]||x[C[e]]}var N=[],k=Object.keys,x={},C={},E=/^(no-?highlight|plain|text)$/i,M=/\blang(?:uage)?-([\w-]+)\b/i,B=/((^(<[^>]+>|\t|)+|(?:\n)))/gm,R="
    ",L={classPrefix:"hljs-",tabReplace:null,useBR:!1,languages:void 0};return e.highlight=d,e.highlightAuto=b,e.fixMarkup=p,e.highlightBlock=f,e.configure=g,e.initHighlighting=_,e.initHighlightingOnLoad=h,e.registerLanguage=v,e.listLanguages=y,e.getLanguage=w,e.inherit=s,e.IR="[a-zA-Z]\\w*",e.UIR="[a-zA-Z_]\\w*",e.NR="\\b\\d+(\\.\\d+)?",e.CNR="(-?)(\\b0[xX][a-fA-F0-9]+|(\\b\\d+(\\.\\d*)?|\\.\\d+)([eE][-+]?\\d+)?)",e.BNR="\\b(0b[01]+)",e.RSR="!|!=|!==|%|%=|&|&&|&=|\\*|\\*=|\\+|\\+=|,|-|-=|/=|/|:|;|<<|<<=|<=|<|===|==|=|>>>=|>>=|>=|>>>|>>|>|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~",e.BE={b:"\\\\[\\s\\S]",r:0},e.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[e.BE]},e.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[e.BE]},e.PWM={b:/\b(a|an|the|are|I'm|isn't|don't|doesn't|won't|but|just|should|pretty|simply|enough|gonna|going|wtf|so|such|will|you|your|they|like|more)\b/},e.C=function(t,r,a){var n=e.inherit({cN:"comment",b:t,e:r,c:[]},a||{});return n.c.push(e.PWM),n.c.push({cN:"doctag",b:"(?:TODO|FIXME|NOTE|BUG|XXX):",r:0}),n},e.CLCM=e.C("//","$"),e.CBCM=e.C("/\\*","\\*/"),e.HCM=e.C("#","$"),e.NM={cN:"number",b:e.NR,r:0},e.CNM={cN:"number",b:e.CNR,r:0},e.BNM={cN:"number",b:e.BNR,r:0},e.CSSNM={cN:"number",b:e.NR+"(%|em|ex|ch|rem|vw|vh|vmin|vmax|cm|mm|in|pt|pc|px|deg|grad|rad|turn|s|ms|Hz|kHz|dpi|dpcm|dppx)?",r:0},e.RM={cN:"regexp",b:/\//,e:/\/[gimuy]*/,i:/\n/,c:[e.BE,{b:/\[/,e:/\]/,r:0,c:[e.BE]}]},e.TM={cN:"title",b:e.IR,r:0},e.UTM={cN:"title",b:e.UIR,r:0},e.METHOD_GUARD={b:"\\.\\s*"+e.UIR,r:0},e.registerLanguage("apache",function(e){var t={cN:"number",b:"[\\$%]\\d+"};return{aliases:["apacheconf"],cI:!0,c:[e.HCM,{cN:"section",b:""},{cN:"attribute",b:/\w+/,r:0,k:{nomarkup:"order deny allow setenv rewriterule rewriteengine rewritecond documentroot sethandler errordocument loadmodule options header listen serverroot servername"},starts:{e:/$/,r:0,k:{literal:"on off all"},c:[{cN:"meta",b:"\\s\\[",e:"\\]$"},{cN:"variable",b:"[\\$%]\\{",e:"\\}",c:["self",t]},t,e.QSM]}}],i:/\S/}}),e.registerLanguage("bash",function(e){var t={cN:"variable",v:[{b:/\$[\w\d#@][\w\d_]*/},{b:/\$\{(.*?)}/}]},r={cN:"string",b:/"/,e:/"/,c:[e.BE,t,{cN:"variable",b:/\$\(/,e:/\)/,c:[e.BE]}]},a={cN:"string",b:/'/,e:/'/};return{aliases:["sh","zsh"],l:/\b-?[a-z\._]+\b/,k:{keyword:"if then else elif fi for while in do done case esac function",literal:"true false",built_in:"break cd continue eval exec exit export getopts hash pwd readonly return shift test times trap umask unset alias bind builtin caller command declare echo enable help let local logout mapfile printf read readarray source type typeset ulimit unalias set shopt autoload bg bindkey bye cap chdir clone comparguments compcall compctl compdescribe compfiles compgroups compquote comptags comptry compvalues dirs disable disown echotc echoti emulate fc fg float functions getcap getln history integer jobs kill limit log noglob popd print pushd pushln rehash sched setcap setopt stat suspend ttyctl unfunction unhash unlimit unsetopt vared wait whence where which zcompile zformat zftp zle zmodload zparseopts zprof zpty zregexparse zsocket zstyle ztcp",_:"-ne -eq -lt -gt -f -d -e -s -l -a"},c:[{cN:"meta",b:/^#![^\n]+sh\s*$/,r:10},{cN:"function",b:/\w[\w\d_]*\s*\(\s*\)\s*\{/,rB:!0,c:[e.inherit(e.TM,{b:/\w[\w\d_]*/})],r:0},e.HCM,r,a,t]}}),e.registerLanguage("coffeescript",function(e){var t={keyword:"in if for while finally new do return else break catch instanceof throw try this switch continue typeof delete debugger super yield import export from as default await then unless until loop of by when and or is isnt not",literal:"true false null undefined yes no on off",built_in:"npm require console print module global window document"},r="[A-Za-z$_][0-9A-Za-z$_]*",a={cN:"subst",b:/#\{/,e:/}/,k:t},n=[e.BNM,e.inherit(e.CNM,{starts:{e:"(\\s*/)?",r:0}}),{cN:"string",v:[{b:/'''/,e:/'''/,c:[e.BE]},{b:/'/,e:/'/,c:[e.BE]},{b:/"""/,e:/"""/,c:[e.BE,a]},{b:/"/,e:/"/,c:[e.BE,a]}]},{cN:"regexp",v:[{b:"///",e:"///",c:[a,e.HCM]},{b:"//[gim]*",r:0},{b:/\/(?![ *])(\\\/|.)*?\/[gim]*(?=\W|$)/}]},{b:"@"+r},{sL:"javascript",eB:!0,eE:!0,v:[{b:"```",e:"```"},{b:"`",e:"`"}]}];a.c=n;var i=e.inherit(e.TM,{b:r}),s="(\\(.*\\))?\\s*\\B[-=]>",c={cN:"params",b:"\\([^\\(]",rB:!0,c:[{b:/\(/,e:/\)/,k:t,c:["self"].concat(n)}]};return{aliases:["coffee","cson","iced"],k:t,i:/\/\*/,c:n.concat([e.C("###","###"),e.HCM,{cN:"function",b:"^\\s*"+r+"\\s*=\\s*"+s,e:"[-=]>",rB:!0,c:[i,c]},{b:/[:\(,=]\s*/,r:0,c:[{cN:"function",b:s,e:"[-=]>",rB:!0,c:[c]}]},{cN:"class",bK:"class",e:"$",i:/[:="\[\]]/,c:[{bK:"extends",eW:!0,i:/[:="\[\]]/,c:[i]},i]},{b:r+":",e:":",rB:!0,rE:!0,r:0}])}}),e.registerLanguage("cpp",function(e){var t={cN:"keyword",b:"\\b[a-z\\d_]*_t\\b"},r={cN:"string",v:[{b:'(u8?|U)?L?"',e:'"',i:"\\n",c:[e.BE]},{b:'(u8?|U)?R"',e:'"',c:[e.BE]},{b:"'\\\\?.",e:"'",i:"."}]},a={cN:"number",v:[{b:"\\b(0b[01']+)"},{b:"(-?)\\b([\\d']+(\\.[\\d']*)?|\\.[\\d']+)(u|U|l|L|ul|UL|f|F|b|B)"},{b:"(-?)(\\b0[xX][a-fA-F0-9']+|(\\b[\\d']+(\\.[\\d']*)?|\\.[\\d']+)([eE][-+]?[\\d']+)?)"}],r:0},n={cN:"meta",b:/#\s*[a-z]+\b/,e:/$/,k:{"meta-keyword":"if else elif endif define undef warning error line pragma ifdef ifndef include"},c:[{b:/\\\n/,r:0},e.inherit(r,{cN:"meta-string"}),{cN:"meta-string",b:/<[^\n>]*>/,e:/$/,i:"\\n"},e.CLCM,e.CBCM]},i=e.IR+"\\s*\\(",s={keyword:"int float while private char catch import module export virtual operator sizeof dynamic_cast|10 typedef const_cast|10 const for static_cast|10 union namespace unsigned long volatile static protected bool template mutable if public friend do goto auto void enum else break extern using asm case typeid short reinterpret_cast|10 default double register explicit signed typename try this switch continue inline delete alignof constexpr decltype noexcept static_assert thread_local restrict _Bool complex _Complex _Imaginary atomic_bool atomic_char atomic_schar atomic_uchar atomic_short atomic_ushort atomic_int atomic_uint atomic_long atomic_ulong atomic_llong atomic_ullong new throw return and or not",built_in:"std string cin cout cerr clog stdin stdout stderr stringstream istringstream ostringstream auto_ptr deque list queue stack vector map set bitset multiset multimap unordered_set unordered_map unordered_multiset unordered_multimap array shared_ptr abort abs acos asin atan2 atan calloc ceil cosh cos exit exp fabs floor fmod fprintf fputs free frexp fscanf isalnum isalpha iscntrl isdigit isgraph islower isprint ispunct isspace isupper isxdigit tolower toupper labs ldexp log10 log malloc realloc memchr memcmp memcpy memset modf pow printf putchar puts scanf sinh sin snprintf sprintf sqrt sscanf strcat strchr strcmp strcpy strcspn strlen strncat strncmp strncpy strpbrk strrchr strspn strstr tanh tan vfprintf vprintf vsprintf endl initializer_list unique_ptr",literal:"true false nullptr NULL"},c=[t,e.CLCM,e.CBCM,a,r];return{aliases:["c","cc","h","c++","h++","hpp"],k:s,i:"",k:s,c:["self",t]},{b:e.IR+"::",k:s},{v:[{b:/=/,e:/;/},{b:/\(/,e:/\)/},{bK:"new throw return else",e:/;/}],k:s,c:c.concat([{b:/\(/,e:/\)/,k:s,c:c.concat(["self"]),r:0}]),r:0},{cN:"function",b:"("+e.IR+"[\\*&\\s]+)+"+i,rB:!0,e:/[{;=]/,eE:!0,k:s,i:/[^\w\s\*&]/,c:[{b:i,rB:!0,c:[e.TM],r:0},{cN:"params",b:/\(/,e:/\)/,k:s,r:0,c:[e.CLCM,e.CBCM,r,a,t]},e.CLCM,e.CBCM,n]},{cN:"class",bK:"class struct",e:/[{;:]/,c:[{b://,c:["self"]},e.TM]}]),exports:{preprocessor:n,strings:r,k:s}}}),e.registerLanguage("cs",function(e){var t={keyword:"abstract as base bool break byte case catch char checked const continue decimal default delegate do double enum event explicit extern finally fixed float for foreach goto if implicit in int interface internal is lock long nameof object operator out override params private protected public readonly ref sbyte sealed short sizeof stackalloc static string struct switch this try typeof uint ulong unchecked unsafe ushort using virtual void volatile while add alias ascending async await by descending dynamic equals from get global group into join let on orderby partial remove select set value var where yield",literal:"null false true"},r={cN:"string",b:'@"',e:'"',c:[{b:'""'}]},a=e.inherit(r,{i:/\n/}),n={cN:"subst",b:"{",e:"}",k:t},i=e.inherit(n,{i:/\n/}),s={cN:"string",b:/\$"/,e:'"',i:/\n/,c:[{b:"{{"},{b:"}}"},e.BE,i]},c={cN:"string",b:/\$@"/,e:'"',c:[{b:"{{"},{b:"}}"},{b:'""'},n]},o=e.inherit(c,{i:/\n/,c:[{b:"{{"},{b:"}}"},{b:'""'},i]});n.c=[c,s,r,e.ASM,e.QSM,e.CNM,e.CBCM],i.c=[o,s,a,e.ASM,e.QSM,e.CNM,e.inherit(e.CBCM,{i:/\n/})];var l={v:[c,s,r,e.ASM,e.QSM]},u=e.IR+"(<"+e.IR+"(\\s*,\\s*"+e.IR+")*>)?(\\[\\])?";return{aliases:["csharp"],k:t,i:/::/,c:[e.C("///","$",{rB:!0,c:[{cN:"doctag",v:[{b:"///",r:0},{b:""},{b:""}]}]}),e.CLCM,e.CBCM,{cN:"meta",b:"#",e:"$",k:{"meta-keyword":"if else elif endif define undef warning error line region endregion pragma checksum"}},l,e.CNM,{bK:"class interface",e:/[{;=]/,i:/[^\s:]/,c:[e.TM,e.CLCM,e.CBCM]},{bK:"namespace",e:/[{;=]/,i:/[^\s:]/,c:[e.inherit(e.TM,{b:"[a-zA-Z](\\.?\\w)*"}),e.CLCM,e.CBCM]},{cN:"meta",b:"^\\s*\\[",eB:!0,e:"\\]",eE:!0,c:[{cN:"meta-string",b:/"/,e:/"/}]},{bK:"new return throw await else",r:0},{cN:"function",b:"("+u+"\\s+)+"+e.IR+"\\s*\\(",rB:!0,e:/[{;=]/,eE:!0,k:t,c:[{b:e.IR+"\\s*\\(",rB:!0,c:[e.TM],r:0},{cN:"params",b:/\(/,e:/\)/,eB:!0,eE:!0,k:t,r:0,c:[l,e.CNM,e.CBCM]},e.CLCM,e.CBCM]}]}}),e.registerLanguage("css",function(e){var t="[a-zA-Z-][a-zA-Z0-9_-]*",r={b:/[A-Z\_\.\-]+\s*:/,rB:!0,e:";",eW:!0,c:[{cN:"attribute",b:/\S/,e:":",eE:!0,starts:{eW:!0,eE:!0,c:[{b:/[\w-]+\(/,rB:!0,c:[{cN:"built_in",b:/[\w-]+/},{b:/\(/,e:/\)/,c:[e.ASM,e.QSM]}]},e.CSSNM,e.QSM,e.ASM,e.CBCM,{cN:"number",b:"#[0-9A-Fa-f]+"},{cN:"meta",b:"!important"}]}}]};return{cI:!0,i:/[=\/|'\$]/,c:[e.CBCM,{cN:"selector-id",b:/#[A-Za-z0-9_-]+/},{cN:"selector-class",b:/\.[A-Za-z0-9_-]+/},{cN:"selector-attr",b:/\[/,e:/\]/,i:"$"},{cN:"selector-pseudo",b:/:(:)?[a-zA-Z0-9\_\-\+\(\)"'.]+/},{b:"@(font-face|page)",l:"[a-z-]+",k:"font-face page"},{b:"@",e:"[{;]",i:/:/,c:[{cN:"keyword",b:/\w+/},{b:/\s/,eW:!0,eE:!0,r:0,c:[e.ASM,e.QSM,e.CSSNM]}]},{cN:"selector-tag",b:t,r:0},{b:"{",e:"}",i:/\S/,c:[e.CBCM,r]}]}}),e.registerLanguage("diff",function(e){return{aliases:["patch"],c:[{cN:"meta",r:10,v:[{b:/^@@ +\-\d+,\d+ +\+\d+,\d+ +@@$/},{b:/^\*\*\* +\d+,\d+ +\*\*\*\*$/},{b:/^\-\-\- +\d+,\d+ +\-\-\-\-$/}]},{cN:"comment",v:[{b:/Index: /,e:/$/},{b:/={3,}/,e:/$/},{b:/^\-{3}/,e:/$/},{b:/^\*{3} /,e:/$/},{b:/^\+{3}/,e:/$/},{b:/\*{5}/,e:/\*{5}$/}]},{cN:"addition",b:"^\\+",e:"$"},{cN:"deletion",b:"^\\-",e:"$"},{cN:"addition",b:"^\\!",e:"$"}]}}),e.registerLanguage("http",function(e){var t="HTTP/[0-9\\.]+";return{aliases:["https"],i:"\\S",c:[{b:"^"+t,e:"$",c:[{cN:"number",b:"\\b\\d{3}\\b"}]},{b:"^[A-Z]+ (.*?) "+t+"$",rB:!0,e:"$",c:[{cN:"string",b:" ",e:" ",eB:!0,eE:!0},{b:t},{cN:"keyword",b:"[A-Z]+"}]},{cN:"attribute",b:"^\\w",e:": ",eE:!0,i:"\\n|\\s|=",starts:{e:"$",r:0}},{b:"\\n\\n",starts:{sL:[],eW:!0}}]}}),e.registerLanguage("ini",function(e){var t={cN:"string",c:[e.BE],v:[{b:"'''",e:"'''",r:10},{b:'"""',e:'"""',r:10},{b:'"',e:'"'},{b:"'",e:"'"}]};return{aliases:["toml"],cI:!0,i:/\S/,c:[e.C(";","$"),e.HCM,{cN:"section",b:/^\s*\[+/,e:/\]+/},{b:/^[a-z0-9\[\]_-]+\s*=\s*/,e:"$",rB:!0,c:[{cN:"attr",b:/[a-z0-9\[\]_-]+/},{b:/=/,eW:!0,r:0,c:[{cN:"literal",b:/\bon|off|true|false|yes|no\b/},{cN:"variable",v:[{b:/\$[\w\d"][\w\d_]*/},{b:/\$\{(.*?)}/}]},t,{cN:"number",b:/([\+\-]+)?[\d]+_[\d_]+/},e.NM]}]}]}}),e.registerLanguage("java",function(e){var t="[À-ʸa-zA-Z_$][À-ʸa-zA-Z_$0-9]*",r=t+"(<"+t+"(\\s*,\\s*"+t+")*>)?",a="false synchronized int abstract float private char boolean static null if const for true while long strictfp finally protected import native final void enum else break transient catch instanceof byte super volatile case assert short package default double public try this switch continue throws protected public private module requires exports do",n="\\b(0[bB]([01]+[01_]+[01]+|[01]+)|0[xX]([a-fA-F0-9]+[a-fA-F0-9_]+[a-fA-F0-9]+|[a-fA-F0-9]+)|(([\\d]+[\\d_]+[\\d]+|[\\d]+)(\\.([\\d]+[\\d_]+[\\d]+|[\\d]+))?|\\.([\\d]+[\\d_]+[\\d]+|[\\d]+))([eE][-+]?\\d+)?)[lLfF]?",i={cN:"number",b:n,r:0};return{aliases:["jsp"],k:a,i:/<\/|#/,c:[e.C("/\\*\\*","\\*/",{r:0,c:[{b:/\w+@/,r:0},{cN:"doctag",b:"@[A-Za-z]+"}]}),e.CLCM,e.CBCM,e.ASM,e.QSM,{cN:"class",bK:"class interface",e:/[{;=]/,eE:!0,k:"class interface",i:/[:"\[\]]/,c:[{bK:"extends implements"},e.UTM]},{bK:"new throw return else",r:0},{cN:"function",b:"("+r+"\\s+)+"+e.UIR+"\\s*\\(",rB:!0,e:/[{;=]/,eE:!0,k:a,c:[{b:e.UIR+"\\s*\\(",rB:!0,r:0,c:[e.UTM]},{cN:"params",b:/\(/,e:/\)/,k:a,r:0,c:[e.ASM,e.QSM,e.CNM,e.CBCM]},e.CLCM,e.CBCM]},i,{cN:"meta",b:"@[A-Za-z]+"}]}}),e.registerLanguage("javascript",function(e){var t="[A-Za-z$_][0-9A-Za-z$_]*",r={keyword:"in of if for while finally var new function do return void else break catch instanceof with throw case default try this switch continue typeof delete let yield const export super debugger as async await static import from as",literal:"true false null undefined NaN Infinity",built_in:"eval isFinite isNaN parseFloat parseInt decodeURI decodeURIComponent encodeURI encodeURIComponent escape unescape Object Function Boolean Error EvalError InternalError RangeError ReferenceError StopIteration SyntaxError TypeError URIError Number Math Date String RegExp Array Float32Array Float64Array Int16Array Int32Array Int8Array Uint16Array Uint32Array Uint8Array Uint8ClampedArray ArrayBuffer DataView JSON Intl arguments require module console window document Symbol Set Map WeakSet WeakMap Proxy Reflect Promise"},a={cN:"number",v:[{b:"\\b(0[bB][01]+)"},{b:"\\b(0[oO][0-7]+)"},{b:e.CNR}],r:0},n={cN:"subst",b:"\\$\\{",e:"\\}",k:r,c:[]},i={cN:"string",b:"`",e:"`",c:[e.BE,n]};n.c=[e.ASM,e.QSM,i,a,e.RM];var s=n.c.concat([e.CBCM,e.CLCM]);return{aliases:["js","jsx"],k:r,c:[{cN:"meta",r:10,b:/^\s*['"]use (strict|asm)['"]/},{cN:"meta",b:/^#!/,e:/$/},e.ASM,e.QSM,i,e.CLCM,e.CBCM,a,{b:/[{,]\s*/,r:0,c:[{b:t+"\\s*:",rB:!0,r:0,c:[{cN:"attr",b:t,r:0}]}]},{b:"("+e.RSR+"|\\b(case|return|throw)\\b)\\s*",k:"return throw case",c:[e.CLCM,e.CBCM,e.RM,{cN:"function",b:"(\\(.*?\\)|"+t+")\\s*=>",rB:!0,e:"\\s*=>",c:[{cN:"params",v:[{b:t},{b:/\(\s*\)/},{b:/\(/,e:/\)/,eB:!0,eE:!0,k:r,c:s}]}]},{b://,sL:"xml",c:[{b:/<\w+\s*\/>/,skip:!0},{b:/<\w+/,e:/(\/\w+|\w+\/)>/,skip:!0,c:[{b:/<\w+\s*\/>/,skip:!0},"self"]}]}],r:0},{cN:"function",bK:"function",e:/\{/,eE:!0,c:[e.inherit(e.TM,{b:t}),{cN:"params",b:/\(/,e:/\)/,eB:!0,eE:!0,c:s}],i:/\[|%/},{b:/\$[(.]/},e.METHOD_GUARD,{cN:"class",bK:"class",e:/[{;=]/,eE:!0,i:/[:"\[\]]/,c:[{bK:"extends"},e.UTM]},{bK:"constructor",e:/\{/,eE:!0}],i:/#(?!!)/}}),e.registerLanguage("json",function(e){var t={literal:"true false null"},r=[e.QSM,e.CNM],a={e:",",eW:!0,eE:!0,c:r,k:t},n={b:"{",e:"}",c:[{cN:"attr",b:/"/,e:/"/,c:[e.BE],i:"\\n"},e.inherit(a,{b:/:/})],i:"\\S"},i={b:"\\[",e:"\\]",c:[e.inherit(a)],i:"\\S"};return r.splice(r.length,0,n,i),{c:r,k:t,i:"\\S"}}),e.registerLanguage("makefile",function(e){var t={cN:"variable",v:[{b:"\\$\\("+e.UIR+"\\)",c:[e.BE]},{b:/\$[@%`]+/}]}]}]};return{aliases:["html","xhtml","rss","atom","xjb","xsd","xsl","plist"],cI:!0,c:[{cN:"meta",b:"",r:10,c:[{b:"\\[",e:"\\]"}]},e.C("",{r:10}),{b:"<\\!\\[CDATA\\[",e:"\\]\\]>",r:10},{b:/<\?(php)?/,e:/\?>/,sL:"php",c:[{b:"/\\*",e:"\\*/",skip:!0}]},{cN:"tag",b:"|$)",e:">",k:{name:"style"},c:[r],starts:{e:"",rE:!0,sL:["css","xml"]}},{cN:"tag",b:"|$)",e:">",k:{name:"script"},c:[r],starts:{e:"",rE:!0,sL:["actionscript","javascript","handlebars","xml"]}},{cN:"meta",v:[{b:/<\?xml/,e:/\?>/,r:10},{b:/<\?\w+/,e:/\?>/}]},{cN:"tag",b:"",c:[{cN:"name",b:/[^\/><\s]+/,r:0},r]}]}}),e.registerLanguage("markdown",function(e){return{aliases:["md","mkdown","mkd"],c:[{cN:"section",v:[{b:"^#{1,6}",e:"$"},{b:"^.+?\\n[=-]{2,}$"}]},{b:"<",e:">",sL:"xml",r:0},{cN:"bullet",b:"^([*+-]|(\\d+\\.))\\s+"},{cN:"strong",b:"[*_]{2}.+?[*_]{2}"},{cN:"emphasis",v:[{b:"\\*.+?\\*"},{b:"_.+?_",r:0}]},{cN:"quote",b:"^>\\s+",e:"$"},{cN:"code",v:[{b:"^```w*s*$",e:"^```s*$"},{b:"`.+?`"},{b:"^( {4}| )",e:"$",r:0}]},{b:"^[-\\*]{3,}",e:"$"},{b:"\\[.+?\\][\\(\\[].*?[\\)\\]]",rB:!0,c:[{cN:"string",b:"\\[",e:"\\]",eB:!0,rE:!0,r:0},{cN:"link",b:"\\]\\(",e:"\\)",eB:!0,eE:!0},{cN:"symbol",b:"\\]\\[",e:"\\]",eB:!0,eE:!0}],r:10},{b:/^\[[^\n]+\]:/,rB:!0,c:[{cN:"symbol",b:/\[/,e:/\]/,eB:!0,eE:!0},{cN:"link",b:/:\s*/,e:/$/,eB:!0}]}]}}),e.registerLanguage("nginx",function(e){var t={cN:"variable",v:[{b:/\$\d+/},{b:/\$\{/,e:/}/},{b:"[\\$\\@]"+e.UIR}]},r={eW:!0,l:"[a-z/_]+",k:{literal:"on off yes no true false none blocked debug info notice warn error crit select break last permanent redirect kqueue rtsig epoll poll /dev/poll"},r:0,i:"=>",c:[e.HCM,{cN:"string",c:[e.BE,t],v:[{b:/"/,e:/"/},{b:/'/,e:/'/}]},{b:"([a-z]+):/",e:"\\s",eW:!0,eE:!0,c:[t]},{cN:"regexp",c:[e.BE,t],v:[{b:"\\s\\^",e:"\\s|{|;",rE:!0},{b:"~\\*?\\s+",e:"\\s|{|;",rE:!0},{b:"\\*(\\.[a-z\\-]+)+"},{b:"([a-z\\-]+\\.)+\\*"}]},{cN:"number",b:"\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(:\\d{1,5})?\\b"},{cN:"number",b:"\\b\\d+[kKmMgGdshdwy]*\\b",r:0},t]};return{aliases:["nginxconf"],c:[e.HCM,{b:e.UIR+"\\s+{",rB:!0,e:"{",c:[{cN:"section",b:e.UIR}],r:0},{b:e.UIR+"\\s",e:";|{",rB:!0,c:[{cN:"attribute",b:e.UIR,starts:r}],r:0}],i:"[^\\s\\}]"}}),e.registerLanguage("objectivec",function(e){var t={cN:"built_in",b:"\\b(AV|CA|CF|CG|CI|CL|CM|CN|CT|MK|MP|MTK|MTL|NS|SCN|SK|UI|WK|XC)\\w+"},r={keyword:"int float while char export sizeof typedef const struct for union unsigned long volatile static bool mutable if do return goto void enum else break extern asm case short default double register explicit signed typename this switch continue wchar_t inline readonly assign readwrite self @synchronized id typeof nonatomic super unichar IBOutlet IBAction strong weak copy in out inout bycopy byref oneway __strong __weak __block __autoreleasing @private @protected @public @try @property @end @throw @catch @finally @autoreleasepool @synthesize @dynamic @selector @optional @required @encode @package @import @defs @compatibility_alias __bridge __bridge_transfer __bridge_retained __bridge_retain __covariant __contravariant __kindof _Nonnull _Nullable _Null_unspecified __FUNCTION__ __PRETTY_FUNCTION__ __attribute__ getter setter retain unsafe_unretained nonnull nullable null_unspecified null_resettable class instancetype NS_DESIGNATED_INITIALIZER NS_UNAVAILABLE NS_REQUIRES_SUPER NS_RETURNS_INNER_POINTER NS_INLINE NS_AVAILABLE NS_DEPRECATED NS_ENUM NS_OPTIONS NS_SWIFT_UNAVAILABLE NS_ASSUME_NONNULL_BEGIN NS_ASSUME_NONNULL_END NS_REFINED_FOR_SWIFT NS_SWIFT_NAME NS_SWIFT_NOTHROW NS_DURING NS_HANDLER NS_ENDHANDLER NS_VALUERETURN NS_VOIDRETURN",literal:"false true FALSE TRUE nil YES NO NULL",built_in:"BOOL dispatch_once_t dispatch_queue_t dispatch_sync dispatch_async dispatch_once"},a=/[a-zA-Z@][a-zA-Z0-9_]*/,n="@interface @class @protocol @implementation";return{aliases:["mm","objc","obj-c"],k:r,l:a,i:""}]}]},{cN:"class",b:"("+n.split(" ").join("|")+")\\b",e:"({|$)",eE:!0,k:n,l:a,c:[e.UTM]},{b:"\\."+e.UIR,r:0}]}}),e.registerLanguage("perl",function(e){var t="getpwent getservent quotemeta msgrcv scalar kill dbmclose undef lc ma syswrite tr send umask sysopen shmwrite vec qx utime local oct semctl localtime readpipe do return format read sprintf dbmopen pop getpgrp not getpwnam rewinddir qqfileno qw endprotoent wait sethostent bless s|0 opendir continue each sleep endgrent shutdown dump chomp connect getsockname die socketpair close flock exists index shmgetsub for endpwent redo lstat msgctl setpgrp abs exit select print ref gethostbyaddr unshift fcntl syscall goto getnetbyaddr join gmtime symlink semget splice x|0 getpeername recv log setsockopt cos last reverse gethostbyname getgrnam study formline endhostent times chop length gethostent getnetent pack getprotoent getservbyname rand mkdir pos chmod y|0 substr endnetent printf next open msgsnd readdir use unlink getsockopt getpriority rindex wantarray hex system getservbyport endservent int chr untie rmdir prototype tell listen fork shmread ucfirst setprotoent else sysseek link getgrgid shmctl waitpid unpack getnetbyname reset chdir grep split require caller lcfirst until warn while values shift telldir getpwuid my getprotobynumber delete and sort uc defined srand accept package seekdir getprotobyname semop our rename seek if q|0 chroot sysread setpwent no crypt getc chown sqrt write setnetent setpriority foreach tie sin msgget map stat getlogin unless elsif truncate exec keys glob tied closedirioctl socket readlink eval xor readline binmode setservent eof ord bind alarm pipe atan2 getgrent exp time push setgrent gt lt or ne m|0 break given say state when",r={cN:"subst",b:"[$@]\\{",e:"\\}",k:t},a={b:"->{",e:"}"},n={v:[{b:/\$\d/},{b:/[\$%@](\^\w\b|#\w+(::\w+)*|{\w+}|\w+(::\w*)*)/},{b:/[\$%@][^\s\w{]/,r:0}]},i=[e.BE,r,n],s=[n,e.HCM,e.C("^\\=\\w","\\=cut",{eW:!0}),a,{cN:"string",c:i,v:[{b:"q[qwxr]?\\s*\\(",e:"\\)",r:5},{b:"q[qwxr]?\\s*\\[",e:"\\]",r:5},{b:"q[qwxr]?\\s*\\{",e:"\\}",r:5},{b:"q[qwxr]?\\s*\\|",e:"\\|",r:5},{b:"q[qwxr]?\\s*\\<",e:"\\>",r:5},{b:"qw\\s+q",e:"q",r:5},{b:"'",e:"'",c:[e.BE]},{b:'"',e:'"'},{b:"`",e:"`",c:[e.BE]},{b:"{\\w+}",c:[],r:0},{b:"-?\\w+\\s*\\=\\>",c:[],r:0}]},{cN:"number",b:"(\\b0[0-7_]+)|(\\b0x[0-9a-fA-F_]+)|(\\b[1-9][0-9_]*(\\.[0-9_]+)?)|[0_]\\b",r:0},{b:"(\\/\\/|"+e.RSR+"|\\b(split|return|print|reverse|grep)\\b)\\s*",k:"split return print reverse grep",r:0,c:[e.HCM,{cN:"regexp",b:"(s|tr|y)/(\\\\.|[^/])*/(\\\\.|[^/])*/[a-z]*",r:10},{cN:"regexp",b:"(m|qr)?/",e:"/[a-z]*",c:[e.BE],r:0}]},{cN:"function",bK:"sub",e:"(\\s*\\(.*?\\))?[;{]",eE:!0,r:5,c:[e.TM]},{b:"-\\w\\b",r:0},{b:"^__DATA__$",e:"^__END__$",sL:"mojolicious",c:[{b:"^@@.*",e:"$",cN:"comment"}]}];return r.c=s,a.c=s,{aliases:["pl","pm"],l:/[\w\.]+/,k:t,c:s}}),e.registerLanguage("php",function(e){var t={b:"\\$+[a-zA-Z_-ÿ][a-zA-Z0-9_-ÿ]*"},r={cN:"meta",b:/<\?(php)?|\?>/},a={cN:"string",c:[e.BE,r],v:[{b:'b"',e:'"'},{b:"b'",e:"'"},e.inherit(e.ASM,{i:null}),e.inherit(e.QSM,{i:null})]},n={v:[e.BNM,e.CNM]};return{aliases:["php3","php4","php5","php6"],cI:!0,k:"and include_once list abstract global private echo interface as static endswitch array null if endwhile or const for endforeach self var while isset public protected exit foreach throw elseif include __FILE__ empty require_once do xor return parent clone use __CLASS__ __LINE__ else break print eval new catch __METHOD__ case exception default die require __FUNCTION__ enddeclare final try switch continue endfor endif declare unset true false trait goto instanceof insteadof __DIR__ __NAMESPACE__ yield finally",c:[e.HCM,e.C("//","$",{c:[r]}),e.C("/\\*","\\*/",{c:[{cN:"doctag",b:"@[A-Za-z]+"}]}),e.C("__halt_compiler.+?;",!1,{eW:!0,k:"__halt_compiler",l:e.UIR}),{cN:"string",b:/<<<['"]?\w+['"]?$/,e:/^\w+;?$/,c:[e.BE,{cN:"subst",v:[{b:/\$\w+/},{b:/\{\$/,e:/\}/}]}]},r,{cN:"keyword",b:/\$this\b/},t,{b:/(::|->)+[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*/},{cN:"function",bK:"function",e:/[;{]/,eE:!0,i:"\\$|\\[|%",c:[e.UTM,{cN:"params",b:"\\(",e:"\\)",c:["self",t,e.CBCM,a,n]}]},{cN:"class",bK:"class interface",e:"{",eE:!0,i:/[:\(\$"]/,c:[{bK:"extends implements"},e.UTM]},{bK:"namespace",e:";",i:/[\.']/,c:[e.UTM]},{bK:"use",e:";",c:[e.UTM]},{b:"=>"},a,n]}}),e.registerLanguage("python",function(e){var t={keyword:"and elif is global as in if from raise for except finally print import pass return exec else break not with class assert yield try while continue del or def lambda async await nonlocal|10 None True False",built_in:"Ellipsis NotImplemented"},r={cN:"meta",b:/^(>>>|\.\.\.) /},a={cN:"subst",b:/\{/,e:/\}/,k:t,i:/#/},n={cN:"string",c:[e.BE],v:[{b:/(u|b)?r?'''/,e:/'''/,c:[r],r:10},{b:/(u|b)?r?"""/,e:/"""/,c:[r],r:10},{b:/(fr|rf|f)'''/,e:/'''/,c:[r,a]},{b:/(fr|rf|f)"""/,e:/"""/,c:[r,a]},{b:/(u|r|ur)'/,e:/'/,r:10},{b:/(u|r|ur)"/,e:/"/,r:10},{b:/(b|br)'/,e:/'/},{b:/(b|br)"/,e:/"/},{b:/(fr|rf|f)'/,e:/'/,c:[a]},{b:/(fr|rf|f)"/,e:/"/,c:[a]},e.ASM,e.QSM]},i={cN:"number",r:0,v:[{b:e.BNR+"[lLjJ]?"},{b:"\\b(0o[0-7]+)[lLjJ]?"},{b:e.CNR+"[lLjJ]?"}]},s={cN:"params",b:/\(/,e:/\)/,c:["self",r,i,n]};return a.c=[n,i,r],{aliases:["py","gyp"],k:t,i:/(<\/|->|\?)|=>/,c:[r,i,n,e.HCM,{v:[{cN:"function",bK:"def"},{cN:"class",bK:"class"}],e:/:/,i:/[${=;\n,]/,c:[e.UTM,s,{b:/->/,eW:!0,k:"None"}]},{cN:"meta",b:/^[\t ]*@/,e:/$/},{b:/\b(print|exec)\(/}]}}),e.registerLanguage("ruby",function(e){ -var t="[a-zA-Z_]\\w*[!?=]?|[-+~]\\@|<<|>>|=~|===?|<=>|[<>]=?|\\*\\*|[-/+%^&*~`|]|\\[\\]=?",r={keyword:"and then defined module in return redo if BEGIN retry end for self when next until do begin unless END rescue else break undef not super class case require yield alias while ensure elsif or include attr_reader attr_writer attr_accessor",literal:"true false nil"},a={cN:"doctag",b:"@[A-Za-z]+"},n={b:"#<",e:">"},i=[e.C("#","$",{c:[a]}),e.C("^\\=begin","^\\=end",{c:[a],r:10}),e.C("^__END__","\\n$")],s={cN:"subst",b:"#\\{",e:"}",k:r},c={cN:"string",c:[e.BE,s],v:[{b:/'/,e:/'/},{b:/"/,e:/"/},{b:/`/,e:/`/},{b:"%[qQwWx]?\\(",e:"\\)"},{b:"%[qQwWx]?\\[",e:"\\]"},{b:"%[qQwWx]?{",e:"}"},{b:"%[qQwWx]?<",e:">"},{b:"%[qQwWx]?/",e:"/"},{b:"%[qQwWx]?%",e:"%"},{b:"%[qQwWx]?-",e:"-"},{b:"%[qQwWx]?\\|",e:"\\|"},{b:/\B\?(\\\d{1,3}|\\x[A-Fa-f0-9]{1,2}|\\u[A-Fa-f0-9]{4}|\\?\S)\b/},{b:/<<(-?)\w+$/,e:/^\s*\w+$/}]},o={cN:"params",b:"\\(",e:"\\)",endsParent:!0,k:r},l=[c,n,{cN:"class",bK:"class module",e:"$|;",i:/=/,c:[e.inherit(e.TM,{b:"[A-Za-z_]\\w*(::\\w+)*(\\?|\\!)?"}),{b:"<\\s*",c:[{b:"("+e.IR+"::)?"+e.IR}]}].concat(i)},{cN:"function",bK:"def",e:"$|;",c:[e.inherit(e.TM,{b:t}),o].concat(i)},{b:e.IR+"::"},{cN:"symbol",b:e.UIR+"(\\!|\\?)?:",r:0},{cN:"symbol",b:":(?!\\s)",c:[c,{b:t}],r:0},{cN:"number",b:"(\\b0[0-7_]+)|(\\b0x[0-9a-fA-F_]+)|(\\b[1-9][0-9_]*(\\.[0-9_]+)?)|[0_]\\b",r:0},{b:"(\\$\\W)|((\\$|\\@\\@?)(\\w+))"},{cN:"params",b:/\|/,e:/\|/,k:r},{b:"("+e.RSR+"|unless)\\s*",k:"unless",c:[n,{cN:"regexp",c:[e.BE,s],i:/\n/,v:[{b:"/",e:"/[a-z]*"},{b:"%r{",e:"}[a-z]*"},{b:"%r\\(",e:"\\)[a-z]*"},{b:"%r!",e:"![a-z]*"},{b:"%r\\[",e:"\\][a-z]*"}]}].concat(i),r:0}].concat(i);s.c=l,o.c=l;var u="[>?]>",d="[\\w#]+\\(\\w+\\):\\d+:\\d+>",b="(\\w+-)?\\d+\\.\\d+\\.\\d(p\\d+)?[^>]+>",p=[{b:/^\s*=>/,starts:{e:"$",c:l}},{cN:"meta",b:"^("+u+"|"+d+"|"+b+")",starts:{e:"$",c:l}}];return{aliases:["rb","gemspec","podspec","thor","irb"],k:r,i:/\/\*/,c:i.concat(p).concat(l)}}),e.registerLanguage("shell",function(e){return{aliases:["console"],c:[{cN:"meta",b:"^\\s{0,3}[\\w\\d\\[\\]()@-]*[>%$#]",starts:{e:"$",sL:"bash"}}]}}),e.registerLanguage("sql",function(e){var t=e.C("--","$");return{cI:!0,i:/[<>{}*#]/,c:[{bK:"begin end start commit rollback savepoint lock alter create drop rename call delete do handler insert load replace select truncate update set show pragma grant merge describe use explain help declare prepare execute deallocate release unlock purge reset change stop analyze cache flush optimize repair kill install uninstall checksum restore check backup revoke comment",e:/;/,eW:!0,l:/[\w\.]+/,k:{keyword:"abort abs absolute acc acce accep accept access accessed accessible account acos action activate add addtime admin administer advanced advise aes_decrypt aes_encrypt after agent aggregate ali alia alias allocate allow alter always analyze ancillary and any anydata anydataset anyschema anytype apply archive archived archivelog are as asc ascii asin assembly assertion associate asynchronous at atan atn2 attr attri attrib attribu attribut attribute attributes audit authenticated authentication authid authors auto autoallocate autodblink autoextend automatic availability avg backup badfile basicfile before begin beginning benchmark between bfile bfile_base big bigfile bin binary_double binary_float binlog bit_and bit_count bit_length bit_or bit_xor bitmap blob_base block blocksize body both bound buffer_cache buffer_pool build bulk by byte byteordermark bytes cache caching call calling cancel capacity cascade cascaded case cast catalog category ceil ceiling chain change changed char_base char_length character_length characters characterset charindex charset charsetform charsetid check checksum checksum_agg child choose chr chunk class cleanup clear client clob clob_base clone close cluster_id cluster_probability cluster_set clustering coalesce coercibility col collate collation collect colu colum column column_value columns columns_updated comment commit compact compatibility compiled complete composite_limit compound compress compute concat concat_ws concurrent confirm conn connec connect connect_by_iscycle connect_by_isleaf connect_by_root connect_time connection consider consistent constant constraint constraints constructor container content contents context contributors controlfile conv convert convert_tz corr corr_k corr_s corresponding corruption cos cost count count_big counted covar_pop covar_samp cpu_per_call cpu_per_session crc32 create creation critical cross cube cume_dist curdate current current_date current_time current_timestamp current_user cursor curtime customdatum cycle data database databases datafile datafiles datalength date_add date_cache date_format date_sub dateadd datediff datefromparts datename datepart datetime2fromparts day day_to_second dayname dayofmonth dayofweek dayofyear days db_role_change dbtimezone ddl deallocate declare decode decompose decrement decrypt deduplicate def defa defau defaul default defaults deferred defi defin define degrees delayed delegate delete delete_all delimited demand dense_rank depth dequeue des_decrypt des_encrypt des_key_file desc descr descri describ describe descriptor deterministic diagnostics difference dimension direct_load directory disable disable_all disallow disassociate discardfile disconnect diskgroup distinct distinctrow distribute distributed div do document domain dotnet double downgrade drop dumpfile duplicate duration each edition editionable editions element ellipsis else elsif elt empty enable enable_all enclosed encode encoding encrypt end end-exec endian enforced engine engines enqueue enterprise entityescaping eomonth error errors escaped evalname evaluate event eventdata events except exception exceptions exchange exclude excluding execu execut execute exempt exists exit exp expire explain export export_set extended extent external external_1 external_2 externally extract failed failed_login_attempts failover failure far fast feature_set feature_value fetch field fields file file_name_convert filesystem_like_logging final finish first first_value fixed flash_cache flashback floor flush following follows for forall force form forma format found found_rows freelist freelists freepools fresh from from_base64 from_days ftp full function general generated get get_format get_lock getdate getutcdate global global_name globally go goto grant grants greatest group group_concat group_id grouping grouping_id groups gtid_subtract guarantee guard handler hash hashkeys having hea head headi headin heading heap help hex hierarchy high high_priority hosts hour http id ident_current ident_incr ident_seed identified identity idle_time if ifnull ignore iif ilike ilm immediate import in include including increment index indexes indexing indextype indicator indices inet6_aton inet6_ntoa inet_aton inet_ntoa infile initial initialized initially initrans inmemory inner innodb input insert install instance instantiable instr interface interleaved intersect into invalidate invisible is is_free_lock is_ipv4 is_ipv4_compat is_not is_not_null is_used_lock isdate isnull isolation iterate java join json json_exists keep keep_duplicates key keys kill language large last last_day last_insert_id last_value lax lcase lead leading least leaves left len lenght length less level levels library like like2 like4 likec limit lines link list listagg little ln load load_file lob lobs local localtime localtimestamp locate locator lock locked log log10 log2 logfile logfiles logging logical logical_reads_per_call logoff logon logs long loop low low_priority lower lpad lrtrim ltrim main make_set makedate maketime managed management manual map mapping mask master master_pos_wait match matched materialized max maxextents maximize maxinstances maxlen maxlogfiles maxloghistory maxlogmembers maxsize maxtrans md5 measures median medium member memcompress memory merge microsecond mid migration min minextents minimum mining minus minute minvalue missing mod mode model modification modify module monitoring month months mount move movement multiset mutex name name_const names nan national native natural nav nchar nclob nested never new newline next nextval no no_write_to_binlog noarchivelog noaudit nobadfile nocheck nocompress nocopy nocycle nodelay nodiscardfile noentityescaping noguarantee nokeep nologfile nomapping nomaxvalue nominimize nominvalue nomonitoring none noneditionable nonschema noorder nopr nopro noprom nopromp noprompt norely noresetlogs noreverse normal norowdependencies noschemacheck noswitch not nothing notice notrim novalidate now nowait nth_value nullif nulls num numb numbe nvarchar nvarchar2 object ocicoll ocidate ocidatetime ociduration ociinterval ociloblocator ocinumber ociref ocirefcursor ocirowid ocistring ocitype oct octet_length of off offline offset oid oidindex old on online only opaque open operations operator optimal optimize option optionally or oracle oracle_date oradata ord ordaudio orddicom orddoc order ordimage ordinality ordvideo organization orlany orlvary out outer outfile outline output over overflow overriding package pad parallel parallel_enable parameters parent parse partial partition partitions pascal passing password password_grace_time password_lock_time password_reuse_max password_reuse_time password_verify_function patch path patindex pctincrease pctthreshold pctused pctversion percent percent_rank percentile_cont percentile_disc performance period period_add period_diff permanent physical pi pipe pipelined pivot pluggable plugin policy position post_transaction pow power pragma prebuilt precedes preceding precision prediction prediction_cost prediction_details prediction_probability prediction_set prepare present preserve prior priority private private_sga privileges procedural procedure procedure_analyze processlist profiles project prompt protection public publishingservername purge quarter query quick quiesce quota quotename radians raise rand range rank raw read reads readsize rebuild record records recover recovery recursive recycle redo reduced ref reference referenced references referencing refresh regexp_like register regr_avgx regr_avgy regr_count regr_intercept regr_r2 regr_slope regr_sxx regr_sxy reject rekey relational relative relaylog release release_lock relies_on relocate rely rem remainder rename repair repeat replace replicate replication required reset resetlogs resize resource respect restore restricted result result_cache resumable resume retention return returning returns reuse reverse revoke right rlike role roles rollback rolling rollup round row row_count rowdependencies rowid rownum rows rtrim rules safe salt sample save savepoint sb1 sb2 sb4 scan schema schemacheck scn scope scroll sdo_georaster sdo_topo_geometry search sec_to_time second section securefile security seed segment select self sequence sequential serializable server servererror session session_user sessions_per_user set sets settings sha sha1 sha2 share shared shared_pool short show shrink shutdown si_averagecolor si_colorhistogram si_featurelist si_positionalcolor si_stillimage si_texture siblings sid sign sin size size_t sizes skip slave sleep smalldatetimefromparts smallfile snapshot some soname sort soundex source space sparse spfile split sql sql_big_result sql_buffer_result sql_cache sql_calc_found_rows sql_small_result sql_variant_property sqlcode sqldata sqlerror sqlname sqlstate sqrt square standalone standby start starting startup statement static statistics stats_binomial_test stats_crosstab stats_ks_test stats_mode stats_mw_test stats_one_way_anova stats_t_test_ stats_t_test_indep stats_t_test_one stats_t_test_paired stats_wsr_test status std stddev stddev_pop stddev_samp stdev stop storage store stored str str_to_date straight_join strcmp strict string struct stuff style subdate subpartition subpartitions substitutable substr substring subtime subtring_index subtype success sum suspend switch switchoffset switchover sync synchronous synonym sys sys_xmlagg sysasm sysaux sysdate sysdatetimeoffset sysdba sysoper system system_user sysutcdatetime table tables tablespace tan tdo template temporary terminated tertiary_weights test than then thread through tier ties time time_format time_zone timediff timefromparts timeout timestamp timestampadd timestampdiff timezone_abbr timezone_minute timezone_region to to_base64 to_date to_days to_seconds todatetimeoffset trace tracking transaction transactional translate translation treat trigger trigger_nestlevel triggers trim truncate try_cast try_convert try_parse type ub1 ub2 ub4 ucase unarchived unbounded uncompress under undo unhex unicode uniform uninstall union unique unix_timestamp unknown unlimited unlock unpivot unrecoverable unsafe unsigned until untrusted unusable unused update updated upgrade upped upper upsert url urowid usable usage use use_stored_outlines user user_data user_resources users using utc_date utc_timestamp uuid uuid_short validate validate_password_strength validation valist value values var var_samp varcharc vari varia variab variabl variable variables variance varp varraw varrawc varray verify version versions view virtual visible void wait wallet warning warnings week weekday weekofyear wellformed when whene whenev wheneve whenever where while whitespace with within without work wrapped xdb xml xmlagg xmlattributes xmlcast xmlcolattval xmlelement xmlexists xmlforest xmlindex xmlnamespaces xmlpi xmlquery xmlroot xmlschema xmlserialize xmltable xmltype xor year year_to_month years yearweek",literal:"true false null",built_in:"array bigint binary bit blob boolean char character date dec decimal float int int8 integer interval number numeric real record serial serial8 smallint text varchar varying void"},c:[{cN:"string",b:"'",e:"'",c:[e.BE,{b:"''"}]},{cN:"string",b:'"',e:'"',c:[e.BE,{b:'""'}]},{cN:"string",b:"`",e:"`",c:[e.BE]},e.CNM,e.CBCM,t]},e.CBCM,t]}}),e}); +/*! highlight.js v9.12.0 | BSD3 License | git.io/hljslicense */ +!function(e){var t="object"==typeof window&&window||"object"==typeof self&&self;"undefined"!=typeof exports?e(exports):t&&(t.hljs=e({}),"function"==typeof define&&define.amd&&define([],function(){return t.hljs}))}(function(e){function t(e){return e.replace(/&/g,"&").replace(//g,">")}function r(e){return e.nodeName.toLowerCase()}function a(e,t){var r=e&&e.exec(t);return r&&0===r.index}function n(e){return E.test(e)}function i(e){var t,r,a,i,s=e.className+" ";if(s+=e.parentNode?e.parentNode.className:"",r=M.exec(s))return w(r[1])?r[1]:"no-highlight";for(s=s.split(/\s+/),t=0,a=s.length;a>t;t++)if(i=s[t],n(i)||w(i))return i}function s(e){var t,r={},a=Array.prototype.slice.call(arguments,1);for(t in e)r[t]=e[t];return a.forEach(function(e){for(t in e)r[t]=e[t]}),r}function c(e){var t=[];return function a(e,n){for(var i=e.firstChild;i;i=i.nextSibling)3===i.nodeType?n+=i.nodeValue.length:1===i.nodeType&&(t.push({event:"start",offset:n,node:i}),n=a(i,n),r(i).match(/br|hr|img|input/)||t.push({event:"stop",offset:n,node:i}));return n}(e,0),t}function o(e,a,n){function i(){return e.length&&a.length?e[0].offset!==a[0].offset?e[0].offset"}function c(e){u+=""}function o(e){("start"===e.event?s:c)(e.node)}for(var l=0,u="",d=[];e.length||a.length;){var b=i();if(u+=t(n.substring(l,b[0].offset)),l=b[0].offset,b===e){d.reverse().forEach(c);do o(b.splice(0,1)[0]),b=i();while(b===e&&b.length&&b[0].offset===l);d.reverse().forEach(s)}else"start"===b[0].event?d.push(b[0].node):d.pop(),o(b.splice(0,1)[0])}return u+t(n.substr(l))}function l(e){return e.v&&!e.cached_variants&&(e.cached_variants=e.v.map(function(t){return s(e,{v:null},t)})),e.cached_variants||e.eW&&[s(e)]||[e]}function u(e){function t(e){return e&&e.source||e}function r(r,a){return new RegExp(t(r),"m"+(e.cI?"i":"")+(a?"g":""))}function a(n,i){if(!n.compiled){if(n.compiled=!0,n.k=n.k||n.bK,n.k){var s={},c=function(t,r){e.cI&&(r=r.toLowerCase()),r.split(" ").forEach(function(e){var r=e.split("|");s[r[0]]=[t,r[1]?Number(r[1]):1]})};"string"==typeof n.k?c("keyword",n.k):k(n.k).forEach(function(e){c(e,n.k[e])}),n.k=s}n.lR=r(n.l||/\w+/,!0),i&&(n.bK&&(n.b="\\b("+n.bK.split(" ").join("|")+")\\b"),n.b||(n.b=/\B|\b/),n.bR=r(n.b),n.e||n.eW||(n.e=/\B|\b/),n.e&&(n.eR=r(n.e)),n.tE=t(n.e)||"",n.eW&&i.tE&&(n.tE+=(n.e?"|":"")+i.tE)),n.i&&(n.iR=r(n.i)),null==n.r&&(n.r=1),n.c||(n.c=[]),n.c=Array.prototype.concat.apply([],n.c.map(function(e){return l("self"===e?n:e)})),n.c.forEach(function(e){a(e,n)}),n.starts&&a(n.starts,i);var o=n.c.map(function(e){return e.bK?"\\.?("+e.b+")\\.?":e.b}).concat([n.tE,n.i]).map(t).filter(Boolean);n.t=o.length?r(o.join("|"),!0):{exec:function(){return null}}}}a(e)}function d(e,r,n,i){function s(e,t){var r,n;for(r=0,n=t.c.length;n>r;r++)if(a(t.c[r].bR,e))return t.c[r]}function c(e,t){if(a(e.eR,t)){for(;e.endsParent&&e.parent;)e=e.parent;return e}return e.eW?c(e.parent,t):void 0}function o(e,t){return!n&&a(t.iR,e)}function l(e,t){var r=v.cI?t[0].toLowerCase():t[0];return e.k.hasOwnProperty(r)&&e.k[r]}function p(e,t,r,a){var n=a?"":L.classPrefix,i='',i+t+s}function m(){var e,r,a,n;if(!N.k)return t(E);for(n="",r=0,N.lR.lastIndex=0,a=N.lR.exec(E);a;)n+=t(E.substring(r,a.index)),e=l(N,a),e?(M+=e[1],n+=p(e[0],t(a[0]))):n+=t(a[0]),r=N.lR.lastIndex,a=N.lR.exec(E);return n+t(E.substr(r))}function f(){var e="string"==typeof N.sL;if(e&&!x[N.sL])return t(E);var r=e?d(N.sL,E,!0,k[N.sL]):b(E,N.sL.length?N.sL:void 0);return N.r>0&&(M+=r.r),e&&(k[N.sL]=r.top),p(r.language,r.value,!1,!0)}function g(){C+=null!=N.sL?f():m(),E=""}function _(e){C+=e.cN?p(e.cN,"",!0):"",N=Object.create(e,{parent:{value:N}})}function h(e,t){if(E+=e,null==t)return g(),0;var r=s(t,N);if(r)return r.skip?E+=t:(r.eB&&(E+=t),g(),r.rB||r.eB||(E=t)),_(r,t),r.rB?0:t.length;var a=c(N,t);if(a){var n=N;n.skip?E+=t:(n.rE||n.eE||(E+=t),g(),n.eE&&(E=t));do N.cN&&(C+=R),N.skip||(M+=N.r),N=N.parent;while(N!==a.parent);return a.starts&&_(a.starts,""),n.rE?0:t.length}if(o(t,N))throw new Error('Illegal lexeme "'+t+'" for mode "'+(N.cN||"")+'"');return E+=t,t.length||1}var v=w(e);if(!v)throw new Error('Unknown language: "'+e+'"');u(v);var y,N=i||v,k={},C="";for(y=N;y!==v;y=y.parent)y.cN&&(C=p(y.cN,"",!0)+C);var E="",M=0;try{for(var B,S,$=0;;){if(N.t.lastIndex=$,B=N.t.exec(r),!B)break;S=h(r.substring($,B.index),B[0]),$=B.index+S}for(h(r.substr($)),y=N;y.parent;y=y.parent)y.cN&&(C+=R);return{r:M,value:C,language:e,top:N}}catch(A){if(A.message&&-1!==A.message.indexOf("Illegal"))return{r:0,value:t(r)};throw A}}function b(e,r){r=r||L.languages||k(x);var a={r:0,value:t(e)},n=a;return r.filter(w).forEach(function(t){var r=d(t,e,!1);r.language=t,r.r>n.r&&(n=r),r.r>a.r&&(n=a,a=r)}),n.language&&(a.second_best=n),a}function p(e){return L.tabReplace||L.useBR?e.replace(B,function(e,t){return L.useBR&&"\n"===e?"
    ":L.tabReplace?t.replace(/\t/g,L.tabReplace):""}):e}function m(e,t,r){var a=t?C[t]:r,n=[e.trim()];return e.match(/\bhljs\b/)||n.push("hljs"),-1===e.indexOf(a)&&n.push(a),n.join(" ").trim()}function f(e){var t,r,a,s,l,u=i(e);n(u)||(L.useBR?(t=document.createElementNS("http://www.w3.org/1999/xhtml","div"),t.innerHTML=e.innerHTML.replace(/\n/g,"").replace(//g,"\n")):t=e,l=t.textContent,a=u?d(u,l,!0):b(l),r=c(t),r.length&&(s=document.createElementNS("http://www.w3.org/1999/xhtml","div"),s.innerHTML=a.value,a.value=o(r,c(s),l)),a.value=p(a.value),e.innerHTML=a.value,e.className=m(e.className,u,a.language),e.result={language:a.language,re:a.r},a.second_best&&(e.second_best={language:a.second_best.language,re:a.second_best.r}))}function g(e){L=s(L,e)}function _(){if(!_.called){_.called=!0;var e=document.querySelectorAll("pre code");N.forEach.call(e,f)}}function h(){addEventListener("DOMContentLoaded",_,!1),addEventListener("load",_,!1)}function v(t,r){var a=x[t]=r(e);a.aliases&&a.aliases.forEach(function(e){C[e]=t})}function y(){return k(x)}function w(e){return e=(e||"").toLowerCase(),x[e]||x[C[e]]}var N=[],k=Object.keys,x={},C={},E=/^(no-?highlight|plain|text)$/i,M=/\blang(?:uage)?-([\w-]+)\b/i,B=/((^(<[^>]+>|\t|)+|(?:\n)))/gm,R="
    ",L={classPrefix:"hljs-",tabReplace:null,useBR:!1,languages:void 0};return e.highlight=d,e.highlightAuto=b,e.fixMarkup=p,e.highlightBlock=f,e.configure=g,e.initHighlighting=_,e.initHighlightingOnLoad=h,e.registerLanguage=v,e.listLanguages=y,e.getLanguage=w,e.inherit=s,e.IR="[a-zA-Z]\\w*",e.UIR="[a-zA-Z_]\\w*",e.NR="\\b\\d+(\\.\\d+)?",e.CNR="(-?)(\\b0[xX][a-fA-F0-9]+|(\\b\\d+(\\.\\d*)?|\\.\\d+)([eE][-+]?\\d+)?)",e.BNR="\\b(0b[01]+)",e.RSR="!|!=|!==|%|%=|&|&&|&=|\\*|\\*=|\\+|\\+=|,|-|-=|/=|/|:|;|<<|<<=|<=|<|===|==|=|>>>=|>>=|>=|>>>|>>|>|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~",e.BE={b:"\\\\[\\s\\S]",r:0},e.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[e.BE]},e.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[e.BE]},e.PWM={b:/\b(a|an|the|are|I'm|isn't|don't|doesn't|won't|but|just|should|pretty|simply|enough|gonna|going|wtf|so|such|will|you|your|they|like|more)\b/},e.C=function(t,r,a){var n=e.inherit({cN:"comment",b:t,e:r,c:[]},a||{});return n.c.push(e.PWM),n.c.push({cN:"doctag",b:"(?:TODO|FIXME|NOTE|BUG|XXX):",r:0}),n},e.CLCM=e.C("//","$"),e.CBCM=e.C("/\\*","\\*/"),e.HCM=e.C("#","$"),e.NM={cN:"number",b:e.NR,r:0},e.CNM={cN:"number",b:e.CNR,r:0},e.BNM={cN:"number",b:e.BNR,r:0},e.CSSNM={cN:"number",b:e.NR+"(%|em|ex|ch|rem|vw|vh|vmin|vmax|cm|mm|in|pt|pc|px|deg|grad|rad|turn|s|ms|Hz|kHz|dpi|dpcm|dppx)?",r:0},e.RM={cN:"regexp",b:/\//,e:/\/[gimuy]*/,i:/\n/,c:[e.BE,{b:/\[/,e:/\]/,r:0,c:[e.BE]}]},e.TM={cN:"title",b:e.IR,r:0},e.UTM={cN:"title",b:e.UIR,r:0},e.METHOD_GUARD={b:"\\.\\s*"+e.UIR,r:0},e.registerLanguage("apache",function(e){var t={cN:"number",b:"[\\$%]\\d+"};return{aliases:["apacheconf"],cI:!0,c:[e.HCM,{cN:"section",b:""},{cN:"attribute",b:/\w+/,r:0,k:{nomarkup:"order deny allow setenv rewriterule rewriteengine rewritecond documentroot sethandler errordocument loadmodule options header listen serverroot servername"},starts:{e:/$/,r:0,k:{literal:"on off all"},c:[{cN:"meta",b:"\\s\\[",e:"\\]$"},{cN:"variable",b:"[\\$%]\\{",e:"\\}",c:["self",t]},t,e.QSM]}}],i:/\S/}}),e.registerLanguage("bash",function(e){var t={cN:"variable",v:[{b:/\$[\w\d#@][\w\d_]*/},{b:/\$\{(.*?)}/}]},r={cN:"string",b:/"/,e:/"/,c:[e.BE,t,{cN:"variable",b:/\$\(/,e:/\)/,c:[e.BE]}]},a={cN:"string",b:/'/,e:/'/};return{aliases:["sh","zsh"],l:/\b-?[a-z\._]+\b/,k:{keyword:"if then else elif fi for while in do done case esac function",literal:"true false",built_in:"break cd continue eval exec exit export getopts hash pwd readonly return shift test times trap umask unset alias bind builtin caller command declare echo enable help let local logout mapfile printf read readarray source type typeset ulimit unalias set shopt autoload bg bindkey bye cap chdir clone comparguments compcall compctl compdescribe compfiles compgroups compquote comptags comptry compvalues dirs disable disown echotc echoti emulate fc fg float functions getcap getln history integer jobs kill limit log noglob popd print pushd pushln rehash sched setcap setopt stat suspend ttyctl unfunction unhash unlimit unsetopt vared wait whence where which zcompile zformat zftp zle zmodload zparseopts zprof zpty zregexparse zsocket zstyle ztcp",_:"-ne -eq -lt -gt -f -d -e -s -l -a"},c:[{cN:"meta",b:/^#![^\n]+sh\s*$/,r:10},{cN:"function",b:/\w[\w\d_]*\s*\(\s*\)\s*\{/,rB:!0,c:[e.inherit(e.TM,{b:/\w[\w\d_]*/})],r:0},e.HCM,r,a,t]}}),e.registerLanguage("coffeescript",function(e){var t={keyword:"in if for while finally new do return else break catch instanceof throw try this switch continue typeof delete debugger super yield import export from as default await then unless until loop of by when and or is isnt not",literal:"true false null undefined yes no on off",built_in:"npm require console print module global window document"},r="[A-Za-z$_][0-9A-Za-z$_]*",a={cN:"subst",b:/#\{/,e:/}/,k:t},n=[e.BNM,e.inherit(e.CNM,{starts:{e:"(\\s*/)?",r:0}}),{cN:"string",v:[{b:/'''/,e:/'''/,c:[e.BE]},{b:/'/,e:/'/,c:[e.BE]},{b:/"""/,e:/"""/,c:[e.BE,a]},{b:/"/,e:/"/,c:[e.BE,a]}]},{cN:"regexp",v:[{b:"///",e:"///",c:[a,e.HCM]},{b:"//[gim]*",r:0},{b:/\/(?![ *])(\\\/|.)*?\/[gim]*(?=\W|$)/}]},{b:"@"+r},{sL:"javascript",eB:!0,eE:!0,v:[{b:"```",e:"```"},{b:"`",e:"`"}]}];a.c=n;var i=e.inherit(e.TM,{b:r}),s="(\\(.*\\))?\\s*\\B[-=]>",c={cN:"params",b:"\\([^\\(]",rB:!0,c:[{b:/\(/,e:/\)/,k:t,c:["self"].concat(n)}]};return{aliases:["coffee","cson","iced"],k:t,i:/\/\*/,c:n.concat([e.C("###","###"),e.HCM,{cN:"function",b:"^\\s*"+r+"\\s*=\\s*"+s,e:"[-=]>",rB:!0,c:[i,c]},{b:/[:\(,=]\s*/,r:0,c:[{cN:"function",b:s,e:"[-=]>",rB:!0,c:[c]}]},{cN:"class",bK:"class",e:"$",i:/[:="\[\]]/,c:[{bK:"extends",eW:!0,i:/[:="\[\]]/,c:[i]},i]},{b:r+":",e:":",rB:!0,rE:!0,r:0}])}}),e.registerLanguage("cpp",function(e){var t={cN:"keyword",b:"\\b[a-z\\d_]*_t\\b"},r={cN:"string",v:[{b:'(u8?|U)?L?"',e:'"',i:"\\n",c:[e.BE]},{b:'(u8?|U)?R"',e:'"',c:[e.BE]},{b:"'\\\\?.",e:"'",i:"."}]},a={cN:"number",v:[{b:"\\b(0b[01']+)"},{b:"(-?)\\b([\\d']+(\\.[\\d']*)?|\\.[\\d']+)(u|U|l|L|ul|UL|f|F|b|B)"},{b:"(-?)(\\b0[xX][a-fA-F0-9']+|(\\b[\\d']+(\\.[\\d']*)?|\\.[\\d']+)([eE][-+]?[\\d']+)?)"}],r:0},n={cN:"meta",b:/#\s*[a-z]+\b/,e:/$/,k:{"meta-keyword":"if else elif endif define undef warning error line pragma ifdef ifndef include"},c:[{b:/\\\n/,r:0},e.inherit(r,{cN:"meta-string"}),{cN:"meta-string",b:/<[^\n>]*>/,e:/$/,i:"\\n"},e.CLCM,e.CBCM]},i=e.IR+"\\s*\\(",s={keyword:"int float while private char catch import module export virtual operator sizeof dynamic_cast|10 typedef const_cast|10 const for static_cast|10 union namespace unsigned long volatile static protected bool template mutable if public friend do goto auto void enum else break extern using asm case typeid short reinterpret_cast|10 default double register explicit signed typename try this switch continue inline delete alignof constexpr decltype noexcept static_assert thread_local restrict _Bool complex _Complex _Imaginary atomic_bool atomic_char atomic_schar atomic_uchar atomic_short atomic_ushort atomic_int atomic_uint atomic_long atomic_ulong atomic_llong atomic_ullong new throw return and or not",built_in:"std string cin cout cerr clog stdin stdout stderr stringstream istringstream ostringstream auto_ptr deque list queue stack vector map set bitset multiset multimap unordered_set unordered_map unordered_multiset unordered_multimap array shared_ptr abort abs acos asin atan2 atan calloc ceil cosh cos exit exp fabs floor fmod fprintf fputs free frexp fscanf isalnum isalpha iscntrl isdigit isgraph islower isprint ispunct isspace isupper isxdigit tolower toupper labs ldexp log10 log malloc realloc memchr memcmp memcpy memset modf pow printf putchar puts scanf sinh sin snprintf sprintf sqrt sscanf strcat strchr strcmp strcpy strcspn strlen strncat strncmp strncpy strpbrk strrchr strspn strstr tanh tan vfprintf vprintf vsprintf endl initializer_list unique_ptr",literal:"true false nullptr NULL"},c=[t,e.CLCM,e.CBCM,a,r];return{aliases:["c","cc","h","c++","h++","hpp"],k:s,i:"",k:s,c:["self",t]},{b:e.IR+"::",k:s},{v:[{b:/=/,e:/;/},{b:/\(/,e:/\)/},{bK:"new throw return else",e:/;/}],k:s,c:c.concat([{b:/\(/,e:/\)/,k:s,c:c.concat(["self"]),r:0}]),r:0},{cN:"function",b:"("+e.IR+"[\\*&\\s]+)+"+i,rB:!0,e:/[{;=]/,eE:!0,k:s,i:/[^\w\s\*&]/,c:[{b:i,rB:!0,c:[e.TM],r:0},{cN:"params",b:/\(/,e:/\)/,k:s,r:0,c:[e.CLCM,e.CBCM,r,a,t]},e.CLCM,e.CBCM,n]},{cN:"class",bK:"class struct",e:/[{;:]/,c:[{b://,c:["self"]},e.TM]}]),exports:{preprocessor:n,strings:r,k:s}}}),e.registerLanguage("cs",function(e){var t={keyword:"abstract as base bool break byte case catch char checked const continue decimal default delegate do double enum event explicit extern finally fixed float for foreach goto if implicit in int interface internal is lock long nameof object operator out override params private protected public readonly ref sbyte sealed short sizeof stackalloc static string struct switch this try typeof uint ulong unchecked unsafe ushort using virtual void volatile while add alias ascending async await by descending dynamic equals from get global group into join let on orderby partial remove select set value var where yield",literal:"null false true"},r={cN:"string",b:'@"',e:'"',c:[{b:'""'}]},a=e.inherit(r,{i:/\n/}),n={cN:"subst",b:"{",e:"}",k:t},i=e.inherit(n,{i:/\n/}),s={cN:"string",b:/\$"/,e:'"',i:/\n/,c:[{b:"{{"},{b:"}}"},e.BE,i]},c={cN:"string",b:/\$@"/,e:'"',c:[{b:"{{"},{b:"}}"},{b:'""'},n]},o=e.inherit(c,{i:/\n/,c:[{b:"{{"},{b:"}}"},{b:'""'},i]});n.c=[c,s,r,e.ASM,e.QSM,e.CNM,e.CBCM],i.c=[o,s,a,e.ASM,e.QSM,e.CNM,e.inherit(e.CBCM,{i:/\n/})];var l={v:[c,s,r,e.ASM,e.QSM]},u=e.IR+"(<"+e.IR+"(\\s*,\\s*"+e.IR+")*>)?(\\[\\])?";return{aliases:["csharp"],k:t,i:/::/,c:[e.C("///","$",{rB:!0,c:[{cN:"doctag",v:[{b:"///",r:0},{b:""},{b:""}]}]}),e.CLCM,e.CBCM,{cN:"meta",b:"#",e:"$",k:{"meta-keyword":"if else elif endif define undef warning error line region endregion pragma checksum"}},l,e.CNM,{bK:"class interface",e:/[{;=]/,i:/[^\s:]/,c:[e.TM,e.CLCM,e.CBCM]},{bK:"namespace",e:/[{;=]/,i:/[^\s:]/,c:[e.inherit(e.TM,{b:"[a-zA-Z](\\.?\\w)*"}),e.CLCM,e.CBCM]},{cN:"meta",b:"^\\s*\\[",eB:!0,e:"\\]",eE:!0,c:[{cN:"meta-string",b:/"/,e:/"/}]},{bK:"new return throw await else",r:0},{cN:"function",b:"("+u+"\\s+)+"+e.IR+"\\s*\\(",rB:!0,e:/[{;=]/,eE:!0,k:t,c:[{b:e.IR+"\\s*\\(",rB:!0,c:[e.TM],r:0},{cN:"params",b:/\(/,e:/\)/,eB:!0,eE:!0,k:t,r:0,c:[l,e.CNM,e.CBCM]},e.CLCM,e.CBCM]}]}}),e.registerLanguage("css",function(e){var t="[a-zA-Z-][a-zA-Z0-9_-]*",r={b:/[A-Z\_\.\-]+\s*:/,rB:!0,e:";",eW:!0,c:[{cN:"attribute",b:/\S/,e:":",eE:!0,starts:{eW:!0,eE:!0,c:[{b:/[\w-]+\(/,rB:!0,c:[{cN:"built_in",b:/[\w-]+/},{b:/\(/,e:/\)/,c:[e.ASM,e.QSM]}]},e.CSSNM,e.QSM,e.ASM,e.CBCM,{cN:"number",b:"#[0-9A-Fa-f]+"},{cN:"meta",b:"!important"}]}}]};return{cI:!0,i:/[=\/|'\$]/,c:[e.CBCM,{cN:"selector-id",b:/#[A-Za-z0-9_-]+/},{cN:"selector-class",b:/\.[A-Za-z0-9_-]+/},{cN:"selector-attr",b:/\[/,e:/\]/,i:"$"},{cN:"selector-pseudo",b:/:(:)?[a-zA-Z0-9\_\-\+\(\)"'.]+/},{b:"@(font-face|page)",l:"[a-z-]+",k:"font-face page"},{b:"@",e:"[{;]",i:/:/,c:[{cN:"keyword",b:/\w+/},{b:/\s/,eW:!0,eE:!0,r:0,c:[e.ASM,e.QSM,e.CSSNM]}]},{cN:"selector-tag",b:t,r:0},{b:"{",e:"}",i:/\S/,c:[e.CBCM,r]}]}}),e.registerLanguage("diff",function(e){return{aliases:["patch"],c:[{cN:"meta",r:10,v:[{b:/^@@ +\-\d+,\d+ +\+\d+,\d+ +@@$/},{b:/^\*\*\* +\d+,\d+ +\*\*\*\*$/},{b:/^\-\-\- +\d+,\d+ +\-\-\-\-$/}]},{cN:"comment",v:[{b:/Index: /,e:/$/},{b:/={3,}/,e:/$/},{b:/^\-{3}/,e:/$/},{b:/^\*{3} /,e:/$/},{b:/^\+{3}/,e:/$/},{b:/\*{5}/,e:/\*{5}$/}]},{cN:"addition",b:"^\\+",e:"$"},{cN:"deletion",b:"^\\-",e:"$"},{cN:"addition",b:"^\\!",e:"$"}]}}),e.registerLanguage("http",function(e){var t="HTTP/[0-9\\.]+";return{aliases:["https"],i:"\\S",c:[{b:"^"+t,e:"$",c:[{cN:"number",b:"\\b\\d{3}\\b"}]},{b:"^[A-Z]+ (.*?) "+t+"$",rB:!0,e:"$",c:[{cN:"string",b:" ",e:" ",eB:!0,eE:!0},{b:t},{cN:"keyword",b:"[A-Z]+"}]},{cN:"attribute",b:"^\\w",e:": ",eE:!0,i:"\\n|\\s|=",starts:{e:"$",r:0}},{b:"\\n\\n",starts:{sL:[],eW:!0}}]}}),e.registerLanguage("ini",function(e){var t={cN:"string",c:[e.BE],v:[{b:"'''",e:"'''",r:10},{b:'"""',e:'"""',r:10},{b:'"',e:'"'},{b:"'",e:"'"}]};return{aliases:["toml"],cI:!0,i:/\S/,c:[e.C(";","$"),e.HCM,{cN:"section",b:/^\s*\[+/,e:/\]+/},{b:/^[a-z0-9\[\]_-]+\s*=\s*/,e:"$",rB:!0,c:[{cN:"attr",b:/[a-z0-9\[\]_-]+/},{b:/=/,eW:!0,r:0,c:[{cN:"literal",b:/\bon|off|true|false|yes|no\b/},{cN:"variable",v:[{b:/\$[\w\d"][\w\d_]*/},{b:/\$\{(.*?)}/}]},t,{cN:"number",b:/([\+\-]+)?[\d]+_[\d_]+/},e.NM]}]}]}}),e.registerLanguage("java",function(e){var t="[À-ʸa-zA-Z_$][À-ʸa-zA-Z_$0-9]*",r=t+"(<"+t+"(\\s*,\\s*"+t+")*>)?",a="false synchronized int abstract float private char boolean static null if const for true while long strictfp finally protected import native final void enum else break transient catch instanceof byte super volatile case assert short package default double public try this switch continue throws protected public private module requires exports do",n="\\b(0[bB]([01]+[01_]+[01]+|[01]+)|0[xX]([a-fA-F0-9]+[a-fA-F0-9_]+[a-fA-F0-9]+|[a-fA-F0-9]+)|(([\\d]+[\\d_]+[\\d]+|[\\d]+)(\\.([\\d]+[\\d_]+[\\d]+|[\\d]+))?|\\.([\\d]+[\\d_]+[\\d]+|[\\d]+))([eE][-+]?\\d+)?)[lLfF]?",i={cN:"number",b:n,r:0};return{aliases:["jsp"],k:a,i:/<\/|#/,c:[e.C("/\\*\\*","\\*/",{r:0,c:[{b:/\w+@/,r:0},{cN:"doctag",b:"@[A-Za-z]+"}]}),e.CLCM,e.CBCM,e.ASM,e.QSM,{cN:"class",bK:"class interface",e:/[{;=]/,eE:!0,k:"class interface",i:/[:"\[\]]/,c:[{bK:"extends implements"},e.UTM]},{bK:"new throw return else",r:0},{cN:"function",b:"("+r+"\\s+)+"+e.UIR+"\\s*\\(",rB:!0,e:/[{;=]/,eE:!0,k:a,c:[{b:e.UIR+"\\s*\\(",rB:!0,r:0,c:[e.UTM]},{cN:"params",b:/\(/,e:/\)/,k:a,r:0,c:[e.ASM,e.QSM,e.CNM,e.CBCM]},e.CLCM,e.CBCM]},i,{cN:"meta",b:"@[A-Za-z]+"}]}}),e.registerLanguage("javascript",function(e){var t="[A-Za-z$_][0-9A-Za-z$_]*",r={keyword:"in of if for while finally var new function do return void else break catch instanceof with throw case default try this switch continue typeof delete let yield const export super debugger as async await static import from as",literal:"true false null undefined NaN Infinity",built_in:"eval isFinite isNaN parseFloat parseInt decodeURI decodeURIComponent encodeURI encodeURIComponent escape unescape Object Function Boolean Error EvalError InternalError RangeError ReferenceError StopIteration SyntaxError TypeError URIError Number Math Date String RegExp Array Float32Array Float64Array Int16Array Int32Array Int8Array Uint16Array Uint32Array Uint8Array Uint8ClampedArray ArrayBuffer DataView JSON Intl arguments require module console window document Symbol Set Map WeakSet WeakMap Proxy Reflect Promise"},a={cN:"number",v:[{b:"\\b(0[bB][01]+)"},{b:"\\b(0[oO][0-7]+)"},{b:e.CNR}],r:0},n={cN:"subst",b:"\\$\\{",e:"\\}",k:r,c:[]},i={cN:"string",b:"`",e:"`",c:[e.BE,n]};n.c=[e.ASM,e.QSM,i,a,e.RM];var s=n.c.concat([e.CBCM,e.CLCM]);return{aliases:["js","jsx"],k:r,c:[{cN:"meta",r:10,b:/^\s*['"]use (strict|asm)['"]/},{cN:"meta",b:/^#!/,e:/$/},e.ASM,e.QSM,i,e.CLCM,e.CBCM,a,{b:/[{,]\s*/,r:0,c:[{b:t+"\\s*:",rB:!0,r:0,c:[{cN:"attr",b:t,r:0}]}]},{b:"("+e.RSR+"|\\b(case|return|throw)\\b)\\s*",k:"return throw case",c:[e.CLCM,e.CBCM,e.RM,{cN:"function",b:"(\\(.*?\\)|"+t+")\\s*=>",rB:!0,e:"\\s*=>",c:[{cN:"params",v:[{b:t},{b:/\(\s*\)/},{b:/\(/,e:/\)/,eB:!0,eE:!0,k:r,c:s}]}]},{b://,sL:"xml",c:[{b:/<\w+\s*\/>/,skip:!0},{b:/<\w+/,e:/(\/\w+|\w+\/)>/,skip:!0,c:[{b:/<\w+\s*\/>/,skip:!0},"self"]}]}],r:0},{cN:"function",bK:"function",e:/\{/,eE:!0,c:[e.inherit(e.TM,{b:t}),{cN:"params",b:/\(/,e:/\)/,eB:!0,eE:!0,c:s}],i:/\[|%/},{b:/\$[(.]/},e.METHOD_GUARD,{cN:"class",bK:"class",e:/[{;=]/,eE:!0,i:/[:"\[\]]/,c:[{bK:"extends"},e.UTM]},{bK:"constructor",e:/\{/,eE:!0}],i:/#(?!!)/}}),e.registerLanguage("json",function(e){var t={literal:"true false null"},r=[e.QSM,e.CNM],a={e:",",eW:!0,eE:!0,c:r,k:t},n={b:"{",e:"}",c:[{cN:"attr",b:/"/,e:/"/,c:[e.BE],i:"\\n"},e.inherit(a,{b:/:/})],i:"\\S"},i={b:"\\[",e:"\\]",c:[e.inherit(a)],i:"\\S"};return r.splice(r.length,0,n,i),{c:r,k:t,i:"\\S"}}),e.registerLanguage("makefile",function(e){var t={cN:"variable",v:[{b:"\\$\\("+e.UIR+"\\)",c:[e.BE]},{b:/\$[@%`]+/}]}]}]};return{aliases:["html","xhtml","rss","atom","xjb","xsd","xsl","plist"],cI:!0,c:[{cN:"meta",b:"",r:10,c:[{b:"\\[",e:"\\]"}]},e.C("",{r:10}),{b:"<\\!\\[CDATA\\[",e:"\\]\\]>",r:10},{b:/<\?(php)?/,e:/\?>/,sL:"php",c:[{b:"/\\*",e:"\\*/",skip:!0}]},{cN:"tag",b:"|$)",e:">",k:{name:"style"},c:[r],starts:{e:"",rE:!0,sL:["css","xml"]}},{cN:"tag",b:"|$)",e:">",k:{name:"script"},c:[r],starts:{e:"",rE:!0,sL:["actionscript","javascript","handlebars","xml"]}},{cN:"meta",v:[{b:/<\?xml/,e:/\?>/,r:10},{b:/<\?\w+/,e:/\?>/}]},{cN:"tag",b:"",c:[{cN:"name",b:/[^\/><\s]+/,r:0},r]}]}}),e.registerLanguage("markdown",function(e){return{aliases:["md","mkdown","mkd"],c:[{cN:"section",v:[{b:"^#{1,6}",e:"$"},{b:"^.+?\\n[=-]{2,}$"}]},{b:"<",e:">",sL:"xml",r:0},{cN:"bullet",b:"^([*+-]|(\\d+\\.))\\s+"},{cN:"strong",b:"[*_]{2}.+?[*_]{2}"},{cN:"emphasis",v:[{b:"\\*.+?\\*"},{b:"_.+?_",r:0}]},{cN:"quote",b:"^>\\s+",e:"$"},{cN:"code",v:[{b:"^```w*s*$",e:"^```s*$"},{b:"`.+?`"},{b:"^( {4}| )",e:"$",r:0}]},{b:"^[-\\*]{3,}",e:"$"},{b:"\\[.+?\\][\\(\\[].*?[\\)\\]]",rB:!0,c:[{cN:"string",b:"\\[",e:"\\]",eB:!0,rE:!0,r:0},{cN:"link",b:"\\]\\(",e:"\\)",eB:!0,eE:!0},{cN:"symbol",b:"\\]\\[",e:"\\]",eB:!0,eE:!0}],r:10},{b:/^\[[^\n]+\]:/,rB:!0,c:[{cN:"symbol",b:/\[/,e:/\]/,eB:!0,eE:!0},{cN:"link",b:/:\s*/,e:/$/,eB:!0}]}]}}),e.registerLanguage("nginx",function(e){var t={cN:"variable",v:[{b:/\$\d+/},{b:/\$\{/,e:/}/},{b:"[\\$\\@]"+e.UIR}]},r={eW:!0,l:"[a-z/_]+",k:{literal:"on off yes no true false none blocked debug info notice warn error crit select break last permanent redirect kqueue rtsig epoll poll /dev/poll"},r:0,i:"=>",c:[e.HCM,{cN:"string",c:[e.BE,t],v:[{b:/"/,e:/"/},{b:/'/,e:/'/}]},{b:"([a-z]+):/",e:"\\s",eW:!0,eE:!0,c:[t]},{cN:"regexp",c:[e.BE,t],v:[{b:"\\s\\^",e:"\\s|{|;",rE:!0},{b:"~\\*?\\s+",e:"\\s|{|;",rE:!0},{b:"\\*(\\.[a-z\\-]+)+"},{b:"([a-z\\-]+\\.)+\\*"}]},{cN:"number",b:"\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}(:\\d{1,5})?\\b"},{cN:"number",b:"\\b\\d+[kKmMgGdshdwy]*\\b",r:0},t]};return{aliases:["nginxconf"],c:[e.HCM,{b:e.UIR+"\\s+{",rB:!0,e:"{",c:[{cN:"section",b:e.UIR}],r:0},{b:e.UIR+"\\s",e:";|{",rB:!0,c:[{cN:"attribute",b:e.UIR,starts:r}],r:0}],i:"[^\\s\\}]"}}),e.registerLanguage("objectivec",function(e){var t={cN:"built_in",b:"\\b(AV|CA|CF|CG|CI|CL|CM|CN|CT|MK|MP|MTK|MTL|NS|SCN|SK|UI|WK|XC)\\w+"},r={keyword:"int float while char export sizeof typedef const struct for union unsigned long volatile static bool mutable if do return goto void enum else break extern asm case short default double register explicit signed typename this switch continue wchar_t inline readonly assign readwrite self @synchronized id typeof nonatomic super unichar IBOutlet IBAction strong weak copy in out inout bycopy byref oneway __strong __weak __block __autoreleasing @private @protected @public @try @property @end @throw @catch @finally @autoreleasepool @synthesize @dynamic @selector @optional @required @encode @package @import @defs @compatibility_alias __bridge __bridge_transfer __bridge_retained __bridge_retain __covariant __contravariant __kindof _Nonnull _Nullable _Null_unspecified __FUNCTION__ __PRETTY_FUNCTION__ __attribute__ getter setter retain unsafe_unretained nonnull nullable null_unspecified null_resettable class instancetype NS_DESIGNATED_INITIALIZER NS_UNAVAILABLE NS_REQUIRES_SUPER NS_RETURNS_INNER_POINTER NS_INLINE NS_AVAILABLE NS_DEPRECATED NS_ENUM NS_OPTIONS NS_SWIFT_UNAVAILABLE NS_ASSUME_NONNULL_BEGIN NS_ASSUME_NONNULL_END NS_REFINED_FOR_SWIFT NS_SWIFT_NAME NS_SWIFT_NOTHROW NS_DURING NS_HANDLER NS_ENDHANDLER NS_VALUERETURN NS_VOIDRETURN",literal:"false true FALSE TRUE nil YES NO NULL",built_in:"BOOL dispatch_once_t dispatch_queue_t dispatch_sync dispatch_async dispatch_once"},a=/[a-zA-Z@][a-zA-Z0-9_]*/,n="@interface @class @protocol @implementation";return{aliases:["mm","objc","obj-c"],k:r,l:a,i:""}]}]},{cN:"class",b:"("+n.split(" ").join("|")+")\\b",e:"({|$)",eE:!0,k:n,l:a,c:[e.UTM]},{b:"\\."+e.UIR,r:0}]}}),e.registerLanguage("perl",function(e){var t="getpwent getservent quotemeta msgrcv scalar kill dbmclose undef lc ma syswrite tr send umask sysopen shmwrite vec qx utime local oct semctl localtime readpipe do return format read sprintf dbmopen pop getpgrp not getpwnam rewinddir qqfileno qw endprotoent wait sethostent bless s|0 opendir continue each sleep endgrent shutdown dump chomp connect getsockname die socketpair close flock exists index shmgetsub for endpwent redo lstat msgctl setpgrp abs exit select print ref gethostbyaddr unshift fcntl syscall goto getnetbyaddr join gmtime symlink semget splice x|0 getpeername recv log setsockopt cos last reverse gethostbyname getgrnam study formline endhostent times chop length gethostent getnetent pack getprotoent getservbyname rand mkdir pos chmod y|0 substr endnetent printf next open msgsnd readdir use unlink getsockopt getpriority rindex wantarray hex system getservbyport endservent int chr untie rmdir prototype tell listen fork shmread ucfirst setprotoent else sysseek link getgrgid shmctl waitpid unpack getnetbyname reset chdir grep split require caller lcfirst until warn while values shift telldir getpwuid my getprotobynumber delete and sort uc defined srand accept package seekdir getprotobyname semop our rename seek if q|0 chroot sysread setpwent no crypt getc chown sqrt write setnetent setpriority foreach tie sin msgget map stat getlogin unless elsif truncate exec keys glob tied closedirioctl socket readlink eval xor readline binmode setservent eof ord bind alarm pipe atan2 getgrent exp time push setgrent gt lt or ne m|0 break given say state when",r={cN:"subst",b:"[$@]\\{",e:"\\}",k:t},a={b:"->{",e:"}"},n={v:[{b:/\$\d/},{b:/[\$%@](\^\w\b|#\w+(::\w+)*|{\w+}|\w+(::\w*)*)/},{b:/[\$%@][^\s\w{]/,r:0}]},i=[e.BE,r,n],s=[n,e.HCM,e.C("^\\=\\w","\\=cut",{eW:!0}),a,{cN:"string",c:i,v:[{b:"q[qwxr]?\\s*\\(",e:"\\)",r:5},{b:"q[qwxr]?\\s*\\[",e:"\\]",r:5},{b:"q[qwxr]?\\s*\\{",e:"\\}",r:5},{b:"q[qwxr]?\\s*\\|",e:"\\|",r:5},{b:"q[qwxr]?\\s*\\<",e:"\\>",r:5},{b:"qw\\s+q",e:"q",r:5},{b:"'",e:"'",c:[e.BE]},{b:'"',e:'"'},{b:"`",e:"`",c:[e.BE]},{b:"{\\w+}",c:[],r:0},{b:"-?\\w+\\s*\\=\\>",c:[],r:0}]},{cN:"number",b:"(\\b0[0-7_]+)|(\\b0x[0-9a-fA-F_]+)|(\\b[1-9][0-9_]*(\\.[0-9_]+)?)|[0_]\\b",r:0},{b:"(\\/\\/|"+e.RSR+"|\\b(split|return|print|reverse|grep)\\b)\\s*",k:"split return print reverse grep",r:0,c:[e.HCM,{cN:"regexp",b:"(s|tr|y)/(\\\\.|[^/])*/(\\\\.|[^/])*/[a-z]*",r:10},{cN:"regexp",b:"(m|qr)?/",e:"/[a-z]*",c:[e.BE],r:0}]},{cN:"function",bK:"sub",e:"(\\s*\\(.*?\\))?[;{]",eE:!0,r:5,c:[e.TM]},{b:"-\\w\\b",r:0},{b:"^__DATA__$",e:"^__END__$",sL:"mojolicious",c:[{b:"^@@.*",e:"$",cN:"comment"}]}];return r.c=s,a.c=s,{aliases:["pl","pm"],l:/[\w\.]+/,k:t,c:s}}),e.registerLanguage("php",function(e){var t={b:"\\$+[a-zA-Z_-ÿ][a-zA-Z0-9_-ÿ]*"},r={cN:"meta",b:/<\?(php)?|\?>/},a={cN:"string",c:[e.BE,r],v:[{b:'b"',e:'"'},{b:"b'",e:"'"},e.inherit(e.ASM,{i:null}),e.inherit(e.QSM,{i:null})]},n={v:[e.BNM,e.CNM]};return{aliases:["php3","php4","php5","php6"],cI:!0,k:"and include_once list abstract global private echo interface as static endswitch array null if endwhile or const for endforeach self var while isset public protected exit foreach throw elseif include __FILE__ empty require_once do xor return parent clone use __CLASS__ __LINE__ else break print eval new catch __METHOD__ case exception default die require __FUNCTION__ enddeclare final try switch continue endfor endif declare unset true false trait goto instanceof insteadof __DIR__ __NAMESPACE__ yield finally",c:[e.HCM,e.C("//","$",{c:[r]}),e.C("/\\*","\\*/",{c:[{cN:"doctag",b:"@[A-Za-z]+"}]}),e.C("__halt_compiler.+?;",!1,{eW:!0,k:"__halt_compiler",l:e.UIR}),{cN:"string",b:/<<<['"]?\w+['"]?$/,e:/^\w+;?$/,c:[e.BE,{cN:"subst",v:[{b:/\$\w+/},{b:/\{\$/,e:/\}/}]}]},r,{cN:"keyword",b:/\$this\b/},t,{b:/(::|->)+[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*/},{cN:"function",bK:"function",e:/[;{]/,eE:!0,i:"\\$|\\[|%",c:[e.UTM,{cN:"params",b:"\\(",e:"\\)",c:["self",t,e.CBCM,a,n]}]},{cN:"class",bK:"class interface",e:"{",eE:!0,i:/[:\(\$"]/,c:[{bK:"extends implements"},e.UTM]},{bK:"namespace",e:";",i:/[\.']/,c:[e.UTM]},{bK:"use",e:";",c:[e.UTM]},{b:"=>"},a,n]}}),e.registerLanguage("python",function(e){var t={keyword:"and elif is global as in if from raise for except finally print import pass return exec else break not with class assert yield try while continue del or def lambda async await nonlocal|10 None True False",built_in:"Ellipsis NotImplemented"},r={cN:"meta",b:/^(>>>|\.\.\.) /},a={cN:"subst",b:/\{/,e:/\}/,k:t,i:/#/},n={cN:"string",c:[e.BE],v:[{b:/(u|b)?r?'''/,e:/'''/,c:[r],r:10},{b:/(u|b)?r?"""/,e:/"""/,c:[r],r:10},{b:/(fr|rf|f)'''/,e:/'''/,c:[r,a]},{b:/(fr|rf|f)"""/,e:/"""/,c:[r,a]},{b:/(u|r|ur)'/,e:/'/,r:10},{b:/(u|r|ur)"/,e:/"/,r:10},{b:/(b|br)'/,e:/'/},{b:/(b|br)"/,e:/"/},{b:/(fr|rf|f)'/,e:/'/,c:[a]},{b:/(fr|rf|f)"/,e:/"/,c:[a]},e.ASM,e.QSM]},i={cN:"number",r:0,v:[{b:e.BNR+"[lLjJ]?"},{b:"\\b(0o[0-7]+)[lLjJ]?"},{b:e.CNR+"[lLjJ]?"}]},s={cN:"params",b:/\(/,e:/\)/,c:["self",r,i,n]};return a.c=[n,i,r],{aliases:["py","gyp"],k:t,i:/(<\/|->|\?)|=>/,c:[r,i,n,e.HCM,{v:[{cN:"function",bK:"def"},{cN:"class",bK:"class"}],e:/:/,i:/[${=;\n,]/,c:[e.UTM,s,{b:/->/,eW:!0,k:"None"}]},{cN:"meta",b:/^[\t ]*@/,e:/$/},{b:/\b(print|exec)\(/}]}}),e.registerLanguage("ruby",function(e){ +var t="[a-zA-Z_]\\w*[!?=]?|[-+~]\\@|<<|>>|=~|===?|<=>|[<>]=?|\\*\\*|[-/+%^&*~`|]|\\[\\]=?",r={keyword:"and then defined module in return redo if BEGIN retry end for self when next until do begin unless END rescue else break undef not super class case require yield alias while ensure elsif or include attr_reader attr_writer attr_accessor",literal:"true false nil"},a={cN:"doctag",b:"@[A-Za-z]+"},n={b:"#<",e:">"},i=[e.C("#","$",{c:[a]}),e.C("^\\=begin","^\\=end",{c:[a],r:10}),e.C("^__END__","\\n$")],s={cN:"subst",b:"#\\{",e:"}",k:r},c={cN:"string",c:[e.BE,s],v:[{b:/'/,e:/'/},{b:/"/,e:/"/},{b:/`/,e:/`/},{b:"%[qQwWx]?\\(",e:"\\)"},{b:"%[qQwWx]?\\[",e:"\\]"},{b:"%[qQwWx]?{",e:"}"},{b:"%[qQwWx]?<",e:">"},{b:"%[qQwWx]?/",e:"/"},{b:"%[qQwWx]?%",e:"%"},{b:"%[qQwWx]?-",e:"-"},{b:"%[qQwWx]?\\|",e:"\\|"},{b:/\B\?(\\\d{1,3}|\\x[A-Fa-f0-9]{1,2}|\\u[A-Fa-f0-9]{4}|\\?\S)\b/},{b:/<<(-?)\w+$/,e:/^\s*\w+$/}]},o={cN:"params",b:"\\(",e:"\\)",endsParent:!0,k:r},l=[c,n,{cN:"class",bK:"class module",e:"$|;",i:/=/,c:[e.inherit(e.TM,{b:"[A-Za-z_]\\w*(::\\w+)*(\\?|\\!)?"}),{b:"<\\s*",c:[{b:"("+e.IR+"::)?"+e.IR}]}].concat(i)},{cN:"function",bK:"def",e:"$|;",c:[e.inherit(e.TM,{b:t}),o].concat(i)},{b:e.IR+"::"},{cN:"symbol",b:e.UIR+"(\\!|\\?)?:",r:0},{cN:"symbol",b:":(?!\\s)",c:[c,{b:t}],r:0},{cN:"number",b:"(\\b0[0-7_]+)|(\\b0x[0-9a-fA-F_]+)|(\\b[1-9][0-9_]*(\\.[0-9_]+)?)|[0_]\\b",r:0},{b:"(\\$\\W)|((\\$|\\@\\@?)(\\w+))"},{cN:"params",b:/\|/,e:/\|/,k:r},{b:"("+e.RSR+"|unless)\\s*",k:"unless",c:[n,{cN:"regexp",c:[e.BE,s],i:/\n/,v:[{b:"/",e:"/[a-z]*"},{b:"%r{",e:"}[a-z]*"},{b:"%r\\(",e:"\\)[a-z]*"},{b:"%r!",e:"![a-z]*"},{b:"%r\\[",e:"\\][a-z]*"}]}].concat(i),r:0}].concat(i);s.c=l,o.c=l;var u="[>?]>",d="[\\w#]+\\(\\w+\\):\\d+:\\d+>",b="(\\w+-)?\\d+\\.\\d+\\.\\d(p\\d+)?[^>]+>",p=[{b:/^\s*=>/,starts:{e:"$",c:l}},{cN:"meta",b:"^("+u+"|"+d+"|"+b+")",starts:{e:"$",c:l}}];return{aliases:["rb","gemspec","podspec","thor","irb"],k:r,i:/\/\*/,c:i.concat(p).concat(l)}}),e.registerLanguage("shell",function(e){return{aliases:["console"],c:[{cN:"meta",b:"^\\s{0,3}[\\w\\d\\[\\]()@-]*[>%$#]",starts:{e:"$",sL:"bash"}}]}}),e.registerLanguage("sql",function(e){var t=e.C("--","$");return{cI:!0,i:/[<>{}*#]/,c:[{bK:"begin end start commit rollback savepoint lock alter create drop rename call delete do handler insert load replace select truncate update set show pragma grant merge describe use explain help declare prepare execute deallocate release unlock purge reset change stop analyze cache flush optimize repair kill install uninstall checksum restore check backup revoke comment",e:/;/,eW:!0,l:/[\w\.]+/,k:{keyword:"abort abs absolute acc acce accep accept access accessed accessible account acos action activate add addtime admin administer advanced advise aes_decrypt aes_encrypt after agent aggregate ali alia alias allocate allow alter always analyze ancillary and any anydata anydataset anyschema anytype apply archive archived archivelog are as asc ascii asin assembly assertion associate asynchronous at atan atn2 attr attri attrib attribu attribut attribute attributes audit authenticated authentication authid authors auto autoallocate autodblink autoextend automatic availability avg backup badfile basicfile before begin beginning benchmark between bfile bfile_base big bigfile bin binary_double binary_float binlog bit_and bit_count bit_length bit_or bit_xor bitmap blob_base block blocksize body both bound buffer_cache buffer_pool build bulk by byte byteordermark bytes cache caching call calling cancel capacity cascade cascaded case cast catalog category ceil ceiling chain change changed char_base char_length character_length characters characterset charindex charset charsetform charsetid check checksum checksum_agg child choose chr chunk class cleanup clear client clob clob_base clone close cluster_id cluster_probability cluster_set clustering coalesce coercibility col collate collation collect colu colum column column_value columns columns_updated comment commit compact compatibility compiled complete composite_limit compound compress compute concat concat_ws concurrent confirm conn connec connect connect_by_iscycle connect_by_isleaf connect_by_root connect_time connection consider consistent constant constraint constraints constructor container content contents context contributors controlfile conv convert convert_tz corr corr_k corr_s corresponding corruption cos cost count count_big counted covar_pop covar_samp cpu_per_call cpu_per_session crc32 create creation critical cross cube cume_dist curdate current current_date current_time current_timestamp current_user cursor curtime customdatum cycle data database databases datafile datafiles datalength date_add date_cache date_format date_sub dateadd datediff datefromparts datename datepart datetime2fromparts day day_to_second dayname dayofmonth dayofweek dayofyear days db_role_change dbtimezone ddl deallocate declare decode decompose decrement decrypt deduplicate def defa defau defaul default defaults deferred defi defin define degrees delayed delegate delete delete_all delimited demand dense_rank depth dequeue des_decrypt des_encrypt des_key_file desc descr descri describ describe descriptor deterministic diagnostics difference dimension direct_load directory disable disable_all disallow disassociate discardfile disconnect diskgroup distinct distinctrow distribute distributed div do document domain dotnet double downgrade drop dumpfile duplicate duration each edition editionable editions element ellipsis else elsif elt empty enable enable_all enclosed encode encoding encrypt end end-exec endian enforced engine engines enqueue enterprise entityescaping eomonth error errors escaped evalname evaluate event eventdata events except exception exceptions exchange exclude excluding execu execut execute exempt exists exit exp expire explain export export_set extended extent external external_1 external_2 externally extract failed failed_login_attempts failover failure far fast feature_set feature_value fetch field fields file file_name_convert filesystem_like_logging final finish first first_value fixed flash_cache flashback floor flush following follows for forall force form forma format found found_rows freelist freelists freepools fresh from from_base64 from_days ftp full function general generated get get_format get_lock getdate getutcdate global global_name globally go goto grant grants greatest group group_concat group_id grouping grouping_id groups gtid_subtract guarantee guard handler hash hashkeys having hea head headi headin heading heap help hex hierarchy high high_priority hosts hour http id ident_current ident_incr ident_seed identified identity idle_time if ifnull ignore iif ilike ilm immediate import in include including increment index indexes indexing indextype indicator indices inet6_aton inet6_ntoa inet_aton inet_ntoa infile initial initialized initially initrans inmemory inner innodb input insert install instance instantiable instr interface interleaved intersect into invalidate invisible is is_free_lock is_ipv4 is_ipv4_compat is_not is_not_null is_used_lock isdate isnull isolation iterate java join json json_exists keep keep_duplicates key keys kill language large last last_day last_insert_id last_value lax lcase lead leading least leaves left len lenght length less level levels library like like2 like4 likec limit lines link list listagg little ln load load_file lob lobs local localtime localtimestamp locate locator lock locked log log10 log2 logfile logfiles logging logical logical_reads_per_call logoff logon logs long loop low low_priority lower lpad lrtrim ltrim main make_set makedate maketime managed management manual map mapping mask master master_pos_wait match matched materialized max maxextents maximize maxinstances maxlen maxlogfiles maxloghistory maxlogmembers maxsize maxtrans md5 measures median medium member memcompress memory merge microsecond mid migration min minextents minimum mining minus minute minvalue missing mod mode model modification modify module monitoring month months mount move movement multiset mutex name name_const names nan national native natural nav nchar nclob nested never new newline next nextval no no_write_to_binlog noarchivelog noaudit nobadfile nocheck nocompress nocopy nocycle nodelay nodiscardfile noentityescaping noguarantee nokeep nologfile nomapping nomaxvalue nominimize nominvalue nomonitoring none noneditionable nonschema noorder nopr nopro noprom nopromp noprompt norely noresetlogs noreverse normal norowdependencies noschemacheck noswitch not nothing notice notrim novalidate now nowait nth_value nullif nulls num numb numbe nvarchar nvarchar2 object ocicoll ocidate ocidatetime ociduration ociinterval ociloblocator ocinumber ociref ocirefcursor ocirowid ocistring ocitype oct octet_length of off offline offset oid oidindex old on online only opaque open operations operator optimal optimize option optionally or oracle oracle_date oradata ord ordaudio orddicom orddoc order ordimage ordinality ordvideo organization orlany orlvary out outer outfile outline output over overflow overriding package pad parallel parallel_enable parameters parent parse partial partition partitions pascal passing password password_grace_time password_lock_time password_reuse_max password_reuse_time password_verify_function patch path patindex pctincrease pctthreshold pctused pctversion percent percent_rank percentile_cont percentile_disc performance period period_add period_diff permanent physical pi pipe pipelined pivot pluggable plugin policy position post_transaction pow power pragma prebuilt precedes preceding precision prediction prediction_cost prediction_details prediction_probability prediction_set prepare present preserve prior priority private private_sga privileges procedural procedure procedure_analyze processlist profiles project prompt protection public publishingservername purge quarter query quick quiesce quota quotename radians raise rand range rank raw read reads readsize rebuild record records recover recovery recursive recycle redo reduced ref reference referenced references referencing refresh regexp_like register regr_avgx regr_avgy regr_count regr_intercept regr_r2 regr_slope regr_sxx regr_sxy reject rekey relational relative relaylog release release_lock relies_on relocate rely rem remainder rename repair repeat replace replicate replication required reset resetlogs resize resource respect restore restricted result result_cache resumable resume retention return returning returns reuse reverse revoke right rlike role roles rollback rolling rollup round row row_count rowdependencies rowid rownum rows rtrim rules safe salt sample save savepoint sb1 sb2 sb4 scan schema schemacheck scn scope scroll sdo_georaster sdo_topo_geometry search sec_to_time second section securefile security seed segment select self sequence sequential serializable server servererror session session_user sessions_per_user set sets settings sha sha1 sha2 share shared shared_pool short show shrink shutdown si_averagecolor si_colorhistogram si_featurelist si_positionalcolor si_stillimage si_texture siblings sid sign sin size size_t sizes skip slave sleep smalldatetimefromparts smallfile snapshot some soname sort soundex source space sparse spfile split sql sql_big_result sql_buffer_result sql_cache sql_calc_found_rows sql_small_result sql_variant_property sqlcode sqldata sqlerror sqlname sqlstate sqrt square standalone standby start starting startup statement static statistics stats_binomial_test stats_crosstab stats_ks_test stats_mode stats_mw_test stats_one_way_anova stats_t_test_ stats_t_test_indep stats_t_test_one stats_t_test_paired stats_wsr_test status std stddev stddev_pop stddev_samp stdev stop storage store stored str str_to_date straight_join strcmp strict string struct stuff style subdate subpartition subpartitions substitutable substr substring subtime subtring_index subtype success sum suspend switch switchoffset switchover sync synchronous synonym sys sys_xmlagg sysasm sysaux sysdate sysdatetimeoffset sysdba sysoper system system_user sysutcdatetime table tables tablespace tan tdo template temporary terminated tertiary_weights test than then thread through tier ties time time_format time_zone timediff timefromparts timeout timestamp timestampadd timestampdiff timezone_abbr timezone_minute timezone_region to to_base64 to_date to_days to_seconds todatetimeoffset trace tracking transaction transactional translate translation treat trigger trigger_nestlevel triggers trim truncate try_cast try_convert try_parse type ub1 ub2 ub4 ucase unarchived unbounded uncompress under undo unhex unicode uniform uninstall union unique unix_timestamp unknown unlimited unlock unpivot unrecoverable unsafe unsigned until untrusted unusable unused update updated upgrade upped upper upsert url urowid usable usage use use_stored_outlines user user_data user_resources users using utc_date utc_timestamp uuid uuid_short validate validate_password_strength validation valist value values var var_samp varcharc vari varia variab variabl variable variables variance varp varraw varrawc varray verify version versions view virtual visible void wait wallet warning warnings week weekday weekofyear wellformed when whene whenev wheneve whenever where while whitespace with within without work wrapped xdb xml xmlagg xmlattributes xmlcast xmlcolattval xmlelement xmlexists xmlforest xmlindex xmlnamespaces xmlpi xmlquery xmlroot xmlschema xmlserialize xmltable xmltype xor year year_to_month years yearweek",literal:"true false null",built_in:"array bigint binary bit blob boolean char character date dec decimal float int int8 integer interval number numeric real record serial serial8 smallint text varchar varying void"},c:[{cN:"string",b:"'",e:"'",c:[e.BE,{b:"''"}]},{cN:"string",b:'"',e:'"',c:[e.BE,{b:'""'}]},{cN:"string",b:"`",e:"`",c:[e.BE]},e.CNM,e.CBCM,t]},e.CBCM,t]}}),e}); diff --git a/js/jquery-2.1.1.min.js b/js/jquery-2.1.1.min.js old mode 100755 new mode 100644 diff --git a/js/modernizr-2.8.3.min.js b/js/modernizr-2.8.3.min.js old mode 100755 new mode 100644 diff --git a/js/theme.js b/js/theme.js old mode 100755 new mode 100644 diff --git a/media/how_to_jitpack.mp4 b/media/how_to_jitpack.mp4 old mode 100755 new mode 100644 diff --git a/search.html b/search.html old mode 100755 new mode 100644 diff --git a/search/lunr.js b/search/lunr.js old mode 100755 new mode 100644 diff --git a/search/main.js b/search/main.js old mode 100755 new mode 100644 diff --git a/search/search_index.json b/search/search_index.json old mode 100755 new mode 100644 index e375849..bd9d456 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Welcome to NewPipe. This site is/should be a beginner friendly tutorial and documentation for people who want to use or write services for the NewPipe Extractor . However, it also contains several notes about how to maintain NewPipe. It is an addition to our auto generated jdoc documentation . Please be aware that it is in its early stages, so help and feedback is always appreciated :D Introduction The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accessed like a normal API. The extractor is the core of the popular YouTube and streaming app NewPipe for Android. It is entirely independent from said platforms and also available for additional platforms as well. The beauty behind this framework is that it takes care of the extracting process, error handling etc. so you can focus on what is important: Scraping the website. It focuses on making it possible for the creator of a scraper for a streaming service to create the best outcome with the least amount of written code.","title":"Welcome to NewPipe."},{"location":"#welcome-to-newpipe","text":"This site is/should be a beginner friendly tutorial and documentation for people who want to use or write services for the NewPipe Extractor . However, it also contains several notes about how to maintain NewPipe. It is an addition to our auto generated jdoc documentation . Please be aware that it is in its early stages, so help and feedback is always appreciated :D","title":"Welcome to NewPipe."},{"location":"#introduction","text":"The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accessed like a normal API. The extractor is the core of the popular YouTube and streaming app NewPipe for Android. It is entirely independent from said platforms and also available for additional platforms as well. The beauty behind this framework is that it takes care of the extracting process, error handling etc. so you can focus on what is important: Scraping the website. It focuses on making it possible for the creator of a scraper for a streaming service to create the best outcome with the least amount of written code.","title":"Introduction"},{"location":"00_Prepare_everything/","text":"Before You Start These documents will guide you through the process of creating your own Extractor service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube and SoundCloud. The whole documentation consists of this page, which explains the general concept of the NewPipeExtractor, as well as our Jdoc setup. IMPORTANT!!! This is likely to be the worst documentation you have ever read, so do not hesitate to report if you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community and are open for everyone to help :) Setting Up Your Dev Environment First and foremost, you need to meet the following conditions in order to write your own service. What You Need to Know: A basic understanding of git Good Java knowledge A good understanding of web technology A basic understanding of unit testing and JUnit A thorough understanding of how to contribute to the NewPipe project Tools/Programs You Will Need: A dev environment/ide that supports: git Java 8 Gradle Unit testing IDEA Community (Strongly recommended, but not required) A Github account A lot of patience and excitement ;D After making sure all these conditions are provided, fork the NewPipeExtractor using the fork button . This is so you have a personal repository to develop on. Next, clone this repository into your local folder in which you want to work in. Then, import the cloned project into your IDE and run it. If all the checks are green, you did everything right! You can proceed to the next chapter. Importing the NewPipe Extractor in IntelliJ IDEA If you use IntelliJ IDEA, you should know the easy way of importing the NewPipe extractor. If you don't, here's how to do it: git clone the extractor onto your computer locally. Start IntelliJ Idea and click Import Project . Select the root directory of the NewPipe Extractor Select \" Import Project from external Model \" and then choose Gradle . In the next window, select \" Use gradle 'wrapper' task configuration \". Running \"test\" in Android Studio/IntelliJ IDEA Go to Run > Edit Configurations > Add New Configuration and select \"Gradle\". As Gradle Project, select NewPipeExtractor. As a task, add \"test\". Now save and you should be able to run. Inclusion Criteria for Services After creating you own service, you will need to submit it to our NewPipeExtractor repository. However, in order to include your changes, you need to follow these rules: Stick to our Code contribution guidelines Do not send services that present content we don't allow on NewPipe. You must be willing to maintain your service after submission. Be patient and make the requested changes when one of our maintainers rejects your code. Content That is Permitted: Any content that is not in the list of prohibited content . Any kind of pornography or NSFW content that does not violate US law. Advertising, which may need to be approved beforehand. Content That is NOT Permitted: Content that is considered NSFL (Not Safe For Life) Content that is prohibited by US federal law (Sexualization of minors, any form of violence, violations of human rights, etc). Copyrighted media, without the consent of the copyright holder/publisher.","title":"Before You Start"},{"location":"00_Prepare_everything/#before-you-start","text":"These documents will guide you through the process of creating your own Extractor service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube and SoundCloud. The whole documentation consists of this page, which explains the general concept of the NewPipeExtractor, as well as our Jdoc setup. IMPORTANT!!! This is likely to be the worst documentation you have ever read, so do not hesitate to report if you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community and are open for everyone to help :)","title":"Before You Start"},{"location":"00_Prepare_everything/#setting-up-your-dev-environment","text":"First and foremost, you need to meet the following conditions in order to write your own service.","title":"Setting Up Your Dev Environment"},{"location":"00_Prepare_everything/#what-you-need-to-know","text":"A basic understanding of git Good Java knowledge A good understanding of web technology A basic understanding of unit testing and JUnit A thorough understanding of how to contribute to the NewPipe project","title":"What You Need to Know:"},{"location":"00_Prepare_everything/#toolsprograms-you-will-need","text":"A dev environment/ide that supports: git Java 8 Gradle Unit testing IDEA Community (Strongly recommended, but not required) A Github account A lot of patience and excitement ;D After making sure all these conditions are provided, fork the NewPipeExtractor using the fork button . This is so you have a personal repository to develop on. Next, clone this repository into your local folder in which you want to work in. Then, import the cloned project into your IDE and run it. If all the checks are green, you did everything right! You can proceed to the next chapter.","title":"Tools/Programs You Will Need:"},{"location":"00_Prepare_everything/#importing-the-newpipe-extractor-in-intellij-idea","text":"If you use IntelliJ IDEA, you should know the easy way of importing the NewPipe extractor. If you don't, here's how to do it: git clone the extractor onto your computer locally. Start IntelliJ Idea and click Import Project . Select the root directory of the NewPipe Extractor Select \" Import Project from external Model \" and then choose Gradle . In the next window, select \" Use gradle 'wrapper' task configuration \".","title":"Importing the NewPipe Extractor in IntelliJ IDEA"},{"location":"00_Prepare_everything/#running-test-in-android-studiointellij-idea","text":"Go to Run > Edit Configurations > Add New Configuration and select \"Gradle\". As Gradle Project, select NewPipeExtractor. As a task, add \"test\". Now save and you should be able to run.","title":"Running \"test\" in Android Studio/IntelliJ IDEA"},{"location":"00_Prepare_everything/#inclusion-criteria-for-services","text":"After creating you own service, you will need to submit it to our NewPipeExtractor repository. However, in order to include your changes, you need to follow these rules: Stick to our Code contribution guidelines Do not send services that present content we don't allow on NewPipe. You must be willing to maintain your service after submission. Be patient and make the requested changes when one of our maintainers rejects your code.","title":"Inclusion Criteria for Services"},{"location":"00_Prepare_everything/#content-that-is-permitted","text":"Any content that is not in the list of prohibited content . Any kind of pornography or NSFW content that does not violate US law. Advertising, which may need to be approved beforehand.","title":"Content That is Permitted:"},{"location":"00_Prepare_everything/#content-that-is-not-permitted","text":"Content that is considered NSFL (Not Safe For Life) Content that is prohibited by US federal law (Sexualization of minors, any form of violence, violations of human rights, etc). Copyrighted media, without the consent of the copyright holder/publisher.","title":"Content That is NOT Permitted:"},{"location":"01_Concept_of_the_extractor/","text":"Concept of the Extractor The Collector/Extractor Pattern Before you start coding your own service, you need to understand the basic concept of the extractor itself. There is a pattern you will find all over the code, called the extractor/collector pattern. The idea behind it is that the extractor would produce fragments of data, and the collector would collect them and assemble that data into a readable format for the front end. The collector also controls the parsing process, and takes care of error handling. So, if the extractor fails at any point, the collector will decide whether or not it should continue parsing. This requires the extractor to be made out of multiple methods, one method for every data field the collector wants to have. The collectors are provided by NewPipe. You need to take care of the extractors. Usage in the Front End A typical call for retrieving data from a website would look like this: Info info; try { // Create a new Extractor with a given context provided as parameter. Extractor extractor = new Extractor(some_meta_info); // Retrieves the data form extractor and builds info package. info = Info.getInfo(extractor); } catch(Exception e) { // handle errors when collector decided to break up extraction } Typical Implementation of a Single Data Extractor The typical implementation of a single data extractor, on the other hand, would look like this: class MyExtractor extends FutureExtractor { public MyExtractor(RequiredInfo requiredInfo, ForExtraction forExtraction) { super(requiredInfo, forExtraction); ... } @Override public void fetch() { // Actually fetch the page data here } @Override public String someDataFiled() throws ExtractionException { //The exception needs to be thrown if someting failed // get piece of information and return it } ... // More datafields } Collector/Extractor Pattern for Lists Information can be represented as a list. In NewPipe, a list is represented by a InfoItemsCollector . A InfoItemCollector will collect and assemble a list of InfoItem . For each item that should be extracted, a new Extractor must be created, and given to the InfoItemCollector via commit() . If you are implementing a list for your service you need to extend InfoItem containing the extracted information and implement an InfoItemExtractor , that will return the data of one InfoItem. A common implementation would look like this: private MyInfoItemCollector collectInfoItemsFromElement(Element e) { MyInfoItemCollector collector = new MyInfoItemCollector(getServiceId()); for(final Element li : element.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return collector; } InfoItems Encapsulated in Pages When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail, and its creator. Such info can be called list header . When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. This is why a list in NewPipe lists are chopped down into smaller lists called InfoItemsPage s. Each page has its own URL, and needs to be extracted separately. Additional metadata about the list and extracting multiple pages can be handled by a ListExtractor , and its ListExtractor.InfoItemsPage . For extracting list header information it behaves like a regular extractor. For handling InfoItemsPages it adds methods such as: getInitialPage() which will return the first page of InfoItems. getNextPageUrl() If a second Page of InfoItems is available this will return the URL pointing to them. getPage() returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the getNextPageUrl() method of the previous page. The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular web page, but all the others as an AJAX request.","title":"Concept of the Extractor"},{"location":"01_Concept_of_the_extractor/#concept-of-the-extractor","text":"","title":"Concept of the Extractor"},{"location":"01_Concept_of_the_extractor/#the-collectorextractor-pattern","text":"Before you start coding your own service, you need to understand the basic concept of the extractor itself. There is a pattern you will find all over the code, called the extractor/collector pattern. The idea behind it is that the extractor would produce fragments of data, and the collector would collect them and assemble that data into a readable format for the front end. The collector also controls the parsing process, and takes care of error handling. So, if the extractor fails at any point, the collector will decide whether or not it should continue parsing. This requires the extractor to be made out of multiple methods, one method for every data field the collector wants to have. The collectors are provided by NewPipe. You need to take care of the extractors.","title":"The Collector/Extractor Pattern"},{"location":"01_Concept_of_the_extractor/#usage-in-the-front-end","text":"A typical call for retrieving data from a website would look like this: Info info; try { // Create a new Extractor with a given context provided as parameter. Extractor extractor = new Extractor(some_meta_info); // Retrieves the data form extractor and builds info package. info = Info.getInfo(extractor); } catch(Exception e) { // handle errors when collector decided to break up extraction }","title":"Usage in the Front End"},{"location":"01_Concept_of_the_extractor/#typical-implementation-of-a-single-data-extractor","text":"The typical implementation of a single data extractor, on the other hand, would look like this: class MyExtractor extends FutureExtractor { public MyExtractor(RequiredInfo requiredInfo, ForExtraction forExtraction) { super(requiredInfo, forExtraction); ... } @Override public void fetch() { // Actually fetch the page data here } @Override public String someDataFiled() throws ExtractionException { //The exception needs to be thrown if someting failed // get piece of information and return it } ... // More datafields }","title":"Typical Implementation of a Single Data Extractor"},{"location":"01_Concept_of_the_extractor/#collectorextractor-pattern-for-lists","text":"Information can be represented as a list. In NewPipe, a list is represented by a InfoItemsCollector . A InfoItemCollector will collect and assemble a list of InfoItem . For each item that should be extracted, a new Extractor must be created, and given to the InfoItemCollector via commit() . If you are implementing a list for your service you need to extend InfoItem containing the extracted information and implement an InfoItemExtractor , that will return the data of one InfoItem. A common implementation would look like this: private MyInfoItemCollector collectInfoItemsFromElement(Element e) { MyInfoItemCollector collector = new MyInfoItemCollector(getServiceId()); for(final Element li : element.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return collector; }","title":"Collector/Extractor Pattern for Lists"},{"location":"01_Concept_of_the_extractor/#infoitems-encapsulated-in-pages","text":"When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail, and its creator. Such info can be called list header . When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. This is why a list in NewPipe lists are chopped down into smaller lists called InfoItemsPage s. Each page has its own URL, and needs to be extracted separately. Additional metadata about the list and extracting multiple pages can be handled by a ListExtractor , and its ListExtractor.InfoItemsPage . For extracting list header information it behaves like a regular extractor. For handling InfoItemsPages it adds methods such as: getInitialPage() which will return the first page of InfoItems. getNextPageUrl() If a second Page of InfoItems is available this will return the URL pointing to them. getPage() returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the getNextPageUrl() method of the previous page. The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular web page, but all the others as an AJAX request.","title":"InfoItems Encapsulated in Pages"},{"location":"02_Concept_of_LinkHandler/","text":"Concept of the LinkHandler The LinkHandler represent links to resources like videos, search requests, channels, etc. The idea is that a video can have multiple links pointing to it, but it has one unique ID that represents it, like this example: oHg5SJYRHA0 can be represented as: https://www.youtube.com/watch?v=oHg5SJYRHA0 (the default URL for YouTube) https://youtu.be/oHg5SJYRHA0 (the shortened link) https://m.youtube.com/watch?v=oHg5SJYRHA0 (the link for mobile devices) Importand notes about LinkHandler: A simple LinkHandler will contain the default URL, the ID, and the original URL. LinkHandler s are read only. LinkHandler s are also used to determine which part of the extractor can handle a certain link. In order to get one you must either call fromUrl() or fromId() of the the corresponding LinkHandlerFactory . Every type of resource has its own LinkHandlerFactory . Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc. Usage The typical usage for obtaining a LinkHandler would look like this: LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory(); LinkHandler myVideo = myLinkHandlerFactory.fromUrl(\"https://my.service.com/the_video\"); Implementation In order to use LinkHandler for your service, you must override the appropriate LinkHandlerFactory. eg: class MyStreamLinkHandlerFactory extends LinkHandlerFactory { @Override public String getId(String url) throws ParsingException { // Return the ID based on the URL. } @Override public String getUrl(String id) throws ParsingException { // Return the URL based on the ID given. } @Override public boolean onAcceptUrl(String url) throws ParsingException { // Return true if this LinkHanlderFactory can handle this type of link } } ListLinkHandler and SearchQueryHandler List based resources, like channels and playlists, can be sorted and filtered. Therefore these type of resources don't just use a LinkHandler, but a class called ListLinkHandler , which inherits from LinkHandler and adds the field ContentFilter , which is used to filter by resource type, like stream or playlist, and SortFilter , which is used to sort by name, date, or view count. !!ATTENTION!! Be careful when you implement a content filter: No selected filter equals all filters selected. If your get an empty content filter list in your extractor, make sure you return everything. By all means, use \"if\" statements like contentFilter.contains(\"video\") || contentFilter.isEmpty() . ListLinkHandler are also created by overriding the ListLinkHandlerFactory additionally to the abstract methods this factory inherits from the LinkHandlerFactory you can override getAvailableContentFilter() and getAvailableSortFilter() . Through these you can tell the front end which kind of filter your service supports. SearchQueryHandler You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the same search request might have a different outcome depending on the country or the time you send the request. This is why the idea of an \"ID\" is replaced by a \"SearchString\" in the SearchQueryHandler These work like regular ListLinkHandler, except that you don't have to implement the methods onAcceptUrl() and getId() when overriding SearchQueryHandlerFactory .","title":"Concept of the LinkHandler"},{"location":"02_Concept_of_LinkHandler/#concept-of-the-linkhandler","text":"The LinkHandler represent links to resources like videos, search requests, channels, etc. The idea is that a video can have multiple links pointing to it, but it has one unique ID that represents it, like this example: oHg5SJYRHA0 can be represented as: https://www.youtube.com/watch?v=oHg5SJYRHA0 (the default URL for YouTube) https://youtu.be/oHg5SJYRHA0 (the shortened link) https://m.youtube.com/watch?v=oHg5SJYRHA0 (the link for mobile devices)","title":"Concept of the LinkHandler"},{"location":"02_Concept_of_LinkHandler/#importand-notes-about-linkhandler","text":"A simple LinkHandler will contain the default URL, the ID, and the original URL. LinkHandler s are read only. LinkHandler s are also used to determine which part of the extractor can handle a certain link. In order to get one you must either call fromUrl() or fromId() of the the corresponding LinkHandlerFactory . Every type of resource has its own LinkHandlerFactory . Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc.","title":"Importand notes about LinkHandler:"},{"location":"02_Concept_of_LinkHandler/#usage","text":"The typical usage for obtaining a LinkHandler would look like this: LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory(); LinkHandler myVideo = myLinkHandlerFactory.fromUrl(\"https://my.service.com/the_video\");","title":"Usage"},{"location":"02_Concept_of_LinkHandler/#implementation","text":"In order to use LinkHandler for your service, you must override the appropriate LinkHandlerFactory. eg: class MyStreamLinkHandlerFactory extends LinkHandlerFactory { @Override public String getId(String url) throws ParsingException { // Return the ID based on the URL. } @Override public String getUrl(String id) throws ParsingException { // Return the URL based on the ID given. } @Override public boolean onAcceptUrl(String url) throws ParsingException { // Return true if this LinkHanlderFactory can handle this type of link } }","title":"Implementation"},{"location":"02_Concept_of_LinkHandler/#listlinkhandler-and-searchqueryhandler","text":"List based resources, like channels and playlists, can be sorted and filtered. Therefore these type of resources don't just use a LinkHandler, but a class called ListLinkHandler , which inherits from LinkHandler and adds the field ContentFilter , which is used to filter by resource type, like stream or playlist, and SortFilter , which is used to sort by name, date, or view count. !!ATTENTION!! Be careful when you implement a content filter: No selected filter equals all filters selected. If your get an empty content filter list in your extractor, make sure you return everything. By all means, use \"if\" statements like contentFilter.contains(\"video\") || contentFilter.isEmpty() . ListLinkHandler are also created by overriding the ListLinkHandlerFactory additionally to the abstract methods this factory inherits from the LinkHandlerFactory you can override getAvailableContentFilter() and getAvailableSortFilter() . Through these you can tell the front end which kind of filter your service supports.","title":"ListLinkHandler and SearchQueryHandler"},{"location":"02_Concept_of_LinkHandler/#searchqueryhandler","text":"You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the same search request might have a different outcome depending on the country or the time you send the request. This is why the idea of an \"ID\" is replaced by a \"SearchString\" in the SearchQueryHandler These work like regular ListLinkHandler, except that you don't have to implement the methods onAcceptUrl() and getId() when overriding SearchQueryHandlerFactory .","title":"SearchQueryHandler"},{"location":"03_Implement_a_service/","text":"Implementing a Service Services, or better service connectors, are the parts of NewPipe which communicate with an actual service like YouTube. This page will describe how you can implement and add your own services to the extractor. Please make sure you read and understand the Concept of Extractors and the Concept of LinkHandler before continuing. Required and Optional Parts Your service does not have to implement everything; some parts are optional. This is because not all services support every feature other services support. For example, it might be that a certain service does not support channels. If so, you can leave out the implementation of channels, and make the corresponding factory method of the your StreamingService implementation return null . The frontend will handle the lack of having channels. However, if you start to implement one of the optional parts of the list below, you will have to implement all of its parts/classes. NewPipe will crash if you only implement the extractor for the list item of a channel, but not the channel extractor itself. The Parts of a Service: Head of Service Stream Search Playlist (optional) Channel (optional) Kiosk (optional) Allowed Libraries The NewPipe Extractor already includes a lot of usable tools and external libraries that should make extracting easy. For some specific (tiny) tasks, Regex is allowed. Here you can take a look at the Parser , which will give you a little help with that. Use Regex with care!!! Avoid it as often as possible. It's better to ask us to introduce a new library than start using Regex to often. Html/XML Parsing: jsoup JSON Parsing: nanojson JavaScript Parsing/Execution: Mozilla Rhino Link detection in strings: AutoLink If you need to introduce new libraries, please tell us before you do so. Head of Service First of all, if you want to create a new service, you should create a new package below org.schabi.newpipe.services , with the name of your service as package name. Parts Required to be Implemented: StreamingService ServiceInfo StreamingService is a factory class that will return objects of all important parts of your service. Every extractor, handler, and info type you add and should be part of your implementation, must be instantiated using an instance of this class. You can see it as a factory for all objects of your implementation. ServiceInfo will return some metadata about your service such as the name, capabilities, the author's name, and their email address for further notice and maintenance issues. Remember, after extending this class, you need to return an instance of it by through your implementation of StreamingService.getServiceInfo() . When these two classes are extended by you, you need to add your StreamingService to the ServiceList of NewPipe. This way, your service will become an official part of the NewPipe Extractor. Every service has an ID, which will be set when this list gets created. You need to set this ID by entering it in the constructor. So when adding your service just give it the ID of the previously last service in the list incremented by one. Stream Streams are considered single entities of video or audio. They have metadata like a title, a description, next/related videos, a thumbnail and comments. To obtain the URL to the actual stream data, as well as its metadata, StreamExtractor is used. The LinkHandlerFactory will represent a link to such a stream. StreamInfoItemExtractor will extract one item in a list of items representing such streams, like a search result or a playlist. Since every streaming service (obviously) provides streams, this is required to implement. Otherwise, your service was pretty useless :) Parts Required to be Implemented: StreamExtractor StreamInfoItemExtractor LinkHandlerFactory Search The SearchExtractor is also required to be implemented. It will take a search query represented as SearchQueryHandler and return a list of search results. Since many services support suggestions as you type, you will also want to implement a SuggestionExtractor . This will make it possible for the frontend to also display a suggestion while typing. Parts Required to be Implemented: SearchExtractor SearchQueryHandlerFactory SuggestionExtractor (optional) Playlist Playlists are lists of streams provided by the service (you might not have to be concerned over locally saved playlists, those will be handled by the frontend). A playlist may only contain StreamInfoItems , but no other InfoItem types. Parts Required to be Implemented: PlaylistExtractor PlayListInfoItemExtractor ListLinkHandlerFactory Channel A Channel is mostly a Playlist , the only difference is that it does not only represent a simple list of streams, but also a user, a channel, or any entity that could be represented as a user. This is why the metadata supported by the ChannelExtractor differs from the one of a playlist. Parts Required to be Implemented: ChannelExtractor ChannelInfoItemExtractor ListLinkHandlerFactory Kiosk A kiosk is a list of InfoItems which will be displayed on the main page of NewPipe. A kiosk is mostly similar to the content displayed on the main page of a video platform. A kiosk could be something like \"Top 20\", \"Charts\", \"News\", \"Creators Selection\" etc. Kiosks are controversial; many people may not like them. If you also don't like them, please consider your users and refrain from denying support for them. Your service would look pretty empty if you select it and no video is being displayed. Also, you should not override the preference of the user, since users of NewPipe can decide by the settings whether they want to see the kiosk page or not. Multiple Kiosks Most services will implement more than one kiosk, so a service might have a \"Top 20\" for different categories like \"Country Music\", \"Techno\", etc. This is why the extractor will let you implement multiple KioskExtractors . Since different kiosk pages might also differ with their HTML structure, every page you want to support has to be implemented as its own KioskExtractor . However, if the pages are similar, you can use the same implementation, but set the page type when you instantiate your KioskExtractor through the KioskList.KioskExtractorFactory . Every kiosk you implement needs to be added to your KioskList which you return with your StreamingService implementation. It is also important to set the default kiosk. This will be the kiosk that will be shown by the first start of your service. An example implementation of the getKioskList() could look like this: @Override public KioskList getKioskList() throws ExtractionException { KioskList list = new KioskList(getServiceId()); list.addKioskEntry(new KioskList.KioskExtractorFactory() { @Override public KioskExtractor createNewKiosk(StreamingService streamingService, String url, String id, Localization local) throws ExtractionException { return new YoutubeTrendingExtractor(YoutubeService.this, new YoutubeTrendingLinkHandlerFactory().fromUrl(url), id, local); } }, new YoutubeTrendingLinkHandlerFactory(), \"Trending\"); list.setDefaultKiosk(\"Trending\"); return list; } Parts Required to be Implemented: KioskList.KioskExtractorFactory KioskExtractor ListLinkHandlerFactory","title":"Implementing a Service"},{"location":"03_Implement_a_service/#implementing-a-service","text":"Services, or better service connectors, are the parts of NewPipe which communicate with an actual service like YouTube. This page will describe how you can implement and add your own services to the extractor. Please make sure you read and understand the Concept of Extractors and the Concept of LinkHandler before continuing.","title":"Implementing a Service"},{"location":"03_Implement_a_service/#required-and-optional-parts","text":"Your service does not have to implement everything; some parts are optional. This is because not all services support every feature other services support. For example, it might be that a certain service does not support channels. If so, you can leave out the implementation of channels, and make the corresponding factory method of the your StreamingService implementation return null . The frontend will handle the lack of having channels. However, if you start to implement one of the optional parts of the list below, you will have to implement all of its parts/classes. NewPipe will crash if you only implement the extractor for the list item of a channel, but not the channel extractor itself. The Parts of a Service: Head of Service Stream Search Playlist (optional) Channel (optional) Kiosk (optional)","title":"Required and Optional Parts"},{"location":"03_Implement_a_service/#allowed-libraries","text":"The NewPipe Extractor already includes a lot of usable tools and external libraries that should make extracting easy. For some specific (tiny) tasks, Regex is allowed. Here you can take a look at the Parser , which will give you a little help with that. Use Regex with care!!! Avoid it as often as possible. It's better to ask us to introduce a new library than start using Regex to often. Html/XML Parsing: jsoup JSON Parsing: nanojson JavaScript Parsing/Execution: Mozilla Rhino Link detection in strings: AutoLink If you need to introduce new libraries, please tell us before you do so.","title":"Allowed Libraries"},{"location":"03_Implement_a_service/#head-of-service","text":"First of all, if you want to create a new service, you should create a new package below org.schabi.newpipe.services , with the name of your service as package name. Parts Required to be Implemented: StreamingService ServiceInfo StreamingService is a factory class that will return objects of all important parts of your service. Every extractor, handler, and info type you add and should be part of your implementation, must be instantiated using an instance of this class. You can see it as a factory for all objects of your implementation. ServiceInfo will return some metadata about your service such as the name, capabilities, the author's name, and their email address for further notice and maintenance issues. Remember, after extending this class, you need to return an instance of it by through your implementation of StreamingService.getServiceInfo() . When these two classes are extended by you, you need to add your StreamingService to the ServiceList of NewPipe. This way, your service will become an official part of the NewPipe Extractor. Every service has an ID, which will be set when this list gets created. You need to set this ID by entering it in the constructor. So when adding your service just give it the ID of the previously last service in the list incremented by one.","title":"Head of Service"},{"location":"03_Implement_a_service/#stream","text":"Streams are considered single entities of video or audio. They have metadata like a title, a description, next/related videos, a thumbnail and comments. To obtain the URL to the actual stream data, as well as its metadata, StreamExtractor is used. The LinkHandlerFactory will represent a link to such a stream. StreamInfoItemExtractor will extract one item in a list of items representing such streams, like a search result or a playlist. Since every streaming service (obviously) provides streams, this is required to implement. Otherwise, your service was pretty useless :) Parts Required to be Implemented: StreamExtractor StreamInfoItemExtractor LinkHandlerFactory","title":"Stream"},{"location":"03_Implement_a_service/#search","text":"The SearchExtractor is also required to be implemented. It will take a search query represented as SearchQueryHandler and return a list of search results. Since many services support suggestions as you type, you will also want to implement a SuggestionExtractor . This will make it possible for the frontend to also display a suggestion while typing. Parts Required to be Implemented: SearchExtractor SearchQueryHandlerFactory SuggestionExtractor (optional)","title":"Search"},{"location":"03_Implement_a_service/#playlist","text":"Playlists are lists of streams provided by the service (you might not have to be concerned over locally saved playlists, those will be handled by the frontend). A playlist may only contain StreamInfoItems , but no other InfoItem types. Parts Required to be Implemented: PlaylistExtractor PlayListInfoItemExtractor ListLinkHandlerFactory","title":"Playlist"},{"location":"03_Implement_a_service/#channel","text":"A Channel is mostly a Playlist , the only difference is that it does not only represent a simple list of streams, but also a user, a channel, or any entity that could be represented as a user. This is why the metadata supported by the ChannelExtractor differs from the one of a playlist. Parts Required to be Implemented: ChannelExtractor ChannelInfoItemExtractor ListLinkHandlerFactory","title":"Channel"},{"location":"03_Implement_a_service/#kiosk","text":"A kiosk is a list of InfoItems which will be displayed on the main page of NewPipe. A kiosk is mostly similar to the content displayed on the main page of a video platform. A kiosk could be something like \"Top 20\", \"Charts\", \"News\", \"Creators Selection\" etc. Kiosks are controversial; many people may not like them. If you also don't like them, please consider your users and refrain from denying support for them. Your service would look pretty empty if you select it and no video is being displayed. Also, you should not override the preference of the user, since users of NewPipe can decide by the settings whether they want to see the kiosk page or not.","title":"Kiosk"},{"location":"03_Implement_a_service/#multiple-kiosks","text":"Most services will implement more than one kiosk, so a service might have a \"Top 20\" for different categories like \"Country Music\", \"Techno\", etc. This is why the extractor will let you implement multiple KioskExtractors . Since different kiosk pages might also differ with their HTML structure, every page you want to support has to be implemented as its own KioskExtractor . However, if the pages are similar, you can use the same implementation, but set the page type when you instantiate your KioskExtractor through the KioskList.KioskExtractorFactory . Every kiosk you implement needs to be added to your KioskList which you return with your StreamingService implementation. It is also important to set the default kiosk. This will be the kiosk that will be shown by the first start of your service. An example implementation of the getKioskList() could look like this: @Override public KioskList getKioskList() throws ExtractionException { KioskList list = new KioskList(getServiceId()); list.addKioskEntry(new KioskList.KioskExtractorFactory() { @Override public KioskExtractor createNewKiosk(StreamingService streamingService, String url, String id, Localization local) throws ExtractionException { return new YoutubeTrendingExtractor(YoutubeService.this, new YoutubeTrendingLinkHandlerFactory().fromUrl(url), id, local); } }, new YoutubeTrendingLinkHandlerFactory(), \"Trending\"); list.setDefaultKiosk(\"Trending\"); return list; } Parts Required to be Implemented: KioskList.KioskExtractorFactory KioskExtractor ListLinkHandlerFactory","title":"Multiple Kiosks"},{"location":"04_Run_changes_in_App/","text":"Testing Your Changes in the App You should develop and test your changes with the JUnit environment that is provided by the NewPipe Extractor and IDEA. If you want to try it with the actual fronted, you need to follow these steps. Setup Android Studio First, you'll want to set up a working Android Studio environment. To do this, download Studio from developer.android.com , and follow the instructions on how to set it up. Get the NewPipe Code and Run it. In order to get it, you simply clone or download it from the current dev branch github.com/TeamNewPipe/NewPipe.git . You can then build and run it following these instructions . Also, make sure you are comfortable with adb since you might experience some trouble running your compiled app on a real device, especially under Linux, where you sometimes have to adjust the udev rules in order to make your device accessible . Run Your Changes on the Extractor In order to use the extractor in our app, we use jitpack . This is a build service that can build maven *.jar packages for Android and Java based on GitHub or GitLab repositories. To use the extractor through jitpack, you need to push it to your online repository of your copy that you host either on GitHub or GitLab . It's important to host it on one of both. To copy your repository URL in HTTP format, go to jitpack and paste it there. From here, you can grab the latest commit via GET IT button. I recomend not to use a SNAPSHOT, since I am not sure when snapshot is built. An \"implementation\" string will be generated for you. Copy this string and replace the implementation 'com.github.TeamNewPipe:NewPipeExtractor:' line in the file /app/build.gradle with it. Your browser does not support the video tag. If everything synced well, then you should only see a screen with OK signs. Now you can compile and run NewPipe with the new extractor. Troubleshooting If something went wrong on jitpack site, you can check their build log, by selecting the commit you tried to build and click on that little paper symbol next to the GET IT button. If it's red, it means that the build failed.","title":"Testing Your Changes in the App"},{"location":"04_Run_changes_in_App/#testing-your-changes-in-the-app","text":"You should develop and test your changes with the JUnit environment that is provided by the NewPipe Extractor and IDEA. If you want to try it with the actual fronted, you need to follow these steps.","title":"Testing Your Changes in the App"},{"location":"04_Run_changes_in_App/#setup-android-studio","text":"First, you'll want to set up a working Android Studio environment. To do this, download Studio from developer.android.com , and follow the instructions on how to set it up.","title":"Setup Android Studio"},{"location":"04_Run_changes_in_App/#get-the-newpipe-code-and-run-it","text":"In order to get it, you simply clone or download it from the current dev branch github.com/TeamNewPipe/NewPipe.git . You can then build and run it following these instructions . Also, make sure you are comfortable with adb since you might experience some trouble running your compiled app on a real device, especially under Linux, where you sometimes have to adjust the udev rules in order to make your device accessible .","title":"Get the NewPipe Code and Run it."},{"location":"04_Run_changes_in_App/#run-your-changes-on-the-extractor","text":"In order to use the extractor in our app, we use jitpack . This is a build service that can build maven *.jar packages for Android and Java based on GitHub or GitLab repositories. To use the extractor through jitpack, you need to push it to your online repository of your copy that you host either on GitHub or GitLab . It's important to host it on one of both. To copy your repository URL in HTTP format, go to jitpack and paste it there. From here, you can grab the latest commit via GET IT button. I recomend not to use a SNAPSHOT, since I am not sure when snapshot is built. An \"implementation\" string will be generated for you. Copy this string and replace the implementation 'com.github.TeamNewPipe:NewPipeExtractor:' line in the file /app/build.gradle with it. Your browser does not support the video tag. If everything synced well, then you should only see a screen with OK signs. Now you can compile and run NewPipe with the new extractor.","title":"Run Your Changes on the Extractor"},{"location":"04_Run_changes_in_App/#troubleshooting","text":"If something went wrong on jitpack site, you can check their build log, by selecting the commit you tried to build and click on that little paper symbol next to the GET IT button. If it's red, it means that the build failed.","title":"Troubleshooting"},{"location":"05_releasing/","text":"Releasing a New NewPipe Version This site is meant for those who want to maintain NewPipe, or just want to know how releasing works. Differences Between Regular and Hotfix Releases NewPipe is a web crawler. That means it does not use a web API, but instead tries to scrape the data from the website, this however has the disadvantage of the app to break instantly when YouTube changes something. We do not know when this happen. Therefore, maintainers need to act quickly when it happens, and reduce our downtime as much as possible. The entire release cycle is therefore designed around this issue. There is a difference between a release that introduces new features and a release that fixes an issue that occurred because YouTube, or some other service, changed their website (typically called a shutdown). Lets have a look at the characteristics of a regular release , and then the characteristics of a hotfix release . Regular Releases Regular releases are normal releases like they are done in any other app. Releases are always stored on master branch. The latest commit on master is always equal to the currently released version. No development is done on master. This ensures that we always have one branch with a stable/releasable version. Feature Branching When developing, the dev branch is used. Pushing to dev directly, however, is not allowed, since QA and testing should be done first before adding something to it. This ensures that the dev version works as stable a possible. In order to change something on the app, one may want to fork the dev branch and develop the changes in their own branch (this is called feature branching). Make sure that both the dev branches, as well as the master branches of the extractor and the frontend, are compatible with each other. If a change is done on the API to the extractor, make sure that frontend is compatible, or changed to become compatible, with these changes. If the PR that should make the frontend compatible again can not be merged, please do not merge the corresponding PR on the extractor either. This should make sure that any developer can run his changes on the fronted at any time. Merging Features/Bugfixes After finishing a feature, one should open up a Pull Reuqest to the dev branch. From here, a maintainer can do Code review and Quality Assurance (QA) . If you are a maintainer, please take care about the code architecture so corrosion or code shifting can be prevented. Please also prioritize code quality over functionality. In short: cool function but bad code = no merge. Focus on leaving the code as clean as possible. You, as a maintainer, should build the app and put the signed APK into the description of that new pull request. This way, other people can test the feature/bugfix and help with QA. You may not need to do this every time. It is enough to do it on bigger pull requests. After the maintainer merges the new feature into the dev branch, he should add the title of the pull request or a summary of the changes into the release notes . Creating a New Release Once there are enough features together, and the maintainers believe that NewPipe is ready for a new release, they should create a new release. Be aware of the rule that a release should never be done on a Friday. For NewPipe, this means: Don't do a release if you don't have time for it!!! Below is a list of things you will want to do: Fork the dev branch into a new release_x.y.z branch. Increase the version number Merge weblate changes from the dev branch at https://hosted.weblate.org/git/newpipe/strings/ . Copy the release notes from the GitHub version draft into the corresponding fastlane file (see release notes ). Open up a pull request form the new release_x.y.z branch into the master branch. Create an issue pointing to the new pull request. The reason for opening an issue is that from my perception, people read issues more than pull requests. Put the release-note into this pull request. Build a signed release version of NewPipe using schabis signing keys. This is a release candidate (RC). Name the build apk file NewPipe__RC1.apk . Zip it and post it to the head of the release issue. This way, others can test the release candidate. Test and QA the new version with the help of others. Leave the PR open for a few days and advertise it to help testing. While being in release phase no new pull requests must be merged into dev branch. This procedure does not have to be done for the extractor as extractor will be tested together with the fronted. Quickfixes When issuing a new release, you will most likely encounter bugs that might not have existed in previous versions. These are called regressions . If you find a regression during release phase, you are allowed to push fixes directly into the release branch without having to fork a branch away from it. All maintainers have to be aware that they might be required to fix regressions, so plan your release at a time when you are available. Do not introduce new features during the release phase. When you have pushed a quickfix, you will want to update the release candidate you put into the issue corresponding to the release pull request . Increment the version number in the filename of the release candidate. e.g. NewPipe__RC2.apk etc. Don't update the actual version number. :P Releasing Once the glorious day of all days has come, and you fulfill the ceremony of releasing. After going through the release procedure of creating a new release and maybe a few quickfixes on the new release, this is what you should do when releasing: Click \"Merge Pull Request\". Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Make sure the draft name equals the tag name. Make sure to not have forgotten anything. Click \"Publish Release\". Rebase quickfix changes back into dev if quickfixes were made. Hotfix Releases As aforementioned, NewPipe is a web crawler and could break at any moment. In order to keep the downtime of NewPipe as low as possible, when such a shutdown happens, we allow hotfixes . A hotfix allows work on the master branch instead of the dev branch. A hotfix MUST NOT contain any features or unrelated bugfixes. A hotfix may only focus on fixing what caused the shutdown. Hotfix Branch Hotfixes work on the master branch. The dev branch has experimental changes that might have not been tested properly enough to be released, if at all. The master branch should always be the latest stable version of NewPipe. If the master branch breaks due to a shutdown, you should fix the master branch. Of course you are not allowed to push to master directly so you will have to open up a hotfix branch. If someone else is pushing a hotfix into master, and it works this can be considered as hotfix branch as well. Releasing If you fixed the issue and found it to be tested and reviewed well enough, you may release it. You don't need to undergo the full release procedure of a regular release, which takes more time to release. Keep in mind that if the hotfix might turn out to be broken after release, you should release another hotfix. It is important to release quickly for the sake of keeping NewPipe alive, and after all, a slightly broken version of NewPipe is better then a non-functional version \u00af\\_(\u30c4)_/\u00af. Here's what you do when releasing a hotfix: Click \"Merge Pull Request\" Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Create a new release draft and write the down the fix into the release notes. Copy the release note into the fastlane directory of releases. Increment the small minor version number and the versionCode . Click \"Publish Release\". Rebase the hotfix back into dev branch. Version Nomenclature The version nomenclature of NewPipe is simple. Major : The major version number (the number before the first dot) was 0 for years. The reason for this changed over time. First, I wanted this number to switch to 1 once NewPipe was feature complete. Now, I rather think of incrementing this number to 1 once we can ensure that NewPipe runs stable (part of which this documentation should help). After this, well, God knows what happens if we ever reach 1. \u00af\\_(\u30c4)_/\u00af Minor : The minor version number (the number after the first dot) will be incremented if there is a major feature added to the app. Small Minor : The small minor (the number after the second dot) will be incremented if there are bug fixes or minor features added to the app. Version Nomenclature of the Extractor The extractor is always released together with the app, therefore the version number of the extractor is identical to the one of NewPipe itself. Version Code In Android, an app can also have a versionCode . This code is a long integer and can be incremented by any value to show a device that a new version is there. For NewPipe, the version code will be incremented by 10 regardless of the change of the major or minor version number. The version codes between the 10 steps are reserved for our internal F-Droid build server. Release Notes Release notes should tell what was changed in the new version of the app. The release nodes for NewPipe are stored in the GitHub draft for a new release . When a maintainer wants to add changes to the release note, but there is no draft for a new version, they should create one. Changes can be categorized into three types: New : New features that god added to the app. Improved : Improvements to the app or existing features Fixes : Bugfixes When releasing a new version of NewPipe, before actually clicking \"Release\", the maintainer should copy the release notes from the draft and put it into a file called .txt (whereas needs to be the version code of the incoming release). This file must be stored in the directory /fastlane/metadata/android/en-US/changelogs . This way, F-Droid will be able to show the changes done to the app.","title":"Releasing a New NewPipe Version"},{"location":"05_releasing/#releasing-a-new-newpipe-version","text":"This site is meant for those who want to maintain NewPipe, or just want to know how releasing works.","title":"Releasing a New NewPipe Version"},{"location":"05_releasing/#differences-between-regular-and-hotfix-releases","text":"NewPipe is a web crawler. That means it does not use a web API, but instead tries to scrape the data from the website, this however has the disadvantage of the app to break instantly when YouTube changes something. We do not know when this happen. Therefore, maintainers need to act quickly when it happens, and reduce our downtime as much as possible. The entire release cycle is therefore designed around this issue. There is a difference between a release that introduces new features and a release that fixes an issue that occurred because YouTube, or some other service, changed their website (typically called a shutdown). Lets have a look at the characteristics of a regular release , and then the characteristics of a hotfix release .","title":"Differences Between Regular and Hotfix Releases"},{"location":"05_releasing/#regular-releases","text":"Regular releases are normal releases like they are done in any other app. Releases are always stored on master branch. The latest commit on master is always equal to the currently released version. No development is done on master. This ensures that we always have one branch with a stable/releasable version.","title":"Regular Releases"},{"location":"05_releasing/#feature-branching","text":"When developing, the dev branch is used. Pushing to dev directly, however, is not allowed, since QA and testing should be done first before adding something to it. This ensures that the dev version works as stable a possible. In order to change something on the app, one may want to fork the dev branch and develop the changes in their own branch (this is called feature branching). Make sure that both the dev branches, as well as the master branches of the extractor and the frontend, are compatible with each other. If a change is done on the API to the extractor, make sure that frontend is compatible, or changed to become compatible, with these changes. If the PR that should make the frontend compatible again can not be merged, please do not merge the corresponding PR on the extractor either. This should make sure that any developer can run his changes on the fronted at any time.","title":"Feature Branching"},{"location":"05_releasing/#merging-featuresbugfixes","text":"After finishing a feature, one should open up a Pull Reuqest to the dev branch. From here, a maintainer can do Code review and Quality Assurance (QA) . If you are a maintainer, please take care about the code architecture so corrosion or code shifting can be prevented. Please also prioritize code quality over functionality. In short: cool function but bad code = no merge. Focus on leaving the code as clean as possible. You, as a maintainer, should build the app and put the signed APK into the description of that new pull request. This way, other people can test the feature/bugfix and help with QA. You may not need to do this every time. It is enough to do it on bigger pull requests. After the maintainer merges the new feature into the dev branch, he should add the title of the pull request or a summary of the changes into the release notes .","title":"Merging Features/Bugfixes"},{"location":"05_releasing/#creating-a-new-release","text":"Once there are enough features together, and the maintainers believe that NewPipe is ready for a new release, they should create a new release. Be aware of the rule that a release should never be done on a Friday. For NewPipe, this means: Don't do a release if you don't have time for it!!! Below is a list of things you will want to do: Fork the dev branch into a new release_x.y.z branch. Increase the version number Merge weblate changes from the dev branch at https://hosted.weblate.org/git/newpipe/strings/ . Copy the release notes from the GitHub version draft into the corresponding fastlane file (see release notes ). Open up a pull request form the new release_x.y.z branch into the master branch. Create an issue pointing to the new pull request. The reason for opening an issue is that from my perception, people read issues more than pull requests. Put the release-note into this pull request. Build a signed release version of NewPipe using schabis signing keys. This is a release candidate (RC). Name the build apk file NewPipe__RC1.apk . Zip it and post it to the head of the release issue. This way, others can test the release candidate. Test and QA the new version with the help of others. Leave the PR open for a few days and advertise it to help testing. While being in release phase no new pull requests must be merged into dev branch. This procedure does not have to be done for the extractor as extractor will be tested together with the fronted.","title":"Creating a New Release"},{"location":"05_releasing/#quickfixes","text":"When issuing a new release, you will most likely encounter bugs that might not have existed in previous versions. These are called regressions . If you find a regression during release phase, you are allowed to push fixes directly into the release branch without having to fork a branch away from it. All maintainers have to be aware that they might be required to fix regressions, so plan your release at a time when you are available. Do not introduce new features during the release phase. When you have pushed a quickfix, you will want to update the release candidate you put into the issue corresponding to the release pull request . Increment the version number in the filename of the release candidate. e.g. NewPipe__RC2.apk etc. Don't update the actual version number. :P","title":"Quickfixes"},{"location":"05_releasing/#releasing","text":"Once the glorious day of all days has come, and you fulfill the ceremony of releasing. After going through the release procedure of creating a new release and maybe a few quickfixes on the new release, this is what you should do when releasing: Click \"Merge Pull Request\". Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Make sure the draft name equals the tag name. Make sure to not have forgotten anything. Click \"Publish Release\". Rebase quickfix changes back into dev if quickfixes were made.","title":"Releasing"},{"location":"05_releasing/#hotfix-releases","text":"As aforementioned, NewPipe is a web crawler and could break at any moment. In order to keep the downtime of NewPipe as low as possible, when such a shutdown happens, we allow hotfixes . A hotfix allows work on the master branch instead of the dev branch. A hotfix MUST NOT contain any features or unrelated bugfixes. A hotfix may only focus on fixing what caused the shutdown.","title":"Hotfix Releases"},{"location":"05_releasing/#hotfix-branch","text":"Hotfixes work on the master branch. The dev branch has experimental changes that might have not been tested properly enough to be released, if at all. The master branch should always be the latest stable version of NewPipe. If the master branch breaks due to a shutdown, you should fix the master branch. Of course you are not allowed to push to master directly so you will have to open up a hotfix branch. If someone else is pushing a hotfix into master, and it works this can be considered as hotfix branch as well.","title":"Hotfix Branch"},{"location":"05_releasing/#releasing_1","text":"If you fixed the issue and found it to be tested and reviewed well enough, you may release it. You don't need to undergo the full release procedure of a regular release, which takes more time to release. Keep in mind that if the hotfix might turn out to be broken after release, you should release another hotfix. It is important to release quickly for the sake of keeping NewPipe alive, and after all, a slightly broken version of NewPipe is better then a non-functional version \u00af\\_(\u30c4)_/\u00af. Here's what you do when releasing a hotfix: Click \"Merge Pull Request\" Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Create a new release draft and write the down the fix into the release notes. Copy the release note into the fastlane directory of releases. Increment the small minor version number and the versionCode . Click \"Publish Release\". Rebase the hotfix back into dev branch.","title":"Releasing"},{"location":"05_releasing/#version-nomenclature","text":"The version nomenclature of NewPipe is simple. Major : The major version number (the number before the first dot) was 0 for years. The reason for this changed over time. First, I wanted this number to switch to 1 once NewPipe was feature complete. Now, I rather think of incrementing this number to 1 once we can ensure that NewPipe runs stable (part of which this documentation should help). After this, well, God knows what happens if we ever reach 1. \u00af\\_(\u30c4)_/\u00af Minor : The minor version number (the number after the first dot) will be incremented if there is a major feature added to the app. Small Minor : The small minor (the number after the second dot) will be incremented if there are bug fixes or minor features added to the app.","title":"Version Nomenclature"},{"location":"05_releasing/#version-nomenclature-of-the-extractor","text":"The extractor is always released together with the app, therefore the version number of the extractor is identical to the one of NewPipe itself.","title":"Version Nomenclature of the Extractor"},{"location":"05_releasing/#version-code","text":"In Android, an app can also have a versionCode . This code is a long integer and can be incremented by any value to show a device that a new version is there. For NewPipe, the version code will be incremented by 10 regardless of the change of the major or minor version number. The version codes between the 10 steps are reserved for our internal F-Droid build server.","title":"Version Code"},{"location":"05_releasing/#release-notes","text":"Release notes should tell what was changed in the new version of the app. The release nodes for NewPipe are stored in the GitHub draft for a new release . When a maintainer wants to add changes to the release note, but there is no draft for a new version, they should create one. Changes can be categorized into three types: New : New features that god added to the app. Improved : Improvements to the app or existing features Fixes : Bugfixes When releasing a new version of NewPipe, before actually clicking \"Release\", the maintainer should copy the release notes from the draft and put it into a file called .txt (whereas needs to be the version code of the incoming release). This file must be stored in the directory /fastlane/metadata/android/en-US/changelogs . This way, F-Droid will be able to show the changes done to the app.","title":"Release Notes"},{"location":"06_documentation/","text":"About This Documentation The documentation you are currently reading was written using mkdocs . It is a tool that will generate a static website based on markdown files. Markdown has the advantage that it is simple to read and write, and that there are several tools that can translate a markdown file into languages like HTML or LaTeX. Installation Mkdocs is written in Python and is distributed through the Python internal package manager pip , thus you need to get python and pip running on your operating system first. Windows Download the latest Python3 version. When running the setup program, make sure to tick, \"Add Python 3.x to PATH\". Install Python. Open PowerShell or cmd.exe and type: pip3 install mkdocs . MacOS MacOS already includes Python, however, pip is still missing. The easiest and most nondestructive way is to install the MacOS package manager, homebrew , first. The advantage of homebrew is that it will only modify your home directory, and not the root dir, so your OS will not be tampered with. Install homebrew . Install Python from homebrew, which will also install pip. Enter this command: brew install python . Install mkdocs: pip3 install mkdocs Linux/*BSD Linux/*BSD also has Python preinstalled. Most distributions also contain pip by default. If it is not installed, you may need to figure out how to install pip3 through the package manager of your system. Install pip3 with these commands according to distributions: Ubuntu/Mint : apt install python3-pip Fedora/CentOS : sudo dnf install python3-pip Arch/Manjaro : sudo pacman -S python-pip openSuse : sudo zypper install python-pip *BSD : You are already advanced enough to know how you can force the bits on your disk to become pip by meditating upon it. Run pip3 install mkdocs to install mkdocs only for the current user, or run sudo pip3 install mkdocs to install mkdocs systemwide. Last one has the higher chance to work properly. Android/ChromeOS This might sound funny, but according to the growing amount of Chromebooks and Android tablets with keyboards, this might actually be useful. Install the Termux App from f-droid . Launch Termux and type apt update Install Python and git with the command: apt install git python Install mkdocs with pip install mkdocs . From herein, everything will be the same as on Desktop. If you want to edit the files, you can (besides vim or emacs which are available through Termux) use your preferred text editor on Android. This is possible by opening the files with the Termux integration of the build in android file manager: Updating Sometimes, mkdocs changes the way of how it serves, or the syntax will differ. This is why you should make sure to always run the latest version of mkdocs. To check, simply run pip3 install --upgrade mkdocs or sudo pip3 install --upgrade mkdocs if you installed pip system wide on a Linux/BSD* system. Using mkdocs In order to extend this documentation, you have to clone it from its GitHub repository . When you clone it, you will find a mkdocs.yml file, and a docs directory inside. The yaml file is the config file while in the directory docs the documentation files are stored. Here is a guide about how to use mkdocs. Write and Deploy If you are writing a documentation page and want a live preview of it, you can enter the root directory of this documentation project, and then run mkdocs serve this will start the mkdocs internal web server on port 8000 . So all you have to do is type localhost:8000 into the address bar of your browser, and here you go. If you modify a file, and save it, mkdocs will reload the page and show you the new content. If you want to deploy the page so it will be up to date at the GitHub pages , simply type mkdocs gh-deploy . However, please be aware that this will not push your changes to the master branch of the repository. So, you still have to commit and push your changes to the actual git repository of this documentation. Please be aware that only privileged maintainers can do this.","title":"About This Documentation"},{"location":"06_documentation/#about-this-documentation","text":"The documentation you are currently reading was written using mkdocs . It is a tool that will generate a static website based on markdown files. Markdown has the advantage that it is simple to read and write, and that there are several tools that can translate a markdown file into languages like HTML or LaTeX.","title":"About This Documentation"},{"location":"06_documentation/#installation","text":"Mkdocs is written in Python and is distributed through the Python internal package manager pip , thus you need to get python and pip running on your operating system first.","title":"Installation"},{"location":"06_documentation/#windows","text":"Download the latest Python3 version. When running the setup program, make sure to tick, \"Add Python 3.x to PATH\". Install Python. Open PowerShell or cmd.exe and type: pip3 install mkdocs .","title":"Windows"},{"location":"06_documentation/#macos","text":"MacOS already includes Python, however, pip is still missing. The easiest and most nondestructive way is to install the MacOS package manager, homebrew , first. The advantage of homebrew is that it will only modify your home directory, and not the root dir, so your OS will not be tampered with. Install homebrew . Install Python from homebrew, which will also install pip. Enter this command: brew install python . Install mkdocs: pip3 install mkdocs","title":"MacOS"},{"location":"06_documentation/#linuxbsd","text":"Linux/*BSD also has Python preinstalled. Most distributions also contain pip by default. If it is not installed, you may need to figure out how to install pip3 through the package manager of your system. Install pip3 with these commands according to distributions: Ubuntu/Mint : apt install python3-pip Fedora/CentOS : sudo dnf install python3-pip Arch/Manjaro : sudo pacman -S python-pip openSuse : sudo zypper install python-pip *BSD : You are already advanced enough to know how you can force the bits on your disk to become pip by meditating upon it. Run pip3 install mkdocs to install mkdocs only for the current user, or run sudo pip3 install mkdocs to install mkdocs systemwide. Last one has the higher chance to work properly.","title":"Linux/*BSD"},{"location":"06_documentation/#androidchromeos","text":"This might sound funny, but according to the growing amount of Chromebooks and Android tablets with keyboards, this might actually be useful. Install the Termux App from f-droid . Launch Termux and type apt update Install Python and git with the command: apt install git python Install mkdocs with pip install mkdocs . From herein, everything will be the same as on Desktop. If you want to edit the files, you can (besides vim or emacs which are available through Termux) use your preferred text editor on Android. This is possible by opening the files with the Termux integration of the build in android file manager:","title":"Android/ChromeOS"},{"location":"06_documentation/#updating","text":"Sometimes, mkdocs changes the way of how it serves, or the syntax will differ. This is why you should make sure to always run the latest version of mkdocs. To check, simply run pip3 install --upgrade mkdocs or sudo pip3 install --upgrade mkdocs if you installed pip system wide on a Linux/BSD* system.","title":"Updating"},{"location":"06_documentation/#using-mkdocs","text":"In order to extend this documentation, you have to clone it from its GitHub repository . When you clone it, you will find a mkdocs.yml file, and a docs directory inside. The yaml file is the config file while in the directory docs the documentation files are stored. Here is a guide about how to use mkdocs.","title":"Using mkdocs"},{"location":"06_documentation/#write-and-deploy","text":"If you are writing a documentation page and want a live preview of it, you can enter the root directory of this documentation project, and then run mkdocs serve this will start the mkdocs internal web server on port 8000 . So all you have to do is type localhost:8000 into the address bar of your browser, and here you go. If you modify a file, and save it, mkdocs will reload the page and show you the new content. If you want to deploy the page so it will be up to date at the GitHub pages , simply type mkdocs gh-deploy . However, please be aware that this will not push your changes to the master branch of the repository. So, you still have to commit and push your changes to the actual git repository of this documentation. Please be aware that only privileged maintainers can do this.","title":"Write and Deploy"},{"location":"07_maintainers_view/","text":"Maintainers View So I want to document some of the views i have when maintaining NewPipe. Keep it Streamlined NewPipe is a Player for online videos on a smart phone, by means it is used for entertainment reason. This means it does not have to be some professional application, and it does not have to be complicated to be used. However NewPipe might not focus on the casual user completely as there are many features that are a bit more \"tecki\" and may require some knowledge about technology, however all in all NewPipe should be easy to use, even for not teck guys. NewPipe does not have to be a air plane cockpit: Don't add to much special features . If people want to do professionally things with Videos they might use professional tools. Design the UI so it does make sense to the user . Try to make it comply with material design guidelines . Don't add to much features : Think about the Betamax vs. VHS phenomena or the Unix principle of having one program designed for one specific task: If you add to much functionality you add complexity and this is not appealing to the user. Focus on what NewPipe should be, and make it be only that. Bugfixes ] Disclaimer: This is a meme maybe in real live it is different. Pleas no shit storm. Always go for Bugfixes , as the best application with the best features does not help much if it is broken, or annoying to use. Now if a program is in an early stage it is quite understandable that many things brake. This is one reason why NewPipe still has no 1 in the beginning of its version number. However by now NewPipe is in a stage where there should be a strong focus on stability. If there are multiple Pull requests open, check the ones with the bugfixes first. Do not add to much features every version, as every feature will inevitable introduce more bugs. It is quite ok, if PRs stay open for a while (not to long though). If there are bugs that are stale, or open for a while bump them from time to time, so devs know that there is still something left to fix. Never accept bugs. From my perception the community does not like to fix bugs, this is why you as a maintainer should especially focus on perusing bugs. Features Well features are also something that can cause a headache. You should always see adding features critical and question whether that features does make sense, is useful and would actually be an advantage for the app. You should not blindly say yes to features even if they are small, however you should also not directly say no as well. Think about it, may be even for days before deciding whether you want to accept a feature or not. If you are not sure, try it, look into the code, speak with the developer, and then make a decision and justify it. The criteria whether to add a feature or not should be: Is the features just requested by one or two people or was the feature requested by multiple people? Is the code of the feature written well? Is it a quick and hacky solution and could a proper solution be implemented later on? Does the amount of code justify the outcome? Maybe people will send a pull request that will add a frequently requested feature, but is implemented in a hacky way, than don't add it, as you might get into trouble with that solution later on. Either through problems of extending the feature, by introducing to much bugs or simply by braking the architecture or the philosophy of NewPipe. If so don't add it. PRs If a PR contains one or more features/bugs be curious. The more stuff a PR changes the longer it will take to be added. Also there might be things you are ok with, but then there are other parts that are not ok with and because of these you can't merge it. This is why you should insist to make the dev chop down the PR into multiple smaller PRs if it's possible. Community When you talk to the community stay friendly and respectful, and make sure a friendly and respectful tone will stay. When you have a bad day just don't go to GitHub (an advice from my experience ;D ).","title":"Maintainers View"},{"location":"07_maintainers_view/#maintainers-view","text":"So I want to document some of the views i have when maintaining NewPipe.","title":"Maintainers View"},{"location":"07_maintainers_view/#keep-it-streamlined","text":"NewPipe is a Player for online videos on a smart phone, by means it is used for entertainment reason. This means it does not have to be some professional application, and it does not have to be complicated to be used. However NewPipe might not focus on the casual user completely as there are many features that are a bit more \"tecki\" and may require some knowledge about technology, however all in all NewPipe should be easy to use, even for not teck guys. NewPipe does not have to be a air plane cockpit: Don't add to much special features . If people want to do professionally things with Videos they might use professional tools. Design the UI so it does make sense to the user . Try to make it comply with material design guidelines . Don't add to much features : Think about the Betamax vs. VHS phenomena or the Unix principle of having one program designed for one specific task: If you add to much functionality you add complexity and this is not appealing to the user. Focus on what NewPipe should be, and make it be only that.","title":"Keep it Streamlined"},{"location":"07_maintainers_view/#bugfixes","text":"] Disclaimer: This is a meme maybe in real live it is different. Pleas no shit storm. Always go for Bugfixes , as the best application with the best features does not help much if it is broken, or annoying to use. Now if a program is in an early stage it is quite understandable that many things brake. This is one reason why NewPipe still has no 1 in the beginning of its version number. However by now NewPipe is in a stage where there should be a strong focus on stability. If there are multiple Pull requests open, check the ones with the bugfixes first. Do not add to much features every version, as every feature will inevitable introduce more bugs. It is quite ok, if PRs stay open for a while (not to long though). If there are bugs that are stale, or open for a while bump them from time to time, so devs know that there is still something left to fix. Never accept bugs. From my perception the community does not like to fix bugs, this is why you as a maintainer should especially focus on perusing bugs.","title":"Bugfixes"},{"location":"07_maintainers_view/#features","text":"Well features are also something that can cause a headache. You should always see adding features critical and question whether that features does make sense, is useful and would actually be an advantage for the app. You should not blindly say yes to features even if they are small, however you should also not directly say no as well. Think about it, may be even for days before deciding whether you want to accept a feature or not. If you are not sure, try it, look into the code, speak with the developer, and then make a decision and justify it. The criteria whether to add a feature or not should be: Is the features just requested by one or two people or was the feature requested by multiple people? Is the code of the feature written well? Is it a quick and hacky solution and could a proper solution be implemented later on? Does the amount of code justify the outcome? Maybe people will send a pull request that will add a frequently requested feature, but is implemented in a hacky way, than don't add it, as you might get into trouble with that solution later on. Either through problems of extending the feature, by introducing to much bugs or simply by braking the architecture or the philosophy of NewPipe. If so don't add it.","title":"Features"},{"location":"07_maintainers_view/#prs","text":"If a PR contains one or more features/bugs be curious. The more stuff a PR changes the longer it will take to be added. Also there might be things you are ok with, but then there are other parts that are not ok with and because of these you can't merge it. This is why you should insist to make the dev chop down the PR into multiple smaller PRs if it's possible.","title":"PRs"},{"location":"07_maintainers_view/#community","text":"When you talk to the community stay friendly and respectful, and make sure a friendly and respectful tone will stay. When you have a bad day just don't go to GitHub (an advice from my experience ;D ).","title":"Community"}]} \ No newline at end of file +{"config":{"lang":["en"],"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Welcome to NewPipe. This site is/should be a beginner friendly tutorial and documentation for people who want to use or write services for the NewPipe Extractor . However, it also contains several notes about how to maintain NewPipe. It is an addition to our auto generated jdoc documentation . Please be aware that it is in its early stages, so help and feedback is always appreciated :D Introduction The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accessed like a normal API. The extractor is the core of the popular YouTube and streaming app NewPipe for Android. It is entirely independent from said platforms and also available for additional platforms as well. The beauty behind this framework is that it takes care of the extracting process, error handling etc. so you can focus on what is important: Scraping the website. It focuses on making it possible for the creator of a scraper for a streaming service to create the best outcome with the least amount of written code.","title":"Welcome to NewPipe."},{"location":"#welcome-to-newpipe","text":"This site is/should be a beginner friendly tutorial and documentation for people who want to use or write services for the NewPipe Extractor . However, it also contains several notes about how to maintain NewPipe. It is an addition to our auto generated jdoc documentation . Please be aware that it is in its early stages, so help and feedback is always appreciated :D","title":"Welcome to NewPipe."},{"location":"#introduction","text":"The NewPipeExtractor is a Java framework for scraping video platform websites in a way that they can be accessed like a normal API. The extractor is the core of the popular YouTube and streaming app NewPipe for Android. It is entirely independent from said platforms and also available for additional platforms as well. The beauty behind this framework is that it takes care of the extracting process, error handling etc. so you can focus on what is important: Scraping the website. It focuses on making it possible for the creator of a scraper for a streaming service to create the best outcome with the least amount of written code.","title":"Introduction"},{"location":"00_Prepare_everything/","text":"Before You Start These documents will guide you through the process of understanding or creating your own Extractor service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube, SoundCloud and MediaCCC. The whole documentation consists of this page and Jdoc setup, which explains the general concept of the NewPipeExtractor. IMPORTANT!!! This is likely to be the worst documentation you have ever read, so do not hesitate to report if you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community and are open for everyone to help :) Setting Up Your Dev Environment First and foremost, you need to meet the following conditions in order to write your own service. What You Need to Know: A basic understanding of git Good Java knowledge A good understanding of web technology A basic understanding of unit testing and JUnit A thorough understanding of how to contribute to the NewPipe project Tools/Programs You Will Need: A dev environment/ide that supports: git Java 8 Gradle Unit testing IDEA Community (Strongly recommended, but not required) A Github account A lot of patience and excitement ;D After making sure all these conditions are provided, fork the NewPipeExtractor using the fork button . This is so you have a personal repository to develop on. Next, clone this repository into your local folder in which you want to work in. Then, import the cloned project into your IDE and run it. If all the checks are green, you did everything right! You can proceed to the next chapter. Importing the NewPipe Extractor in IntelliJ IDEA If you use IntelliJ IDEA, you should know the easy way of importing the NewPipe extractor. If you don't, here's how to do it: git clone the extractor onto your computer locally. Start IntelliJ Idea and click Import Project . Select the root directory of the NewPipe Extractor Select \" Import Project from external Model \" and then choose Gradle . In the next window, select \" Use gradle 'wrapper' task configuration \". Running \"test\" in Android Studio/IntelliJ IDEA Go to Run > Edit Configurations > Add New Configuration and select \"Gradle\". As Gradle Project, select NewPipeExtractor. As a task, add \"test\". Now save and you should be able to run. Inclusion Criteria for Services After creating you own service, you will need to submit it to our NewPipeExtractor repository. However, in order to include your changes, you need to follow these rules: Stick to our Code contribution guidelines Do not send services that present content we don't allow on NewPipe. You must be willing to maintain your service after submission. Be patient and make the requested changes when one of our maintainers rejects your code. Content That is Permitted: Any content that is not in the list of prohibited content . Any kind of pornography or NSFW content that does not violate US law. Advertising, which may need to be approved beforehand. Content That is NOT Permitted: Content that is considered NSFL (Not Safe For Life) Content that is prohibited by US federal law (Sexualization of minors, any form of violence, violations of human rights, etc). Copyrighted media, without the consent of the copyright holder/publisher.","title":"Before You Start"},{"location":"00_Prepare_everything/#before-you-start","text":"These documents will guide you through the process of understanding or creating your own Extractor service of which will enable NewPipe to access additional streaming services, such as the currently supported YouTube, SoundCloud and MediaCCC. The whole documentation consists of this page and Jdoc setup, which explains the general concept of the NewPipeExtractor. IMPORTANT!!! This is likely to be the worst documentation you have ever read, so do not hesitate to report if you find any spelling errors, incomplete parts or you simply don't understand something. We are an open community and are open for everyone to help :)","title":"Before You Start"},{"location":"00_Prepare_everything/#setting-up-your-dev-environment","text":"First and foremost, you need to meet the following conditions in order to write your own service.","title":"Setting Up Your Dev Environment"},{"location":"00_Prepare_everything/#what-you-need-to-know","text":"A basic understanding of git Good Java knowledge A good understanding of web technology A basic understanding of unit testing and JUnit A thorough understanding of how to contribute to the NewPipe project","title":"What You Need to Know:"},{"location":"00_Prepare_everything/#toolsprograms-you-will-need","text":"A dev environment/ide that supports: git Java 8 Gradle Unit testing IDEA Community (Strongly recommended, but not required) A Github account A lot of patience and excitement ;D After making sure all these conditions are provided, fork the NewPipeExtractor using the fork button . This is so you have a personal repository to develop on. Next, clone this repository into your local folder in which you want to work in. Then, import the cloned project into your IDE and run it. If all the checks are green, you did everything right! You can proceed to the next chapter.","title":"Tools/Programs You Will Need:"},{"location":"00_Prepare_everything/#importing-the-newpipe-extractor-in-intellij-idea","text":"If you use IntelliJ IDEA, you should know the easy way of importing the NewPipe extractor. If you don't, here's how to do it: git clone the extractor onto your computer locally. Start IntelliJ Idea and click Import Project . Select the root directory of the NewPipe Extractor Select \" Import Project from external Model \" and then choose Gradle . In the next window, select \" Use gradle 'wrapper' task configuration \".","title":"Importing the NewPipe Extractor in IntelliJ IDEA"},{"location":"00_Prepare_everything/#running-test-in-android-studiointellij-idea","text":"Go to Run > Edit Configurations > Add New Configuration and select \"Gradle\". As Gradle Project, select NewPipeExtractor. As a task, add \"test\". Now save and you should be able to run.","title":"Running \"test\" in Android Studio/IntelliJ IDEA"},{"location":"00_Prepare_everything/#inclusion-criteria-for-services","text":"After creating you own service, you will need to submit it to our NewPipeExtractor repository. However, in order to include your changes, you need to follow these rules: Stick to our Code contribution guidelines Do not send services that present content we don't allow on NewPipe. You must be willing to maintain your service after submission. Be patient and make the requested changes when one of our maintainers rejects your code.","title":"Inclusion Criteria for Services"},{"location":"00_Prepare_everything/#content-that-is-permitted","text":"Any content that is not in the list of prohibited content . Any kind of pornography or NSFW content that does not violate US law. Advertising, which may need to be approved beforehand.","title":"Content That is Permitted:"},{"location":"00_Prepare_everything/#content-that-is-not-permitted","text":"Content that is considered NSFL (Not Safe For Life) Content that is prohibited by US federal law (Sexualization of minors, any form of violence, violations of human rights, etc). Copyrighted media, without the consent of the copyright holder/publisher.","title":"Content That is NOT Permitted:"},{"location":"01_Concept_of_the_extractor/","text":"Concept of the Extractor The Collector/Extractor Pattern Before you start coding your own service, you need to understand the basic concept of the extractor itself. There is a pattern you will find all over the code, called the extractor/collector pattern. The idea behind it is that the extractor would produce fragments of data, and the collector would collect them and assemble that data into a readable format for the front end. The collector also controls the parsing process, and takes care of error handling. So, if the extractor fails at any point, the collector will decide whether or not it should continue parsing. This requires the extractor to be made out of multiple methods, one method for every data field the collector wants to have. The collectors are provided by NewPipe. You need to take care of the extractors. Usage in the Front End A typical call for retrieving data from a website would look like this: Info info; try { // Create a new Extractor with a given context provided as parameter. Extractor extractor = new Extractor(some_meta_info); // Retrieves the data form extractor and builds info package. info = Info.getInfo(extractor); } catch(Exception e) { // handle errors when collector decided to break up extraction } Typical Implementation of a Single Data Extractor The typical implementation of a single data extractor, on the other hand, would look like this: class MyExtractor extends FutureExtractor { public MyExtractor(RequiredInfo requiredInfo, ForExtraction forExtraction) { super(requiredInfo, forExtraction); ... } @Override public void fetch() { // Actually fetch the page data here } @Override public String someDataFiled() throws ExtractionException { //The exception needs to be thrown if someting failed // get piece of information and return it } ... // More datafields } Collector/Extractor Pattern for Lists Information can be represented as a list. In NewPipe, a list is represented by a InfoItemsCollector . A InfoItemsCollector will collect and assemble a list of InfoItem . For each item that should be extracted, a new Extractor must be created, and given to the InfoItemsCollector via commit() . If you are implementing a list in your service you need to implement an InfoItemExtractor , that will be able to retreve data for one and only one InfoItem. This extractor will then be comitted to the InfoItemsCollector that can collect the type of InfoItems you want to generate. A common implementation would look like this: private SomeInfoItemCollector collectInfoItemsFromElement(Element e) { // See *Some* as something like Stream or Channel // e.g. StreamInfoItemsCollector, and ChannelInfoItemsCollector are provided by NP SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId()); for(final Element li : element.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return collector; } ListExtractor There is more to know about lists: When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail, and its creator. Such info can be called list header . When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. Both of these Problems are fixed by the ListExtractor which takes care about extracting additional metadata about the liast, and by chopping down lists into several pages, so called InfoItemsPage s. Each page has its own URL, and needs to be extracted separately. For extracting list header information a ListExtractor behaves like a regular extractor. For handling InfoItemsPages it adds methods such as: getInitialPage() which will return the first page of InfoItems. getNextPageUrl() If a second Page of InfoItems is available this will return the URL pointing to them. getPage() returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the getNextPageUrl() method of the previous page. The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular web page, but all the others as an AJAX request. An InfoItemsPage itself has two constructors which take these parameters: - The InfoitemsCollector of the list that the page should represent - A nextPageUrl which represents the url of the following page (may be null if not page follows). - Optionally errors which is a list of Exceptions that may have happened during extracton. Here is a simplified reference implementation of a list extractor that only extracts pages, but not metadata: class MyListExtractor extends ListExtractor { ... private Document document; ... public InfoItemsPage getPage(pageUrl) throws ExtractionException { SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId()); document = myFunctionToGetThePageHTMLWhatever(pageUrl); //remember this part from the simple list extraction for(final Element li : document.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return new InfoItemsPage(collector, myFunctionToGetTheNextPageUrl(document)); } public InfoItemsPage getInitialPage() { //document here got initialzied by the fetch() function. return getPage(getTheCurrentPageUrl(document)); } ... }","title":"Concept of the Extractor"},{"location":"01_Concept_of_the_extractor/#concept-of-the-extractor","text":"","title":"Concept of the Extractor"},{"location":"01_Concept_of_the_extractor/#the-collectorextractor-pattern","text":"Before you start coding your own service, you need to understand the basic concept of the extractor itself. There is a pattern you will find all over the code, called the extractor/collector pattern. The idea behind it is that the extractor would produce fragments of data, and the collector would collect them and assemble that data into a readable format for the front end. The collector also controls the parsing process, and takes care of error handling. So, if the extractor fails at any point, the collector will decide whether or not it should continue parsing. This requires the extractor to be made out of multiple methods, one method for every data field the collector wants to have. The collectors are provided by NewPipe. You need to take care of the extractors.","title":"The Collector/Extractor Pattern"},{"location":"01_Concept_of_the_extractor/#usage-in-the-front-end","text":"A typical call for retrieving data from a website would look like this: Info info; try { // Create a new Extractor with a given context provided as parameter. Extractor extractor = new Extractor(some_meta_info); // Retrieves the data form extractor and builds info package. info = Info.getInfo(extractor); } catch(Exception e) { // handle errors when collector decided to break up extraction }","title":"Usage in the Front End"},{"location":"01_Concept_of_the_extractor/#typical-implementation-of-a-single-data-extractor","text":"The typical implementation of a single data extractor, on the other hand, would look like this: class MyExtractor extends FutureExtractor { public MyExtractor(RequiredInfo requiredInfo, ForExtraction forExtraction) { super(requiredInfo, forExtraction); ... } @Override public void fetch() { // Actually fetch the page data here } @Override public String someDataFiled() throws ExtractionException { //The exception needs to be thrown if someting failed // get piece of information and return it } ... // More datafields }","title":"Typical Implementation of a Single Data Extractor"},{"location":"01_Concept_of_the_extractor/#collectorextractor-pattern-for-lists","text":"Information can be represented as a list. In NewPipe, a list is represented by a InfoItemsCollector . A InfoItemsCollector will collect and assemble a list of InfoItem . For each item that should be extracted, a new Extractor must be created, and given to the InfoItemsCollector via commit() . If you are implementing a list in your service you need to implement an InfoItemExtractor , that will be able to retreve data for one and only one InfoItem. This extractor will then be comitted to the InfoItemsCollector that can collect the type of InfoItems you want to generate. A common implementation would look like this: private SomeInfoItemCollector collectInfoItemsFromElement(Element e) { // See *Some* as something like Stream or Channel // e.g. StreamInfoItemsCollector, and ChannelInfoItemsCollector are provided by NP SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId()); for(final Element li : element.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return collector; }","title":"Collector/Extractor Pattern for Lists"},{"location":"01_Concept_of_the_extractor/#listextractor","text":"There is more to know about lists: When a streaming site shows a list of items, it usually offers some additional information about that list like its title, a thumbnail, and its creator. Such info can be called list header . When a website shows a long list of items it usually does not load the whole list, but only a part of it. In order to get more items you may have to click on a next page button, or scroll down. Both of these Problems are fixed by the ListExtractor which takes care about extracting additional metadata about the liast, and by chopping down lists into several pages, so called InfoItemsPage s. Each page has its own URL, and needs to be extracted separately. For extracting list header information a ListExtractor behaves like a regular extractor. For handling InfoItemsPages it adds methods such as: getInitialPage() which will return the first page of InfoItems. getNextPageUrl() If a second Page of InfoItems is available this will return the URL pointing to them. getPage() returns a ListExtractor.InfoItemsPage by its URL which was retrieved by the getNextPageUrl() method of the previous page. The reason why the first page is handled special is because many Websites such as YouTube will load the first page of items like a regular web page, but all the others as an AJAX request. An InfoItemsPage itself has two constructors which take these parameters: - The InfoitemsCollector of the list that the page should represent - A nextPageUrl which represents the url of the following page (may be null if not page follows). - Optionally errors which is a list of Exceptions that may have happened during extracton. Here is a simplified reference implementation of a list extractor that only extracts pages, but not metadata: class MyListExtractor extends ListExtractor { ... private Document document; ... public InfoItemsPage getPage(pageUrl) throws ExtractionException { SomeInfoItemCollector collector = new SomeInfoItemCollector(getServiceId()); document = myFunctionToGetThePageHTMLWhatever(pageUrl); //remember this part from the simple list extraction for(final Element li : document.children()) { collector.commit(new InfoItemExtractor() { @Override public String getName() throws ParsingException { ... } @Override public String getUrl() throws ParsingException { ... } ... } return new InfoItemsPage(collector, myFunctionToGetTheNextPageUrl(document)); } public InfoItemsPage getInitialPage() { //document here got initialzied by the fetch() function. return getPage(getTheCurrentPageUrl(document)); } ... }","title":"ListExtractor"},{"location":"02_Concept_of_LinkHandler/","text":"Concept of the LinkHandler The LinkHandler represent links to resources like videos, search requests, channels, etc. The idea is that a video can have multiple links pointing to it, but it has one unique ID that represents it, like this example: oHg5SJYRHA0 can be represented as: https://www.youtube.com/watch?v=oHg5SJYRHA0 (the default URL for YouTube) https://youtu.be/oHg5SJYRHA0 (the shortened link) https://m.youtube.com/watch?v=oHg5SJYRHA0 (the link for mobile devices) Importand notes about LinkHandler: A simple LinkHandler will contain the default URL, the ID, and the original URL. LinkHandler s are read only. LinkHandler s are also used to determine which part of the extractor can handle a certain link. In order to get one you must either call fromUrl() or fromId() of the the corresponding LinkHandlerFactory . Every type of resource has its own LinkHandlerFactory . Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc. Usage The typical usage for obtaining a LinkHandler would look like this: LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory(); LinkHandler myVideo = myLinkHandlerFactory.fromUrl(\"https://my.service.com/the_video\"); Implementation In order to use LinkHandler for your service, you must override the appropriate LinkHandlerFactory. eg: class MyStreamLinkHandlerFactory extends LinkHandlerFactory { @Override public String getId(String url) throws ParsingException { // Return the ID based on the URL. } @Override public String getUrl(String id) throws ParsingException { // Return the URL based on the ID given. } @Override public boolean onAcceptUrl(String url) throws ParsingException { // Return true if this LinkHanlderFactory can handle this type of link } } ListLinkHandler and SearchQueryHandler List based resources, like channels and playlists, can be sorted and filtered. Therefore these type of resources don't just use a LinkHandler, but a class called ListLinkHandler , which inherits from LinkHandler and adds the field ContentFilter , which is used to filter by resource type, like stream or playlist, and SortFilter , which is used to sort by name, date, or view count. !!ATTENTION!! Be careful when you implement a content filter: No selected filter equals all filters selected. If your get an empty content filter list in your extractor, make sure you return everything. By all means, use \"if\" statements like contentFilter.contains(\"video\") || contentFilter.isEmpty() . ListLinkHandler are also created by overriding the ListLinkHandlerFactory additionally to the abstract methods this factory inherits from the LinkHandlerFactory you can override getAvailableContentFilter() and getAvailableSortFilter() . Through these you can tell the front end which kind of filter your service supports. SearchQueryHandler You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the same search request might have a different outcome depending on the country or the time you send the request. This is why the idea of an \"ID\" is replaced by a \"SearchString\" in the SearchQueryHandler These work like regular ListLinkHandler, except that you don't have to implement the methods onAcceptUrl() and getId() when overriding SearchQueryHandlerFactory .","title":"Concept of the LinkHandler"},{"location":"02_Concept_of_LinkHandler/#concept-of-the-linkhandler","text":"The LinkHandler represent links to resources like videos, search requests, channels, etc. The idea is that a video can have multiple links pointing to it, but it has one unique ID that represents it, like this example: oHg5SJYRHA0 can be represented as: https://www.youtube.com/watch?v=oHg5SJYRHA0 (the default URL for YouTube) https://youtu.be/oHg5SJYRHA0 (the shortened link) https://m.youtube.com/watch?v=oHg5SJYRHA0 (the link for mobile devices)","title":"Concept of the LinkHandler"},{"location":"02_Concept_of_LinkHandler/#importand-notes-about-linkhandler","text":"A simple LinkHandler will contain the default URL, the ID, and the original URL. LinkHandler s are read only. LinkHandler s are also used to determine which part of the extractor can handle a certain link. In order to get one you must either call fromUrl() or fromId() of the the corresponding LinkHandlerFactory . Every type of resource has its own LinkHandlerFactory . Eg. YoutubeStreamLinkHandler, YoutubeChannelLinkHandler, etc.","title":"Importand notes about LinkHandler:"},{"location":"02_Concept_of_LinkHandler/#usage","text":"The typical usage for obtaining a LinkHandler would look like this: LinkHandlerFactory myLinkHandlerFactory = new MyStreamLinkHandlerFactory(); LinkHandler myVideo = myLinkHandlerFactory.fromUrl(\"https://my.service.com/the_video\");","title":"Usage"},{"location":"02_Concept_of_LinkHandler/#implementation","text":"In order to use LinkHandler for your service, you must override the appropriate LinkHandlerFactory. eg: class MyStreamLinkHandlerFactory extends LinkHandlerFactory { @Override public String getId(String url) throws ParsingException { // Return the ID based on the URL. } @Override public String getUrl(String id) throws ParsingException { // Return the URL based on the ID given. } @Override public boolean onAcceptUrl(String url) throws ParsingException { // Return true if this LinkHanlderFactory can handle this type of link } }","title":"Implementation"},{"location":"02_Concept_of_LinkHandler/#listlinkhandler-and-searchqueryhandler","text":"List based resources, like channels and playlists, can be sorted and filtered. Therefore these type of resources don't just use a LinkHandler, but a class called ListLinkHandler , which inherits from LinkHandler and adds the field ContentFilter , which is used to filter by resource type, like stream or playlist, and SortFilter , which is used to sort by name, date, or view count. !!ATTENTION!! Be careful when you implement a content filter: No selected filter equals all filters selected. If your get an empty content filter list in your extractor, make sure you return everything. By all means, use \"if\" statements like contentFilter.contains(\"video\") || contentFilter.isEmpty() . ListLinkHandler are also created by overriding the ListLinkHandlerFactory additionally to the abstract methods this factory inherits from the LinkHandlerFactory you can override getAvailableContentFilter() and getAvailableSortFilter() . Through these you can tell the front end which kind of filter your service supports.","title":"ListLinkHandler and SearchQueryHandler"},{"location":"02_Concept_of_LinkHandler/#searchqueryhandler","text":"You cannot point to a search request with an ID like you point to a playlist or a channel, simply because one and the same search request might have a different outcome depending on the country or the time you send the request. This is why the idea of an \"ID\" is replaced by a \"SearchString\" in the SearchQueryHandler These work like regular ListLinkHandler, except that you don't have to implement the methods onAcceptUrl() and getId() when overriding SearchQueryHandlerFactory .","title":"SearchQueryHandler"},{"location":"03_Implement_a_service/","text":"Implementing a Service Services, or better service connectors, are the parts of NewPipe which communicate with an actual service like YouTube. This page will describe how you can implement and add your own services to the extractor. Please make sure you read and understand the Concept of Extractors and the Concept of LinkHandler before continuing. Required and Optional Parts Your service does not have to implement everything; some parts are optional. This is because not all services support every feature other services support. For example, it might be that a certain service does not support channels. If so, you can leave out the implementation of channels, and make the corresponding factory method of the your StreamingService implementation return null . The frontend will handle the lack of having channels. However, if you start to implement one of the optional parts of the list below, you will have to implement all of its parts/classes. NewPipe will crash if you only implement the extractor for the list item of a channel, but not the channel extractor itself. The Parts of a Service: Head of Service Stream Search Playlist (optional) Channel (optional) Kiosk (optional) Allowed Libraries The NewPipe Extractor already includes a lot of usable tools and external libraries that should make extracting easy. For some specific (tiny) tasks, Regex is allowed. Here you can take a look at the Parser , which will give you a little help with that. Use Regex with care!!! Avoid it as often as possible. It's better to ask us to introduce a new library than start using Regex to often. Html/XML Parsing: jsoup JSON Parsing: nanojson JavaScript Parsing/Execution: Mozilla Rhino Link detection in strings: AutoLink If you need to introduce new libraries, please tell us before you do so. Head of Service First of all, if you want to create a new service, you should create a new package below org.schabi.newpipe.services , with the name of your service as package name. Parts Required to be Implemented: StreamingService ServiceInfo StreamingService is a factory class that will return objects of all important parts of your service. Every extractor, handler, and info type you add and should be part of your implementation, must be instantiated using an instance of this class. You can see it as a factory for all objects of your implementation. ServiceInfo will return some metadata about your service such as the name, capabilities, the author's name, and their email address for further notice and maintenance issues. Remember, after extending this class, you need to return an instance of it by through your implementation of StreamingService.getServiceInfo() . When these two classes are extended by you, you need to add your StreamingService to the ServiceList of NewPipe. This way, your service will become an official part of the NewPipe Extractor. Every service has an ID, which will be set when this list gets created. You need to set this ID by entering it in the constructor. So when adding your service just give it the ID of the previously last service in the list incremented by one. Stream Streams are considered single entities of video or audio. They have metadata like a title, a description, next/related videos, a thumbnail and comments. To obtain the URL to the actual stream data, as well as its metadata, StreamExtractor is used. The LinkHandlerFactory will represent a link to such a stream. StreamInfoItemExtractor will extract one item in a list of items representing such streams, like a search result or a playlist. Since every streaming service (obviously) provides streams, this is required to implement. Otherwise, your service was pretty useless :) Parts Required to be Implemented: StreamExtractor StreamInfoItemExtractor LinkHandlerFactory Search The SearchExtractor is also required to be implemented. It will take a search query represented as SearchQueryHandler and return a list of search results. Since many services support suggestions as you type, you will also want to implement a SuggestionExtractor . This will make it possible for the frontend to also display a suggestion while typing. Parts Required to be Implemented: SearchExtractor SearchQueryHandlerFactory SuggestionExtractor (optional) Playlist Playlists are lists of streams provided by the service (you might not have to be concerned over locally saved playlists, those will be handled by the frontend). A playlist may only contain StreamInfoItems , but no other InfoItem types. Parts Required to be Implemented: PlaylistExtractor PlayListInfoItemExtractor ListLinkHandlerFactory Channel A Channel is mostly a Playlist , the only difference is that it does not only represent a simple list of streams, but also a user, a channel, or any entity that could be represented as a user. This is why the metadata supported by the ChannelExtractor differs from the one of a playlist. Parts Required to be Implemented: ChannelExtractor ChannelInfoItemExtractor ListLinkHandlerFactory Kiosk A kiosk is a list of InfoItems which will be displayed on the main page of NewPipe. A kiosk is mostly similar to the content displayed on the main page of a video platform. A kiosk could be something like \"Top 20\", \"Charts\", \"News\", \"Creators Selection\" etc. Kiosks are controversial; many people may not like them. If you also don't like them, please consider your users and refrain from denying support for them. Your service would look pretty empty if you select it and no video is being displayed. Also, you should not override the preference of the user, since users of NewPipe can decide by the settings whether they want to see the kiosk page or not. Multiple Kiosks Most services will implement more than one kiosk, so a service might have a \"Top 20\" for different categories like \"Country Music\", \"Techno\", etc. This is why the extractor will let you implement multiple KioskExtractors . Since different kiosk pages might also differ with their HTML structure, every page you want to support has to be implemented as its own KioskExtractor . However, if the pages are similar, you can use the same implementation, but set the page type when you instantiate your KioskExtractor through the KioskList.KioskExtractorFactory . Every kiosk you implement needs to be added to your KioskList which you return with your StreamingService implementation. It is also important to set the default kiosk. This will be the kiosk that will be shown by the first start of your service. An example implementation of the getKioskList() could look like this: @Override public KioskList getKioskList() throws ExtractionException { KioskList list = new KioskList(getServiceId()); list.addKioskEntry(new KioskList.KioskExtractorFactory() { @Override public KioskExtractor createNewKiosk(StreamingService streamingService, String url, String id, Localization local) throws ExtractionException { return new YoutubeTrendingExtractor(YoutubeService.this, new YoutubeTrendingLinkHandlerFactory().fromUrl(url), id, local); } }, new YoutubeTrendingLinkHandlerFactory(), \"Trending\"); list.setDefaultKiosk(\"Trending\"); return list; } Parts Required to be Implemented: KioskList.KioskExtractorFactory KioskExtractor ListLinkHandlerFactory","title":"Implementing a Service"},{"location":"03_Implement_a_service/#implementing-a-service","text":"Services, or better service connectors, are the parts of NewPipe which communicate with an actual service like YouTube. This page will describe how you can implement and add your own services to the extractor. Please make sure you read and understand the Concept of Extractors and the Concept of LinkHandler before continuing.","title":"Implementing a Service"},{"location":"03_Implement_a_service/#required-and-optional-parts","text":"Your service does not have to implement everything; some parts are optional. This is because not all services support every feature other services support. For example, it might be that a certain service does not support channels. If so, you can leave out the implementation of channels, and make the corresponding factory method of the your StreamingService implementation return null . The frontend will handle the lack of having channels. However, if you start to implement one of the optional parts of the list below, you will have to implement all of its parts/classes. NewPipe will crash if you only implement the extractor for the list item of a channel, but not the channel extractor itself. The Parts of a Service: Head of Service Stream Search Playlist (optional) Channel (optional) Kiosk (optional)","title":"Required and Optional Parts"},{"location":"03_Implement_a_service/#allowed-libraries","text":"The NewPipe Extractor already includes a lot of usable tools and external libraries that should make extracting easy. For some specific (tiny) tasks, Regex is allowed. Here you can take a look at the Parser , which will give you a little help with that. Use Regex with care!!! Avoid it as often as possible. It's better to ask us to introduce a new library than start using Regex to often. Html/XML Parsing: jsoup JSON Parsing: nanojson JavaScript Parsing/Execution: Mozilla Rhino Link detection in strings: AutoLink If you need to introduce new libraries, please tell us before you do so.","title":"Allowed Libraries"},{"location":"03_Implement_a_service/#head-of-service","text":"First of all, if you want to create a new service, you should create a new package below org.schabi.newpipe.services , with the name of your service as package name. Parts Required to be Implemented: StreamingService ServiceInfo StreamingService is a factory class that will return objects of all important parts of your service. Every extractor, handler, and info type you add and should be part of your implementation, must be instantiated using an instance of this class. You can see it as a factory for all objects of your implementation. ServiceInfo will return some metadata about your service such as the name, capabilities, the author's name, and their email address for further notice and maintenance issues. Remember, after extending this class, you need to return an instance of it by through your implementation of StreamingService.getServiceInfo() . When these two classes are extended by you, you need to add your StreamingService to the ServiceList of NewPipe. This way, your service will become an official part of the NewPipe Extractor. Every service has an ID, which will be set when this list gets created. You need to set this ID by entering it in the constructor. So when adding your service just give it the ID of the previously last service in the list incremented by one.","title":"Head of Service"},{"location":"03_Implement_a_service/#stream","text":"Streams are considered single entities of video or audio. They have metadata like a title, a description, next/related videos, a thumbnail and comments. To obtain the URL to the actual stream data, as well as its metadata, StreamExtractor is used. The LinkHandlerFactory will represent a link to such a stream. StreamInfoItemExtractor will extract one item in a list of items representing such streams, like a search result or a playlist. Since every streaming service (obviously) provides streams, this is required to implement. Otherwise, your service was pretty useless :) Parts Required to be Implemented: StreamExtractor StreamInfoItemExtractor LinkHandlerFactory","title":"Stream"},{"location":"03_Implement_a_service/#search","text":"The SearchExtractor is also required to be implemented. It will take a search query represented as SearchQueryHandler and return a list of search results. Since many services support suggestions as you type, you will also want to implement a SuggestionExtractor . This will make it possible for the frontend to also display a suggestion while typing. Parts Required to be Implemented: SearchExtractor SearchQueryHandlerFactory SuggestionExtractor (optional)","title":"Search"},{"location":"03_Implement_a_service/#playlist","text":"Playlists are lists of streams provided by the service (you might not have to be concerned over locally saved playlists, those will be handled by the frontend). A playlist may only contain StreamInfoItems , but no other InfoItem types. Parts Required to be Implemented: PlaylistExtractor PlayListInfoItemExtractor ListLinkHandlerFactory","title":"Playlist"},{"location":"03_Implement_a_service/#channel","text":"A Channel is mostly a Playlist , the only difference is that it does not only represent a simple list of streams, but also a user, a channel, or any entity that could be represented as a user. This is why the metadata supported by the ChannelExtractor differs from the one of a playlist. Parts Required to be Implemented: ChannelExtractor ChannelInfoItemExtractor ListLinkHandlerFactory","title":"Channel"},{"location":"03_Implement_a_service/#kiosk","text":"A kiosk is a list of InfoItems which will be displayed on the main page of NewPipe. A kiosk is mostly similar to the content displayed on the main page of a video platform. A kiosk could be something like \"Top 20\", \"Charts\", \"News\", \"Creators Selection\" etc. Kiosks are controversial; many people may not like them. If you also don't like them, please consider your users and refrain from denying support for them. Your service would look pretty empty if you select it and no video is being displayed. Also, you should not override the preference of the user, since users of NewPipe can decide by the settings whether they want to see the kiosk page or not.","title":"Kiosk"},{"location":"03_Implement_a_service/#multiple-kiosks","text":"Most services will implement more than one kiosk, so a service might have a \"Top 20\" for different categories like \"Country Music\", \"Techno\", etc. This is why the extractor will let you implement multiple KioskExtractors . Since different kiosk pages might also differ with their HTML structure, every page you want to support has to be implemented as its own KioskExtractor . However, if the pages are similar, you can use the same implementation, but set the page type when you instantiate your KioskExtractor through the KioskList.KioskExtractorFactory . Every kiosk you implement needs to be added to your KioskList which you return with your StreamingService implementation. It is also important to set the default kiosk. This will be the kiosk that will be shown by the first start of your service. An example implementation of the getKioskList() could look like this: @Override public KioskList getKioskList() throws ExtractionException { KioskList list = new KioskList(getServiceId()); list.addKioskEntry(new KioskList.KioskExtractorFactory() { @Override public KioskExtractor createNewKiosk(StreamingService streamingService, String url, String id, Localization local) throws ExtractionException { return new YoutubeTrendingExtractor(YoutubeService.this, new YoutubeTrendingLinkHandlerFactory().fromUrl(url), id, local); } }, new YoutubeTrendingLinkHandlerFactory(), \"Trending\"); list.setDefaultKiosk(\"Trending\"); return list; } Parts Required to be Implemented: KioskList.KioskExtractorFactory KioskExtractor ListLinkHandlerFactory","title":"Multiple Kiosks"},{"location":"04_Run_changes_in_App/","text":"Testing Your Changes in the App You should develop and test your changes with the JUnit environment that is provided by the NewPipe Extractor and IDEA. If you want to try it with the actual fronted, you need to follow these steps. Setup Android Studio First, you'll want to set up a working Android Studio environment. To do this, download Studio from developer.android.com , and follow the instructions on how to set it up. Get the NewPipe Code and Run it. In order to get it, you simply clone or download it from the current dev branch github.com/TeamNewPipe/NewPipe.git . You can then build and run it following these instructions . Also, make sure you are comfortable with adb since you might experience some trouble running your compiled app on a real device, especially under Linux, where you sometimes have to adjust the udev rules in order to make your device accessible . Run Your Changes on the Extractor In order to use the extractor in our app, we use jitpack . This is a build service that can build maven *.jar packages for Android and Java based on GitHub or GitLab repositories. To use the extractor through jitpack, you need to push it to your online repository of your copy that you host either on GitHub or GitLab . It's important to host it on one of both. To copy your repository URL in HTTP format, go to jitpack and paste it there. From here, you can grab the latest commit via GET IT button. I recomend not to use a SNAPSHOT, since I am not sure when snapshot is built. An \"implementation\" string will be generated for you. Copy this string and replace the implementation 'com.github.TeamNewPipe:NewPipeExtractor:' line in the file /app/build.gradle with it. Your browser does not support the video tag. If everything synced well, then you should only see a screen with OK signs. Now you can compile and run NewPipe with the new extractor. Troubleshooting If something went wrong on jitpack site, you can check their build log, by selecting the commit you tried to build and click on that little paper symbol next to the GET IT button. If it's red, it means that the build failed.","title":"Testing Your Changes in the App"},{"location":"04_Run_changes_in_App/#testing-your-changes-in-the-app","text":"You should develop and test your changes with the JUnit environment that is provided by the NewPipe Extractor and IDEA. If you want to try it with the actual fronted, you need to follow these steps.","title":"Testing Your Changes in the App"},{"location":"04_Run_changes_in_App/#setup-android-studio","text":"First, you'll want to set up a working Android Studio environment. To do this, download Studio from developer.android.com , and follow the instructions on how to set it up.","title":"Setup Android Studio"},{"location":"04_Run_changes_in_App/#get-the-newpipe-code-and-run-it","text":"In order to get it, you simply clone or download it from the current dev branch github.com/TeamNewPipe/NewPipe.git . You can then build and run it following these instructions . Also, make sure you are comfortable with adb since you might experience some trouble running your compiled app on a real device, especially under Linux, where you sometimes have to adjust the udev rules in order to make your device accessible .","title":"Get the NewPipe Code and Run it."},{"location":"04_Run_changes_in_App/#run-your-changes-on-the-extractor","text":"In order to use the extractor in our app, we use jitpack . This is a build service that can build maven *.jar packages for Android and Java based on GitHub or GitLab repositories. To use the extractor through jitpack, you need to push it to your online repository of your copy that you host either on GitHub or GitLab . It's important to host it on one of both. To copy your repository URL in HTTP format, go to jitpack and paste it there. From here, you can grab the latest commit via GET IT button. I recomend not to use a SNAPSHOT, since I am not sure when snapshot is built. An \"implementation\" string will be generated for you. Copy this string and replace the implementation 'com.github.TeamNewPipe:NewPipeExtractor:' line in the file /app/build.gradle with it. Your browser does not support the video tag. If everything synced well, then you should only see a screen with OK signs. Now you can compile and run NewPipe with the new extractor.","title":"Run Your Changes on the Extractor"},{"location":"04_Run_changes_in_App/#troubleshooting","text":"If something went wrong on jitpack site, you can check their build log, by selecting the commit you tried to build and click on that little paper symbol next to the GET IT button. If it's red, it means that the build failed.","title":"Troubleshooting"},{"location":"05_releasing/","text":"Releasing a New NewPipe Version This site is meant for those who want to maintain NewPipe, or just want to know how releasing works. Differences Between Regular and Hotfix Releases NewPipe is a web crawler. That means it does not use a web API, but instead tries to scrape the data from the website, this however has the disadvantage of the app to break instantly when YouTube changes something. We do not know when this happen. Therefore, maintainers need to act quickly when it happens, and reduce our downtime as much as possible. The entire release cycle is therefore designed around this issue. There is a difference between a release that introduces new features and a release that fixes an issue that occurred because YouTube, or some other service, changed their website (typically called a shutdown). Lets have a look at the characteristics of a regular release , and then the characteristics of a hotfix release . Regular Releases Regular releases are normal releases like they are done in any other app. Releases are always stored on master branch. The latest commit on master is always equal to the currently released version. No development is done on master. This ensures that we always have one branch with a stable/releasable version. Feature Branching When developing, the dev branch is used. Pushing to dev directly, however, is not allowed, since QA and testing should be done first before adding something to it. This ensures that the dev version works as stable a possible. In order to change something on the app, one may want to fork the dev branch and develop the changes in their own branch (this is called feature branching). Make sure that both the dev branches, as well as the master branches of the extractor and the frontend, are compatible with each other. If a change is done on the API to the extractor, make sure that frontend is compatible, or changed to become compatible, with these changes. If the PR that should make the frontend compatible again can not be merged, please do not merge the corresponding PR on the extractor either. This should make sure that any developer can run his changes on the fronted at any time. Merging Features/Bugfixes After finishing a feature, one should open up a Pull Reuqest to the dev branch. From here, a maintainer can do Code review and Quality Assurance (QA) . If you are a maintainer, please take care about the code architecture so corrosion or code shifting can be prevented. Please also prioritize code quality over functionality. In short: cool function but bad code = no merge. Focus on leaving the code as clean as possible. You, as a maintainer, should build the app and put the signed APK into the description of that new pull request. This way, other people can test the feature/bugfix and help with QA. You may not need to do this every time. It is enough to do it on bigger pull requests. After the maintainer merges the new feature into the dev branch, he should add the title of the pull request or a summary of the changes into the release notes . Creating a New Release Once there are enough features together, and the maintainers believe that NewPipe is ready for a new release, they should create a new release. Be aware of the rule that a release should never be done on a Friday. For NewPipe, this means: Don't do a release if you don't have time for it!!! Below is a list of things you will want to do: Fork the dev branch into a new release_x.y.z branch. Increase the version number Merge weblate changes from the dev branch at https://hosted.weblate.org/git/newpipe/strings/ . Copy the release notes from the GitHub version draft into the corresponding fastlane file (see release notes ). Open up a pull request form the new release_x.y.z branch into the master branch. Create an issue pointing to the new pull request. The reason for opening an issue is that from my perception, people read issues more than pull requests. Put the release-note into this pull request. Build a signed release version of NewPipe using schabis signing keys. This is a release candidate (RC). Name the build apk file NewPipe__RC1.apk . Zip it and post it to the head of the release issue. This way, others can test the release candidate. Test and QA the new version with the help of others. Leave the PR open for a few days and advertise it to help testing. While being in release phase no new pull requests must be merged into dev branch. This procedure does not have to be done for the extractor as extractor will be tested together with the fronted. Quickfixes When issuing a new release, you will most likely encounter bugs that might not have existed in previous versions. These are called regressions . If you find a regression during release phase, you are allowed to push fixes directly into the release branch without having to fork a branch away from it. All maintainers have to be aware that they might be required to fix regressions, so plan your release at a time when you are available. Do not introduce new features during the release phase. When you have pushed a quickfix, you will want to update the release candidate you put into the issue corresponding to the release pull request . Increment the version number in the filename of the release candidate. e.g. NewPipe__RC2.apk etc. Don't update the actual version number. :P Releasing Once the glorious day of all days has come, and you fulfill the ceremony of releasing. After going through the release procedure of creating a new release and maybe a few quickfixes on the new release, this is what you should do when releasing: Click \"Merge Pull Request\". Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Make sure the draft name equals the tag name. Make sure to not have forgotten anything. Click \"Publish Release\". Rebase quickfix changes back into dev if quickfixes were made. Hotfix Releases As aforementioned, NewPipe is a web crawler and could break at any moment. In order to keep the downtime of NewPipe as low as possible, when such a shutdown happens, we allow hotfixes . A hotfix allows work on the master branch instead of the dev branch. A hotfix MUST NOT contain any features or unrelated bugfixes. A hotfix may only focus on fixing what caused the shutdown. Hotfix Branch Hotfixes work on the master branch. The dev branch has experimental changes that might have not been tested properly enough to be released, if at all. The master branch should always be the latest stable version of NewPipe. If the master branch breaks due to a shutdown, you should fix the master branch. Of course you are not allowed to push to master directly so you will have to open up a hotfix branch. If someone else is pushing a hotfix into master, and it works this can be considered as hotfix branch as well. Releasing If you fixed the issue and found it to be tested and reviewed well enough, you may release it. You don't need to undergo the full release procedure of a regular release, which takes more time to release. Keep in mind that if the hotfix might turn out to be broken after release, you should release another hotfix. It is important to release quickly for the sake of keeping NewPipe alive, and after all, a slightly broken version of NewPipe is better then a non-functional version \u00af\\_(\u30c4)_/\u00af. Here's what you do when releasing a hotfix: Click \"Merge Pull Request\" Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Create a new release draft and write the down the fix into the release notes. Copy the release note into the fastlane directory of releases. Increment the small minor version number and the versionCode . Click \"Publish Release\". Rebase the hotfix back into dev branch. Version Nomenclature The version nomenclature of NewPipe is simple. Major : The major version number (the number before the first dot) was 0 for years. The reason for this changed over time. First, I wanted this number to switch to 1 once NewPipe was feature complete. Now, I rather think of incrementing this number to 1 once we can ensure that NewPipe runs stable (part of which this documentation should help). After this, well, God knows what happens if we ever reach 1. \u00af\\_(\u30c4)_/\u00af Minor : The minor version number (the number after the first dot) will be incremented if there is a major feature added to the app. Small Minor : The small minor (the number after the second dot) will be incremented if there are bug fixes or minor features added to the app. Version Nomenclature of the Extractor The extractor is always released together with the app, therefore the version number of the extractor is identical to the one of NewPipe itself. Version Code In Android, an app can also have a versionCode . This code is a long integer and can be incremented by any value to show a device that a new version is there. For NewPipe, the version code will be incremented by 10 regardless of the change of the major or minor version number. The version codes between the 10 steps are reserved for our internal F-Droid build server. Release Notes Release notes should tell what was changed in the new version of the app. The release nodes for NewPipe are stored in the GitHub draft for a new release . When a maintainer wants to add changes to the release note, but there is no draft for a new version, they should create one. Changes can be categorized into three types: New : New features that god added to the app. Improved : Improvements to the app or existing features Fixes : Bugfixes When releasing a new version of NewPipe, before actually clicking \"Release\", the maintainer should copy the release notes from the draft and put it into a file called .txt (whereas needs to be the version code of the incoming release). This file must be stored in the directory /fastlane/metadata/android/en-US/changelogs . This way, F-Droid will be able to show the changes done to the app.","title":"Releasing a New NewPipe Version"},{"location":"05_releasing/#releasing-a-new-newpipe-version","text":"This site is meant for those who want to maintain NewPipe, or just want to know how releasing works.","title":"Releasing a New NewPipe Version"},{"location":"05_releasing/#differences-between-regular-and-hotfix-releases","text":"NewPipe is a web crawler. That means it does not use a web API, but instead tries to scrape the data from the website, this however has the disadvantage of the app to break instantly when YouTube changes something. We do not know when this happen. Therefore, maintainers need to act quickly when it happens, and reduce our downtime as much as possible. The entire release cycle is therefore designed around this issue. There is a difference between a release that introduces new features and a release that fixes an issue that occurred because YouTube, or some other service, changed their website (typically called a shutdown). Lets have a look at the characteristics of a regular release , and then the characteristics of a hotfix release .","title":"Differences Between Regular and Hotfix Releases"},{"location":"05_releasing/#regular-releases","text":"Regular releases are normal releases like they are done in any other app. Releases are always stored on master branch. The latest commit on master is always equal to the currently released version. No development is done on master. This ensures that we always have one branch with a stable/releasable version.","title":"Regular Releases"},{"location":"05_releasing/#feature-branching","text":"When developing, the dev branch is used. Pushing to dev directly, however, is not allowed, since QA and testing should be done first before adding something to it. This ensures that the dev version works as stable a possible. In order to change something on the app, one may want to fork the dev branch and develop the changes in their own branch (this is called feature branching). Make sure that both the dev branches, as well as the master branches of the extractor and the frontend, are compatible with each other. If a change is done on the API to the extractor, make sure that frontend is compatible, or changed to become compatible, with these changes. If the PR that should make the frontend compatible again can not be merged, please do not merge the corresponding PR on the extractor either. This should make sure that any developer can run his changes on the fronted at any time.","title":"Feature Branching"},{"location":"05_releasing/#merging-featuresbugfixes","text":"After finishing a feature, one should open up a Pull Reuqest to the dev branch. From here, a maintainer can do Code review and Quality Assurance (QA) . If you are a maintainer, please take care about the code architecture so corrosion or code shifting can be prevented. Please also prioritize code quality over functionality. In short: cool function but bad code = no merge. Focus on leaving the code as clean as possible. You, as a maintainer, should build the app and put the signed APK into the description of that new pull request. This way, other people can test the feature/bugfix and help with QA. You may not need to do this every time. It is enough to do it on bigger pull requests. After the maintainer merges the new feature into the dev branch, he should add the title of the pull request or a summary of the changes into the release notes .","title":"Merging Features/Bugfixes"},{"location":"05_releasing/#creating-a-new-release","text":"Once there are enough features together, and the maintainers believe that NewPipe is ready for a new release, they should create a new release. Be aware of the rule that a release should never be done on a Friday. For NewPipe, this means: Don't do a release if you don't have time for it!!! Below is a list of things you will want to do: Fork the dev branch into a new release_x.y.z branch. Increase the version number Merge weblate changes from the dev branch at https://hosted.weblate.org/git/newpipe/strings/ . Copy the release notes from the GitHub version draft into the corresponding fastlane file (see release notes ). Open up a pull request form the new release_x.y.z branch into the master branch. Create an issue pointing to the new pull request. The reason for opening an issue is that from my perception, people read issues more than pull requests. Put the release-note into this pull request. Build a signed release version of NewPipe using schabis signing keys. This is a release candidate (RC). Name the build apk file NewPipe__RC1.apk . Zip it and post it to the head of the release issue. This way, others can test the release candidate. Test and QA the new version with the help of others. Leave the PR open for a few days and advertise it to help testing. While being in release phase no new pull requests must be merged into dev branch. This procedure does not have to be done for the extractor as extractor will be tested together with the fronted.","title":"Creating a New Release"},{"location":"05_releasing/#quickfixes","text":"When issuing a new release, you will most likely encounter bugs that might not have existed in previous versions. These are called regressions . If you find a regression during release phase, you are allowed to push fixes directly into the release branch without having to fork a branch away from it. All maintainers have to be aware that they might be required to fix regressions, so plan your release at a time when you are available. Do not introduce new features during the release phase. When you have pushed a quickfix, you will want to update the release candidate you put into the issue corresponding to the release pull request . Increment the version number in the filename of the release candidate. e.g. NewPipe__RC2.apk etc. Don't update the actual version number. :P","title":"Quickfixes"},{"location":"05_releasing/#releasing","text":"Once the glorious day of all days has come, and you fulfill the ceremony of releasing. After going through the release procedure of creating a new release and maybe a few quickfixes on the new release, this is what you should do when releasing: Click \"Merge Pull Request\". Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Make sure the draft name equals the tag name. Make sure to not have forgotten anything. Click \"Publish Release\". Rebase quickfix changes back into dev if quickfixes were made.","title":"Releasing"},{"location":"05_releasing/#hotfix-releases","text":"As aforementioned, NewPipe is a web crawler and could break at any moment. In order to keep the downtime of NewPipe as low as possible, when such a shutdown happens, we allow hotfixes . A hotfix allows work on the master branch instead of the dev branch. A hotfix MUST NOT contain any features or unrelated bugfixes. A hotfix may only focus on fixing what caused the shutdown.","title":"Hotfix Releases"},{"location":"05_releasing/#hotfix-branch","text":"Hotfixes work on the master branch. The dev branch has experimental changes that might have not been tested properly enough to be released, if at all. The master branch should always be the latest stable version of NewPipe. If the master branch breaks due to a shutdown, you should fix the master branch. Of course you are not allowed to push to master directly so you will have to open up a hotfix branch. If someone else is pushing a hotfix into master, and it works this can be considered as hotfix branch as well.","title":"Hotfix Branch"},{"location":"05_releasing/#releasing_1","text":"If you fixed the issue and found it to be tested and reviewed well enough, you may release it. You don't need to undergo the full release procedure of a regular release, which takes more time to release. Keep in mind that if the hotfix might turn out to be broken after release, you should release another hotfix. It is important to release quickly for the sake of keeping NewPipe alive, and after all, a slightly broken version of NewPipe is better then a non-functional version \u00af\\_(\u30c4)_/\u00af. Here's what you do when releasing a hotfix: Click \"Merge Pull Request\" Create a GPG signed tag with the name v0.x.y . Merge dev into master on the extractor. Create a GPG signed tag with the name v0.x.y on the extractor. Create a new release draft and write the down the fix into the release notes. Copy the release note into the fastlane directory of releases. Increment the small minor version number and the versionCode . Click \"Publish Release\". Rebase the hotfix back into dev branch.","title":"Releasing"},{"location":"05_releasing/#version-nomenclature","text":"The version nomenclature of NewPipe is simple. Major : The major version number (the number before the first dot) was 0 for years. The reason for this changed over time. First, I wanted this number to switch to 1 once NewPipe was feature complete. Now, I rather think of incrementing this number to 1 once we can ensure that NewPipe runs stable (part of which this documentation should help). After this, well, God knows what happens if we ever reach 1. \u00af\\_(\u30c4)_/\u00af Minor : The minor version number (the number after the first dot) will be incremented if there is a major feature added to the app. Small Minor : The small minor (the number after the second dot) will be incremented if there are bug fixes or minor features added to the app.","title":"Version Nomenclature"},{"location":"05_releasing/#version-nomenclature-of-the-extractor","text":"The extractor is always released together with the app, therefore the version number of the extractor is identical to the one of NewPipe itself.","title":"Version Nomenclature of the Extractor"},{"location":"05_releasing/#version-code","text":"In Android, an app can also have a versionCode . This code is a long integer and can be incremented by any value to show a device that a new version is there. For NewPipe, the version code will be incremented by 10 regardless of the change of the major or minor version number. The version codes between the 10 steps are reserved for our internal F-Droid build server.","title":"Version Code"},{"location":"05_releasing/#release-notes","text":"Release notes should tell what was changed in the new version of the app. The release nodes for NewPipe are stored in the GitHub draft for a new release . When a maintainer wants to add changes to the release note, but there is no draft for a new version, they should create one. Changes can be categorized into three types: New : New features that god added to the app. Improved : Improvements to the app or existing features Fixes : Bugfixes When releasing a new version of NewPipe, before actually clicking \"Release\", the maintainer should copy the release notes from the draft and put it into a file called .txt (whereas needs to be the version code of the incoming release). This file must be stored in the directory /fastlane/metadata/android/en-US/changelogs . This way, F-Droid will be able to show the changes done to the app.","title":"Release Notes"},{"location":"06_documentation/","text":"About This Documentation The documentation you are currently reading was written using mkdocs . It is a tool that will generate a static website based on markdown files. Markdown has the advantage that it is simple to read and write, and that there are several tools that can translate a markdown file into languages like HTML or LaTeX. Installation Mkdocs is written in Python and is distributed through the Python internal package manager pip , thus you need to get python and pip running on your operating system first. Windows Download the latest Python3 version. When running the setup program, make sure to tick, \"Add Python 3.x to PATH\". Install Python. Open PowerShell or cmd.exe and type: pip3 install mkdocs . MacOS MacOS already includes Python, however, pip is still missing. The easiest and most nondestructive way is to install the MacOS package manager, homebrew , first. The advantage of homebrew is that it will only modify your home directory, and not the root dir, so your OS will not be tampered with. Install homebrew . Install Python from homebrew, which will also install pip. Enter this command: brew install python . Install mkdocs: pip3 install mkdocs Linux/*BSD Linux/*BSD also has Python preinstalled. Most distributions also contain pip by default. If it is not installed, you may need to figure out how to install pip3 through the package manager of your system. Install pip3 with these commands according to distributions: Ubuntu/Mint : apt install python3-pip Fedora/CentOS : sudo dnf install python3-pip Arch/Manjaro : sudo pacman -S python-pip openSuse : sudo zypper install python-pip *BSD : You are already advanced enough to know how you can force the bits on your disk to become pip by meditating upon it. Run pip3 install mkdocs to install mkdocs only for the current user, or run sudo pip3 install mkdocs to install mkdocs systemwide. Last one has the higher chance to work properly. Android/ChromeOS This might sound funny, but according to the growing amount of Chromebooks and Android tablets with keyboards, this might actually be useful. Install the Termux App from f-droid . Launch Termux and type apt update Install Python and git with the command: apt install git python Install mkdocs with pip install mkdocs . From herein, everything will be the same as on Desktop. If you want to edit the files, you can (besides vim or emacs which are available through Termux) use your preferred text editor on Android. This is possible by opening the files with the Termux integration of the build in android file manager: Updating Sometimes, mkdocs changes the way of how it serves, or the syntax will differ. This is why you should make sure to always run the latest version of mkdocs. To check, simply run pip3 install --upgrade mkdocs or sudo pip3 install --upgrade mkdocs if you installed pip system wide on a Linux/BSD* system. Using mkdocs In order to extend this documentation, you have to clone it from its GitHub repository . When you clone it, you will find a mkdocs.yml file, and a docs directory inside. The yaml file is the config file while in the directory docs the documentation files are stored. Here is a guide about how to use mkdocs. Write and Deploy If you are writing a documentation page and want a live preview of it, you can enter the root directory of this documentation project, and then run mkdocs serve this will start the mkdocs internal web server on port 8000 . So all you have to do is type localhost:8000 into the address bar of your browser, and here you go. If you modify a file, and save it, mkdocs will reload the page and show you the new content. If you want to deploy the page so it will be up to date at the GitHub pages , simply type mkdocs gh-deploy . However, please be aware that this will not push your changes to the master branch of the repository. So, you still have to commit and push your changes to the actual git repository of this documentation. Please be aware that only privileged maintainers can do this.","title":"About This Documentation"},{"location":"06_documentation/#about-this-documentation","text":"The documentation you are currently reading was written using mkdocs . It is a tool that will generate a static website based on markdown files. Markdown has the advantage that it is simple to read and write, and that there are several tools that can translate a markdown file into languages like HTML or LaTeX.","title":"About This Documentation"},{"location":"06_documentation/#installation","text":"Mkdocs is written in Python and is distributed through the Python internal package manager pip , thus you need to get python and pip running on your operating system first.","title":"Installation"},{"location":"06_documentation/#windows","text":"Download the latest Python3 version. When running the setup program, make sure to tick, \"Add Python 3.x to PATH\". Install Python. Open PowerShell or cmd.exe and type: pip3 install mkdocs .","title":"Windows"},{"location":"06_documentation/#macos","text":"MacOS already includes Python, however, pip is still missing. The easiest and most nondestructive way is to install the MacOS package manager, homebrew , first. The advantage of homebrew is that it will only modify your home directory, and not the root dir, so your OS will not be tampered with. Install homebrew . Install Python from homebrew, which will also install pip. Enter this command: brew install python . Install mkdocs: pip3 install mkdocs","title":"MacOS"},{"location":"06_documentation/#linuxbsd","text":"Linux/*BSD also has Python preinstalled. Most distributions also contain pip by default. If it is not installed, you may need to figure out how to install pip3 through the package manager of your system. Install pip3 with these commands according to distributions: Ubuntu/Mint : apt install python3-pip Fedora/CentOS : sudo dnf install python3-pip Arch/Manjaro : sudo pacman -S python-pip openSuse : sudo zypper install python-pip *BSD : You are already advanced enough to know how you can force the bits on your disk to become pip by meditating upon it. Run pip3 install mkdocs to install mkdocs only for the current user, or run sudo pip3 install mkdocs to install mkdocs systemwide. Last one has the higher chance to work properly.","title":"Linux/*BSD"},{"location":"06_documentation/#androidchromeos","text":"This might sound funny, but according to the growing amount of Chromebooks and Android tablets with keyboards, this might actually be useful. Install the Termux App from f-droid . Launch Termux and type apt update Install Python and git with the command: apt install git python Install mkdocs with pip install mkdocs . From herein, everything will be the same as on Desktop. If you want to edit the files, you can (besides vim or emacs which are available through Termux) use your preferred text editor on Android. This is possible by opening the files with the Termux integration of the build in android file manager:","title":"Android/ChromeOS"},{"location":"06_documentation/#updating","text":"Sometimes, mkdocs changes the way of how it serves, or the syntax will differ. This is why you should make sure to always run the latest version of mkdocs. To check, simply run pip3 install --upgrade mkdocs or sudo pip3 install --upgrade mkdocs if you installed pip system wide on a Linux/BSD* system.","title":"Updating"},{"location":"06_documentation/#using-mkdocs","text":"In order to extend this documentation, you have to clone it from its GitHub repository . When you clone it, you will find a mkdocs.yml file, and a docs directory inside. The yaml file is the config file while in the directory docs the documentation files are stored. Here is a guide about how to use mkdocs.","title":"Using mkdocs"},{"location":"06_documentation/#write-and-deploy","text":"If you are writing a documentation page and want a live preview of it, you can enter the root directory of this documentation project, and then run mkdocs serve this will start the mkdocs internal web server on port 8000 . So all you have to do is type localhost:8000 into the address bar of your browser, and here you go. If you modify a file, and save it, mkdocs will reload the page and show you the new content. If you want to deploy the page so it will be up to date at the GitHub pages , simply type mkdocs gh-deploy . However, please be aware that this will not push your changes to the master branch of the repository. So, you still have to commit and push your changes to the actual git repository of this documentation. Please be aware that only privileged maintainers can do this.","title":"Write and Deploy"},{"location":"07_maintainers_view/","text":"Maintainers View So I want to document some of the views i have when maintaining NewPipe. Keep it Streamlined NewPipe is a Player for online videos on a smart phone, by means it is used for entertainment reason. This means it does not have to be some professional application, and it does not have to be complicated to be used. However NewPipe might not focus on the casual user completely as there are many features that are a bit more \"tecki\" and may require some knowledge about technology, however all in all NewPipe should be easy to use, even for not teck guys. NewPipe does not have to be a air plane cockpit: Don't add to much special features . If people want to do professionally things with Videos they might use professional tools. Design the UI so it does make sense to the user . Try to make it comply with material design guidelines . Don't add to much features : Think about the Betamax vs. VHS phenomena or the Unix principle of having one program designed for one specific task: If you add to much functionality you add complexity and this is not appealing to the user. Focus on what NewPipe should be, and make it be only that. Bugfixes ] Disclaimer: This is a meme maybe in real live it is different. Pleas no shit storm. Always go for Bugfixes , as the best application with the best features does not help much if it is broken, or annoying to use. Now if a program is in an early stage it is quite understandable that many things brake. This is one reason why NewPipe still has no 1 in the beginning of its version number. However by now NewPipe is in a stage where there should be a strong focus on stability. If there are multiple Pull requests open, check the ones with the bugfixes first. Do not add to much features every version, as every feature will inevitable introduce more bugs. It is quite ok, if PRs stay open for a while (not to long though). If there are bugs that are stale, or open for a while bump them from time to time, so devs know that there is still something left to fix. Never accept bugs. From my perception the community does not like to fix bugs, this is why you as a maintainer should especially focus on perusing bugs. Features Well features are also something that can cause a headache. You should always see adding features critical and question whether that features does make sense, is useful and would actually be an advantage for the app. You should not blindly say yes to features even if they are small, however you should also not directly say no as well. Think about it, may be even for days before deciding whether you want to accept a feature or not. If you are not sure, try it, look into the code, speak with the developer, and then make a decision and justify it. The criteria whether to add a feature or not should be: Is the features just requested by one or two people or was the feature requested by multiple people? Is the code of the feature written well? Is it a quick and hacky solution and could a proper solution be implemented later on? Does the amount of code justify the outcome? Maybe people will send a pull request that will add a frequently requested feature, but is implemented in a hacky way, than don't add it, as you might get into trouble with that solution later on. Either through problems of extending the feature, by introducing to much bugs or simply by braking the architecture or the philosophy of NewPipe. If so don't add it. PRs If a PR contains one or more features/bugs be curious. The more stuff a PR changes the longer it will take to be added. Also there might be things you are ok with, but then there are other parts that are not ok with and because of these you can't merge it. This is why you should insist to make the dev chop down the PR into multiple smaller PRs if it's possible. Community When you talk to the community stay friendly and respectful, and make sure a friendly and respectful tone will stay. When you have a bad day just don't go to GitHub (an advice from my experience ;D ).","title":"Maintainers View"},{"location":"07_maintainers_view/#maintainers-view","text":"So I want to document some of the views i have when maintaining NewPipe.","title":"Maintainers View"},{"location":"07_maintainers_view/#keep-it-streamlined","text":"NewPipe is a Player for online videos on a smart phone, by means it is used for entertainment reason. This means it does not have to be some professional application, and it does not have to be complicated to be used. However NewPipe might not focus on the casual user completely as there are many features that are a bit more \"tecki\" and may require some knowledge about technology, however all in all NewPipe should be easy to use, even for not teck guys. NewPipe does not have to be a air plane cockpit: Don't add to much special features . If people want to do professionally things with Videos they might use professional tools. Design the UI so it does make sense to the user . Try to make it comply with material design guidelines . Don't add to much features : Think about the Betamax vs. VHS phenomena or the Unix principle of having one program designed for one specific task: If you add to much functionality you add complexity and this is not appealing to the user. Focus on what NewPipe should be, and make it be only that.","title":"Keep it Streamlined"},{"location":"07_maintainers_view/#bugfixes","text":"] Disclaimer: This is a meme maybe in real live it is different. Pleas no shit storm. Always go for Bugfixes , as the best application with the best features does not help much if it is broken, or annoying to use. Now if a program is in an early stage it is quite understandable that many things brake. This is one reason why NewPipe still has no 1 in the beginning of its version number. However by now NewPipe is in a stage where there should be a strong focus on stability. If there are multiple Pull requests open, check the ones with the bugfixes first. Do not add to much features every version, as every feature will inevitable introduce more bugs. It is quite ok, if PRs stay open for a while (not to long though). If there are bugs that are stale, or open for a while bump them from time to time, so devs know that there is still something left to fix. Never accept bugs. From my perception the community does not like to fix bugs, this is why you as a maintainer should especially focus on perusing bugs.","title":"Bugfixes"},{"location":"07_maintainers_view/#features","text":"Well features are also something that can cause a headache. You should always see adding features critical and question whether that features does make sense, is useful and would actually be an advantage for the app. You should not blindly say yes to features even if they are small, however you should also not directly say no as well. Think about it, may be even for days before deciding whether you want to accept a feature or not. If you are not sure, try it, look into the code, speak with the developer, and then make a decision and justify it. The criteria whether to add a feature or not should be: Is the features just requested by one or two people or was the feature requested by multiple people? Is the code of the feature written well? Is it a quick and hacky solution and could a proper solution be implemented later on? Does the amount of code justify the outcome? Maybe people will send a pull request that will add a frequently requested feature, but is implemented in a hacky way, than don't add it, as you might get into trouble with that solution later on. Either through problems of extending the feature, by introducing to much bugs or simply by braking the architecture or the philosophy of NewPipe. If so don't add it.","title":"Features"},{"location":"07_maintainers_view/#prs","text":"If a PR contains one or more features/bugs be curious. The more stuff a PR changes the longer it will take to be added. Also there might be things you are ok with, but then there are other parts that are not ok with and because of these you can't merge it. This is why you should insist to make the dev chop down the PR into multiple smaller PRs if it's possible.","title":"PRs"},{"location":"07_maintainers_view/#community","text":"When you talk to the community stay friendly and respectful, and make sure a friendly and respectful tone will stay. When you have a bad day just don't go to GitHub (an advice from my experience ;D ).","title":"Community"}]} \ No newline at end of file diff --git a/search/worker.js b/search/worker.js old mode 100755 new mode 100644 diff --git a/sitemap.xml b/sitemap.xml old mode 100755 new mode 100644 index 76d0a7f..b74709b --- a/sitemap.xml +++ b/sitemap.xml @@ -2,47 +2,47 @@ None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily None - 2019-04-07 + 2019-07-02 daily \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz old mode 100755 new mode 100644 index 787860f..feeffd7 Binary files a/sitemap.xml.gz and b/sitemap.xml.gz differ