174db2c55SMasatake YAMATO.. NOT REVIEWED YET 274db2c55SMasatake YAMATO 3eb56edb2SHiroo HAYASHI.. _multiple_parsers: 4eb56edb2SHiroo HAYASHI 574db2c55SMasatake YAMATORunning multiple parsers on an input file 674db2c55SMasatake YAMATO--------------------------------------------------------------------- 774db2c55SMasatake YAMATO 8b45a42b3SHiroo HAYASHIUniversal Ctags provides parser developers two ways, *guest parser* (guest/host) 9b45a42b3SHiroo HAYASHIand *subparser* (sub/base), to run multiple parsers for an input file. 1074db2c55SMasatake YAMATO 11b45a42b3SHiroo HAYASHIThis section shows concepts behind the running multiple parsers and real 12b45a42b3SHiroo HAYASHIexamples. 1374db2c55SMasatake YAMATO 1499fa9565SMasatake YAMATO.. _host-guest-parsers: 1574db2c55SMasatake YAMATO 16b45a42b3SHiroo HAYASHIGuest parser: Applying a parser to specified areas of input file 1774db2c55SMasatake YAMATO~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1874db2c55SMasatake YAMATO 19b45a42b3SHiroo HAYASHI*Guest parser* (guest/host) considers the case that an input file has areas 2074db2c55SMasatake YAMATOwritten in languages different from the language for the input file. 2174db2c55SMasatake YAMATO 223cd8570eSHiroo HAYASHIA *host parser* parses the input file and detects the areas. 233cd8570eSHiroo HAYASHIThe host parser schedules *guest parsers* parsing the areas. 243cd8570eSHiroo HAYASHIThe guest parsers parse the areas. 2574db2c55SMasatake YAMATO 2674db2c55SMasatake YAMATO.. figure:: area-and-parsers.svg 2774db2c55SMasatake YAMATO :scale: 80% 2874db2c55SMasatake YAMATO 29b45a42b3SHiroo HAYASHI.. NOTE: We don't have --list-guest-parser option as --list-subparser for 30b45a42b3SHiroo HAYASHI a subparser, because a guest parser does not define a new parser and it just 31b45a42b3SHiroo HAYASHI uses parsers which are already defined. 32b45a42b3SHiroo HAYASHI 33b45a42b3SHiroo HAYASHICommand line interface 34b45a42b3SHiroo HAYASHI......................................................................... 35b45a42b3SHiroo HAYASHI 36b45a42b3SHiroo HAYASHIRunning guest parser can be controlled with guest (``g``) extras flag. 37b45a42b3SHiroo HAYASHIBy default it is disabled. To turning on the feature running 38b45a42b3SHiroo HAYASHIguest parser, specify ``--extras=+g``. 39b45a42b3SHiroo HAYASHI 40b45a42b3SHiroo HAYASHIIf ``--fields=+E`` is given, all tags generated by a guest parser is marked 41b45a42b3SHiroo HAYASHI``guest`` in their ``extras:`` fields. 42b45a42b3SHiroo HAYASHI 43b45a42b3SHiroo HAYASHIExamples of guest parser 4474db2c55SMasatake YAMATO...................................................................... 4574db2c55SMasatake YAMATO 4610454529SMasatake YAMATO{CSS,JavaScript}/HTML parser combination 4774db2c55SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 4874db2c55SMasatake YAMATO 493cd8570eSHiroo HAYASHIFor an HTML file, you may want to run HTML parser, of course. The 503cd8570eSHiroo HAYASHIHTML file may have CSS areas and JavaScript areas. In other hand 51dccba5efSHiroo HAYASHIUniversal Ctags has both CSS and JavaScript parsers. Don't you 5274db2c55SMasatake YAMATOthink it is useful if you can apply these parsers to the areas? 5374db2c55SMasatake YAMATO 5474db2c55SMasatake YAMATOIn this case, HTML has responsible to detect the CSS and 5574db2c55SMasatake YAMATOJavaScript areas and record the positions of the areas. 5674db2c55SMasatake YAMATOThe HTML parser schedules delayed invocations of CSS and 5774db2c55SMasatake YAMATOJavaScript parsers on the area with promise API. 5874db2c55SMasatake YAMATO 5986bcb5c2SHiroo HAYASHIHere HTML parser is a *host parser*. CSS and JavaScript parsers 6086bcb5c2SHiroo HAYASHIare *guest parsers*. 6174db2c55SMasatake YAMATO 62b45a42b3SHiroo HAYASHISee ":ref:`The new HTML parser <html>`" and "`parsers/html.c 63b45a42b3SHiroo HAYASHI<https://github.com/universal-ctags/ctags/blob/master/parsers/html.c>`_". 6474db2c55SMasatake YAMATO 6574db2c55SMasatake YAMATO 6610454529SMasatake YAMATOC/Yacc parser combination 6774db2c55SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 6874db2c55SMasatake YAMATO 69dccba5efSHiroo HAYASHIA yacc file has some areas written in C. Universal Ctags has both YACC 7099fa9565SMasatake YAMATOand C parsers. You may want to run C parser for the areas from YACC 7199fa9565SMasatake YAMATOparser. 7274db2c55SMasatake YAMATO 7374db2c55SMasatake YAMATOHere YACC parser is a host parser. C parser is a guest parser. 74b45a42b3SHiroo HAYASHISee "`parsers/yacc.c 75b45a42b3SHiroo HAYASHI<https://github.com/universal-ctags/ctags/blob/master/parsers/yacc.c>`_". 7674db2c55SMasatake YAMATO 7774db2c55SMasatake YAMATO 7810454529SMasatake YAMATOPod/Perl parser combination 7974db2c55SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 8074db2c55SMasatake YAMATO 8199fa9565SMasatake YAMATOPod (Plain Old Documentation) is a language for documentation. The language 8299fa9565SMasatake YAMATOcan be used not only in a stand alone file but also it can be 8399fa9565SMasatake YAMATOused inside a Perl script. 8474db2c55SMasatake YAMATO 85dccba5efSHiroo HAYASHIUniversal Ctags has both parsers for Perl and Pod. 8699fa9565SMasatake YAMATOThe Perl parser recognizes the area where Pod document is 8799fa9565SMasatake YAMATOembedded in a Perl script and schedules applying pod parser 8899fa9565SMasatake YAMATOas a guest parser on the area. 8974db2c55SMasatake YAMATO 9004cce070SHiroo HAYASHI.. _base-sub-parsers: 913f73955fSMasatake YAMATO 92b45a42b3SHiroo HAYASHISubparser: Tagging definitions of higher (upper) level language 9321b5e450SMasatake YAMATO~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 9421b5e450SMasatake YAMATO 9521b5e450SMasatake YAMATOBackground 9621b5e450SMasatake YAMATO...................................................................... 9721b5e450SMasatake YAMATO 9821b5e450SMasatake YAMATOConsider an application written in language X. The application has 9921b5e450SMasatake YAMATOits domain own concepts. Developers of the application may try to 10021b5e450SMasatake YAMATOexpress the concepts in the syntax of language X. 10121b5e450SMasatake YAMATO 10221b5e450SMasatake YAMATOIn language X level, the developer can define functions, variables, types, and 10321b5e450SMasatake YAMATOso on. Further more, if the syntax of X allows, the developers want to 10421b5e450SMasatake YAMATOdefine higher level (= application level) things for implementing the 10521b5e450SMasatake YAMATOdomain own concepts. 10621b5e450SMasatake YAMATO 10721b5e450SMasatake YAMATOLet me show the part of source code of SPY-WARS, an imaginary game application. 10821b5e450SMasatake YAMATOIt is written in scheme language, a dialect of lisp. 1092b8661bbSK.Takata(Here `gauche <https://practical-scheme.net/gauche/index.html>`_ is considered 11021b5e450SMasatake YAMATOas the implementation of scheme interpreter). 11121b5e450SMasatake YAMATO 11221b5e450SMasatake YAMATO.. code-block:: scheme 11321b5e450SMasatake YAMATO 11421b5e450SMasatake YAMATO (define agent-tables (make-hash-table)) 11521b5e450SMasatake YAMATO (define-class <agent> () 11621b5e450SMasatake YAMATO ((rights :init-keyword :rights) 11721b5e450SMasatake YAMATO (responsibilities :init-keyword :responsibilities))) 11821b5e450SMasatake YAMATO 11921b5e450SMasatake YAMATO (define-macro (define-agent name rights responsibilities) 12021b5e450SMasatake YAMATO `(hash-table-put! agent-tables ',name 12121b5e450SMasatake YAMATO (make <agent> 12221b5e450SMasatake YAMATO :rights ',rights 12321b5e450SMasatake YAMATO :responsibilities ',responsibilities))) 12421b5e450SMasatake YAMATO 12521b5e450SMasatake YAMATO (define-agent Bond (kill ...) ...) 12621b5e450SMasatake YAMATO (define-agent Bourne ...) 12721b5e450SMasatake YAMATO 12821b5e450SMasatake YAMATO ... 12921b5e450SMasatake YAMATO 13086bcb5c2SHiroo HAYASHI``define``, ``define-class``, and ``define-macro`` are keywords of scheme 13121b5e450SMasatake YAMATOfor defining a variable, class and macro. Therefore scheme parser of 13286bcb5c2SHiroo HAYASHIctags should make tags for ``agent-tables`` with variable kind, 13386bcb5c2SHiroo HAYASHI``<agent>`` with class kind, and ``define-agent`` with macro kind. 13421b5e450SMasatake YAMATOThere is no discussion here. 13521b5e450SMasatake YAMATO 13686bcb5c2SHiroo HAYASHI NOTE: To be exactly ``define-class`` and ``define-macro`` are not the part 13721b5e450SMasatake YAMATO of scheme language. They are part of gauche. That means three parsers 13821b5e450SMasatake YAMATO are stacked: scheme, gosh, and SPY-WARS. 13921b5e450SMasatake YAMATO 14086bcb5c2SHiroo HAYASHIThe interesting things here are ``Bond`` and ``Bourne``. 14121b5e450SMasatake YAMATO 14221b5e450SMasatake YAMATO.. code-block:: scheme 14321b5e450SMasatake YAMATO 14421b5e450SMasatake YAMATO (define-agent Bond (kill ...) ...) 14521b5e450SMasatake YAMATO (define-agent Bourne ...) 14621b5e450SMasatake YAMATO 14721b5e450SMasatake YAMATOIn scheme parser level, the two expressions define nothing; the two 14886bcb5c2SHiroo HAYASHIexpressions are just macro (``define-agent``) expansions. 14921b5e450SMasatake YAMATO 15021b5e450SMasatake YAMATOHowever, in the application level, they define agents as the 151dccba5efSHiroo HAYASHImacro name shown. In this level Universal Ctags should capture 15286bcb5c2SHiroo HAYASHI``Bond`` and ``Bourne``. The question is which parser should 15321b5e450SMasatake YAMATOcapture them? scheme parser should not; define-agent is not part of 15421b5e450SMasatake YAMATOscheme language. Newly defined SPY-WARS parser is the answer. 15521b5e450SMasatake YAMATO 15686bcb5c2SHiroo HAYASHIThough ``define-agent`` is just a macro in scheme parser level, 15721b5e450SMasatake YAMATOit is keyword in SPY-WARS parser. SPY-WARS parser makes a 15886bcb5c2SHiroo HAYASHItag for a token next to ``define-agent``. 15921b5e450SMasatake YAMATO 16021b5e450SMasatake YAMATOThe above example illustrates levels of language in an input 16121b5e450SMasatake YAMATOfile. scheme is used as the base language. With the base language we 16221b5e450SMasatake YAMATOcan assume an imaginary higher level language named SPY-WARS is used 16321b5e450SMasatake YAMATOto write the application. To parse the source code of the application 16421b5e450SMasatake YAMATOwritten in two stacked language, ctags uses the two stacked parsers. 16521b5e450SMasatake YAMATO 16621b5e450SMasatake YAMATOMaking higher level language is very popular technique in the 16786bcb5c2SHiroo HAYASHIlanguages of lisp family (see "`On Lisp 16886bcb5c2SHiroo HAYASHI<http://www.paulgraham.com/onlisp.html>`_" for more details). 16921b5e450SMasatake YAMATOHowever, it is not special to lisp. 17021b5e450SMasatake YAMATO 17121b5e450SMasatake YAMATOFollowing code is taken from linux kernel written in C: 17221b5e450SMasatake YAMATO 17321b5e450SMasatake YAMATO.. code-block:: C 17421b5e450SMasatake YAMATO 17521b5e450SMasatake YAMATO DEFINE_EVENT(mac80211_msg_event, mac80211_info, 17621b5e450SMasatake YAMATO TP_PROTO(struct va_format *vaf), 17721b5e450SMasatake YAMATO TP_ARGS(vaf) 17821b5e450SMasatake YAMATO ); 17921b5e450SMasatake YAMATO 18021b5e450SMasatake YAMATOThere is no concept EVENT in C language, however it make sense in the 18121b5e450SMasatake YAMATOsource tree of linux kernel. So we can consider linux parser, based on 18286bcb5c2SHiroo HAYASHIC parser, which tags ``mac80211_msg_event`` as ``event`` kind. 18321b5e450SMasatake YAMATO 18421b5e450SMasatake YAMATO 18521b5e450SMasatake YAMATOTerms 18621b5e450SMasatake YAMATO...................................................................... 18721b5e450SMasatake YAMATO 18821b5e450SMasatake YAMATOBase parser and subparser 18921b5e450SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 19021b5e450SMasatake YAMATOIn the context of the SPY-WARS example, scheme parser is called a *base 19121b5e450SMasatake YAMATOparser*. The SPY-WARS is called a *subparser*. A base parser tags 19221b5e450SMasatake YAMATOdefinitions found in lower level view. A subparser on the base parser tags 19321b5e450SMasatake YAMATOdefinitions found in higher level view. This relationship can be nested. 1943cd8570eSHiroo HAYASHIA subparser can be a base parser for another subparser. 19521b5e450SMasatake YAMATO 19621b5e450SMasatake YAMATO.. figure:: stack-and-parsers.svg 19721b5e450SMasatake YAMATO :scale: 80% 19821b5e450SMasatake YAMATO 19921b5e450SMasatake YAMATOAt a glance the relationship between two parsers are similar to the 20021b5e450SMasatake YAMATOrelationship guest parser and host parser description in 201b45a42b3SHiroo HAYASHI":ref:`host-guest-parsers`". 202a3ebbe2bSMasatake YAMATOHowever, they are different. Though a guest 20321b5e450SMasatake YAMATOparser can run stand-alone, a subparser cannot; a subparser needs help 20421b5e450SMasatake YAMATOfrom base parser to work. 20521b5e450SMasatake YAMATO 20621b5e450SMasatake YAMATOTop down parser choice and bottom up parser choice 20721b5e450SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 20821b5e450SMasatake YAMATO 20986bcb5c2SHiroo HAYASHIThere are two ways to run a subparser: *top down* or *bottom up* parser 21021b5e450SMasatake YAMATOchoices. 21121b5e450SMasatake YAMATO 212dccba5efSHiroo HAYASHIUniversal Ctags can chose a subparser :ref:`automatically <guessing>`. 21321b5e450SMasatake YAMATOMatching file name patterns and extensions are the typical ways for 21486bcb5c2SHiroo HAYASHIchoosing. A user can choose a subparser with ``--language-force=`` option. 21521b5e450SMasatake YAMATOChoosing a parser in these deterministic way is called *top down*. 21621b5e450SMasatake YAMATOWhen a parser is chosen as a subparser in the top down way, the 21721b5e450SMasatake YAMATOsubparser must call its base parser. The base parser may call methods 21821b5e450SMasatake YAMATOdefined in the subparser. 21921b5e450SMasatake YAMATO 220dccba5efSHiroo HAYASHIUniversal Ctags uses *bottom up* choice when the top down way 22121b5e450SMasatake YAMATOdoesn't work; a given file name doesn't match any patterns and 22221b5e450SMasatake YAMATOextensions of subparsers and the user doesn't specify 22386bcb5c2SHiroo HAYASHI``--language-force=`` explicitly. In choosing a subparser bottom up way 22421b5e450SMasatake YAMATOit is assumed that a base parser for the subparser can be chosen 22521b5e450SMasatake YAMATOby top down way. During a base parser running, the base parser tries 22621b5e450SMasatake YAMATOto detect use of higher level languages in the input file. As shown 22721b5e450SMasatake YAMATOlater in this section, the base parser utilizes methods defined in its 22821b5e450SMasatake YAMATOsubparsers for the detection. If the base parser detects the use of a 22921b5e450SMasatake YAMATOhigher level language, a subparser for the higher level language is 23021b5e450SMasatake YAMATOchosen. Choosing a parser in this non-deterministic way (dynamic way) 23121b5e450SMasatake YAMATOis called *bottom up*. 23221b5e450SMasatake YAMATO 2338fc51ccaSHiroo HAYASHI============================== ================= 2348fc51ccaSHiroo HAYASHIinput file subparser choices 2358fc51ccaSHiroo HAYASHI============================== ================= 2368fc51ccaSHiroo HAYASHI*<SUB_LANG>* (``input.sub``) top down 2378fc51ccaSHiroo HAYASHI*<BASE_LANG>* (``input.base``) bottom up 2388fc51ccaSHiroo HAYASHI============================== ================= 2398fc51ccaSHiroo HAYASHI 240dccba5efSHiroo HAYASHIHere is an example. Universal Ctags has both m4 parser and Autoconf 24121b5e450SMasatake YAMATOparser. The m4 parser is a base parser. The Autoconf parser is a 24286bcb5c2SHiroo HAYASHIsubparser based on the m4 parser. If ``configure.ac`` is given as an 24321b5e450SMasatake YAMATOinput file, Autoconf parser is chosen automatically because the 24486bcb5c2SHiroo HAYASHIAutoconf parser has ``configure.ac`` in its patterns list. Based on the 245dccba5efSHiroo HAYASHIpattern matching, Universal Ctags chooses the Autoconf parser 24621b5e450SMasatake YAMATOautomatically (top down choice). 24721b5e450SMasatake YAMATO 24886bcb5c2SHiroo HAYASHIIf ``input.m4`` is given as an input file, the Autoconf parser is 24921b5e450SMasatake YAMATOnot chosen. Instead the m4 parser is chosen automatically because 25086bcb5c2SHiroo HAYASHIthe m4 parser has ``.m4`` in its extension list. The m4 parser passes 25121b5e450SMasatake YAMATOevery token finding in the input file to the 25221b5e450SMasatake YAMATOAutoconf parser. The Autoconf parser gets the chance to probe 25321b5e450SMasatake YAMATOwhether the Autoconf parser itself can handle the input or not; if 25486bcb5c2SHiroo HAYASHIa token name is started with ``AC_``, the Autoconf parser 25521b5e450SMasatake YAMATOreports "this is Autoconf input though its file extension 25686bcb5c2SHiroo HAYASHIis ``.m4``" to the m4 parser. As the result the Autoconf parser is 25721b5e450SMasatake YAMATOchosen (bottom up choice). 25821b5e450SMasatake YAMATO 25921b5e450SMasatake YAMATOSome subparsers can be chosen both top down and bottom up ways. Some 26021b5e450SMasatake YAMATOsubparser can be chosen only top down way or bottom up ways. 26121b5e450SMasatake YAMATO 262eb56edb2SHiroo HAYASHI.. _multiple_parsers_directions: 26321b5e450SMasatake YAMATO 264eb56edb2SHiroo HAYASHIDirection flags 26521b5e450SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 26621b5e450SMasatake YAMATO 267*755aeae5SMasatake YAMATO.. TESTCASE: Units/flags-langdef-directions.r 268*755aeae5SMasatake YAMATO 269eb56edb2SHiroo HAYASHI*Direction flags* specify how a base parser and a subparser work together. You 270eb56edb2SHiroo HAYASHIcan choose directions by putting a long flag after 271eb56edb2SHiroo HAYASHI``--langdef=<SUB_LANG>{base=<BASE_LANG>}``. 27221b5e450SMasatake YAMATO 273eb56edb2SHiroo HAYASHIThe following three direction flags are available; 274eb56edb2SHiroo HAYASHI 275eb56edb2SHiroo HAYASHI``shared`` (default): 276eb56edb2SHiroo HAYASHI For an input file of *<BASE_LANG>* (ex. ``input.base``), tags captured by 277eb56edb2SHiroo HAYASHI both *<BASE_LANG>* and *<SUB_LANG>* parser are recorded to tags file. 278eb56edb2SHiroo HAYASHI 279eb56edb2SHiroo HAYASHI For an input file of *<SUB_LANG>* (ex. ``input.sub``), tags captured by only 280eb56edb2SHiroo HAYASHI *<SUB_LANG>* parser are recorded to tags file. 281eb56edb2SHiroo HAYASHI 282eb56edb2SHiroo HAYASHI =============== ===================== ====================== 283eb56edb2SHiroo HAYASHI input file tags of *<BASE_LANG>* tags of *<SUB_LANG>* 284eb56edb2SHiroo HAYASHI =============== ===================== ====================== 285eb56edb2SHiroo HAYASHI ``input.base`` recorded recorded 286eb56edb2SHiroo HAYASHI ``input.sub`` --- recorded 287eb56edb2SHiroo HAYASHI =============== ===================== ====================== 288eb56edb2SHiroo HAYASHI 289eb56edb2SHiroo HAYASHI``dedicated``: 290eb56edb2SHiroo HAYASHI For an input file of *<BASE_LANG>* (ex. ``input.base``), tags captured by 291eb56edb2SHiroo HAYASHI only *<BASE_LANG>* parser are recorded to tags file. 292eb56edb2SHiroo HAYASHI 293eb56edb2SHiroo HAYASHI For an input file of *<SUB_LANG>* (ex. ``input.sub``), tags captured by both 294eb56edb2SHiroo HAYASHI *<BASE_LANG>* and *<SUB_LANG>* parser are recorded to tags file. 295eb56edb2SHiroo HAYASHI 296eb56edb2SHiroo HAYASHI =============== ===================== ====================== 297eb56edb2SHiroo HAYASHI input file tags of *<BASE_LANG>* tags of *<SUB_LANG>* 298eb56edb2SHiroo HAYASHI =============== ===================== ====================== 299eb56edb2SHiroo HAYASHI ``input.base`` recorded --- 300eb56edb2SHiroo HAYASHI ``input.sub`` recorded recorded 301eb56edb2SHiroo HAYASHI =============== ===================== ====================== 302eb56edb2SHiroo HAYASHI 303eb56edb2SHiroo HAYASHI``bidirectional``: 304eb56edb2SHiroo HAYASHI For an input file of both *<BASE_LANG>* (ex. ``input.base``) and 305eb56edb2SHiroo HAYASHI *<SUB_LANG>* (ex. ``input.sub``), tags captured by both *<BASE_LANG>* and 306eb56edb2SHiroo HAYASHI <SUB_LANG> parser are recorded to tags file. 307eb56edb2SHiroo HAYASHI 308eb56edb2SHiroo HAYASHI =============== ===================== ====================== 309eb56edb2SHiroo HAYASHI input file tags of *<BASE_LANG>* tags of *<SUB_LANG>* 310eb56edb2SHiroo HAYASHI =============== ===================== ====================== 311eb56edb2SHiroo HAYASHI ``input.base`` recorded recorded 312eb56edb2SHiroo HAYASHI ``input.sub`` recorded recorded 313eb56edb2SHiroo HAYASHI =============== ===================== ====================== 314eb56edb2SHiroo HAYASHI 315eb56edb2SHiroo HAYASHIIf no direction flag is specified, it implies ``{shared}``. 316eb56edb2SHiroo HAYASHI 317eb56edb2SHiroo HAYASHISee ":ref:`optlib_directions`" in ":ref:`optlib`" for examples of using the 318eb56edb2SHiroo HAYASHIdirection flags. 31921b5e450SMasatake YAMATO 3208fc51ccaSHiroo HAYASHIListing subparsers 3218fc51ccaSHiroo HAYASHI......................................................................... 3228fc51ccaSHiroo HAYASHISubparsers can be listed with ``--list-subparser``: 32321b5e450SMasatake YAMATO 3248fc51ccaSHiroo HAYASHI.. code-block:: console 32521b5e450SMasatake YAMATO 3268fc51ccaSHiroo HAYASHI $ ctags --options=./linux.ctags --list-subparsers=C 3278fc51ccaSHiroo HAYASHI #NAME BASEPARSER DIRECTION 3288fc51ccaSHiroo HAYASHI linux C base => sub {shared} 32921b5e450SMasatake YAMATO 3309d278fdcSMasatake YAMATOCommand line interface 3318fc51ccaSHiroo HAYASHI......................................................................... 3321b2f09f1SMasatake YAMATO 3333cd8570eSHiroo HAYASHIRunning subparser can be controlled with subparser (``s``) extras flag. 3341b2f09f1SMasatake YAMATOBy default it is enabled. To turning off the feature running 33586bcb5c2SHiroo HAYASHIsubparser, specify ``--extras=-s``. 3361b2f09f1SMasatake YAMATO 3371b2f09f1SMasatake YAMATO 338b45a42b3SHiroo HAYASHIExamples of subparser 33921b5e450SMasatake YAMATO...................................................................... 34021b5e450SMasatake YAMATO 34121b5e450SMasatake YAMATOAutomake/Make parser combination 34221b5e450SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 34321b5e450SMasatake YAMATO 34421b5e450SMasatake YAMATOSimply to say the syntax of Automake is the subset of Make. However, 34521b5e450SMasatake YAMATOthe Automake parser has interests in Make macros having special 34686bcb5c2SHiroo HAYASHIsuffixes: ``_PROGRAMS``, ``_LTLIBRARIES``, and ``_SCRIPTS`` so on. 34721b5e450SMasatake YAMATO 34821b5e450SMasatake YAMATOHere is an example of input for Automake: 34921b5e450SMasatake YAMATO 35021b5e450SMasatake YAMATO.. code-block:: Make 35121b5e450SMasatake YAMATO 35221b5e450SMasatake YAMATO bin_PROGRAMS = ctags 35321b5e450SMasatake YAMATO ctags_CPPFLAGS = \ 35421b5e450SMasatake YAMATO -I. \ 35521b5e450SMasatake YAMATO -I$(srcdir) \ 35621b5e450SMasatake YAMATO -I$(srcdir)/main 35721b5e450SMasatake YAMATO 35886bcb5c2SHiroo HAYASHIFrom the point of the view of the Make parser, ``bin_PROGRAMS`` is a just 35986bcb5c2SHiroo HAYASHIa macro; the Make parser tags ``bin_PROGRAMS`` as a macro. The Make parser 36086bcb5c2SHiroo HAYASHIdoesn't tag ``ctags`` being right side of '``=``' because it is not a new 36121b5e450SMasatake YAMATOname: just a value assigned to bin_PROGRAMS. However, for the Automake 36286bcb5c2SHiroo HAYASHIparser ``ctags`` is a new name; the Automake parser tags ``ctags`` with 36386bcb5c2SHiroo HAYASHIkind ``Program``. The Automake parser can tag it with getting help from 36421b5e450SMasatake YAMATOthe Make parser. 36521b5e450SMasatake YAMATO 36621b5e450SMasatake YAMATOThe Automake parser is an exclusive subparser. It is chosen in top 36786bcb5c2SHiroo HAYASHIdown way; an input file name ``Makefile.am`` gives enough information for 36821b5e450SMasatake YAMATOchoosing the Automake parser. 36921b5e450SMasatake YAMATO 37021b5e450SMasatake YAMATOTo give chances to the Automake parser to capture Automake own 37121b5e450SMasatake YAMATOdefinitions, The Make parser provides following interface in 37286bcb5c2SHiroo HAYASHI``parsers/make.h``: 37321b5e450SMasatake YAMATO 37421b5e450SMasatake YAMATO.. code-block:: C 37521b5e450SMasatake YAMATO 37621b5e450SMasatake YAMATO struct sMakeSubparser { 37721b5e450SMasatake YAMATO subparser subparser; 37821b5e450SMasatake YAMATO 37921b5e450SMasatake YAMATO void (* valueNotify) (makeSubparser *s, char* name); 38021b5e450SMasatake YAMATO void (* directiveNotify) (makeSubparser *s, char* name); 38121b5e450SMasatake YAMATO void (* newMacroNotify) (makeSubparser *s, 38221b5e450SMasatake YAMATO char* name, 38321b5e450SMasatake YAMATO bool withDefineDirective, 38421b5e450SMasatake YAMATO bool appending); 38521b5e450SMasatake YAMATO }; 38621b5e450SMasatake YAMATO 38721b5e450SMasatake YAMATOThe Automake parser defines methods for tagging Automake own definitions 38886bcb5c2SHiroo HAYASHIin a ``struct sMakeSubparser`` type variable, and runs the Make parser by 38986bcb5c2SHiroo HAYASHIcalling ``scheduleRunningBaseparser`` function. 39021b5e450SMasatake YAMATO 39121b5e450SMasatake YAMATOThe Make parser tags Make own definitions in an input file. In 39221b5e450SMasatake YAMATOaddition Make parser calls the methods during parsing the input file. 39321b5e450SMasatake YAMATO 39421b5e450SMasatake YAMATO.. code-block:: console 39521b5e450SMasatake YAMATO 39645e335abSHiroo HAYASHI $ ctags --fields=+lK --extras=+r -o - Makefile.am 39721b5e450SMasatake YAMATO bin Makefile.am /^bin_PROGRAMS = ctags$/;" directory language:Automake 39821b5e450SMasatake YAMATO bin_PROGRAMS Makefile.am /^bin_PROGRAMS = ctags$/;" macro language:Make 39921b5e450SMasatake YAMATO ctags Makefile.am /^bin_PROGRAMS = ctags$/;" program language:Automake directory:bin 40021b5e450SMasatake YAMATO ctags_CPPFLAGS Makefile.am /^ctags_CPPFLAGS = \\$/;" macro language:Make 40121b5e450SMasatake YAMATO 40286bcb5c2SHiroo HAYASHI``bin_PROGRAMS`` and ``ctags_CPPFLAGS`` are tagged as macros of Make. 40386bcb5c2SHiroo HAYASHIIn addition ``bin`` is tagged as directory, and ``ctags`` as program of Automake. 40421b5e450SMasatake YAMATO 40586bcb5c2SHiroo HAYASHI``bin`` is tagged in a callback function assigned to ``newMacroFound`` method. 40686bcb5c2SHiroo HAYASHI``ctags`` is tagged in a callback function assigned to ``valuesFound`` method. 40721b5e450SMasatake YAMATO 4083cd8570eSHiroo HAYASHI``--extras=+r`` is used in the example. Reference (``r``) extra is needed to 40986bcb5c2SHiroo HAYASHItag ``bin``. ``bin`` is not defined in the line, ``bin_PROGRAMS =``. 41086bcb5c2SHiroo HAYASHI``bin`` is referenced as a name of directory where programs are 41186bcb5c2SHiroo HAYASHIstored. Therefore ``r`` is needed. 41221b5e450SMasatake YAMATO 41386bcb5c2SHiroo HAYASHIFor tagging ``ctags``, the Automake parser must recognize 41486bcb5c2SHiroo HAYASHI``bin`` in ``bin_PROGRAMS`` first. ``ctags`` is tagged 41586bcb5c2SHiroo HAYASHIbecause it is specified as a value for ``bin_PROGRAMS``. 41686bcb5c2SHiroo HAYASHIAs the result ``r`` is also needed to tag ``ctags``. 41721b5e450SMasatake YAMATO 41821b5e450SMasatake YAMATOOnly Automake related tags are emitted if Make parser is 41921b5e450SMasatake YAMATOdisabled. 42021b5e450SMasatake YAMATO 42121b5e450SMasatake YAMATO.. code-block:: console 42221b5e450SMasatake YAMATO 42345e335abSHiroo HAYASHI $ ctags --languages=-Make --fields=+lKr --extras=+r -o - Makefile.am 4249062cbc8SMasatake YAMATO bin Makefile.am /^bin_PROGRAMS = ctags$/;" directory language:Automake roles:program 42521b5e450SMasatake YAMATO ctags Makefile.am /^bin_PROGRAMS = ctags$/;" program language:Automake directory:bin 42621b5e450SMasatake YAMATO 42721b5e450SMasatake YAMATOAutoconf/M4 parser combination 42821b5e450SMasatake YAMATO,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 42921b5e450SMasatake YAMATO 430dccba5efSHiroo HAYASHIUniversal Ctags uses m4 parser as a base parser and Autoconf parse as 43186bcb5c2SHiroo HAYASHIa subparser for ``configure.ac`` input file. 43221b5e450SMasatake YAMATO 43321b5e450SMasatake YAMATO.. code-block:: Autoconf 43421b5e450SMasatake YAMATO 43521b5e450SMasatake YAMATO AC_DEFUN([PRETTY_VAR_EXPAND], 43621b5e450SMasatake YAMATO [$(eval "$as_echo_n" $(eval "$as_echo_n" "${$1}"))]) 43721b5e450SMasatake YAMATO 43821b5e450SMasatake YAMATOThe m4 parser finds no definition here. However, Autoconf parser finds 43986bcb5c2SHiroo HAYASHI``PRETTY_VAR_EXPAND`` as a macro definition. Syntax like ``(...)`` is part 44021b5e450SMasatake YAMATOof M4 language. So Autoconf parser is implemented as a subparser of 44121b5e450SMasatake YAMATOm4 parser. The most parts of tokens in input files are handled by 44286bcb5c2SHiroo HAYASHIM4. Autoconf parser gives hints for parsing ``configure.ac`` and 44321b5e450SMasatake YAMATOregisters callback functions to 44421b5e450SMasatake YAMATOAutoconf parser. 445
