1====================================================================== 2Other changes 3====================================================================== 4 5:Maintainer: Masatake YAMATO <yamato@redhat.com> 6 7.. contents:: `Table of contents` 8 :depth: 3 9 :local: 10 11---- 12 13Many changes have been introduced in Universal Ctags. Use git-log to 14review changes not enumerated here, especially in language parsers. 15 16New and extended options 17--------------------------------------------------------------------- 18 19``--exclude-exception``, an option complementing ``--exclude`` 20~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 21 22See :ref:`option_input_output_file` in :ref:`ctags(1) <ctags(1)>`. 23 24``--maxdepth`` option 25~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 26 27See :ref:`option_input_output_file` in :ref:`ctags(1) <ctags(1)>`. 28 29``--input-encoding=ENCODING`` and ``--output-encoding=ENCODING`` 30~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 31 32.. TODO: Review... 33 34People may use their own native language in source code comments (or 35sometimes in identifiers) and in such cases encoding may become an issue. 36Nowadays UTF-8 is the most widely used encoding, but some source codes 37still use legacy encodings like latin1, cp932 and so on. These options 38are useful for such files. 39 40ctags doesn't consider the input encoding; it just reads input as a 41sequence of bytes and uses them as is when writing tags entries. 42 43On the other hand Vim does consider input encoding. When loading a 44file, Vim converts the file contents into an internal format with one 45of the encodings specified in its `fileencodings` option. 46 47As a result of this difference, Vim cannot always move the cursor to 48the definition of a tag as users expect when attempting to match the 49patterns in a tags file. 50 51The good news is that there is a way to notify Vim of the encoding 52used in a tags file with the ``TAG_FILE_ENCODING`` pseudo-tag. 53 54Two new options have been introduced (``--input-encoding=IN`` and 55``--output-encoding=OUT``). 56 57Using the encoding specified with these options ctags converts input 58from ``IN`` to ``OUT``. ctags uses the converted strings when writing 59the pattern parts of each tag line. As a result the tags output is 60encoded in ``OUT`` encoding. 61 62In addition ``OUT`` is specified at the top the tags file as the 63value for the ``TAG_FILE_ENCODING`` pseudo-tag. The default value of 64``OUT`` is UTF-8. 65 66NOTE: Converted input is NOT passed to language parsers. 67The parsers still deal with input as a byte sequence. 68 69With ``--input-encoding-<LANG>=IN``, you can specify a specific input 70encoding for ``LANG``. It overrides the global default value given 71with ``--input-encoding``. 72 73The example usage can be found in *Tmain/{input,output}-encoding-option.d*. 74 75Acceptable ``IN`` and ``OUT`` values can be listed with *iconv -l* or 76*iconv --list*. It is platform dependant. 77 78To enable the option, libiconv is needed on your platform. 79On Windows mingw (without msys2), you must specify ``WITH_ICONV=yes`` 80like this:: 81 82 C:\dev\ctags>mingw32-make -f mk_mingw.mak WITH_ICONV=yes 83 84``--list-features`` helps you to know whether your ctags executable 85links to libiconv or not. You will find ``iconv`` in the output if it 86links to. 87 88See also :ref:`option_output_format` in :ref:`ctags(1) <ctags(1)>`. 89 90``--map-<LANG>`` option 91~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 92 93.. IN MAN PAGE 94 95``--map-<LANG>`` is newly introduced to control the file name 96to language mappings (langmap) with finer granularity than 97``--langmap`` allows. 98 99A langmap entry is defined as a pair; the name of the language and a 100file name extension (or pattern). 101 102Here we use "spec" as a generic term representing both file name 103extensions and patterns. 104 105``--langmap`` maps specs to languages exclusively:: 106 107 $ ctags --langdef=FOO --langmap=FOO:+.ABC \ 108 --langdef=BAR --langmap=BAR:+.ABC \ 109 --list-maps | grep '\*.ABC$' 110 BAR *.ABC 111 112Though language `FOO` is added before `BAR`, only `BAR` is set as a 113handler for the spec `*.ABC`. 114 115Universal Ctags enables multiple parsers to be configured for a spec. 116The appropriate parser for a given input file can then be chosen by a 117variety of internal guessing strategies (see :ref:`Determining file language 118<guessing>`). 119 120Let's see how specs can be mapped non-exclusively with 121``--map-<LANG>``:: 122 123 $ ctags --langdef=FOO --map-FOO=+.ABC \ 124 --langdef=BAR --map-BAR=+.ABC \ 125 --list-maps | grep '\*.ABC$' 126 FOO *.ABC 127 BAR *.ABC 128 129Both `FOO` and `BAR` are registered as handlers for the spec `*.ABC`. 130 131``--map-<LANG>`` can also be used for removing a langmap entry.:: 132 133 $ ctags --langdef=FOO --map-FOO=+.ABC \ 134 --langdef=BAR --map-BAR=+.ABC \ 135 --map-FOO=-.ABC --list-maps | grep '\*.ABC$' 136 BAR *.ABC 137 138 $ ctags --langdef=FOO --map-FOO=+.ABC \ 139 --langdef=BAR --map-BAR=+.ABC \ 140 --map-BAR=-.ABC --list-maps | grep '\*.ABC$' 141 FOO *.ABC 142 143 $ ctags --langdef=FOO --map-FOO=+.ABC \ 144 --langdef=BAR --map-BAR=+.ABC \ 145 --map-BAR=-.ABC --map-FOO=-.ABC --list-maps | grep '\*.ABC$' 146 (NOTHING) 147 148``--langmap`` provides a way to manipulate the langmap in a 149spec-centric manner and ``--map-<LANG>`` provides a way to manipulate 150the langmap in a parser-centric manner. 151 152See also :ref:`option_lang_mapping` in :ref:`ctags(1) <ctags(1)>`. 153 154Guessing parser from file contents (``-G`` option) 155~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 156 157See :ref:`guessing` in :ref:`ctags(1) <ctags(1)>`. 158 159Including line number to pattern field 160~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 161 162Use ``--excmd=number``. 163See :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 164 165Long names in kinds, fields, and extra options 166~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 167 168A letter is used for specifying a kind, a field, or an extra entry. 169In Universal Ctags a name can also be used. 170 171Surround the name with braces (`{` and `}`) in values assigned to the 172options, ``--kind-<LANG>=``, ``--fields=``, or ``--extras=``. 173 174.. code-block:: console 175 176 $ ctags --kinds-C=+L-d ... 177 178This command line uses the letters, `L` for enabling the label kind 179and `d` for disabling the macro kind of C. The command line can be 180rewritten with the associated names. 181 182.. code-block:: console 183 184 $ ctags --kinds-C='+{label}-{macro}' ... 185 186The quotes are needed because braces are interpreted as meta 187characters by the shell. 188 189The available names can be listed with ``--list-kinds-full``, 190``--list-fields``, or ``--list-extras``. 191 192See also :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 193 194Wildcard in options 195~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 196 197For the purpose of gathering as much as information as possible from 198source code the "wildcard"(``*``) option value has been introduced. 199 200``--extras=*`` 201 Enables all extra tags. 202 203``--fields=*`` 204 Enables all available fields. 205 206``--kinds-<LANG>=*`` 207 Enables all available kinds for ``LANG``. 208 209``--kinds-all=*`` 210 Enables all available kinds for all available language parsers. 211 212See also :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 213 214Extra tag entries (``--extras``) 215~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 216``--extra`` option in Exuberant Ctags is renamed to ``--extras`` (plural) in 217Universal Ctags for making consistent with ``--kinds-<LANG>`` and ``--fields``. 218 219These extra tag entries are newly introduced. 220 221``F`` 222 Replacement for --file-scope. 223 224``p`` 225 Include pseudo-tags. 226 227.. 228 NOT REVIEWED YET 229 230See also :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 231 232Kinds synchronization 233~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 234See the description about ``--kinds-<LANG>`` and ``--list-kinds-full`` 235option on :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 236 237Enabling/disabling pseudo-tags (``--pseudo-tags`` option) 238~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 239 240.. IN MAN PAGE 241 242See :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>` and 243:ref:`ctags-client-tools(7) <ctags-client-tools(7)>` about the option. 244 245``--put-field-prefix`` options 246~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 247 248See :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 249 250"always" and "never" as an argument for ``--tag-relative`` 251~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 252 253``--tag-relative`` option is extend. 254See :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>`. 255 256Defining a parser specific extra 257~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 258 259A new ``--_extradef-<LANG>=name,description`` option allows you to 260defining a parser specific extra which turning on and off can be 261referred from a regex based parser for ``<LANG>``. 262 263See :ref:`Conditional tagging with extras <extras>` for more details. 264 265Defining a CPreProcessor macro from command line 266~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 267 268Newly introduced ``-D`` option extends the function provided by 269``-I`` option. 270 271``-D`` emulates the behaviour of the corresponding gcc option: 272it defines a C preprocessor macro. 273 274See :ref:`option_tags_file_contents` in :ref:`ctags(1) <ctags(1)>` and 275:ref:`cxx` for more details. 276 277Options for inspecting ctags internals 278~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 279 280Exuberant Ctags provides a way to inspect its internals via 281``--list-kinds``, ``--list-languages``, and ``--list-maps``. 282 283This idea has been expanded in Universal Ctags with 284``--list-kinds-full``, ``--list-map-extensions``, ``--list-extras``, 285``--list-features``, ``--list-fields``, ``--list-map-patterns``, and 286``--list-pseudo-tags`` being added. 287 288The original three ``--list-`` options are not changed for 289compatibility reasons, however, the newly introduced options are 290recommended for all future use. 291 292By default, interactive use is assumed and ctags tries aligning the 293list output in columns for easier reading. 294 295When ``--machinable`` is given before a ``--list-`` option, ctags 296outputs the list in a format more suitable for processing by scripts. 297Tab characters are used as separators between columns. The alignment 298of columns is never considered when ``--machinable`` is given. 299 300Currently only ``--list-extras``, ``--list-fields`` and 301``--list-kinds-full`` support ``--machinable`` output. 302 303These new ``--list-`` options also print a column header, a line 304representing the name of each column. The header may help users and 305scripts to understand and recognize the columns. Ignoring the column 306header is easy because it starts with a `#` character. 307 308``--with-list-header=no`` suppresses output of the column header. 309 310See also :ref:`option_listing` in :ref:`ctags(1) <ctags(1)>`. 311 312Notice messages and ``--quiet`` 313~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 314There were 3 classes of message in Exuberant Ctags. 315In addition to them Universal Ctags introduced a new class of message, *notice*. 316 317*fatal* 318 A critical error has occurred and ctags aborts the execution. 319 320*warning* 321 An error has occurred but ctags continues the execution. 322 323*notice* (new) 324 It is less important than *warning* but more important for users than *verbose*. 325 326*verbose* 327 Mainly used for debugging purposes. 328 329Generally the user can ignore *notice* class messages and ``--quiet`` 330can be used to disable them. 331 332*verbose* class messages are disabled by default, and ``--verbose`` or ``-V`` 333can be used to enable them. 334 335See also :ref:`option_misc` in :ref:`ctags(1) <ctags(1)>`. 336 337Skipping utf-8 BOM 338~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 339 340The three bytes sequence(``\xEF\xBB\xBF``) at the head of an input 341file is skipped when parsing. 342 343TODO: 344 345* Do the same in guessing and selecting parser stage. 346* Refect the BOM detection to encoding option 347 348Interactive mode 349~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 350 351A new ``--_interactive`` option launches a JSON based command REPL which 352can be used to control ctags generation programmatically. 353 354See :ref:`interactive-mode` for more details. 355 356PCRE2 regular expression 357~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 358 359With ``{pcre2}`` (or ``p``) flag, PCRE2 expressions can be used in 360``--regex-<LANG>=``, ``--mline-regex-<LANG>=``, and 361``--_mtable-regex-<LANG>=`` if the ctags is built with ``pcre2`` library. 362 363 364Incompatible changes in command line 365--------------------------------------------------------------------- 366 367.. NOT REVIEWED YET 368 369``-D`` option 370~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 371 372For a ctags binary that had debugging output enabled in the build config 373stage, ``-D`` was used for specifying the level of debugging 374output. It is changed to ``-d``. This change is not critical because 375``-D`` option was not described in ctags.1 man page. 376 377Instead ``-D`` is used for defining a macro in CPreProcessor parser. 378 379Changes imported from Exuberant Ctags 380--------------------------------------------------------------------- 381See "Exuberant Ctags" in "Tracking other projects" for detailed 382information regarding imported changes. 383 384Some changes have also been imported from Fedora and Debian. 385 386Parser related changes 387--------------------------------------------------------------------- 388 389New parsers 390~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 391The following parsers have been added: 392 393* Abaqus 394* Abc 395* Ada 396* AnsiblePlaybook *libyaml* 397* Asciidoc 398* Autoconf 399* Automake 400* AutoIt 401* BibTeX 402* Clojure 403* CMake *optlib* 404* CSS 405* Ctags option library *optlib* 406* CUDA 407* D 408* DBusIntrospect *libxml* 409* Diff 410* DTD 411* DTS 412* Elixir *optlib* 413* Elm *peg/packcc* 414* Falcon 415* FrontMatter *only YAML syntax, running as a guest on R?Markdown* 416* FunctionParameters *perl based subparser* 417* Gdbinit script *optlib* 418* GemSpec *Ruby based subparser* 419* GDScript 420* Glade *libxml* 421* Go 422* Haskell 423* Haxe 424* iPythonCell *optlib*, *pthon based subparser* 425* Inko *optlib* 426* JavaProperties 427* JSON 428* Julia 429* Kconfig *optlib* 430* Kotlin *peg/packcc* 431* GNU linker script(LdScript) 432* LEX *optlib* 433* Man page *optlib* 434* Markdown 435* Maven2 *libxml* 436* MesonBuild (Meson) *optlib* 437* MesonOptions *optlib+script* 438* Moose *perl based subparser* 439* Myrddin 440* M4 441* NSIS 442* ObjectiveC 443* Org *optlib* 444* OpenAPI (3.x.x / Swagger 2.0) *Yaml based subparser* 445* Passwd *optlib* 446* PuppetManifest *optlib* 447* Perl6 448* Pod *optlib* 449* PowerShell 450* PropertyList(plist) *libxml* 451* Protobuf 452* PythonLoggingConfig 453* QemuHX *optlib* 454* QtMoc 455* R 456* R6Class *R based subparser* 457* Rake *Ruby based subparser* 458* RDoc *optlib pcre2* 459* RelaxNG *libxml* 460* ReStructuredText 461* RMarkdown *Markdown based subparser* 462* Robot 463* RpmMacros *optlib* 464* RpmSpec 465* RSpec *Ruby based subparser* 466* Rust 467* S4Class *R based subparser* 468* SCSS *optlib* 469* SystemdUnit 470* SystemTap *optlib* 471* SystemVerilog 472* SVG *libxml* 473* TclOO (see :ref:`The new Tcl parser <tcl>`) 474* Thrift *peg/packcc* 475* TTCN 476* Txt2tags 477* TypeScript 478* Varlink *peg/packcc* 479* WindRes 480* XSLT v1.0 *libxml* 481* Yacc 482* Yaml *libyaml* 483* YumRepo 484* Zephir 485 486See :ref:`optlib` for details on *optlib*. 487Libxml2 is required to use the parser(s) marked with *libxml*. 488Libyaml is required to use the parser(s) marked with *libyaml*. 489pcre2 is required to use the parser(s) marked with *pcre2*. 490 491TIPS: you can list newly introduced parsers if you also have 492Exuberant Ctags installed with following command line: 493 494.. code-block:: console 495 496 $ diff -ruN <(universal-ctags --list-languages) <(exuberant-ctags --list-languages) | grep '^[-+]' 497 498Fully improved parsers 499~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 500* C (see :ref:`The new C/C++ parser <cxx>`) 501* C++ (see :ref:`The new C/C++ parser <cxx>`) 502* Python (see :ref:`The new Python parser <python>`) 503* HTML (see :ref:`The new HTML parser <html>`) 504* Tcl (see :ref:`The new Tcl parser <tcl>`) 505* ITcl (see :ref:`The new Tcl parser <tcl>`) 506* Ant (rewritten with *libxml*) 507* PHP 508* Verilog/SystemVerilog 509 510Automatically expanding CPreProcessor macros defined in the same input file (HIGHLY EXPERIMENTAL) 511~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 512 513See :ref:`The new C/C++ parser <cxx>` for more details. 514 515Readtags 516--------------------------------------------------------------------- 517 518Printing line numbers with ``-n`` 519~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 520See :ref:`readtags(1) <readtags(1)>`. 521 522Filtering in readtags command 523~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 524See :ref:`readtags(1) <readtags(1)>`. 525 526readtags has ability to find tag entries by name. 527 528The concept of filtering is inspired by the display filter of 529Wireshark. You can specify more complex conditions for searching. 530 531All symbols starting with `$` represent a field of a tag entry which 532is being tested against the S expression. Most will evaluate as a 533string or `#f`. It evaluates to `#f` when the field doesn't exist. 534 535The `scope` field holds structured data: the kind and name of the 536upper scope combined with `:`. The hold the value is stored to 537`$scope`. The kind part is mapped to `$scope-kind`, and the name part 538to `$scope-name`. 539 540`$scope-kind` and `$scope-name` can only be used if the input tags 541file is generated by ctags with ``--fields=+Z``. 542 543`$` is a generic accessor for accessing extension fields. 544`$` takes one argument: the name of an extension field. 545It returns the value of the field as a string if a value 546is given, or `#f`. 547 548Following examples shows how `prefix?`, `suffix?`, and 549`substr?` work. 550:: 551 552 (prefix? "TARGET" "TA") 553 => #t 554 555 (prefix? "TARGET" "RGET") 556 => #f 557 558 (prefix? "TARGET" "RGE") 559 => #f 560 561 (suffix? "TARGET" "TA") 562 => #f 563 564 (suffix? "TARGET" "RGET") 565 => #t 566 567 (suffix? "TARGET" "RGE") 568 => #f 569 570 (substr? "TARGET" "TA") 571 => #t 572 573 (suffix? "TARGET" "RGET") 574 => #t 575 576 (suffix? "TARGET" "RGE") 577 => #t 578 579 (and (suffix? "TARGET" "TARGET") 580 (prefix? "TARGET" "TARGET") 581 (substr? "TARGET" "TARGET") 582 => #t 583 584 585Sorting in readtags command 586~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 587readtags can sort the tag entries before printing. 588You can specify the way to sort with -S option. Like ``-Q`` option, ``-S`` 589also takes an S expression. 590 591See :ref:`readtags(1) <readtags(1)>`. 592 593 594Listing pseudo tags with ``-D`` 595~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 596See :ref:`readtags(1) <readtags(1)>`. 597