| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Once a source file has been parsed, the following APIs can be used to write programs that use the token stream most effectively.
| 9.1 Token Queries | Getting info about a parsed token (nonterminal). | |
| 9.2 Nonterminal streams | Working with streams of nonterminals. | |
| 9.3 Nonterminals at point | Finding nonterminals at point. | |
| 9.4 Nonterminal sorting | Reorganizing streams. | |
| 9.5 Nonterminal completion | Completing read functions. | |
| 9.6 Override Methods | Language dependent functions covering conversion to text strings, language dependent queries and local context information | |
| 9.7 Parser Hooks | How to know when tags change. | |
| 9.8 Programming Examples | Simple programming examples. |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
When writing programs that use the bovinator, the following functions are needed to find get details out of a nonterminal.
nil if they are equivalent.
Use eq to test of two tokens are the same. Use this function if tokens
are being copied and regrouped to test for if two tokens represent the same
thing, but may be constructed of different cons cells.
'variable, 'function, 'type, or other.
nil means no parents.
The car of the list is either the parent class, or a list
of parent classes. The cdr of the list is the list of
interfaces, or abstract classes which are parents of TOKEN.
nil if TOKEN is a destructor function.
nil if it doesn't exist.
For override methods that query a token, see See section 9.6.2 Token Details.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
These functions take some key, and returns information found inside the nonterminal stream. Some will return one token (the first matching item found.) Others will return a list of all items matching a given criterion. All these functions work regardless of a buffer being in memory or not.
nil, search children of tokens.
If SEARCH-INCLUDE is non-nil, search include files.
nil, don't do
the median calculation, and return nil.
nil, don't do
the median calculation, and return nil.
This function will find the topmost item, and recurse until no more
details are available of findable.
nil if an element of STREAM will be included
in the new list.
If optional argument SEARCH-PARTS is non-nil, all sub-parts of tokens
are searched. The over-loadable function semantic-nonterminal-children is
used for the searching child lists. If SEARCH-PARTS is the symbol
'positiononly, then only children that have positional information are
searched.
If SEARCH-INCLUDES is non-nil, then all include files are also
searched for matches.
nil if an element of STREAM will be included
in the new list.
If optional argument SEARCH-PARTS, all sub-parts of tokens are searched.
The over-loadable function semantic-nonterminal-children is used for
searching.
If SEARCH-INCLUDES is non-nil, then all include files are also
searched for matches.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
When you just want to get at a nonterminal the cursor is on, there is
a more efficient mechanism than using
semantic-find-nonterminal-by-position. This mechanism
directly queries the overlays the parsing step leaves in the buffer.
This provides for very rapid retrieval of what function or variable
the cursor is currently in.
These functions query the current buffer's overlay system for tokens.
nil, use the current point.
Optional BUFFER is used if POSITIONORMARKER is a number, otherwise the current
buffer is used. This finds all tokens covering the specified position
by checking for all overlays covering the current spot. They are then sorted
from largest to smallest via the start location.
nil if there is no parent.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Sometimes it is important to reorganize a token stream into a form that is better for display to a user. It is important to not use functions with side effects when doing this, and that could effect the token cache.
There are some existing utility functions which will reorganize the token list for you.
This function uses semantic-nonterminal-external-member-p to determine when a potential child is an externally defined member.
Note: Applications which use this function must account for token types which do not have a position, but have children which *do* have positions.
Applications should use semantic-mark-external-member-function
to modify all tokens which are found as externally defined to some
type. For example, changing the token type for generating extra
buckets with the bucket function.
'type for metaparents.
A metaparent is a made-up type semantic token used to hold the child list
of orphaned members of a named type.
adopted property.
This function should be locally bound by a program that needs
to add additional behaviors into the token list.
This function is called with one argument which is a shallow copy
of the token to be modified. This function should return the
token (or a copy of it) which is then integrated into the
revised token list.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
These functions provide ways reading the names of items in a buffer with completion.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
These functions are called `override methods' because they provide generic behaviors, which a given language can override. For example, finding a dependency file in Emacs lisp can be done with the `locate-library' command (which overrides the default behavior.) In C, a dependency can be found by searching a generic search path which can be passed in via a variable.
| 9.6.1 Token->Text | Converting Tokens into text strings | |
| 9.6.2 Token Details | Arbitrary token detail fetching | |
| 9.6.3 Local Context | Deriving information about a language specific local context. | |
| 9.6.4 Making New Methods | How to add your own methods for a tool |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Any given token consists of Meta information which is best viewed in some textual form. This could be as simple as the token's name, or as a prototype to be added to header file in C. Not only are there several default converters from a Token into text, but there is also some convenient variables that can be used with them. Use these variables to allow options on output forms when displaying tokens in your programs.
font-lock.
:type field of a customizable variable.
Every token to text conversion function must take the same parameters, which are TOKEN, the token to be converted, PARENT, the containing parent (like a structure which contains a variable), and COLOR, which is a flag specifying that color should be applied to the returned string.
When creating, or using these strings, particularly with color, use concat to build up larger strings instead of format. This will preserve text properties.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
These functions help derive information about tokens that may not be obvious for non-traditional languages with their own token types.
nonterminal-children
is to return type parts for a type, and arguments for a function.
If optional argument POSITIONALONLY is non-nil, then only return valid
children if they contain positions. Some languages may choose to create
lists of children without position/overlay information.
If this function is overridden, use semantic-nonterminal-children-default to also include the default behavior, and merely extend your own.
Note for language authors: If a mode defines a language that has tokens in it with overlays that should not be considered children, you should still return them with this function. If you do not, then token re-parsing, and database saving will fail.
The default behavior, if not overridden with
nonterminal-external-member-parent is get the 'parent extra
specifier of TOKEN.
If this function is overridden, use semantic-nonterminal-external-member-parent-default to also include the default behavior, and merely extend your own.
nil if PARENT is the parent of TOKEN.
TOKEN is an external member of PARENT when it is somehow tagged
as having PARENT as it's parent.
The default behavior, if not overridden with
nonterminal-external-member-p is to match 'parent extra specifier in
the name of TOKEN.
If this function is overridden, use
semantic-nonterminal-external-member-children-p-default to also
include the default behavior, and merely extend your own.
nil, then also search files in
the Semantic Database. If USEDB is a list of databases, search those
databases.
Children in this case are functions or types which are members of TOKEN, such as the parts of a type, but which are not defined inside the class. C++ and CLOS both permit methods of a class to be defined outside the bounds of the class' definition.
The default behavior, if not overridden with
nonterminal-external-member-children is to search using
semantic-nonterminal-external-member-p in all top level definitions
with a parent of TOKEN.
If this function is overridden, use semantic-nonterminal-external-member-children-default to also include the default behavior, and merely extend your own.
nil - No special protection. Language dependent.
'public - Anyone can access this TOKEN.
'private - Only methods in the local scope can access TOKEN.
'friend - Like private, except some outer scopes are allowed
access to token.
Some languages may choose to provide additional return symbols specific
to themselves. Use of this function should allow for this.
The default behavior (if not overridden with nonterminal-protection
is to return a symbol based on type modifiers.
nil if TOKEN is abstract.
Optional PARENT is the parent token of TOKEN.
In UML, abstract methods and classes have special meaning and behavior
in how methods are overridden. In UML, abstract methods are italicized.
The default behavior (if not overridden with nonterminal-abstract
is to return true if abstract is in the type modifiers.
nil if TOKEN is leaf.
Optional PARENT is the parent token of TOKEN.
In UML, leaf methods and classes have special meaning and behavior.
The default behavior (if not overridden with nonterminal-leaf
is to return true if leaf is in the type modifiers.
nil if TOKEN is static.
Optional PARENT is the parent token of TOKEN.
In UML, static methods and attributes mean that they are allocated
in the parent class, and are not instance specific.
UML notation specifies that STATIC entries are underlined.
The default behavior (if not overridden with nonterminal-static
is to return true if static is in the type modifiers.
semantic-dependency-include-path for searching. Always searches
`.' first, then searches additional paths.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
nil if there are no more context levels.
Overloaded functions using up-context take no parameters.
nil if there is no upper context.
The default behavior uses semantic-up-context. It can
be overridden with beginning-of-context.
nil if there is no upper context.
Be default, this uses semantic-up-context, and assumes parenthetical
block delimiters. This can be overridden with end-of-context.
bovine-inner-scope to
parse tokens at the beginning of the context.
This can be overridden with get-local-variables.
get-local-arguments.
get-all-local-variables.
Optional argument POINT is the location to start getting the variables from.
These next set of functions handle local context parsing. This means looking at the code (locally) and navigating, and fetching information such as a the type of the parameter the cursor may be typing in.
semantic-command-separation-character.
Override with end-of-command.
semantic-command-separation-character.
Override with beginning-of-command.
ctxt-current-symbol.
nil if there is nothing relevant.
Override with ctxt-current-assignment.
ctxt-current-function.
ctxt-current-argument.
ctxt-scoped-types.
For details on using these functions to get more detailed information about the current context: See section 10.4 Analysis of the current context.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If you write a program that uses the stream of tokens in a persistent display or database, it is necessary to know when tokens change so that your displays can be updated. This is especially important as tokens can be replaced, changed, or deleted, and the associated overlays will then throw errors when you try to use them. Complete integration with token changes can be achieved via several very important hooks.
One interesting way to interact with the parser is to let it know that changes you are going to make will not require re-parsing.
nil, modifications do not require a reparse.
This prevents tokens from being marked dirty, and it
prevents top level edits from causing a cache check.
Use this when writing programs that could cause a full
reparse, but will not change the tag structure, such
as adding or updating top-level comments.
Next, it is sometimes useful to know what the current parsing state is. These function can let you know what level of re-parsing may be needed. Careful choices on when to reparse can make your program much faster.
nil if the current buffer needs a full reparse.
Optional argument CHECKCACHE indicates if the cache check should be made.
nil if the current buffer needs a partial reparse.
This only returns non-nil if semantic-bovine-toplevel-full-reparse-needed-p
returns nil.
Optional argument CHECKCACHE indicates if the cache check should be made
when checking semantic-bovine-toplevel-full-reparse-needed-p.
If you need very close interaction with the user's editing, then these two hooks can be used to find out when a given tag is being changed. These hooks could even be used to cut down on re-parsing if used correctly.
For all hooks, make sure you are careful to add it as a local hook if you only want to effect a single buffer. Setting it globally can cause unwanted effects if your program is concerned with a single buffer.
semantic-clean-token-hooks, unless a full
reparse is done instead.
semantic-dirty-token-hooks. This hook is not called for tokens
marked dirty if the buffer is completely re-parsed. In that case, use
semantic-after-toplevel-cache-change-hook.
after-change-function.
Lastly, if you just want to know when a buffer changes, use this hook.
This hook should not be used any more.
Use semantic-after-toplevel-cache-change-hook instead.
Hook functions must take one argument, which is the new list of tokens associated with this buffer.
For language specific hooks, make sure you define this as a local hook.
Hook functions must take one argument, which is the list of tokens updated among the ones associated with this buffer.
For language specific hooks, make sure you define this as a local hook.
semantic-after-toplevel-cache-change-hook which is also called
during a flush when the cache is given a new value of nil.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Here are some simple examples that use different aspects of the semantic library APIs. For fully functional example programs with lots of comments, see the file `semantic-examples.el'.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If you need a command that asks the user for a token name, you can get full range completion using the query functions 9.5 Nonterminal completion.
(interactive (list (semantic-read-symbol "Symbol: "))) |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If you have the name of a function or variable, and need to find its location in a buffer, you need a search function. There is a wide range of searches you can perform 9.2 Nonterminal streams.
(semantic-find-nonterminal-by-name "some-name" (current-buffer) t ;; look inside structures and classes for these symbols nil) ;; do not look inside header files. |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If you have the name of a function or variable, and need to find its location somewhere in a project, you need to use the Semantic Database 11.3 Semantic Database. There are many search functions similar to the ones found in 9.2 Nonterminal streams.
The Semantic Database is interesting in that the return structure is not
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If you have a nonterminal token, or a list of them, you may want to find their position in a buffer.
(semanticdb-find-nonterminal-by-name "symbol" nil ;; Defaults to the current project's database list. t ;; Search inside types nil ;; Do not search include files nil ;; Only search files in the same mode (all C files) t ;; When a token is found, make sure it is loaded in a buffer. ) |
Of interesting note above, semanticdb can find symbols in files that are not loaded into an Emacs buffer. These tokens do not have an associated overlay, and the function semantic-token-buffer will fail.
The last parameter's tells the search function to find-file-noselect any file in which a matching token was found. This will allow you to merge all the tokens into a completion list, or other flat list needed by most functions that use association lists.
If you do not ask semanticdb to load those files, you will need to
explicitly request the database object (found in the car of
each sublist) get the file loaded. It is useful to not auto find all
files if you don't need to jump to that token.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
A nonterminal token is a rather unpleasant Lisp structure when trying to decipher what is going on. As such, there is a wide range of functions available that can convert a token into a human readable, and colorful string 9.6.1 Token->Text.
If you program interfaces with lots of users, you will probably want to have your program define a configurable variable that will let users change the visible portion of your program.
(defcustom my-summary-function 'semantic-uml-prototype-nonterminal "*Function to use when showing info about my token." :group 'my-program :type semantic-token->text-custom-list) |
Note the special type provided by semantic.
Next, you can call this function to create a string.
(funcall my-summary-function token
token-parent
t ; use color
)
|
In this case, token-parent is an optional argument. In many cases, parent is not used by the outputting function. The parent may be a struct or class that contains token, or nil for top-level definitions. In particular, C++ needs the parent to correctly calculate the protection of each method.
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |