9. History and Projects

9.1 News: Changes in Recent Versions of X-Symbol

This is the complete history of X-Symbol. It just lists the major changes before Version 3.0.

9.1.1 Changes in X-Symbol 4.5.1

Version 4.5.1 has not yet been announced.

• Various bug fixes and minor changes.

9.1.2 Changes in X-Symbol 4.5

Version 4.5 has been released on March 2003 as beta.

• X-Symbol finally respects the Mule coding system of each individual buffer.

• Bug fix: would mess up encoding of math-mode characters with token language bib. Other conversion fixes for languages bib and texi.

• Bug fix (workaround for bug in XEmacs): auto-save files would have length 0.

• Bug fix (Emacs): package now works with package crypt/crypt++.

• Token language sgml: always encode characters to entity references by default (where defined by the HTML standard). Include hm--html-mode, html-helper-mode, remove sgml-mode as typical major modes which use X-Symbol.

• Token language tex: support some symbols of package `stmaryrd.sty'.

• Change the auto-style, formerly auto-mode, mechanism.

• Image support when running on Emacs.

• New input method Quail, a usual Mule input method.

• Corrected Latin-5 definitions. Support Latin-5 ("Turkish") on XEmacs running under Windows.

• X-Symbol works with Emacs/XEmacs running under a character terminal.

• Improments for external languages. Super-/subscript matching of token languages has changed.

• X-Symbol can use package format and does not require special fonts for super-/subscripts with Emacs-21.4+. Still open whether this will be used....

• Dropped support for XEmacs-20.3.

• Various bug fixes and minor changes.

9.1.3 Changes in X-Symbol 4.2 to 4.4

Version 4.4 has been released on June 2002 as beta.

• Token language TeX has changed: no excessive use of braces anymore, no excessive normalization, and aware of environments {tabbing} and {verbatim}, and macro \verb. Reading and saving "old-encoded" files works without changes in the file (the buffer looks different), there is also a command to remove the unwanted braces around accented letters.

• New token language "BibTeX Macro" (bib, similar to old tex), used for BibTeX files.

• Nuked executables, the Lisp conversion for all languages is now 2-5 times faster.

• Latin-9 support. Latin-9 font included in distribution.

• Works with XEmacs-21.4+ on Windows. Of course, it just supports a limited number of characters and no super- and subscripts there due to missing fonts.

• More likely to save 8bit characters in the file by default: also look for 8bit characters in the file when visiting the file, also inspect master file (TeX-master) with token language tex.

• New buffer-local variable x-symbol-unique: when non-nil, decodes much less tokens to avoid near to all normalizations, used for TeX's style files (but X-Symbol is not automatically turned on). Dropped token language utex.

• Menu changes, new commands: submenu "Conversion", menu items "Copy Encoded", "Paste Decoded" and others.

• Special coding for preview-latex. Using X-Symbol now only gives a 10% overhead of previews parsing time.

• X-Symbol now works with WhizzyTeX.

• The interface for defining a token language has changed, it is also much more general, useful for ProofGeneral.

• Changed final bytes of ISO 2022 escape sequence for X-Symbol charsets since Emacs reserves the characters `0-9' for itself. Does XEmacs has any policy here (it also uses `?')?

• Dropped workaround for minor bug in XEmacs-20.X.

• Various bug fixes and minor changes.

9.1.4 Changes in X-Symbol 4.1

Version 4.1 has been released on Mar 2002 as beta.

• X-Symbol works with Emacs-21.1 or higher. Porting is not complete, yet.

• New token language "TeXinfo command" (texi).

• Slightly different definition of "valid character".

• Remove the "local if set" and "default: ..." submenu stuff.

9.1.5 Changes in X-Symbol 3.4

Version 3.4 has been released on Mar 2002.

• Moved to SourceForge.net. Added files for nicer HTML output of manual.

• Would sometimes perform strange conversions when global-flyspell-mode is enabled.

• Bug fixes: command M-x write-region would always save the whole buffer if X-Symbol is enabled for that buffer, writing a remote file via ange-ftp would not work (was OK with efs).

• Automatically deduce default coding via locale -ck LC_CTYPE.

• Issue warning when running on XEmacs-21.0 to XEmacs-21.1.8. Update manual: XEmacs user package directory is `~/.xemacs/packages'.

• Directories ending with `//' in image search paths are recursive.

• New characters used for token languages "TeX macro" and "Isabelle symbol".

• Make sure to convert just the first part of a multi-part image.

• Source distribution includes files for building an RPM package, all files also compile without Mule support.

• Minor changes. Manual changes.

9.1.6 Changes in X-Symbol 3.3

Version 3.3 has been released on Jan 1999.

• Package X-Symbol is really a proper XEmacs package: no need to create fonts and to set the font path. With XEmacs/no-Mule, I still recommend to create the executables (type M-x x-symbol-exec-create).

• New functions used for interaction with Emacs package comint. This is necessary for new token language "Isabelle symbol", to be distributed with Emacs package ProofGeneral.

• New characters used for token languages "TeX macro" and "Isabelle symbol".

• Minor changes. Manual changes.

9.1.7 Changes in X-Symbol 3.2

Version 3.2 has been released on Dec 1998.

• Package X-Symbol is a proper XEmacs package. The installation process is much easier (using the binary package). It has changed, though! The use of file `x-symbol-site.el' is deprecated.

• Reverting the buffer and using vc commands do not encode characters when not using crypt. (This did not work always.)

• Workaround for bug (segfault) in XEmacs-21/Mule betas.

• Command x-symbol-package-bug is less restrictive. Please use this command to contact the maintainer.

• Bug fixes. Minor changes. Manual changes.

9.1.8 Changes in X-Symbol 3.1

Version 3.1 has been released on Oct 1998.

• TeX macro \mu is represented by a character in the Adobe Symbol font, not in a Latin-{1,3,5} font anymore.

• Support for most SGML entities in HTML-4.0 specification.

• Additional characters for \therefore/∴, ‾ and €.

• Package X-Symbol has been customized.

• The documentation has been completed (as TeXinfo file).

• Handle special URL prefixes `file:', `http:' for images.

• Bug fixes, configuration changes.

9.1.9 Changes in X-Symbol 3.0

Version 3.0 has been released on Sep 1998 as beta.

• Package X-Symbol now works on XEmacs with and without Mule support. Dropped support for XEmacs 19.13 to 19.16/20.2.

• Full support of token language sgml (executables, subscripts, images).

• X-Symbol is a proper minor mode.

• Easier (automatic) 8bit character control (e.g., for \times \pm,...). By default, the encoding when saving only writes 8bit characters, if `\usepackage[latinn]{inputenc}' with n=1,2,3,5 was found in the first 10000 characters of the file (including commentary).

• Package X-Symbol can be easily extended with new token languages and fonts due to its modular design. It consistently handles situations where an entry for an additional character defines the same preferred key binding (and context) as for a previously defined character

• Key bindings have completely changed. They are now consistent with the contexts of input method Context (which have changed a bit).

• The keys @ and ! are not used anymore as Modify- and Rotate-Key. The Rotate key (instead of the Modify-Key) is used to "Greek"ify the previous Ascii char.

• Input method Aggressive Context is now called input method Electric and is much more restrictive (using package texmathp with language "TeX macro").

• Easier installation despite many additional features.

• Supports more characters.

• Nicer grid, info in echo area.

• Better cooperation with packages: vc (check-out does not convert characters), reftex (no strange characters `\237', help with label creation), auctex, ispell, font-latex (no annoyances with \exists).

• Safer use of executables.

• The code has completely changed. You have to redo your installation.

9.1.10 Changes in Old Releases.

This sections gives just an overview of the major changes in the releases.

Version 2.6 has been released on Oct 1998.

• Fixed serious bug when used under tty.

Version 2.5 has been released on Mar 1998.

• Image support.

Version 2.4 has been released on Mar 1997.

• Token language sgml. (X-Symbol can handle more then token language tex.)

• Input method Aggressive Context (precursor of input method Electric), input method Context has been much improved.

• Fixed performance bug when saving a file with package crypt.

• Control of Conversion and 8bit character has changed.

Version 2.3 has been released on Sep 1996.

• Distributed with own font for more math characters.

• Info for the character around point in echo area.

Version 2.2 has been released on June 1996.

• Input method Grid. Help when using input method Keyboard.

• Control of Conversion and 8bit character has changed.

Version 2.1 has been released on April 1996.

• Fixed serious performance bug when loading files with font-lock/lazy-lock. Use executables for conversion of large buffers.

• The package iso-cvt is not integrated anymore. Now this package can also convert to/from Latin-1 characters, it is much faster.

• Menu support, including input method Menu.

• isearch works with X-Symbol characters.

• First multi-file version.

Version 1.4 has been released on Feb 1996.

• Provide some kind of "poor man's Mule" to remove most Nomule-Problems.

Version 1.3 has been released on Jan 1996.

• Input method Abbrev (precursor of input method Token).

• Super-/subscript support.

Version 1.2 has been released on Jan 1996. It was the first release.

• Conversion between characters and TeX tokens. Do so automatically when visiting a file and saving the buffer.

• Input method Keyboard.

9.2 Wishlist: Projects for X-Symbol

You are encouraged to try to provide a solution to one of the problems of this section. In fact, it is quite unlikely that I do it myself without any contributions from you, see also 9.3 Open Questions.

Providing a solution to these problems is the second way of making your name appear in 9.4 Acknowledgments.

9.2.1 Wishlist: Additional Token Languages

Making a contribution here would require just a basic knowledge of Emacs and X-Symbol. In fact, I would do the non-trivial part of the Emacs Lisp part (see section 7.4 Extending Package X-Symbol) for general-interest token languages (e.g., AmsTeX).

It is likely that this would require additional fonts: available fonts (e.g., IPA font), hand-crafted, or generated (see section 9.2.2 Wishlist: Generated Fonts).

9.2.2 Wishlist: Generated Fonts

A specific direction of font generation would be from `.bdf' or `.pcf' font files to Windows fonts to get rid of the limited support for XEmacs on Windows (see section 2.1 Requirements). If you have successfully converted X-Symbol's fonts from the Unix format to the Windows format (via bdftofon or whatever) or if you have free and real Latin-N fonts for Windows, please let me know! I would also appreciate if you would actively try to get those missing Windows fonts.

The general direction is to automatically generate the `.bdf' or `.fon' fonts from other sources. This would have various advantages:

• We could easily create different sizes for our symbol font.

• It would be quite simple to create a font for AmsTeX macros, etc., which would be displayed as X-Symbol characters by package X-Symbol.

• We could easily create different sizes for our symbol font.

• We would have fonts for both X11 and Windows.

New fonts for X-Symbol are being worked on. You can find material to generate them at the web pages of X-Symbol. Quite a few problems needs to be fixed though, so it is considered as experimental. You are welcome to try, fix and report on the X-Symbol development mailing list.

General open design issues (i.e., they could be re-thought for the currently used handcrafted fonts, too) are:

• Different TeX macros (same appearance, different TeX class = different spacing) use the same MetaFont character, e.g., \dagger and \dag. Therefore, we need different X11 characters for them.

• Some Ascii characters have a special meaning in TeX. The corresponding MetaFont character is therefore produced by a TeX macro, e.g., `{' by \{. We need a X11 character which looks similar to the character but not exactly like it.

We could ask the question whether we should really distinguish the characters by appearance...we have the minibuffer info for the X-Symbol character anyway.... Here are the options:

• distinguished by size/underlining/miscellaneous (currently used),
• distinguished by different spacing (my current favorite),
• not distinguished

9.2.3 Wishlist: Changes in Emacs/XEmacs

Changes in Emacs and/or XEmacs would improve package X-Symbol, too:

• In Emacs: a package system similar to XEmacs' one. The installation would be easier.

• The package ispell assumes the buffer contents to be the same as the file contents and does not provide any hook to fix this. This should be fixed in ispell (it will be better in Emacs-21.4), see 2.6.4 Miscellaneous Packages.

• Some versions control commands turn off font-lock. This should be changed.

• Provide a face property raise: we wouldn't need extra fonts for super- and subscripts. Emacs: it's already a display property, make it a face property, too (or make font-lock set properties other than faces). XEmacs: no such property, yet.

• You are sometimes unnecessarily asked (because X-Symbol will encode the corresponding characters anyway) for a safe coding system. In Emacs (will be fixed in Emacs-21.4) for non-default Latin characters. In XEmacs, for all non-default characters if you use package latin-unity (see section 2.6.3 File I/O Packages).

• In Emacs, will be fixed in 21.4. Using isearch and the input method Grid would not work.

• In XEmacs, fixed in 21.X. In after-insert-file-functions, there should be a possibility to get to know the start position of the region which is inserted. If insert-file-contents is called with argument replace being non-nil, it is not always point.

• In Emacs and XEmacs, will be fixed in Emacs-21.4. Make possibility to change buffers in write-region-annotate-functions official, see 7.6.3 Alternative Auto Conversion Methods, have a way to get the original buffer.

• Since font-lock uses duplicable text properties in some cases, I need a function like insert-buffer-substring-without-extents. (Currently, I remove the extents afterwards, which looks slow for me.)

• In XEmacs. Run hooks in post-command-hook even if command exits with an error or quit (as it is in Emacs) or having some post-error-or-quit-hook. See section 8.1 Problems under XEmacs/no-Mule.

• In XEmacs. There are some bugs in package custom/widget (still in XEmacs-21.4) which are visible during the customization of X-Symbol.

9.2.4 Wishlist: Changes in LaTeX

Changes in LaTeX, especially `inputenc.sty', would improve package X-Symbol, too:

• To make the definition of the character U00B5 consistent with Unicode, `inputenc.sty' should define the character to stand not for the token \mu (U03BC is the right character), but for an extra token, e.g., something like \textmicro. X-Symbol uses \mathmicro here in order to avoid changing \mu to the character U00B5 if you have chosen to store 8bit characters.

• Use same encoding for both text and math, i.e. use periodcentered for both \textperiodcentered (the default) and \cdot. At least provide text-and-math versions for characters where no alternative is more obvious than the other. If that is not possible, always choose text mode except for \lnot, \pm, \times and \division: use \textonesuperior for U00B9, \texttwosuperior for U00B2, and \textthreesuperior for U00B3.

• The TeX macros \textcent, \textcurrency, \textbrokenbar, \textyen are defined as not available with OT1 and T1 font encoding. This should be changed.

9.2.5 Various Projects for X-Symbol

The following suggestions seem to be useful, though not essential:

• It would be nice if we could print the buffer contents. Currently, you see strange characters instead X-Symbol's own characters.

  Printing non-standard fonts is only possible via the Emacs package ps-print. A newer version of ps-print might be probably already capable of doing it. Thus, you are encouraged to help the XEmacs team updating this package.

9.2.6 Rejected Suggestions for X-Symbol

The following suggestions seem to be not useful enough to be worth the additional effort and increased package size. I might be convinced otherwise by patches (i.e., code, not text), though:

• It would be nice if X-Symbol would replace the token with the last character of the token if this is possible (see section 4.2 Input Method Token: Replace Token by Character), not just with the next character. Well, during typing, this is not really annoying and after a while, you will use input method Token only for very short tokens.

9.3 Open Questions

This section lists some minor open questions.

• Loading file `x-symbol.el' will initialize package X-Symbol (via function x-symbol-initialize), since all functions will need the initialization. In my opinion, this is no problem, since all customization options are defined an other files which do not require file `x-symbol.el'. Thus, customizing package X-Symbol will not initialize package X-Symbol.

  The alternative would be to call function x-symbol-initialize in every function which can be autoloaded. This seems quite tedious to me. Also, I do not see a reason not to call x-symbol-initialize top-level in file `x-symbol.el'. If I am wrong here, please let me know (with an explanation). Batch-compilation might be an issue...

• When is necessary to set x-symbol-auto-conversion-method to slowest? Of course, it is only necessary when using crypt. Is the other necessary condition to use the computer pool of the University of Edinburgh?

9.4 Acknowledgments

Stefan Monnier did many of the changes necessary for porting X-Symbol to Emacs-21. Fortunately, he not only changed X-Symbol to use a quite different API on Emacs for things like charsets and menus, he also made the necessary changes in Emacs itself. Before that, Sang-Min Lee started porting X-Symbol to Emacs-20.4, which was important for moving the status of the Emacs port of X-Symbol from "todo" to "in work".

David Kastrup demonstrated that the old way of encoding characters to TeX macros generally inhibited ligatures and kerns, i.e., it was worse than expected. He also discussed the details of how to do the encoding and decoding right. Christophe Raffalli suggested to use a decode method which can be used for a larger class of token languages. He also proved that it is faster.

Solofo Ramangalahy is working on scripts to generate X-Symbol fonts from other sources. This has various advantages and is discussed in more detail at 9.2.2 Wishlist: Generated Fonts. His work is now available at the X-Symbol download area.

Package math-mode by Renaud Marlet and the extension of it by Julian Bradfield gave the basic idea for the following features: supporting TeX's math macros, input methods token, context/electric, super-/subscript support. The shell script makesub is a merge and change of the scripts makesupers and makesub by Julian.

The font `xsymb0', which is distributed with this package, is a minor modification (appearance) of the Adobe symbol font, thanks to its non-restrictive copyright. You may use the Adobe font instead. The special images are from package frame-icon.

The idea for Help during an X-Symbol key sequence is from package x-compose. The general idea for showing some info in the echo area is from package eldoc. The trick which stops expand-abbrev is from package mail-abbrevs. The idea for x-symbol-image-cache-directories is from package fast-lock. The code for image command parsing is influenced by some code in package font-lock. The code around x-symbol-image-delete-extents is based on some code in package bib-cite.

Thanks for patches/reports/suggestions to: Adrian Aichner, Vladimir Alexiev, David Aspinall, Masayuki Ataka, Neal Becker, Matthias Berberich, Stefano Bianchi, Janusz S. Bien, Uwe Brauer, Alastair Burt, John Collins, Laurent Descamps, Frederic Devernay, Carsten Dominik, Steve Dunham, Michael Ebner, Stephen Eglen, Paul Furnanz, Jeffrey Grandy, Clemens Gr@"opl, Kenichi Handa, Meik Hellmund, Ryurick M. Hristev, Adriaan Joubert, Marcin Kasperski, David Kastrup, Richard Ketchersid, Felix E. Klee, Gerwin Klein, Thomas Kleymann, Ekkehard Koehler, Fred Labrosse, Jan-Ake Larsson, Bernhard Lehner, Stefan Monnier, Harald Muehlboeck, Karsten Muehlmann, Jakub Narebski, Peter Møller Neergaard, Raymond Nijssen, David von Oheimb, Alex Ott, Sudeep Kumar Palat, Arshak Petrosyan, Jim Radford, Christophe Raffalli, Solofo Ramangalahy, Alex Russell, Marciano Siniscalchi, Richard M. Stallman, Axel Thimm, Eli Tziperman, Jan Vroonhof, Markus Wenzel, Sabine Wetzel, Pierre-Henri Wuillemin, Roland Zumkeller, Marco Zunino, Gerard Zwaan.

Thanks for general information to: Per Abrahamsen, Steve L. Baur, Kenichi Handa, David Kastrup, Gerd Moellmann, Stefan Monnier, Primoz Peterlin, Martin Ramsch, Peter Schmitt, Toby Speight, Jan Vroonhof, Eli Zaretskii.

I made use of information from the following URLs:

  http://www.fmi.uni-passau.de/~ramsch/iso8859-1.html
  http://czyborra.com/charsets/iso8859.html
  http://www.bbsinc.com/iso8859.html
  http://www.bbsinc.com/iso8879.html
  http://ppewww.ph.gla.ac.uk/~flavell/charset/internat.html
  http://ppewww.ph.gla.ac.uk/~flavell/iso8859/iso8859-pointers.html
  http://sizif.mf.uni-lj.si/linux/cee/iso8859-2.html

I do not intend to update this list in the future--this is just an "Acknowledgment" section.