com.ibm.icu.text
public abstract class Transliterator extends Object
Transliterator
is an abstract class that
transliterates text from one format to another. The most common
kind of transliterator is a script, or alphabet, transliterator.
For example, a Russian to Latin transliterator changes Russian text
written in Cyrillic characters to phonetically equivalent Latin
characters. It does not translate Russian to English!
Transliteration, unlike translation, operates on characters, without
reference to the meanings of words and sentences.
Although script conversion is its most common use, a
transliterator can actually perform a more general class of tasks.
In fact, Transliterator
defines a very general API
which specifies only that a segment of the input text is replaced
by new text. The particulars of this conversion are determined
entirely by subclasses of Transliterator
.
Transliterators are stateless
Transliterator
objects are stateless; they
retain no information between calls to
transliterate()
. As a result, threads may share
transliterators without synchronizing them. This might seem to
limit the complexity of the transliteration operation. In
practice, subclasses perform complex transliterations by delaying
the replacement of text until it is known that no other
replacements are possible. In other words, although the
Transliterator
objects are stateless, the source text
itself embodies all the needed information, and delayed operation
allows arbitrary complexity.
Batch transliteration
The simplest way to perform transliteration is all at once, on a
string of existing text. This is referred to as batch
transliteration. For example, given a string input
and a transliterator t
, the call
String result = t.transliterate(input);
will transliterate it and return the result. Other methods allow
the client to specify a substring to be transliterated and to use
{@link Replaceable} objects instead of strings, in order to
preserve out-of-band information (such as text styles).
Keyboard transliteration
Somewhat more involved is keyboard, or incremental transliteration. This is the transliteration of text that is arriving from some source (typically the user's keyboard) one character at a time, or in some other piecemeal fashion.
In keyboard transliteration, a Replaceable
buffer
stores the text. As text is inserted, as much as possible is
transliterated on the fly. This means a GUI that displays the
contents of the buffer may show text being modified as each new
character arrives.
Consider the simple RuleBasedTransliterator
:
th>{theta}
t>{tau}
When the user types 't', nothing will happen, since the
transliterator is waiting to see if the next character is 'h'. To
remedy this, we introduce the notion of a cursor, marked by a '|'
in the output string:
t>|{tau}
{tau}h>{theta}
Now when the user types 't', tau appears, and if the next character
is 'h', the tau changes to a theta. This is accomplished by
maintaining a cursor position (independent of the insertion point,
and invisible in the GUI) across calls to
transliterate()
. Typically, the cursor will
be coincident with the insertion point, but in a case like the one
above, it will precede the insertion point.
Keyboard transliteration methods maintain a set of three indices
that are updated with each call to
transliterate()
, including the cursor, start,
and limit. These indices are changed by the method, and they are
passed in and out via a Position object. The start
index
marks the beginning of the substring that the transliterator will
look at. It is advanced as text becomes committed (but it is not
the committed index; that's the cursor
). The
cursor
index, described above, marks the point at
which the transliterator last stopped, either because it reached
the end, or because it required more characters to disambiguate
between possible inputs. The cursor
can also be
explicitly set by rules in a RuleBasedTransliterator
.
Any characters before the cursor
index are frozen;
future keyboard transliteration calls within this input sequence
will not change them. New text is inserted at the
limit
index, which marks the end of the substring that
the transliterator looks at.
Because keyboard transliteration assumes that more characters
are to arrive, it is conservative in its operation. It only
transliterates when it can do so unambiguously. Otherwise it waits
for more characters to arrive. When the client code knows that no
more characters are forthcoming, perhaps because the user has
performed some input termination operation, then it should call
finishTransliteration()
to complete any
pending transliterations.
Inverses
Pairs of transliterators may be inverses of one another. For
example, if transliterator A transliterates characters by
incrementing their Unicode value (so "abc" -> "def"), and
transliterator B decrements character values, then A
is an inverse of B and vice versa. If we compose A
with B in a compound transliterator, the result is the
indentity transliterator, that is, a transliterator that does not
change its input text.
The Transliterator
method getInverse()
returns a transliterator's inverse, if one exists, or
null
otherwise. However, the result of
getInverse()
usually will not be a true
mathematical inverse. This is because true inverse transliterators
are difficult to formulate. For example, consider two
transliterators: AB, which transliterates the character 'A'
to 'B', and BA, which transliterates 'B' to 'A'. It might
seem that these are exact inverses, since
"A" x AB -> "B"where 'x' represents transliteration. However,
"B" x BA -> "A"
"ABCD" x AB -> "BBCD"so AB composed with BA is not the identity. Nonetheless, BA may be usefully considered to be AB's inverse, and it is on this basis that AB
"BBCD" x BA -> "AACD"
.getInverse()
could legitimately return
BA.
IDs and display names
A transliterator is designated by a short identifier string or ID. IDs follow the format source-destination, where source describes the entity being replaced, and destination describes the entity replacing source. The entities may be the names of scripts, particular sequences of characters, or whatever else it is that the transliterator converts to or from. For example, a transliterator from Russian to Latin might be named "Russian-Latin". A transliterator from keyboard escape sequences to Latin-1 characters might be named "KeyboardEscape-Latin1". By convention, system entity names are in English, with the initial letters of words capitalized; user entity names may follow any format so long as they do not contain dashes.
In addition to programmatic IDs, transliterator objects have display names for presentation in user interfaces, returned by {@link #getDisplayName}.
Factory methods and registration
In general, client code should use the factory method
getInstance()
to obtain an instance of a
transliterator given its ID. Valid IDs may be enumerated using
getAvailableIDs()
. Since transliterators are
stateless, multiple calls to getInstance()
with the
same ID will return the same object.
In addition to the system transliterators registered at startup,
user transliterators may be registered by calling
registerInstance()
at run time. To register a
transliterator subclass without instantiating it (until it is
needed), users may call registerClass()
.
Composed transliterators
In addition to built-in system transliterators like "Latin-Greek", there are also built-in composed transliterators. These are implemented by composing two or more component transliterators. For example, if we have scripts "A", "B", "C", and "D", and we want to transliterate between all pairs of them, then we need to write 12 transliterators: "A-B", "A-C", "A-D", "B-A",..., "D-A", "D-B", "D-C". If it is possible to convert all scripts to an intermediate script "M", then instead of writing 12 rule sets, we only need to write 8: "A~M", "B~M", "C~M", "D~M", "M~A", "M~B", "M~C", "M~D". (This might not seem like a big win, but it's really 2n vs. n2 - n, so as n gets larger the gain becomes significant. With 9 scripts, it's 18 vs. 72 rule sets, a big difference.) Note the use of "~" rather than "-" for the script separator here; this indicates that the given transliterator is intended to be composed with others, rather than be used as is.
Composed transliterators can be instantiated as usual. For example, the system transliterator "Devanagari-Gujarati" is a composed transliterator built internally as "Devanagari~InterIndic;InterIndic~Gujarati". When this transliterator is instantiated, it appears externally to be a standard transliterator (e.g., getID() returns "Devanagari-Gujarati").
Subclassing
Subclasses must implement the abstract method
handleTransliterate()
.
Subclasses should override
the transliterate()
method taking a
Replaceable
and the transliterate()
method taking a String
and StringBuffer
if the performance of these methods can be improved over the
performance obtained by the default implementations in this class.
Copyright © IBM Corporation 1999. All rights reserved.
UNKNOWN: ICU 2.0
Nested Class Summary | |
---|---|
static interface | Transliterator.Factory
The factory interface for transliterators. |
static class | Transliterator.Position
Position structure for incremental transliteration. |
Field Summary | |
---|---|
static int | FORWARD
Direction constant indicating the forward direction in a transliterator,
e.g., the forward rules of a RuleBasedTransliterator. |
static int | REVERSE
Direction constant indicating the reverse direction in a transliterator,
e.g., the reverse rules of a RuleBasedTransliterator. |
Constructor Summary | |
---|---|
protected | Transliterator(String ID, UnicodeFilter filter)
Default constructor. |
Method Summary | |
---|---|
protected String | baseToRules(boolean escapeUnprintable)
Returns a rule string for this transliterator. |
static Transliterator | createFromRules(String ID, String rules, int dir)
Returns a Transliterator object constructed from
the given rule string. |
void | filteredTransliterate(Replaceable text, Transliterator.Position index, boolean incremental)
Transliterate a substring of text, as specified by index, taking filters
into account. |
void | finishTransliteration(Replaceable text, Transliterator.Position index)
Finishes any pending transliterations that were waiting for
more characters. |
static Enumeration | getAvailableIDs()
Returns an enumeration over the programmatic names of registered
Transliterator objects. |
static Enumeration | getAvailableSources()
Returns an enumeration over the source names of registered
transliterators. |
static Enumeration | getAvailableTargets(String source)
Returns an enumeration over the target names of registered
transliterators having a given source name. |
static Enumeration | getAvailableVariants(String source, String target)
Returns an enumeration over the variant names of registered
transliterators having a given source name and target name. |
static String | getDisplayName(String ID)
Returns a name for this transliterator that is appropriate for
display to the user in the default locale. |
static String | getDisplayName(String id, Locale inLocale)
Returns a name for this transliterator that is appropriate for
display to the user in the given locale. |
static String | getDisplayName(String id, ULocale inLocale)
Returns a name for this transliterator that is appropriate for
display to the user in the given locale. |
Transliterator[] | getElements()
Return the elements that make up this transliterator. |
UnicodeFilter | getFilter()
Returns the filter used by this transliterator, or null
if this transliterator uses no filter. |
String | getID()
Returns a programmatic identifier for this transliterator.
|
static Transliterator | getInstance(String ID)
Returns a Transliterator object given its ID.
|
static Transliterator | getInstance(String ID, int dir)
Returns a Transliterator object given its ID.
|
Transliterator | getInverse()
Returns this transliterator's inverse. |
int | getMaximumContextLength()
Returns the length of the longest context required by this transliterator.
|
UnicodeSet | getSourceSet()
Returns the set of all characters that may be modified in the
input text by this Transliterator. |
UnicodeSet | getTargetSet()
Returns the set of all characters that may be generated as
replacement text by this transliterator. |
protected UnicodeSet | handleGetSourceSet()
Framework method that returns the set of all characters that
may be modified in the input text by this Transliterator,
ignoring the effect of this object's filter. |
protected abstract void | handleTransliterate(Replaceable text, Transliterator.Position pos, boolean incremental)
Abstract method that concrete subclasses define to implement
their transliteration algorithm. |
static void | registerAlias(String aliasID, String realID)
Register an ID as an alias of another ID. |
static void | registerClass(String ID, Class transClass, String displayName)
Registers a subclass of Transliterator with the
system. |
static void | registerFactory(String ID, Transliterator.Factory factory)
Register a factory object with the given ID. |
static void | registerInstance(Transliterator trans)
Register a Transliterator object with the given ID. |
void | setFilter(UnicodeFilter filter)
Changes the filter used by this transliterator. |
protected void | setID(String id)
Set the programmatic identifier for this transliterator. |
protected void | setMaximumContextLength(int a)
Method for subclasses to use to set the maximum context length. |
String | toRules(boolean escapeUnprintable)
Returns a rule string for this transliterator. |
int | transliterate(Replaceable text, int start, int limit)
Transliterates a segment of a string, with optional filtering.
|
void | transliterate(Replaceable text)
Transliterates an entire string in place. |
String | transliterate(String text)
Transliterate an entire string and returns the result. |
void | transliterate(Replaceable text, Transliterator.Position index, String insertion)
Transliterates the portion of the text buffer that can be
transliterated unambiguosly after new text has been inserted,
typically as a result of a keyboard event. |
void | transliterate(Replaceable text, Transliterator.Position index, int insertion)
Transliterates the portion of the text buffer that can be
transliterated unambiguosly after a new character has been
inserted, typically as a result of a keyboard event. |
void | transliterate(Replaceable text, Transliterator.Position index)
Transliterates the portion of the text buffer that can be
transliterated unambiguosly. |
static void | unregister(String ID)
Unregisters a transliterator or class. |
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
Parameters: ID the string identifier for this transliterator filter the filter. Any character for which filter.contains() returns false will not be altered by this transliterator. If filter is null then no filtering is applied.
UNKNOWN: ICU 2.0
Parameters: escapeUnprintable if true, then unprintable characters will be converted to escape form backslash-'u' or backslash-'U'.
UNKNOWN: ICU 2.0
Transliterator
object constructed from
the given rule string. This will be a RuleBasedTransliterator,
if the rule string contains only rules, or a
CompoundTransliterator, if it contains ID blocks, or a
NullTransliterator, if it contains ID blocks which parse as
empty for the given direction.UNKNOWN: ICU 2.0
Parameters: text the text to be transliterated index the position indices incremental if TRUE, then assume more characters may be inserted at index.limit, and postpone processing to accomodate future incoming characters
UNKNOWN: ICU 2.0
transliterate()
.Parameters: text the buffer holding transliterated and untransliterated text. index the array of indices previously passed to {@link #transliterate}
UNKNOWN: ICU 2.0
Transliterator
objects. This includes both system
transliterators and user transliterators registered using
registerClass()
. The enumerated names may be
passed to getInstance()
.
Returns: An Enumeration
over String
objects
See Also: Transliterator Transliterator
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
java.text
package.
If no localized names exist in the system resource bundles,
a name is synthesized using a localized
MessageFormat
pattern from the resource data. The
arguments to this pattern are an integer followed by one or two
strings. The integer is the number of strings, either 1 or 2.
The strings are formed by splitting the ID for this
transliterator at the first '-'. If there is no '-', then the
entire ID forms the only string.
Parameters: inLocale the Locale in which the display name should be localized.
See Also: java.text.MessageFormat
UNKNOWN: ICU 2.0
java.text
package.
If no localized names exist in the system resource bundles,
a name is synthesized using a localized
MessageFormat
pattern from the resource data. The
arguments to this pattern are an integer followed by one or two
strings. The integer is the number of strings, either 1 or 2.
The strings are formed by splitting the ID for this
transliterator at the first '-'. If there is no '-', then the
entire ID forms the only string.
Parameters: inLocale the ULocale in which the display name should be localized.
See Also: java.text.MessageFormat
UNKNOWN: ICU 3.2 This API might change or be removed in a future release.
If this transliterator is not composed of other transliterators, then this method will return an array of length one containing a reference to this transliterator.
Returns: an array of one or more transliterators that make up this transliterator
UNKNOWN: ICU 3.0 This API might change or be removed in a future release.
UNKNOWN: ICU 2.0
getInstance()
, it
will return this object, if it has been registered.See Also: Transliterator Transliterator
UNKNOWN: ICU 2.0
Transliterator
object given its ID.
The ID must be either a system transliterator ID or a ID registered
using registerClass()
.
Parameters: ID a valid ID, as enumerated by getAvailableIDs()
Returns: A Transliterator
object with the given ID
Throws: IllegalArgumentException if the given ID is invalid.
UNKNOWN: ICU 2.0
Transliterator
object given its ID.
The ID must be either a system transliterator ID or a ID registered
using registerClass()
.
Parameters: ID a valid ID, as enumerated by getAvailableIDs()
dir either FORWARD or REVERSE. If REVERSE then the
inverse of the given ID is instantiated.
Returns: A Transliterator
object with the given ID
Throws: IllegalArgumentException if the given ID is invalid.
See Also: Transliterator Transliterator Transliterator
UNKNOWN: ICU 2.0
getID()
returns "A-B", then this method will return the result of
getInstance("B-A")
, or null
if that
call fails.
Subclasses with knowledge of their inverse may wish to override this method.
Returns: a transliterator that is an inverse, not necessarily
exact, of this transliterator, or null
if no such
transliterator is registered.
See Also: Transliterator
UNKNOWN: ICU 2.0
setMaximumContextLength()
.
For example, if a transliterator translates "ddd" (where
d is any digit) to "555" when preceded by "(ddd)", then the preceding
context length is 5, the length of "(ddd)".
Returns: The maximum number of preceding context characters this transliterator needs to examine
UNKNOWN: ICU 2.0
See Also: Transliterator Transliterator
UNKNOWN: ICU 2.2
See Also: Transliterator
UNKNOWN: ICU 2.2
Returns: the set of characters that this transliterator may modify. The set may be modified, so subclasses should return a newly-created object.
See Also: Transliterator Transliterator
UNKNOWN: ICU 2.2
originalStart
refer to the value of
pos.start
upon entry.
incremental
is false, then this method
should transliterate all characters between
pos.start
and pos.limit
. Upon return
pos.start
must == pos.limit
.incremental
is true, then this method
should transliterate all characters between
pos.start
and pos.limit
that can be
unambiguously transliterated, regardless of future insertions
of text at pos.limit
. Upon return,
pos.start
should be in the range
[originalStart
, pos.limit
).
pos.start
should be positioned such that
characters [originalStart
,
pos.start
) will not be changed in the future by this
transliterator and characters [pos.start
,
pos.limit
) are unchanged.Implementations of this method should also obey the following invariants:
pos.limit
and pos.contextLimit
should be updated to reflect changes in length of the text
between pos.start
and pos.limit
. The
difference pos.contextLimit - pos.limit
should
not change.pos.contextStart
should not change.pos.start
nor
pos.limit
should be less than
originalStart
.originalStart
and text after
pos.limit
should not change.pos.contextStart
and text after
pos.contextLimit
should be ignored.Subclasses may safely assume that all characters in
[pos.start
, pos.limit
) are filtered.
In other words, the filter has already been applied by the time
this method is called. See
filteredTransliterate()
.
This method is not for public consumption. Calling
this method directly will transliterate
[pos.start
, pos.limit
) without
applying the filter. End user code should call
transliterate()
instead of this method. Subclass code
should call filteredTransliterate()
instead of
this method.
Parameters: text the buffer holding transliterated and
untransliterated text
pos the indices indicating the start, limit, context
start, and context limit of the text.
incremental if true, assume more text may be inserted at
pos.limit
and act accordingly. Otherwise,
transliterate all text between pos.start
and
pos.limit
and move pos.start
up to
pos.limit
.
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: aliasID The new ID being registered. realID The existing ID that the new ID should be an alias of.
UNKNOWN: ICU 3.4.1 This API might change or be removed in a future release.
Transliterator
with the
system. This subclass must have a public constructor taking no
arguments. When that constructor is called, the resulting
object must return the ID
passed to this method if
its getID()
method is called.
Parameters: ID the result of getID()
for this
transliterator transClass a subclass of Transliterator
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: ID the ID of this transliterator factory the factory object
UNKNOWN: ICU 2.0
Parameters: trans the Transliterator object
UNKNOWN: ICU 2.2
Callers must take care if a transliterator is in use by multiple threads. The filter should not be changed by one thread while another thread may be transliterating.
UNKNOWN: ICU 2.0
UNKNOWN: ICU 2.0
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: escapeUnprintable if true, then unprintable characters will be converted to escape form backslash-'u' or backslash-'U'.
UNKNOWN: ICU 2.0
Parameters: text the string to be transliterated start the beginning index, inclusive; 0 <= start
<= limit
. limit the ending index, exclusive; start <= limit
<= text.length()
.
Returns: The new limit index. The text previously occupying [start,
limit)
has been transliterated, possibly to a string of a different
length, at [start,
new-limit)
, where
new-limit is the return value. If the input offsets are out of bounds,
the returned value is -1 and the input string remains unchanged.
UNKNOWN: ICU 2.0
Parameters: text the string to be transliterated
UNKNOWN: ICU 2.0
Parameters: text the string to be transliterated
Returns: The transliterated text
UNKNOWN: ICU 2.0
insertion
will be inserted into text
at index.contextLimit
, advancing
index.contextLimit
by insertion.length()
.
Then the transliterator will try to transliterate characters of
text
between index.start
and
index.contextLimit
. Characters before
index.start
will not be changed.
Upon return, values in index
will be updated.
index.contextStart
will be advanced to the first
character that future calls to this method will read.
index.start
and index.contextLimit
will
be adjusted to delimit the range of text that future calls to
this method may change.
Typical usage of this method begins with an initial call
with index.contextStart
and index.contextLimit
set to indicate the portion of text
to be
transliterated, and index.start == index.contextStart
.
Thereafter, index
can be used without
modification in future calls, provided that all changes to
text
are made via this method.
This method assumes that future calls may be made that will insert new text into the buffer. As a result, it only performs unambiguous transliterations. After the last call to this method, there may be untransliterated text that is waiting for more input to resolve an ambiguity. In order to perform these pending transliterations, clients should call {@link #finishTransliteration} after the last call to this method has been made.
Parameters: text the buffer holding transliterated and untransliterated text index the start and limit of the text, the position
of the cursor, and the start and limit of transliteration. insertion text to be inserted and possibly
transliterated into the translation buffer at
index.contextLimit
. If null
then no text
is inserted.
Throws: IllegalArgumentException if index
is invalid
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: text the buffer holding transliterated and
untransliterated text index the start and limit of the text, the position
of the cursor, and the start and limit of transliteration. insertion text to be inserted and possibly
transliterated into the translation buffer at
index.contextLimit
.
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: text the buffer holding transliterated and untransliterated text index the start and limit of the text, the position of the cursor, and the start and limit of transliteration.
See Also: Transliterator
UNKNOWN: ICU 2.0
Parameters: ID the ID of the transliterator or class
See Also: Transliterator
UNKNOWN: ICU 2.0