This document does not describe current Twisted best practice. The progress of this document is currently being tracked in issue 1139.
Goals
This document describes serialized Python configuration objects which can be used for running Twisted applications.
Users of this document should be have a Twisted server application that
they are going to deploy. In general, if you are going to write a mktap
plugin, it is the last stage of developing and deploying a Twisted
application. The majority of users will use the .tac
Python
configuration files without needing to use the mktap serialization.
Therefore, people writing Twisted applications should review the Application howto before this document, and
consider using the .tac
mechanism described there.
What is a Plug-In?
Python makes it very easy to dynamically load and evaluate programs. The
plug-in system for Twisted, twisted.python.plugin
, is
a way to find (without loading) and then load plug-ins for particular
systems.
See The Twisted Plugin system for more
information on using twisted.python.plugin
.
Applications and plugins in Twisted
Application objects
The central concept that a Twisted system administrator will work with are
files that contain Application
instances serialized in various formats
optimized for different uses. .TAP
files are optimized for speed
of loading and saving, .TAX
files are editable by administrators
familiar with XML syntax, and .TAS
files are generated Python
source code, most useful for developers. The two command-line programs which
work with these files are mktap
and twistd
. The mktap
utility
create .TA*
files from simple command-line arguments, and the
twistd
daemon will load and run those files.
Application
objects and related infrastructure are covered in
more depth in the Application howto.
TAP plugins
The most prevalent kind of plug-in is the TAP
(Twisted
Application builder) type. These are relatively simple to get started with.
Let's look at an excerpt from Twisted's own plugins.tml for an example of
registering one:
# ... register("Twisted Web Automated TAP builder", "twisted.tap.web", description=""" Builds a Twisted Application instance that contains a general-purpose web server, which can serve from a filesystem or application resource. """, type="tap", tapname="web") # ...
plugins.tml
will be a list of calls to one function:
register(name, module, type=plugin_type, description=user_description [, **plugin_specific_data])
name
is a free-form string, to be displayed to the user in presentation contexts (like a web page, or a list-box in a GUI).module
is a string which must be the fully-qualified name of a Python module.type
is the name of the system you are plugging in to. Be sure to spell this right, or Twisted won't find your plug-in at all!**plugin_specific_data
is a dictionary of information associated with the plug-in, specific to thetype
of plug-in it is. Note that some plug-in types may require a specific bit of data in order to work.
Note the tapname
parameter given in
the example above. This parameter is an example of **plugin_specific_data
. The parameter tapname
is only used by "tap"
-type modules. It indicates what name to use
on the mktap
command line. In English, this
particular call to register
means When the user
types
.
mktap web
, it selects the module twisted.tap.web
to handle the rest of the arguments
Now that you understand how to register a plug-in, let's move along to writing your first one.
Writing a mktap plugin: the Twisted Quotes example
As an example, we are going to work on a Quote of the Day application,
TwistedQuotes
described in the Designing
Twisted Applications document.
Set up the project directory
See the description of setting up the TwistedQuotes example.
Creating the extension to mktap
is done through
implementing a
module that follows the mktap
plug-in interface,
and then
registering it to be found and loaded by twisted.python.plugin
.
As described above, registration is done by adding a call to
register
in the file
TwistedQuotes/plugins.tml
from twisted.application import internet # services that run TCP/SSL/etc. from TwistedQuotes import quoteproto # Protocol and Factory from TwistedQuotes import quoters # "give me a quote" code from twisted.python import usage # twisted command-line processing class Options(usage.Options): optParameters = [["port", "p", 8007, "Port number to listen on for QOTD protocol."], ["static", "s", "An apple a day keeps the doctor away.", "A static quote to display."], ["file", "f", None, "A fortune-format text file to read quotes from."]] def makeService(config): """Return a service that will be attached to the application.""" if config["file"]: # If I was given a "file" option... # Read quotes from a file, selecting a random one each time, quoter = quoters.FortuneQuoter([config['file']]) else: # otherwise, # read a single quote from the command line (or use the default). quoter = quoters.StaticQuoter(config['static']) port = int(config["port"]) # TCP port to listen on factory = quoteproto.QOTDFactory(quoter) # here we create a QOTDFactory # Finally, set up our factory, with its custom quoter, to create QOTD # protocol instances when events arrive on the specified port. return internet.TCPServer(port, factory)
This module has to conform to a fairly simple interface. It must have a
class called Options
which is a subclass of twisted.python.usage.Options
. It must also have a function
makeService(config)
, which will be
passed an instance of the
Options
class defined in the module itself, TwistedQuotes.quotetap.Options
. Command-line options
given on the mktap
command line fill in the values
in Options
and are used in makeService
to make the actual connections between
objects. makeService
is expected to return an object implementing
IService
. This can
be a Service
subclass,
a MultiService
collection
of sub-services, a TCPServer
serving a protocol factory, and so on.
A more detailed discussion of twisted.python.usage.Options
can be found in the document Using usage.Options
.
Once the TwistedQuotes application design is complete, we pull it them
together by writing a TML file which allows the mktap
utility to find our protocol module.
register("Quote of the Day TAP Builder", "TwistedQuotes.quotetap", description=""" Example of a TAP builder module. """, type="tap", tapname="qotd")
Now the QOTD server is ready to be instantiated! Let's start up a server and get a quote from it.
% mktap qotd Saving qotd application to qotd.tap... Saved. % twistd -f qotd.tap % nc localhost 8007 An apple a day keeps the doctor away. % kill `cat twistd.pid`
Let's walk through the above example. First, we run mktap
specifying the Application type
(qotd
) to create.
mktap
reads in our plugins.tml
file, instantiates an Application
object, fills in the appropriate data, and
serializes it out to a qotd.tap
file. Next, we
launch the server using the twistd daemon, passing qotd.tap
as a command line option. The server launches,
listens on the default port from quotetap.py
. Next,
we run nc
to connect to the running server. In this
step, the QOTDFactory
creates a Quoter
instance, which responds to our network connection
by sending a quote string (in this case, the default quote) over our
connection, and then closes the connection. Finally, we shutdown the server by
killing it via a saved out process id file.
(nc
is the netcat
utility, which no UNIX system should be without.)
After reading this (and following along with your own example, of course), you should be familiar with the process of getting your own Twisted code with unique functionality in it running inside of a server. You should be familiar with the concept of a drop-in and a plug-in, and understand both how to create them and how to install them from other people on your system.
By following the rules set out at the beginning of this HOWTO, we have accidentally implemented another piece of useful functionality.
% mktap Usage: mktap [options] <command> [command options] Options: -x, --xml DEPRECATED: same as --type=xml -s, --source DEPRECATED: same as --type=source -e, --encrypted Encrypt file before writing -p, --progress Show progress of plugin loading -d, --debug Show debug information for plugin loading -u, --uid= [default: 1000] -g, --gid= [default: 1000] -a, --append= An existing .tap file to append the plugin to, rather than creating a new one. -t, --type= The output format to use; this can be 'pickle', 'xml', or 'source'. [default: pickle] --help display this message Commands: ftp An FTP server. im A multi-protocol chat client. inetd mail An email service. manhole An interactive remote debugger service. news News Server portforward A simple port-forwarder. qotd Example of a TAP builder module. socks A SOCKSv4 proxy service. ssh telnet A simple, telnet-based remote debugging service. toc An AIM TOC service. web A general-purpose web server which can serve from a filesystem or application resource. words A chat service.
Not only does our Options
class get
instantiated by mktap
directly, the user can query mktap for interactive
help! This is just one small benefit to using Twisted as it was designed. As
more tools that use the
style of
plug-in, more useful functionality will become available from Twisted Quotes.
For example, a graphical tool could provide not just help messages at the
command line, but a listing of all available TAP types and forms for each, for
the user to enter information.tap