This module implements AMP, the Asynchronous Messaging Protocol.
AMP is a protocol for sending multiple asynchronous request/response pairs over the same connection. Requests and responses are both collections of key/value pairs.
AMP is a very simple protocol which is not an application. This module is a "protocol construction kit" of sorts; it attempts to be the simplest wire-level implementation of Deferreds. AMP provides the following base-level features:
- Asynchronous request/response handling (hence the name)
- Requests and responses are both key/value pairs
- Binary transfer of all data: all data is length-prefixed. Your application will never need to worry about quoting.
- Command dispatching (like HTTP Verbs): the protocol is extensible, and multiple AMP sub-protocols can be grouped together easily.
The protocol implementation also provides a few additional features which are not part of the core wire protocol, but are nevertheless very useful:
- Tight TLS integration, with an included StartTLS command.
- Handshaking to other protocols: because AMP has well-defined message boundaries and maintains all incoming and outgoing requests for you, you can start a connection over AMP and then switch to another protocol. This makes it ideal for firewall-traversal applications where you may have only one forwarded port but multiple applications that want to use it.
Using AMP with Twisted is simple. Each message is a command, with a response. You begin by defining a command type. Commands specify their input and output in terms of the types that they expect to see in the request and response key-value pairs. Here's an example of a command that adds two integers, 'a' and 'b':
class Sum(amp.Command): arguments = [('a', amp.Integer()), ('b', amp.Integer())] response = [('total', amp.Integer())]
Once you have specified a command, you need to make it part of a protocol, and define a responder for it. Here's a 'JustSum' protocol that includes a responder for our 'Sum' command:
class JustSum(amp.AMP): def sum(self, a, b): total = a + b print 'Did a sum: %d + %d = %d' % (a, b, total) return {'total': total} Sum.responder(sum)
Later, when you want to actually do a sum, the following expression will return a Deferred
which will fire with the result:
ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback( lambda p: p.callRemote(Sum, a=13, b=81)).addCallback( lambda result: result['total'])
Command responders may also return Deferreds, causing the response to be sent only once the Deferred fires:
class DelayedSum(amp.AMP): def slowSum(self, a, b): total = a + b result = defer.Deferred() reactor.callLater(3, result.callback, {'total': total}) return result Sum.responder(slowSum)
This is transparent to the caller.
You can also define the propagation of specific errors in AMP. For example, for the slightly more complicated case of division, we might have to deal with division by zero:
class Divide(amp.Command): arguments = [('numerator', amp.Integer()), ('denominator', amp.Integer())] response = [('result', amp.Float())] errors = {ZeroDivisionError: 'ZERO_DIVISION'}
The 'errors' mapping here tells AMP that if a responder to Divide emits a ZeroDivisionError
, then the other side should be informed that an error of the type 'ZERO_DIVISION' has occurred. Writing a responder which takes advantage of this is very simple - just raise your exception normally:
class JustDivide(amp.AMP): def divide(self, numerator, denominator): result = numerator / denominator print 'Divided: %d / %d = %d' % (numerator, denominator, total) return {'result': result} Divide.responder(divide)
On the client side, the errors mapping will be used to determine what the 'ZERO_DIVISION' error means, and translated into an asynchronous exception, which can be handled normally as any Deferred
would be:
def trapZero(result): result.trap(ZeroDivisionError) print "Divided by zero: returning INF" return 1e1000 ClientCreator(reactor, amp.AMP).connectTCP(...).addCallback( lambda p: p.callRemote(Divide, numerator=1234, denominator=0) ).addErrback(trapZero)
For a complete, runnable example of both of these commands, see the files in the Twisted repository:
doc/core/examples/ampserver.py doc/core/examples/ampclient.py
On the wire, AMP is a protocol which uses 2-byte lengths to prefix keys and values, and empty keys to separate messages:
<2-byte length><key><2-byte length><value> <2-byte length><key><2-byte length><value> ... <2-byte length><key><2-byte length><value> <NUL><NUL> # Empty Key == End of Message
And so on. Because it's tedious to refer to lengths and NULs constantly, the documentation will refer to packets as if they were newline delimited, like so:
C: _command: sum C: _ask: ef639e5c892ccb54 C: a: 13 C: b: 81 S: _answer: ef639e5c892ccb54 S: total: 94
Notes:
In general, the order of keys is arbitrary. Specific uses of AMP may impose an ordering requirement, but unless this is specified explicitly, any ordering may be generated and any ordering must be accepted. This applies to the command-related keys _command and _ask as well as any other keys.
Values are limited to the maximum encodable size in a 16-bit length, 65535 bytes.
Keys are limited to the maximum encodable size in a 8-bit length, 255 bytes. Note that we still use 2-byte lengths to encode keys. This small redundancy has several features:
- If an implementation becomes confused and starts emitting corrupt data, or gets keys confused with values, many common errors will be signalled immediately instead of delivering obviously corrupt packets.
- A single NUL will separate every key, and a double NUL separates messages. This provides some redundancy when debugging traffic dumps.
- NULs will be present at regular intervals along the protocol, providing some padding for otherwise braindead C implementations of the protocol, so that <stdio.h> string functions will see the NUL and stop.
- This makes it possible to run an AMP server on a port also used by a plain-text protocol, and easily distinguish between non-AMP clients (like web browsers) which issue non-NUL as the first byte, and AMP clients, which always issue NUL as the first byte.
Interface |
|
An IArgumentType can serialize a Python object into an AMP box and deserialize information from an AMP box back into a Python object. |
Interface |
|
An application object which can receive AmpBox objects and dispatch them appropriately. |
Interface |
|
A transport which can send AmpBox objects. |
Interface |
|
An application object which can look up appropriate responder methods for AMP commands. |
Class | AMP |
This protocol is an AMP connection. See the module docstring for protocol details. |
Class |
|
I am a packet in the AMP protocol, much like a regular bytes:bytes dictionary. |
Class |
|
Convert a list of dictionaries into a list of AMP boxes on the wire. |
Class |
|
Base-class of all objects that take values from Amp packets and convert them into objects for Python functions. |
Class |
|
A protocol for receiving AmpBox es - key/value pairs - via length-prefixed strings. A box is composed of: |
Class |
|
Encode True or False as "True" or "False" on the wire. |
Class |
|
A BoxDispatcher dispatches '_ask', '_answer', and '_error' AmpBox es, both incoming and outgoing, to their appropriate destinations. |
Class |
|
Subclass me to specify an AMP Command. |
Class |
|
A CommandLocator is a collection of responders to AMP Command s, with the help of the Command.responder decorator. |
Class |
|
Encodes datetime.datetime instances. |
Class |
|
Encodes decimal.Decimal instances. |
Class |
|
Encode and decode file descriptors for exchange over a UNIX domain socket. |
Class |
|
Encode floating-point values on the wire as their repr. |
Class |
|
Encode any integer values of any size on the wire as the string representation. |
Class |
|
Encode and decode lists of instances of a single other argument type. |
Class |
|
Encode and decode filepath.FilePath instances as paths on the wire. |
Class |
|
Use this command to switch from something Amp-derived to a different protocol mid-connection. This can be useful to use amp as the connection-startup negotiation phase. Since TLS is a different layer entirely, you can use Amp to negotiate the security parameters of your connection, then switch to a different protocol, and the connection will remain secured. |
Class |
|
I am an AmpBox that, upon being sent, terminates the connection. |
Class |
|
Implement the AMP.locateResponder method to do simple, string-based dispatch. |
Class |
|
Use, or subclass, me to implement a command that starts TLS. |
Class |
|
Don't do any conversion at all; just pass through 'str'. |
Class |
|
Encode a unicode string on the wire as UTF-8. |
Exception |
|
Base class of all Amp-related exceptions. |
Exception |
|
A bad value was returned from a local command; we were unable to coerce it. |
Exception |
|
It was impossible to negotiate a compatible version of the protocol with the other end of the connection. |
Exception |
|
You didn't pass all the required arguments. |
Exception |
|
This error indicates that the wire-level protocol was malformed. |
Exception |
|
You can't have empty boxes on the connection. This is raised when you receive or attempt to send one. |
Exception |
|
This is an implementation limitation; TLS may only be started once per connection. |
Exception |
|
Connections which have been switched to other protocols can no longer accept traffic at the AMP level. This is raised when you try to send it. |
Exception |
|
This error indicates that something went wrong on the remote end of the connection, and the error was serialized and transmitted to you. |
Exception |
|
One of the protocol's length limitations was violated. |
Exception |
|
A command received via amp could not be dispatched. |
Exception |
|
This means that an error whose type we can't identify was raised from the other side. |
Constant | ANSWER |
Marker for an Answer packet. |
Constant | ASK |
Marker for an Ask packet. |
Constant | COMMAND |
Marker for a Command packet. |
Constant | ERROR |
Marker for an AMP box of error type. |
Constant | ERROR |
Marker for an AMP box containing the code of an error. |
Constant | ERROR |
Marker for an AMP box containing the description of the error. |
Constant | MAX |
Undocumented |
Constant | MAX |
The maximum length of a message. |
Constant | PROTOCOL |
Undocumented |
Constant | PYTHON |
Undocumented |
Constant | UNHANDLED |
Undocumented |
Constant | UNKNOWN |
Undocumented |
Class | _ |
This metaclass keeps track of all of the Command.responder-decorated methods defined since the last CommandLocator subclass was defined. It assumes (usually correctly, but unfortunately not necessarily so) that those commands responders were all declared as methods of the class being defined... |
Class | _ |
Metaclass hack to establish reverse-mappings for 'errors' and 'fatalErrors' as class vars. |
Class | _ |
_DescriptorExchanger is a mixin for BinaryBoxProtocol which adds support for receiving file descriptors, a feature offered by IUNIXTransport . |
Class | _ |
Local arguments are never actually relayed across the wire. This is just a shim so that StartTLS can pretend to have some arguments: if arguments acquire documentation properties, replace this with something nicer later. |
Class | _ |
This is for peers which don't want to use a local certificate. Used by AMP because AMP's internal language is all about certificates and this duck-types in the appropriate place; this API isn't really stable though, so it's not exposed anywhere public. |
Class | _ |
A box receiver which records all boxes received. |
Class | _ |
Implementation detail of ProtocolSwitchCommand: I am an AmpBox which sets up state for the protocol to switch. |
Class | _ |
I am an AmpBox that, upon being sent, initiates a TLS connection. |
Function | _objects |
Convert a dictionary of python objects to an AmpBox, converting through a given arglist. |
Function | _strings |
Convert an AmpBox to a dictionary of python objects, converting through a given arglist. |
Function | _wire |
(Private) Normalize an argument name from the wire for use with Python code. If the return value is going to be a python keyword it will be capitalized. If it contains any dashes they will be replaced with underscores. |
Type Variable | _ |
Undocumented |
Type Variable | _ |
Undocumented |
Variable | _log |
Undocumented |
Convert a dictionary of python objects to an AmpBox, converting through a given arglist.
Parameters | |
objects | a dict mapping names to python objects |
arglist | a list of 2-tuples of strings and Argument objects, as described in Command.arguments . |
strings | [OUT PARAMETER] An object providing the dict interface which will be populated with serialized data. |
proto | an AMP instance. |
Returns | |
The converted dictionary mapping names to encoded argument strings (identical to strings). |
Convert an AmpBox to a dictionary of python objects, converting through a given arglist.
Parameters | |
strings | an AmpBox (or dict of strings) |
arglist | a list of 2-tuples of strings and Argument objects, as described in Command.arguments . |
proto | an AMP instance. |
Returns | |
the converted dictionary mapping names to argument objects. |
(Private) Normalize an argument name from the wire for use with Python code. If the return value is going to be a python keyword it will be capitalized. If it contains any dashes they will be replaced with underscores.
The rationale behind this method is that AMP should be an inherently multi-language protocol, so message keys may contain all manner of bizarre bytes. This is not a complete solution; there are still forms of arguments that this implementation will be unable to parse. However, Python identifiers share a huge raft of properties with identifiers from many other languages, so this is a 'good enough' effort for now. We deal explicitly with dashes because that is the most likely departure: Lisps commonly use dashes to separate method names, so protocols initially implemented in a lisp amp dialect may use dashes in argument or command names.
Parameters | |
key:bytes | a bytes, looking something like 'foo-bar-baz' or 'from' |
Returns | |
a native string which is a valid python identifier, looking something like 'foo_bar_baz' or 'From'. |