twisted.protocols.basic.NetstringReceiver

class documentation

class NetstringReceiver(protocol.Protocol): (source)

A protocol that sends and receives netstrings.

See http://cr.yp.to/proto/netstrings.txt for the specification of netstrings. Every netstring starts with digits that specify the length of the data. This length specification is separated from the data by a colon. The data is terminated with a comma.

Override stringReceived to handle received netstrings. This method is called with the netstring payload as a single argument whenever a complete netstring is received.

Security features:

Messages are limited in size, useful if you don't want someone sending you a 500MB netstring (change self.MAX_LENGTH to the maximum length you wish to accept).
The connection is lost if an illegal message is received.

Method	`dataReceived`	Receives some characters of a netstring.
Method	`makeConnection`	Initializes the protocol.
Method	`sendString`	Sends a netstring.
Method	`stringReceived`	Override this for notification when each complete string is received.
Constant	`MAX_LENGTH`	Defines the maximum length of netstrings that can be received.
Instance Variable	`brokenPeer`	Indicates if the connection is still functional
Method	`_checkForTrailingComma`	Checks if the netstring has a trailing comma at the expected position.
Method	`_checkPartialLengthSpecification`	Makes sure that the received data represents a valid number.
Method	`_checkStringSize`	Checks the sanity of lengthAsString.
Method	`_consumeData`	Consumes the content of `self._remainingData`.
Method	`_consumeLength`	Consumes the length portion of `self._remainingData`.
Method	`_consumePayload`	Consumes the payload portion of `self._remainingData`.
Method	`_extractLength`	Attempts to extract the length information of a netstring.
Method	`_extractPayload`	Extracts payload information from `self._remainingData`.
Method	`_handleParseError`	Terminates the connection and sets the flag `self.brokenPeer`.
Method	`_maxLengthSize`	Calculate and return the string size of `self.MAX_LENGTH`.
Method	`_payloadComplete`	Checks if enough data have been received to complete the netstring.
Method	`_prepareForPayloadConsumption`	Sets up variables necessary for consuming the payload of a netstring.
Method	`_processLength`	Processes the length definition of a netstring.
Method	`_processPayload`	Processes the actual payload with `stringReceived`.
Constant	`_LENGTH`	A pattern describing all strings that contain a netstring length specification. Examples for length specifications are `b'0:'`, `b'12:'`, and `b'179:'`. `b'007:'` is not a valid length specification, since leading zeros are not allowed.
Constant	`_LENGTH_PREFIX`	A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed.
Constant	`_MISSING_COMMA`	Undocumented
Constant	`_MISSING_LENGTH`	Undocumented
Constant	`_OVERFLOW`	Undocumented
Constant	`_TOO_LONG`	Undocumented
Instance Variable	`_currentPayloadSize`	Undocumented
Instance Variable	`_expectedPayloadSize`	Holds the payload size plus one for the trailing comma.
Instance Variable	`_PARSING_LENGTH`	Indicates that the `NetstringReceiver` is in the state of parsing the length portion of a netstring.
Instance Variable	`_PARSING_PAYLOAD`	Indicates that the `NetstringReceiver` is in the state of parsing the payload portion (data and trailing comma) of a netstring.
Instance Variable	`_payload`	Holds the payload portion of a netstring including the trailing comma
Instance Variable	`_remainingData`	Holds the chunk of data that has not yet been consumed
Instance Variable	`_state`	Indicates if the protocol is consuming the length portion (`PARSING_LENGTH`) or the payload (`PARSING_PAYLOAD`) of a netstring

Inherited from Protocol:

Method	`connectionLost`	Called when the connection is shut down.
Method	`logPrefix`	Return a prefix matching the class name, to identify log messages related to this protocol instance.
Class Variable	`factory`	Undocumented

Inherited from BaseProtocol (via Protocol):

Method	`connectionMade`	Called when a connection is made.
Instance Variable	`connected`	Undocumented
Instance Variable	`transport`	Undocumented

def dataReceived(self, data): (source) ¶

from twisted.internet.interfaces.IProtocol

overrides twisted.internet.protocol.Protocol.dataReceived

Receives some characters of a netstring.

Whenever a complete netstring is received, this method extracts its payload and calls stringReceived to process it.

Parameters
data:`bytes`	A chunk of data representing a (possibly partial) netstring

def makeConnection(self, transport): (source) ¶

from twisted.internet.interfaces.IProtocol

overrides twisted.internet.protocol.BaseProtocol.makeConnection

Initializes the protocol.

def sendString(self, string): (source) ¶

Sends a netstring.

Wraps up string by adding length information and a trailing comma; writes the result to the transport.

Parameters
string:`bytes`	The string to send. The necessary framing (length prefix, etc) will be added.

def stringReceived(self, string): (source) ¶

Override this for notification when each complete string is received.

Parameters
string:`bytes`	The complete string which was received with all framing (length prefix, etc) removed.
Raises
`NotImplementedError`	because the method has to be implemented by the child class.

MAX_LENGTH: int = (source) ¶

Defines the maximum length of netstrings that can be received.

Value

brokenPeer: int = (source) ¶

Indicates if the connection is still functional

def _checkForTrailingComma(self): (source) ¶

Checks if the netstring has a trailing comma at the expected position.

Raises
`NetstringParseError`	if the last payload character is anything but a comma.

def _checkPartialLengthSpecification(self): (source) ¶

Makes sure that the received data represents a valid number.

Checks if self._remainingData represents a number smaller or equal to self.MAX_LENGTH.

Raises
`NetstringParseError`	if `self._remainingData` is no number or is too big (checked by `_extractLength`).

def _checkStringSize(self, lengthAsString): (source) ¶

Checks the sanity of lengthAsString.

Checks if the size of the length specification exceeds the size of the string representing self.MAX_LENGTH. If this is not the case, the number represented by lengthAsString is certainly bigger than self.MAX_LENGTH, and a NetstringParseError can be raised.

This method should make sure that netstrings with extremely long length specifications are refused before even attempting to convert them to an integer (which might trigger a MemoryError).

def _consumeData(self): (source) ¶

Consumes the content of self._remainingData.

Raises
`IncompleteNetstring`	if `self._remainingData` does not contain enough data to complete the current netstring.
`NetstringParseError`	if the received data do not form a valid netstring.

def _consumeLength(self): (source) ¶

Consumes the length portion of self._remainingData.

Raises
`IncompleteNetstring`	if `self._remainingData` contains a partial length specification (digits without trailing comma).
`NetstringParseError`	if the received data do not form a valid netstring.

def _consumePayload(self): (source) ¶

Consumes the payload portion of self._remainingData.

If the payload is complete, checks for the trailing comma and processes the payload. If not, raises an IncompleteNetstring exception.

Raises
`IncompleteNetstring`	if the payload received so far contains fewer characters than expected.
`NetstringParseError`	if the payload does not end with a comma.

def _extractLength(self, lengthAsString): (source) ¶

Attempts to extract the length information of a netstring.

Parameters
lengthAsString:`bytes`	A chunk of data starting with a length specification
Returns
`int`	The length of the netstring
Raises
`NetstringParseError`	if the number is bigger than `self.MAX_LENGTH`.

def _extractPayload(self): (source) ¶

Extracts payload information from self._remainingData.

Splits self._remainingData at the end of the netstring. The first part becomes self._payload, the second part is stored in self._remainingData.

If the netstring is not yet complete, the whole content of self._remainingData is moved to self._payload.

def _handleParseError(self): (source) ¶

Terminates the connection and sets the flag self.brokenPeer.

def _maxLengthSize(self): (source) ¶

Calculate and return the string size of self.MAX_LENGTH.

Returns
`float`	The size of the string representation for `self.MAX_LENGTH`

def _payloadComplete(self): (source) ¶

Checks if enough data have been received to complete the netstring.

Returns
`bool`	`True` iff the received data contain at least as many characters as specified in the length section of the netstring

def _prepareForPayloadConsumption(self): (source) ¶

Sets up variables necessary for consuming the payload of a netstring.

def _processLength(self, lengthMatch): (source) ¶

Processes the length definition of a netstring.

Extracts and stores in self._expectedPayloadSize the number representing the netstring size. Removes the prefix representing the length specification from self._remainingData.

Parameters
lengthMatch:`re.Match`	A regular expression match object matching a netstring length specification
Raises
`NetstringParseError`	if the received netstring does not start with a number or the number is bigger than `self.MAX_LENGTH`.

def _processPayload(self): (source) ¶

Processes the actual payload with stringReceived.

Strips self._payload of the trailing comma and calls stringReceived with the result.

_LENGTH: re.Match = (source) ¶

A pattern describing all strings that contain a netstring length specification. Examples for length specifications are b'0:', b'12:', and b'179:'. b'007:' is not a valid length specification, since leading zeros are not allowed.

Value

re.compile(rb'(0|[1-9]\d*)(:)')

_LENGTH_PREFIX: re.Match = (source) ¶

A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed.

Value

re.compile(rb'(0|[1-9]\d*)$')

_MISSING_COMMA: str = (source) ¶

Undocumented

Value

'The received netstring is not terminated by a comma.'

_MISSING_LENGTH: str = (source) ¶

Undocumented

Value

'The received netstring does not start with a length specification.'

_OVERFLOW: str = (source) ¶

Undocumented

Value

'The length specification of the received netstring cannot be represented in Pyt↵
hon - it causes an OverflowError!'

_TOO_LONG: str = (source) ¶

Undocumented

Value

'The received netstring is longer than the maximum %s specified by self.MAX_LENG↵
TH'

_currentPayloadSize = (source) ¶

Undocumented

_expectedPayloadSize: int = (source) ¶

Holds the payload size plus one for the trailing comma.

_PARSING_LENGTH: int = (source) ¶

Indicates that the NetstringReceiver is in the state of parsing the length portion of a netstring.

_PARSING_PAYLOAD: int = (source) ¶

Indicates that the NetstringReceiver is in the state of parsing the payload portion (data and trailing comma) of a netstring.

_payload: BytesIO = (source) ¶

Holds the payload portion of a netstring including the trailing comma

_remainingData: string = (source) ¶

Holds the chunk of data that has not yet been consumed

_state: int = (source) ¶

Indicates if the protocol is consuming the length portion (PARSING_LENGTH) or the payload (PARSING_PAYLOAD) of a netstring