class documentation

class NetstringReceiver(protocol.Protocol): (source)

View In Hierarchy

A protocol that sends and receives netstrings.

See http://cr.yp.to/proto/netstrings.txt for the specification of netstrings. Every netstring starts with digits that specify the length of the data. This length specification is separated from the data by a colon. The data is terminated with a comma.

Override stringReceived to handle received netstrings. This method is called with the netstring payload as a single argument whenever a complete netstring is received.

Security features:

  1. Messages are limited in size, useful if you don't want someone sending you a 500MB netstring (change self.MAX_LENGTH to the maximum length you wish to accept).
  2. The connection is lost if an illegal message is received.
Method dataReceived Receives some characters of a netstring.
Method makeConnection Initializes the protocol.
Method sendString Sends a netstring.
Method stringReceived Override this for notification when each complete string is received.
Constant MAX_LENGTH Defines the maximum length of netstrings that can be received.
Instance Variable brokenPeer Indicates if the connection is still functional
Method _checkForTrailingComma Checks if the netstring has a trailing comma at the expected position.
Method _checkPartialLengthSpecification Makes sure that the received data represents a valid number.
Method _checkStringSize Checks the sanity of lengthAsString.
Method _consumeData Consumes the content of self._remainingData.
Method _consumeLength Consumes the length portion of self._remainingData.
Method _consumePayload Consumes the payload portion of self._remainingData.
Method _extractLength Attempts to extract the length information of a netstring.
Method _extractPayload Extracts payload information from self._remainingData.
Method _handleParseError Terminates the connection and sets the flag self.brokenPeer.
Method _maxLengthSize Calculate and return the string size of self.MAX_LENGTH.
Method _payloadComplete Checks if enough data have been received to complete the netstring.
Method _prepareForPayloadConsumption Sets up variables necessary for consuming the payload of a netstring.
Method _processLength Processes the length definition of a netstring.
Method _processPayload Processes the actual payload with stringReceived.
Constant _LENGTH A pattern describing all strings that contain a netstring length specification. Examples for length specifications are b'0:', b'12:', and b'179:'. b'007:' is not a valid length specification, since leading zeros are not allowed.
Constant _LENGTH_PREFIX A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed.
Constant _MISSING_COMMA Undocumented
Constant _MISSING_LENGTH Undocumented
Constant _OVERFLOW Undocumented
Constant _TOO_LONG Undocumented
Instance Variable _currentPayloadSize Undocumented
Instance Variable _expectedPayloadSize Holds the payload size plus one for the trailing comma.
Instance Variable _PARSING_LENGTH Indicates that the NetstringReceiver is in the state of parsing the length portion of a netstring.
Instance Variable _PARSING_PAYLOAD Indicates that the NetstringReceiver is in the state of parsing the payload portion (data and trailing comma) of a netstring.
Instance Variable _payload Holds the payload portion of a netstring including the trailing comma
Instance Variable _remainingData Holds the chunk of data that has not yet been consumed
Instance Variable _state Indicates if the protocol is consuming the length portion (PARSING_LENGTH) or the payload (PARSING_PAYLOAD) of a netstring

Inherited from Protocol:

Method connectionLost Called when the connection is shut down.
Method logPrefix Return a prefix matching the class name, to identify log messages related to this protocol instance.
Class Variable factory Undocumented

Inherited from BaseProtocol (via Protocol):

Method connectionMade Called when a connection is made.
Instance Variable connected Undocumented
Instance Variable transport Undocumented
def dataReceived(self, data): (source)

Receives some characters of a netstring.

Whenever a complete netstring is received, this method extracts its payload and calls stringReceived to process it.

Parameters
data:bytesA chunk of data representing a (possibly partial) netstring
def makeConnection(self, transport): (source)
def sendString(self, string): (source)

Sends a netstring.

Wraps up string by adding length information and a trailing comma; writes the result to the transport.

Parameters
string:bytesThe string to send. The necessary framing (length prefix, etc) will be added.
def stringReceived(self, string): (source)

Override this for notification when each complete string is received.

Parameters
string:bytesThe complete string which was received with all framing (length prefix, etc) removed.
Raises
NotImplementedErrorbecause the method has to be implemented by the child class.
MAX_LENGTH: int = (source)

Defines the maximum length of netstrings that can be received.

Value
99999
brokenPeer: int = (source)

Indicates if the connection is still functional

def _checkForTrailingComma(self): (source)

Checks if the netstring has a trailing comma at the expected position.

Raises
NetstringParseErrorif the last payload character is anything but a comma.
def _checkPartialLengthSpecification(self): (source)

Makes sure that the received data represents a valid number.

Checks if self._remainingData represents a number smaller or equal to self.MAX_LENGTH.

Raises
NetstringParseErrorif self._remainingData is no number or is too big (checked by _extractLength).
def _checkStringSize(self, lengthAsString): (source)

Checks the sanity of lengthAsString.

Checks if the size of the length specification exceeds the size of the string representing self.MAX_LENGTH. If this is not the case, the number represented by lengthAsString is certainly bigger than self.MAX_LENGTH, and a NetstringParseError can be raised.

This method should make sure that netstrings with extremely long length specifications are refused before even attempting to convert them to an integer (which might trigger a MemoryError).

def _consumeData(self): (source)

Consumes the content of self._remainingData.

Raises
IncompleteNetstringif self._remainingData does not contain enough data to complete the current netstring.
NetstringParseErrorif the received data do not form a valid netstring.
def _consumeLength(self): (source)

Consumes the length portion of self._remainingData.

Raises
IncompleteNetstringif self._remainingData contains a partial length specification (digits without trailing comma).
NetstringParseErrorif the received data do not form a valid netstring.
def _consumePayload(self): (source)

Consumes the payload portion of self._remainingData.

If the payload is complete, checks for the trailing comma and processes the payload. If not, raises an IncompleteNetstring exception.

Raises
IncompleteNetstringif the payload received so far contains fewer characters than expected.
NetstringParseErrorif the payload does not end with a comma.
def _extractLength(self, lengthAsString): (source)

Attempts to extract the length information of a netstring.

Parameters
lengthAsString:bytesA chunk of data starting with a length specification
Returns
intThe length of the netstring
Raises
NetstringParseErrorif the number is bigger than self.MAX_LENGTH.
def _extractPayload(self): (source)

Extracts payload information from self._remainingData.

Splits self._remainingData at the end of the netstring. The first part becomes self._payload, the second part is stored in self._remainingData.

If the netstring is not yet complete, the whole content of self._remainingData is moved to self._payload.

def _handleParseError(self): (source)

Terminates the connection and sets the flag self.brokenPeer.

def _maxLengthSize(self): (source)

Calculate and return the string size of self.MAX_LENGTH.

Returns
floatThe size of the string representation for self.MAX_LENGTH
def _payloadComplete(self): (source)

Checks if enough data have been received to complete the netstring.

Returns
boolTrue iff the received data contain at least as many characters as specified in the length section of the netstring
def _prepareForPayloadConsumption(self): (source)

Sets up variables necessary for consuming the payload of a netstring.

def _processLength(self, lengthMatch): (source)

Processes the length definition of a netstring.

Extracts and stores in self._expectedPayloadSize the number representing the netstring size. Removes the prefix representing the length specification from self._remainingData.

Parameters
lengthMatch:re.MatchA regular expression match object matching a netstring length specification
Raises
NetstringParseErrorif the received netstring does not start with a number or the number is bigger than self.MAX_LENGTH.
def _processPayload(self): (source)

Processes the actual payload with stringReceived.

Strips self._payload of the trailing comma and calls stringReceived with the result.

_LENGTH: re.Match = (source)

A pattern describing all strings that contain a netstring length specification. Examples for length specifications are b'0:', b'12:', and b'179:'. b'007:' is not a valid length specification, since leading zeros are not allowed.

Value
re.compile(rb'(0|[1-9]\d*)(:)')
_LENGTH_PREFIX: re.Match = (source)

A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed.

Value
re.compile(rb'(0|[1-9]\d*)$')
_MISSING_COMMA: str = (source)

Undocumented

Value
'The received netstring is not terminated by a comma.'
_MISSING_LENGTH: str = (source)

Undocumented

Value
'The received netstring does not start with a length specification.'
_OVERFLOW: str = (source)

Undocumented

Value
'The length specification of the received netstring cannot be represented in Pyt
hon - it causes an OverflowError!'
_TOO_LONG: str = (source)

Undocumented

Value
'The received netstring is longer than the maximum %s specified by self.MAX_LENG
TH'
_currentPayloadSize = (source)

Undocumented

_expectedPayloadSize: int = (source)

Holds the payload size plus one for the trailing comma.

_PARSING_LENGTH: int = (source)

Indicates that the NetstringReceiver is in the state of parsing the length portion of a netstring.

_PARSING_PAYLOAD: int = (source)

Indicates that the NetstringReceiver is in the state of parsing the payload portion (data and trailing comma) of a netstring.

_payload: BytesIO = (source)

Holds the payload portion of a netstring including the trailing comma

_remainingData: string = (source)

Holds the chunk of data that has not yet been consumed

_state: int = (source)

Indicates if the protocol is consuming the length portion (PARSING_LENGTH) or the payload (PARSING_PAYLOAD) of a netstring