twisted.web.http._ChunkedTransferDecoder

class documentation

class _ChunkedTransferDecoder: (source)

Constructor: _ChunkedTransferDecoder(dataCallback, finishCallback)

View In Hierarchy

Protocol for decoding chunked Transfer-Encoding, as defined by RFC 7230, section 4.1. This protocol can interpret the contents of a request or response body which uses the chunked Transfer-Encoding. It cannot interpret any of the rest of the HTTP protocol.

It may make sense for _ChunkedTransferDecoder to be an actual IProtocol implementation. Currently, the only user of this class will only ever call dataReceived on it. However, it might be an improvement if the user could connect this to a transport and deliver connection lost notification. This way, `dataCallback` becomes `self.transport.write` and perhaps `finishCallback` becomes `self.transport.loseConnection()` (although I'm not sure where the extra data goes in that case). This could also allow this object to indicate to the receiver of data that the stream was not completely received, an error case which should be noticed. -exarkun

Method __init__ Undocumented
Method dataReceived Interpret data from a request or response body which uses the chunked Transfer-Encoding.
Method noMoreData Verify that all data has been received. If it has not been, raise _DataLoss.
Instance Variable dataCallback A one-argument callable which will be invoked each time application data is received. This callback is not reentrant.
Instance Variable finishCallback A one-argument callable which will be invoked when the terminal chunk is received. It will be invoked with all bytes which were delivered to this protocol which came after the terminal chunk.
Instance Variable length Counter keeping track of how many more bytes in a chunk there are to receive.
Instance Variable state One of 'CHUNK_LENGTH', 'CRLF', 'TRAILER', 'BODY', or 'FINISHED'. For 'CHUNK_LENGTH', data for the chunk length line is currently being read. For 'CRLF', the CR LF pair which follows each chunk is being read...
Method _dataReceived_BODY Deliver any available chunk data to the dataCallback. When all the remaining data for the chunk arrives, switch to state 'CRLF'.
Method _dataReceived_CHUNK_LENGTH Read the chunk size line, ignoring any extensions.
Method _dataReceived_CRLF Await the carriage return and line feed characters that are the end of chunk marker that follow the chunk data.
Method _dataReceived_FINISHED Once finishCallback has been invoked receipt of additional data raises RuntimeError because it represents a programming error in the caller.
Method _dataReceived_TRAILER Collect trailer headers if received and finish at the terminal zero-length chunk. Then invoke finishCallback and switch to state 'FINISHED'.
Instance Variable _buffer Accumulated received data for the current state. At each state transition this is truncated at the front so that index 0 is where the next state shall begin.
Instance Variable _maxTrailerHeadersSize Maximum bytes for trailer header from the response.
Instance Variable _receivedTrailerHeadersSize Bytes received so far for the tailer headers.
Instance Variable _start While in the 'CHUNK_LENGTH' and 'TRAILER' states, tracks the index into the buffer at which search for CRLF should resume. Resuming the search at this position avoids doing quadratic work if the chunk length line arrives over many calls to ...
Instance Variable _trailerHeaders Accumulates raw/unparsed trailer headers. See https://github.com/twisted/twisted/issues/12014
def __init__(self, dataCallback: Callable[[bytes], None], finishCallback: Callable[[bytes], None]): (source)

Undocumented

def dataReceived(self, data: bytes): (source)

Interpret data from a request or response body which uses the chunked Transfer-Encoding.

def noMoreData(self): (source)

Verify that all data has been received. If it has not been, raise _DataLoss.

dataCallback = (source)

A one-argument callable which will be invoked each time application data is received. This callback is not reentrant.

finishCallback = (source)

A one-argument callable which will be invoked when the terminal chunk is received. It will be invoked with all bytes which were delivered to this protocol which came after the terminal chunk.

length = (source)

Counter keeping track of how many more bytes in a chunk there are to receive.

state: str = (source)

One of 'CHUNK_LENGTH', 'CRLF', 'TRAILER', 'BODY', or 'FINISHED'. For 'CHUNK_LENGTH', data for the chunk length line is currently being read. For 'CRLF', the CR LF pair which follows each chunk is being read. For 'TRAILER', the CR LF pair which follows the terminal 0-length chunk is currently being read. For 'BODY', the contents of a chunk are being read. For 'FINISHED', the last chunk has been completely read and no more input is valid.

def _dataReceived_BODY(self) -> bool: (source)

Deliver any available chunk data to the dataCallback. When all the remaining data for the chunk arrives, switch to state 'CRLF'.

Returns
boolTrue to continue processing of any buffered data.
def _dataReceived_CHUNK_LENGTH(self) -> bool: (source)

Read the chunk size line, ignoring any extensions.

Returns
boolTrue once the line has been read and removed from self._buffer. False when more data is required.
Raises
_MalformedChunkedDataErrorwhen the chunk size cannot be decoded or the length of the line exceeds maxChunkSizeLineLength.
def _dataReceived_CRLF(self) -> bool: (source)

Await the carriage return and line feed characters that are the end of chunk marker that follow the chunk data.

Returns
boolTrue when the CRLF have been read, otherwise False.
Raises
_MalformedChunkedDataErrorwhen anything other than CRLF are received.
def _dataReceived_FINISHED(self) -> bool: (source)

Once finishCallback has been invoked receipt of additional data raises RuntimeError because it represents a programming error in the caller.

def _dataReceived_TRAILER(self) -> bool: (source)

Collect trailer headers if received and finish at the terminal zero-length chunk. Then invoke finishCallback and switch to state 'FINISHED'.

Returns
boolFalse, as there is either insufficient data to continue, or no data remains.
_buffer = (source)

Accumulated received data for the current state. At each state transition this is truncated at the front so that index 0 is where the next state shall begin.

_maxTrailerHeadersSize: int = (source)

Maximum bytes for trailer header from the response.

_receivedTrailerHeadersSize: int = (source)

Bytes received so far for the tailer headers.

_start: int = (source)

While in the 'CHUNK_LENGTH' and 'TRAILER' states, tracks the index into the buffer at which search for CRLF should resume. Resuming the search at this position avoids doing quadratic work if the chunk length line arrives over many calls to dataReceived.

_trailerHeaders: List[bytearray] = (source)

Accumulates raw/unparsed trailer headers. See https://github.com/twisted/twisted/issues/12014