Stream manipulation

Note

Certain constructs are available only for seekable or tellable streams (in-memory and files). Sockets and pipes do not support seeking, so you’ll have to first read the data from the stream and parse it in-memory, or use experimental Rebuffered() wrapper.

Field wrappers

Pointer allows for non-sequential construction. The pointer first changes the stream position, does the construction, and restores the original stream position.

class construct.core.Pointer(offset, subcon)

Jumps in the stream forth and back for one field.

Parsing and building seeks the stream to new location, processes subcon, and seeks back to original location. Size is not defined.

Offset can be positive, indicating a position from stream beginning forward, or negative, indicating a position from EOF backwards.

Parameters:
  • offset – integer or context lambda, positive or negative
  • subcon – Construct instance
Raises:
  • StreamError – requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes
  • StreamError – stream is not seekable and tellable

Can propagate any exception from the lambda, possibly non-ConstructError.

Example:

>>> d = Pointer(8, Bytes(1))
>>> d.parse(b"abcdefghijkl")
b'i'
>>> d.build(b"Z")
b'\x00\x00\x00\x00\x00\x00\x00\x00Z'

Peek parses the subconstruct but restores the stream position afterwards (it does peeking). Building does nothing, it does NOT defer to subcon when building.

class construct.core.Peek(subcon)

Peeks at the stream.

Parsing sub-parses (and returns None if failed), then reverts stream to original position. Building does nothing (its NOT deferred). Size is defined as 0 because there is no building.

This class is used in Union class to parse each member.

Parameters:

subcon – Construct instance

Raises:
  • StreamError – requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes
  • StreamError – stream is not seekable and tellable

Example:

>>> d = Sequence(Peek(Int8ub), Peek(Int16ub))
>>> d.parse(b"\x01\x02")
[1, 258]
>>> d.sizeof()
0

Pure side effects

Seek makes a jump within the stream and leaves it at that point. It does not read or write anything to the stream by itself.

construct.core.Seek(at, whence=0)

Seeks the stream.

Parsing and building seek the stream to given location (and whence), and return stream.seek() return value. Size is not defined.

See also

Analog Pointer wrapper that has same side effect but also processes a subcon, and also seeks back.

Parameters:
  • at – integer or context lambda, where to jump to
  • whence – optional, integer or context lambda, is the offset from beginning (0) or from current position (1) or from EOF (2), default is 0
Raises:

StreamError – stream is not seekable

Can propagate any exception from the lambda, possibly non-ConstructError.

Example:

>>> d = (Seek(5) >> Byte)
>>> d.parse(b"01234x")
[5, 120]

>>> d = (Bytes(10) >> Seek(5) >> Byte)
>>> d.build([b"0123456789", None, 255])
b'01234\xff6789'

Tell checks the current stream position and returns it, also putting it into the context. It does not read or write anything to the stream by itself.

construct.core.Tell()

Tells the stream.

Parsing and building return current stream offset using using stream.tell(). Size is defined as 0 because parsing and building does not consume or add into the stream.

Tell is useful for adjusting relative offsets to absolute positions, or to measure sizes of Constructs. To get an absolute pointer, use a Tell plus a relative offset. To get a size, place two Tells and measure their difference using a Compute field. However, its recommended to use RawCopy instead of manually extracting two positions and computing difference.

Raises:StreamError – stream is not tellable

Example:

>>> d = Struct("num"/VarInt, "offset"/Tell)
>>> d.parse(b"X")
Container(num=88)(offset=1)
>>> d.build(dict(num=88))
b'X'
construct.core.Pass()

No-op construct, useful as default cases for Switch and Enum.

Parsing returns None. Building does nothing. Size is 0 by definition.

Example:

>>> Pass.parse(b"")
None
>>> Pass.build(None)
b''
>>> Pass.sizeof()
0
construct.core.Terminated()

Asserts end of stream (EOF). You can use it to ensure no more unparsed data follows in the stream.

Parsing checks if stream reached EOF, and raises TerminatedError if not. Building does nothing. Size is defined as 0 because parsing and building does not consume or add into the stream.

Raises:TerminatedError – stream not at EOF when parsing

Example:

>>> Terminated.parse(b"")
None
>>> Terminated.parse(b"remaining")
construct.core.TerminatedError: expected end of stream

Stream wrappers

class construct.core.Restreamed(subcon, decoder, decoderunit, encoder, encoderunit, sizecomputer)

Transforms bytes between the underlying stream and the subcon.

Used internally to implement Bitwise Bytewise ByteSwapped BitsSwapped .

Warning

Remember that subcon must consume or produce an amount of bytes that is a multiple of encoding or decoding units. For example, in a Bitwise context you should process a multiple of 8 bits or the stream will fail during parsing/building.

Warning

Do NOT use seeking/telling classes inside Restreamed context.

Parameters:
  • subcon – Construct instance, subcon which will operate on the buffer
  • encoder – function that takes bytes and returns bytes (used when building)
  • encoderunit – integer ratio, encoder takes that many bytes at once
  • decoder – function that takes bytes and returns bytes (used when parsing)
  • decoderunit – integer ratio, decoder takes that many bytes at once
  • sizecomputer – function that computes amount of bytes outputed

Can propagate any exception from the lambda, possibly non-ConstructError.

Can also raise arbitrary exceptions in its implementation.

Example:

Bitwise  <--> Restreamed(subcon, bits2bytes, 8, bytes2bits, 1, lambda n: n//8)
Bytewise <--> Restreamed(subcon, bytes2bits, 1, bits2bytes, 8, lambda n: n*8)