Core API: Tunneling

construct.RawCopy(subcon)

Returns a dict containing both parsed subcon, the raw bytes that were consumed by it, starting and ending offset in the stream, and the amount of bytes. Builds either from raw bytes or a value used by subcon.

Context does contain a dict with data (if built from raw bytes) or with both (if built from value or parsed).

Example:

>>>> RawCopy(Byte).parse(b"\xff")
Container(data='\xff')(value=255)(offset1=0L)(offset2=1L)(length=1L)
...
>>>> RawCopy(Byte).build(dict(data=b"\xff"))
'\xff'
>>>> RawCopy(Byte).build(dict(value=255))
'\xff'
construct.ByteSwapped(subcon)

Swap the byte order within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

Int24ul <--> ByteSwapped(Int24ub)
construct.BitsSwapped(subcon)

Swap the bit order within each byte within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

>>>> Bitwise(Bytes(8)).parse(b"\x01")
'\x00\x00\x00\x00\x00\x00\x00\x01'
>>>> BitsSwapped(Bitwise(Bytes(8))).parse(b"\x01")
'\x01\x00\x00\x00\x00\x00\x00\x00'
construct.Prefixed(lengthfield, subcon, includelength=False)

Parses the length field. Then reads that amount of bytes and parses the subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes. When building, data is prefixed by its length. Optionally, length field can include its own size.

See also

The VarInt encoding should be preferred over Byte and fixed size fields. VarInt is more compact and does never overflow.

Parameters:
  • lengthfield – a subcon used for storing the length
  • subcon – the subcon used for storing the value
  • includelength – optional, whether length field should include own size

Example:

>>> Prefixed(VarInt, GreedyBytes).parse(b"\x05hello?????")
b'hello'

>>>> Prefixed(VarInt, Byte[:]).parse(b"\x03\x01\x02\x03?????")
[1, 2, 3]
construct.Checksum(checksumfield, hashfunc, bytesfunc)

A field that is build or validated by a hash of a given byte range.

Parameters:
  • checksumfield – a subcon field that reads the checksum, usually Bytes(int)
  • hashfunc – a function taking bytes and returning whatever checksumfield takes when building
  • bytesfunc – a function taking context and returning the bytes or object to be hashed, usually this.rawcopy1.data alike

Example:

import hashlib
d = Struct(
    "fields" / RawCopy(Struct(
        "a" / Byte,
        "b" / Byte,
    )),
    "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data),
)
data = d.build(dict(fields=dict(value=dict(a=1,b=2))))
# returned b'\x01\x02\xbd\xd8\x1a\xb23\xbc\xebj\xd23\xcd\x18qP\x93 \xa1\x8d\x035\xa8\x91\xcf\x98s\t\x90\xe8\x92>\x1d\xda\x04\xf35\x8e\x9c~\x1c=\x16\xb1o@\x8c\xfa\xfbj\xf52T\xef0#\xed$6S8\x08\xb6\xca\x993'
construct.Compressed(subcon, encoding, level=None)

Compresses and decompresses underlying stream when processing the subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end. This construct should be used with Prefixed() or entire stream.

Parameters:
  • subcon – the subcon used for storing the value
  • encoding – any of the module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings
  • level – optional, an int between 0..9, lzma discards it

Example:

Compressed(GreedyBytes, "zlib")

Prefixed(VarInt, Compressed(GreedyBytes, "zlib"))
Struct("inner"/above)

Compressed(Struct(...), "zlib")