Core API: Tunneling

construct.RawCopy(subcon)

Returns a dict containing both parsed subcon, the raw bytes that were consumed by it, starting and ending offset in the stream, and the amount of bytes. Builds either from raw bytes or a value used by subcon.

Context does contain a dict with data (if built from raw bytes) or with both (if built from value) during building.

Example:

>>>> RawCopy(Byte).parse(b"\xff")
Container(data='\xff')(value=255)(offset1=0L)(offset2=1L)(length=1L)
...
>>>> RawCopy(Byte).build(dict(data=b"\xff"))
'\xff'
>>>> RawCopy(Byte).build(dict(value=255))
'\xff'
construct.ByteSwapped(subcon)

Swap the byte order within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

Int24ul <--> ByteSwapped(Int24ub)
construct.BitsSwapped(subcon)

Swap the bit order within each byte within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

>>>> Bitwise(Bytes(8)).parse(b"\x01")
'\x00\x00\x00\x00\x00\x00\x00\x01'
>>>> BitsSwapped(Bitwise(Bytes(8))).parse(b"\x01")
'\x01\x00\x00\x00\x00\x00\x00\x00'
construct.Prefixed(lengthfield, subcon)

Parses the length field. Then reads that amount of bytes and parses the subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes. When building, data is prefixed by its length.

See also

The VarInt encoding should be preferred over Byte and fixed size fields. VarInt is more compact and does never overflow.

Note

If lengthfield is fixed size, Prefixed will seek back to write the length afterwards, which will break on non-seekable streams.

Parameters:
  • lengthfield – a subcon used for storing the length
  • subcon – the subcon used for storing the value

Example:

>>> Prefixed(VarInt, GreedyBytes).parse(b"\x05hello????remainins")
b'hello'

>>>> Prefixed(VarInt, Byte[:]).parse(b"\x03\x01\x02\x03????following")
[1, 2, 3]
construct.Checksum(checksumfield, hashfunc, bytesfunc)

A field that is build or validated by a hash of a given byte range.

Parameters:
  • checksumfield – a subcon field that reads the checksum, usually Bytes(int)
  • hashfunc – a function taking bytes and returning whatever checksumfield takes when building
  • bytesfunc – a function taking context and returning the bytes to be hashed, usually this.rawcopy1.data alike

Example:

import hashlib
d = Struct(
    "fields" / RawCopy(Struct(
        "a" / Byte,
        "b" / Byte,
    )),
    "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data),
)
data = d.build(dict(fields=dict(value=dict(a=1,b=2))))
# returned b'\x01\x02\xbd\xd8\x1a\xb23\xbc\xebj\xd23\xcd\x18qP\x93 \xa1\x8d\x035\xa8\x91\xcf\x98s\t\x90\xe8\x92>\x1d\xda\x04\xf35\x8e\x9c~\x1c=\x16\xb1o@\x8c\xfa\xfbj\xf52T\xef0#\xed$6S8\x08\xb6\xca\x993'
construct.Compressed(subcon, encoding, level=None)

Compresses or decompresses underlying stream when processing the subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end.

See also

This construct should either be used with Prefixed() or on entire stream.

Parameters:
  • subcon – the subcon used for storing the value
  • encoding – any of the module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings
  • level – optinal, an int between 0..9, lzma discards that

Example:

Compressed(GreedyBytes, "zlib")

Prefixed(VarInt, Compressed(GreedyBytes, "zlib"))
Struct("inner"/above)

Compressed(Struct(...), "zlib")