construct.core – entire module

class construct.core.Adapter(subcon)

Abstract adapter parent class.

Needs to implement _decode() and _encode().

Parameters:subcon – the construct to wrap
class construct.core.Aligned(modulus, subcon, pattern='x00')

Appends additional null bytes to achieve a length that is shortest multiple of a modulus.

Parameters:
  • modulus – the modulus to final length, an int or a context->int function
  • subcon – the subcon to align
  • pattern – optional, the padding pattern (default is x00)

Example:

>>> Aligned(4, Int16ub).build(1)
b'\x00\x01\x00\x00'
>>> Aligned(4, Int16ub).parse(_)
1
>>> Aligned(4, Int16ub).sizeof()
4
construct.core.AlignedStruct(modulus, *subcons, **kw)

Makes a structure where each field is aligned to the same modulus.

See also

Uses Aligned() and ~construct.core.Struct.

Parameters:
  • modulus – passed to each member
  • *subcons – the subcons that make up this structure
  • pattern – optional, keyword parameter passed to each member

Example:

>>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).build(dict(a=1,b=5))
b'\x01\x00\x00\x00\x00\x05\x00\x00'
>>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).parse(_)
Container(a=1)(b=5)
>>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).sizeof()
8
construct.core.Array(count, subcon)

A homogenous array of elements. The array will iterate through exactly count elements. Will raise RangeError if less elements are found.

See also

Base Range() construct.

Parameters:
  • count – int or a function that takes context and returns the number of elements
  • subcon – the subcon to process individual elements

Example:

>>> Byte[5].build(range(5))
b'\x00\x01\x02\x03\x04'
>>> Byte[5].parse(_)
[0, 1, 2, 3, 4]

>>> Array(5, Byte).build(range(5))
b'\x00\x01\x02\x03\x04'
>>> Array(5, Byte).parse(_)
[0, 1, 2, 3, 4]
construct.core.BitStruct(*subcons)

Makes a structure inside a Bitwise.

See also

Uses Bitwise() and Struct().

Parameters:*subcons – the subcons that make up this structure

Example:

>>> BitStruct("field"/Octet).build(dict(field=5))
b'\x05'
>>> BitStruct("field"/Octet).parse(_)
Container(field=5)
>>> BitStruct("field"/Octet).sizeof()
1

>>> format = BitStruct(
...     "a" / Flag,
...     "b" / Nibble,
...     "c" / BitsInteger(10),
...     "d" / Padding(1),
... )
>>> format.parse(b"\xbe\xef")
Container(a=True)(b=7)(c=887)(d=None)
>>> format.sizeof()
2
class construct.core.BitsInteger(length, signed=False, swapped=False, bytesize=8)

A byte field, that parses into and builds from integers as opposed to b-strings. This is similar to Bit/Nibble/Octet fields but can have arbitrary sizes. This must be enclosed in Bitwise.

Parameters:
  • length – number of bits in the field, or a context function that returns int
  • signed – whether the value is signed (two’s complement), default is False (unsigned)
  • swapped – whether to swap byte order (little endian), default is False (big endian)
  • bytesize – size of byte as used for byte swapping (if swapped), default is 8

Example:

>>> Bitwise(BitsInteger(8)).parse(b"\x10")
16
>>> Bitwise(BitsInteger(8)).build(255)
b'\xff'
>>> Bitwise(BitsInteger(8)).sizeof()
1
construct.core.BitsSwapped(subcon)

Swap the bit order within each byte within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

>>>> Bitwise(Bytes(8)).parse(b"\x01")
'\x00\x00\x00\x00\x00\x00\x00\x01'
>>>> BitsSwapped(Bitwise(Bytes(8))).parse(b"\x01")
'\x01\x00\x00\x00\x00\x00\x00\x00'
construct.core.Bitwise(subcon)

Converts the stream from bytes to bits, and passes the bitstream to underlying subcon.

See also

Analog Bytewise() that transforms subset of bits back to bytes.

Warning

Do not use pointers inside.

Parameters:subcon – any field that works with bits like: BitStruct BitsNumber Bit Nibble Octet

Example:

>>> Bitwise(Octet).parse(b"\xff")
255
>>> Bitwise(Octet).build(1)
b'\x01'
>>> Bitwise(Octet).sizeof()
1
construct.core.ByteSwapped(subcon)

Swap the byte order within boundaries of the given subcon.

Parameters:subcon – the subcon on top of byte swapped bytes

Example:

Int24ul <--> ByteSwapped(Int24ub)
class construct.core.Bytes(length)

A field consisting of a specified number of bytes. Builds from a b-string, or an integer (although deprecated and BytesInteger should be used).

See also

Analog BytesInteger() that parses and builds from integers.

Parameters:length – an int or a function that takes context and returns int

Example:

>>> Bytes(4).parse(b"beef")
b'beef'
>>> Bytes(4).build(_)
b'beef'
>>> Bytes(4).build(255)
b'\x00\x00\x00\xff'
>>> Bytes(4).sizeof()
4
class construct.core.BytesInteger(length, signed=False, swapped=False, bytesize=1)

A byte field, that parses into and builds from integers as opposed to b-strings. This is similar to Int* fields but can have arbitrary size.

See also

Analog BitsInteger() that operates on bits.

Parameters:
  • length – number of bytes in the field, or a function that takes context and returns int
  • signed – whether the value is signed (two’s complement), default is False (unsigned)
  • swapped – whether to swap byte order (little endian), default is False (big endian)
  • bytesize – size of byte as used for byte swapping (if swapped), default is 1

Example:

>>> BytesInteger(4).parse(b"abcd")
1633837924
>>> BytesInteger(4).build(1)
b'\x00\x00\x00\x01'
>>> BytesInteger(4).sizeof()
4
construct.core.Bytewise(subcon)

Converts the stream from bits back to bytes. Needs to be used within Bitwise.

Parameters:subcon – any field that works with bytes like: Bytes BytesInteger Int* Struct

Example:

>>> Bitwise(Bytewise(Byte)).parse(b"\xff")
255
>>> Bitwise(Bytewise(Byte)).build(63)
b'?'
>>> Bitwise(Bytewise(Byte)).sizeof()
1
construct.core.CString(terminators='\x00', encoding=None)

A string ending in a terminator b-string character.

CString is similar to the strings of C.

By default, the terminator is the NULL byte (b’x00’). Terminators field can be a longer b-string, and any of the characters breaks parsing. First terminator byte is used when building.

Parameters:
  • terminators – sequence of valid terminators, first is used when building, all are used when parsing
  • encoding – encoding (e.g. “utf8”) or None for bytes

Example:

>>> CString(encoding="utf8").build("Афон")
b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00'
>>> CString(encoding="utf8").parse(_)
'Афон'
class construct.core.Check(func)

Checks for a condition, and raises ValidationError if the check fails.

Example:

Check(lambda ctx: len(ctx.payload.data) == ctx.payload_len)

Check(len_(this.payload.data) == this.payload_len)
class construct.core.Checksum(checksumfield, hashfunc, bytesfunc)

A field that is build or validated by a hash of a given byte range.

Parameters:
  • checksumfield – a subcon field that reads the checksum, usually Bytes(int)
  • hashfunc – a function taking bytes and returning whatever checksumfield takes when building
  • bytesfunc – a function taking context and returning the bytes or object to be hashed, usually this.rawcopy1.data alike

Example:

import hashlib
d = Struct(
    "fields" / RawCopy(Struct(
        "a" / Byte,
        "b" / Byte,
    )),
    "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data),
)
data = d.build(dict(fields=dict(value=dict(a=1,b=2))))
# returned b'\x01\x02\xbd\xd8\x1a\xb23\xbc\xebj\xd23\xcd\x18qP\x93 \xa1\x8d\x035\xa8\x91\xcf\x98s\t\x90\xe8\x92>\x1d\xda\x04\xf35\x8e\x9c~\x1c=\x16\xb1o@\x8c\xfa\xfbj\xf52T\xef0#\xed$6S8\x08\xb6\xca\x993'
class construct.core.Compressed(subcon, encoding, level=None)

Compresses and decompresses underlying stream when processing the subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end. This construct should be used with Prefixed() or entire stream.

Parameters:
  • subcon – the subcon used for storing the value
  • encoding – any of the module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings
  • level – optional, an int between 0..9, lzma discards it

Example:

Compressed(GreedyBytes, "zlib")

Prefixed(VarInt, Compressed(GreedyBytes, "zlib"))
Struct("inner"/above)

Compressed(Struct(...), "zlib")
class construct.core.Computed(func)

A computed value. Underlying byte stream is unaffected. When parsing func(context) provides the value.

Parameters:func – a function that takes context and returns the computed value
Example::
>>> st = Struct(
...     "width" / Byte,
...     "height" / Byte,
...     "total" / Computed(this.width * this.height),
... )
>>> st.parse(b"12")
Container(width=49)(height=50)(total=2450)
>>> st.build(dict(width=4,height=5))
b'\x04\x05'
>>> Computed(lambda ctx: os.urandom(10)).parse(b"")
b'\x98\xc2\xec\x10\x07\xf5\x8e\x98\xc2\xec'
class construct.core.Const(subcon, value=None)

Constant field enforcing a constant value. It is used for file signatures, to validate that the given pattern exists. When parsed, the value must match.

Note that a variable length subcon may still provide positive verification. Const does not consume a precomputed amount of bytes, but depends on the subcon to read the appropriate amount. Consider for example, a field that eats null bytes and returns following byte, then compares to one. When parsing, both b”x00x00x01” and b”x01” will be parsed and checked OK.

Parameters:
  • subcon – the subcon used to build value from, or a b-string value itself
  • value – optional, the expected value
Raises:

ConstError – when parsed data does not match specified value, or building from wrong value

Example:

>>> Const(b"IHDR").build(None)
b'IHDR'
>>> Const(b"IHDR").parse(b"JPEG")
construct.core.ConstError: expected b'IHDR' but parsed b'JPEG'

>>> Const(Int32ul, 16).build(None)
b'\x10\x00\x00\x00'
class construct.core.Construct

The mother of all constructs.

This object is generally not directly instantiated, and it does not directly implement parsing and building, so it is largely only of interest to subclass implementors. There are also other abstract classes.

The external user API:

  • parse()
  • parse_stream()
  • build()
  • build_stream()
  • sizeof()

Subclass authors should not override the external methods. Instead, another API is available:

  • _parse()
  • _build()
  • _sizeof()

And stateful copying:

  • __getstate__()
  • __setstate__()

All constructs have a name and flags. The name is used for naming struct members and context dictionaries. Note that the name can either be a string, or None if the name is not needed. A single underscore (“_”) is a reserved name. The name should be descriptive, short, and valid as a Python identifier, although these rules are not enforced.The flags specify additional behavioral information about this construct. Flags are used by enclosing constructs to determine a proper course of action. Flags are often inherited from inner subconstructs but that depends on each class behavior.

build(obj, context=None, **kw)

Build an object in memory.

Returns:bytes
build_stream(obj, stream, context=None, **kw)

Build an object directly into a stream.

Returns:None
parse(data, context=None, **kw)

Parse an in-memory buffer.

Strings, buffers, memoryviews, and other complete buffers can be parsed with this method.

parse_stream(stream, context=None, **kw)

Parse a stream.

Files, pipes, sockets, and other streaming sources of data are handled by this method.

sizeof(context=None, **kw)

Calculate the size of this object, optionally using a context.

Some constructs have no fixed size and can only know their size for a given hunk of data. These constructs will raise an error if they are not passed a context.

Parameters:context – a container
Returns:int of the length of this construct
Raises:SizeofError – the size could not be determined
class construct.core.Default(subcon, value)

Allows to make a field have a default value, which comes handly when building a Struct from a dict with missing keys.

Example:

>>> Struct("a"/Default(Byte,0)).build(dict(a=1))
b'\x01'
>>> Struct("a"/Default(Byte,0)).build(dict())
b'\x00'
class construct.core.Embedded(subcon)

Embeds a struct into the enclosing struct, merging fields. Can also embed sequences into sequences. Name is also inherited.

Parameters:subcon – the struct to embed

Example:

>>> Struct("a"/Byte, Embedded(Struct("b"/Byte)), "c"/Byte).parse(b"abc")
Container(a=97)(b=98)(c=99)
>>> Struct("a"/Byte, Embedded(Struct("b"/Byte)), "c"/Byte).build(_)
b'abc'
construct.core.EmbeddedBitStruct(*subcons)

Makes an embedded BitStruct.

See also

Uses Bitwise() and Embedded() and Struct().

Parameters:*subcons – the subcons that make up this structure
construct.core.Enum(subcon, default=NotImplemented, **mapping)

A set of named values mapping. Can build both from names and values.

Parameters:
  • subcon – the subcon to map
  • **mapping – keyword arguments which serve as the encoding mapping
  • default – an optional, keyword-only argument that specifies the default value to use when the mapping is undefined. if not given, and exception is raised when the mapping is undefined. use Pass topass the unmapped value as-is

Example:

>>> Enum(Byte,a=1,b=2).parse(b"\x01")
'a'
>>> Enum(Byte,a=1,b=2).parse(b"\x08")
construct.core.MappingError: no decoding mapping for 8

>>> Enum(Byte,a=1,b=2).build("a")
b'\x01'
>>> Enum(Byte,a=1,b=2).build(1)
b'\x01'
class construct.core.ExprAdapter(subcon, encoder, decoder)

A generic adapter that takes encoder and decoder as parameters. You can use ExprAdapter instead of writing a full-blown class when only a simple expression is needed.

Parameters:
  • subcon – the subcon to adapt
  • encoder – a function that takes (obj, context) and returns an encoded version of obj, or None for identity
  • decoder – a function that takes (obj, context) and returns an decoded version of obj, or None for identity

Example:

Ident = ExprAdapter(Byte,
    encoder = lambda obj,ctx: obj+1,
    decoder = lambda obj,ctx: obj-1, )
class construct.core.ExprValidator(subcon, validator)

A generic adapter that takes validator as parameter. You can use ExprValidator instead of writing a full-blown class when only a simple expression is needed.

Parameters:
  • subcon – the subcon to adapt
  • encoder – a function that takes (obj, context) and returns a bool

Example:

OneOf = ExprValidator(Byte,
    validator = lambda obj,ctx: obj in [1,3,5])
construct.core.Filter(predicate, subcon)

Filters a list leaving only the elements that passed through the validator.

Parameters:
  • subcon – a construct to validate, usually a Range or Array or Sequence
  • predicate – a function taking (obj, context) and returning a bool

Example:

>>> Filter(obj_ != 0, Byte[:]).parse(b"\x00\x02\x00")
[2]
>>> Filter(obj_ != 0, Byte[:]).build([0,1,0,2,0])
b'\x01\x02'
class construct.core.FlagsEnum(subcon, **flags)

A set of flag values mapping. Each flag is extracted from the number, resulting in a FlagsContainer dict that has each key assigned True or False.

Parameters:
  • subcon – the subcon to extract
  • **flags – a dictionary mapping flag-names to their value

Example:

>>> FlagsEnum(Byte,a=1,b=2,c=4,d=8).parse(b"\x03")
Container(c=False)(b=True)(a=True)(d=False)
>>> FlagsEnum(Byte,a=1,b=2,c=4,d=8).build(_)
b'\x03'
class construct.core.FocusedSeq(parsebuildfrom, *subcons, **kw)

Parses and builds a sequence where only one subcon value is returned from parsing or taken into building, other fields are parsed and discarded or built from nothing. This is a replacement for SeqOfOne.

Parameters:
  • parsebuildfrom – which subcon to use, an int or str, or a context lambda returning an int or str
  • *subcons – a list of members
  • **kw – a list of members (works ONLY on python 3.6)

Excample:

>>> d = FocusedSeq("num", Const(b"MZ"), "num"/Byte, Terminated)
>>> d = FocusedSeq(1,     Const(b"MZ"), "num"/Byte, Terminated)

>>> d.parse(b"MZ\xff")
255
>>> d.build(255)
b'MZ\xff'
class construct.core.FormatField(endianity, format)

A field that uses struct module to pack and unpack data. This is used to implement basic Int* fields.

See struct documentation for instructions on crafting format strings.

Parameters:
  • endianity – endianness character like: < > =
  • format – format character like: f d B H L Q b h l q

Example:

>>> FormatField(">","H").parse(b"\x01\x00")
256
>>> FormatField(">","H").build(18)
b'\x00\x12'
>>> FormatField(">","H").sizeof()
2
construct.core.GreedyRange(subcon)

A homogenous array of elements that parses until end of stream and builds from all elements.

Parameters:subcon – the subcon to process individual elements

Example:

>>> GreedyRange(Byte).build(range(10))
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'
>>> GreedyRange(Byte).parse(_)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
construct.core.GreedyString(encoding=None)

A string that reads the rest of the stream until EOF, and writes a given string as is. If no encoding is given, this is essentially GreedyBytes.

Parameters:encoding – encoding (e.g. “utf8”) or None for bytes

See also

Analog to GreedyBytes and the same when no enoding is used.

Example:

>>> GreedyString(encoding="utf8").build("Афон")
b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd'
>>> GreedyString(encoding="utf8").parse(_)
'Афон'
construct.core.Hex(subcon)

Adapter for hex-dumping b-strings. It returns a hex dump when parsing, and un-dumps when building.

Example:

>>> Hex(GreedyBytes).parse(b"abcd")
b'61626364'
>>> Hex(GreedyBytes).build("01020304")
b'\x01\x02\x03\x04'
construct.core.HexDump(subcon, linesize=16)

Adapter for hex-dumping b-strings. It returns a hex dump when parsing, and un-dumps when building.

Parameters:
  • linesize – default 16 bytes per line
  • buildraw – by default build takes the same format that parse returns, set to build from a b-string directly

Example:

>>> HexDump(Bytes(10)).parse(b"12345abc;/")
'0000   31 32 33 34 35 61 62 63 3b 2f                     12345abc;/       \n'
construct.core.If(predicate, subcon)

An if-then conditional construct. If the predicate indicates True, the subcon will be used for parsing and building, otherwise parsing returns None and building is no-op.

Parameters:
  • predicate – a function taking context and returning a bool
  • subcon – the subcon that will be used if the predicate returns True

Example:

>>> If(this.x > 0, Byte).build(255, dict(x=1))
b'\xff'
>>> If(this.x > 0, Byte).build(255, dict(x=0))
b''
construct.core.IfThenElse(predicate, thensubcon, elsesubcon)

An if-then-else conditional construct. If the predicate indicates True, thensubcon will be used, otherwise elsesubcon will be used.

Parameters:
  • predicate – a function taking context and returning a bool
  • thensubcon – the subcon that will be used if the predicate indicates True
  • elsesubcon – the subcon that will be used if the predicate indicates False

Example:

>>> IfThenElse(this.x > 0, VarInt, Byte).build(255, dict(x=1))
b'\xff\x01'
>>> IfThenElse(this.x > 0, VarInt, Byte).build(255, dict(x=0))
b'\xff'
class construct.core.Indexing(subcon, count, index, empty=None)

Adapter for indexing a list (getting a single item from that list). Works with Range and Sequence and their lazy equivalents.

Parameters:
  • subcon – the subcon to index
  • count – expected number of elements, needed during building
  • index – the index of the list to get
  • empty – value to fill the list with during building

Example:

???
class construct.core.LazyBound(subconfunc)

A lazy-bound construct that binds to the construct only at runtime. Useful for recursive data structures (like linked lists or trees), where a construct needs to refer to itself (while it doesn’t exist yet).

Parameters:subconfunc – a context function returning a Construct (derived) instance, can also return Pass or itself

Example:

>>> st = Struct(
...     "value"/Byte,
...     "next"/If(this.value > 0, LazyBound(lambda ctx: st)),
... )
...
>>> st.parse(b"\x05\x09\x00")
Container(value=5)(next=Container(value=9)(next=Container(value=0)(next=None)))
...
>>> print(st.parse(b"\x05\x09\x00"))
Container:
    value = 5
    next = Container:
        value = 9
        next = Container:
            value = 0
            next = None
class construct.core.LazyRange(min, max, subcon)

Equivalent to Range construct, but members are parsed on demand. Works only with fixed size subcon.

See also

Equivalent to Range().

class construct.core.LazySequence(*subcons, **kw)

Equivalent to Sequence construct, however fixed size members are parsed on demand, others are parsed immediately. If entire sequence is fixed size then entire parse is essentially one seek.

See also

Equivalent to Sequence().

class construct.core.LazyStruct(*subcons, **kw)

Equivalent to Struct construct, however fixed size members are parsed on demand, others are parsed immediately. If entire struct is fixed size then entire parse is essentially one seek.

See also

Equivalent to Struct().

class construct.core.Mapping(subcon, decoding, encoding, decdefault=NotImplemented, encdefault=NotImplemented)

Adapter that maps objects to other objects. Translates objects before parsing and before

Parameters:
  • subcon – the subcon to map
  • decoding – the decoding (parsing) mapping as a dict
  • encoding – the encoding (building) mapping as a dict
  • decdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if Pass is used, the unmapped object will be passed as-is
  • encdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if Pass is used, the unmapped object will be passed as-is

Example:

???
class construct.core.NamedTuple(tuplename, tuplefields, subcon)

Both arrays, structs and sequences can be mapped to a namedtuple from collections module. To create a named tuple, you need to provide a name and a sequence of fields, either a string with space-separated names or a list of strings. Just like the standard namedtuple does.

Example:

>>> NamedTuple("coord", "x y z", Byte[3]).parse(b"123")
coord(x=49, y=50, z=51)
>>> NamedTuple("coord", "x y z", Byte >> Byte >> Byte).parse(b"123")
coord(x=49, y=50, z=51)
>>> NamedTuple("coord", "x y z", Struct("x"/Byte, "y"/Byte, "z"/Byte)).parse(b"123")
coord(x=49, y=50, z=51)
construct.core.NoneOf(subcon, invalids)

Validates that the object is none of the listed values, both during parsing and building.

Parameters:
  • subcon – a construct to validate
  • invalids – a collection implementing in

See also

Look at OneOf() for examples, works the same.

class construct.core.OnDemand(subcon)

Allows for on-demand (lazy) parsing. When parsing, it will return a parameterless function that when called, will return the parsed value. Object is cached after first parsing, so non-deterministic subcons will be affected. Works only with fixed size subcon.

Parameters:subcon – the subcon to read/write on demand, must be fixed size

Example:

>>> OnDemand(Byte).parse(b"\xff")
<function OnDemand._parse.<locals>.<lambda> at 0x7fdc241cfc80>
>>> _()
255
>>> OnDemand(Byte).build(16)
b'\x10'

Can also re-build from the lambda returned at parsing.

>>> OnDemand(Byte).parse(b"\xff")
<function OnDemand._parse.<locals>.<lambda> at 0x7fcbd9855f28>
>>> OnDemand(Byte).build(_)
b'\xff'
construct.core.OnDemandPointer(offset, subcon)

An on-demand pointer. Is both lazy and jumps to a position before reading.

See also

Base OnDemand() and Pointer() construct.

Parameters:
  • offset – an int or a context function that returns absolute stream position, where the construction would take place, can return negative integer as position from the end backwards
  • subcon – the subcon that will be parsed or built at the offset stream position

Example:

>>> OnDemandPointer(lambda ctx: 2, Byte).parse(b"\x01\x02\x03garbage")
<function OnDemand._parse.<locals>.effectuate at 0x7f6f011ad510>
>>> _()
3
construct.core.OneOf(subcon, valids)

Validates that the object is one of the listed values, both during parsing and building.

Parameters:
  • subcon – a construct to validate
  • valids – a collection implementing in

Example:

>>> OneOf(Byte, [1,2,3]).parse(b"\x01")
1
>>> OneOf(Byte, [1,2,3]).parse(b"\x08")
construct.core.ValidationError: ('invalid object', 8)

>>> OneOf(Bytes(1), b"1234567890").parse(b"4")
b'4'
>>> OneOf(Bytes(1), b"1234567890").parse(b"?")
construct.core.ValidationError: ('invalid object', b'?')

>>> OneOf(Bytes(2), b"1234567890").parse(b"78")
b'78'
>>> OneOf(Bytes(2), b"1234567890").parse(b"19")
construct.core.ValidationError: ('invalid object', b'19')
construct.core.Optional(subcon)

Makes an optional construct, that tries to parse the subcon. If parsing fails, returns None. If building fails, writes nothing.

Note: sizeof returns subcon size, although no bytes could be consumed or produced. Just something to consider.

Parameters:subcon – the subcon to optionally parse or build

Example:

>>> Optional(Int64ul).parse(b"1234")
>>> Optional(Int64ul).parse(b"12345678")
4050765991979987505

>>> Optional(Int64ul).build(1)
b'\x01\x00\x00\x00\x00\x00\x00\x00'
>>> Optional(Int64ul).build("1")
b''
class construct.core.Padded(length, subcon, pattern='x00', strict=False)

Appends additional null bytes to achieve a fixed length.

Example:

>>> Padded(4, Byte).build(255)
b'\xff\x00\x00\x00'
>>> Padded(4, Byte).parse(_)
255
>>> Padded(4, Byte).sizeof()
4

>>> Padded(4, VarInt).build(1)
b'\x01\x00\x00\x00'
>>> Padded(4, VarInt).build(70000)
b'\xf0\xa2\x04\x00'
construct.core.Padding(length, pattern='\x00', strict=False)

A padding field that adds bytes when building, discards bytes when parsing.

Parameters:
  • length – length of the padding, an int or a function taking context and returning an int
  • pattern – padding pattern as b-string character, default is b”x00” null character
  • strict – whether to verify during parsing that the stream contains the pattern, raises an exception if actual padding differs from the pattern, default is False

Example:

>>> (Padding(4) >> Bytes(4)).parse(b"????abcd")
[None, b'abcd']
>>> (Padding(4) >> Bytes(4)).build(_)
b'\x00\x00\x00\x00abcd'
>>> (Padding(4) >> Bytes(4)).sizeof()
8

>>> Padding(4).build(None)
b'\x00\x00\x00\x00'
>>> Padding(4, strict=True).parse(b"****")
construct.core.PaddingError: expected b'\x00\x00\x00\x00', found b'****'
construct.core.PascalString(lengthfield, encoding=None)

A length-prefixed string.

PascalString is named after the string types of Pascal, which are length-prefixed. Lisp strings also follow this convention.

The length field will not appear in the same dict, when parsing. Only the string will be returned. When building, actual length is prepended before the encoded string. The length field can be variable length (such as VarInt). Stored length is in bytes, not characters.

Parameters:
  • lengthfield – a field used to parse and build the length
  • encoding – encoding (eg. “utf8”) or None for bytes

Example:

>>> PascalString(VarInt, encoding="utf8").build("Афон")
b'\x08\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd'
>>> PascalString(VarInt, encoding="utf8").parse(_)
'Афон'
class construct.core.Peek(subcon)

Peeks at the stream. Parses without changing the stream position. If the end of the stream is reached when peeking, returns None. Sizeof returns 0 by design because build does not put anything into the stream. Building is no-op.

See also

The Union() class.

Parameters:subcon – the subcon to peek at

Example:

>>> Sequence(Peek(Byte), Peek(Int16ub)).parse(b"\x01\x02")
[1, 258]
>>> Sequence(Peek(Byte), Peek(Int16ub)).sizeof()
0
class construct.core.Pointer(offset, subcon)

Changes the stream position to a given offset, where the construction should take place, and restores the stream position when finished.

See also

Analog OnDemandPointer() field, which also seeks to a given offset.

Parameters:
  • offset – an int or a function that takes context and returns absolute stream position, where the construction would take place, can return negative integer as position from the end backwards
  • subcon – the subcon to use at the offset

Example:

>>> Pointer(8, Bytes(1)).parse(b"abcdefghijkl")
b'i'
>>> Pointer(8, Bytes(1)).build(b"x")
b'\x00\x00\x00\x00\x00\x00\x00\x00x'
>>> Pointer(8, Bytes(1)).sizeof()
0
class construct.core.Prefixed(lengthfield, subcon, includelength=False)

Parses the length field. Then reads that amount of bytes and parses the subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes. When building, data is prefixed by its length. Optionally, length field can include its own size.

See also

The VarInt encoding should be preferred over Byte and fixed size fields. VarInt is more compact and does never overflow.

Parameters:
  • lengthfield – a subcon used for storing the length
  • subcon – the subcon used for storing the value
  • includelength – optional, whether length field should include own size

Example:

>>> Prefixed(VarInt, GreedyBytes).parse(b"\x05hello?????")
b'hello'

>>>> Prefixed(VarInt, Byte[:]).parse(b"\x03\x01\x02\x03?????")
[1, 2, 3]
construct.core.PrefixedArray(lengthfield, subcon)

An array prefixed by a length field (as opposed to prefixed by byte count, see Prefixed()).

Parameters:
  • lengthfield – field parsing and building an integer
  • subcon – subcon to process individual elements

Example:

>>> PrefixedArray(Byte, Byte).build(range(5))
b'\x05\x00\x01\x02\x03\x04'
>>> PrefixedArray(Byte, Byte).parse(_)
[0, 1, 2, 3, 4]
class construct.core.Range(min, max, subcon)

A homogenous array of elements. The array will iterate through between min to max times. If an exception occurs (EOF, validation error), the repeater exits cleanly. If less than min units have been successfully parsed, a RangeError is raised.

See also

Analog GreedyRange() that parses until end of stream.

Note

This object requires a seekable stream for parsing.

Parameters:
  • min – the minimal count
  • max – the maximal count
  • subcon – the subcon to process individual elements

Example:

>>> Range(3, 5, Byte).build([1,2,3,4])
b'\x01\x02\x03\x04'
>>> Range(3, 5, Byte).parse(_)
[1, 2, 3, 4]

>>> Range(3, 5, Byte).build([1,2])
construct.core.RangeError: expected from 3 to 5 elements, found 2
>>> Range(3, 5, Byte).build([1,2,3,4,5,6])
construct.core.RangeError: expected from 3 to 5 elements, found 6
class construct.core.RawCopy(subcon)

Returns a dict containing both parsed subcon, the raw bytes that were consumed by it, starting and ending offset in the stream, and the amount of bytes. Builds either from raw bytes or a value used by subcon.

Context does contain a dict with data (if built from raw bytes) or with both (if built from value or parsed).

Example:

>>>> RawCopy(Byte).parse(b"\xff")
Container(data='\xff')(value=255)(offset1=0L)(offset2=1L)(length=1L)
...
>>>> RawCopy(Byte).build(dict(data=b"\xff"))
'\xff'
>>>> RawCopy(Byte).build(dict(value=255))
'\xff'
class construct.core.Rebuffered(subcon, tailcutoff=None)

Caches bytes from the underlying stream, so it becomes seekable and tellable. Also makes the stream blocking, in case it came from a socket or a pipe. Optionally, stream can forget bytes that went a certain amount of bytes beyond the current offset, allowing only a limited seeking capability while allowing to process an endless stream.

Warning

Experimental implementation. May not be mature enough.

Parameters:
  • subcon – the subcon which will operate on the buffered stream
  • tailcutoff – optional, amount of bytes kept in buffer, by default buffers everything

Example:

Rebuffered(RepeatUntil(lambda obj,ctx: ?,Byte), tailcutoff=1024).parse_stream(endless_nonblocking_stream)
class construct.core.Rebuild(subcon, func)

Parses the field like normal, but computes the value for building from a function. Useful for length and count fields when Prefixed and PrefixedArray cannot be used.

Example:

>>> st = Struct(
...     "count" / Rebuild(Byte, len_(this.items)),
...     "items" / Byte[this.count],
... )
>>> st.build(dict(items=[1,2,3]))
b'\x03\x01\x02\x03'
class construct.core.Renamed(newname, subcon)

Renames an existing construct. This creates a wrapper so underlying subcon retains it’s original name, which in general means just a None. Can be used to give same construct few different names. Used internally by / operator.

Also this wrapper is responsible for building a path (a chain of names) that gets attached to error message when parsing, building, or sizeof fails. A field that is not named does not appear on the path.

Parameters:
  • newname – the new name
  • subcon – the subcon to rename

Example:

>>> "name" / Int32ul
<Renamed: name>
>>> Renamed("name", Int32ul)
<Renamed: name>
class construct.core.RepeatUntil(predicate, subcon)

An array that repeats until the predicate indicates it to stop. Note that the last element (which caused the repeat to exit) is included in the return value.

Parameters:
  • predicate – a predicate function that takes (obj, list, context) and returns True to break or False to continue
  • subcon – the subcon used to parse and build each element

Example:

>>> RepeatUntil(lambda x,lst,ctx: x>7, Byte).build(range(20))
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08'
>>> RepeatUntil(lambda x,lst,ctx: x>7, Byte).parse(b"\x01\xff\x02")
[1, 255]
>>> RepeatUntil(lambda x,lst,ctx: lst[-2:]==[0,0], Byte).parse(b"\x01\x00\x00\xff")
[1, 0, 0]
class construct.core.Restreamed(subcon, encoder, encoderunit, decoder, decoderunit, sizecomputer)

Transforms bytes between the underlying stream and the subcon.

When the parsing or building is done, the wrapper stream is closed. If read buffer or write buffer is not empty, error is raised.

See also

Both Bitwise() and Bytewise() are implemented using Restreamed.

Warning

Remember that subcon must consume or produce an amount of bytes that is a multiple of encoding or decoding units. For example, in a Bitwise context you should process a multiple of 8 bits or the stream will fail after parsing/building. Also do NOT use pointers inside.

Parameters:
  • subcon – the subcon which will operate on the buffer
  • encoder – a function that takes a b-string and returns a b-string (used when building)
  • encoderunit – ratio as int, encoder takes that many bytes at once
  • decoder – a function that takes a b-string and returns a b-string (used when parsing)
  • decoderunit – ratio as int, decoder takes that many bytes at once
  • sizecomputer – a function that computes amount of bytes outputed by some bytes

Example:

Bitwise  <--> Restreamed(subcon, bits2bytes, 8, bytes2bits, 1, lambda n: n//8)
Bytewise <--> Restreamed(subcon, bytes2bits, 1, bits2bytes, 8, lambda n: n*8)
class construct.core.Seek(at, whence=0)

Sets a new stream position when parsing or building. Seeks are useful when many other fields follow the jump. Pointer works when there is only one field to look at, but when there is more to be done, Seek may come useful.

See also

Analog Pointer() wrapper that has same side effect but also processed a subcon.

Parameters:
  • at – where to jump to, can ne an int or a context lambda
  • whence – is the offset from beginning (0) or from current position (1) or from ending (2), can be an int or a context lambda, default is 0

Example:

>>> (Seek(5) >> Byte).parse(b"01234x")
[5, 120]
>>> (Bytes(10) >> Seek(5) >> Byte).build([b"0123456789", None, 255])
b'01234\xff6789'
class construct.core.Select(*subcons, **kw)

Selects the first matching subconstruct. It will literally try each of the subconstructs, until one matches.

Parameters:
  • subcons – the subcons to try (order sensitive)
  • includename – indicates whether to include the name of the selected subcon in the return value of parsing, default is false

Example:

>>> Select(Int32ub, CString(encoding="utf8")).build(1)
b'\x00\x00\x00\x01'
>>> Select(Int32ub, CString(encoding="utf8")).build("Афон")
b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00'

Note that this syntax works ONLY on python 3.6 due to unordered keyword arguments:
>>> Select(num=Int32ub, text=CString(encoding="utf8"))
class construct.core.Sequence(*subcons, **kw)

A sequence of unnamed constructs. The elements are parsed and built in the order they are defined.

See also

Can be nested easily, and embedded using Embedded() wrapper that merges entries into parent’s entries.

Parameters:subcons – a sequence of subconstructs that make up this sequence

Example:

>>> (Byte >> Byte).build([1, 2])
b'\x01\x02'
>>> (Byte >> Byte).parse(_)
[1, 2]
>>> (Byte >> Byte).sizeof()
2

>>> Sequence(Byte, CString(), Float32b).build([255, b"hello", 123])
b'\xffhello\x00B\xf6\x00\x00'
>>> Sequence(Byte, CString(), Float32b).parse(_)
[255, b'hello', 123.0]
class construct.core.Slicing(subcon, count, start, stop, step=1, empty=None)

Adapter for slicing a list (getting a slice from that list). Works with Range and Sequence and their lazy equivalents.

Parameters:
  • subcon – the subcon to slice
  • count – expected number of elements, needed during building
  • start – start index (or None for entire list)
  • stop – stop index (or None for up-to-end)
  • step – step (or 1 for every element)
  • empty – value to fill the list with during building

Example:

???
class construct.core.StopIf(condfunc)

Checks for a condition, and stops a Struct Sequence Range from parsing or building.

Warning

May break sizeof methods. Unsure.

Example:

Struct('x'/Byte, StopIf(this.x == 0), 'y'/Byte)

Sequence('x'/Byte, StopIf(this.x == 0), 'y'/Byte)

GreedyRange(FocusedSeq(0, 'x'/Byte, StopIf(this.x == 0)))
construct.core.String(length, encoding=None, padchar='\x00', paddir='right', trimdir='right')

A configurable, fixed-length or variable-length string field.

When parsing, the byte string is stripped of pad character (as specified) from the direction (as specified) then decoded (as specified). Length is a constant integer or a function of the context. When building, the string is encoded (as specified) then padded (as specified) from the direction (as specified) or trimmed as bytes (as specified).

The padding character and direction must be specified for padding to work. The trim direction must be specified for trimming to work.

Parameters:
  • length – length in bytes (not unicode characters), as int or context function
  • encoding – encoding (e.g. “utf8”) or None for bytes
  • padchar – b-string character to pad out strings (by default b”x00”)
  • paddir – direction to pad out strings (one of: right left both)
  • trimdir – direction to trim strings (one of: right left)

Example:

>>> String(10).build(b"hello")
b'hello\x00\x00\x00\x00\x00'
>>> String(10).parse(_)
b'hello'
>>> String(10).sizeof()
10

>>> String(10, encoding="utf8").build("Афон")
b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00\x00'
>>> String(10, encoding="utf8").parse(_)
'Афон'

>>> String(10, padchar=b"XYZ", paddir="center").build(b"abc")
b'XXXabcXXXX'
>>> String(10, padchar=b"XYZ", paddir="center").parse(b"XYZabcXYZY")
b'abc'

>>> String(10, trimdir="right").build(b"12345678901234567890")
b'1234567890'
class construct.core.StringEncoded(subcon, encoding)

Used internally.

class construct.core.StringPaddedTrimmed(length, subcon, padchar='x00', paddir='right', trimdir='right')

Used internally.

class construct.core.Struct(*subcons, **kw)

A sequence of usually named constructs, similar to structs in C. The elements are parsed and built in the order they are defined.

Some fields do not need to be named, since they are built from None anyway. See Const Padding Pass Terminated.

See also

Can be nested easily, and embedded using Embedded() wrapper that merges members into parent’s members.

Parameters:subcons – a sequence of subconstructs that make up this structure

Example:

>>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).parse(b"\x01abc")
Container(a=1)(data=b'ab')(data2=b'c')
>>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).build(_)
b'\x01abc'
>>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).build(dict(a=5, data=b"??", data2=b"hello"))
b'\x05??hello'

>>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).build({})
b'MZ\x00\x00'
>>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).parse(_)
Container()
>>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).sizeof()
4

Note that this syntax works ONLY on python 3.6 due to unordered keyword arguments:
>>> Struct(a=Byte, b=Byte, c=Byte, d=Byte)
class construct.core.Subconstruct(subcon)

Abstract subconstruct (wraps an inner construct, inheriting its name and flags). Parsing and building is by default deferred to subcon, so it sizeof.

Subconstructs wrap an inner Construct, inheriting its name and flags.

Parameters:subcon – the construct to wrap
class construct.core.Switch(keyfunc, cases, default=<NoDefault: None>, includekey=False)

A conditional branch. Switch will choose the case to follow based on the return value of keyfunc. If no case is matched, and no default value is given, SwitchError will be raised.

Warning

You can use Embedded(Switch(...)) but not Switch(Embedded(...)). Sames applies to If and IfThenElse macros.

Parameters:
  • keyfunc – a context function that returns a key which will choose a case, or a constant
  • cases – a dictionary mapping keys to subcons
  • default – a default field to use when the key is not found in the cases. if not supplied, an exception will be raised when the key is not found. Pass can be used for do-nothing
  • includekey – whether to include the key in the return value of parsing, defualt is False

Example:

>>> Switch(this.n, { 1:Byte, 2:Int32ub }).build(5, dict(n=1))
b'\x05'
>>> Switch(this.n, { 1:Byte, 2:Int32ub }).build(5, dict(n=2))
b'\x00\x00\x00\x05'
class construct.core.SymmetricAdapter(subcon)

Abstract adapter parent class.

Needs to implement _decode() only. Encoding is done by same method.

Parameters:subcon – the construct to wrap
construct.core.SymmetricMapping(subcon, mapping, default=NotImplemented)

Defines a symmetrical mapping, same mapping is used on parsing and building.

See also

Based on Mapping().

Parameters:
  • subcon – the subcon to map
  • encoding – the mapping as a dict
  • decdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if Pass is used, the unmapped object will be passed as-is

Example:

???
class construct.core.Union(parsefrom, *subcons, **kw)

Treats the same data as multiple constructs (similar to C union statement) so you can “look” at the data in multiple views.

When parsing, all fields read the same data bytes, but stream remains at initial offset if None, unless parsefrom selects a subcon by index or name. When building, the first subcon that can find an entry in the dict (or builds from None, so it does not require an entry) is automatically selected.

Warning

If you skip the parsefrom parameter then stream will be left back at the starting offset. Many users fail to use this class properly.

Parameters:
  • parsefrom – how to leave stream after parsing, can be integer index or string name selecting a subcon, None (leaves stream at initial offset, the default), a context lambda returning either of previously mentioned
  • subcons – subconstructs (order and name sensitive)

Example:

>>> Union(0, "raw"/Bytes(8), "ints"/Int32ub[2], "shorts"/Int16ub[4], "chars"/Byte[8]).parse(b"12345678")
Container(raw=b'12345678')(ints=[825373492, 892745528])(shorts=[12594, 13108, 13622, 14136])(chars=[49, 50, 51, 52, 53, 54, 55, 56])

>>> Union(0, "raw"/Bytes(8), "ints"/Int32ub[2], "shorts"/Int16ub[4], "chars"/Byte[8]).build(dict(chars=range(8)))
b'\x00\x01\x02\x03\x04\x05\x06\x07'

Note that this syntax works ONLY on python 3.6 due to unordered keyword arguments:
>>> Union(0, raw=Bytes(8), ints=Int32ub[2], shorts=Int16ub[4], chars=Byte[8])
>>> Union(0, raw=Bytes(8), ints=Int32ub[2], shorts=Int16ub[4], chars=Byte[8])
class construct.core.Validator(subcon)

Abstract class: validates a condition on the encoded/decoded object.

Needs to implement _validate() that returns bool.

Parameters:subcon – the subcon to validate
construct.core.setglobalstringencoding(encoding)

Sets the encoding globally for all String PascalString CString GreedyString instances.

Parameters:encoding – a string like “utf8” etc or None, which means working with bytes