Core API: Abstract classes

class construct.Construct

The mother of all constructs.

This object is generally not directly instantiated, and it does not directly implement parsing and building, so it is largely only of interest to subclass implementors. There are also other abstract classes sitting on top of this one.

The external user API:

  • parse()
  • parse_stream()
  • build()
  • build_stream()
  • sizeof()
  • compile()
  • benchmark()
  • testcompiled()

Subclass authors should not override the external methods. Instead, another API is available:

  • _parse()
  • _build()
  • _sizeof()
  • _emitdecompiled()
  • _emitparse()
  • _emitbuild()
  • __getstate__()
  • __setstate__()

Attributes and Inheritance:

All constructs have a name and flags. The name is used for naming struct members and context dictionaries. Note that the name can be a string, or None by default. A single underscore “_” is a reserved name, used as up-level in nested containers. The name should be descriptive, short, and valid as a Python identifier, although these rules are not enforced. The flags specify additional behavioral information about this construct. Flags are used by enclosing constructs to determine a proper course of action. Flags are often inherited from inner subconstructs but that depends on each class.

benchmark(sampledata)

Measures performance of your construct (its parsing and building runtime), both for this instance and its compiled equivalent (does not fail if its not compilable). Uses timeit module over 1000 samples.

You need to provide a sample data for parsing testing. This data gets parsed into an object that gets reused for building testing. Sizeof is not tested.

Returns:string containing runtimes and descriptions
build(obj, **kw)

Build an object in memory (a bytes object).

Whenever data cannot be written, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **kw.

Parameters:**kw – context entries, usually empty
Returns:bytes
Raises:ConstructError – raised for any reason
build_stream(obj, stream, **kw)

Build an object directly into a stream.

Whenever data cannot be written, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **kw.

Parameters:**kw – context entries, usually empty
Raises:ConstructError – raised for any reason
compile()

Transforms a construct into another construct that does same thing (has same parsing and building semantics) but is faster (has better performance). Compiled instances compile into itself, obviously. This method returns a Compiled instance.

There are restrictions on what can be compiled (see documentation site, Compilation chapter). Some classes do not compile or compile only in certain circumstances.

Returned instance has additional source field and tofile method, aside of regular parse build sizeof.

Returns:Compiled instance
Raises:NotImplementedError – raised for any reason
parse(data, **kw)

Parse an in-memory buffer (often bytes object). Strings, buffers, memoryviews, and other complete buffers can be parsed with this method.

Whenever data cannot be read, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **kw.

Parameters:**kw – context entries, usually empty
Returns:some value, usually based on bytes read from the stream but sometimes it is computed from nothing or from the context dictionary, sometimes its non-deterministic
Raises:ConstructError – raised for any reason
parse_stream(stream, **kw)

Parse a stream. Files, pipes, sockets, and other streaming sources of data are handled by this method.

Whenever data cannot be read, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **kw.

Parameters:**kw – context entries, usually empty
Returns:some value, usually based on bytes read from the stream but sometimes it is computed from nothing or from the context dictionary, sometimes its non-deterministic
Raises:ConstructError – raised for any reason
sizeof(**kw)

Calculate the size of this object, optionally using a context.

Some constructs have fixed size (like FormatField), some have variable-size and can determine their size given a context entry (like Bytes(this.otherfield1)), and some cannot determine their size (like VarInt).

Whenever size cannot be determined, SizeofError is raised. This method is NOT ALLOWED to raise any other exception, even if eg. context dictionary is missing a key, or subcon propagates ConstructError-derivative exception.

Context entries are passed only as keyword parameters **kw.

Parameters:**kw – context entries, usually empty
Returns:integer if computable, SizeofError otherwise
Raises:SizeofError – size could not be determined in actual context, or is impossible to be determined
testcompiled(sampledata)

Checks correctness of compiled equivalent class by comparing parse and build results of both this and compiled instances.

You need to provide a sample data for parsing testing. This data gets parsed into an object that gets reused for building testing. Sizeof is not tested.

class construct.Subconstruct(subcon)

Abstract subconstruct (wraps an inner construct, inheriting its name and flags). Parsing and building is by default deferred to subcon, same as sizeof.

Parameters:subcon – Construct instance
class construct.Adapter(subcon)

Abstract adapter class.

Needs to implement _decode() for parsing and _encode() for building.

Parameters:subcon – Construct instance
class construct.SymmetricAdapter(subcon)

Abstract adapter class.

Needs to implement _decode() only, for both parsing and building.

Parameters:subcon – Construct instance
class construct.Validator(subcon)

Abstract class that validates a condition on the encoded/decoded object.

Needs to implement _validate() that returns a bool (or a truthy value)

Parameters:subcon – Construct instance
class construct.Tunnel(subcon)

Abstract class that allows other constructs to read part of the stream as if they were reading the entrie stream. See Prefixed for example.

Needs to implement _decode() for parsing and _encode() for building.

class construct.Compiled(source, defersubcon, parsefunc=None, buildfunc=None, sizefunc=None)

Used internally.

tofile(filename)

Saves the source field into a text file (preferably with .py extension).

class construct.CompilableMacro(subcon, emitparsefunc)

Used internally.

class construct.Decompiled(parsefunc)

Used internally.