Core API: Abstract classes

class construct.Construct

The mother of all constructs.

This object is generally not directly instantiated, and it does not directly implement parsing and building, so it is largely only of interest to subclass implementors. There are also other abstract classes sitting on top of this one.

The external user API:

  • parse
  • parse_stream
  • parse_file
  • build
  • build_stream
  • build_file
  • sizeof
  • compile
  • benchmark

Subclass authors should not override the external methods. Instead, another API is available:

  • _parse
  • _build
  • _sizeof
  • _actualsize
  • _emitparse
  • _emitbuild
  • __getstate__
  • __setstate__

Attributes and Inheritance:

All constructs have a name and flags. The name is used for naming struct members and context dictionaries. Note that the name can be a string, or None by default. A single underscore “_” is a reserved name, used as up-level in nested containers. The name should be descriptive, short, and valid as a Python identifier, although these rules are not enforced. The flags specify additional behavioral information about this construct. Flags are used by enclosing constructs to determine a proper course of action. Flags are often inherited from inner subconstructs but that depends on each class.

benchmark(sampledata, filename=None)

Measures performance of your construct (its parsing and building runtime), both for the original instance and the compiled instance. Uses timeit module, over at min 1 sample, and at max over 1 second time.

Optionally, results are saved to a text file for later inspection. Otherwise you can print the result string to terminal.

Also this method checks correctness, by comparing parsing/building results from both instances.

Parameters:
  • sampledata – bytes, a valid blob parsable by this construct
  • filename – optional, string, source is saved to that file
Returns:

string containing measurements

build(obj, **contextkw)

Build an object in memory (a bytes object).

Whenever data cannot be written, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **contextkw.

Parameters:**contextkw – context entries, usually empty
Returns:bytes
Raises:ConstructError – raised for any reason
build_file(obj, filename, **contextkw)

Build an object into a closed binary file. See build().

build_stream(obj, stream, **contextkw)

Build an object directly into a stream. See build().

compile(filename=None)

Transforms a construct into another construct that does same thing (has same parsing and building semantics) but is much faster when parsing. Already compiled instances just compile into itself.

Optionally, partial source code can be saved to a text file. This is meant only to inspect the generated code, not to import it from external scripts.

Returns:Compiled instance
parse(data, **contextkw)

Parse an in-memory buffer (often bytes object). Strings, buffers, memoryviews, and other complete buffers can be parsed with this method.

Whenever data cannot be read, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated.

Context entries are passed only as keyword parameters **contextkw.

Parameters:**contextkw – context entries, usually empty
Returns:some value, usually based on bytes read from the stream but sometimes it is computed from nothing or from the context dictionary, sometimes its non-deterministic
Raises:ConstructError – raised for any reason
parse_file(filename, **contextkw)

Parse a closed binary file. See parse().

parse_stream(stream, **contextkw)

Parse a stream. Files, pipes, sockets, and other streaming sources of data are handled by this method. See parse().

sizeof(**contextkw)

Calculate the size of this object, optionally using a context.

Some constructs have fixed size (like FormatField), some have variable-size and can determine their size given a context entry (like Bytes(this.otherfield1)), and some cannot determine their size (like VarInt).

Whenever size cannot be determined, SizeofError is raised. This method is NOT ALLOWED to raise any other exception, even if eg. context dictionary is missing a key, or subcon propagates ConstructError-derivative exception.

Context entries are passed only as keyword parameters **contextkw.

Parameters:**contextkw – context entries, usually empty
Returns:integer if computable, SizeofError otherwise
Raises:SizeofError – size could not be determined in actual context, or is impossible to be determined
class construct.Subconstruct(subcon)

Abstract subconstruct (wraps an inner construct, inheriting its name and flags). Parsing and building is by default deferred to subcon, same as sizeof.

Parameters:subcon – Construct instance
class construct.Adapter(subcon)

Abstract adapter class.

Needs to implement _decode() for parsing and _encode() for building.

Parameters:subcon – Construct instance
class construct.SymmetricAdapter(subcon)

Abstract adapter class.

Needs to implement _decode() only, for both parsing and building.

Parameters:subcon – Construct instance
class construct.Validator(subcon)

Abstract class that validates a condition on the encoded/decoded object.

Needs to implement _validate() that returns a bool (or a truthy value)

Parameters:subcon – Construct instance
class construct.Tunnel(subcon)

Abstract class that allows other constructs to read part of the stream as if they were reading the entrie stream. See Prefixed for example.

Needs to implement _decode() for parsing and _encode() for building.

class construct.Compiled(source, defersubcon, parsefunc)

Used internally.

benchmark(sampledata, filename=None)

Measures performance of your construct (its parsing and building runtime), both for the original instance and the compiled instance. Uses timeit module, over at min 1 sample, and at max over 1 second time.

Optionally, results are saved to a text file for later inspection. Otherwise you can print the result string to terminal.

Also this method checks correctness, by comparing parsing/building results from both instances.

Parameters:
  • sampledata – bytes, a valid blob parsable by this construct
  • filename – optional, string, source is saved to that file
Returns:

string containing measurements

compile(filename=None)

Transforms a construct into another construct that does same thing (has same parsing and building semantics) but is much faster when parsing. Already compiled instances just compile into itself.

Optionally, partial source code can be saved to a text file. This is meant only to inspect the generated code, not to import it from external scripts.

Returns:Compiled instance