Generator Customization

Because semiwrap’s code generation is intended to be a semi-automated process (except for simple C++ code), a rich set of per-{class/function/parameter} configuration options can be specified in per-file YAML configurations.

Additionally, some headers are too complex for the autogenerator to completely process, so when this occurs you must manually specify required information in the YAML file.

Most files generated by semiwrap are customizable.

Note

semiwrap is designed for the RobotPy project and may contain defaults that aren’t appropriate for all projects. If you find that you need more customization, file an issue on github and let’s talk about it!

Location of customization file

By default customization files are located in the semiwrap directory. The name of the YAML file is the key in the extension module headers table:

[tool.semiwrap.extension_modules."PACKAGE.NAME".headers]
# yaml file is `semiwrap/demo.yml`
demo = "include/demo.h"

Autogeneration

The default values for these YAML files should be generated via the semiwrap command line tool:

semiwrap update-yaml --write

This can be a good way to get the boilerplate out of the way when you need to provide customizations.

Manual pybind11 APIs

You can write your own custom .cpp files and add them to the python extensions generated by semiwrap. In your meson.build after you include the generated semiwrap subdir, add files to NAME_sources.

subdir('semiwrap')

my_module_sources += files(
   'src/my_module/main.cpp',
)

Reference

The following strctures describe the dictionaries that are read from the YAML files. The toplevel structure is AutowrapConfigYaml.

class semiwrap.config.autowrap_yml.AutowrapConfigYaml(defaults=<factory>, strip_prefixes=<factory>, extra_includes=<factory>, extra_includes_first=<factory>, inline_code=None, attributes=<factory>, classes=<factory>, functions=<factory>, enums=<factory>, templates=<factory>, typealias=<factory>, encoding='utf-8-sig')

Format of the files specified by [tool.semiwrap.extension_modules."PACKAGENAME".headers]. Each key in this data structure is a toplevel key in the YAML file.

Parameters:

  • defaults (Defaults)

  • strip_prefixes (List[str])

  • extra_includes (List[str])

  • extra_includes_first (List[str])

  • inline_code (Optional[str])

  • attributes (Dict[str, PropData])

  • classes (Dict[str, ClassData])

  • functions (Dict[str, FunctionData])

  • enums (Dict[str, EnumData])

  • templates (Dict[str, TemplateData])

  • typealias (List[str])

  • encoding (str)

attributes: Dict[str, PropData]

Key is the attribute (variable) name

attributes:
  my_variable:
    # customizations here, see PropData
classes: Dict[str, ClassData]

Key is the namespace + class name

classes:
  NAMESPACE::CLASSNAME:
    # customizations here, see ClassData
defaults: Defaults

Provides a mechanism to specify certain default behaviors

encoding: str = 'utf-8-sig'

Encoding to use when opening this header file

enums: Dict[str, EnumData]

Key is the enum name, for enums at global scope

enums:
  MyEnum:
    # customizations here, see EnumData
extra_includes: List[str]

Adds #include <FILENAME> directives to the top of the autogenerated C++ file, after autodetected include dependencies are inserted.

extra_includes:
- foo.h
extra_includes_first: List[str]

Adds #include <FILENAME> directives after semiwrap.h is included, but before any autodetected include dependencies. Only use this when dealing with broken headers.

extra_includes_first:
- foo.h
functions: Dict[str, FunctionData]

Key is the function name

functions:
  fn_name:
    # customizations here, see FunctionData
inline_code: str | None = None

Specify raw C++ code that will be inserted at the end of the autogenerated file, inside a function. This is useful for extending your classes or providing other customizations. The following C++ variables are available:

  • m is the py::module instance

  • cls_CLASSNAME are py::class instances

  • … lots of other things too

The trampoline class (useful for accessing protected items) is available at {CLASSNAME}_Trampoline

To see the full list, run a build and look at the generated code at build/*/gensrc/**/*.cpp

Recommend that you use the YAML multiline syntax to specify it:

inline_code: |
  cls_CLASSNAME.def("get42", []() { return 42; });
strip_prefixes: List[str]

When specified,

strip_prefixes:
- FOO_  # any functions that begin with FOO_ will have it stripped
templates: Dict[str, TemplateData]

Instantiates a template. Key is the name to give to the Python type.

templates:
  ClassName:
    # customizations here, see TemplateData
typealias: List[str]

Extra ‘using’ directives to insert into the trampoline and the wrapping scope

typealias:
# makes SomeClass available in the wrapping scope
- some_namespace::SomeClass
class semiwrap.config.autowrap_yml.BufferData(type, src, len, minsz=None)

Specify that a parameter uses the buffer protocol

Parameters:

  • type (BufferType)

  • src (str)

  • len (str)

  • minsz (Optional[int])

len: str

Name of the C++ length parameter. An out-only parameter, it will be set to the size of the python buffer, and will be returned so the caller can determine how many bytes were written

minsz: int | None = None

If specified, the minimum size of the python buffer

src: str

Name of C++ parameter that the buffer will use

type: BufferType

Indicates what type of python buffer is required

class semiwrap.config.autowrap_yml.BufferType(value)
IN = 'in'

The buffer must indicate that it is readable (such as bytes, or bytearray)

INOUT = 'inout'

The buffer must indicate that it readable or writeable (such as a bytearray)

OUT = 'out'

The buffer must indicate that it is writeable (such as a bytearray)

class semiwrap.config.autowrap_yml.ClassData(doc=None, doc_append=None, ignore=False, ignored_bases=<factory>, base_qualnames=<factory>, attributes=<factory>, enums=<factory>, methods=<factory>, is_polymorphic=None, force_no_trampoline=False, force_no_default_constructor=False, force_multiple_inheritance=False, force_depends=<factory>, force_type_casters=<factory>, nodelete=False, rename=None, subpackage=None, typealias=<factory>, constants=<factory>, template_params=None, template_inline_code='', trampoline_inline_code=None, inline_code=None)
Parameters:

  • doc (Optional[str])

  • doc_append (Optional[str])

  • ignore (bool)

  • ignored_bases (List[str])

  • base_qualnames (Dict[str, str])

  • attributes (Dict[str, PropData])

  • enums (Dict[str, EnumData])

  • methods (Dict[str, FunctionData])

  • is_polymorphic (Optional[bool])

  • force_no_trampoline (bool)

  • force_no_default_constructor (bool)

  • force_multiple_inheritance (bool)

  • force_depends (List[str])

  • force_type_casters (List[str])

  • nodelete (bool)

  • rename (Optional[str])

  • subpackage (Optional[str])

  • typealias (List[str])

  • constants (List[str])

  • template_params (Optional[List[str]])

  • template_inline_code (str)

  • trampoline_inline_code (Optional[str])

  • inline_code (Optional[str])

attributes: Dict[str, PropData]

Customize attributes of the class

classes:
  MyClass:
    attributes:
      m_attr:
base_qualnames: Dict[str, str]

Specify fully qualified names for the bases. If the base has a template parameter, you must include it. Only needed if it can’t be automatically detected directly from the text.

constants: List[str]

Fully-qualified pre-existing constant that will be inserted into the trampoline and wrapping scopes as a constexpr

doc: str | None = None

Docstring for the class

doc_append: str | None = None

Text to append to the (autoconverted) docstring

enums: Dict[str, EnumData]

Customize enums

classes:
  MyClass:
    enums:
      MyEnum:
force_depends: List[str]

If there are circular dependencies, this will help you resolve them manually. TODO: make it so we don’t need this

force_multiple_inheritance: bool = False

pybind11 will detect multiple inheritance automatically if a class directly derives from multiple classes. However, If the class derives from classes that participate in multiple inheritance, pybind11 won’t detect it automatically, so this flag is needed.

force_no_default_constructor: bool = False

Disable generating a default constructor for this class

force_no_trampoline: bool = False

Disable generating a trampoline for this class

force_type_casters: List[str]

Use this to bring in type casters for a particular type that may have been hidden (for example, with a typedef or definition in another file), instead of explicitly including the header. This should be the full namespace of the type.

ignore: bool = False

Ignore this class

ignored_bases: List[str]

List of bases to ignore. Name must include any template specializations.

inline_code: str | None = None

This will insert code right before the semicolon ending the class py definition. You can use this to easily insert additional custom functions without using the global inline_code mechanism.

is_polymorphic: bool | None = None

Inform the autogenerator that this class is polymorphic

methods: Dict[str, FunctionData]

Customize methods

classes:
  MyClass:
    methods:
      mymethod:
nodelete: bool = False

If the object shouldn’t be deleted by pybind11, use this. Disables implicit constructors.

rename: str | None = None

Set the python name of the class to this

subpackage: str | None = None

If specified, put the class in a sub.pack.age. Ignored for functions attached to a class. When template parameters are used, must define subpackage on template instances instead

template_inline_code: str = ''

If this is a template class, the specified C++ code is inserted into the template definition

template_params: List[str] | None = None

If this is a template class, a list of the parameters if it can’t be autodetected (currently can’t autodetect). If there is no space in the parameter, then it is assumed to be a ‘typename’, otherwise the parameter is split by space and the first item is the type and the second parameter is the name (useful for integral templates)

trampoline_inline_code: str | None = None

If this class has an associated trampoline, add this code inline at the bottom of the trampoline class. This is rarely useful.

typealias: List[str]

Extra ‘using’ directives to insert into the trampoline and the wrapping scope

class semiwrap.config.autowrap_yml.Defaults(ignore=False, report_ignored_missing=True, subpackage=None, references_are_out_param=False)

Defaults to apply to everything

defaults:
  ignore: true
  report_ignored_missing: false
Parameters:

  • ignore (bool)

  • report_ignored_missing (bool)

  • subpackage (Optional[str])

  • references_are_out_param (bool)

ignore: bool = False

Set this to True to ignore functions/enums/classes at namespace scope by default if they aren’t present in the YAML

references_are_out_param: bool = False

If specified, any non-const Foo& parameters will be assumed to be an ‘out’ parameter

report_ignored_missing: bool = True

If False and default ignore is True, don’t report missing items

subpackage: str | None = None

All items will be added to this subpackage unless specified otherwise

class semiwrap.config.autowrap_yml.EnumData(doc=None, doc_append=None, ignore=False, rename=None, value_prefix=None, subpackage=None, values=<factory>, inline_code=None, arithmetic=False)
Parameters:

  • doc (Optional[str])

  • doc_append (Optional[str])

  • ignore (bool)

  • rename (Optional[str])

  • value_prefix (Optional[str])

  • subpackage (Optional[str])

  • values (Dict[str, EnumValue])

  • inline_code (Optional[str])

  • arithmetic (bool)

arithmetic: bool = False

Tell pybind11 to create an enumeration that also supports rudimentary arithmetic and bit-level operations like comparisons, and, or, xor, negation, etc.

doc: str | None = None

Set your own docstring for the enum

doc_append: str | None = None

Text to append to the (autoconverted) docstring

ignore: bool = False

If set to True, this property is not made available to python

inline_code: str | None = None

This will insert code right before the semicolon ending the enum py definition. You can use this to easily insert additional custom values without using the global inline_code mechanism.

rename: str | None = None

Set the python name of this enum to the specified string

subpackage: str | None = None

If specified, put the enum in a sub.pack.age (ignored for enums that are part of classes)

value_prefix: str | None = None

Remove this prefix from autogenerated enum value name

values: Dict[str, EnumValue]

Enum value configuration

class semiwrap.config.autowrap_yml.EnumValue(ignore=False, rename=None, doc=None, doc_append=None)
Parameters:

  • ignore (bool)

  • rename (Optional[str])

  • doc (Optional[str])

  • doc_append (Optional[str])

doc: str | None = None

Docstring for the enum value

doc_append: str | None = None

Text to append to the (autoconverted) docstring

ignore: bool = False

If set to True, this property is not made available to python

rename: str | None = None

Set the python name of this enum value to the specified string

class semiwrap.config.autowrap_yml.FunctionData(ignore=False, ignore_pure=False, ignore_py=False, ifdef=None, ifndef=None, cpp_code=None, doc=None, doc_append=None, disable_none=None, internal=False, rename=None, param_override=<factory>, subpackage=None, no_release_gil=None, buffers=<factory>, keepalive=None, return_value_policy=ReturnValuePolicy.automatic, template_impls=None, trampoline_cpp_code=None, virtual_xform=None, overloads=<factory>)

Customize the way the autogenerator binds a function.

functions:
  # for non-overloaded functions, just specify the name + customizations
  name_of_non_overloaded_fn:
    # add customizations for function here

  # For overloaded functions, specify the name, but each overload
  # separately
  my_overloaded_fn:
    overloads:
      int, int:
        # customizations for `my_overloaded_fn(int, int)`
      int, int, int:
        # customizations for `my_overloaded_fn(int, int, int)`
Parameters:

  • ignore (bool)

  • ignore_pure (bool)

  • ignore_py (bool)

  • ifdef (Optional[str])

  • ifndef (Optional[str])

  • cpp_code (Optional[str])

  • doc (Optional[str])

  • doc_append (Optional[str])

  • disable_none (Optional[bool])

  • internal (bool)

  • rename (Optional[str])

  • param_override (Dict[str, ParamData])

  • subpackage (Optional[str])

  • no_release_gil (Optional[bool])

  • buffers (List[BufferData])

  • keepalive (Optional[List[Tuple[int, int]]])

  • return_value_policy (ReturnValuePolicy)

  • template_impls (Optional[List[List[str]]])

  • trampoline_cpp_code (Optional[str])

  • virtual_xform (Optional[str])

  • overloads (Dict[str, OverloadData])

overloads: Dict[str, OverloadData]

See above

class semiwrap.config.autowrap_yml.OverloadData(ignore=False, ignore_pure=False, ignore_py=False, ifdef=None, ifndef=None, cpp_code=None, doc=None, doc_append=None, disable_none=None, internal=False, rename=None, param_override=<factory>, subpackage=None, no_release_gil=None, buffers=<factory>, keepalive=None, return_value_policy=ReturnValuePolicy.automatic, template_impls=None, trampoline_cpp_code=None, virtual_xform=None)

See also

FunctionData

Parameters:

  • ignore (bool)

  • ignore_pure (bool)

  • ignore_py (bool)

  • ifdef (Optional[str])

  • ifndef (Optional[str])

  • cpp_code (Optional[str])

  • doc (Optional[str])

  • doc_append (Optional[str])

  • disable_none (Optional[bool])

  • internal (bool)

  • rename (Optional[str])

  • param_override (Dict[str, ParamData])

  • subpackage (Optional[str])

  • no_release_gil (Optional[bool])

  • buffers (List[BufferData])

  • keepalive (Optional[List[Tuple[int, int]]])

  • return_value_policy (ReturnValuePolicy)

  • template_impls (Optional[List[List[str]]])

  • trampoline_cpp_code (Optional[str])

  • virtual_xform (Optional[str])

buffers: List[BufferData]

Configures parameters that can receive objects that implement the buffer protocol

cpp_code: str | None = None

Use this code instead of the generated code

disable_none: bool | None = None

Disallow implicit conversions from None for all parameters. See also disable_none in ParamData.

doc: str | None = None

Docstring for the function, will attempt to convert Doxygen docs if omitted

doc_append: str | None = None

Text to append to the (autoconverted) docstring for the function

ifdef: str | None = None

Generate this in an #ifdef

ifndef: str | None = None

Generate this in an #ifndef

ignore: bool = False

If True, don’t wrap this

ignore_pure: bool = False

If True, don’t wrap this, but provide a pure virtual implementation

ignore_py: bool = False

Most of the time, you will want to specify ignore instead.

If True, don’t expose this function to python. If a trampoline is supposed to be generated, it will still be generated. You will likely want to use trampoline_cpp_code if you specify this.

internal: bool = False

If True, prepends an underscore to the python name

keepalive: List[Tuple[int, int]] | None = None

Adds py::keep_alive<x,y> to the function. Overrides automatic keepalive support, which retains references passed to constructors. https://pybind11.readthedocs.io/en/stable/advanced/functions.html#keep-alive

no_release_gil: bool | None = None

By default, semiwrap will release the GIL whenever a wrapped function is called.

param_override: Dict[str, ParamData]

Mechanism to override individual parameters. For example, to rename the first parameter of void my_function(int param1):

functions:
    my_function:
        param_override:
             param1:
rename: str | None = None

Use this to set the name of the function as exposed to python

When applied to a constructor, creates a static method for the overload instead.

return_value_policy: ReturnValuePolicy = 'automatic'

https://pybind11.readthedocs.io/en/stable/advanced/functions.html#return-value-policies

subpackage: str | None = None

If specified, put the function in a sub.pack.age

template_impls: List[List[str]] | None = None

If this is a function template, this is a list of instantiations that you wish to provide. This is a list of lists, where the inner list is the template parameters for that function

If there are ‘auto’ parameters, each one is treated as a unique template argument. If the return value is ‘auto’, it also must be specified here

trampoline_cpp_code: str | None = None

Specify custom C++ code for the virtual function trampoline

virtual_xform: str | None = None

Specify a transformation lambda to be used when this virtual function is called from C++. This inline code should be a lambda that has the same arguments as the original C++ virtual function, except the first argument will be a py::function with the python overload

cpp_code should also be specified for this to be useful

For example, to transform a function that takes an iostream into a function that returns a string:

cpp_code: |
  [](MyClass* self) {
    return "string";
  }
virtual_xform: |
  [](py::function fn, MyClass* self, std::iostream &is) {
     std::string d = py::cast(fn());
     is << d;
  }
class semiwrap.config.autowrap_yml.ParamData(name=None, x_type=None, default=None, no_default=False, disable_none=None, disable_type_caster_default_cast=False, force_out=False, array_size=None, ignore=False)

Various ways to modify parameters

Parameters:

  • name (Optional[str])

  • x_type (Optional[str])

  • default (Optional[str])

  • no_default (Optional[bool])

  • disable_none (Optional[bool])

  • disable_type_caster_default_cast (bool)

  • force_out (bool)

  • array_size (Optional[int])

  • ignore (bool)

array_size: int | None = None

Force an array size

default: str | None = None

Default value for parameter

disable_none: bool | None = None

Disallow implicit conversions from None. This defaults to True for built in types and types that are obviously std::function (does not handle all cases, in which case this should be explicitly specified)

disable_type_caster_default_cast: bool = False

Disables a default cast caused by default_arg_cast

force_out: bool = False

Force this to be an ‘out’ parameter

See also

Out parameters

ignore: bool = False

Ignore this parameter

name: str | None = None

Set parameter name to this

no_default: bool | None = False

Disable default value

x_type: str | None = None

Change C++ type emitted

class semiwrap.config.autowrap_yml.PropAccess(value)

Whether a property is writable.

auto = 'auto'

Determine read/read-write automatically:

  • If a struct/union, default to readwrite

  • If a class, default to readwrite if a basic type that isn’t a reference, otherwise default to readonly

readonly = 'readonly'

Allow python users access to the value, but ensure it can’t change. This is useful for properties that are defined directly in the class

readwrite = 'readwrite'

Allows python users to read/write the value

class semiwrap.config.autowrap_yml.PropData(ignore=False, rename=None, access=PropAccess.auto, doc=None, doc_append=None)
Parameters:

  • ignore (bool)

  • rename (Optional[str])

  • access (PropAccess)

  • doc (Optional[str])

  • doc_append (Optional[str])

access: PropAccess = 'auto'

Python code access to this property

doc: str | None = None

Docstring for the property (only available on class properties)

doc_append: str | None = None

Text to append to the (autoconverted) docstring

ignore: bool = False

If set to True, this property is not made available to python

rename: str | None = None

Set the python name of this property to the specified string

class semiwrap.config.autowrap_yml.ReturnValuePolicy(value)

See pybind11 documentation for what each of these values mean.

automatic = 'automatic'

pybind11 determines the policy to use

automatic_reference = 'automatic_reference'

pybind11 determines the policy to use, but falls back to a reference

copy = 'copy'

Create a new copy of the returned object, which will be owned by Python.

move = 'move'

Use std::move to move the return value contents into a new instance that will be owned by Python.

reference = 'reference'

Reference an existing object, but do not take ownership.

reference_internal = 'reference_internal'

Indicates that the lifetime of the return value is tied to the lifetime of a parent object

take_ownership = 'take_ownership'

Reference an existing object (i.e. do not create a new copy) and take ownership.

class semiwrap.config.autowrap_yml.TemplateData(qualname, params, subpackage=None, doc=None, doc_append=None)

Instantiates a template as a python type. To customize the class, add it to the classes key and specify the template type.

Code to be wrapped:

template <typename T>
class MyClass {};

To bind MyClass<int> as the python class MyIntClass, add this to your YAML:

classes:
  MyClass:
    template_params:
    - T

templates:
  MyIntClass:
    qualname: MyClass
    params:
    - int
Parameters:

  • qualname (str)

  • params (List[Union[str, int]])

  • subpackage (Optional[str])

  • doc (Optional[str])

  • doc_append (Optional[str])

doc: str | None = None

Set the docstring for the template instance

doc_append: str | None = None

Text to append to the (autoconverted) docstring for the template instance

params: List[str | int]

Template parameters to use

qualname: str

Fully qualified name of instantiated class

subpackage: str | None = None

If specified, put the template instantiation in a sub.pack.age