roberto package

Submodules

roberto.conda module

roberto.config module

Define the Roberto’s configuration.

class roberto.config.RobertoConfig(overrides=None, defaults=None, system_prefix=None, user_prefix=None, project_location=None, runtime_path=None, lazy=False)

Bases: Config

A specialized Invoke configuration for Roberto.

The main modifications in behavior w.r.t. the vanilla Config are:

• The roberto prefix.

• A project config file .roberto.* can be loaded from the current directory.

• Default configuration, which can be extended overridden by user.

• Config finalization, filling in some blanks with sensible defaults.

prefix = 'roberto'

load_base_conf_files()

Load also the local project config file.

set_project_location(path)

Override the default mechanism of Invoke to find project config.

Parameters

path (str) – This argument is ignored and the local directory is used instead.

load_shell_env()

Call _finalize after Invoke has loaded the complete config.

static global_defaults()

Set the global default configuration, before loading any other config.

Return type

dict

__contains__(key)

Return type

bool

__delattr__(name)

Implement delattr(self, name).

Return type

None

__delitem__(key)

Return type

None

__dict__ = mappingproxy({'__module__': 'roberto.config', '__doc__': 'A specialized Invoke configuration for Roberto.\n\n    The main modifications in behavior w.r.t. the vanilla Config are:\n\n    - The `roberto` prefix.\n    - A project config file `.roberto.*` can be loaded from the current directory.\n    - Default configuration, which can be extended overridden by user.\n    - Config finalization, filling in some blanks with sensible defaults.\n\n    ', 'prefix': 'roberto', 'load_base_conf_files': <function RobertoConfig.load_base_conf_files>, 'set_project_location': <function RobertoConfig.set_project_location>, 'load_shell_env': <function RobertoConfig.load_shell_env>, '_finalize': <function RobertoConfig._finalize>, 'global_defaults': <staticmethod object>, '__annotations__': {}})

__eq__(other)

Return self==value.

Return type

bool

__getattr__(key)

Return type

Any

__getitem__(key)

Return type

Any

__hash__ = None

__init__(overrides=None, defaults=None, system_prefix=None, user_prefix=None, project_location=None, runtime_path=None, lazy=False)

Creates a new config object.

Parameters

• defaults (dict) – A dict containing default (lowest level) config data. Default: global_defaults.

• overrides (dict) – A dict containing override-level config data. Default: {}.

• system_prefix (str) –

  Base path for the global config file location; combined with the prefix and file suffixes to arrive at final file path candidates.

  Default: /etc/ (thus e.g. /etc/invoke.yaml or /etc/invoke.json).

• user_prefix (str) –

  Like system_prefix but for the per-user config file. These variables are joined as strings, not via path-style joins, so they may contain partial file paths; for the per-user config file this often means a leading dot, to make the final result a hidden file on most systems.

  Default: ~/. (e.g. ~/.invoke.yaml).

• project_location (str) – Optional directory path of the currently loaded .Collection (as loaded by .Loader). When non-empty, will trigger seeking of per-project config files in this directory.

• runtime_path (str) –

  Optional file path to a runtime configuration file.

  Used to fill the penultimate slot in the config hierarchy. Should be a full file path to an existing file, not a directory path or a prefix.

• lazy (bool) –

  Whether to automatically load some of the lower config levels.

  By default (lazy=False), __init__ automatically calls load_system and load_user to load system and user config files, respectively.

  For more control over what is loaded when, you can say lazy=True, and no automatic loading is done.

  Note

  If you give defaults and/or overrides as __init__ kwargs instead of waiting to use load_defaults or load_overrides afterwards, those will still end up ‘loaded’ immediately.

__iter__()

Return type

Iterator[Dict[str, Any]]

__len__()

Return type

int

__module__ = 'roberto.config'

__repr__()

Return repr(self).

Return type

str

__setattr__(key, value)

Implement setattr(self, name, value).

Return type

None

__setitem__(key, value)

Return type

None

__weakref__

list of weak references to the object (if defined)

clear()

Return type

None

clone(into=None)

Return a copy of this configuration object.

The new object will be identical in terms of configured sources and any loaded (or user-manipulated) data, but will be a distinct object with as little shared mutable state as possible.

Specifically, all dict values within the config are recursively recreated, with non-dict leaf values subjected to copy.copy (note: not copy.deepcopy, as this can cause issues with various objects such as compiled regexen or threading locks, often found buried deep within rich aggregates like API or DB clients).

The only remaining config values that may end up shared between a config and its clone are thus those ‘rich’ objects that do not copy.copy cleanly, or compound non-dict objects (such as lists or tuples).

Parameters

into (Optional[Type[Config]]) –

A .Config subclass that the new clone should be “upgraded” to.

Used by client libraries which have their own .Config subclasses that e.g. define additional defaults; cloning “into” one of these subclasses ensures that any new keys/subtrees are added gracefully, without overwriting anything that may have been pre-defined.

Default: None (just clone into another regular .Config).

Return type

Config

Returns

A .Config, or an instance of the class given to into.

Raises

TypeError, if into is given a value and that value is not a .Config subclass.

New in version 1.0.

env_prefix = None

file_prefix = None

classmethod from_data(data, root=None, keypath=())

Alternate constructor for ‘baby’ DataProxies used as sub-dict values.

Allows creating standalone DataProxy objects while also letting subclasses like .Config define their own __init__ without muddling the two.

Parameters

• data (dict) – This particular DataProxy’s personal data. Required, it’s the Data being Proxied.

• root (Optional[DataProxy]) – Optional handle on a root DataProxy/Config which needs notification on data updates.

• keypath (tuple) – Optional tuple describing the path of keys leading to this DataProxy’s location inside the root structure. Required if root was given (and vice versa.)

Return type

DataProxy

New in version 1.0.

load_collection(data, merge=True)

Update collection-driven config data.

.load_collection is intended for use by the core task execution machinery, which is responsible for obtaining collection-driven data. See collection-configuration for details. :rtype: None

New in version 1.0.

load_defaults(data, merge=True)

Set or replace the ‘defaults’ configuration level, from data.

Parameters

• data (dict) – The config data to load as the defaults level.

• merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

load_overrides(data, merge=True)

Set or replace the ‘overrides’ configuration level, from data.

Parameters

• data (dict) – The config data to load as the overrides level.

• merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

load_project(merge=True)

Load a project-level config file, if possible.

Checks the configured _project_prefix value derived from the path given to set_project_location, which is typically set to the directory containing the loaded task collection.

Thus, if one were to run the CLI tool against a tasks collection /home/myuser/code/tasks.py, load_project would seek out files like /home/myuser/code/invoke.yml.

Parameters

merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

load_runtime(merge=True)

Load a runtime-level config file, if one was specified.

When the CLI framework creates a Config, it sets _runtime_path, which is a full path to the requested config file. This method attempts to load that file.

Parameters

merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

load_system(merge=True)

Load a system-level config file, if possible.

Checks the configured _system_prefix path, which defaults to /etc, and will thus load files like /etc/invoke.yml.

Parameters

merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

load_user(merge=True)

Load a user-level config file, if possible.

Checks the configured _user_prefix path, which defaults to ~/., and will thus load files like ~/.invoke.yml.

Parameters

merge (bool) – Whether to merge the loaded data into the central config. Default: True.

Return type

None

Returns

None.

New in version 1.0.

merge()

Merge all config sources, in order. :rtype: None

New in version 1.0.

pop(*args)

Return type

Any

popitem()

Return type

Any

set_runtime_path(path)

Set the runtime config file path. :rtype: None

New in version 1.0.

setdefault(*args)

Return type

Any

update(*args, **kwargs)

Return type

None

roberto.program module

Define the Roberto program.

roberto.requirements module

Installation of requirements.

roberto.requirements.compute_req_hash(req_items, req_fns)

Compute a hash from all parameters that affect installed packages.

Parameters

• req_items (Set[str]) – A set of requirement items.

• req_fns (Set[str]) – A set of requirement files.

Returns

hashdigest – The hex digest of the sha256 hash of all development requirements.

Return type

str

roberto.requirements.check_install_requirements(fn_skip, req_hash)

Check if reinstallation of requirements is desired.

Parameters

• fn_skip (str) – File where the requirements hash is stored.

• req_has – The (new) requirement hash.

Returns

True if reinstallation is desired.

Return type

install

roberto.requirements.install_requirements_conda(ctx)

Install all requirements, including tools used by Roberto.

roberto.requirements.convert_requires(fn_requires, fn_requirements)

Convert requires.txt from an egg-info to a requirements.txt file.

roberto.requirements.install_requirements_pip(ctx)

Install requirements in the virtual environment.

roberto.tasks module

Tasks in Roberto’s workflow.

Due to the decorators, no documentation is generated for this module. Use rob --help to get an overview of Roberto’s task.

roberto.tasks.with_stdout_header(f)

Print a header for a task.

roberto.testenvs module

Tools to set up virtual environments.

All classes implemented here are stateless. They do not have attributes and store state information in the configuration object instead. They are essentially groups of methods with compatible API, yet with implementation details that may differ.

roberto.testenvs.append_activate(ctx, line)

Append a line to the activate script and show it.

roberto.testenvs.install_macosx_sdk(ctx)

Install MacOSX SDK if on OSX if needed.

roberto.testenvs.init_testenv(cfg)

Initialize a test environment defined in the configuration.

roberto.testenvs.init_testenv_conda(cfg)

Initialize a testenv using Conda.

roberto.testenvs.install_conda(ctx)

Install miniconda if not present yet.

roberto.testenvs.setup_conda(ctx)

Set up a conda testing environment.

roberto.testenvs.nuke_conda(ctx)

Erase the conda environment.

roberto.testenvs.init_testenv_venv(cfg)

Initialize a test environment using Python’s built-in venv.

roberto.testenvs.setup_venv(ctx)

Make sure there is a virtual environment and activate it.

roberto.testenvs.nuke_venv(ctx)

Erase the virtual environment.

roberto.testenvs.init_testenv_none(cfg)

Initialize a testenv without really doing anything.

roberto.testenvs.setup_none(ctx)

Do nothing. Stub for API consistency.

roberto.testenvs.nuke_none(ctx)

Do nothing. Stub for API consistency.

roberto.tools module

Tools contribute functionality to tasks.

roberto.tools.initialize_tools(tools_config)

Build a dictionary with available tools.

roberto.tools.execute_tools(ctx, tool_cls)

Execute all tools of a given class for all packages.

class roberto.tools.Tool

Bases: object

Tool base class.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

Parameters

• ctx – Invoke Context instance.

• package – Package configuration.

• fmtkwargs – A dictionary used for formatting arguments.

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Tool base class.', '__init_subclass__': <classmethod object>, 'execute': <function Tool.execute>, '_run_commands': <staticmethod object>, '__dict__': <attribute '__dict__' of 'Tool' objects>, '__weakref__': <attribute '__weakref__' of 'Tool' objects>, '__annotations__': {}})

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.WriteVersion(name, template, destination)

Bases: Tool

Write version info to a file.

__init__(name, template, destination)

Initialize a WriteVersion instance.

Parameters

• name – The name of the tool

• template – Template for the source code with the version info.

• destination – The destination to write the file too.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Write version info to a file.', '__init__': <function WriteVersion.__init__>, 'execute': <function WriteVersion.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.Lint(name, commands_master, commands_feature)

Bases: Tool

Base class for linters.

__init__(name, commands_master, commands_feature)

Initialize a Lint instance.

Parameters

• name – The name of the tool

• commands_master – Executed when working in the merge_branch, usually master.

• commands_feature – Executed when working in a feature branch.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Base class for linters.', '__init__': <function Lint.__init__>, 'execute': <function Lint.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.LintStatic(name, commands_master, commands_feature)

Bases: Lint

Stub for static linters.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for static linters.', '__annotations__': {}})

__init__(name, commands_master, commands_feature)

Initialize a Lint instance.

Parameters

• name – The name of the tool

• commands_master – Executed when working in the merge_branch, usually master.

• commands_feature – Executed when working in a feature branch.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.LintDynamic(name, commands_master, commands_feature)

Bases: Lint

Stub for dynamic linters.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for dynamic linters.', '__annotations__': {}})

__init__(name, commands_master, commands_feature)

Initialize a Lint instance.

Parameters

• name – The name of the tool

• commands_master – Executed when working in the merge_branch, usually master.

• commands_feature – Executed when working in a feature branch.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.BuildInPlace(name, check_vars, commands, extra_vars)

Bases: Tool

Build in-place and check and/or extend environment variables.

__init__(name, check_vars, commands, extra_vars)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• check_vars – A list of variable names to print before building, just to help detecting issues.

• commands – Build commands.

• extra_vars – Additions to environment variables needed after this build.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Build in-place and check and/or extend environment variables.', '__init__': <function BuildInPlace.__init__>, '_check_vars': <function BuildInPlace._check_vars>, '_update_extra_vars': <function BuildInPlace._update_extra_vars>, 'execute': <function BuildInPlace.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.RunCommands(name, commands)

Bases: Tool

Base class for all tools that just execute some commands.

__init__(name, commands)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• commands – Commands to run.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Base class for all tools that just execute some commands.', '__init__': <function RunCommands.__init__>, 'execute': <function RunCommands.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.TestInPlace(name, commands)

Bases: RunCommands

Stub for in-place testing tools.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for in-place testing tools.', '__annotations__': {}})

__init__(name, commands)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• commands – Commands to run.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.UploadCoverage(name, commands)

Bases: RunCommands

Stub for coverage upload.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for coverage upload.', '__annotations__': {}})

__init__(name, commands)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• commands – Commands to run.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.BuildDocs(name, commands)

Bases: RunCommands

Stub for documentation builds.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for documentation builds.', '__annotations__': {}})

__init__(name, commands)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• commands – Commands to run.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.UploadDocs

Bases: Tool

Base class for all docu upload tools.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Base class for all docu upload tools.', '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

Parameters

• ctx – Invoke Context instance.

• package – Package configuration.

• fmtkwargs – A dictionary used for formatting arguments.

class roberto.tools.UploadDocsGit(name, docroot, docbranch, docremote, deploy_labels)

Bases: UploadDocs

Squash-push documentation to a git branch.

__init__(name, docroot, docbranch, docremote, deploy_labels)

Initialize an UploadDocsGit instance.

Parameters

• name – The name of the tool

• docroot – The subdirectory where the documentation can be found.

• docbranch – The branch to which the documentation must be pushed.

• docremote – The remote repository for the documentation.

• deploy_labels – The kind of releases for which documentation must be uploaded.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Squash-push documentation to a git branch.', '__init__': <function UploadDocsGit.__init__>, 'execute': <function UploadDocsGit.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

class roberto.tools.BuildPackage(name, commands)

Bases: RunCommands

Stub for building a package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Stub for building a package.', '__annotations__': {}})

__init__(name, commands)

Initialize a BuiltInPlace instance.

Parameters

• name – The name of the tool

• commands – Commands to run.

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

class roberto.tools.Deploy(name, deploy_vars, include_sha256, noarch_asset_patterns, binary_asset_patterns, deploy_labels, commands)

Bases: Tool

Upload software packages to distribution servers.

__init__(name, deploy_vars, include_sha256, noarch_asset_patterns, binary_asset_patterns, deploy_labels, commands)

Initialize an UploadDocsGit instance.

Parameters

• name – The name of the tool

• deploy_vars – Essential environment variables for deployment. Their presence is checked.

• include_sha256 – If True, sha256 checksums are generated for the assets

• noarch_asset_patterns – Glob pattern for architecture-independent assets.

• binary_asset_patterns – Glob pattern for architecture-dependent assets.

• deploy_labels – The kind of releases for which documentation must be uploaded.

• commands – Bash command to perform the deployment.

execute(ctx, package, fmtkwargs)

Execute the tool for a given package.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'roberto.tools', '__doc__': 'Upload software packages to distribution servers.', '__init__': <function Deploy.__init__>, 'execute': <function Deploy.execute>, '__annotations__': {}})

classmethod __init_subclass__(**kwargs)

Register any new sub class.

__module__ = 'roberto.tools'

__weakref__

list of weak references to the object (if defined)

roberto.utils module

Utilities used by tasks in Roberto’s workflow.

roberto.utils.parse_git_describe(git_describe)

Parse the output of git describe --tags.

Parameters

git_describe (str) – The output of git describe

Returns

version_info – A dictionary with version information.

Return type

dict

exception roberto.utils.TagError(message, tag)

Bases: Exception

An exception raised when a version tag is invalid.

__init__(message, tag)

Initialize an TagError.

Parameters

• message (str) – Describe the error.

• tag (str) – The tag having the problem, which will be added to the error message.

__cause__

exception cause

__context__

exception context

__delattr__(name, /)

Implement delattr(self, name).

__dict__ = mappingproxy({'__module__': 'roberto.utils', '__doc__': 'An exception raised when a version tag is invalid.', '__init__': <function TagError.__init__>, '__weakref__': <attribute '__weakref__' of 'TagError' objects>, '__annotations__': {}})

__getattribute__(name, /)

Return getattr(self, name).

__module__ = 'roberto.utils'

__new__(**kwargs)

__reduce__()

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value, /)

Implement setattr(self, name, value).

__setstate__()

__str__()

Return str(self).

__suppress_context__

__traceback__

__weakref__

list of weak references to the object (if defined)

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

roberto.utils.sanitize_branch(ctx, branch)

Attempt to fix the presence of a branch.

The branch is checked with rev-parse. If not present, try to set it to origin/branch. If that does not work, try to fetch it.

Parameters

• ctx (Context) – A invoke.Context instance.

• branch (str) – The branch to resurrect.

roberto.utils.check_env_var(name)

Check if an environment variable is set and non-empty.

Parameters

name (str) – The environment variable to be checked.

roberto.utils.need_deployment(ctx, prefix, binary, deploy_labels)

Return True if deployment is needed.

Parameters

• ctx (Context) – A invoke.Context instance.

• prefix (str) – The type of deployment, used for message printed to stdout.

• binary (bool) – Whether or not a binary is being deployment. If binary, it may be of interest to deploy for different architectures, which is controlled by a config.deploy_binary, intead of config.deploy_source.

• deploy_labels (List[str]) – A list of valid deploy labels for this release. If the current deploy label matches any of the ones in the list, deployment is needed.

Return type

bool

roberto.utils.write_sha256_sum(fn_asset)

Make a sha256 checksum file, print it and return the checksum filename.

Parameters

fn_asset (str) – The file of which the hash will be computed.

Returns

fn_asset_hash – The filename of the file containing the hash.

Return type

str

roberto.venv module

roberto.version module

Load version info or, when that failed, resort to bogus values.

Module contents

Roberto package.