flexeval.cli#

CLI commands.

Functions

`global_callback`(ctx[, log_level])	FlexEval offers a number of CLI commands for convenience.
`main`()
`run`(eval_run_yaml_path)	Run FlexEval using the given YAML Eval Run configuration.
`run_eval_by_name`(input_data, database_path, ...)	Run an eval by name.
`summarize_metrics`([eval_run_yaml_path, ...])	Print a summary of computed metrics.

flexeval.cli.global_callback(ctx: ~typer.models.Context, log_level: ~flexeval.log_utils.Annotated[~flexeval.log_utils.LogLevel, <typer.models.OptionInfo object at 0x7fdf9b0cb220>] = 'info')[source]#: FlexEval offers a number of CLI commands for convenience.

flexeval.cli.run(eval_run_yaml_path: ~pathlib.Annotated[~pathlib.Path, <typer.models.ArgumentInfo object at 0x7fdf9b0cb7f0>])[source]#: Run FlexEval using the given YAML Eval Run configuration.

flexeval.cli.run_eval_by_name(input_data: ~typing.Annotated[list[~pathlib.Path], <typer.models.OptionInfo object at 0x7fdf9b0cae60>], database_path: ~pathlib.Annotated[~pathlib.Path, <typer.models.OptionInfo object at 0x7fdf9b0cb8e0>], eval_name: str, evals_path: ~pathlib.Path, config_path: ~pathlib.Path, clear_tables: bool = False)[source]#: Run an eval by name.

flexeval.cli.summarize_metrics(eval_run_yaml_path: ~types.Annotated[~pathlib.Path | None, <typer.models.ArgumentInfo object at 0x7fdf9b0cb4f0>] | None = None, database_path: ~types.Annotated[~pathlib.Path | None, <typer.models.OptionInfo object at 0x7fdf9b0cae90>] | None = None)[source]#: Print a summary of computed metrics.