CLI#
FlexEval offers a command-line interface.
flexeval --help
# OR
python -m flexeval --help
The help output contains the list of valid commands:
[1m [0m
[1m [0m[1mUsage: [0m[1mpython [0m[1m-m[0m[1m flexeval [OPTIONS] COMMAND [ARGS]...[0m[1m [0m[1m [0m
[1m [0m
FlexEval offers a number of CLI commands for convenience.
[2mâ•─[0m[2m Options [0m[2m───────────────────────────────────────────────────────────────────[0m[2m─╮[0m
[2m│[0m [1m-[0m[1m-log[0m[1m-level[0m [1;2m[[0m[1mcritical[0m[1;2m|[0m[1merror[0m[1;2m|[0m[1mwarning[0m Log level to use. [2m│[0m
[2m│[0m [1;2m|[0m[1minfo[0m[1;2m|[0m[1mdebug[0m[1;2m][0m[1m [0m [2m[default: info] [0m [2m│[0m
[2m│[0m [1m-[0m[1m-install[0m[1m-completion[0m [1m [0m Install completion for [2m│[0m
[2m│[0m the current shell. [2m│[0m
[2m│[0m [1m-[0m[1m-show[0m[1m-completion[0m [1m [0m Show completion for the [2m│[0m
[2m│[0m current shell, to copy [2m│[0m
[2m│[0m it or customize the [2m│[0m
[2m│[0m installation. [2m│[0m
[2m│[0m [1m-[0m[1m-help[0m [1m [0m Show this message and [2m│[0m
[2m│[0m exit. [2m│[0m
[2m╰──────────────────────────────────────────────────────────────────────────────╯[0m
[2mâ•─[0m[2m Commands [0m[2m──────────────────────────────────────────────────────────────────[0m[2m─╮[0m
[2m│[0m [1mrun [0m[1m [0m Run FlexEval using the given YAML Eval Run [2m│[0m
[2m│[0m [1m [0m configuration. [2m│[0m
[2m│[0m [1msummarize-metrics [0m[1m [0m Print a summary of computed metrics. [2m│[0m
[2m│[0m [1mrun-eval-by-name [0m[1m [0m Run an eval by name. [2m│[0m
[2m╰──────────────────────────────────────────────────────────────────────────────╯[0m
See the Basic CLI usage vignette for a working example of CLI usage.