flexeval.classes.eval_runner#

Classes

EvalRunner(evalrun)

Class for maintaining database connection, logs, and run state Does not need to write anything to database itself.
class flexeval.classes.eval_runner.EvalRunner(evalrun: EvalRun)[source]#

Bases: object

Class for maintaining database connection, logs, and run state Does not need to write anything to database itself.

add_file_logger()[source]#
database: SqliteDatabase#
get_database_path() → Path[source]#
initialize_database()[source]#

Initializes database and tables. If config.clear_tables, then current contents of tables are dropped.

initialize_logger(add_stream_handler: bool = False)[source]#

Configure the logger for this class.

Parameters:

add_stream_handler (bool, optional) – If True, will add a stream handler at the INFO level. Defaults to False.

load_env()[source]#
load_evaluation_settings()[source]#

This function parses our eval suite and puts it in the data structure we’ll need for easy use at run-time.

shutdown_logging()[source]#