flexeval.classes.thread#

Classes

Thread(**kwargs)

Class for holding a single thread / conversation This corresponds to a single row in a jsonl file or a single 'thread_id' in a langgraph checkpoint database

class flexeval.classes.thread.Thread(**kwargs)[source]#

Bases: BaseModel

Class for holding a single thread / conversation This corresponds to a single row in a jsonl file or a single ‘thread_id’ in a langgraph checkpoint database

dataset = <ForeignKeyField: Thread.dataset>#

eval_run_thread_id = <TextField: Thread.eval_run_thread_id>#

format_input_for_rubric()[source]#

get_content(include_toolcalls=True)[source]#: Content is a list of dictionaries where each dictionary contains the role and content of messages and tool calls in the turn. Each tool call appears after the message it’s associated with. If toolcalls are not desired, pass False to include_toolcalls.

get_formatted_prompt(include_system_prompt=False)[source]#

id = <IntegerField: Thread.id>#

jsonl_thread_id = <TextField: Thread.jsonl_thread_id>#

langgraph_thread_id = <TextField: Thread.langgraph_thread_id>#

messages#: !! processed by numpydoc !!

metadata = <TextField: Thread.metadata>#

metadata_dict#

Descriptor that provides dict-like access to a JSON text field.

Example: class SomeModel(pw.Model):

some_field = pw.TextField(default=”{}”) some_field_dict = JsonView(text_field_attr_name=”some_field”)

metrics_list#: !! processed by numpydoc !!

system_prompt = <TextField: Thread.system_prompt>#

toolcalls#: !! processed by numpydoc !!

turns#: !! processed by numpydoc !!