flexeval.metrics.access

Utility functions for accessing metrics.

Functions

count_dict_values(lst)	Convenience function for counting key values.
get_all_metrics()
get_first_user_message_for_threads(thread_ids)	Get the first user message in each thread.

flexeval.metrics.access.count_dict_values(lst: list[dict]) → dict[str, Counter]

Convenience function for counting key values.

Parameters:

lst (list[dict]) – List of dictionaries.

Returns:

counter for each key that appears in the dicts in lst.

Return type:

dict[str, Counter]

flexeval.metrics.access.get_all_metrics() → list[dict]

flexeval.metrics.access.get_first_user_message_for_threads(thread_ids: set) → list[dict]

Get the first user message in each thread.

Parameters:

thread_ids (set) – The set of thread IDs to retrieve messages for.

Returns:

An iterable of messages.

Return type:

list[dict]