Skip to content

Expert user

ExpertUser: simulated user with column-level dropout and a growing known-features set.

Inherits from LangchainAgent and UserSimulator. Prompts are loaded from simulator_prompts.json.

ExpertUser(spec, dataset, seed=0, max_features_per_turn=5, early_stop_on_xstar=False, search_features=None, experience_features=None, credence_features=None, *args, model_name='gpt-5-nano', model_kwargs={}, max_react_steps=25, randomly_choose_initial=False, initial_dropout_rate=None, use_oracle_item_representations=False, use_actual_item_values=False, use_item_jsons=True, use_structured_actions=False, parser_reasoning_effort='medium', max_text_len=None, proactive_user=False, **kwargs)

Bases: LangchainAgent, UserSimulator

LLM-powered user simulator with SEC-aware feature revelation.

ExpertUser simulates a shopper whose preferences are encoded in a :class:~coshop.data.dataset.Specification. It uses a LangChain ReAct agent to respond to the shopping assistant and a set of helper classes to track which features have been revealed, parse the assistant's messages, and compare items.

Benchmark modes

  • Natural language mode (use_structured_actions=False, default): The simulator responds in free-form text, including questions and answers.
  • Structured action mode (use_structured_actions=True): The simulator responds with structured actions (parsed by :class:~coshop.user_simulator.helpers.structured_action_message_parser.StructuredActionMessageParser).

Feature revelation

Features are drawn from three SEC categories (search_features, experience_features, credence_features). Initially, only search features that pass through the dataset's dropout settings are known. As the conversation progresses, the simulator may reveal additional features up to max_features_per_turn per turn.

Official benchmark settings (as configured by evaluate_agent.py): use_structured_actions=False, max_features_per_turn=5, early_stop_on_xstar=True, proactive_user=False, use_oracle_item_representations=False, use_actual_item_values=False.

Parameters:

Name Type Description Default
spec Specification

Task specification containing xstar, z-variants, utility function, historical data, and simulator persona.

required
dataset

Dataset instance; provides catalog, true_features (column descriptions), and representation.

required
seed int

Random seed (currently unused; reserved for future use).

0
max_features_per_turn int

Maximum number of new preference features the simulator may reveal in a single turn. Official setting: 5.

5
early_stop_on_xstar bool

If True, the simulator stops the conversation once it believes xstar has been found. Official setting: True.

False
search_features Optional[List[str]]

List of column names classified as Search features for this episode. If all three SEC lists are None, all feature columns are treated as search features.

None
experience_features Optional[List[str]]

List of column names classified as Experience features (revealed after the user interacts with the product).

None
credence_features Optional[List[str]]

List of column names classified as Credence features (hard-to-verify, revealed last).

None
model_name str

LLM model name for the ReAct agent and all helpers. Defaults to "gpt-5-nano".

'gpt-5-nano'
model_kwargs dict

Extra keyword arguments forwarded to the LLM client (e.g. api_base, reasoning_effort).

{}
max_react_steps int

Maximum ReAct loop iterations per turn.

25
randomly_choose_initial bool

If True, the simulator's initial known features are drawn randomly rather than from spec.initial_known_features. Use together with initial_dropout_rate.

False
initial_dropout_rate Optional[float]

Fraction of features to drop from the initial feature set when randomly_choose_initial=True. None means no dropout.

None
use_oracle_item_representations bool

If True, the simulator sees the ground-truth (non-vagueified) item representations when evaluating items shown by the agent.

False
use_actual_item_values bool

If True, the simulator uses exact feature values instead of vagueified ones when describing its preferences.

False
use_item_jsons bool

If True, item representations are formatted as JSON dicts instead of the default paragraph format.

True
use_structured_actions bool

If True, use structured-action mode; otherwise natural-language mode (official default).

False
parser_reasoning_effort str

Reasoning effort level for the response parsing LLM. One of "low", "medium" (default), "high".

'medium'
max_text_len Optional[int]

Maximum character length of item text shown to the simulator. None disables truncation.

None
proactive_user bool

If True, the simulator proactively hints at features it needs even when the agent does not ask. Official setting: False.

False
**kwargs

Additional keyword arguments forwarded to :class:~coshop.utils.langchain_agent.LangchainAgent.

{}

get_current_z()

Return the current preference string built from known_features.

get_state_history()

Return history of state per turn (z at end of turn, features added by source) plus existing histories.

rank_items(items, mode='agentic', **kwargs)

Rank a provided list of item_ids using either: - agentic: LLM ranks from item text and conversation context. - parser: LOTUS feature-match on item descriptions vs known features; score = (# True) / (# active columns); False and None count as fails. - otherwise (e.g. rank): column-based utility (catalog vs xstar) restricted to known features.

rank_items_initial_state(items, mode='agentic', **kwargs)

Rank items as the simulator would have at the very start of the conversation, using only the initial known features / initial system message (i.e. before any clarifications or feedback updates).

Same mode values as :meth:rank_items (agentic, parser, or catalog rank).

reset()

Reset the agent to its initial state.

set_budget_tracker(budget_tracker)

Store the budget tracker for checking questions/items limits.