Input Configuration
Configure personas, base inputs, and input generation settings in configs/input.yaml.
Overview
The input configuration file defines:
- Personas: Different user types with characteristics and goals
- Base Inputs: Core test inputs to generate variations from
- Generation Settings: How to create input variations (LLM or deterministic)
- Variation Strategies: Transformations to apply (rephrase, verbose, etc.)
This configuration drives the fluxloop generate inputs command.
File Location
fluxloop/my-agent/
└── configs/
└── input.yaml # ← This file
Complete Example
# FluxLoop Input Configuration
# ------------------------------------------------------------
# Defines personas, base inputs, and generation modes.
# Adjust personas/goals/strategies based on your target scenarios.
personas:
- name: novice_user
description: A user new to the system
characteristics:
- Asks basic questions
- May use incorrect terminology
- Needs detailed explanations
language: en
expertise_level: novice
goals:
- Understand system capabilities
- Complete basic tasks
# Tip: Add persona-specific context that can be injected into prompts.
- name: expert_user
description: An experienced power user
characteristics:
- Uses technical terminology
- Asks complex questions
- Expects efficient responses
language: en
expertise_level: expert
goals:
- Optimize workflows
- Access advanced features
# Tip: Include any tone/style expectations in characteristics.
base_inputs:
- input: "How do I get started?"
expected_intent: help
# Provide optional 'metadata' or 'expected' fields to guide evaluation.
# ------------------------------------------------------------
# Input generation settings
# - variation_strategies: transformations applied when synthesizing inputs.
# - variation_count / temperature: tune diversity of generated samples.
# - inputs_file: location where generated inputs will be saved/loaded.
variation_strategies:
- rephrase
- verbose
- error_prone
variation_count: 2
variation_temperature: 0.7
inputs_file: inputs/generated.yaml
input_generation:
mode: llm
llm:
enabled: true
provider: openai
model: gpt-4o-mini
api_key: null
# Replace provider/model/api_key according to your LLM setup.
Personas
Overview
Personas represent different user types that will interact with your agent. Each persona has:
- Identity (name, description)
- Characteristics (behavior patterns)
- Expertise level
- Goals and objectives
- Language preferences
Persona Structure
personas:
- name: string # Required: unique identifier
description: string # Required: human-readable description
characteristics: list # Optional: behavioral traits
language: string # Optional: ISO language code (default: "en")
expertise_level: string # Optional: novice/intermediate/expert
goals: list # Optional: user objectives
Example Personas
Novice User
- name: novice_user
description: A user new to the system
characteristics:
- Asks basic questions
- May use incorrect terminology
- Needs detailed explanations
- Prefers step-by-step guidance
language: en
expertise_level: novice
goals:
- Understand system capabilities
- Complete basic tasks
- Learn fundamental concepts
Expert User
- name: expert_user
description: An experienced power user
characteristics:
- Uses technical terminology
- Asks complex questions
- Expects efficient responses
- Wants advanced features
language: en
expertise_level: expert
goals:
- Optimize workflows
- Access advanced features
- Automate repetitive tasks
Non-Native Speaker
- name: non_native_speaker
description: User with limited English proficiency
characteristics:
- Uses simple vocabulary
- May have grammatical errors
- Shorter sentences
- Literal interpretations
language: en
expertise_level: intermediate
goals:
- Get clear, simple answers
- Avoid complex terminology
Frustrated User
- name: frustrated_user
description: User experiencing issues or delays
characteristics:
- Expresses frustration or urgency
- May be impatient
- Wants quick solutions
- Previous attempts failed
language: en
expertise_level: intermediate
goals:
- Resolve issue quickly
- Get reassurance
- Understand what went wrong
Persona Fields
| Field | Type | Required | Description |
|---|---|---|---|
name | string | Yes | Unique persona identifier (snake_case) |
description | string | Yes | Human-readable persona description |
characteristics | list[string] | No | Behavioral traits and patterns |
language | string | No | ISO language code (e.g., en, es, fr) |
expertise_level | string | No | novice, intermediate, expert |
goals | list[string] | No | User objectives and motivations |
How Personas Are Used
- Input Generation: LLM uses persona characteristics to generate realistic variations
- Multi-Turn: Supervisor uses persona to craft follow-up questions
- Evaluation: Results can be grouped by persona for analysis
- Reporting: Dashboard shows performance per persona type
Base Inputs
Overview
Base inputs are the core test cases from which variations are generated. Each base input represents a distinct user intent or scenario.
Base Input Structure
base_inputs:
- input: string # Required: the actual input text
expected_intent: string # Optional: expected intent/category
metadata: object # Optional: custom fields
persona: string # Optional: default persona for this input
Example Base Inputs
Simple Base Inputs
base_inputs:
- input: "How do I get started?"
expected_intent: help
- input: "What are the pricing plans?"
expected_intent: pricing_inquiry
- input: "I can't log in to my account"
expected_intent: troubleshooting
With Metadata
base_inputs:
- input: "How do I reset my password?"
expected_intent: account_management
metadata:
category: authentication
priority: high
typical_persona: novice_user
- input: "Show me advanced filtering options"
expected_intent: feature_inquiry
metadata:
category: features
complexity: high
typical_persona: expert_user
With Expected Outputs
base_inputs:
- input: "What's the status of order #12345?"
expected_intent: order_inquiry
metadata:
expected_tool_calls: ["get_order_status"]
expected_response_type: "structured_data"
test_data:
order_id: "12345"
Base Input Fields
| Field | Type | Required | Description |
|---|---|---|---|
input | string | Yes | The actual input text |
expected_intent | string | No | Expected intent/category (for evaluation) |
metadata | object | No | Any custom metadata |
persona | string | No | Default persona for this input |
Design Guidelines
Good Base Inputs:
- Cover diverse user intents
- Represent real-world scenarios
- Include edge cases (errors, ambiguity)
- Vary in complexity
- Test different features
Examples:
base_inputs:
# Happy path
- input: "How do I create a new project?"
expected_intent: getting_started
# Edge case - ambiguous
- input: "Can you help me?"
expected_intent: help_general
# Edge case - error scenario
- input: "Why isn't this working?!?"
expected_intent: troubleshooting
# Complex multi-step
- input: "I need to export my data, format it as CSV, and email it"
expected_intent: complex_task
# Non-English (if supported)
- input: "¿Cómo puedo empezar?"
expected_intent: help
metadata:
language: es
Variation Strategies
Available Strategies
| Strategy | Description | Use Case |
|---|---|---|
rephrase | Reword input keeping same meaning | Test semantic understanding |
verbose | Add more details and context | Test handling of lengthy input |
concise | Shorten to minimal form | Test with terse users |
error_prone | Add typos, grammar errors | Test robustness |
formal | Use formal language | Test tone adaptation |
casual | Use casual/slang language | Test informal input |
technical | Add technical terminology | Test expert users |
simple | Simplify vocabulary | Test novice users |
Configuration
variation_strategies:
- rephrase # Always useful
- verbose # Test long-winded users
- error_prone # Test robustness
Examples of Variations
Base Input:
"How do I get started?"
Variations:
# rephrase
- "What are the first steps to begin?"
- "Where should I start?"
- "How can I initiate the process?"
# verbose
- "Hello, I'm completely new to this system and I was wondering if you could help me understand how I should get started with using it?"
# error_prone
- "how do i get strated?"
- "How do i gte started?"
# casual
- "yo how do i get going with this?"
# formal
- "I would like to inquire about the initial steps required to commence."
# technical (if applicable)
- "What's the initialization procedure for the system?"
Input Generation Settings
variation_count
Type: integer
Default: 2
Range: 1-10
Number of variations to generate per base input × persona combination.
variation_count: 2 # Generate 2 variations each
Formula:
Total Inputs = base_inputs × personas × variation_count
Example:
- 5 base inputs
- 2 personas
- 3 variations
- Total: 30 generated inputs
variation_temperature
Type: float
Default: 0.7
Range: 0.0-2.0
Controls diversity of LLM-generated variations.
variation_temperature: 0.7 # Balanced diversity
| Temperature | Effect | Use Case |
|---|---|---|
0.0-0.3 | Very conservative, minimal changes | Subtle rephrasing |
0.4-0.7 | Balanced diversity | General use (recommended) |
0.8-1.2 | More creative variations | Stress testing |
1.3-2.0 | Highly diverse, unpredictable | Edge case discovery |
inputs_file
Type: string
Default: inputs/generated.yaml
Path where generated inputs will be saved (relative to project root).
inputs_file: inputs/generated.yaml
File Structure:
# inputs/generated.yaml
- input: "How can I begin?"
persona: novice_user
metadata:
base_input: "How do I get started?"
variation_strategy: rephrase
variation_index: 0
generated_at: "2025-01-17T14:30:22Z"
- input: "Hello, I'm new and wondering how to get started..."
persona: novice_user
metadata:
base_input: "How do I get started?"
variation_strategy: verbose
variation_index: 1
LLM Configuration
input_generation.mode
Type: string
Options: llm | deterministic
Default: llm
Generation mode to use.
input_generation:
mode: llm # or deterministic
LLM Mode:
- Uses language model to generate variations
- More natural and diverse
- Requires API key
- Non-deterministic (different each run)
Deterministic Mode:
- Uses rule-based transformations
- Reproducible results
- No API key needed
- Faster and cheaper
LLM Settings
input_generation:
mode: llm
llm:
enabled: true
provider: openai
model: gpt-4o-mini
api_key: null # Set via OPENAI_API_KEY env var
| Field | Type | Description |
|---|---|---|
enabled | boolean | Enable LLM generation |
provider | string | openai, anthropic, gemini |
model | string | Model name (e.g., gpt-4o-mini) |
api_key | string | null | API key (use env var instead) |
Supported Providers
OpenAI
llm:
provider: openai
model: gpt-4o-mini # or gpt-4o, gpt-4-turbo
api_key: null # Set OPENAI_API_KEY
Anthropic
llm:
provider: anthropic
model: claude-3-haiku-20240307 # or claude-3-5-sonnet-20241022
api_key: null # Set ANTHROPIC_API_KEY
Google Gemini
llm:
provider: gemini
model: gemini-2.0-flash-exp # or gemini-2.5-flash
api_key: null # Set GEMINI_API_KEY
Setting API Keys
Via CLI:
fluxloop config set-llm openai sk-your-key --model gpt-4o-mini
Via .env file:
# .env
OPENAI_API_KEY=sk-your-key
Complete Examples
Minimal Configuration
personas:
- name: user
description: Regular user
base_inputs:
- input: "Hello"
- input: "Help me"
input_generation:
mode: deterministic
Standard Configuration
personas:
- name: novice_user
description: New to the system
expertise_level: novice
- name: expert_user
description: Experienced user
expertise_level: expert
base_inputs:
- input: "How do I get started?"
expected_intent: help
- input: "What are the advanced features?"
expected_intent: features
variation_strategies:
- rephrase
- verbose
variation_count: 3
variation_temperature: 0.7
inputs_file: inputs/generated.yaml
input_generation:
mode: llm
llm:
provider: openai
model: gpt-4o-mini
Advanced Configuration
personas:
- name: novice_user
description: First-time user
characteristics:
- Asks basic questions
- Needs guidance
- May make mistakes
language: en
expertise_level: novice
goals:
- Learn the basics
- Complete first task
- name: expert_user
description: Power user
characteristics:
- Uses shortcuts
- Expects efficiency
- Knows advanced features
language: en
expertise_level: expert
goals:
- Automate workflows
- Optimize performance
- name: non_english_user
description: ESL user
characteristics:
- Simple vocabulary
- Grammatical errors
- Direct questions
language: en
expertise_level: intermediate
goals:
- Clear communication
- Avoid confusion
base_inputs:
- input: "How do I create a new project?"
expected_intent: project_creation
metadata:
category: getting_started
complexity: easy
- input: "I need to bulk import 1000 items from CSV"
expected_intent: data_import
metadata:
category: advanced
complexity: high
- input: "Error 500 when I click submit"
expected_intent: error_report
metadata:
category: troubleshooting
priority: high
variation_strategies:
- rephrase
- verbose
- error_prone
- casual
- formal
variation_count: 5
variation_temperature: 0.8
inputs_file: inputs/generated.yaml
input_generation:
mode: llm
llm:
enabled: true
provider: openai
model: gpt-4o
api_key: null
Usage
Generate Inputs
# Generate with default settings
fluxloop generate inputs
# Generate specific number
fluxloop generate inputs --limit 50
# Use LLM mode
fluxloop generate inputs --mode llm
# Use deterministic mode
fluxloop generate inputs --mode deterministic
# Overwrite existing
fluxloop generate inputs --overwrite
View Generated Inputs
# View in terminal
cat inputs/generated.yaml
# Count total
wc -l inputs/generated.yaml
# View with syntax highlighting
bat inputs/generated.yaml
Tips and Best Practices
Persona Design
- Start with 2-3 personas - Don't overcomplicate initially
- Make them distinct - Clear differences in behavior
- Based on real users - Draw from actual user research
- Document reasoning - Add comments explaining choices
Base Input Selection
- Cover main user journeys - Critical paths first
- Include edge cases - Errors, ambiguity, unusual requests
- Test boundaries - Very short, very long inputs
- Real-world examples - Use actual user queries if available
Generation Strategy
For Development:
mode: deterministic # Fast, reproducible
variation_count: 2 # Minimal
variation_strategies:
- rephrase
For Testing:
mode: llm # Diverse, realistic
variation_count: 3 # Moderate
variation_temperature: 0.7
variation_strategies:
- rephrase
- verbose
- error_prone
For Production Validation:
mode: llm # Maximum coverage
variation_count: 5 # Extensive
variation_temperature: 0.9
variation_strategies:
- rephrase
- verbose
- error_prone
- casual
- formal
See Also
- Simulation Configuration - Runner and experiment settings
- Evaluation Configuration - Evaluators configuration
- generate Command - Input generation CLI
- Basic Workflow - Complete workflow guide