# User Clustering POC - Plan Summary

## Objective
Cluster users based on their content creation patterns using 4 input dataframes.

## Expected Clusters
- **Bots**: High creation volume, no consumption, consistent patterns
- **Casual Users**: Moderate activity, uses lyrics generation (gpt_description_prompt in metadata)
- **Pro Serious Users**: High pro_user_ratio, quality content, diverse feature usage
- **Others**: Various other patterns to be discovered

## Key Components

### Input Data
- `total_clip_df`: Main user activity data (starting pool of users)
- `boosts_action_df`: Additional clip metadata and engagement
- `reaction_df`: User reactions to clips (consumption behavior)
- `playlist_clip_df`: Playlist participation data (join by clip_id to track user behavior)

### Feature Categories (45+ features)
1. **Temporal**: Creation frequency, consistency, active days, burst patterns
2. **Bot Detection**: Consumption ratio, reaction patterns, creation timing
3. **Content Quality**: Duration, public ratio, deletion rate, pro status, lyrics generation
4. **Engagement**: Play counts, downloads, shares, upvotes
5. **Creation Source**: Diversity and distribution of creation methods
6. **Model Usage**: Model preferences, diversity, switch rate, adoption patterns
7. **Task Type**: Task diversity, specialization, primary tasks used
8. **Platform/Source**: Web vs mobile usage, iOS vs Android preference, cross-platform behavior
9. **Prompt Behavior**: Length, reuse, uniqueness, diversity
10. **Playlist**: Participation rate, positioning, playlist count

### Technical Approach
- Feature scaling with StandardScaler
- K-means clustering with k = 4 to 6 clusters only
- Elbow method and silhouette analysis for optimal k
- Cluster on data subset (10k-50k users) for memory efficiency
- Cluster stability validation
- Map clusters to expected types

### Deliverables
- User-cluster assignments CSV
- Cluster profiles with feature means
- Cluster-to-type mapping (bot, casual, pro, other)
- Saved k-means model
- Feature importance by cluster