# Orphy Assistant Architecture

## Overview

Orphy (named after Orpheus) is the AI assistant for Suno Spaces. Users can mention `@suno` in messages to request song creation, room management, DJ duties, and production assistance. Orphy acts as a mix between a producer, producer assistant, and DJ.

## Core Capabilities

### Primary Functions
1. **Song Creation**: Generate songs from text prompts
2. **DJ Duties**: Manage room playback queue, suggest songs, create playlists
3. **Production Assistance**: Provide feedback, suggest edits, explain music theory
4. **Room Management**: Control room settings, playback, and radio mode

### Future Extensions
- Remix and sampling operations
- Lyrics generation and editing
- Video generation for songs
- Image generation for album art
- Collaborative editing suggestions
- Music theory explanations and tutorials

## Architecture

### High-Level Flow

```
User Message (@suno) → Message Handler → Intent Classification → Tool Selection → Atom Creation/Action → Response
```

### Components

#### 1. Message Handler (Convex Mutation)

**Purpose**: Detect and process messages that mention `@suno`

**Location**: `convex/assistant.ts`

**Responsibilities**:
- Listen for new messages containing `@suno` mention
- Extract user intent and context
- Queue assistant request for processing
- Create placeholder response message
- Trigger assistant action

**Implementation**:
```typescript
// convex/assistant.ts
export const handleAssistantMention = mutation({
  args: {
    messageId: v.id('messages'),
    roomId: v.id('rooms'),
    spaceId: v.id('spaces'),
  },
  handler: async (ctx, args) => {
    const message = await ctx.db.get(args.messageId);
    // Extract @suno mentions and content
    // Create assistant response message with status='processing'
    // Schedule assistant action
  },
});
```

#### 2. Intent Classifier (Convex Action with OpenAI)

**Purpose**: Determine what the user wants Orphy to do

**Location**: `convex/assistant.ts`

**Model**: OpenAI GPT-4o-mini (fast, cost-effective)

**Intents**:
- `generate_song`: Create a new song from prompt
- `manage_queue`: Add/remove songs from room queue
- `playback_control`: Play, pause, skip, adjust volume
- `suggest_music`: Recommend songs based on context
- `explain`: Answer questions about music, theory, or production
- `room_settings`: Change room configuration

**Implementation**:
```typescript
// convex/assistant.ts
export const classifyIntent = action({
  args: {
    messageContent: v.string(),
    roomContext: v.object({ /* room state */ }),
  },
  handler: async (ctx, args) => {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        { role: 'system', content: ASSISTANT_SYSTEM_PROMPT },
        { role: 'user', content: args.messageContent },
      ],
      tools: ASSISTANT_TOOLS,
      tool_choice: 'auto',
    });

    return response.choices[0].message.tool_calls;
  },
});
```

#### 3. Tool Router (Convex Action)

**Purpose**: Execute the appropriate tool based on classified intent

**Location**: `convex/assistant.ts`

**Tools**:

##### Tool: `generate_song`
- **Parameters**: `{ prompt: string, tags: string, makeInstrumental?: boolean }`
- **Handler**: Calls Suno API client to generate songs
- **Prompt Format**: Must use section tags ([Verse], [Chorus], [PreChorus], [Bridge], [Outro])
- **Lyrics Guidelines**:
  - Structure: Verse → PreChorus → Chorus, then repeat variations
  - Catchy, memorable choruses (hook-driven)
  - Avoid excessive rhyming - natural flow preferred
  - Conversational language over overly poetic
- **Returns**: Atom IDs for created songs
- **Response**: "🎵 Creating 2 songs for you: '{prompt}'. I'll let you know when they're ready!"

##### Tool: `manage_queue`
- **Parameters**: `{ action: 'add' | 'remove' | 'clear', atomIds?: string[] }`
- **Handler**: Updates room queue in database
- **Returns**: Updated queue
- **Response**: "✅ Queue updated! Now playing X songs."

##### Tool: `playback_control`
- **Parameters**: `{ action: 'play' | 'pause' | 'skip' | 'volume', value?: number }`
- **Handler**: Updates room playback state
- **Returns**: New playback state
- **Response**: "▶️ Now playing [song title]" or "⏸️ Playback paused"

##### Tool: `suggest_music`
- **Parameters**: `{ context?: string, mood?: string, genre?: string }`
- **Handler**: Queries atoms table and uses OpenAI for recommendations
- **Returns**: List of suggested atom IDs
- **Response**: "🎧 Here are some suggestions: [song list with reasons]"

##### Tool: `explain`
- **Parameters**: `{ question: string, context?: object }`
- **Handler**: Uses OpenAI to generate educational response
- **Returns**: Markdown-formatted explanation
- **Response**: Detailed answer with examples

##### Tool: `room_settings`
- **Parameters**: `{ setting: string, value: any }`
- **Handler**: Updates room configuration
- **Returns**: Updated room settings
- **Response**: "⚙️ Room settings updated!"

#### 4. Response Generator (Convex Mutation)

**Purpose**: Update assistant message with final response

**Location**: `convex/assistant.ts`

**Responsibilities**:
- Update placeholder message with actual response
- Attach created atoms to message
- Set status to 'completed' or 'failed'
- Notify users in room

**Implementation**:
```typescript
export const updateAssistantResponse = mutation({
  args: {
    messageId: v.id('messages'),
    content: v.string(),
    atomIds: v.optional(v.array(v.id('atoms'))),
    status: v.union(v.literal('completed'), v.literal('failed')),
  },
  handler: async (ctx, args) => {
    await ctx.db.patch(args.messageId, {
      content: args.content,
      atomReferences: args.atomIds,
      status: args.status,
      updatedAt: Date.now(),
    });
  },
});
```

## Data Model Extensions

### Messages Table Updates

Add assistant-specific fields to messages:

```typescript
// convex/schema.ts
messages: defineTable({
  // ... existing fields

  // Assistant-specific fields
  isAssistantMessage: v.boolean(), // true if sent by Orphy
  assistantRequestId: v.optional(v.id('assistantRequests')), // link to request
  assistantStatus: v.optional(
    v.union(
      v.literal('pending'),
      v.literal('processing'),
      v.literal('completed'),
      v.literal('failed')
    )
  ),
})
```

### New Table: Assistant Requests

Track assistant requests for debugging and analytics:

```typescript
// convex/schema.ts
assistantRequests: defineTable({
  userId: v.id('users'),
  spaceId: v.id('spaces'),
  roomId: v.id('rooms'),
  messageId: v.id('messages'), // Original user message
  responseMessageId: v.optional(v.id('messages')), // Orphy's response

  rawContent: v.string(), // User's message content
  classifiedIntent: v.string(), // Detected intent
  toolCalls: v.array(v.object({
    toolName: v.string(),
    parameters: v.any(),
    result: v.optional(v.any()),
  })),

  status: v.union(
    v.literal('pending'),
    v.literal('processing'),
    v.literal('completed'),
    v.literal('failed')
  ),
  errorMessage: v.optional(v.string()),

  createdAt: v.number(),
  completedAt: v.optional(v.number()),
})
  .index('by_user', ['userId'])
  .index('by_room', ['roomId'])
  .index('by_status', ['status'])
```

## System Prompts

### Assistant System Prompt

```typescript
const ASSISTANT_SYSTEM_PROMPT = `You are Orphy, the AI assistant for Suno Spaces. You help users create music, manage playback, and provide production assistance.

Your personality:
- Friendly and enthusiastic about music
- Professional but casual (like a helpful studio engineer)
- Concise responses (1-3 sentences usually)
- Use music emojis occasionally (🎵 🎧 🎶 ✨)

Your capabilities:
- Generate songs from text prompts using Suno's API
- Manage room playback queues
- Control playback (play, pause, skip)
- Suggest songs based on mood, genre, or context
- Answer questions about music theory and production
- Adjust room settings

When a user mentions you:
1. Understand their intent
2. Choose the appropriate tool
3. Execute the action
4. Respond with a friendly confirmation

Always be helpful, creative, and music-focused.`;
```

### Tool Definitions (OpenAI Function Calling)

```typescript
const ASSISTANT_TOOLS = [
  {
    type: 'function',
    function: {
      name: 'generate_song',
      description: 'Generate a new song. Must write original lyrics with proper formatting.',
      parameters: {
        type: 'object',
        properties: {
          prompt: {
            type: 'string',
            description: 'Actual lyrics with section tags: [Verse], [Chorus], [PreChorus], [Bridge]. Make choruses catchy. Avoid excessive rhyming.',
          },
          tags: {
            type: 'string',
            description: 'Comma-separated genre/style tags (e.g., "indie rock, energetic")',
          },
          makeInstrumental: {
            type: 'boolean',
            description: 'Whether to create an instrumental version',
            default: false,
          },
        },
        required: ['prompt', 'tags'],
      },
    },
  },
  {
    type: 'function',
    function: {
      name: 'manage_queue',
      description: 'Add, remove, or clear songs from the room queue',
      parameters: {
        type: 'object',
        properties: {
          action: {
            type: 'string',
            enum: ['add', 'remove', 'clear'],
            description: 'The queue action to perform',
          },
          atomIds: {
            type: 'array',
            items: { type: 'string' },
            description: 'Song atom IDs to add or remove',
          },
        },
        required: ['action'],
      },
    },
  },
  // ... more tools
];
```

## UI Components

### Assistant Message Bubble

Display assistant messages with special styling:

```tsx
// components/AssistantMessage.tsx
export function AssistantMessage({ message }) {
  return (
    <div className="assistant-message">
      <Avatar src="/orphy-avatar.png" name="Orphy" />
      <div className="message-content">
        <div className="message-header">
          <span className="assistant-badge">🎵 Orphy</span>
          {message.assistantStatus === 'processing' && (
            <LoadingSpinner />
          )}
        </div>
        <div className="message-text">{message.content}</div>
        {message.atomReferences && (
          <div className="attached-atoms">
            {message.atomReferences.map(atomId => (
              <AtomCard key={atomId} atomId={atomId} />
            ))}
          </div>
        )}
      </div>
    </div>
  );
}
```

### Progress Indicator

Show real-time progress for song generation:

```tsx
// components/SongGenerationProgress.tsx
export function SongGenerationProgress({ requestId }) {
  const request = useQuery(api.assistant.getRequest, { id: requestId });

  if (!request || request.status === 'completed') return null;

  return (
    <div className="generation-progress">
      <div className="progress-bar">
        <div
          className="progress-fill"
          style={{ width: `${request.progress ?? 0}%` }}
        />
      </div>
      <p className="progress-text">
        {request.status === 'pending' && '⏳ Queuing your request...'}
        {request.status === 'processing' && '🎵 Generating songs...'}
      </p>
    </div>
  );
}
```

## Error Handling

### Common Errors

1. **Rate Limiting**: Suno API rate limits exceeded
   - **Response**: "⏰ I'm a bit overwhelmed right now. Can you try again in a few minutes?"

2. **Invalid Prompt**: Prompt doesn't meet content guidelines
   - **Response**: "❌ I couldn't process that prompt. Can you try rephrasing?"

3. **Generation Failed**: Suno API returns error
   - **Response**: "😔 Something went wrong with song generation. Let's try again?"

4. **Insufficient Permissions**: User lacks permission for action
   - **Response**: "🔒 You don't have permission to do that in this room."

### Error Logging

```typescript
export const logAssistantError = mutation({
  args: {
    requestId: v.id('assistantRequests'),
    error: v.object({
      code: v.string(),
      message: v.string(),
      stack: v.optional(v.string()),
    }),
  },
  handler: async (ctx, args) => {
    await ctx.db.patch(args.requestId, {
      status: 'failed',
      errorMessage: args.error.message,
      completedAt: Date.now(),
    });

    // Log to monitoring service (future)
    console.error('Assistant error:', args.error);
  },
});
```

## Implementation Plan

### Phase 1: Basic Song Generation (MVP)
**Goal**: Users can mention @suno to generate songs

- [ ] Set up OpenAI API client in Convex
- [ ] Implement message handler to detect @suno mentions
- [ ] Create assistantRequests table
- [ ] Implement intent classifier with OpenAI
- [ ] Implement `generate_song` tool using Suno client
- [ ] Create assistant response generator
- [ ] Build AssistantMessage UI component
- [ ] Add progress indicators for song generation
- [ ] Test end-to-end flow: @suno → intent → generation → response

**Acceptance Criteria**:
- User types "@suno create a chill lofi beat"
- Orphy responds: "🎵 Creating 2 songs for you: 'a chill lofi beat'. I'll let you know when they're ready!"
- Progress indicator shows during generation
- When complete, songs appear as atoms attached to Orphy's message

### Phase 2: Queue Management
**Goal**: Orphy can manage room playback

- [ ] Implement room queue data structure
- [ ] Create `manage_queue` tool
- [ ] Create `playback_control` tool
- [ ] Build queue UI component
- [ ] Implement playback controls in room header
- [ ] Add synchronized playback with presence
- [ ] Test queue operations via @suno commands

### Phase 3: Music Suggestions
**Goal**: Orphy can recommend songs

- [ ] Implement `suggest_music` tool
- [ ] Build recommendation algorithm (OpenAI + atom metadata)
- [ ] Create suggestion UI component
- [ ] Add context awareness (room history, user preferences)
- [ ] Test various suggestion prompts

### Phase 4: Production Assistance
**Goal**: Orphy can answer questions and provide feedback

- [ ] Implement `explain` tool
- [ ] Build knowledge base for music theory
- [ ] Create rich markdown response renderer
- [ ] Add support for multi-turn conversations (threads)
- [ ] Test educational prompts

### Phase 5: Advanced Features
**Goal**: Extend Orphy's capabilities

- [ ] Remix and sampling tools
- [ ] Lyrics generation
- [ ] Video generation for songs
- [ ] Image generation for album art
- [ ] Custom voice for Orphy (future)
- [ ] Voice command support (future)

## Security & Privacy

### Permissions
- Only space members can invoke Orphy
- Orphy respects room permissions
- Private rooms: Only members can request Orphy actions
- Atom permissions: Orphy can only reference atoms user has access to

### Content Moderation
- All prompts pass through OpenAI's moderation API
- Inappropriate content is rejected before reaching Suno API
- Failed moderation logged for review

### Rate Limiting
- Per-user rate limits (e.g., 10 requests per hour)
- Per-space rate limits (e.g., 50 requests per hour)
- Graceful degradation with clear error messages

## Analytics & Monitoring

### Metrics to Track
- **Usage**: Requests per day, requests per user, requests per space
- **Performance**: Average response time, generation success rate
- **Intent Distribution**: Which tools are used most frequently
- **Error Rate**: Failed requests by error type
- **User Satisfaction**: Implicit feedback (did user engage with result?)

### Logging
- Log all assistant requests with full context
- Log tool executions and results
- Log errors with stack traces
- Enable querying for debugging and analytics

## Testing Strategy

### Unit Tests
- Test intent classification with various prompts
- Test each tool handler independently
- Test error handling paths

### Integration Tests
- Test full flow: message → classification → tool → response
- Test with Suno API (use test tokens)
- Test rate limiting and error scenarios

### User Testing
- Internal alpha with Suno team
- Collect feedback on assistant personality
- Iterate on prompt engineering and tool design

## Future Enhancements

### Context Awareness
- Remember previous requests in conversation
- Learn user preferences over time
- Suggest actions proactively based on room activity

### Multi-Modal Interactions
- Voice commands ("Hey Orphy, play something upbeat")
- Visual prompts (upload image for song inspiration)
- Gesture controls on mobile/VR

### Collaboration Features
- Multi-user jamming sessions coordinated by Orphy
- Real-time collaboration on song editing
- Group voting on Orphy's suggestions

### Personalization
- Custom Orphy personalities per space
- User-specific assistant preferences
- Fine-tuned models for specific music genres

## Technical Considerations

### OpenAI API Costs
- GPT-4o-mini is cost-effective (~$0.15 per 1M input tokens)
- Most requests will be small (< 1000 tokens)
- Estimated cost: ~$0.0001 per request
- Budget for 100k requests/month: ~$10/month

### Suno API Integration
- Already built (`lib/sunoClient.ts`)
- Handles async song generation
- Polls for completion with configurable intervals

### Convex Actions vs Scheduled Functions
- Use **Actions** for immediate response (< 30s)
- Use **Scheduled Functions** for long-running operations (song generation)
- Actions can schedule functions for background work

### Real-Time Updates
- Use Convex subscriptions for live progress updates
- Client automatically re-renders when message status changes
- No need for manual polling from frontend

## Message Attachments (Atoms in Messages)

### Overview

Users can now attach atoms (songs, videos, etc.) to messages. This feature enables:
- Sharing songs directly in chat conversations
- Attaching generated songs to messages
- Building conversations around specific musical content
- Creating playlists through chat interactions

### Architecture

#### Backend Changes

**Message Schema** (`convex/schema.ts`):
- Messages already support `atomReferences: v.optional(v.array(v.id("atoms")))`
- This field stores IDs of atoms attached to the message

**Message Send Mutation** (`convex/messages.ts`):
```typescript
export const send = mutation({
  args: {
    roomId: v.id("rooms"),
    content: v.string(),
    mentions: v.optional(v.array(v.union(v.id("users"), v.literal("orphy")))),
    atomIds: v.optional(v.array(v.id("atoms"))),  // NEW: Optional atom attachments
  },
  handler: async (ctx, args) => {
    // ... authorization checks
    const messageId = await ctx.db.insert("messages", {
      // ... other fields
      atomReferences: args.atomIds,  // Store attached atoms
    });
    return messageId;
  },
});
```

#### Frontend Changes

**SongAtomRow Component** (`components/SongAtomRow.tsx`):
- Added `onAttachToMessage` callback prop
- New "Attach to Message" button (MessageSquarePlus icon)
- Button appears on song rows when callback is provided
- Clicking attaches the song's atom ID to the current message draft

**Space Page Message Composer** (`app/space/[id]/page.tsx`):

1. **State Management**:
   - `attachedAtomIds`: Array of atom IDs to be sent with the message
   - `showAttachMenu`: Controls visibility of the attachment menu

2. **Attachment Dock**:
   - Displays above message input when atoms are attached
   - Shows thumbnail, title, and remove button for each attached atom
   - Uses `atoms.getByIds` query to fetch atom metadata

3. **Message Input Bar**:
   - "+" button on the left side opens attachment menu
   - Menu shows "Record" and "Upload file" options (currently disabled)
   - Input field for typing message text
   - Send button submits message with attachments

4. **Workflow**:
   ```
   User clicks attach button on song row
   → handleAttachAtom() adds atom ID to attachedAtomIds state
   → Attachment dock displays the atom preview
   → User types message and clicks Send
   → sendMessage() includes atomIds in mutation
   → Message is created with atomReferences
   → Attached atoms clear from draft
   ```

### UI Components

**Attachment Dock** (appears above message input):
```tsx
{attachedAtoms && attachedAtoms.length > 0 && (
  <div className="attachment-dock">
    <div className="header">Attached ({attachedAtoms.length})</div>
    {attachedAtoms.map(atom => (
      <div className="atom-preview">
        <img src={atom.metadata.albumArtUrl} />
        <div className="title">{atom.metadata.tags}</div>
        <button onClick={() => handleRemoveAttachment(atom._id)}>×</button>
      </div>
    ))}
  </div>
)}
```

**Attachment Menu** (opens from "+" button):
```tsx
<div className="attachment-menu">
  <button disabled>
    <Mic /> Record (Coming soon)
  </button>
  <button disabled>
    <Upload /> Upload file (Coming soon)
  </button>
</div>
```

**Message Display**:
- Messages with `atomReferences` automatically render attached atoms using `SongAtomRow`
- Each atom can be played, liked, or attached to a new message
- Located in `MessageItem` component at `app/space/[id]/page.tsx:122-134`

### User Flow Examples

#### Example 1: Sharing a Song
1. User sees a song they like in chat
2. Clicks the MessageSquarePlus button on the song row
3. Song appears in attachment dock above message input
4. User types "Check this out!" and sends
5. Message appears in chat with song attachment

#### Example 2: Orphy Song Generation
1. User mentions @orphy: "create a chill lofi beat"
2. Orphy generates two songs
3. Songs automatically attached to Orphy's response message
4. Songs render as playable SongAtomRows in the message
5. Other users can play, like, or re-attach songs to their own messages

#### Example 3: Building a Playlist via Chat
1. User A: "Here's a great intro track" [attaches song]
2. User B: "This would be perfect next" [attaches another song]
3. User C: "And then this for the buildup" [attaches third song]
4. Conversation creates a sequential playlist discussion

### Future Enhancements

#### Planned Features (from "+" menu):
- **Record**: Record audio directly in the browser and upload as an atom
- **Upload File**: Upload audio files, images, or videos as atoms
- **Drag & Drop**: Drag atoms from message history to attachment dock
- **Bulk Attach**: Select multiple songs and attach all at once

#### Advanced Features:
- **Inline Playback**: Play attached songs without leaving the message
- **Playlist Creation**: Convert message thread into a room queue/playlist
- **Atom Collections**: Group multiple atoms into a collection (album, EP)
- **Rich Previews**: Show waveforms, spectrograms, or video thumbnails
- **Reactions to Atoms**: React to specific atoms within a message

### Technical Notes

**TypeScript Checks**:
After making changes to message handling, always run:
```bash
npx tsc --noEmit --project convex/tsconfig.json
```

**Performance Considerations**:
- Attachment dock uses `atoms.getByIds` query with "skip" when no atoms attached
- Avoids unnecessary queries and re-renders
- Atom metadata cached by Convex's subscription system

**Access Control**:
- Messages can only attach atoms from the same space
- Users must have access to the room to see attached atoms
- Atom permissions respected when rendering (future enhancement)

## References

- OpenAI Function Calling: https://platform.openai.com/docs/guides/function-calling
- Convex Actions: https://docs.convex.dev/functions/actions
- Suno API Client: `lib/sunoClient.ts`
- Message Schema: `convex/schema.ts`