# Extended Features Documentation

This document describes all the extended features added to the AI Toolkit that are not part of the original codebase. It includes instructions on how to apply these changes to an unchanged AI Toolkit installation.

## Table of Contents

1. [New Features Overview](#new-features-overview)
2. [Files to Override](#files-to-override)
3. [New Files to Add](#new-files-to-add)
4. [Feature Details](#feature-details)
5. [Installation Instructions](#installation-instructions)

---

## New Features Overview

### 1. **Advanced Queue** (New Feature)
An enhanced version of the Training Queue with advanced filtering, pagination, search, and bulk operations.

**Location:** `/jobs-extended`

**Features:**
- Pagination for idle entries with options: 25, 50, 100, 250, 500, 1000, 2000, 5000, or "all"
- Search by job name
- Status filter dropdown
- Bulk actions:
  - **Stop All Queued Jobs**: Marks all queued jobs as stopped (leaves running jobs intact)
  - **Resume Stopped Jobs**: Changes all stopped jobs back to queued status

### 2. **From Dataset to Queue** (New Feature)
Allows users to batch-create training jobs from datasets using templates.

**Location:** `/batch-queue`

**Features:**
- Select training template (wan, zimage, etc.)
- View available datasets that haven't been trained yet
- Select multiple datasets and submit them to the training queue
- Automatically redirects to Advanced Queue after submission

### 3. **UI Improvements**
- Fixed context menu text color (dark gray → white) for better visibility
- Added tooltips to all action buttons in JobActionBar
- Improved user experience with hover hints

---

## Files to Override

These files were **modified** from the original AI Toolkit. You need to replace them with the extended versions:

### 1. `ui/src/components/Sidebar.tsx`
**Changes:**
- Added "Advanced Queue" navigation item linking to `/jobs-extended`
- Added "From Dataset to Queue" navigation item linking to `/batch-queue`
- Uses `BrainCircuit` icon for Advanced Queue and `ListPlus` icon for From Dataset to Queue

**Location in navigation:** 
- "Advanced Queue" appears between "Training Queue" and "From Dataset to Queue"
- "From Dataset to Queue" appears after "Advanced Queue"

### 2. `ui/src/components/JobActionBar.tsx`
**Changes:**
- Fixed context menu text color: Added `text-gray-200` class to menu items for better visibility on dark background
- Added `title` attributes (tooltips) to all action buttons:
  - Play button: "Start Job"
  - Pause button: "Stop Job"
  - X button: "Remove from Queue"
  - Eye button: "View Job"
  - Pen button: "Edit Job"
  - Trash button: "Delete Job"
  - Cog button: "More Options"

---

## New Files to Add

These are completely new files that don't exist in the original AI Toolkit:

### 1. `ui/src/app/batch-queue/page.tsx`
**Purpose:** Main page component for the "From Dataset to Queue" feature

**Description:**
- Allows users to select a training template and multiple datasets
- Filters datasets to show only those that haven't been trained yet
- Creates batch jobs and redirects to Advanced Queue after submission
- Includes template selection, dataset filtering, and multi-select functionality

**Note:** This file redirects to `/jobs-extended` after batch job creation (not the original `/jobs` route)

### 2. `ui/src/app/jobs-extended/page.tsx`
**Purpose:** Main page component for the Advanced Queue route

**Description:** 
- Renders the Advanced Queue page with TopBar and MainContent layout
- Uses `JobsTableExtended` component to display the enhanced queue interface

### 3. `ui/src/components/JobsTableExtended.tsx`
**Purpose:** Enhanced version of JobsTable with advanced features

**Features:**
- Pagination controls for idle entries
- Search input for filtering by job name
- Status filter dropdown
- Bulk action buttons (Stop All Queued, Resume Stopped)
- All original JobsTable functionality (GPU sections, queue controls, etc.)

**Dependencies:**
- Uses existing hooks: `useJobsList`, `useQueueList`, `useGPUInfo`
- Uses existing components: `UniversalTable`, `JobActionBar`
- Uses existing utilities: `startQueue`, `stopQueue`, `apiClient`

### 4. `ui/src/app/api/jobs/bulk/route.ts`
**Purpose:** API endpoint for bulk job operations

**Endpoints:**
- `POST /api/jobs/bulk` with action parameter:
  - `action: 'stop-queued'` - Stops all queued jobs (marks as stopped)
  - `action: 'resume-stopped'` - Resumes all stopped jobs (changes to queued)

**Request Body:**
```json
{
  "action": "stop-queued" | "resume-stopped"
}
```

**Response:**
```json
{
  "success": true,
  "count": 5,
  "message": "Stopped 5 queued job(s)"
}
```

### 5. `ui/src/app/api/batch-queue/route.ts`
**Purpose:** API endpoint for batch job creation from datasets

**Endpoints:**
- `POST /api/batch-queue` - Creates multiple training jobs from template and dataset list

**Request Body:**
```json
{
  "template": "wan",
  "datasets": ["aaliyah", "abbiecornish"],
  "gpu_ids": "0"
}
```

**Response:**
```json
{
  "created": 2,
  "errors": 0,
  "errorDetails": []
}
```

### 6. `ui/src/app/api/templates/list/route.ts`
**Purpose:** API endpoint to list available training templates

**Endpoints:**
- `GET /api/templates/list` - Returns array of template names (e.g., ["wan", "zimage"])

### 7. `ui/src/app/api/templates/[name]/route.ts`
**Purpose:** API endpoint to get template content

**Endpoints:**
- `GET /api/templates/[name]` - Returns YAML content of the specified template

### 8. `ui/src/app/api/output/list/route.ts`
**Purpose:** API endpoint to list existing output folders

**Endpoints:**
- `GET /api/output/list` - Returns array of output folder names

### 9. `ui/src/hooks/useTemplateList.tsx`
**Purpose:** React hook to fetch and cache available templates list

### 10. `ui/src/hooks/useOutputList.tsx`
**Purpose:** React hook to fetch and cache output folder names

---

## Feature Details

### Advanced Queue Features

#### Pagination
- Default page size: 25 entries
- Options: 25, 50, 100, 250, 500, 1000, 2000, 5000, or "all"
- Only applies to the "Idle" section (completed/stopped/failed jobs)
- Active jobs (running/queued) are shown in GPU-specific sections without pagination

#### Search
- Real-time filtering by job name
- Case-insensitive search
- Works in combination with status filter
- Resets to page 1 when search query changes

#### Status Filter
- Filter options: "all" plus all unique statuses found in idle jobs
- Common statuses: completed, stopped, failed, error
- Works in combination with search

#### Bulk Actions

**Stop All Queued Jobs:**
- Finds all jobs with status `'queued'`
- Marks them as `'stopped'` (does NOT affect running jobs)
- Shows confirmation dialog before execution
- Displays success message with count of stopped jobs

**Resume Stopped Jobs:**
- Finds all jobs with status `'stopped'`
- Changes status to `'queued'`
- Assigns proper queue positions (increments of 1000)
- Ensures queue exists for each job's GPU
- Shows confirmation dialog before execution
- Displays success message with count of resumed jobs

### From Dataset to Queue Features

This is a new feature that allows batch creation of training jobs from datasets using templates (see `BATCH_QUEUE_FEATURE.md` for full details).

**Key Components:**
- Template selection dropdown
- Dataset filtering (excludes already trained datasets)
- Multi-select checkboxes
- Batch job creation
- Error handling for duplicate job names
- Redirects to Advanced Queue after submission (not the original `/jobs` route)

**API Dependencies:**
- `/api/templates/list` - Lists available templates
- `/api/templates/[name]` - Gets template YAML content
- `/api/output/list` - Lists existing output folders
- `/api/batch-queue` - Creates batch jobs from template and datasets

---

## Installation Instructions

To apply these extended features to an unchanged AI Toolkit:

### Step 1: Backup Original Files
Before making changes, backup the following files:
```bash
cp ui/src/components/Sidebar.tsx ui/src/components/Sidebar.tsx.backup
cp ui/src/components/JobActionBar.tsx ui/src/components/JobActionBar.tsx.backup
```

### Step 2: Override Modified Files
Replace the following files with the extended versions:

1. **`ui/src/components/Sidebar.tsx`**
   - Copy the extended version that includes both "Advanced Queue" and "From Dataset to Queue" menu items

2. **`ui/src/components/JobActionBar.tsx`**
   - Copy the extended version with fixed menu colors and tooltips

### Step 3: Add New Files
Create the following new files:

#### Frontend Pages:
1. **`ui/src/app/jobs-extended/page.tsx`**
   - Create the directory if it doesn't exist: `ui/src/app/jobs-extended/`
   - Add the page component file

2. **`ui/src/app/batch-queue/page.tsx`**
   - Create the directory if it doesn't exist: `ui/src/app/batch-queue/`
   - Add the batch queue page component

#### Frontend Components:
3. **`ui/src/components/JobsTableExtended.tsx`**
   - Add the enhanced table component

#### API Routes:
4. **`ui/src/app/api/jobs/bulk/route.ts`**
   - Create the directory structure: `ui/src/app/api/jobs/bulk/`
   - Add the bulk operations API route

5. **`ui/src/app/api/batch-queue/route.ts`**
   - Create the directory structure: `ui/src/app/api/batch-queue/`
   - Add the batch queue API route

6. **`ui/src/app/api/templates/list/route.ts`**
   - Create the directory structure: `ui/src/app/api/templates/list/`
   - Add the templates list API route

7. **`ui/src/app/api/templates/[name]/route.ts`**
   - Create the directory structure: `ui/src/app/api/templates/[name]/`
   - Add the template content API route

8. **`ui/src/app/api/output/list/route.ts`**
   - Create the directory structure: `ui/src/app/api/output/list/`
   - Add the output list API route

#### React Hooks:
9. **`ui/src/hooks/useTemplateList.tsx`**
   - Add the template list hook

10. **`ui/src/hooks/useOutputList.tsx`**
    - Add the output list hook

### Step 4: Verify Dependencies
Ensure all dependencies are installed. The extended features use:
- Existing React hooks (no new dependencies)
- Existing UI components (no new dependencies)
- Existing API utilities (no new dependencies)
- Headless UI components (should already be installed)

### Step 5: Test Installation
1. Start the development server: `npm run dev`
2. Navigate to the sidebar and verify:
   - "Advanced Queue" menu item appears
   - "From Dataset to Queue" menu item exists
3. Test the Advanced Queue:
   - Navigate to `/jobs-extended`
   - Verify pagination controls appear
   - Test search functionality
   - Test status filter
   - Test bulk action buttons
4. Test batch-queue redirect:
   - Submit jobs from batch-queue page
   - Verify redirect to `/jobs-extended` instead of `/jobs`

### Step 6: Database
No database migrations are required. The extended features use existing database schema and tables.

---

## File Structure Summary

```
ui/
├── src/
│   ├── app/
│   │   ├── api/
│   │   │   ├── batch-queue/
│   │   │   │   └── route.ts              [NEW]
│   │   │   ├── jobs/
│   │   │   │   └── bulk/
│   │   │   │       └── route.ts          [NEW]
│   │   │   ├── output/
│   │   │   │   └── list/
│   │   │   │       └── route.ts          [NEW]
│   │   │   └── templates/
│   │   │       ├── list/
│   │   │       │   └── route.ts          [NEW]
│   │   │       └── [name]/
│   │   │           └── route.ts           [NEW]
│   │   ├── batch-queue/
│   │   │   └── page.tsx                  [NEW]
│   │   └── jobs-extended/
│   │       └── page.tsx                  [NEW]
│   ├── components/
│   │   ├── JobActionBar.tsx              [MODIFIED - colors + tooltips]
│   │   ├── JobsTableExtended.tsx         [NEW]
│   │   └── Sidebar.tsx                   [MODIFIED - new menu items]
│   └── hooks/
│       ├── useTemplateList.tsx           [NEW]
│       └── useOutputList.tsx             [NEW]
└── README_EXTENDED.md                    [NEW - this file]
```

---

## Notes

- **Minimal Changes Philosophy**: All changes were designed to minimize modifications to the original codebase
- **No Breaking Changes**: All extended features are additive and don't break existing functionality
- **Backward Compatible**: The original `/jobs` route and `JobsTable` component remain unchanged
- **Shared Components**: Extended features reuse existing components, hooks, and utilities where possible

---

## Troubleshooting

### Issue: Advanced Queue page not found (404)
**Solution:** Ensure `ui/src/app/jobs-extended/page.tsx` exists and the directory structure is correct.

### Issue: Bulk actions not working
**Solution:** 
- Verify `ui/src/app/api/jobs/bulk/route.ts` exists
- Check that the API route is accessible at `/api/jobs/bulk`
- Check browser console for errors

### Issue: Menu text not visible in context menu
**Solution:** Ensure `JobActionBar.tsx` has `text-gray-200` class on menu items.

### Issue: Tooltips not showing
**Solution:** Verify `title` attributes are present on all buttons in `JobActionBar.tsx`.

---

## Future Enhancements

Potential improvements for the Advanced Queue:
- Export filtered results
- Advanced sorting options
- Column visibility toggles
- Bulk edit capabilities
- Job grouping by template or dataset