Files

johnny2211 d3c90ea447 Add video sentence feature with playlist functionality

2026-01-18 14:56:28 +01:00

19 KiB

Raw Blame History

Video Rečenica (Video Sentence Player) - Implementation Guide

Overview

The Video Rečenica feature allows users to watch sequential playback of sign language videos for complete sentences. Each word's video plays in order with synchronized highlighting of the current token.

Architecture

Data Flow

User navigates to Video Rečenica
    ↓
Loads sentence from:
  - sentenceStore (current working sentence from Znakopis)
  - OR selected document/page from Oblak
    ↓
For each token in sentence:
  - Get term data (already includes media from database)
  - Extract video URL from term.media array
  - Build playlist of videos
    ↓
Sequential video playback:
  - Play video for token[0]
  - Highlight token[0] in sentence list
  - On video end → advance to token[1]
  - Repeat until all tokens played
    ↓
User controls: play/pause, next/prev, speed, loop, autoplay

Implementation Tasks

1. Backend API (Optional - Can use existing data)

Note: The existing document API already returns all necessary data (tokens with terms and media). No new backend endpoint is strictly required, but you may optionally create a playlist helper endpoint.

Optional Endpoint: GET /api/playlists/generate

Query params: sentenceId or documentId + pageIndex + sentenceIndex
Returns: Array of playlist items

{
  playlist: [
    {
      tokenId: string,
      termId: string,
      displayText: string,
      videoUrl: string,
      durationMs: number
    }
  ]
}

Alternative: Use existing document API and transform data on frontend.

2. Frontend Components

A. Create `VideoSentence.tsx` Page

Location: packages/frontend/src/pages/VideoSentence.tsx

Layout:

Two-column grid: 60% video player (left) + 40% sentence panel (right)
Responsive: stack vertically on mobile

State Management:

- currentTokens: SentenceToken[] (from sentenceStore or loaded document)
- currentTokenIndex: number (which token is playing)
- isPlaying: boolean
- playbackSpeed: number (0.5, 0.75, 1.0, 1.25, 1.5, 2.0)
- loopMode: 'none' | 'sentence' | 'word'
- autoplay: boolean

Key Features:

Load sentence from sentenceStore on mount
Option to load from saved documents (document selector)
Build video playlist from tokens
Handle video playback state
Sync highlighting with current video

B. Create `VideoPlayer.tsx` Component

Location: packages/frontend/src/components/video-sentence/VideoPlayer.tsx

Props:

interface VideoPlayerProps {
  videoUrl: string | null;
  isPlaying: boolean;
  playbackSpeed: number;
  onVideoEnd: () => void;
  onPlayPause: () => void;
  onNext: () => void;
  onPrevious: () => void;
}

Features:

Large video element with controls
Custom control bar (play/pause, next/prev, speed selector)
Preload next video for smooth transitions
Handle video errors gracefully (show placeholder if no video)
Display current word being signed
Fullscreen support (optional)

Implementation Notes:

Use HTML5 <video> element
Set playbackRate property for speed control
Listen to ended event to advance to next token
Use ref to control video programmatically
Preload strategy: Create hidden video element for next video

C. Create `SentencePanel.tsx` Component

Location: packages/frontend/src/components/video-sentence/SentencePanel.tsx

Props:

interface SentencePanelProps {
  tokens: SentenceToken[];
  currentTokenIndex: number;
  onTokenClick: (index: number) => void;
  selectedDocument: Document | null;
  currentPageIndex: number;
  onLoadDocument: (documentId: string) => void;
  onPageChange: (pageIndex: number) => void;
}

Features:

Display all tokens in sentence
Highlight current token (e.g., bold, colored background, border)
Allow clicking tokens to jump to that position
Show document selector (reuse from Znakopis)
Show page navigation if multi-page document
Display sentence list if multiple sentences on page

Visual Design:

Token list: horizontal wrap or vertical list
Current token: distinct visual treatment (e.g., orange background, bold)
Clickable tokens: hover effect
Empty state: "Nema učitane rečenice" with instructions

D. Create `PlaybackControls.tsx` Component

Location: packages/frontend/src/components/video-sentence/PlaybackControls.tsx

Props:

interface PlaybackControlsProps {
  isPlaying: boolean;
  playbackSpeed: number;
  loopMode: 'none' | 'sentence' | 'word';
  autoplay: boolean;
  onPlayPause: () => void;
  onNext: () => void;
  onPrevious: () => void;
  onSpeedChange: (speed: number) => void;
  onLoopModeChange: (mode: 'none' | 'sentence' | 'word') => void;
  onAutoplayToggle: () => void;
  canGoNext: boolean;
  canGoPrevious: boolean;
}

Features:

Play/Pause button (large, prominent)
Previous/Next buttons
Speed selector dropdown (0.5x, 0.75x, 1x, 1.25x, 1.5x, 2x)
Loop mode toggle (none, loop sentence, loop word)
Autoplay toggle
Disable next/previous when at boundaries

Icons: Use lucide-react icons (Play, Pause, SkipForward, SkipBack, Repeat, Gauge)

3. Routing

Update: packages/frontend/src/App.tsx

Replace placeholder route:

<Route path="/video-sentence" element={<ProtectedRoute><VideoSentence /></ProtectedRoute>} />

4. Video Playlist Logic

Helper Function: Create utility to build playlist from tokens

Location: packages/frontend/src/lib/videoPlaylist.ts

import { SentenceToken } from '../stores/sentenceStore';

export interface PlaylistItem {
  tokenId: string;
  termId: string;
  displayText: string;
  videoUrl: string | null;
  durationMs: number | null;
}

export function buildPlaylist(tokens: SentenceToken[]): PlaylistItem[] {
  return tokens.map(token => {
    // Get first video media for the term
    const videoMedia = token.term.media?.find(m => m.mediaType === 'VIDEO');

    return {
      tokenId: token.id,
      termId: token.termId,
      displayText: token.displayText,
      videoUrl: videoMedia?.url || null,
      durationMs: videoMedia?.durationMs || null,
    };
  });
}

export function getVideoUrl(relativeUrl: string | null): string | null {
  if (!relativeUrl) return null;

  // If already absolute URL, return as-is
  if (relativeUrl.startsWith('http')) return relativeUrl;

  // Construct full URL using backend base URL
  const baseUrl = import.meta.env.VITE_API_URL || 'http://localhost:3000';
  return `${baseUrl}${relativeUrl}`;
}

5. State Management

Option A: Use local component state (recommended for simplicity)

Keep playback state in VideoSentence component
Pass down via props

Option B: Create videoPlayerStore (if needed for cross-component state)

// packages/frontend/src/stores/videoPlayerStore.ts
interface VideoPlayerState {
  currentTokenIndex: number;
  isPlaying: boolean;
  playbackSpeed: number;
  loopMode: 'none' | 'sentence' | 'word';
  autoplay: boolean;
  // actions...
}

6. Key Implementation Details

Video Playback Flow

// In VideoSentence.tsx
const handleVideoEnd = () => {
  if (loopMode === 'word') {
    // Replay current video
    videoRef.current?.play();
    return;
  }

  if (currentTokenIndex < tokens.length - 1) {
    // Advance to next token
    setCurrentTokenIndex(prev => prev + 1);
  } else {
    // End of sentence
    if (loopMode === 'sentence') {
      // Restart from beginning
      setCurrentTokenIndex(0);
    } else {
      // Stop playback
      setIsPlaying(false);
    }
  }
};

// Auto-play next video when index changes
useEffect(() => {
  if (isPlaying && videoRef.current) {
    videoRef.current.play();
  }
}, [currentTokenIndex, isPlaying]);

Preloading Next Video

// Create hidden video element for next video
const [nextVideoUrl, setNextVideoUrl] = useState<string | null>(null);

useEffect(() => {
  if (currentTokenIndex < playlist.length - 1) {
    const nextItem = playlist[currentTokenIndex + 1];
    setNextVideoUrl(getVideoUrl(nextItem.videoUrl));
  }
}, [currentTokenIndex, playlist]);

// In render:
{nextVideoUrl && (
  <video
    src={nextVideoUrl}
    preload="auto"
    style={{ display: 'none' }}
  />
)}

Error Handling

// Handle missing videos gracefully
const currentVideo = playlist[currentTokenIndex];

if (!currentVideo.videoUrl) {
  // Show placeholder: "Video nije dostupan za ovu riječ"
  // Display word text prominently
  // Auto-advance after 2 seconds
  setTimeout(() => handleVideoEnd(), 2000);
}

// Handle video load errors
<video
  onError={(e) => {
    console.error('Video load error:', e);
    toast.error(`Greška pri učitavanju videa za "${currentVideo.displayText}"`);
    // Auto-advance to next video
    handleVideoEnd();
  }}
/>

7. UI/UX Considerations

Empty State

When no sentence is loaded:

┌─────────────────────────────────────┐
│                                     │
│   📹 Video Rečenica                 │
│                                     │
│   Nema učitane rečenice             │
│                                     │
│   Kako započeti:                    │
│   1. Dodajte riječi u Rječniku      │
│   2. Uredite rečenicu u Znakopisu   │
│   3. Vratite se ovdje za reprodukciju│
│                                     │
│   ILI                               │
│                                     │
│   [Učitaj spremljeni dokument]      │
│                                     │
└─────────────────────────────────────┘

Active Playback Layout

┌──────────────────────────────────────────────────────────────┐
│  Video Rečenica                                              │
├────────────────────────────┬─────────────────────────────────┤
│                            │  Dokument: Moja rečenica        │
│                            │  Stranica: 1 / 2                │
│                            │                                 │
│   ┌──────────────────────┐ │  Rečenica 1:                    │
│   │                      │ │  ┌───────────────────────────┐  │
│   │                      │ │  │ Ja      volim     školu   │  │
│   │   [VIDEO PLAYER]     │ │  │ [ACTIVE] [normal] [normal]│  │
│   │                      │ │  └───────────────────────────┘  │
│   │   Currently: "Ja"    │ │                                 │
│   │                      │ │  Rečenica 2:                    │
│   └──────────────────────┘ │  ┌───────────────────────────┐  │
│                            │  │ Učitelj  uči   učenike    │  │
│   ┌──────────────────────┐ │  │ [normal] [normal] [normal]│  │
│   │ [◀] [▶] [⏸] [🔁] [⚙]│ │  └───────────────────────────┘  │
│   │ Speed: 1.0x          │ │                                 │
│   └──────────────────────┘ │  [Učitaj drugi dokument ▼]      │
│                            │  [◀ Stranica] [Stranica ▶]      │
└────────────────────────────┴─────────────────────────────────┘

Token Highlighting Styles

/* Normal token */
.token {
  padding: 8px 12px;
  border-radius: 6px;
  background: #f3f4f6;
  cursor: pointer;
  transition: all 0.2s;
}

/* Active/playing token */
.token-active {
  background: #fb923c; /* orange-400 */
  color: white;
  font-weight: 600;
  transform: scale(1.05);
  box-shadow: 0 4px 6px rgba(251, 146, 60, 0.3);
}

/* Hover state */
.token:hover {
  background: #e5e7eb;
  transform: translateY(-2px);
}

8. Testing Checklist

Basic Functionality:

Load sentence from sentenceStore
Load sentence from saved document
Play button starts video playback
Pause button pauses video
Videos play sequentially in correct order
Current token is highlighted during playback
Highlighting updates when video changes

Navigation:

Next button skips to next video
Previous button goes to previous video
Clicking a token jumps to that video
Next/Previous disabled at boundaries

Playback Controls:

Speed control changes playback rate (0.5x - 2x)
Loop sentence restarts from beginning after last video
Loop word replays current video continuously
Autoplay automatically starts playback when sentence loads

Multi-sentence/Multi-page:

Can switch between sentences on same page
Can navigate between pages
Playback state resets when changing sentence/page
Document selector loads different documents

Error Handling:

Missing video shows placeholder
Video load error shows toast and auto-advances
Empty sentence shows helpful message
No crash when term has no media

Performance:

Next video preloads for smooth transition
No lag when switching videos
Responsive on mobile devices

9. File Structure Summary

packages/frontend/src/
├── pages/
│   └── VideoSentence.tsx          # Main page component
├── components/
│   └── video-sentence/
│       ├── VideoPlayer.tsx        # Video player with controls
│       ├── SentencePanel.tsx      # Sentence list with highlighting
│       └── PlaybackControls.tsx   # Playback control buttons
├── lib/
│   └── videoPlaylist.ts           # Playlist utilities
└── stores/
    └── videoPlayerStore.ts        # (Optional) Playback state store

10. Implementation Order

Phase 1: Basic Structure
- Create VideoSentence.tsx page with layout
- Add route in App.tsx
- Create basic VideoPlayer component
- Create basic SentencePanel component
- Test with sentenceStore data
Phase 2: Playback Logic
- Implement video playback state management
- Add sequential playback (auto-advance on video end)
- Add token highlighting sync
- Create videoPlaylist.ts utilities
Phase 3: Controls
- Create PlaybackControls component
- Implement play/pause, next/previous
- Add speed control
- Add loop modes
Phase 4: Document Integration
- Add document selector to SentencePanel
- Implement loading from saved documents
- Add page navigation
- Add sentence selection (if multiple sentences)
Phase 5: Polish
- Add video preloading
- Improve error handling
- Add empty states
- Responsive design
- Performance optimization

11. Code Examples

VideoSentence.tsx (Skeleton)

import { useState, useEffect } from 'react';
import { Layout } from '../components/layout/Layout';
import { VideoPlayer } from '../components/video-sentence/VideoPlayer';
import { SentencePanel } from '../components/video-sentence/SentencePanel';
import { PlaybackControls } from '../components/video-sentence/PlaybackControls';
import { useSentenceStore } from '../stores/sentenceStore';
import { buildPlaylist, getVideoUrl } from '../lib/videoPlaylist';

function VideoSentence() {
  const { currentTokens } = useSentenceStore();
  const [currentTokenIndex, setCurrentTokenIndex] = useState(0);
  const [isPlaying, setIsPlaying] = useState(false);
  const [playbackSpeed, setPlaybackSpeed] = useState(1.0);
  const [loopMode, setLoopMode] = useState<'none' | 'sentence' | 'word'>('none');

  const playlist = buildPlaylist(currentTokens);
  const currentItem = playlist[currentTokenIndex];
  const videoUrl = currentItem ? getVideoUrl(currentItem.videoUrl) : null;

  const handleVideoEnd = () => {
    // Implementation from section 6
  };

  const handleNext = () => {
    if (currentTokenIndex < playlist.length - 1) {
      setCurrentTokenIndex(prev => prev + 1);
    }
  };

  const handlePrevious = () => {
    if (currentTokenIndex > 0) {
      setCurrentTokenIndex(prev => prev - 1);
    }
  };

  return (
    <Layout>
      <div className="space-y-6">
        <h1 className="text-3xl font-bold">Video Rečenica</h1>

        <div className="grid grid-cols-1 lg:grid-cols-5 gap-6">
          {/* Video Player - 60% */}
          <div className="lg:col-span-3">
            <VideoPlayer
              videoUrl={videoUrl}
              isPlaying={isPlaying}
              playbackSpeed={playbackSpeed}
              onVideoEnd={handleVideoEnd}
              onPlayPause={() => setIsPlaying(!isPlaying)}
              onNext={handleNext}
              onPrevious={handlePrevious}
            />
            <PlaybackControls
              isPlaying={isPlaying}
              playbackSpeed={playbackSpeed}
              loopMode={loopMode}
              onPlayPause={() => setIsPlaying(!isPlaying)}
              onNext={handleNext}
              onPrevious={handlePrevious}
              onSpeedChange={setPlaybackSpeed}
              onLoopModeChange={setLoopMode}
              canGoNext={currentTokenIndex < playlist.length - 1}
              canGoPrevious={currentTokenIndex > 0}
            />
          </div>

          {/* Sentence Panel - 40% */}
          <div className="lg:col-span-2">
            <SentencePanel
              tokens={currentTokens}
              currentTokenIndex={currentTokenIndex}
              onTokenClick={setCurrentTokenIndex}
            />
          </div>
        </div>
      </div>
    </Layout>
  );
}

export default VideoSentence;

12. Croatian Translations

Video Rečenica - Video Sentence
Reprodukcija - Playback
Brzina - Speed
Ponavljanje - Loop/Repeat
Automatska reprodukcija - Autoplay
Nema učitane rečenice - No sentence loaded
Trenutno - Currently
Prethodno - Previous
Sljedeće - Next
Pauza - Pause
Reproduciraj - Play
Video nije dostupan - Video not available

Summary

This implementation guide provides a complete roadmap for building the Video Rečenica feature. The key aspects are:

Sequential video playback - Videos play one after another automatically
Synchronized highlighting - Current token is visually highlighted
Playback controls - Play/pause, next/prev, speed, loop modes
Document integration - Load from sentenceStore or saved documents
Error handling - Graceful handling of missing videos
Smooth UX - Preloading, responsive design, clear visual feedback

Start with Phase 1 (basic structure) and progressively add features. Test thoroughly at each phase before moving to the next.

19 KiB Raw Blame History

Video Rečenica (Video Sentence Player) - Implementation Guide

Overview

Architecture

Data Flow

Implementation Tasks

1. Backend API (Optional - Can use existing data)

2. Frontend Components

A. Create VideoSentence.tsx Page

B. Create VideoPlayer.tsx Component

C. Create SentencePanel.tsx Component

D. Create PlaybackControls.tsx Component

3. Routing

4. Video Playlist Logic

5. State Management

6. Key Implementation Details

Video Playback Flow

Preloading Next Video

Error Handling

7. UI/UX Considerations

Empty State

Active Playback Layout

Token Highlighting Styles

8. Testing Checklist

9. File Structure Summary

10. Implementation Order

11. Code Examples

VideoSentence.tsx (Skeleton)

12. Croatian Translations

Summary

19 KiB

Raw Blame History

A. Create `VideoSentence.tsx` Page

B. Create `VideoPlayer.tsx` Component

C. Create `SentencePanel.tsx` Component

D. Create `PlaybackControls.tsx` Component