How to Scale Videos: Parallel Processing, Messenger, and More

Processing video is one of the heaviest tasks a web application can handle. If you are still running shell_exec(‘ffmpeg …’) inside a Controller, you are likely blocking your PHP-FPM threads, frustrating your users, and risking timeouts.

\ In 2026, we don’t do that. We treat video processing as an asynchronous, distributed pipeline.

\ With the release of Symfony 7.4, we have new tools that make this robust and native. We now have a dedicated #[Video] validation constraint, native support for shared directories, and the mature Messenger component to orchestrate parallel pipelines.

\ In this article, we will build a production-grade video processing architecture that:

1. Validates uploads instantly using the new Symfony 7.4 constraints.
2. Offloads heavy lifting to background workers.
3. Parallelizes tasks (transcoding, thumbnails, analysis) simultaneously.

Prerequisites

1. PHP 8.2+ (PHP 8.4 recommended).
2. Symfony 7.4 (symfony/skeleton & symfony/webapp-pack).
3. FFmpeg installed on your server/container (sudo apt install ffmpeg).
4. Composer packages: \n php-ffmpeg/php-ffmpeg(v1.3+) \n symfony/messenger\symfony/validator

The Architecture: Shared Directory Strategy

Before writing code, we must solve the physical storage problem. When a user uploads a video to your Web Container, your Worker Container (which might be on a different server) needs to access it.

\ In Symfony 7.4, we lean into the “Shared Directory” pattern — a standardized location for stateful data shared across nodes (like NFS mounts or Docker Volumes).

\ Infrastructure setup (Conceptual):

• Web Node: Mounts /mnt/shared_storage to ./var/storage.
• Worker Node: Mounts the same volume to ./var/storage.

\ We will configure this path in services.yaml to ensure our code is agnostic of the physical location.

# config/services.yaml parameters: # The absolute path to the shared storage (mapped volume) app.storage_dir: '%kernel.project_dir%/var/storage' services: _defaults: autowire: true autoconfigure: true bind: $storageDir: '%app.storage_dir%'

Validation: The New #[Video] Constraint

In previous versions, we had to rely on the generic File constraint and guess MIME types, or write complex custom validators to check duration and codecs.

\ Symfony 7.4 introduces the native #[Video] constraint. It uses FFmpeg internals (via ffprobe) to validate metadata before you even accept the business logic.

\ Let’s create a DTO for our upload.

// src/Dto/VideoUploadDto.php namespace App\Dto; use Symfony\Component\HttpFoundation\File\UploadedFile; use Symfony\Component\Validator\Constraints as Assert; final readonly class VideoUploadDto { public function __construct( #[Assert\NotBlank] #[Assert\Video( maxSize: '500M', mimeTypes: ['video/mp4', 'video/quicktime', 'video/webm'], minWidth: 1280, maxWidth: 3840, // 4K limit minDuration: 5, // Seconds maxDuration: 3600, allowPortrait: false, // Enforce landscape suggestedExtensions: ['mp4', 'mov'] )] public UploadedFile $file, #[Assert\NotBlank] #[Assert\Length(min: 5, max: 255)] public string $title ) {} }

This constraint requires the ffprobe binary to be executable by the web user.

The Orchestrator: Messenger Pipelines

To process video efficiently, we shouldn’t just run one giant script. We should split the work into Pipelines. When a video is uploaded, we will dispatch a “Manager Message,” which then dispatches sub-tasks to be run in parallel.

The Messages (DTOs)

We use readonly classes for immutable message objects.

// src/Message/ProcessVideoUpload.php namespace App\Message; /** * The Trigger: Dispatched immediately after upload. */ final readonly class ProcessVideoUpload { public function __construct( public string $videoId, public string $filename ) {} }

// src/Message/TranscodeVideo.php namespace App\Message; /** * Sub-Task: Heavy encoding work. */ final readonly class TranscodeVideo { public function __construct( public string $videoId, public string $filename, public string $targetFormat // e.g., 'hls', 'mp4-720p' ) {} }

// src/Message/GenerateThumbnail.php namespace App\Message; /** * Sub-Task: Image extraction. */ final readonly class GenerateThumbnail { public function __construct( public string $videoId, public string $filename, public int $timestamp ) {} }

Configuring Transports

We need at least two transports:

1. async_priority: For fast tasks (thumbnails, database updates).
2. async_heavy: For long-running tasks (transcoding). This prevents a 10-minute 4K encode from blocking your thumbnail generation.

# config/packages/messenger.yaml framework: messenger: failure_transport: failed transports: # Fast lane async_priority: dsn: '%env(MESSENGER_TRANSPORT_DSN)%' options: queue_name: priority_queue # Slow lane (Heavy video processing) async_heavy: dsn: '%env(MESSENGER_TRANSPORT_DSN)%' options: queue_name: video_encoding_queue # Increase timeout for workers on this queue auto_setup: true failed: 'doctrine://default?queue_name=failed' routing: 'App\Message\ProcessVideoUpload': async_priority 'App\Message\GenerateThumbnail': async_priority 'App\Message\TranscodeVideo': async_heavy

The Processing Logic

We will use the php-ffmpeg library. First, install it:

composer require php-ffmpeg/php-ffmpeg

The Manager Handler

This handler receives the initial upload event and “fans out” the work. This is the Parallel Pipeline pattern.

// src/MessageHandler/ProcessVideoUploadHandler.php namespace App\MessageHandler; use App\Message\GenerateThumbnail; use App\Message\ProcessVideoUpload; use App\Message\TranscodeVideo; use Psr\Log\LoggerInterface; use Symfony\Component\Messenger\Attribute\AsMessageHandler; use Symfony\Component\Messenger\MessageBusInterface; #[AsMessageHandler] final readonly class ProcessVideoUploadHandler { public function __construct( private MessageBusInterface $bus, private LoggerInterface $logger ) {} public function __invoke(ProcessVideoUpload $message): void { $this->logger->info("Starting pipeline for video: {$message->videoId}"); // 1. Dispatch Thumbnail Generation (Fast) // We generate 3 thumbnails in parallel by dispatching 3 messages $this->bus->dispatch(new GenerateThumbnail($message->videoId, $message->filename, 5)); $this->bus->dispatch(new GenerateThumbnail($message->videoId, $message->filename, 30)); $this->bus->dispatch(new GenerateThumbnail($message->videoId, $message->filename, 60)); // 2. Dispatch Transcoding (Slow/Heavy) // These will go to the 'async_heavy' transport $this->bus->dispatch(new TranscodeVideo($message->videoId, $message->filename, 'mp4-720p')); $this->bus->dispatch(new TranscodeVideo($message->videoId, $message->filename, 'webm-720p')); $this->logger->info("Pipeline dispatched successfully."); } }

The Heavy Worker Handler

Now, we implement the actual logic using FFmpeg.

// src/MessageHandler/TranscodeVideoHandler.php namespace App\MessageHandler; use App\Message\TranscodeVideo; use FFMpeg\FFMpeg; use FFMpeg\Format\Video\X264; use Psr\Log\LoggerInterface; use Symfony\Component\Filesystem\Filesystem; use Symfony\Component\Messenger\Attribute\AsMessageHandler; #[AsMessageHandler] final readonly class TranscodeVideoHandler { public function __construct( private string $storageDir, private LoggerInterface $logger, ) {} public function __invoke(TranscodeVideo $message): void { $inputFile = $this->storageDir . '/' . $message->filename; $outputFile = $this->storageDir . '/processed/' . $message->videoId . '-' . $message->targetFormat . '.mp4'; // 1. Verify existence (Robustness) if (!file_exists($inputFile)) { // In a shared dir setup, there might be a slight sync delay or error. // Throwing exception triggers Messenger retry policy. throw new \RuntimeException("File not found: $inputFile"); } $this->logger->info("Transcoding {$message->videoId} to {$message->targetFormat}..."); // 2. Initialize FFMpeg $ffmpeg = FFMpeg::create(); $video = $ffmpeg->open($inputFile); // 3. Configure Format (x264 codec, AAC audio) $format = new X264(); $format->setKiloBitrate(1000) ->setAudioChannels(2) ->setAudioKiloBitrate(128); // 4. Save // This is a blocking process that can take minutes. // Because it's in a worker, the user is not waiting. $filesystem = new Filesystem(); $filesystem->mkdir(dirname($outputFile)); $video->save($format, $outputFile); $this->logger->info("Transcoding complete: $outputFile"); } }

The Controller: Tying It Together

The controller’s job is now incredibly simple:

Validate -> Move to Storage -> Dispatch -> Return 202 Accepted.

// src/Controller/VideoController.php namespace App\Controller; use App\Dto\VideoUploadDto; use App\Message\ProcessVideoUpload; use Symfony\Bundle\FrameworkBundle\Controller\AbstractController; use Symfony\Component\HttpFoundation\JsonResponse; use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpKernel\Attribute\MapUploadedFile; use Symfony\Component\Messenger\MessageBusInterface; use Symfony\Component\Routing\Attribute\Route; use Symfony\Component\Uid\Uuid; #[Route('/api/videos')] class VideoController extends AbstractController { public function __construct( private string $storageDir, private MessageBusInterface $bus ) {} #[Route('/upload', methods: ['POST'])] public function upload( // Validates automatically using our DTO constraints #[MapUploadedFile] VideoUploadDto $uploadDto ): JsonResponse { $file = $uploadDto->file; $videoId = Uuid::v7()->toRfc4122(); // 1. Move file to Shared Directory // We use the ID as the filename to avoid collisions $filename = $videoId . '.' . $file->guessExtension(); $file->move($this->storageDir, $filename); // 2. Dispatch the Manager Message $this->bus->dispatch(new ProcessVideoUpload($videoId, $filename)); // 3. Immediate Response return $this->json([ 'status' => 'processing', 'id' => $videoId, 'message' => 'Video accepted. Processing pipelines initiated.' ], 202); } }

Running the Pipelines

To see this parallelism in action, you need to run your workers. In a production environment (like Kubernetes or Docker Swarm), you would scale these deployments independently.

\ Terminal 1 (The Priority Worker): Handles the dispatching and thumbnails.

php bin/console messenger:consume async_priority -vv

\ Terminal 2 (The Heavy Worker): Handles the actual video encoding. You might run 4 or 5 of these containers.

php bin/console messenger:consume async_heavy -vv

Verification

1. POST a 50MB MP4 file to /api/videos/upload.
2. Observe Terminal 1: It receives ProcessVideoUpload, then almost instantly dispatches 5 new messages (3 thumbnails, 2 transcodes). It then processes the thumbnails.
3. Observe Terminal 2: It picks up TranscodeVideo. You will see the CPU usage rise as FFmpeg runs.
4. Check your ./var/storage/processed/ directory. You will see the thumbnails appear quickly, followed later by the transcoded video files.

Conclusion

By leveraging Symfony 7.4, we have transformed a complex problem into a clean, manageable architecture.

• Reliability: The #[Video] constraint ensures no invalid files ever enter our pipeline.
• Scalability: By separating asyncpriority and asyncheavy, thumbnail generation is never blocked by large video encoding.
• Maintainability: The logic is decoupled. You can change the transcoding library or the storage engine (e.g., to S3 using Flysystem) without touching the Controller or the Validation logic.

\ This architecture is “production-ready” but allows for growth. As you scale, you might replace the local Shared Directory with an Object Storage abstraction (using league/flysystem-aws-s3-v3), but the Messenger pipeline concepts remain exactly the same.

\ Video processing doesn’t have to be scary. With the right constraints and queue architecture, it becomes predictable and observable.

\ Have questions about scaling Symfony pipelines? Let’s connect. I share daily tips on modern PHP architecture.

\ LinkedIn: Connect with me [https://www.linkedin.com/in/matthew-mochalkin/]

\