Play Video With ESP32




Introduction: Play Video With ESP32

About: Make it yourself if you cannot buy one!

This Instructables show something about playing video and audio with ESP32.

Step 1: ESP32 Features & Limitations


  • 4 SPI bus, 2 SPI bus available for user space, they are SPI2 and SPI3 or called HSPI and VSPI. Both SPI buses can run at most 80 MHz. Theoretically it can push 320x240 16-bit color pixels to SPI LCD at 60 fps, but it has not yet counted the time overhead required for read and decode the video data.
  • 1-bit / 4-bit SD bus can connect SD card in native protocol
  • I2S internal DAC audio output
  • over 100 KB RAM available for video and audio buffer
  • Fair enough processing power to decode JPEG (play Motion JPEG) and LZW data compression (play Animated GIF)
  • Dual-core version can split read data from SD card, decode and push to SPI LCD into parallel multi-tasks and boost the playback performance


  • not enough internal RAM to have double frame buffer for 320x240 in 16-bit color, it limited the multitask design. It can overcome a bit with external PSRAM though it is slower than internal RAM
  • not enough processing power to decode mp4 video
  • not all ESP32 version have 2 core, the multi-task sample only benefit on dual-core version


Step 2: Video Format


Or called 16-bit color is a raw data format commonly used on the communication between MCU and color display. Each color pixel represented by a 16-bit value, the first 5-bit is red value, following 6-bit is green value and then 5-bit blue value. 16-bit value can make 65536 color variation so it also called 64K colors. So 1 minute 320x240@30 fps video will be sized: 16 * 320 * 240 * 30 * 60 = 2211840000 bits = 276480000 bytes or over 260 MB

Animated GIF

This is a common file format on the web since 1990s. It limit the color variation for each screen up to 256 colors and do not repeat store the pixel that as same color as previous frame. So it can much reduce the file size, especially when each animation frame not change too much details. The LZW compression is designed capable decoded by 1990s computer, so ESP32 also have fair enough processing power to decode it in real time.

Motion JPEG

Or called M-JPEG / MJPEG is a common video compression format for the video capture hardware with limited processing power. It actually simply a concatenation of still JPEG frames. Compare with MPEG or MP4, Motion JPEG no need computationally intensive technique of interframe prediction, every frame is independent. So it requirement less resource to encode and decode.


Step 3: Audio Format


A raw data format for digital audio. ESP32 DAC use 16-bit bit depth, that means each 16-bit data represent a digital sampled analog signal. Most video and song audio commonly use sample rate at 44100 MHz, that means 44100 sampled analog signal for each second. So, 1 minute mono audio PCM raw data will be sized: 16 * 44100 * 60 = 42336000 bits = 5292000 bytes or over 5 MB. The size of stereo audio will be double, i.e. over 10 MB


MPEG Layer 3 is a compressed audio format widely used for song compression since 1990s. It can dramatically reduce file size to under one-tenth of raw PCM format


Step 4: Format Conversion

This project use FFmpeg convert the video into ESP32 readable format.

Please download and install FFmpeg at their official site if not yet:

Convert to PCM audio

ffmpeg -i input.mp4 -f u16be -acodec pcm_u16le -ar 44100 -ac 1 44100_u16le.pcm

Convert to MP3 audio

ffmpeg -i input.mp4 -ar 44100 -ac 1 -q:a 9 44100.mp3

Convert to RGB565

ffmpeg -i input.mp4 -vf "fps=9,scale=-1:176:flags=lanczos,crop=220:in_h:(in_w-220)/2:0" -c:v rawvideo -pix_fmt rgb565be 220_9fps.rgb

Convert to Animated GIF

ffmpeg -i input.mp4 -vf "fps=15,scale=-1:176:flags=lanczos,crop=220:in_h:(in_w-220)/2:0,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop -1 220_15fps.gif

Convert to Motion JPEG

ffmpeg -i input.mp4 -vf "fps=30,scale=-1:176:flags=lanczos,crop=220:in_h:(in_w-220)/2:0" -q:v 9 220_30fps.mjpeg


  • FFmpeg converted Animated GIF can be further optimizered by some web tools, you may search GIF optimizer to find one.

Step 5: Hardware Preparation

ESP32 Dev Board

Any dual-core ESP32 dev board should be ok, this time I am using a TTGO ESP32-Micro.

Color Display

Any color display that Arduino_GFX support should be ok, this time I am using a ILI9225 breakout board with SD card slot.

You can find Arduino_GFX supported color display list at Github:

SD Card

Any SD card should be ok, this time I am using a SanDisk "normal speed" 8 GB micro SD with SD adaptor.


If you want to use headphone only, simply connect headphone pins to pin 26 and GND can listen the audio. Or you can use a tiny amplifier to play audio with speaker.


Some breadboards and breadboard wires

Step 6: SD Interface

ILI9225 LCD breakout board also included a SD crd slot breakout pins. It can be used as SPI bus or 1-bit SD bus. As mentioned in my previous instructables, I prefer using 1-bit SD bus, so this project will base on 1-bit SD bus.

Step 7: Put It Together

The above pictures show the testing platform I am using in this project. The white breadboard is 3D printed, you can download and print it at thingiverse:

The actual connection depends on which hardware you have in hand.

Here are the connection summary:

Vcc     -> LCD Vcc
GND     -> LCD GND
GPIO 2  -> SD D0/MISO -> 1k resistor -> Vcc


Step 8: Program

Arduino IDE

Download and install Arduino IDE if you are not yet do it:

ESP32 Support

Follow the Installation Instructions to add ESP32 support if you re not yet do it:

Arduino_GFX Library

Download latest Arduino_GFX libraries: (press "Clone or Download" -> "Download ZIP")

Import libraries in Arduino IDE. (Arduino IDE "Sketch" Menu -> "Include Library" -> "Add .ZIP Library" -> select downloaded ZIP file)


Download latest ESP8266Audio libraries: (press "Clone or Download" -> "Download ZIP")

Import libraries in Arduino IDE. (Arduino IDE "Sketch" Menu -> "Include Library" -> "Add .ZIP Library" -> select downloaded ZIP file)

RGB565_video Sample Code

Download latest RGB565_video sample code: (press "Clone or Download" -> "Download ZIP")

SD Card Data

Copy the converted files to SD card and insert into LCD card slot

Compile & Upload

  1. Open SDMMC_MJPEG_video_PCM_audio_dualSPI_multitask.ino in Arduino IDE
  2. If you are not using ILI9225, change the new class code (around line 35) to correct class name
  3. Press Arduino IDE "Upload" button
  4. If you failed to upload the program, try detach the connection between ESP32 GPIO 2 and SD D0/MISO
  5. If you find the orientation not correct, change the "rotation" value (0-3) in new class code
  6. If program run well you can try other sample start with SDMMC_*
  7. If you do not have SD card slot or you don't have FFmpeg installed, you can still try SPIFFS_* example

Step 9: Benchmark

Here are the performance summary for different video (220x176) and audio (44100 MHz) format:

FormatFrame per second(fps)
RGB565 + PCM9


  • MJPEG + PCM can reach higher fps but it is unnecessary play in a tiny screen greater than 30 fps
  • RGB565 does not require decode process but the data size is too large and much time consumed at loading data from SD, 4-bit SD bus and faster SD card can improve it a little bit (wild guess can reach around 12 fps)
  • MP3 decode process not yet optimized, it is now dedicate core 0 for MP3 decode and core 1 for playing video

Step 10: Happy Playing!

Now you can play video and audio with your ESP32, it unlocked many possibilities!

I think I will make a tiny vintage TV later...

2 People Made This Project!


  • Puzzles Challenge

    Puzzles Challenge
  • Lamps Challenge

    Lamps Challenge
  • CNC and 3D Printing Contest

    CNC and 3D Printing Contest



3 months ago

Hi, I'm trying to get your SD_GIF_Video example working, but would really appreciate some help with the pinouts, I'm using an Adafruit ESP32 Feather and a ILI9225 screen, but I can adapt the pin numbers once I know what needs to go where. I think I've summarised the key bits in the code extract below from your SD_GIF_Video example that I'm struggling with:

> CS - Is that CS for the ILI9225 or the SD Card?

> SCK - Is that SCK for the ILI9225 or the SD Card?

> What do MOSI and MISO map to on the ILI9225 or the SD card? I have both MOSI and MISO on the SD card but only SDI on the screen?

> What is SS 0?

#define SCK 18

#define MOSI 23

#define MISO 19

#define SS 0

#define TFT_BL 22

// ILI9225 Display

Arduino_DataBus *bus
= new Arduino_ESP32SPI(27 /* DC */, 5 /* CS */, SCK, MOSI, MISO);

Arduino_ILI9225 *gfx
= new Arduino_ILI9225(bus, 33 /* RST */, 3 /* rotation */);

Appreciate any help and advice, many thanks - Richard


Reply 3 months ago

Cs is display CS pin, SS is SD CS pin. All other SPI pins are shared for both devices.


Reply 3 months ago

Thanks, got it working!


Question 4 months ago

Dear Sir,

I try the example "RGB565_video-master/SDMMC_MJPEG_JPEGDEC_MP3_audio"
and have the following problem
(Error compling for board DOIT ESP32 DEVKIT V1.)

C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp: In member function 'bool AudioOutputI2S::SetPinout()':
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp:95:41: error: 'i2s_pin_config_t' has no non-static data member named 'mck_io_num'
.data_in_num = I2S_PIN_NO_CHANGE};
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp: In member function 'bool AudioOutputI2S::begin(bool)':
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp:232:28: error: 'I2S_MCLK_MULTIPLE_DEFAULT' was not declared in this scope
.mclk_multiple = I2S_MCLK_MULTIPLE_DEFAULT, // Unused
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp:233:28: error: 'I2S_BITS_PER_CHAN_DEFAULT' was not declared in this scope
.bits_per_chan = I2S_BITS_PER_CHAN_DEFAULT // Use bits per sample
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputI2S.cpp:234:7: error: 'i2s_config_t' has no non-static data member named 'mclk_multiple'
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp: In constructor 'AudioOutputSPDIF::AudioOutputSPDIF(int, int, int)':
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp:108:22: error: 'I2S_MCLK_MULTIPLE_DEFAULT' was not declared in this scope
.mclk_multiple = I2S_MCLK_MULTIPLE_DEFAULT, // Unused
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp:109:22: error: 'I2S_BITS_PER_CHAN_DEFAULT' was not declared in this scope
.bits_per_chan = I2S_BITS_PER_CHAN_DEFAULT // Use bits per sample
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp:110:3: error: 'i2s_config_t' has no non-static data member named 'mclk_multiple'
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp: In member function 'bool AudioOutputSPDIF::SetPinout(int, int, int)':
C:\Users\User\Documents\Arduino\libraries\ESP8266Audio\src\AudioOutputSPDIF.cpp:159:3: error: 'i2s_pin_config_t' has no non-static data member named 'mck_io_num'
Multiple libraries were found for "WiFi.h"
Used: C:\Users\User\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\WiFi
Not used: C:\Tommy Lai\arduino-1.8.13\libraries\WiFi
Multiple libraries were found for "SD.h"
Used: C:\Users\User\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.6\libraries\SD
Not used: C:\Tommy Lai\arduino-1.8.13\libraries\SD
exit status 1
Error compiling for board DOIT ESP32 DEVKIT V1.

** I am using arduino-1.8.13, and DOIT ESP32 DEVKIT V1
Please help with suggestions at your convenience, thank you.

Thanks & best regards
Tommy Lai


Reply 4 months ago

Thanks for your guidance.


9 months ago



Reply 9 months ago

I have revised SDMMC_MJPEG_JPEGDEC_MP3_audio.ino
You may give it a try.


Reply 9 months ago



Reply 9 months ago

The examples in this project place all code in setup(). If you want repeat play, try move the playing logic to loop(). Some variables should be reused between loop...


Question 10 months ago

Dear Author
My name is "Vvid" trying to integrate Mjpeg movie onto the model track since last Sep (2021) after getting your advices.
This project is mostly finished, and video movie is playbacked succeddfully on the LCD.
Lastly I would like to switch the movie program (video tytle) Prog 1-> Prog B -> so and so.
But after playback of 1 st program, next video program is NOT playback.
What I programed on the scketch is to "goto video start line" with next video tytle.
I imagine I need some "initialization" of Mjpeg playback liblary but I do not know howto.
Would it be possible for you to advise on this issue?

Best Regards


Answer 10 months ago

( Self Solution )
I found an error to cause this issue in MjpegClass.h
Solution is to add following line at the beginning of "class MjpegClass"
in "MjpegClass.h".
" _inputindex = 0; "
After playback some video, this becomes some value, and need to "reset" it to Zero
before starting next video program.


10 months ago

Hello! Thank you so much for posting code for this awesome project! I have been working on getting simultaneous MJPEG + MP3 playback working on an ESP-WROOM-32 and a 1.8" TFT LCD (ST7735) and believe that I have found a bug in the MjpegClass.h code that is present in the "SDMMC_MJPEG_JPEGDEC_MP3_audio" folder. I wanted to share it here in case others run into this issue.

I was having difficulty with the SD_MMC library and switched to using the standard SD library instead. (LCD and SD pins are defined to use the ESP32's hardware SPI pins.) After uploading this code, I was having an issue with getting video and audio playing simultaneously. I could do one or the other, but not both -- having both loaded would result in an ESP32 reset due to watchdog timeout.

In the MjpegClass.h file in this folder, the drawTask was pinned to core 0 instead of core 1. I therefore changed this line to:

xTaskCreatePinnedToCore(drawTask, "drawTask", 1600, &_p, 1, &task, 1);

(Per your text above, I believe that having the drawTask on core 0 was incorrect, since this is the core that is decoding the MP3.) I also needed to set the "enableMultiTask" bool in the mjpeg.setup() call in the main portion of my code to "false."

Doing so allowed everything to work -- with these changes, I am currently getting 20.5 FPS playback for a video encoded at 24FPS (tested using a 23 minute video), which is good enough for my application. Hope this helps someone!

Thanks again for providing the code for this!


Reply 10 months ago

Use Core 1 decode video and Core 0 draw to display can work smooth as you can see at the above demo video. But it is base on SD_MMC 4-bit, SD may be too slow to make it.


Reply 10 months ago

Thank you for your reply! Is it correct that SD_MMC operates at 40MHz SPI frequency? During the SD card initialization, I have specified this as well using the line

if (!SD.begin(SD_CS,SPI,40000000)){...}

but am still only seeing a playback speed of 20.6 frames per second. Therefore I am wondering if SD_MMC would improve over this speed and would be interested to learn more/try it for benchmarking. (With the above line, I am also not successful in getting the Core 1/0 and multi-tasking configuration to work like in your example, so I agree that there definitely seems to be a difference between using SD and SD_MMC.)

I know that you used a different development board for your example above, but do you have any additional advice on getting 1bit SD_MMC working with an ESP-WROOM-32? I have followed your tutorial above and double checked that I am using the correct pins on the ESP32, but am not able to get the SD card to mount. Is there any other advice that you would be able to share?

Thank you for your continued help!


1 year ago

Hi community,
How are you? I hope you are doing well despite this pandemic.
Thanks 陳亮 for posting this instructable.
I need help with a little project.
I just started a course in Arduino I'm a complete newbie.
I saw your tutorial on play video with esp32.
I build plastic kits e.g. robots (Gundams), aircraft, etc.
I'd like to install a 0.96" lcd creen (or similar size) to play videos using Arduino or other microcontrollers.
The link below is what I want to achieve.
On the video above it seems that he presses a button to change the video.
Do you think is possible? Usually, the LCD of this size doesn't have an sd card reader, so I don't have any idea on how to do this.
Anyone up for tutoring?
Please let me know.
Claudio (Juju)


1 year ago

Dear Author
Thank you for your advice on this movie project.
Please lallow me to report something regarding my trial on this.
The Best FR could be 15FPS with 44100 Audio through
SD Para4 and / or SD HSPI(16MHz) to get acceptable IQ
and sound Quality.
My target is to get faster FR with audio through SD.
I have been trying to get LCD Para8 $ SD Para4 by modification of
However I gave it up because I could not solve error regarding
SD_MMS on Audio process.
Then I tried to modify "SD_MJPEG_video_PCM_audio" to get
both of LCD Para8 and SD Para4.
In addition, I compare to the performance between SD Para4 and
When I increase FR from 15 to 20, Image looks better, but playback
speed seems slower than sound.
Image and sound are NOT synchronized.
P.S. If you have some advice to increase FR more with this setting,
I appreciate it.


Reply 1 year ago

I think 15 fps is already very good enough for 320x240 resolution. There is no meaning step up to 20 or 24 fps except it can reach the normal standard, 30 fps.


Reply 1 year ago

Thank you for advice.
I tried to set more than 20MHz clock on SPI , but it was limited below 20MHz by SD library. If I could exceed this, its FR could be increased a littel bit more. Could you plase advise how I can change its library ?
I appreciate it .



Reply 1 year ago

still 4-bit SDMMC is fastest