/_internal

DeepFaceLab code and required packages.

Folders:

Files:

/workspace

Container for all video, image, and model files used in the deepfake project. Place the source and destination videos inside this folder. Place images, faceset, and model files inside appropriate subdirectory.

1) clear workspace.bat

Deletes all data in the workspace subdirectories and rebuilds folder structure. Preserves data_src.* and data_dst.* video files. This file may be deleted in order to prevent accidentally clearing the workspace.

2) extract images from video data_src.bat

Extract source video frame images to /workspace/data_src. Options to set extraction framerate and file type.

• Enter FPS: Sets the framerate (frequency) of extraction. Example: extracting 15 fps from a video that is 30 fps = 15/30 = 1/2 of the frames are extracted.
  [Tooltip: How many frames of every second of the video will be extracted. 0 – full fps.]
• Output Image Format ( png / jpg ): Select compressed JPEG or uncompressed PNG.
  [Tooltip: png is lossless, but extraction is x10 slower for HDD, requires x10 more disk space than jpg.]

3) cut video (drop video on me).bat

Simple video trimmer. Drop the video file directly onto this file to open the options dialogue. The new video file will appear in the same directory as the original file with “_cut” appended to the filename.

• From time: Timecode for the edit in-point.
• To time: Timecode for the edit out-point.
• Specify an audio track id: Select the audio track to use. Inspect file to see alternate audio tracks.
• Bitrate of output file in MB/s: Set the output file bitrate.

3) extract images from video data_dst FULL FPS.bat

Extract data_dst.* destination video frame images to workspace/data_dst. FULL FPS means all video frames will be extracted.

• Output Image Format ( png / jpg ): Select compressed JPG or uncompressed PNG.
  [Tooltip: png is lossless, but extraction is x10 slower for HDD, requires x10 more disk space than jpg.]

3.optional) denoise data_dst images.bat

Denoise destination video frame images.

• Denoise factor ( 1 – 20 ): Set the value (strength) of the denoise algorithm.

4) data_src faceset extract MANUAL.bat

Manual source faceset extractor. Opens interface to manually set faceset alignment landmarks on video frames. Manual mode allows one face selection per frame. Faceset filenames are taken from the original video frame image filename. Console will log the number of images found and faces detected.

• Which GPU index to choose?: Select a single GPU index from the list to run manual extraction.
• Face type ( f / wf / head ): Select the face type for the extraction.
  [Tooltip: Full face / whole face / head. ‘Whole face’ covers full area of face include forehead. ‘head’ covers full head, but requires XSeg for src and dst faceset.]
• Max number of faces from image: Select the maximum number of faces to extract from each frame.
  [Tooltip: If you extract a src faceset that has frames with a large number of faces, it is advisable to set max faces to 3 to speed up extraction. 0 – unlimited]
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]
• Write debug images to aligned_debug? ( y/n ): Choose whether or not to write debug images.

4) data_src faceset extract.bat

Automatic source faceset extractor. Faceset filenames are taken from the original video frame image filename. Console will log the number of images found and faces detected.

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• Face type ( f / wf / head ): Select the face type for the extraction.
  [Tooltip: Full face / whole face / head. ‘Whole face’ covers full area of face include forehead. ‘head’ covers full head, but requires XSeg for src and dst faceset.]
• Max number of faces from image: Select the maximum number of faces to extract from each frame.
  [Tooltip: If you extract a src faceset that has frames with a large number of faces, it is advisable to set max faces to 3 to speed up extraction. 0 – unlimited]
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]
• Write debug images to aligned_debug? ( y/n ): Choose whether or not to write debug images.

4.1) data_src view aligned result.bat

Open source faceset within XNView. Use this image viewer to delete unwanted faceset images.

Read more about XNView on their website.

4.2) data_src sort.bat

Sort the source faceset using various values and algorithms. Faceset image files will be enumerated (renamed) with the new sort order.

Choose Sorting Method:

• [0] blur – Sort by image blurriness based on contrast.
• [1] motion blur – Sort by motion blur.
• [2] face yaw direction – Sort by yaw (horizontal / left-to-right).
• [3] face pitch direction – Sort by pitch (vertical / up-to-down).
• [4] face rect size in source image – Sort by size of the face in the original video frame image (descending).
• [5] histogram similarity – Sort by histogram similarity (descending).
• [6] histogram dissimilarity – Sort by histogram similarity (ascending).
• [7] brightness – Sort by image brightness.
• [8] hue – Sort by image hue.
• [9] amount of black pixels – Sort by amount of black pixels in image (ascending).
• [10] original filename – Sort by order of original filename. Does not recover the original filename.
• [11] one face in image – Sort by the number of faces in the original video frame image (ascending).
• [12] absolute pixel difference – Sort by absolute difference.
• [13] best faces – Sort by multiple methods (w/ blur) and remove similar faces. Select target number of face images to keep. Discarded faces moved to data_src/aligned_trash.
• [14] best faces faster – Sort by multiple methods (w/ face rect size) and remove similar faces. Select target number of face images to keep. Discarded faces moved to data_src/aligned_trash.

4.2) data_src util add landmarks debug images.bat

Add face landmarks debug images. Duplicates faceset images in data_src/aligned, adds landmarks images, appends “_debug” to filename.

To remove debug images before training run 4.1) data_src view aligned result.bat, search for “_debug”, select all search result images and delete.

4.2) data_src util faceset enhance.bat

Enhance source faceset via upscale. Enhanced images will be created in data_src/aligned_enhanced.

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run enhancer. Recommend using identical devices when choosing multiple GPU indexes.
• Merge data_src/aligned_enhanced to data_src/aligned ?: Choose to replace (overwrite) original faceset images with enhanced images.

4.2) data_src util faceset metadata restore.bat

Restore source faceset metadata from meta.dat file.

4.2) data_src util faceset metadata save.bat

Save source faceset metadata as meta.dat file.

• [Console: Now you can edit images.!!! Keep same filenames in the folder. You can change size of images, restoring process will downscale back to original size. After that, use restore metadata.

4.2) data_src util faceset pack.bat

Pack source faceset as data_src/aligned/faceset.pak file.

• Delete original files? ( y / n ) : Choose to deleted original files after packing.

4.2) data_src util faceset resize.bat

Resize and change face type of source faceset images. Resized images will be created in data_src/aligned_resized.

• New image size ( 128-2048 ): Select the new faceset image size (resolution).
• Change face type ( h / mf / f / wf / head / same ) : Select to change the face type.
  [Note: half face / mid face / full face / whole face / head / no change]
• Merge data_src/aligned_resized to data_src/aligned ?: Choose to replace (overwrite) original faceset images with resized images.

4.2) data_src util faceset unpack.bat

Unpack source data_src/aligned/faceset.pak file. Faceset.pak will be deleted.

4.2) data_src util recover original filename.bat

Rename source faceset files with original filename after sorting. Faceset filenames are taken from the original video frame image filename.

5) data_dst faceset extract + manual fix.bat

Automatic destination faceset extractor with option to manually landmark undetected frames. After automatic extraction, opens interface to manually set faceset alignment landmarks on video frames without detected faces. Manual mode allows one face selection per frame. Faceset filenames are taken from the original video frame image filename. Console will log the number of images found and faces detected.

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• Face type ( f / wf / head ): Select the face type for the extraction.
  [Tooltip: Full face / whole face / head. ‘Whole face’ covers full area of face include forehead. ‘head’ covers full head, but requires XSeg for src and dst faceset.]
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]

5) data_dst faceset extract MANUAL.bat

Manual destination faceset extractor. Opens interface to manually set faceset alignment landmarks on video frames. Manual mode allows one face selection per frame. Faceset filenames are taken from the original video frame image filename. Console will log the number of images found and faces detected.

• Which GPU index to choose?: Select a single GPU index from the list to run manual extraction.
• Face type ( f / wf / head ): Select the face type for the extraction.
  [Tooltip: Full face / whole face / head. ‘Whole face’ covers full area of face include forehead. ‘head’ covers full head, but requires XSeg for src and dst faceset.]
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]

5) data_dst faceset extract.bat

Automatic destination faceset extractor. Faceset filenames are taken from the original video frame image filename. Console will log the number of images found and faces detected.

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• Face type ( f / wf / head ): Select the face type for the extraction.
  [Tooltip: Full face / whole face / head. ‘Whole face’ covers full area of face include forehead. ‘head’ covers full head, but requires XSeg for src and dst faceset.]
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]

5) data_dst faceset MANUAL RE-EXTRACT DELETED ALIGNED_DEBUG.bat

Manual destination faceset re-extractor based on deleted aligned_debug images.
First, delete data_dst/aligned_debug images that contain poorly aligned faces.
Opens interface to manually set faceset alignment landmarks on video frames corresponding to deleted debug images. Console will log the number of images found and faces detected.

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• Image size ( 256 – 2048 ): Select the size (resolution) of the extracted faceset image files.
  [Tooltip: Output image size. The higher image size, the worse face-enhancer works. Use higher than 512 value only if the source image is sharp enough and the face does not need to be enhanced.]
• Jpeg quality ( 1 – 100 ): Select the quality (compression) of the extracted faceset image files.
  [Tooltip: Jpeg quality. The higher jpeg quality the larger the output file size.]

5.1) data_dst view aligned results.bat

Open destination faceset within XNView. Use this image viewer to delete unwanted faceset images.

Read more about XNView on their website.

5.1) data_dst view aligned_debug results.bat

Open destination aligned_debug images within XNView. Use this image viewer to find corresponding faceset images with poor alignments.

Before running 5) data_dst faceset MANUAL RE-EXTRACT DELETED ALIGNED_DEBUG.bat, use this viewer to delete debug images containing poorly aligned faces.

Read more about XNView on their website.

5.2) data_dst sort.bat

Sort the destination faceset using various values and algorithms. Faceset image files will be enumerated (renamed) with the new sort order.

Choose Sorting Method:

• [0] blur – Sort by image blurriness based on contrast.
• [1] motion blur – Sort by motion blur.
• [2] face yaw direction – Sort by yaw (horizontal / left-to-right).
• [3] face pitch direction – Sort by pitch (vertical / up-to-down).
• [4] face rect size in source image – Sort by size of the face in the original video frame image (descending).
• [5] histogram similarity – Sort by histogram similarity (descending).
• [6] histogram dissimilarity – Sort by histogram similarity (ascending).
• [7] brightness – Sort by image brightness.
• [8] hue – Sort by image hue.
• [9] amount of black pixels – Sort by amount of black pixels in image (ascending).
• [10] original filename – Sort by order of original filename. Does not recover the original filename.
• [11] one face in image – Sort by the number of faces in the original video frame image (ascending).
• [12] absolute pixel difference – Sort by absolute difference.
• [13] best faces – Sort by multiple methods (w/ blur) and remove similar faces. Select target number of face images to keep. Discarded faces moved to data_src/aligned_trash.
• [14] best faces faster – Sort by multiple methods (w/ face rect size) and remove similar faces. Select target number of face images to keep. Discarded faces moved to data_dst/aligned_trash.

5.2) data_dst util faceset pack.bat

Pack destination faceset as data_dst/aligned/faceset.pak file.

• Delete original files? ( y / n ) : Choose to deleted original files after packing.

5.2) data_dst util faceset resize.bat

Resize and change face type of destination faceset images. Resized images will be created in data_dst/aligned_resized.

• New image size ( 128-2048 ): Select the new faceset image size (resolution).
• Change face type ( h / mf / f / wf / head / same ) : Select to change the face type.
  [Note: half face / mid face / full face / whole face / head / no change]
• Merge data_dst/aligned_resized to data_dst/aligned ?: Choose to replace (overwrite) original faceset images with resized images.

5.2) data_dst util faceset unpack.bat

Unpack destination data_dst/aligned/faceset.pak file. Faceset.pak will be deleted.

5.2) data_dst util recover original filename.bat

Rename destination faceset files with original filename after sorting. Faceset filenames are taken from the original video frame image filename.

5.XSeg Generic) data_dst whole_face mask – apply.bat

Apply generic XSeg whole face mask to destination faceset.

Place generic trained XSeg model files within _internal/model_generic_xseg.

• Which GPU index to choose?: Select a single GPU index from the list to apply the XSeg mask.

5.XSeg Generic) data_src whole_face mask – apply.bat

Apply generic XSeg whole face mask to source faceset.

Place generic trained XSeg model files within _internal/model_generic_xseg.

• Which GPU index to choose?: Select a single GPU index from the list to apply the XSeg mask.

5.XSeg) data_dst mask – edit.bat

Edit destination XSeg mask labels. Opens XSeg Editor to manually label faceset images with XSeg mask polygons.

5.XSeg) data_dst mask – fetch.bat

Copy XSeg labeled destination faceset images to data_dst/aligned_xseg.

• Delete original files? ( y / n ) : Choose to delete the Xseg labeled files from the faceset after fetching.

5.XSeg) data_dst mask – remove.bat

Remove XSeg mask labels from destination faceset images.

• [console: !!! WARNING : LABELED XSEG POLYGONS WILL BE REMOVED FROM THE FRAMES !!!]

5.XSeg) data_dst trained mask – apply.bat

Apply trained XSeg mask to destination faceset.

Requires trained XSeg model. Place trained XSeg model files within workspace/model.

• Which GPU index to choose?: Select a single GPU index from the list to apply the XSeg mask.

5.XSeg) data_dst trained mask – remove.bat

Remove trained XSeg mask from destination faceset. Faceset will retain default generated mask.

• [Console: !!! WARNING : APPLIED XSEG MASKS WILL BE REMOVED FROM THE FRAMES !!!]

5.XSeg) data_src mask – edit.bat

Edit source XSeg mask labels. Opens XSeg Editor to manually label faceset images with XSeg mask polygons.

5.XSeg) data_src mask – fetch.bat

Copy XSeg labeled source faceset images to data_src/aligned_xseg.

• Delete original files? ( y / n ) : Choose to delete the Xseg labeled files from the faceset after fetching.

5.XSeg) data_src mask – remove.bat

Remove XSeg mask labels from source faceset images.

• [Console: !!! WARNING : LABELED XSEG POLYGONS WILL BE REMOVED FROM THE FRAMES !!!]

5.XSeg) data_src trained mask – apply.bat

Apply trained XSeg mask to source faceset.

Requires trained XSeg model. Place trained XSeg model files within workspace/model.

• Which GPU index to choose?: Select a single GPU index from the list to apply the XSeg mask.

5.XSeg) data_src trained mask – remove.bat

Remove trained XSeg mask from source faceset. Faceset will retain default generated mask.

• [Tooltip: !!! WARNING : APPLIED XSEG MASKS WILL BE REMOVED FROM THE FRAMES !!!]

5.XSeg) train.bat

Train XSeg masks using labeled images from source and destination facesets.

First Run:

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• Face type ( h / mf / f / wf / head ): Select the face type for XSeg training.
  [Tooltip: Half / mid face / full face / whole face / head. Choose the same as your deepfake model.]
• Batch_size ( 2 – 16 ): Select the batch size for XSeg training.
  [Tooltip: Larger batch size is better for NN’s generalization, but it can cause Out of Memory error. Tune this value for your videocard manually.]
• Enable pretraining mode ( y / n ): Choose to use the _internal/pretrain_faces faceset for XSeg training.
• [Console: Trying to do the first iteration. If an error occurs, reduce the model parameters.]
• [Console: Windows 10 users IMPORTANT notice. You should set this setting in order to work correctly. https://i.imgur.com/B7cmDCB.jpg ]
• [Note: Developer recommends Windows 10 users enable Hardware-accelerated GPU Scheduling in System > Display > Graphics Settings. Test on / off performance on your system.]

Subsequent Run:

• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.
• [Console: Press enter in 2 seconds to override model settings.]
• Restart training? ( y / n ): Choose to restart XSeg training.
  [Tooltip: Reset model weights and start training from scratch.]
• Batch_size ( 2 – 16 ): Select the batch size for XSeg training.
  [Tooltip: Larger batch size is better for NN’s generalization, but it can cause Out of Memory error. Tune this value for your videocard manually.]
• Enable pretraining mode ( y / n ): Choose to use the _internal/pretrain_faces faceset for XSeg training.

6) export AMP as dfm.bat

Export AMP model as .dfm format to work in DeepFaceLive.

• Choose one of saved models, or enter a name to create a new model.
  [ r ] : rename
  [ d ] : delete
  [Note: Choose the model index to export as dfm.]
• Export quantized? ( y / n ): Quantization reduces precision but results in a smaller model size and faster computation. Read more about Tensorflow Optimization.
  [Tooltip: Makes the exported model faster. If you have problems, disable this option.]

6) export SAEHD as dfm.bat

Export SAEHD model as .dfm format to work in DeepFaceLive.

• Choose one of saved models, or enter a name to create a new model.
  [ r ] : rename
  [ d ] : delete
  [Note: Choose the model index to export as dfm.]
• Export quantized? ( y / n ): Quantization reduces precision but results in a smaller model size and faster computation. Read more about Tensorflow Optimization.
  [Tooltip: Makes the exported model faster. If you have problems, disable this option.]

6) train AMP SRC-SRC.bat

Train AMP model with source faceset as destination faceset.

• In Progress

6) train AMP.bat

Train AMP model. Dst facial expressions are amplified to src.

• In Progress

6) train Quick96.bat

Train Quick96 model. Static model with Resolution 96 and Batch Size 4.

Displays console log and preview window during training.

First Run:

• No saved models found. Enter a name of a new model : Name the model.
• Which GPU indexes to choose?: Select one or more GPU indexes from the list to run extraction. Recommend using identical devices when choosing multiple GPU indexes.

• In Progress

6) train SAEHD.bat

Train SAEHD model.

• In Progress

7) merge AMP.bat

Merge AMP model to destination images.

Generates data_dst/merged and data_dst/merged_mask image sequences.

• In Progress

7) merge Quick96.bat

Merge Quick96 model to destination images.

Generates data_dst/merged and data_dst/merged_mask image sequences.

• In Progress

7) merge SAEHD.bat

Merge SAEHD model to destination images.

Generates data_dst/merged and data_dst/merged_mask image sequences.

• In Progress

8) merged to avi.bat

Merge final image sequence to destination video as AVI.

Generates result.avi and result_mask.avi in the workspace folder.

• Bitrate of output file in MB/s : Select the video bitrate.

8) merged to mov lossless.bat

Merge final image sequence to destination video as lossless MOV.

Generates result.mov and result_mask.mov in the workspace folder.

8) merged to mp4 lossless.bat

Merge final image sequence to destination video as lossless MP4.

Generates result.mp4 and result_mask.mov in the workspace folder.

8) merged to mp4.bat

Merge final image sequence to destination video as compressed MP4.

Generates result.mp4 and result_mask.mp4 in the workspace folder.

• Bitrate of output file in MB/s : Select the video bitrate.

10.misc) make CPU only.bat

Make the software run using CPU only by installing an older version of TensorFlow from the internet.

• [Console: This script will convert this DeepFaceLab folder to work on CPU only without any problems.]
• [Console: This operation cannot be undone.]
• [Console: An internet connection is required.]

10.misc) start EBSynth.bat

Run EbSynth. Transform video by painting over a single frame

Read mode about EbSynth at their website.

changelog.html

DeepFaceLab code changelog updated by the developer.