Search Results for 'DeepFaceLab'

Of the four or so versions of DeepFacelab it’s unclear which would work with my Quadro RTX 6000.

DeepFaceLab_DirectX12_build_05_04_2022.exe
DeepFaceLab_DirectX12_build_11_20_2021.exe
DeepFaceLab_NVIDIA_RTX3000_series_build_11_20_2021.exe
DeepFaceLab_NVIDIA_up_to_RTX2080Ti_build_11_20_2021.exe

I expect it would be one of the last two but it’s really not clear. Can anyone please advise?

I get this error message while running ‘merge SAEHD’

Caching GPU kernels…
Running merger.

Choose one of saved models, or enter a name to create a new model.
[r] : rename
[d] : delete

[0] : p384dfudt – latest
: 0
0
Loading p384dfudt_SAEHD model…

Choose one or several GPU idxs (separated by comma).

[CPU] : CPU
[0] : NVIDIA GeForce RTX 2070
[1] : Intel(R) UHD Graphics

[1] Which GPU indexes to choose? : 0
0

Caching GPU kernels…
Traceback (most recent call last):
File “C:\DeepFaceLab\DeepFaceLab_DirectX12\_internal\DeepFaceLab\mainscripts\Merger.py”, line 53, in main
cpu_only=cpu_only)
File “C:\DeepFaceLab\DeepFaceLab_DirectX12\_internal\DeepFaceLab\models\ModelBase.py”, line 180, in __init__
self.on_initialize_options()
File “C:\DeepFaceLab\DeepFaceLab_DirectX12\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 181, in on_initialize_options
raise Exception(“pretraining_data_path is not defined”)
Exception: pretraining_data_path is not defined

Done.
Press any key to continue . . .

– Please anyone help me how to overcome this issue.

Hi guys! I’ve never created a video with Deepfacelab before. First of all, I heard that if I do pretraining, it will be faster and easier every time I make a df video. When I open the Deepfacelab folder, I directly run the 6) train SAEHD file. But I get an error at the last part because of the values I entered.

I can’t find out why I’m getting this error. I’m about to go crazy…

[n] Enable pretraining mode ( y/n ?:help ) : y
Initializing models: 80%|##################################################4 | 4/5 [01:31<00:22, 22.88s/it]
Error: OOM when allocating tensor with shape[131072,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node src_dst_opt/ms_inter_AB/dense1/weight_0/Assign (defined at C:\Users\Ersin\Desktop\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series_build_11_20_2021\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:37) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

My System Features:

My Graphics Card: RTX 3060
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz 2.30 GHz
RAM: 16GB

I need to know, for rtx 4080 OC GPU, what Deepfacelab2.0 build I should install.

I didn’t find a build that says 4080 or above, so, far I only find out 3000 series, 2080 ti and above, and DeepFaceLab_DirectX12.

Can anyone tell me which one to use, for my GPU, in one video there was a saying for an openCL version if any of the above doesn’t work, but I can’t find that build anymore, in the download link.

I appreciate any help you can provide.

[Ismail111Participant]

I deleted the previous model and made a new model. I tried to do SAEHD training again, but I got the same error again. I can’t find out why I’m getting this error. I’m about to go crazy…

[n] Enable pretraining mode ( y/n ?:help ) : y
Initializing models: 80%|##################################################4 | 4/5 [01:31<00:22, 22.88s/it]
Error: OOM when allocating tensor with shape[131072,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node src_dst_opt/ms_inter_AB/dense1/weight_0/Assign (defined at C:\Users\Ersin\Desktop\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series_build_11_20_2021\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:37) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

Hi guys! I’ve never created a video with Deepfacelab before. First of all, I heard that if I do pretraining, it will be faster and easier every time I make a df video. When I open the Deepfacelab folder, I directly run the 6) train SAEHD file. But I get an error at the last part because of the values I entered.

My System Features:

My Graphics Card: RTX 3060
11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz 2.30 GHz
RAM: 16GB

Can you tell me what path I should follow and what values I should enter?

[seishiruoParticipant]

Leaving it after several minutes I get this.

Error: 2 root error(s) found.
(0) Resource exhausted: failed to allocate memory
[[node mul_81 (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:64) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

[[concat_4/concat/_463]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

(1) Resource exhausted: failed to allocate memory
[[node mul_81 (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:64) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node mul_81:
src_dst_opt/vs_inter/dense1/weight_0/read (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:38)

Input Source operations connected to node mul_81:
src_dst_opt/vs_inter/dense1/weight_0/read (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:38)

Original stack trace for ‘mul_81’:
File “threading.py”, line 884, in _bootstrap
File “threading.py”, line 916, in _bootstrap_inner
File “threading.py”, line 864, in run
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\mainscripts\Trainer.py”, line 58, in trainerThread
debug=debug)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\ModelBase.py”, line 193, in __init__
self.on_initialize()
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 564, in on_initialize
src_dst_loss_gv_op = self.src_dst_opt.get_update_op (nn.average_gv_list (gpu_G_loss_gvs))
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py”, line 64, in get_update_op
v_t = self.beta_2*vs + (1.0-self.beta_2) * tf.square(g-m_t)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\variables.py”, line 1076, in _run_op
return tensor_oper(a.value(), *args, **kwargs)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 1400, in r_binary_op_wrapper
return func(x, y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 1710, in _mul_dispatch
return multiply(x, y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\util\dispatch.py”, line 206, in wrapper
return target(*args, **kwargs)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 530, in multiply
return gen_math_ops.mul(x, y, name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\gen_math_ops.py”, line 6245, in mul
“Mul”, x=x, y=y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\op_def_library.py”, line 750, in _apply_op_helper
attrs=attr_protos, op_def=op_def)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\ops.py”, line 3569, in _create_op_internal
op_def=op_def)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\ops.py”, line 2045, in __init__
self._traceback = tf_stack.extract_stack_for_node(self._c_op)

Traceback (most recent call last):
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1375, in _do_call
return fn(*args)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1360, in _run_fn
target_list, run_metadata)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1453, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: failed to allocate memory
[[{{node mul_81}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

[[concat_4/concat/_463]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

(1) Resource exhausted: failed to allocate memory
[[{{node mul_81}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\mainscripts\Trainer.py”, line 129, in trainerThread
iter, iter_time = model.train_one_iter()
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\ModelBase.py”, line 474, in train_one_iter
losses = self.onTrainOneIter()
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 774, in onTrainOneIter
src_loss, dst_loss = self.src_dst_train (warped_src, target_src, target_srcm, target_srcm_em, warped_dst, target_dst, target_dstm, target_dstm_em)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 584, in src_dst_train
self.target_dstm_em:target_dstm_em,
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 968, in run
run_metadata_ptr)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1191, in _run
feed_dict_tensor, options, run_metadata)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1369, in _do_run
run_metadata)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1394, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
(0) Resource exhausted: failed to allocate memory
[[node mul_81 (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:64) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

[[concat_4/concat/_463]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

(1) Resource exhausted: failed to allocate memory
[[node mul_81 (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:64) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn’t available when running in Eager mode.

0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node mul_81:
src_dst_opt/vs_inter/dense1/weight_0/read (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:38)

Input Source operations connected to node mul_81:
src_dst_opt/vs_inter/dense1/weight_0/read (defined at C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py:38)

Original stack trace for ‘mul_81’:
File “threading.py”, line 884, in _bootstrap
File “threading.py”, line 916, in _bootstrap_inner
File “threading.py”, line 864, in run
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\mainscripts\Trainer.py”, line 58, in trainerThread
debug=debug)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\ModelBase.py”, line 193, in __init__
self.on_initialize()
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 564, in on_initialize
src_dst_loss_gv_op = self.src_dst_opt.get_update_op (nn.average_gv_list (gpu_G_loss_gvs))
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\leras\optimizers\AdaBelief.py”, line 64, in get_update_op
v_t = self.beta_2*vs + (1.0-self.beta_2) * tf.square(g-m_t)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\variables.py”, line 1076, in _run_op
return tensor_oper(a.value(), *args, **kwargs)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 1400, in r_binary_op_wrapper
return func(x, y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 1710, in _mul_dispatch
return multiply(x, y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\util\dispatch.py”, line 206, in wrapper
return target(*args, **kwargs)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\math_ops.py”, line 530, in multiply
return gen_math_ops.mul(x, y, name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\ops\gen_math_ops.py”, line 6245, in mul
“Mul”, x=x, y=y, name=name)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\op_def_library.py”, line 750, in _apply_op_helper
attrs=attr_protos, op_def=op_def)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\ops.py”, line 3569, in _create_op_internal
op_def=op_def)
File “C:\Users\Ferros\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\framework\ops.py”, line 2045, in __init__
self._traceback = tf_stack.extract_stack_for_node(self._c_op)

is there any manual or guide ever written since on the github page there is none

https://github.com/MachineEditor/DeepFaceLab/wiki/New-Training-Options

Running trainer.

Model first run.

Choose one or several GPU idxs (separated by comma).

[CPU] : CPU
[0] : NVIDIA GeForce RTX 3090

[0] Which GPU indexes to choose? : 0
0

[wf] Face type ( h/mf/f/wf/head ?:help ) : wf
wf
[8] Batch_size ( 2-16 ?:help ) : 8
8
[n] Enable pretraining mode ( y/n ) : n
Loading samples: 100%|##############################################################| 654/654 [00:02<00:00, 271.86it/s]
Error: name ‘samples_dat_path’ is not defined
Traceback (most recent call last):
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleLoader.py”, line 54, in load
result = samplelib.PackedFaceset.load(samples_path)
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\PackedFaceset.py”, line 142, in load
samples_configs = pickle.loads ( f.read(sizeof_samples_bytes) )
_pickle.UnpicklingError: invalid load key, ‘\x00′.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\mainscripts\Trainer.py”, line 58, in trainerThread
debug=debug)
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_XSeg\Model.py”, line 17, in __init__
super().__init__(*args, force_model_class_name=’XSeg’, **kwargs)
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\ModelBase.py”, line 193, in __init__
self.on_initialize()
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_XSeg\Model.py”, line 166, in on_initialize
data_format=nn.data_format)
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleGeneratorFaceXSeg.py”, line 26, in __init__
samples = sum([ SampleLoader.load (SampleType.FACE, path) for path in paths ] )
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleGeneratorFaceXSeg.py”, line 26, in <listcomp>
samples = sum([ SampleLoader.load (SampleType.FACE, path) for path in paths ] )
File “D:\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleLoader.py”, line 56, in load
io.log_err(f”Error occured while loading samplelib.PackedFaceset.load {str(samples_dat_path)}, {traceback.format_exc()}”)
NameError: name ‘samples_dat_path’ is not defined

Hello there!
I was testing aroung with DeepFaceLab SaeHD. First I created a pretrained model with around 150.000k samples.
And then I wanted to use this model to train on a source scene and a destination scene.
While in training mode everything worked fine, in no-training mode/normal mode it is not trying to understand faces by recreating the faces. The area where the blurred images of the program trying to understand the face occurs only consists of yellow, red or white images. Like the entire frame is just one tone of color. The first few images seemed like it was trying to detect faces, after that the image turned red and yellow until after a few samples the entire image was yellow.

Is this a common bug? What can I do against it?

Link to see: https://youtu.be/OiVa4ezAeq4

Choose one or several GPU idxs (separated by comma).

[CPU] : CPU
[0] : NVIDIA GeForce RTX 3060

[0] Which GPU indexes to choose? : 0
0

Initializing models: 100%|###############################################################| 5/5 [00:02<00:00, 2.48it/s]
Loading samples: 100%|############################################################| 5757/5757 [00:29<00:00, 194.10it/s]
Loaded 63012 packed faces from C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\workspace\data_dst\aligned
================ Model Summary =================
== ==
== Model name: ook_Quick96 ==
== ==
== Current iteration: 19 ==
== ==
==————– Model Options —————==
== ==
== batch_size: 4 ==
== ==
==—————- Running On —————-==
== ==
== Device index: 0 ==
== Name: NVIDIA GeForce RTX 3060 ==
== VRAM: 9.37GB ==
== ==
================================================
Starting. Press “Enter” to stop training and save model.
[20:08:53][#000111][0346ms][1.8989][2.9003]
[20:09:04][#000219][0095ms][1.5813][2.3569]
[20:09:27][#000459][0105ms][1.2614][2.0420]
Process Process-16:[0094ms][0.8256][1.3022]
Traceback (most recent call last):
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py”, line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleProcessor.py”, line 56, in process
sample_bgr = sample.load_bgr()
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\Sample.py”, line 112, in load_bgr
img = cv2_imread (self.filename, loader_func=self.read_raw_file).astype(np.float32) / 255.0
AttributeError: ‘NoneType’ object has no attribute ‘astype’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “multiprocessing\process.py”, line 258, in _bootstrap
File “multiprocessing\process.py”, line 93, in run
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\core\joblib\SubprocessGenerator.py”, line 54, in process_func
gen_data = next (self.generator_func)
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py”, line 136, in batch_func
raise Exception (“Exception occured in sample %s. Error: %s” % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample 48466.jpg. Error: Traceback (most recent call last):
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py”, line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\SampleProcessor.py”, line 56, in process
sample_bgr = sample.load_bgr()
File “C:\Users\U S E R\Desktop\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\samplelib\Sample.py”, line 112, in load_bgr
img = cv2_imread (self.filename, loader_func=self.read_raw_file).astype(np.float32) / 255.0
AttributeError: ‘NoneType’ object has no attribute ‘astype’

[20:10:48][#001340][0088ms][0.8677][1.7431]

Hi all!

I am starting with deepfacelab and I really don’t know what i am doing yet! I have a model trained with more than 500K iterations using train quick96 but now when I hit mergeQuick96 i get all kinds of import and mempory errors. I tried several times and have been days searching for the whole internet with no luck.

I have lots of lines similar to this:

File “<frozen importlib._bootstrap>”, line 955, in _find_and_load_unlocked
File “<frozen importlib._bootstrap>”, line 656, in _load_unlocked
raise ImportError(msg)

But where it seems to be the main problem and the “solution” is here:

MemoryError
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

* The Python version is: Python3.6 from “D:\deepfakeLab\_internal\python-3.6.8\python.exe”
* The NumPy version is: “1.19.3”

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed: The paging file is too small for this operation to complete.
MemoryError
File “<frozen importlib._bootstrap>”, line 441, in spec_from_loader
File “<frozen importlib._bootstrap_external>”, line 544, in spec_from_file_location
File “<frozen importlib._bootstrap_external>”, line 1157, in find_spec
File “<frozen importlib._bootstrap>”, line 656, in _load_unlocked
File “<frozen importlib._bootstrap>”, line 626, in _load_backward_compatible
ImportError:

I have visited the webpage and read the instructions but I cant figure it out. is there something that i am missing here?

Thanks in advance.
Javi.

someone, can you help me

== models_opt_on_gpu: True ==
== archi: liae-ud ==
== ae_dims: 256 ==
== e_dims: 64 ==
== d_dims: 64 ==
== d_mask_dims: 22 ==
== masked_training: True ==
== eyes_mouth_prio: False ==
== uniform_yaw: False ==
== blur_out_mask: False ==
== adabelief: True ==
== lr_dropout: n ==
== random_warp: True ==
== random_hsv_power: 0.0 ==
== true_face_power: 0.0 ==
== face_style_power: 0.0 ==
== bg_style_power: 0.0 ==
== ct_mode: none ==
== clipgrad: False ==
== pretrain: False ==
== autobackup_hour: 0 ==
== write_preview_history: False ==
== target_iter: 0 ==
== random_src_flip: False ==
== random_dst_flip: True ==
== batch_size: 8 ==
== gan_power: 0.0 ==
== gan_patch_size: 16 ==
== gan_dims: 16 ==
== ==
==—————— Running On ——————==
== ==
== Device index: 0 ==
== Name: NVIDIA GeForce RTX 4090 ==
== VRAM: 20.85GB ==
== ==
====================================================
Starting. Press “Enter” to stop training and save model.
Error: Could not synchronize CUDA stream: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure
Traceback (most recent call last):
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1375, in _do_call
return fn(*args)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1360, in _run_fn
target_list, run_metadata)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1453, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Could not synchronize CUDA stream: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\mainscripts\Trainer.py”, line 129, in trainerThread
iter, iter_time = model.train_one_iter()
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\ModelBase.py”, line 474, in train_one_iter
losses = self.onTrainOneIter()
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 774, in onTrainOneIter
src_loss, dst_loss = self.src_dst_train (warped_src, target_src, target_srcm, target_srcm_em, warped_dst, target_dst, target_dstm, target_dstm_em)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\DeepFaceLab\models\Model_SAEHD\Model.py”, line 584, in src_dst_train
self.target_dstm_em:target_dstm_em,
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 968, in run
run_metadata_ptr)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1191, in _run
feed_dict_tensor, options, run_metadata)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1369, in _do_run
run_metadata)
File “C:\Users\RAY\Downloads\DeepFaceLab\DeepFaceLab_NVIDIA_RTX3000_series\_internal\python-3.6.8\lib\site-packages\tensorflow\python\client\session.py”, line 1394, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InternalError: Could not synchronize CUDA stream: CUDA_ERROR_LAUNCH_FAILED: unspecified launch failure
Done.
Presione una tecla para continuar . . .

my problem is getting worse – I started off where the card was recognized, but didn’t work – couldn’t extract anything (it tried, but everything was crap). Now (after trying to troubleshoot) I don’t even have the option to select GPU, it just automatically goes to CPU – which works but so sloooooooooooow.

As part of troubleshooting I was playing with tensorflow, path, CUDA, etc. so who knows what I’ve f’d up. Seems like it’s supposed to work without installing anything.

GPU works fine other than DFL. Running Windows 10.

Any suggestions to help reset and get the GPU actually working would be appreciated. Cheers.

edit to add: using the DeepFaceLab_NVIDIA_up_to_RTX2080Ti code, but also tried the 3000 one just in case – no joy.

Running trainer.

Model first run.

Choose one or several GPU idxs (separated by comma).

[CPU] : CPU
[0] : NVIDIA GeForce RTX 3070 Ti

[0] Which GPU indexes to choose? : 0
0

[wf] Face type ( h/mf/f/wf/head ?:help ) : wf
wf
[12] Batch_size ( 2-16 ?:help ) : 8
8
[n] Enable pretraining mode ( y/n ) : n
Loading samples: 100%|##########################################################| 26247/26247 [00:52<00:00, 501.79it/s]
Loaded 15843 packed faces from E:\DeepFaceLab_NVIDIA_RTX3000_series\workspace\data_dst\aligned

at here wait for 2 days wont run
CPU 13th Gen Intel(R) Core(TM) i7-13700KF 3.40 GHz
RAM 64.0 GB
RTX 3070 Ti
Vram 8GB