working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT [2024-10-08_12:28:33]: Failed to import xformers. [2024-10-08_12:28:33]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t4_v25 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 20.0 Overriding: sft_loss_scale = 0.0 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 1145 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t3.pt Overriding: model_cache_loss_name = 30b_t3_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t4_v25_b20 [2024-10-08_12:28:34]: ddp init, rank 40, local_rank 0 [2024-10-08_12:28:34]: ddp init, rank 16, local_rank 0 [2024-10-08_12:28:35]: ddp init, rank 41, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 0, local_rank 0 [2024-10-08_12:28:35]: ddp init, rank 44, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 42, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 47, local_rank 7 [2024-10-08_12:28:35]: ddp init, rank 32, local_rank 0 [2024-10-08_12:28:35]: ddp init, rank 7, local_rank 7 [2024-10-08_12:28:35]: ddp init, rank 6, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 8, local_rank 0 [2024-10-08_12:28:35]: ddp init, rank 5, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 3, local_rank 3 [2024-10-08_12:28:35]: ddp init, rank 17, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 2, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 1, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 4, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 24, local_rank 0 [2024-10-08_12:28:35]: ddp init, rank 43, local_rank 3 [2024-10-08_12:28:35]: ddp init, rank 46, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 45, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 19, local_rank 3 NCCL version 2.20.5+cuda12.4 [2024-10-08_12:28:35]: ddp init, rank 23, local_rank 7 [2024-10-08_12:28:35]: ddp init, rank 18, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 22, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 21, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 20, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 36, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 28, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 13, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 35, local_rank 3 [2024-10-08_12:28:35]: ddp init, rank 39, local_rank 7 [2024-10-08_12:28:35]: ddp init, rank 30, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 34, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 31, local_rank 7 [2024-10-08_12:28:35]: ddp init, rank 27, local_rank 3 [2024-10-08_12:28:35]: ddp init, rank 26, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 25, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 14, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 37, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 33, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 29, local_rank 5 [2024-10-08_12:28:35]: ddp init, rank 38, local_rank 6 [2024-10-08_12:28:35]: ddp init, rank 9, local_rank 1 [2024-10-08_12:28:35]: ddp init, rank 10, local_rank 2 [2024-10-08_12:28:35]: ddp init, rank 12, local_rank 4 [2024-10-08_12:28:35]: ddp init, rank 11, local_rank 3 [2024-10-08_12:28:35]: ddp init, rank 15, local_rank 7 [2024-10-08_12:28:47]: ddp init: world size 48 ddp_rank 0. [2024-10-08_12:28:47]: loss discounts for codebooks: [0.308 0.077 0.073 0.07 0.066 0.063 0.059 0.056 0.052 0.049 0.045 0.042 0.038] [2024-10-08_12:29:08]: Total world size 48 [2024-10-08_12:29:08]: logging checkpoint here: /app/suno/checkpoints/2024-10-08_12-29-08 [2024-10-08_12:29:08]: loading data... [2024-10-08_12:29:08]: indexed 100.0% of data [2024-10-08_12:29:08]: 1,110 lines of data_val.bin loaded. [2024-10-08_12:29:11]: indexed 100.0% of data [2024-10-08_12:29:11]: 109,780 lines of data_tr.bin loaded. [2024-10-08_12:29:11]: train data weights: 50.0% perference_0 50.0% perference_1 [2024-10-08_12:29:11]: done loading data [2024-10-08_12:29:12]: GPU capacity: NVIDIA H100 80GB HBM3 (0) with 79.33GiB memory [2024-10-08_12:29:12]: Initializing train model from scratch trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-08_12:35:05]: number of parameters: 32020M [2024-10-08_12:35:05]: finish init train model trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-08_12:35:05]: not compiling model. [2024-10-08_12:35:05]: start loading state dict [2024-10-08_12:35:05]: verifying model args... trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-08_12:35:18]: copying checkpoint file to local cachedir... [2024-10-08_12:37:59]: loading model state_dict on gpu 0 [2024-10-08_12:38:08]: loading model state_dict on gpu 1 [2024-10-08_12:38:16]: loading model state_dict on gpu 2 [2024-10-08_12:38:24]: loading model state_dict on gpu 3 [2024-10-08_12:38:32]: loading model state_dict on gpu 4 [2024-10-08_12:38:40]: loading model state_dict on gpu 5 [2024-10-08_12:38:48]: loading model state_dict on gpu 6 [2024-10-08_12:38:55]: loading model state_dict on gpu 7 [2024-10-08_12:39:03]: finish loading state dict [2024-10-08_12:39:03]: wrapping model in FSDP .... [2024-10-08_12:39:33]: GPU memory usage for model: 11.03GiB(13.90%) [2024-10-08_12:39:33]: applying fsdp activation checkpointing... [2024-10-08_12:39:33]: num decayed parameter tensors: 567, with 671,048,768 parameters [2024-10-08_12:39:33]: num non-decayed parameter tensors: 124, with 430,080 parameters [2024-10-08_12:39:33]: using fused Optimizer: False [2024-10-08_12:39:33]: model setup done [2024-10-08_12:39:33]: Validate random number: 0.9918216287004739 [2024-10-08_12:39:33]: Evaluating [2024-10-08_12:39:33]: Start the ref model loss eval loop. [2024-10-08_12:39:33]: Loading pre-computed cache loss: /app/suno/data/dpo/30b_t4_v25/30b_t3_bt16_cached_loss.json [2024-10-08_12:39:34]: Check if loaded correctly: train 109780 vs 109780 val 1110 vs 1110 [2024-10-08_12:39:34]: Validate random number: 0.3328835026423017 [2024-10-08_12:39:34]: training... [2024-10-08_12:39:40]: Eval -- [81118, 81119] Policy positive loss: tensor([3.5327]), torch.float32, Policy negative loss: tensor([3.7911]), torch.float32 [2024-10-08_12:39:40]: Eval -- Reference positive loss: tensor([3.5327]), torch.float32, Reference negative loss: tensor([3.7911]), torch.float32 [2024-10-08_12:39:40]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:39:45]: Eval -- [102242, 102243] Policy positive loss: tensor([2.0225]), torch.float32, Policy negative loss: tensor([3.5523]), torch.float32 [2024-10-08_12:39:45]: Eval -- Reference positive loss: tensor([2.0225]), torch.float32, Reference negative loss: tensor([3.5523]), torch.float32 [2024-10-08_12:39:45]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:39:50]: Eval -- [52446, 52447] Policy positive loss: tensor([1.9289]), torch.float32, Policy negative loss: tensor([1.6977]), torch.float32 [2024-10-08_12:39:50]: Eval -- Reference positive loss: tensor([1.9289]), torch.float32, Reference negative loss: tensor([1.6977]), torch.float32 [2024-10-08_12:39:50]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([2.3842e-06]), torch.float32, Rejected rewards: tensor([2.3842e-06]), torch.float32 [2024-10-08_12:39:55]: Eval -- [82184, 82185] Policy positive loss: tensor([3.5139]), torch.float32, Policy negative loss: tensor([2.9447]), torch.float32 [2024-10-08_12:39:55]: Eval -- Reference positive loss: tensor([3.5139]), torch.float32, Reference negative loss: tensor([2.9447]), torch.float32 [2024-10-08_12:39:55]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:40:00]: Eval -- [69068, 69069] Policy positive loss: tensor([3.2116]), torch.float32, Policy negative loss: tensor([1.9206]), torch.float32 [2024-10-08_12:40:00]: Eval -- Reference positive loss: tensor([3.2116]), torch.float32, Reference negative loss: tensor([1.9206]), torch.float32 [2024-10-08_12:40:00]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:40:05]: Eval -- [54682, 54683] Policy positive loss: tensor([2.3377]), torch.float32, Policy negative loss: tensor([3.2739]), torch.float32 [2024-10-08_12:40:05]: Eval -- Reference positive loss: tensor([2.3377]), torch.float32, Reference negative loss: tensor([3.2739]), torch.float32 [2024-10-08_12:40:05]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:40:10]: Eval -- [46438, 46439] Policy positive loss: tensor([2.5620]), torch.float32, Policy negative loss: tensor([3.3584]), torch.float32 [2024-10-08_12:40:10]: Eval -- Reference positive loss: tensor([2.5620]), torch.float32, Reference negative loss: tensor([3.3584]), torch.float32 [2024-10-08_12:40:10]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([-4.7684e-06]), torch.float32 [2024-10-08_12:40:15]: Eval -- [15508, 15509] Policy positive loss: tensor([4.1432]), torch.float32, Policy negative loss: tensor([3.7524]), torch.float32 [2024-10-08_12:40:15]: Eval -- Reference positive loss: tensor([4.1432]), torch.float32, Reference negative loss: tensor([3.7524]), torch.float32 [2024-10-08_12:40:15]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:40:21]: Eval -- [97300, 97301] Policy positive loss: tensor([2.5406]), torch.float32, Policy negative loss: tensor([3.4492]), torch.float32 [2024-10-08_12:40:21]: Eval -- Reference positive loss: tensor([2.5406]), torch.float32, Reference negative loss: tensor([3.4492]), torch.float32 [2024-10-08_12:40:21]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:40:26]: Eval -- [82286, 82287] Policy positive loss: tensor([3.1532]), torch.float32, Policy negative loss: tensor([3.0082]), torch.float32 [2024-10-08_12:40:26]: Eval -- Reference positive loss: tensor([3.1532]), torch.float32, Reference negative loss: tensor([3.0082]), torch.float32 [2024-10-08_12:40:26]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:40:31]: Eval -- [90246, 90247] Policy positive loss: tensor([3.8347]), torch.float32, Policy negative loss: tensor([2.9624]), torch.float32 [2024-10-08_12:40:31]: Eval -- Reference positive loss: tensor([3.8347]), torch.float32, Reference negative loss: tensor([2.9624]), torch.float32 [2024-10-08_12:40:31]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([-4.7684e-06]), torch.float32, Rejected rewards: tensor([-4.7684e-06]), torch.float32 [2024-10-08_12:40:36]: Eval -- [33666, 33667] Policy positive loss: tensor([2.3980]), torch.float32, Policy negative loss: tensor([2.3616]), torch.float32 [2024-10-08_12:40:36]: Eval -- Reference positive loss: tensor([2.3980]), torch.float32, Reference negative loss: tensor([2.3616]), torch.float32 [2024-10-08_12:40:36]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:40:41]: Eval -- [414, 415] Policy positive loss: tensor([2.2433]), torch.float32, Policy negative loss: tensor([2.1746]), torch.float32 [2024-10-08_12:40:41]: Eval -- Reference positive loss: tensor([2.2433]), torch.float32, Reference negative loss: tensor([2.1746]), torch.float32 [2024-10-08_12:40:41]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([-4.7684e-06]), torch.float32 [2024-10-08_12:40:46]: Eval -- [642, 643] Policy positive loss: tensor([3.0821]), torch.float32, Policy negative loss: tensor([3.1067]), torch.float32 [2024-10-08_12:40:46]: Eval -- Reference positive loss: tensor([3.0821]), torch.float32, Reference negative loss: tensor([3.1067]), torch.float32 [2024-10-08_12:40:46]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:40:51]: Eval -- [542, 543] Policy positive loss: tensor([3.1036]), torch.float32, Policy negative loss: tensor([3.1540]), torch.float32 [2024-10-08_12:40:51]: Eval -- Reference positive loss: tensor([3.1036]), torch.float32, Reference negative loss: tensor([3.1540]), torch.float32 [2024-10-08_12:40:51]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:40:57]: Eval -- [288, 289] Policy positive loss: tensor([3.4596]), torch.float32, Policy negative loss: tensor([3.4271]), torch.float32 [2024-10-08_12:40:57]: Eval -- Reference positive loss: tensor([3.4596]), torch.float32, Reference negative loss: tensor([3.4271]), torch.float32 [2024-10-08_12:40:57]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([-4.7684e-06]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:41:02]: Eval -- [428, 429] Policy positive loss: tensor([4.1705]), torch.float32, Policy negative loss: tensor([3.7238]), torch.float32 [2024-10-08_12:41:02]: Eval -- Reference positive loss: tensor([4.1705]), torch.float32, Reference negative loss: tensor([3.7238]), torch.float32 [2024-10-08_12:41:02]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:07]: Eval -- [600, 601] Policy positive loss: tensor([3.9567]), torch.float32, Policy negative loss: tensor([4.5490]), torch.float32 [2024-10-08_12:41:07]: Eval -- Reference positive loss: tensor([3.9567]), torch.float32, Reference negative loss: tensor([4.5490]), torch.float32 [2024-10-08_12:41:07]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:12]: Eval -- [384, 385] Policy positive loss: tensor([3.1158]), torch.float32, Policy negative loss: tensor([3.3972]), torch.float32 [2024-10-08_12:41:12]: Eval -- Reference positive loss: tensor([3.1158]), torch.float32, Reference negative loss: tensor([3.3972]), torch.float32 [2024-10-08_12:41:12]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:17]: Eval -- [22, 23] Policy positive loss: tensor([3.4368]), torch.float32, Policy negative loss: tensor([3.3178]), torch.float32 [2024-10-08_12:41:17]: Eval -- Reference positive loss: tensor([3.4368]), torch.float32, Reference negative loss: tensor([3.3178]), torch.float32 [2024-10-08_12:41:17]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([-4.7684e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:22]: Eval -- [158, 159] Policy positive loss: tensor([3.4420]), torch.float32, Policy negative loss: tensor([3.4870]), torch.float32 [2024-10-08_12:41:22]: Eval -- Reference positive loss: tensor([3.4420]), torch.float32, Reference negative loss: tensor([3.4870]), torch.float32 [2024-10-08_12:41:22]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:41:28]: Eval -- [330, 331] Policy positive loss: tensor([3.4709]), torch.float32, Policy negative loss: tensor([2.5878]), torch.float32 [2024-10-08_12:41:28]: Eval -- Reference positive loss: tensor([3.4709]), torch.float32, Reference negative loss: tensor([2.5878]), torch.float32 [2024-10-08_12:41:28]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:33]: Eval -- [458, 459] Policy positive loss: tensor([2.9774]), torch.float32, Policy negative loss: tensor([3.2273]), torch.float32 [2024-10-08_12:41:33]: Eval -- Reference positive loss: tensor([2.9774]), torch.float32, Reference negative loss: tensor([3.2273]), torch.float32 [2024-10-08_12:41:33]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-08_12:41:38]: Eval -- [158, 159] Policy positive loss: tensor([3.4420]), torch.float32, Policy negative loss: tensor([3.4870]), torch.float32 [2024-10-08_12:41:38]: Eval -- Reference positive loss: tensor([3.4420]), torch.float32, Reference negative loss: tensor([3.4870]), torch.float32 [2024-10-08_12:41:38]: Eval -- Preference loss: tensor([0.0006]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([4.7684e-06]), torch.float32 [2024-10-08_12:41:41]: loss estimation took 126.7 seconds. (100.0% of loop) [2024-10-08_12:41:41]: step 0: train loss 3.9043, val loss 3.8817 [2024-10-08_12:41:51]: iter 0: avg_loss 0.001, step_time 137484.8ms, mfu 0.0%, throughput 0k tok/s, total time 137s, memory 42.88GiB(54.05%) [2024-10-08_12:46:10]: iter 25: avg_loss 0.001, step_time 10350.7ms, mfu 130.0%, throughput 82k tok/s, total time 397s, memory 49.24GiB(62.07%) [2024-10-08_12:50:29]: iter 50: avg_loss 0.001, step_time 10350.1ms, mfu 130.0%, throughput 82k tok/s, total time 655s, memory 49.24GiB(62.07%) [2024-10-08_12:54:48]: iter 75: avg_loss 0.001, step_time 10368.2ms, mfu 129.8%, throughput 82k tok/s, total time 914s, memory 49.24GiB(62.07%) [2024-10-08_12:59:07]: iter 100: avg_loss 0.001, step_time 10341.8ms, mfu 130.1%, throughput 82k tok/s, total time 1173s, memory 49.24GiB(62.07%) [2024-10-08_13:03:26]: iter 125: avg_loss 0.001, step_time 10334.7ms, mfu 130.2%, throughput 82k tok/s, total time 1432s, memory 49.24GiB(62.07%) [2024-10-08_13:07:44]: iter 150: avg_loss 0.000, step_time 10344.1ms, mfu 130.1%, throughput 82k tok/s, total time 1690s, memory 49.24GiB(62.07%) [2024-10-08_13:12:03]: iter 175: avg_loss 0.000, step_time 10340.8ms, mfu 130.1%, throughput 82k tok/s, total time 1949s, memory 49.24GiB(62.07%) [2024-10-08_13:16:22]: iter 200: avg_loss 0.001, step_time 10372.3ms, mfu 129.7%, throughput 82k tok/s, total time 2208s, memory 49.24GiB(62.07%) [2024-10-08_13:20:41]: iter 225: avg_loss 0.001, step_time 10326.9ms, mfu 130.3%, throughput 82k tok/s, total time 2467s, memory 49.24GiB(62.07%) [2024-10-08_13:25:00]: iter 250: avg_loss 0.001, step_time 10372.0ms, mfu 129.8%, throughput 82k tok/s, total time 2726s, memory 49.24GiB(62.07%) [2024-10-08_13:29:19]: iter 275: avg_loss 0.001, step_time 10338.0ms, mfu 130.2%, throughput 82k tok/s, total time 2985s, memory 49.24GiB(62.07%) [2024-10-08_13:33:37]: iter 300: avg_loss 0.001, step_time 10338.2ms, mfu 130.2%, throughput 82k tok/s, total time 3243s, memory 49.24GiB(62.07%) [2024-10-08_13:37:57]: iter 325: avg_loss 0.001, step_time 10347.2ms, mfu 130.1%, throughput 82k tok/s, total time 3503s, memory 49.24GiB(62.07%) [2024-10-08_13:42:16]: iter 350: avg_loss 0.001, step_time 10365.9ms, mfu 129.8%, throughput 82k tok/s, total time 3762s, memory 49.24GiB(62.07%) [2024-10-08_13:46:34]: iter 375: avg_loss 0.001, step_time 10368.0ms, mfu 129.8%, throughput 82k tok/s, total time 4021s, memory 49.24GiB(62.07%) [2024-10-08_13:50:53]: iter 400: avg_loss 0.001, step_time 10355.2ms, mfu 130.0%, throughput 82k tok/s, total time 4279s, memory 49.24GiB(62.07%) [2024-10-08_13:55:12]: iter 425: avg_loss 0.000, step_time 10344.3ms, mfu 130.1%, throughput 82k tok/s, total time 4538s, memory 49.24GiB(62.07%) [2024-10-08_13:59:31]: iter 450: avg_loss 0.001, step_time 10374.9ms, mfu 129.7%, throughput 82k tok/s, total time 4797s, memory 49.24GiB(62.07%) [2024-10-08_14:03:50]: iter 475: avg_loss 0.000, step_time 10362.7ms, mfu 129.9%, throughput 82k tok/s, total time 5056s, memory 49.24GiB(62.07%) [2024-10-08_14:08:08]: iter 500: avg_loss 0.000, step_time 10361.4ms, mfu 129.9%, throughput 82k tok/s, total time 5315s, memory 49.24GiB(62.07%) [2024-10-08_14:12:27]: iter 525: avg_loss 0.001, step_time 10310.8ms, mfu 130.5%, throughput 82k tok/s, total time 5573s, memory 49.24GiB(62.07%) [2024-10-08_14:16:46]: iter 550: avg_loss 0.001, step_time 10338.3ms, mfu 130.2%, throughput 82k tok/s, total time 5832s, memory 49.24GiB(62.07%) [2024-10-08_14:21:05]: iter 575: avg_loss 0.001, step_time 10371.6ms, mfu 129.8%, throughput 82k tok/s, total time 6091s, memory 49.24GiB(62.07%) [2024-10-08_14:25:23]: iter 600: avg_loss 0.001, step_time 10334.3ms, mfu 130.2%, throughput 82k tok/s, total time 6350s, memory 49.24GiB(62.07%) [2024-10-08_14:29:43]: iter 625: avg_loss 0.001, step_time 10350.6ms, mfu 130.0%, throughput 82k tok/s, total time 6609s, memory 49.24GiB(62.07%) [2024-10-08_14:34:02]: iter 650: avg_loss 0.001, step_time 10349.0ms, mfu 130.0%, throughput 82k tok/s, total time 6868s, memory 49.24GiB(62.07%) [2024-10-08_14:38:20]: iter 675: avg_loss 0.001, step_time 10336.7ms, mfu 130.2%, throughput 82k tok/s, total time 7127s, memory 49.24GiB(62.07%) [2024-10-08_14:42:39]: iter 700: avg_loss 0.001, step_time 10365.8ms, mfu 129.8%, throughput 82k tok/s, total time 7385s, memory 49.24GiB(62.07%) [2024-10-08_14:46:58]: iter 725: avg_loss 0.001, step_time 10370.8ms, mfu 129.8%, throughput 82k tok/s, total time 7644s, memory 49.24GiB(62.07%) [2024-10-08_14:51:17]: iter 750: avg_loss 0.001, step_time 10349.4ms, mfu 130.0%, throughput 82k tok/s, total time 7903s, memory 49.24GiB(62.07%) [2024-10-08_14:55:36]: iter 775: avg_loss 0.001, step_time 10345.7ms, mfu 130.1%, throughput 82k tok/s, total time 8162s, memory 49.24GiB(62.07%) [2024-10-08_14:59:55]: iter 800: avg_loss 0.001, step_time 10362.9ms, mfu 129.9%, throughput 82k tok/s, total time 8421s, memory 49.24GiB(62.07%) [2024-10-08_15:04:14]: iter 825: avg_loss 0.000, step_time 10354.4ms, mfu 130.0%, throughput 82k tok/s, total time 8680s, memory 49.24GiB(62.07%) [2024-10-08_15:08:32]: iter 850: avg_loss 0.001, step_time 10333.3ms, mfu 130.2%, throughput 82k tok/s, total time 8938s, memory 49.24GiB(62.07%) [2024-10-08_15:12:51]: iter 875: avg_loss 0.001, step_time 10364.6ms, mfu 129.8%, throughput 82k tok/s, total time 9197s, memory 49.24GiB(62.07%) [2024-10-08_15:17:10]: iter 900: avg_loss 0.000, step_time 10348.7ms, mfu 130.0%, throughput 82k tok/s, total time 9456s, memory 49.24GiB(62.07%) [2024-10-08_15:21:29]: iter 925: avg_loss 0.001, step_time 10338.5ms, mfu 130.2%, throughput 82k tok/s, total time 9715s, memory 49.24GiB(62.07%) [2024-10-08_15:25:48]: iter 950: avg_loss 0.001, step_time 10359.5ms, mfu 129.9%, throughput 82k tok/s, total time 9974s, memory 49.24GiB(62.07%) [2024-10-08_15:30:07]: iter 975: avg_loss 0.000, step_time 10338.1ms, mfu 130.2%, throughput 82k tok/s, total time 10233s, memory 49.24GiB(62.07%) [2024-10-08_15:34:25]: iter 1000: avg_loss 0.001, step_time 10345.3ms, mfu 130.1%, throughput 82k tok/s, total time 10492s, memory 49.24GiB(62.07%) [2024-10-08_15:38:44]: iter 1025: avg_loss 0.001, step_time 10333.1ms, mfu 130.2%, throughput 82k tok/s, total time 10750s, memory 49.24GiB(62.07%) [2024-10-08_15:43:03]: iter 1050: avg_loss 0.000, step_time 10354.9ms, mfu 130.0%, throughput 82k tok/s, total time 11009s, memory 49.24GiB(62.07%) [2024-10-08_15:47:21]: iter 1075: avg_loss 0.001, step_time 10340.8ms, mfu 130.1%, throughput 82k tok/s, total time 11268s, memory 49.24GiB(62.07%) [2024-10-08_15:51:40]: iter 1100: avg_loss 0.001, step_time 10339.8ms, mfu 130.2%, throughput 82k tok/s, total time 11526s, memory 49.24GiB(62.07%) [2024-10-08_15:55:59]: iter 1125: avg_loss 0.001, step_time 10363.2ms, mfu 129.9%, throughput 82k tok/s, total time 11785s, memory 49.24GiB(62.07%) [2024-10-08_15:59:08]: Eval -- [42246, 42247] Policy positive loss: tensor([2.6393]), torch.float32, Policy negative loss: tensor([3.1785]), torch.float32 [2024-10-08_15:59:08]: Eval -- Reference positive loss: tensor([2.6294]), torch.float32, Reference negative loss: tensor([3.1693]), torch.float32 [2024-10-08_15:59:08]: Eval -- Preference loss: tensor([0.0007]), torch.float32, Chosen rewards: tensor([-0.1967]), torch.float32, Rejected rewards: tensor([-0.1847]), torch.float32 [2024-10-08_15:59:13]: Eval -- [10936, 10937] Policy positive loss: tensor([2.2751]), torch.float32, Policy negative loss: tensor([2.1054]), torch.float32 [2024-10-08_15:59:13]: Eval -- Reference positive loss: tensor([2.2717]), torch.float32, Reference negative loss: tensor([2.1107]), torch.float32 [2024-10-08_15:59:13]: Eval -- Preference loss: tensor([0.0011]), torch.float32, Chosen rewards: tensor([-0.0677]), torch.float32, Rejected rewards: tensor([0.1071]), torch.float32 [2024-10-08_15:59:18]: Eval -- [78654, 78655] Policy positive loss: tensor([2.6872]), torch.float32, Policy negative loss: tensor([2.6069]), torch.float32 [2024-10-08_15:59:18]: Eval -- Reference positive loss: tensor([2.6881]), torch.float32, Reference negative loss: tensor([2.6004]), torch.float32 [2024-10-08_15:59:18]: Eval -- Preference loss: tensor([0.0003]), torch.float32, Chosen rewards: tensor([0.0186]), torch.float32, Rejected rewards: tensor([-0.1300]), torch.float32 [2024-10-08_15:59:23]: Eval -- [51614, 51615] Policy positive loss: tensor([3.8275]), torch.float32, Policy negative loss: tensor([3.7156]), torch.float32 [2024-10-08_15:59:23]: Eval -- Reference positive loss: tensor([3.8030]), torch.float32, Reference negative loss: tensor([3.7063]), torch.float32 [2024-10-08_15:59:23]: Eval -- Preference loss: tensor([0.0016]), torch.float32, Chosen rewards: tensor([-0.4895]), torch.float32, Rejected rewards: tensor([-0.1858]), torch.float32 [2024-10-08_15:59:28]: Eval -- [57192, 57193] Policy positive loss: tensor([2.5286]), torch.float32, Policy negative loss: tensor([1.7717]), torch.float32 [2024-10-08_15:59:28]: Eval -- Reference positive loss: tensor([2.5247]), torch.float32, Reference negative loss: tensor([1.7657]), torch.float32 [2024-10-08_15:59:28]: Eval -- Preference loss: tensor([0.0005]), torch.float32, Chosen rewards: tensor([-0.0787]), torch.float32, Rejected rewards: tensor([-0.1202]), torch.float32 [2024-10-08_15:59:33]: Eval -- [103380, 103381] Policy positive loss: tensor([2.9756]), torch.float32, Policy negative loss: tensor([4.0036]), torch.float32 [2024-10-08_15:59:33]: Eval -- Reference positive loss: tensor([2.9689]), torch.float32, Reference negative loss: tensor([3.9874]), torch.float32 [2024-10-08_15:59:33]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.1333]), torch.float32, Rejected rewards: tensor([-0.3245]), torch.float32 [2024-10-08_15:59:39]: Eval -- [95586, 95587] Policy positive loss: tensor([2.2539]), torch.float32, Policy negative loss: tensor([1.1729]), torch.float32 [2024-10-08_15:59:39]: Eval -- Reference positive loss: tensor([2.2443]), torch.float32, Reference negative loss: tensor([1.1502]), torch.float32 [2024-10-08_15:59:39]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([-0.1923]), torch.float32, Rejected rewards: tensor([-0.4554]), torch.float32 [2024-10-08_15:59:44]: Eval -- [40268, 40269] Policy positive loss: tensor([2.9166]), torch.float32, Policy negative loss: tensor([3.1490]), torch.float32 [2024-10-08_15:59:44]: Eval -- Reference positive loss: tensor([2.9131]), torch.float32, Reference negative loss: tensor([3.1419]), torch.float32 [2024-10-08_15:59:44]: Eval -- Preference loss: tensor([0.0005]), torch.float32, Chosen rewards: tensor([-0.0699]), torch.float32, Rejected rewards: tensor([-0.1409]), torch.float32 [2024-10-08_15:59:49]: Eval -- [41772, 41773] Policy positive loss: tensor([1.6038]), torch.float32, Policy negative loss: tensor([2.4482]), torch.float32 [2024-10-08_15:59:49]: Eval -- Reference positive loss: tensor([1.5968]), torch.float32, Reference negative loss: tensor([2.4456]), torch.float32 [2024-10-08_15:59:49]: Eval -- Preference loss: tensor([0.0009]), torch.float32, Chosen rewards: tensor([-0.1399]), torch.float32, Rejected rewards: tensor([-0.0524]), torch.float32 [2024-10-08_15:59:54]: Eval -- [66596, 66597] Policy positive loss: tensor([3.0326]), torch.float32, Policy negative loss: tensor([2.9647]), torch.float32 [2024-10-08_15:59:54]: Eval -- Reference positive loss: tensor([3.0326]), torch.float32, Reference negative loss: tensor([2.9576]), torch.float32 [2024-10-08_15:59:54]: Eval -- Preference loss: tensor([0.0003]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([-0.1420]), torch.float32 [2024-10-08_15:59:59]: Eval -- [53020, 53021] Policy positive loss: tensor([3.3739]), torch.float32, Policy negative loss: tensor([3.1360]), torch.float32 [2024-10-08_15:59:59]: Eval -- Reference positive loss: tensor([3.3714]), torch.float32, Reference negative loss: tensor([3.1312]), torch.float32 [2024-10-08_15:59:59]: Eval -- Preference loss: tensor([0.0005]), torch.float32, Chosen rewards: tensor([-0.0481]), torch.float32, Rejected rewards: tensor([-0.0962]), torch.float32 [2024-10-08_16:00:04]: Eval -- [944, 945] Policy positive loss: tensor([1.7465]), torch.float32, Policy negative loss: tensor([1.9360]), torch.float32 [2024-10-08_16:00:04]: Eval -- Reference positive loss: tensor([1.7489]), torch.float32, Reference negative loss: tensor([1.9192]), torch.float32 [2024-10-08_16:00:04]: Eval -- Preference loss: tensor([3.3600e-05]), torch.float32, Chosen rewards: tensor([0.0481]), torch.float32, Rejected rewards: tensor([-0.3360]), torch.float32 [2024-10-08_16:00:10]: Eval -- [526, 527] Policy positive loss: tensor([3.4029]), torch.float32, Policy negative loss: tensor([3.6330]), torch.float32 [2024-10-08_16:00:10]: Eval -- Reference positive loss: tensor([3.4029]), torch.float32, Reference negative loss: tensor([3.6210]), torch.float32 [2024-10-08_16:00:10]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([4.7684e-06]), torch.float32, Rejected rewards: tensor([-0.2415]), torch.float32 [2024-10-08_16:00:15]: Eval -- [1030, 1031] Policy positive loss: tensor([2.3446]), torch.float32, Policy negative loss: tensor([2.7920]), torch.float32 [2024-10-08_16:00:15]: Eval -- Reference positive loss: tensor([2.3455]), torch.float32, Reference negative loss: tensor([2.7815]), torch.float32 [2024-10-08_16:00:15]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.0186]), torch.float32, Rejected rewards: tensor([-0.2087]), torch.float32 [2024-10-08_16:00:20]: Eval -- [162, 163] Policy positive loss: tensor([3.1350]), torch.float32, Policy negative loss: tensor([3.5986]), torch.float32 [2024-10-08_16:00:20]: Eval -- Reference positive loss: tensor([3.1242]), torch.float32, Reference negative loss: tensor([3.5823]), torch.float32 [2024-10-08_16:00:20]: Eval -- Preference loss: tensor([0.0004]), torch.float32, Chosen rewards: tensor([-0.2153]), torch.float32, Rejected rewards: tensor([-0.3256]), torch.float32 [2024-10-08_16:00:25]: Eval -- [1050, 1051] Policy positive loss: tensor([2.9512]), torch.float32, Policy negative loss: tensor([3.2131]), torch.float32 [2024-10-08_16:00:25]: Eval -- Reference positive loss: tensor([2.9489]), torch.float32, Reference negative loss: tensor([3.2051]), torch.float32 [2024-10-08_16:00:25]: Eval -- Preference loss: tensor([0.0004]), torch.float32, Chosen rewards: tensor([-0.0470]), torch.float32, Rejected rewards: tensor([-0.1606]), torch.float32 [2024-10-08_16:00:30]: Eval -- [470, 471] Policy positive loss: tensor([3.2074]), torch.float32, Policy negative loss: tensor([3.2643]), torch.float32 [2024-10-08_16:00:30]: Eval -- Reference positive loss: tensor([3.2038]), torch.float32, Reference negative loss: tensor([3.2629]), torch.float32 [2024-10-08_16:00:30]: Eval -- Preference loss: tensor([0.0007]), torch.float32, Chosen rewards: tensor([-0.0721]), torch.float32, Rejected rewards: tensor([-0.0284]), torch.float32 [2024-10-08_16:00:35]: Eval -- [148, 149] Policy positive loss: tensor([2.6862]), torch.float32, Policy negative loss: tensor([1.2752]), torch.float32 [2024-10-08_16:00:35]: Eval -- Reference positive loss: tensor([2.6924]), torch.float32, Reference negative loss: tensor([1.2531]), torch.float32 [2024-10-08_16:00:35]: Eval -- Preference loss: tensor([1.0180e-05]), torch.float32, Chosen rewards: tensor([0.1224]), torch.float32, Rejected rewards: tensor([-0.4414]), torch.float32 [2024-10-08_16:00:41]: Eval -- [620, 621] Policy positive loss: tensor([3.8800]), torch.float32, Policy negative loss: tensor([3.9185]), torch.float32 [2024-10-08_16:00:41]: Eval -- Reference positive loss: tensor([3.8675]), torch.float32, Reference negative loss: tensor([3.9068]), torch.float32 [2024-10-08_16:00:41]: Eval -- Preference loss: tensor([0.0007]), torch.float32, Chosen rewards: tensor([-0.2513]), torch.float32, Rejected rewards: tensor([-0.2338]), torch.float32 [2024-10-08_16:00:46]: Eval -- [546, 547] Policy positive loss: tensor([3.1906]), torch.float32, Policy negative loss: tensor([2.8792]), torch.float32 [2024-10-08_16:00:46]: Eval -- Reference positive loss: tensor([3.1872]), torch.float32, Reference negative loss: tensor([2.8741]), torch.float32 [2024-10-08_16:00:46]: Eval -- Preference loss: tensor([0.0005]), torch.float32, Chosen rewards: tensor([-0.0677]), torch.float32, Rejected rewards: tensor([-0.1005]), torch.float32 [2024-10-08_16:00:51]: Eval -- [82, 83] Policy positive loss: tensor([3.1042]), torch.float32, Policy negative loss: tensor([2.0439]), torch.float32 [2024-10-08_16:00:51]: Eval -- Reference positive loss: tensor([3.1042]), torch.float32, Reference negative loss: tensor([2.0504]), torch.float32 [2024-10-08_16:00:51]: Eval -- Preference loss: tensor([0.0010]), torch.float32, Chosen rewards: tensor([-4.7684e-06]), torch.float32, Rejected rewards: tensor([0.1300]), torch.float32 [2024-10-08_16:00:56]: Eval -- [358, 359] Policy positive loss: tensor([2.3941]), torch.float32, Policy negative loss: tensor([2.3966]), torch.float32 [2024-10-08_16:00:56]: Eval -- Reference positive loss: tensor([2.4010]), torch.float32, Reference negative loss: tensor([2.3969]), torch.float32 [2024-10-08_16:00:56]: Eval -- Preference loss: tensor([0.0003]), torch.float32, Chosen rewards: tensor([0.1377]), torch.float32, Rejected rewards: tensor([0.0066]), torch.float32 [2024-10-08_16:01:01]: Eval -- [954, 955] Policy positive loss: tensor([3.4277]), torch.float32, Policy negative loss: tensor([3.4214]), torch.float32 [2024-10-08_16:01:01]: Eval -- Reference positive loss: tensor([3.4256]), torch.float32, Reference negative loss: tensor([3.4162]), torch.float32 [2024-10-08_16:01:01]: Eval -- Preference loss: tensor([0.0005]), torch.float32, Chosen rewards: tensor([-0.0415]), torch.float32, Rejected rewards: tensor([-0.1049]), torch.float32 [2024-10-08_16:01:06]: Eval -- [864, 865] Policy positive loss: tensor([3.8200]), torch.float32, Policy negative loss: tensor([3.8075]), torch.float32 [2024-10-08_16:01:06]: Eval -- Reference positive loss: tensor([3.8035]), torch.float32, Reference negative loss: tensor([3.7941]), torch.float32 [2024-10-08_16:01:06]: Eval -- Preference loss: tensor([0.0008]), torch.float32, Chosen rewards: tensor([-0.3300]), torch.float32, Rejected rewards: tensor([-0.2677]), torch.float32 [2024-10-08_16:01:09]: loss estimation took 124.1 seconds. (1.0% of loop) [2024-10-08_16:01:09]: step 1144: train loss 3.9096, val loss 3.8792 [2024-10-08_16:05:13]: saving checkpoint to /app/suno/checkpoints/2024-10-08_12-29-08 [2024-10-08_16:07:25]: saving took 375.8 seconds. (3.1% of loop) [2024-10-08_16:07:48]: iter 1144: avg_loss 0.001, step_time 522712.2ms, mfu 2.6%, throughput 2k tok/s, total time 12494s, memory 49.24GiB(62.07%) [2024-10-08_16:07:48]: done. [2024-10-08_16:07:48]: all seen idxs: 109780 working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE