• Stars
    star
    137
  • Rank 266,121 (Top 6 %)
  • Language
    Python
  • License
    MIT License
  • Created about 4 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Voice Conversion by CycleGAN (语音克隆/语音转换):CycleGAN-VC3

CycleGAN-VC3-PyTorch

standard-readme compliant Donate

中文说明 | English


This code is a PyTorch implementation for paper: CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversion, a nice work on Voice-Conversion/Voice Cloning.

  • Dataset
    • VC
  • Usage
    • Training
    • Example
  • Demo
  • Reference

CycleGAN-VC3

Project Page

Non-parallel voice conversion (VC) is a technique for learning mappings between source and target speeches without using a parallel corpus. Recently, CycleGAN-VC [3] and CycleGAN-VC2 [2] have shown promising results regarding this problem and have been widely used as benchmark methods. However, owing to the ambiguity of the effectiveness of CycleGAN-VC/VC2 for mel-spectrogram conversion, they are typically used for mel-cepstrum conversion even when comparative methods employ mel-spectrogram as a conversion target. To address this, we examined the applicability of CycleGAN-VC/VC2 to mel-spectrogram conversion. Through initial experiments, we discovered that their direct applications compromised the time-frequency structure that should be preserved during conversion. To remedy this, we propose CycleGAN-VC3, an improvement of CycleGAN-VC2 that incorporates time-frequency adaptive normalization (TFAN). Using TFAN, we can adjust the scale and bias of the converted features while reflecting the time-frequency structure of the source mel-spectrogram. We evaluated CycleGAN-VC3 on inter-gender and intra-gender non-parallel VC. A subjective evaluation of naturalness and similarity showed that for every VC pair, CycleGAN-VC3 outperforms or is competitive with the two types of CycleGAN-VC2, one of which was applied to mel-cepstrum and the other to mel-spectrogram.

network comparison Figure 1. We developed time-frequency adaptive normalization (TFAN), which extends instance normalization [5] so that the affine parameters become element-dependent and are determined according to an entire input mel-spectrogram.


This repository contains:

  1. TFAN module code which implemented the TFAN module
  2. model code which implemented the model network.
  3. audio preprocessing script you can use to create cache for training data.
  4. training scripts to train the model.

Table of Contents


Requirement

pip install -r requirements.txt

Usage


Star-History

star-history


Reference

  1. CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversion. Paper, Project
  2. CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion. Paper, Project
  3. Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks. Paper, Project
  4. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Paper, Project, Code
  5. Image-to-Image Translation with Conditional Adversarial Nets. Paper, Project, Code

Donation

If this project help you reduce time to develop, you can give me a cup of coffee :)

AliPay(支付宝)

ali_pay

WechatPay(微信)

wechat_pay

paypal


License

MIT © Kun

More Repositories

1

awesome_LLMs_interview_notes

LLMs interview notes and answers:该仓库主要记录大模型(LLMs)算法工程师相关的面试题和参考答案
1,126
star
2

CycleGAN-VC2

Voice Conversion by CycleGAN (语音克隆/语音转换): CycleGAN-VC2
Python
521
star
3

Vicuna-LoRA-RLHF-PyTorch

A full pipeline to finetune Vicuna LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Vicuna architecture. Basically ChatGPT but with Vicuna
Python
207
star
4

Recurrent-LLM

The open-source LLM implementation of paper: RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text. AI 写小说,AI写作
Python
152
star
5

SecBERT

pretrained BERT model for cyber security text, learned CyberSecurity Knowledge
Python
144
star
6

LAS_Mandarin_PyTorch

Listen, attend and spell Model and a Chinese Mandarin Pretrained model (中文-普通话 ASR模型)
Python
121
star
7

ChatGLM-LoRA-RLHF-PyTorch

A full pipeline to finetune ChatGLM LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the ChatGLM architecture. Basically ChatGPT but with ChatGLM
Python
121
star
8

NLP4CyberSecurity

NLP model and tech for cyber security tasks
Jupyter Notebook
75
star
9

ThreatReportExtractor

Extracting Attack Behavior from Threat Reports
Python
74
star
10

Alpaca-LoRA-RLHF-PyTorch

A full pipeline to finetune Alpaca LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Alpaca architecture. Basically ChatGPT but with Alpaca
Python
54
star
11

nude-detect

Porn Content Pic or Video Recognization
Python
38
star
12

location_clustering

用户地理位置的聚类算法实现—基于DBSCAN和Kmeans的混合算法
Python
25
star
13

awesome_NLP-Interview-Notes

nlp_interview notes and answers: 该仓库主要记录 NLP 算法工程师相关的面试题和参考答案
18
star
14

AI-WAF

AI driven Web Application Firewall
Python
18
star
15

apk-view-tracer

Apk-view-tracer is a trigger tool for Android Dynamic Analysis and can be used in android anti-virus dynamic analysis.
Python
18
star
16

drowsiness-detection

打瞌睡检测,通过检测眼皮对眼球的遮挡程度,判定是否打瞌睡😂
Python
17
star
17

HomoglyphAttacksDetector

Detecting Homoglyph Attacks with CNN model using Computer Vision method
Jupyter Notebook
11
star
18

RepackagedAppDetector

Detect re-packaged app on Android based on fuzzy hash of instructions in dex
8
star
19

Loss-Function-In-PyTorch

Loss Function in PyTorch
Jupyter Notebook
7
star
20

WindowsStoreCrawler

crawl windows application from windows store on windows 8
C#
7
star
21

SpeakerRecognition-ResNet-GhostVLAD

Utterance-level Aggregation For Speaker Recognition In The Wild, using a "thin-ResNet" trunk architecture, and a dictionary-based NetVLAD or GhostVLAD layer to aggregate features across time, that can be trained end-to-end
Python
7
star
22

PrivacyLeakAdvancedDetection

Privacy Leak and Behavior Detect on Android based on method call graph
Java
6
star
23

audio_classification_models.pytorch

audio/voice classification in pytorch implementations
5
star
24

GANs-implementation

GAN models implementation repo
Python
4
star
25

jackaduma

personal profile
4
star
26

speaker_recognition_models.pytorch

speaker recognition / speaker verification models in pytorch implementation
4
star
27

DotNetAppGuard

Decompile &Static Analysis Dot Net App by using java
Java
4
star
28

Speech-Transformer-PyTorch

Python
4
star
29

jackaduma.github.io

CSS
4
star
30

django-cache-machine-mongoengine

Automatic caching and invalidation for Django & Mongodb. using models through the mongoengine ORM.
Python
4
star
31

py-recommender-framework

Recommender Framework implemented by python
Python
4
star
32

LangChain-OpenLLMs

Langchain-OpenLLMs with local knowledge library based on open source LLMs.
Jupyter Notebook
4
star
33

Annotated-Diffusion-Model

The Annotated Diffusion Model
Jupyter Notebook
3
star
34

SecCopilot

2
star
35

malicious-url-detection-with-ML

malicious url detection with machine learning
Python
1
star
36

awesome_AI_in_CyberSecurity_papers

awesome AI in CyberSecurity papers list
1
star
37

phishing-url-detection-with-ML

phishing url detection with machine learning
1
star
38

awesome_AI_in_Speech_papers

awesome AI in Speech papers
1
star
39

weak-password-detection-with-ML

weak password detection with machine learning
1
star